wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
26,837 Commits 34 Branches 312 Tags 292 MiB
Commit Graph

503 Commits

Author SHA1 Message Date
Johannes M. Scheuermann 1312e347be Correct the way how the backup agent makes use of the proxy and add proxy to trace events 2023-09-20 17:52:59 +02:00
Hao Fu 6d9c53f8c4 Add proxy to backup agent via global var 
backup agent itself does not have proxy info.
This changes adds the proxy via a global var.
 2023-09-08 10:27:01 -07:00
Steve Atherton 596ab8d83d Yield between parsing backup layer status JSON documents and between merging them into the combined backup layer doc to avoid SlowTasks when there are many agents running. 2023-09-06 19:24:15 -07:00
Hui Liu 7ca13d8f9c
support blob restore in fdbrestore (#10248) 2023-05-19 14:45:14 -07:00
Hui Liu c59e418d0f
Add TLS command line options to fdbbackup modify (#10239) 2023-05-16 08:50:50 -07:00
Hui Liu 53e68065e7
Support blob manifest backup for fdbbackup cmdline (#10091) 2023-05-08 16:07:22 -07:00
Steve Atherton 53ee26d758 Changed KeyBackedTypes to an actor file. Added TypedKeySelectors for Map and Set classes and getRange() keySelector methods. Added debug macro for KeyBackedTypes. Rewrote KeyBackedRangeMap using keyselectors on KeyBackedMap. 2023-04-18 22:21:19 -07:00
Steve Atherton 216d0be2cf
Add processID, networkAddress, and locality to layer status JSON for Backup Agents. (#9736) 
* Add processID, networkAddress, and locality to layer status JSON for Backup Agents.

* Backup/dr agent determines network address to report in Layer Status only once, when the status updater loop begins, since it is a blocking call which connects to the cluster.  And lots of code cleanup.
 2023-03-17 18:07:03 -07:00
Jingyu Zhou b08291e8e2 Add more trace events 2023-03-03 09:40:44 -08:00
Jingyu Zhou ee5154f478 Refactor decoder to read file as a whole once 
To reduce the number of network requests.
 2023-03-03 09:32:12 -08:00
Jingyu Zhou ad778cbe5e Merge branch 'main' of https://github.com/apple/foundationdb into fix 2023-03-02 09:56:30 -08:00
Jingyu Zhou c5a1fe7e6b Output in HEX format for easy regex matching 2023-02-28 14:59:06 -08:00
Jingyu Zhou aca73a213d Change mutation and KV logging to SevInfo 
Set max length as well to avoid TraceEventOverflow.
 2023-02-28 10:07:04 -08:00
Jingyu Zhou 1313a7fa25 Use KeyspaceSnapshotFile to filter range files 2023-02-27 19:41:08 -08:00
Jingyu Zhou ce123db490 Add a trace event for range files 2023-02-27 14:17:57 -08:00
Jingyu Zhou 944f5a1163 Fix merge conflict issues 2023-02-27 13:35:42 -08:00
Jingyu Zhou 7d98ef47ea Fix -t flag bug for fdbdecode 2023-02-27 13:10:37 -08:00
Vishesh Yadav dd0ea8b0cf Clang-format 2023-02-27 13:10:19 -08:00
Vishesh Yadav 58f8ede450 add command line option 2023-02-27 13:10:06 -08:00
Vishesh Yadav 3e6e31ad0b Use the RangeMapFilters 2023-02-27 13:08:55 -08:00
Jingyu Zhou 9ec9f38a50 Allow fdbbackup, fdbrestore to read keyranges from a file 2023-02-27 13:06:30 -08:00
Jingyu Zhou dd4bc82862 Refactor code 2023-02-27 13:06:01 -08:00
Jingyu Zhou 3ab88442b9 Disable filter validate by default 2023-02-27 12:18:07 -08:00
Jingyu Zhou 8c8e20af5f Use KeyRangeMap for better matching performance 2023-02-27 12:17:26 -08:00
Jingyu Zhou 3c313c8ede Fix filter delimiter and print sub versions 2023-02-27 12:07:55 -08:00
Jingyu Zhou 93a8e00440 Allow fdbdecode to read filters from a file 2023-02-27 12:07:36 -08:00
Vishesh Yadav 64820fbfe5 add filtering 2023-02-27 12:03:23 -08:00
Vishesh Yadav c66fd144c3 fdbdecode: read backup range files 2023-02-27 12:03:10 -08:00
Jingyu Zhou 7f21074916 Use KeyRangeMap for better matching performance 2023-02-27 12:02:52 -08:00
Jingyu Zhou 0a307d05ef Fix filter delimiter and print sub versions 2023-02-27 12:00:37 -08:00
Jingyu Zhou 09444d36ba Allow fdbdecode to read filters from a file 2023-02-27 12:00:24 -08:00
Zhe Wu c5c434eb71 Add more comments in fdbbackup query command, and address comments 2023-02-27 10:00:49 -08:00
Zhe Wu 4fd3a90732 fix clang build error 2023-02-26 12:38:52 -08:00
Zhe Wu ffa3467098 Explicitly using min and max restorable version from backup description in query command in stead of going throw snapshots 2023-02-26 12:17:07 -08:00
Zhe Wu 92a90c78d2 clean format 2023-02-25 18:13:55 -08:00
Zhe Wu 8a88df0e91 Query backup size from a specific snapshot 2023-02-25 17:38:27 -08:00
Zhe Wu a94dd3a430 Fix fdbbackup query returning earliest version 2023-02-25 16:44:45 -08:00
Vaidas Gasiunas f8b1da8bc6
An option to initialize client tracing in setupNetwork (#9209) 
* client_config_tester: use a generic mechanism to set specific network options

* trace_initialize_on_setup option to initialize client traces on network setup without local IP address

* trace_initialize_on_setup: Addressing review comments

* Restore correct formatting

* trace_initialize_on_setup: Update go bindings

* Include PID for identification into trace file names by default

* Use the same naming pattern for trace files in all configurations

* Empty commit
 2023-02-02 10:00:51 +01:00
Nim Wijetunga 05a8a90830
Snapshot Backup Tenant Deletion Support (#9145) 
Tenant deletion support for snapshot backups
 2023-01-26 17:46:14 -08:00
Xiaoxi Wang bbcb3cc018 extract KeyBackedConfig, StorageWiggleData class; solve template resolution problem; solve MV txn and native api conflict by splitting RunTransaction file 2023-01-02 23:34:39 -08:00
sfc-gh-tclinkenbeard 49ab88e912 Fix more clang 15 warnings 2022-12-08 14:25:36 -08:00
FoundationDB CI 86d6106dc1
format source code after switch to clang 15 2022-12-08 17:26:45 +00:00
sfc-gh-tclinkenbeard 82db9415fb Add const qualifier to more methods 
This commit mainly targets get* and is* methods
 2022-11-15 14:57:32 -08:00
Nim Wijetunga 70d30d93aa
Add CLI Argument to Override Snapshot Backup Encryption (#8661) 
* add cli option

* fix compile error

* address pr comments

* trigger build

* trigger build
 2022-11-07 20:39:18 -08:00
Nim Wijetunga 6f37f55917
Restore System Keys First in Backup/Restore Workloads (#8475) 
* system key restore ordering

* restore system keys before regular data

* atomic restore backup fix

* change testing

* fix compile error

* fix compile issue

* fix compile issues

* Trigger Build

* only split restore if encryption is enabled

* revert knob changes

* Update fdbserver/workloads/AtomicSwitchover.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbserver/workloads/AtomicSwitchover.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbserver/workloads/BackupCorrectness.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbserver/workloads/AtomicRestore.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* add todo

* strengthen check

* seperate system restore for atomic restore

* address pr comments

* address pr comments

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2022-10-26 09:38:27 -07:00
Nim Wijetunga 025359a974
Add Options to restore user/system keys in backup agent (#8497) 
* add options to restore user/system keys

* format help commands

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbbackup/backup.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbbackup/backup.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* address pr comments

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2022-10-20 09:58:00 -07:00
Nim Wijetunga 232e23c87e
Snapshot Backup Encryption (#8095) 
* inital commit

* add encryption support

* refactor passing encryption keys

* disable encryption codepath

* pass enable encryption cli arg for fdbbackup

* change encryption key passing

* only enable encryption for backup if tenant mode is required

* revert server knobs

* ignore json trace files

* --amend

* integrate ekp

* add comments

* comments

* comments

* revert knobs

* fix bug

* modify encrypted backup logic

* use common prefix

* check boundaries between begin and end key

* return system encrypt domain

* modify

* add tenant entry cache to backup

* slight revert

* format

* Trigger Build

* format

* use tenant name to fetch keys

* Trigger Build

* Trigger Build

* merge

* fix getting ekp

* fix bug

* address pr comments

* Trigger Build

* Trigger Build

* remove faulty check
 2022-09-29 14:45:47 -07:00
A.J. Beamon fda0d7223d Update backup to include system key ranges needed for tenants. Run simulated backup tests with tenants. 2022-09-22 10:00:13 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Dennis Zhou 80a0816157
flow: switch from hard coded to ApiVersion like ProtocolVersion (#8071) 
* flow: add ApiVersion to replace hard coding api version

Instead of hard coding api value, let's rely on feature versions akin to
ProtocolVersion.

* ApiVersion: remove use of -1 for latest and use LATEST_VERSION
 2022-09-02 09:28:13 +02:00