Commit Graph

862 Commits

Author SHA1 Message Date
He Liu eb4839d046
Fix audit_storage issues (#9265)
* Erase audit from DD after completion.

* Use CLIENT_KNOBS->TOO_MANY as the default number of rows for
get_audit_status.
2023-01-31 15:39:01 -08:00
Jingyu Zhou ceeb988b29
Merge pull request #9266 from saintstack/audit_storage_help
Edit of audit_storage usage and help message.
2023-01-30 20:27:26 -08:00
Josh Slocum d7ebaad6ab
Adding fdbcli and native api force-flush and force-recompact for blob granules (#9243)
* Adding fdbcli and native api force-flush and force-recompact for blob granule ranges

* throwing execption for faster retry instead of breaking loop

* fixing force flush at a non-committed, rolled back version

* fixing ide build
2023-01-30 15:00:57 -06:00
stack 2770767255 Edit of audit_storage usage and help message.
Add explanation of command return. Amend return format (slightly).
2023-01-27 15:56:28 -08:00
Yanqin Jin d461dbdd7b
Extend `metacluster status` on non-management clusters (#9179)
The metacluster status command in fdbcli currently reports some useful metacluster information when run on a
management cluster. We should update this command to report a status even on data clusters of a metacluster and
standalone clusters that do not belong to any metacluster.

- On data clusters, this would report that the cluster is a data cluster as well its name and the name of the metacluster it is a part of.
- On standalone clusters, status should report that the cluster is not part of a metacluster.

Test plan:
- CI
- Manual test
- Added new test `metacluster_fdbcli_tests.py` that can be run with ctest `ctest -R metacluster_fdbcli_tests`
2023-01-26 13:04:22 -08:00
Hui Liu 9310057446
Merge pull request #9220 from sfc-gh-huliu/fix
fix fdbcli status error
2023-01-24 18:50:07 -08:00
Hui Liu 1f07d4ac3e fix fdbcli status error 2023-01-24 09:34:11 -08:00
A.J. Beamon fd13bc04c8 Update the tenant maps to be keyed by ID 2023-01-23 14:09:12 -08:00
Hui Liu c85b984c3a blobrestore to previous point of time 2023-01-19 14:45:01 -08:00
He Liu 00203c8732
Validate Storage part II (#8471)
* Implemented AuditUtils.actor.cpp

Moved AuditUtils to fdbserver/

* Persist AuditStorageState.

* Passed persisted AuditStorageState test.

* Added audit_storage_error to indicate a corruption is caught.

Throw/Send audit_storage_error when there is a data corruption.

Added doAuditStorage() for resuming Audit.

* Load and resume AuditStorage when DD restarts.

* Generate audit id monotonically.

* Fixed minor issue AuditId/Type was not set.

* Adding getLatestAuditStates.

* Improved persisted errors and added AuditStorageCommand.actor.cpp for
fdbcli.

* Added `audit_storage` fdbcli command.

* fmt.

* Fixed null shared_ptr issue.

* Improve audit data.

* Change DDAuditFailed to SevWarn.

* Sev.

* set SERVE_AUDIT_STORAGE_PARALLELISM to 1.

* Moved AuditUtils* to fdbclient/.

* Added getAuditStatus fdbcli command.

* Refactor audit storage fdb cli commands.

* Added auditStorage in sim.

* Cleanup.

* Resolved comments.

* Resolved comments.

* Test disabling audit for sims.

* Cleanup.

Co-authored-by: He Liu <heliu@apple.com>
2023-01-15 21:46:14 -08:00
Ankita Kejriwal fe06114bcd Update the help text for `quota` command 2022-12-14 11:01:33 -08:00
FoundationDB CI 86d6106dc1
format source code after switch to clang 15 2022-12-08 17:26:45 +00:00
Daniel Luan 4be05e5a5d Upgrade C++ Standard to 20 2022-12-06 14:19:06 -08:00
A.J. Beamon 7cc79ccf68 Add tenant name to tenant map entry and remove the encrypted field 2022-12-05 12:26:01 -08:00
Jon Fu d2f0de7a36 use int macro 2022-11-30 14:31:01 -08:00
Jon Fu 3674637d37 address code review comments 2022-11-30 13:10:54 -08:00
Jon Fu 6096262597 fix compile errors 2022-11-29 16:59:20 -08:00
Jon Fu b7cba23126 Merge branch 'main' of github.com:apple/foundationdb into tenant-delete-id 2022-11-29 16:49:23 -08:00
Jon Fu 1ad359a39f add comments and test case 2022-11-29 16:49:04 -08:00
Nim Wijetunga 97713cadff
Update Encryption Mode DB Config Values (#8839)
* add encryption db config

* address pr comments

* address pr comments

* add comments

* add comment

* modify check

* change condition

* address pr comments

* simplify check

* address pr comments
2022-11-22 16:33:59 -08:00
Jon Fu 6f3c05452f introduce tenant deleteId 2022-11-22 11:54:24 -08:00
Hui Liu de8285f11d
Merge pull request #8834 from sfc-gh-huliu/status
Show blob restore in fdbcli status command
2022-11-18 16:03:30 -08:00
Hui Liu 2e62822183 Show blob restore in fdbcli status command 2022-11-17 14:22:59 -08:00
Jon Fu 349f5821d7
Merge pull request #8592 from sfc-gh-jfu/tenant-list-filter
Add option to filter the tenant list command for metacluster
2022-11-17 09:04:58 -08:00
Jon Fu 2f53c6ebd8 remove extra test check 2022-11-15 10:32:19 -08:00
Ankita Kejriwal 3deb9a413b Add fdbcli commands to get, set, and clear storage quota 2022-11-14 18:26:54 -08:00
Jon Fu 2985b255fa remove unnecessary redeclaration 2022-11-14 16:39:13 -08:00
Jon Fu a93eda244f sscanf on argument parsing 2022-11-14 16:09:31 -08:00
Jon Fu 410b4375d2 change fdbcli_tests.py 2022-11-14 13:08:56 -08:00
Jon Fu 25e1721e75 Merge branch 'main' of github.com:apple/foundationdb into tenant-list-filter 2022-11-14 12:30:28 -08:00
Hui Liu 036b6ac4df
Merge pull request #8699 from sfc-gh-huliu/cli2
Add fdbcli blobrestore to start the full restore
2022-11-11 09:23:31 -08:00
Hui Liu 5834517570 Add fdbcli blobrestore to start the full restore 2022-11-11 08:32:23 -08:00
Chaoguang Lin 9b1c8ba932
Disable the quota,suspend commands; Add timeout for fdbcli commands (#8785) 2022-11-10 21:25:11 -08:00
Jon Fu 55367daf28 Merge branch 'main' of github.com:apple/foundationdb into tenant-list-filter 2022-11-10 14:29:32 -08:00
Jon Fu 93fb151e6c add fdbcli error handling, remove invalid state and change some offset logic 2022-11-10 14:29:01 -08:00
Jon Fu 631538537f Merge branch 'main' of github.com:apple/foundationdb into tenant-list-filter 2022-11-09 10:45:45 -08:00
sfc-gh-tclinkenbeard ca572f58fb Combine *_COST_BYTE_FACTOR knobs into a single TAG_THROTTLING_PAGE_SIZE knob 2022-11-08 07:44:01 -08:00
Jon Fu 8ef0411b32 address code review comments and introduce offset parameter 2022-11-03 11:39:39 -07:00
sfc-gh-tclinkenbeard 69f669f36b Change type of value in QuotaCommand.actor.cpp 2022-10-31 18:58:55 -07:00
Jon Fu 098793893e move hints to correct generator 2022-10-26 14:50:59 -07:00
Jon Fu b17c3fecbb add invalid tenant state and assertion in metacluster consistency 2022-10-26 14:37:00 -07:00
sfc-gh-tclinkenbeard 1c119be26d Merge remote-tracking branch 'origin/main' into expose-txn-cost 2022-10-25 13:50:43 -07:00
Jon Fu f21fcf67ce initial commit to allow tenant list filtering in metacluster 2022-10-25 12:01:29 -07:00
sfc-gh-tclinkenbeard 32ae7bb529 Merge remote-tracking branch 'origin/main' into expose-txn-cost 2022-10-23 12:59:07 -07:00
sfc-gh-tclinkenbeard b442705dc7 Change units for tag quota enforcement from pages to bytes 2022-10-23 12:57:19 -07:00
sfc-gh-tclinkenbeard 78b6779f11 Update fdbcli_tests.py quota test 2022-10-22 19:37:33 -07:00
sfc-gh-tclinkenbeard 2a2d52ca50 Improve error messages for fdbcli quota commands 2022-10-22 18:24:48 -07:00
Kevin Hoxha 2dac017ea2 fdbcli: Cancel checkStatus actor when prompting for description in setknob 2022-10-20 21:31:52 -07:00
sfc-gh-tclinkenbeard 8c082dac4e Expand quota tests in fdbcli_tests.py 2022-10-17 18:56:11 -07:00
sfc-gh-tclinkenbeard 7e4f380423 Add tests for fdbcli quota commands 2022-10-17 16:27:45 -07:00