9c786e6d1e
Merge pull request #9854 from sfc-gh-ajbeamon/metacluster-separate-project
...
Metacluster refactoring
2023-04-04 09:43:41 -07:00
f2a406f609
Add blob manifest and mutation log status to "status json" ( #9856 )
2023-04-03 18:30:13 -07:00
807646675c
Refactor the metacluster project into smaller files, and reorganize the namespaces. Move some metacluster and tenant testing helpers into the metacluster project.
2023-03-30 16:20:09 -07:00
e61748c7d5
Move metacluster into its own directory and static library
2023-03-30 16:07:49 -07:00
6c337e5c7a
fix metacluster get segfault ( #9693 )
2023-03-23 14:21:25 -04:00
5504a58a12
Fix formatting
2023-03-13 16:04:25 -07:00
d39cda610a
Merge branch 'main' into metacluster-improvements
...
# Conflicts:
# fdbcli/TenantCommands.actor.cpp
2023-03-13 15:58:39 -07:00
b751d0f87c
Add tenant getId api to fdbcli ( #9658 )
...
* add tenant getId api to fdbcli
* add to fdbcli tests and address review comments
* fix fdbcli tests
2023-03-13 18:06:03 -04:00
45056370b8
Merge branch 'main' into metacluster-improvements
2023-03-13 13:14:09 -07:00
18cf523f49
Merge pull request #9660 from sfc-gh-ajbeamon/tenant-id-restore-safety
...
Disallow repopulating a management cluster from a data cluster with matching tenant ID prefix
2023-03-13 13:12:30 -07:00
ffa0aa4a7b
Update the expected error message from exclude in fdbcli_tests ( #9648 )
2023-03-10 19:49:20 -08:00
6b2185c707
Fix off-by-one in token hint generator for metacluster restore
2023-03-10 18:50:34 -08:00
cbc330697c
Disallow repopulating a management cluster from a data cluster with matching tenant ID prefix unless forced. Remember the largest used tenant ID on the data cluster and use it to update the management cluster tenant ID when force repopulating the same ID.
2023-03-10 15:36:37 -08:00
06decf1141
Fix memory error in fdbcil tenant lock command
2023-03-10 10:31:59 -08:00
fa1f20a9ff
Update the help text for some fdbcli tenant commands
2023-03-07 11:47:21 -08:00
684464cebb
Merge pull request #9594 from sfc-gh-jfu/fix-fdbcli-segfault
...
Fix fdbcli segfault
2023-03-07 08:28:14 -08:00
5ff0bc3f87
Merge pull request #9576 from sfc-gh-satherton/storage-configure-refactor
...
Storage and log engine configuration support / refactor a few things.
2023-03-07 02:10:14 -08:00
7273723a43
Add the hotrange fdbcli command ( #9570 )
...
* Add the hotrange fdbcli command
* Remove the unnecessary state
* Add the doc about the hotrange command
2023-03-06 14:46:52 -08:00
9313e5f653
fix fdbcli segfault
2023-03-06 14:18:46 -08:00
7a0b3c05b9
Merge pull request #9540 from sfc-gh-huliu/timestamp
...
Report restore phase start time and eta
2023-03-06 14:06:23 -08:00
94a9e37583
Merge pull request #9526 from sfc-gh-xwang/fix/main/killall
...
wait extra time to make sure rebootWorker request sent to storage server
2023-03-06 11:25:09 -08:00
50d567b5a5
Refactored some parts of database configuration to support log_engine=<name> and storage_engine=<name> and generate these when converting a DatabaseConfig JSON object to a `configure` command. Refactored `fileconfigure` and simulation setup to use the same JSON -> configure function as the same code was copy/pasted to both places but only one has been kept up to date with new features. Renamed Redwood to `ssd-redwood-1` canonically but the experimental name is still supported for backward compatibility.
2023-03-04 20:52:31 -08:00
b2d497a3b2
Report restore phase start timestamp
2023-03-03 18:09:51 -08:00
e123a51b50
remove the time guard on fdbcli commands in tests
2023-03-01 11:09:47 -08:00
576c2a9c93
wait extra time to make sure rebootWorker request sent to storage server
2023-02-28 22:41:30 -08:00
20874d8575
Merge pull request #9502 from sfc-gh-ajbeamon/metacluster-tenant-lock-support
...
Metacluster tenant lock support
2023-02-27 21:19:03 -07:00
469e77158f
Add metacluster support for tenant locking
2023-02-27 16:53:13 -08:00
29a406948a
Merge pull request #9370 from sfc-gh-mpilman/features/tenant-lock-fdbcli
...
fdbcli commands for tenant lock
2023-02-27 16:18:51 -08:00
ed1079fc7d
Remove wrong comment
2023-02-27 15:27:16 -07:00
1689e36d14
Fix uid extraction from lock command output
2023-02-27 15:24:07 -07:00
bcc05b1058
Improve support for prebuilt boost
2023-02-27 15:38:58 -06:00
6a55f0da26
Add missing newlines in tenant lock fdbcli command output
2023-02-27 14:02:29 -07:00
a0e347c7ba
Merge remote-tracking branch 'origin/main' into features/tenant-lock-fdbcli
2023-02-27 09:09:20 -07:00
217b52103c
Clean up some code for the tenant configure command; Add a second test with an invalid configuration string, this time with an unknown token and a prior valid configuration string
2023-02-25 13:08:02 -08:00
8c3ee768a2
Add an option to allow exceeding the tenant group capacity limit when changing tenant configuration
2023-02-24 21:01:36 -08:00
1c71056e26
Merge pull request #9479 from sfc-gh-nwijetunga/nim/enforce-metacluster-tenant-mode
...
Enforce Disabled Tenant Mode in Metacluster
2023-02-24 19:27:57 -08:00
6b121de6a6
Merge pull request #9464 from jzhou77/fix
...
Add exclude to fdbcli's configure command
2023-02-24 16:31:02 -08:00
836ec1a908
add failure test
2023-02-24 15:42:29 -08:00
c8b7cff10c
fix api test
2023-02-24 15:26:59 -08:00
eca98afcb0
metacluster check tenant mode
2023-02-24 13:59:54 -08:00
01fab190b3
add arrayGenerator for w|rw option
2023-02-24 12:01:43 -07:00
e0df60bd92
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2023-02-24 11:33:29 -07:00
6c15506c36
Fixed tests
2023-02-24 11:32:37 -07:00
82bd5796f5
Fix merge conflicts
2023-02-23 16:04:37 -08:00
33f8e90f9f
Split tenant group metadata ( #9446 )
...
* initial commit to split tenant group metadata
* attempt to fix merge errors
* fix compile errors and adjust existing tests
* fix infinite loop and extra ACTOR tag
* direct assignment instead of store
* direct assign instead of store (missed a few)
2023-02-23 18:11:49 -05:00
1f1dc5e768
Allow a comma separated list of excluded addresses
2023-02-23 14:29:08 -08:00
6ac8720364
Add exclude to fdbcli's configure command
...
Right now this only allows one server address being excluded. This is useful
when the database is unavailable but we want the recruitment to skip some
particular processes.
Manually tested the concept works with a loopback cluster.
2023-02-23 14:28:20 -08:00
c1f80fe471
Merge remote-tracking branch 'origin/main' into features/tenant-lock-fdbcli
2023-02-23 15:16:14 -07:00
193e517cc4
Address review comments and move lock ID into TenantMapEntry
2023-02-23 14:25:36 -07:00
46dd75ed06
Fix compilation issues after merging with main
2023-02-22 16:51:16 -07:00
230170a431
Merge remote-tracking branch 'sfc/features/tenant-lock-fdbcli' into features/tenant-lock-fdbcli
2023-02-22 16:29:28 -07:00
7d7ca0bb34
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2023-02-22 16:28:53 -07:00
b1762916fb
Merge remote-tracking branch 'origin/main' into features/tenant-lock-fdbcli
2023-02-22 16:27:24 -07:00
a5a8c57a38
Fix some merge issues and missing updates to use new boolean parameters
2023-02-22 12:52:58 -08:00
ec79ecce73
Add a boolean parameter for ForceRemove; rename ForceJoinNewMetacluster to ForceJoin
2023-02-22 12:26:24 -08:00
5edf78ddce
fix fdbcli_tests
2023-02-21 13:00:57 -08:00
37fa579587
fix compile issues
2023-02-21 11:46:11 -08:00
2d74d10a91
use MetaclusterAPI namespace over TenantAPI for TenantState
2023-02-21 11:33:36 -08:00
428eb07766
Merge branch 'main' of github.com:apple/foundationdb into split-tenant-metadata
2023-02-21 10:11:35 -08:00
bef39ffc64
Merge pull request #9084 from sfc-gh-akejriwal/fdbcli
...
Add fdbcli commands for checking the status of and clearing old idempotency ids
2023-02-21 08:44:46 -08:00
bfb3ffc509
added c and java apis for granule flush ( #9412 )
2023-02-20 10:28:11 -06:00
edb7a51b7e
Revert "let client supply restore id"
...
This reverts commit 5fe32b8503
2023-02-17 14:37:22 -08:00
5fe32b8503
let client supply restore id
2023-02-17 14:01:58 -08:00
07cdd34f26
Update metacluster_fdbcli_tests.py ( #9406 )
...
Update metacluster_fdbcli_tests.py
This commit has no impact on functionality.
Test plan:
- ctest -R metacluster_fdbcli_tests.py
2023-02-17 12:40:02 -08:00
53fc43a3a6
Merge branch 'main' of github.com:apple/foundationdb into split-tenant-metadata
2023-02-16 14:46:03 -08:00
c3d6ae0213
Throw invalid_tenant_configuration when changing assigned cluster ( #9350 )
...
Since we currently do not support tenant movement, we should as well explicitly disallow changing the assigned
cluster of a tenant during configuration by throwing `invalid_tenant_configuration` for now.
Test plan:
- add coverage for changing assigned cluster during tenant configuration
- fdbcli
- simulation tests
2023-02-16 14:20:59 -08:00
ab15478ef9
Merge branch 'main' of github.com:apple/foundationdb into split-tenant-metadata
2023-02-16 10:04:07 -08:00
b79885d5b9
Merge remote-tracking branch 'origin/main' into features/tenant-lock-fdbcli
...
# Conflicts:
# fdbserver/ApplyMetadataMutation.cpp
2023-02-15 18:37:18 +01:00
7284e691fb
Fix a few minor restore bugs and add a dry-run mode. Some improvements to the fdbcli output.
2023-02-14 12:28:55 -08:00
32684c802b
Add tenant lock commands and fdcli tests
2023-02-14 16:05:21 +01:00
983f677821
Merge branch 'main' of github.com:apple/foundationdb into fdbcli
2023-02-13 16:13:11 -08:00
958ff862e0
Fix some merge issues
2023-02-13 12:59:48 -08:00
98407809d9
Merge branch 'main' into metacluster-mgmt-restore
...
# Conflicts:
# fdbcli/MetaclusterCommands.actor.cpp
# fdbclient/Metacluster.cpp
# fdbclient/include/fdbclient/MetaclusterManagement.actor.h
# fdbserver/workloads/MetaclusterManagementWorkload.actor.cpp
# tests/CMakeLists.txt
2023-02-13 12:30:33 -08:00
bba782cc56
Add command
2023-02-13 18:21:55 +01:00
e6021f8326
Add Jon's metacluster concurrency test and fix various bugs that it found
2023-02-11 15:15:32 -08:00
ee1b48323d
Merge pull request #9346 from sfc-gh-nwijetunga/nim/global-tenant-ids
...
Support for Two Byte Prefix for Tenant IDs
2023-02-11 11:31:24 -08:00
4579a4319d
Merge branch 'main' into storage-quota-in-tenant-metadata-space
2023-02-11 09:04:15 -08:00
ca3fbd2ecf
Merge branch 'main' of github.com:apple/foundationdb into fdbcli
2023-02-10 12:15:43 -08:00
df165e43ad
Since FDB now limits memory by RSS, report RSS in `status details` view.
2023-02-10 11:30:29 -08:00
13eee09ce8
Merge branch 'main' into metacluster-mgmt-restore
2023-02-10 10:58:01 -08:00
4b13c9c211
Make a few minor fixes, refactor some code for clarity, and improve throughput of repopulating a management cluster
2023-02-10 10:41:55 -08:00
f775116214
fix test
2023-02-10 02:26:32 -08:00
f9a68056ac
Add support for modifying a data cluster that is being restored so that we can manage conflicts
2023-02-09 15:33:40 -08:00
b7ef50d1f8
inital commit
2023-02-09 14:32:29 -08:00
2d59c5681d
Bug fixes and test improvements for management cluster restoration
2023-02-09 08:42:23 -08:00
d981d5e9c2
initial commit to separate metadata
2023-02-08 10:44:26 -08:00
42eb82cd30
Merge branch 'main' into storage-quota-in-tenant-metadata-space
2023-02-07 15:11:29 -08:00
0410f7d9a1
Add intial support for restoring a management cluster in a metacluster
2023-02-07 15:08:46 -08:00
99bc126667
Merge branch 'main' into storage-quota-in-tenant-metadata-space
2023-02-06 09:32:24 -08:00
f1dd3aea92
Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/listTenant
2023-02-03 15:27:21 -08:00
b757e8914a
fix BOOST_SYSTEM_NO_LIB redefinition in CI
2023-02-03 13:47:50 -08:00
a447ec98a0
Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/listTenant
2023-01-31 16:25:07 -08:00
eb4839d046
Fix audit_storage issues ( #9265 )
...
* Erase audit from DD after completion.
* Use CLIENT_KNOBS->TOO_MANY as the default number of rows for
get_audit_status.
2023-01-31 15:39:01 -08:00
e383ea2138
push down offset calculation to listTenants; optimize fdbcli tenant list implementation
2023-01-31 14:42:38 -08:00
851b6e7e56
rename APIs
2023-01-31 11:16:17 -08:00
ceeb988b29
Merge pull request #9266 from saintstack/audit_storage_help
...
Edit of audit_storage usage and help message.
2023-01-30 20:27:26 -08:00
56b716040f
Various fixes and cleanup
2023-01-30 13:34:45 -08:00
be5b0f451d
Expand the restore test; fix some bugs mainly around renaming and changing tenant configuration during a restore
2023-01-30 13:34:45 -08:00
af55feb493
Fix some test issues
2023-01-30 13:34:18 -08:00
91f5347551
Add some simulated testing to the metacluster restore process; fix various issues found by the testing.
2023-01-30 13:34:18 -08:00
A.J. Beamon