Commit Graph

207 Commits

Author SHA1 Message Date
Xiaoxi Wang 801c1e8be2 Schema fix 2022-06-29 23:07:04 -07:00
Xiaoxi Wang 6bb4e341f9
Merge pull request #7110 from sfc-gh-xwang/features/ppw-pause-state
Adding paused/running wiggling status to status json and also the last running/paused timestamp
2022-06-16 14:27:18 -07:00
Xiaoxi Wang a311cc28cc solve some comments 2022-06-16 11:07:21 -07:00
Jingyu Zhou b9ff6bc129 Address AJ's comments 2022-06-08 09:38:32 -07:00
Xiaoxi Wang 382f0fc4a2 merge upstream/main 2022-05-17 10:20:51 -07:00
Sagar Vemuri ebf11d5d48 Update schemas with the tenants information 2022-05-16 11:09:10 -07:00
Xiaoxi Wang 2717cee1f9 Merge branch 'features/read-skew' into readaware 2022-05-09 16:12:09 -07:00
Xiaoxi Wang e6615c29a3 add perpetual wiggler state to json 2022-05-09 11:51:08 -07:00
Xiaoxi Wang e9bf7c0851 fix unknown schema check 2022-04-25 17:23:20 -07:00
Xiaoxi Wang 182f244e2e fix schema format 2022-04-25 10:28:45 -07:00
Xiaoxi Wang e8477e15ce Merge remote-tracking branch 'upstream/main' into readaware 2022-04-25 09:55:52 -07:00
Xiaoxi Wang adeaf52db3
Merge branch 'apple:main' into ppw-store 2022-04-25 09:40:45 -07:00
Xiaoxi Wang 269e94cb71 add store type to status json 2022-04-24 22:53:06 -07:00
Xiaoxi Wang 0639810b66 merge upstream/main 2022-04-22 11:09:15 -07:00
Bharadwaj V.R 822eb9ec26
Merge branch 'apple:main' into dbcorever 2022-04-22 08:08:34 -07:00
Zhe Wang 6c9ff6ee5e
Add sharded rocksdb type (#6862)
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Bharadwaj V.R 11f2827c82 Make names even across change and add lowest-compatible-version to status json 2022-04-15 13:57:58 -07:00
Bharadwaj V.R 831bee34f8 Add server version to status json 2022-04-13 08:23:02 -07:00
Xiaoxi Wang aba9d85560 merge main 2022-04-06 09:57:52 -07:00
Steve Atherton b179813989 Updated status schema and fixed spacing. 2022-04-01 17:21:35 -07:00
He Liu dd15489605 rename ssd-rocksdb-experimental as ssd-rocksdb-v1. 2022-03-29 10:53:38 -07:00
Xiaoxi Wang d93b57dd88 conflict solving 2022-03-24 20:45:51 -07:00
Xiaoxi Wang 1b631a9263 solve conflict with main 2022-03-24 16:29:11 -07:00
Josh Slocum f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Josh Slocum e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
A.J. Beamon 72a34945ce Add the ability to disable tenants. Server processes verify the ID of tenants being read or written. 2022-03-06 21:54:21 -08:00
A.J. Beamon 5fa9d3e1b7 Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode. 2022-03-06 21:54:21 -08:00
Xiaoxi Wang e73c0a31e6 add wiggle_server_ids and wiggle_server_addresses in status json 2022-03-02 10:03:23 -08:00
Xiaoxi Wang f48ba989c3 add status 2022-03-01 15:27:27 -08:00
Josh Slocum 38a75a8b89 Merge branch 'main' into blob_integration 2022-02-17 17:47:38 -06:00
Vaidas Gasiunas 092b5cee4b MVC2.0: Rollback added code 2022-02-14 13:50:42 -08:00
Xiaoxi Wang 6dc5921575
createdTime based storage wiggler (#6219)
* add storagemetadata

* add StorageWiggler;

* fix serverMetadataKey bug

* add metadata tracker in storage tracker

* finish StorageWiggler

* update next storage ID

* change pid to server id

* write metadata when seed SS

* add status json fields

* remove pid based ppw iteration

* fix time expression

* fix tss metadata nonexistence; fix transaction retry when retrieving metadata

* fix checkMetadata bug when store type is wrong

* fix remove storage status json

* format code

* refactor updateNextWigglingStoragePID

* seperate storage metadata tracker and store type tracker

* rename pid

* wiggler stats

* fix completion between waitServerListChange and storageRecruiter

* solve review comments

* rename system key

* fix database lock timeout by adding lock_aware

* format code

* status json

* resolve code format/naming comments

* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>

* fix omit start rount

* format code

* status json reset

* solve status json format

* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions

* status timestamp
2022-02-04 15:04:30 -08:00
Ata E Husain Bohra 87ee4cf958 Add new FDB EncryptKeyProxy role
Major changes includes:

1. Add a new FDB role responsible- EncyrptKeyProxy. The role is
   responsible to expose APIs to fetch encyrption keys interacting
   with external Encryption KeyManager interface.
2. The process is a FDB singleton process following similar recruitment
   rules as other singleton processes in the system.
3. Code to recruit the worker process; given the encryption keys are
   needed during recovery (decode TLog records), for now the process
   is co-located in same datacenter as ClusterController.
4. Skeleton process actor code; more functionality will be added in
   subsequent PRs.

NOTE: The code is protected under a SERVER_KNOB with the default
      value as 'false' for now.
2022-01-25 17:38:27 -08:00
Ata E Husain Bohra 936bf5336a
Revert "Revert "Refactor: ClusterController driving cluster-recovery state machine" (#6191)
* Revert "Revert "Refactor: ClusterController driving cluster-recovery state machine""

Major changes includes:
1. Re-revert Sequencer refactor commits listed below (in listed order):
1.a. This reverts commit bb17e194d9.
1.b. This reverts commit d174bb2e06.
1.c. This reverts commit 30b05b469c.

2. Update Status.actor to track ClusterController interface to track
   recovery status.
3. Introduce a ServerKnob to define "cluster recovery trace event"
   prefix; for now keeping it as "Master", however, it should allow
   smooth transition to "Cluster" prefix as it seems more appropriate.
2022-01-06 12:15:51 -08:00
Suraj Gupta 640cee2072 Start BM without config change if config is enabled. 2021-12-10 14:00:34 -06:00
Suraj Gupta fc3376fe8f Move client knob to database config for blob granules. 2021-12-10 14:00:34 -06:00
Steve Atherton c53f5aa110 Renamed redwood to redwood-1-experimental and file extension to .redwood-v1. 2021-11-16 02:15:22 -08:00
Vaidas Gasiunas 40da5a80f9 Merge remote-tracking branch 'apple/master' into multi-version-client-2 2021-10-26 19:29:10 +02:00
Josh Slocum 5f0ec0612a Merge branch 'feature-range-feed' into blob_full 2021-10-13 15:44:35 -05:00
Vaidas Gasiunas 61fb967484 MVC2.0: Add a clientlib metadata attribute for checksum algorithm 2021-10-07 18:06:22 +02:00
Vaidas Gasiunas 5360abb238 MVC2.0: Operation to list uploaded libraries with various filters;
Introducing constants for attribute names and platform values
2021-10-06 18:01:46 +02:00
Vaidas Gasiunas cda0a5f931 Operation to upload client library binary in to system keyspace 2021-10-06 18:01:46 +02:00
Neethu Haneesha Bingi 3e79299898 Locality filter support to perpetual storage wiggler feature. 2021-09-30 10:00:33 -07:00
Suraj Gupta a4bcd3919d Add exclusive process class for Blob Worker.
Also introduces a specific machine in the simulated cluster
to test blob worker (similar to what's done for storage cache).
2021-09-23 16:54:44 -04:00
Suraj Gupta 5fa6c687d6 Add blob manager as a singleton. 2021-09-23 10:45:37 -04:00
Josh Slocum 9992a7b33f Added StorageMigrationType and cli commands 2021-09-14 09:55:41 -05:00
Neethu Haneesha Bingi 66f2518405 exclude to work with any locality data match. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi cbe714acd0 Status json schema update, includelocalities back for consistency check, review comments. 2021-06-23 18:03:27 -07:00
Xiaoxi Wang 454f9e9c89 update json schemas 2021-06-17 20:20:39 +00:00