6dc5921575
createdTime based storage wiggler ( #6219 )
...
* add storagemetadata
* add StorageWiggler;
* fix serverMetadataKey bug
* add metadata tracker in storage tracker
* finish StorageWiggler
* update next storage ID
* change pid to server id
* write metadata when seed SS
* add status json fields
* remove pid based ppw iteration
* fix time expression
* fix tss metadata nonexistence; fix transaction retry when retrieving metadata
* fix checkMetadata bug when store type is wrong
* fix remove storage status json
* format code
* refactor updateNextWigglingStoragePID
* seperate storage metadata tracker and store type tracker
* rename pid
* wiggler stats
* fix completion between waitServerListChange and storageRecruiter
* solve review comments
* rename system key
* fix database lock timeout by adding lock_aware
* format code
* status json
* resolve code format/naming comments
* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>
* fix omit start rount
* format code
* status json reset
* solve status json format
* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions
* status timestamp
2022-02-04 15:04:30 -08:00
13da5d394b
Remove extra StringRef copy
2022-01-26 19:22:40 -08:00
e2e853a30b
Use ObjectWriter to write storage cache data
...
This allows deserialization of the on disk data in lower FDB versions
that don't contain fields added in main.
2022-01-25 19:24:14 -08:00
964d0209ca
Merge pull request #5637 from sfc-gh-ljoswiak/features/data-loss-prevention
...
Data loss protection when joining new cluster
2021-11-15 15:26:32 -08:00
30867750b5
Add protection against storage and tlog data deletion when joining a new cluster
2021-11-09 12:29:47 -08:00
875824b186
MVC2.0: Notify clients about relevant changes of client libraries
2021-10-27 23:43:40 +02:00
40da5a80f9
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 19:29:10 +02:00
37bc41abbb
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 18:51:43 +02:00
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
3f7df58a77
fixed a number of issues
2021-10-19 13:56:52 -07:00
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
5f0ec0612a
Merge branch 'feature-range-feed' into blob_full
2021-10-13 15:44:35 -05:00
5b6f8b2abb
added the known committed version to change feeds
2021-10-11 13:53:36 -07:00
5c642f706e
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
cda0a5f931
Operation to upload client library binary in to system keyspace
2021-10-06 18:01:46 +02:00
3e79299898
Locality filter support to perpetual storage wiggler feature.
2021-09-30 10:00:33 -07:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
a1f6dcc2d5
merge from master
2021-09-22 13:07:38 -07:00
914b1ee960
fmt
2021-09-20 12:01:47 -07:00
95f64f97de
resolved some fmt issues
2021-09-20 11:58:30 -07:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
cf1626f553
Added comments.
2021-09-10 11:10:10 -07:00
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
2ea055371b
Introduced serverKeysTrueEmptyRange to indicate the SS that the new
...
shard is empty.
2021-09-09 14:15:07 -07:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
3b011408f8
Added sequence numbers and locks to blob worker and manager
2021-08-27 16:33:07 -05:00
a278d2977a
renamed range feeds to change feeds, data distribution support almost complete
2021-08-13 14:27:15 -07:00
76a3c47e42
Got full demo working
2021-08-12 10:18:58 -05:00
df6bbb6a08
fix: needed bigEndian instead of littleEndian
2021-08-10 14:37:05 -07:00
921a2cfca1
Merge branch 'feature-range-feed' into blob_full
2021-08-10 11:25:48 -05:00
f1eedf4910
fixed serialization of rangefeed durable keys
2021-08-09 17:15:51 -07:00
e5b0cf20aa
Reader can read blob worker interface and mapping from db, read files, apply mutations, and return result
2021-08-06 11:02:05 -05:00
e83b27387a
POC worker that dumps fdb range to an s3 object
2021-08-06 11:01:37 -05:00
a931f70cfb
Added fdbcli for blobrange
2021-08-06 11:01:37 -05:00
0989c28a6b
made range feeds durable on the storage server
2021-07-30 15:23:42 -07:00
406562b282
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
2021-07-27 17:04:58 -07:00
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
15fe00927a
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbclient/SystemData.cpp
# fdbclient/SystemData.h
# fdbserver/storageserver.actor.cpp
# flow/ProtocolVersion.h
2021-07-12 19:10:14 -07:00
ab0ab765c8
Merge pull request #4965 from sfc-gh-jslocum/tss_mapping_changes
...
TSS Mismatch Changes
2021-06-29 08:24:25 -07:00
4ad5926a25
Snake naming of keys and added comments to all new functions.
2021-06-23 18:03:27 -07:00
73752f441b
exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion.
2021-06-23 18:03:27 -07:00
62355571d0
exclude servers based on locality match
2021-06-23 18:03:27 -07:00
826e70916c
Merge remote-tracking branch 'origin/master' into config-db
2021-06-17 09:47:41 -07:00
56dadaa428
TSS Mismatch Changes
2021-06-11 23:13:16 +00:00
65c49fc1e4
Merge pull request #4934 from sfc-gh-jslocum/tss_quarantine
...
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-11 11:52:42 -07:00
Xiaoxi Wang