ed9f95b299
simplifying blob manager recovery by always treating the DB granule mapping as the source of truth, and persisting all boundary changes there BEFORE updating workers
2022-02-25 14:46:14 -06:00
38a75a8b89
Merge branch 'main' into blob_integration
2022-02-17 17:47:38 -06:00
092b5cee4b
MVC2.0: Rollback added code
2022-02-14 13:50:42 -08:00
6dc5921575
createdTime based storage wiggler ( #6219 )
...
* add storagemetadata
* add StorageWiggler;
* fix serverMetadataKey bug
* add metadata tracker in storage tracker
* finish StorageWiggler
* update next storage ID
* change pid to server id
* write metadata when seed SS
* add status json fields
* remove pid based ppw iteration
* fix time expression
* fix tss metadata nonexistence; fix transaction retry when retrieving metadata
* fix checkMetadata bug when store type is wrong
* fix remove storage status json
* format code
* refactor updateNextWigglingStoragePID
* seperate storage metadata tracker and store type tracker
* rename pid
* wiggler stats
* fix completion between waitServerListChange and storageRecruiter
* solve review comments
* rename system key
* fix database lock timeout by adding lock_aware
* format code
* status json
* resolve code format/naming comments
* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>
* fix omit start rount
* format code
* status json reset
* solve status json format
* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions
* status timestamp
2022-02-04 15:04:30 -08:00
13da5d394b
Remove extra StringRef copy
2022-01-26 19:22:40 -08:00
e2e853a30b
Use ObjectWriter to write storage cache data
...
This allows deserialization of the on disk data in lower FDB versions
that don't contain fields added in main.
2022-01-25 19:24:14 -08:00
1494f8216b
Reworked blob manager recovery to be more efficient and handle overlapping ongoing splits
2022-01-24 14:12:45 -06:00
08f8700636
Fixed a couple bugs in blob manager recovery
2022-01-18 18:47:42 -06:00
9d9cb961a1
reworked blob manager recovery to be more efficient
2022-01-18 14:22:58 -06:00
932f68e1b3
Refactor bstore and add watch key.
2021-12-10 16:43:58 -06:00
22d72ec9dd
init
2021-12-10 16:43:58 -06:00
efb21ca6a1
Merge branch 'master' into blob_integration
2021-11-24 13:17:54 -06:00
964d0209ca
Merge pull request #5637 from sfc-gh-ljoswiak/features/data-loss-prevention
...
Data loss protection when joining new cluster
2021-11-15 15:26:32 -08:00
f140028086
Merge branch 'master' into blob_integration
2021-11-15 15:09:40 -06:00
30867750b5
Add protection against storage and tlog data deletion when joining a new cluster
2021-11-09 12:29:47 -08:00
3b711af061
Fixed a couple rollback issues and endianness of versionstamp version
2021-10-29 12:07:16 -05:00
875824b186
MVC2.0: Notify clients about relevant changes of client libraries
2021-10-27 23:43:40 +02:00
40da5a80f9
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 19:29:10 +02:00
37bc41abbb
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 18:51:43 +02:00
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
3f7df58a77
fixed a number of issues
2021-10-19 13:56:52 -07:00
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
5f0ec0612a
Merge branch 'feature-range-feed' into blob_full
2021-10-13 15:44:35 -05:00
5b6f8b2abb
added the known committed version to change feeds
2021-10-11 13:53:36 -07:00
5c642f706e
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
cda0a5f931
Operation to upload client library binary in to system keyspace
2021-10-06 18:01:46 +02:00
3e79299898
Locality filter support to perpetual storage wiggler feature.
2021-09-30 10:00:33 -07:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
a1f6dcc2d5
merge from master
2021-09-22 13:07:38 -07:00
914b1ee960
fmt
2021-09-20 12:01:47 -07:00
95f64f97de
resolved some fmt issues
2021-09-20 11:58:30 -07:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
cf1626f553
Added comments.
2021-09-10 11:10:10 -07:00
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
2ea055371b
Introduced serverKeysTrueEmptyRange to indicate the SS that the new
...
shard is empty.
2021-09-09 14:15:07 -07:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
3b011408f8
Added sequence numbers and locks to blob worker and manager
2021-08-27 16:33:07 -05:00
a278d2977a
renamed range feeds to change feeds, data distribution support almost complete
2021-08-13 14:27:15 -07:00
76a3c47e42
Got full demo working
2021-08-12 10:18:58 -05:00
df6bbb6a08
fix: needed bigEndian instead of littleEndian
2021-08-10 14:37:05 -07:00
921a2cfca1
Merge branch 'feature-range-feed' into blob_full
2021-08-10 11:25:48 -05:00
f1eedf4910
fixed serialization of rangefeed durable keys
2021-08-09 17:15:51 -07:00
e5b0cf20aa
Reader can read blob worker interface and mapping from db, read files, apply mutations, and return result
2021-08-06 11:02:05 -05:00
e83b27387a
POC worker that dumps fdb range to an s3 object
2021-08-06 11:01:37 -05:00
a931f70cfb
Added fdbcli for blobrange
2021-08-06 11:01:37 -05:00
Josh Slocum