Commit Graph

18262 Commits

Author SHA1 Message Date
Josh Slocum a42c80faa9 Tightening up memory management in the blob worker 2022-02-01 14:52:28 -06:00
Josh Slocum d0e89ecdd5 Including Change Feeds in fetch keys budget 2022-02-01 14:39:10 -06:00
Josh Slocum 25f291cf6d Improving blob manager recovery of client blob ranges 2022-02-01 14:38:54 -06:00
Josh Slocum 2c22d07b39 Several change feed popping fixes for behind SS 2022-02-01 10:27:41 -06:00
Josh Slocum 45f4235f2b Fixing worker dying in BM recovery race 2022-01-31 20:59:58 -06:00
Josh Slocum d46ef148bb Fixing granule revoke on recover 2022-01-31 19:59:41 -06:00
Josh Slocum 30d8c593c8 Fix for new manager recovery 2022-01-31 19:59:41 -06:00
Josh Slocum d0113a6776 Added mechanism for blob manager to poll blob workers for their granule assignments, and used that to improve manager recovery 2022-01-31 19:59:41 -06:00
Josh Slocum 25a0d857fa Fixing issue with change_feed_popped 2022-01-31 17:19:52 -06:00
Josh Slocum d4931348c1 Fix for new change feed movement 2022-01-31 11:18:47 -06:00
Josh Slocum 066944ad1c Merge branch 'blob_integration' of github.com:apple/foundationdb into blob_integration 2022-01-31 10:45:16 -06:00
Josh Slocum f2b9eb1d4b Fixing potential bug of mixed-version FDB result in granule verifier 2022-01-31 10:44:09 -06:00
Josh Slocum d6dc468793 Fixing potential bug of mixed-version FDB result in granule verifier 2022-01-31 09:51:18 -06:00
Josh Slocum 05f27eac50 range mover fix 2022-01-31 08:27:37 -06:00
Josh Slocum c3cece552c Fixes for previous 2022-01-28 15:19:00 -06:00
Josh Slocum bc9572028f Added AtomicOps tests for blob granules 2022-01-28 14:40:43 -06:00
Josh Slocum cf7ed4ee40 Simplify change feed fetching to not inject anything at all during fetch and rely on reading from the remote feeds 2022-01-28 14:21:26 -06:00
Josh Slocum 2d3b216f60 Persisting change feed popped version in all cases to enable popped checking 2022-01-28 12:27:04 -06:00
Josh Slocum ac1fd056dd Added change feed popped read guard for blob workers 2022-01-28 10:45:33 -06:00
Josh Slocum 10c3cc870f Fixing new race in change feed cleanup while still working for fetch+cleanup race 2022-01-28 09:26:36 -06:00
Josh Slocum cfbb3f5b2b Adding random prefix to blob worker generated files 2022-01-28 07:43:34 -06:00
Josh Slocum df1a21564b Fixed Blob Worker Rollback issue 2022-01-28 06:21:07 -06:00
Josh Slocum 91df569fdd Fetch Change Feed metadata at the start of fetchKeys to fix mutation race 2022-01-27 18:56:07 -06:00
Josh Slocum c26e11c2c3 Adding extra debugid to change feed stream requests to track things better 2022-01-27 18:53:27 -06:00
Josh Slocum bc7b76b407 Fix for gotAll in getChangeFeedMutations 2022-01-27 14:25:08 -06:00
Josh Slocum bf6ed8348f A couple workload improvements 2022-01-27 10:26:35 -06:00
Josh Slocum 74c8167745 Not using krm for blob manager worker mapping recovery 2022-01-27 10:26:35 -06:00
Josh Slocum ce5faa470b Merge branch 'ss_cf_fix' into blob_integration 2022-01-26 16:49:08 -06:00
Josh Slocum 992fc8ed7c Using non-expanded clears in change feeds 2022-01-26 15:28:58 -06:00
Josh Slocum 311a7656c0 Fixing change feed getting stuck not quite caught up, which causes empty version problems 2022-01-26 12:22:20 -06:00
Josh Slocum 826f956990 Fixed manager recovery order of in progress splits 2022-01-26 08:38:24 -06:00
Josh Slocum c2d334d843 Fixing more bugs in new workload 2022-01-25 19:36:51 -06:00
Josh Slocum 005421dfc8 Fixing final check bug in new workload 2022-01-25 16:53:44 -06:00
Josh Slocum 14cc0a8b02 Got BlobGranuleCorrectnessWorkload passing a single test 2022-01-25 15:46:29 -06:00
Josh Slocum 672b7ab89d Added new test for blob granules and had more consistent naming 2022-01-24 15:15:27 -06:00
Josh Slocum 1494f8216b Reworked blob manager recovery to be more efficient and handle overlapping ongoing splits 2022-01-24 14:12:45 -06:00
Josh Slocum 4262241c92 Removed incorrect assert 2022-01-24 12:51:07 -06:00
Josh Slocum 4d7d1f0e8e Rollback tracking fix for blob worker 2022-01-24 12:50:41 -06:00
Josh Slocum 42a36dc756 Fixed Blob Manager recruitment error and Blob Worker monitoring error 2022-01-24 09:46:37 -06:00
Josh Slocum 1180eb6e44 Fixed uncaught error in blob worker requests 2022-01-21 18:02:30 -06:00
Josh Slocum 9ccffe911b Added exponential backoff for re-establishing blob worker monitoring 2022-01-21 14:21:00 -06:00
Evan Tschannen 966d9d9c15
Merge pull request #6273 from sfc-gh-etschannen/blob_integration
fix: do not report errors when the blob verifier is actor_cancelled
2022-01-21 12:06:34 -08:00
Evan Tschannen fee9e932e7 fix: do not report errors when the blob verifier is actor_cancelled 2022-01-21 12:06:00 -08:00
Josh Slocum 7f9bce40a7 retrying GRVs in BlobGranuleVerifier 2022-01-21 13:36:56 -06:00
Josh Slocum 558779d782 Fix open granule races 2022-01-21 13:20:15 -06:00
Josh Slocum 269dfb79cf Fixes to checkBlobWorkerList 2022-01-21 10:29:54 -06:00
Josh Slocum be98fc9a58 Fixed incorrect assert in blob manager 2022-01-21 08:28:36 -06:00
Josh Slocum a8f1d6fa00 StorageMetrics and and halting fixes 2022-01-20 19:27:25 -06:00
Josh Slocum a7a6e0ee9c More blob worker failure fixes 2022-01-20 19:27:25 -06:00
Josh Slocum 951b28498b Fixed a couple issues with manager recovery and granule locks 2022-01-20 19:27:25 -06:00