19076ad4d2
Changed blob manager checkin to by async and to have blob worker speculatively consume change feed ahead of re-snapshot
2021-10-27 16:23:51 -05:00
134aef6011
Fix compiler warning.
2021-10-27 10:40:03 -04:00
fd50a011b1
Address PR comments.
2021-10-26 21:39:41 -04:00
5a9d9921d0
Fixes and final cleanup for BM failure handling
2021-10-26 16:16:00 -04:00
17b30f188a
Working impl
2021-10-26 16:16:00 -04:00
99606482ea
initial thoughts
2021-10-26 16:16:00 -04:00
e57e2bec5f
Improve documentation.
2021-10-25 12:19:28 -04:00
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
e2e852e515
Mitigate transitive includes.
2021-10-18 10:49:25 -04:00
a9f23773ad
Default all debug flags to false.
2021-10-18 10:10:05 -04:00
90ce8bbe5b
Refactor retry loop to splitRange.
2021-10-18 09:56:47 -04:00
d1fe9d4c50
Remove old comments from BM.
2021-10-14 19:25:34 -04:00
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
f3c44c568f
fixing merge conflicts
2021-10-13 16:26:44 -05:00
cfb8368da6
Address PR comments.
2021-10-13 14:56:17 -04:00
ef67feed67
Clean up blob manager changes.
2021-10-13 14:40:26 -04:00
266a5b06fa
Fix infinite loop.
2021-10-13 14:40:26 -04:00
423a67f448
trying to fix infinite loop
2021-10-13 14:40:26 -04:00
dfb9655c57
Handle blob work failure
2021-10-13 14:40:26 -04:00
4e4a2534da
review comments and moving priority yield to correct spot
2021-10-06 09:24:48 -05:00
6a24ef9258
adding priorities to blob worker and fixing monitoring in blob manager
2021-10-05 16:51:19 -05:00
282f9d35cd
Cleanup comments and debugging code.
2021-10-04 11:07:08 -04:00
4d54669ccd
Recruit the blob workers via blob manager.
...
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
8fb7c45e65
even more bug fixes
2021-09-25 10:30:27 -05:00
fa1fe5f08b
added blob worker rollbacks that handle (most) cases
2021-09-24 17:52:36 -05:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
889ae3f255
Address PR comments.
...
Remove nukeBlobWorkerData, simplify checkManagerLock call and make
final choose...when consistent.
2021-09-23 14:09:56 -04:00
d675e6b143
Kill BM by returning, instead of throwing.
2021-09-23 11:41:57 -04:00
8c49f1f238
Improve actor management and handling of failure in BM.
2021-09-23 10:58:38 -04:00
5fa6c687d6
Add blob manager as a singleton.
2021-09-23 10:45:37 -04:00
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
c9b8bdffbe
Passes a single correctness test!
2021-09-15 17:18:04 -05:00
e2a51a4fe7
Fixing up after change feed updates
2021-09-10 11:49:41 -05:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
fccfe3af78
Address PR comments.
2021-09-02 12:09:37 -04:00
c27d8f3336
Improve assignment of range granules.
...
Previously, we randomly picked a worker to assign a range to.
Now, we pick a worker that has the least number of range granules
already assigned, ultimately distributing the workload in a more
efficient manner.
Future iterations should also consider the number of read/writes
that a worker is already handling when picking a worker to assign
a range to. This could prevent us from assigning a range to a worker
that is already hot.
2021-09-01 23:36:52 -04:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
bbeec49533
added range mover
2021-08-30 13:59:53 -05:00
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
3b011408f8
Added sequence numbers and locks to blob worker and manager
2021-08-27 16:33:07 -05:00
8d49c98a41
Added simulation workload for blob granules and fixed some bugs
2021-08-26 13:48:05 -05:00
ad899565e9
Using local files for blob granules in simulation
2021-08-24 14:15:14 -05:00
5259af787d
Switched blob implementation to use backup container
2021-08-24 13:47:47 -05:00
2ae447eaaa
Refactored blob worker/manager to be in separate files
2021-08-23 14:16:09 -05:00
b3491d7338
Cleaned up code for demo
2021-08-12 14:30:33 -05:00
247602753d
Blob worker handles multiple ranges now
2021-08-10 11:19:43 -05:00
Josh Slocum