Commit Graph

17237 Commits

Author SHA1 Message Date
Josh Slocum 6360fdf5e3
Merge pull request #16 from sfc-gh-sgupta/blob-worker-failure
Blob worker failure handling
2021-10-13 14:12:04 -05:00
Suraj Gupta 180e806086 Add new test to cmakelists 2021-10-13 15:10:52 -04:00
Suraj Gupta cfb8368da6 Address PR comments. 2021-10-13 14:56:17 -04:00
Suraj Gupta d002df3b21 Clean up blob worker changes. 2021-10-13 14:40:26 -04:00
Suraj Gupta ef67feed67 Clean up blob manager changes. 2021-10-13 14:40:26 -04:00
Suraj Gupta 266a5b06fa Fix infinite loop. 2021-10-13 14:40:26 -04:00
Suraj Gupta 423a67f448 trying to fix infinite loop 2021-10-13 14:40:26 -04:00
Suraj Gupta dfb9655c57 Handle blob work failure 2021-10-13 14:40:26 -04:00
Josh Slocum abdf5d434c
Merge pull request #18 from sfc-gh-sgupta/gate-blob-related-work
Add a knob to gate blob-related work.
2021-10-13 13:18:19 -05:00
Suraj Gupta 2ec8781224 Merge knobs into one. 2021-10-13 14:00:37 -04:00
Suraj Gupta bde90f058b Gate the client API. 2021-10-13 13:26:27 -04:00
Suraj Gupta 9d4b55c7fe Gate the blob verifier as well. 2021-10-13 11:45:51 -04:00
Suraj Gupta a163619fbc Change default val for knob. 2021-10-13 09:58:09 -04:00
Suraj Gupta 5a6a052c55 Add a knob to gate blob-related work. 2021-10-13 09:48:02 -04:00
Josh Slocum 31664ddb78 Fixing change feed error retry in pop 2021-10-07 10:07:18 -05:00
Josh Slocum 54d5e8366d fixing another change feed hang error 2021-10-06 16:04:01 -05:00
Josh Slocum bbe7c7ca9f found change feed liveness bug 2021-10-06 12:32:16 -05:00
Josh Slocum 3eb49d38df
Merge pull request #17 from sfc-gh-jslocum/bw_priorities
adding priorities to blob worker and fixing monitoring in blob manager
2021-10-06 09:40:08 -05:00
Josh Slocum 4e4a2534da review comments and moving priority yield to correct spot 2021-10-06 09:24:48 -05:00
Josh Slocum 6a24ef9258 adding priorities to blob worker and fixing monitoring in blob manager 2021-10-05 16:51:19 -05:00
Josh Slocum 0e043bf456
Merge pull request #15 from sfc-gh-sgupta/blob-worker-recruitment
Blob worker recruitment.
2021-10-05 09:03:06 -05:00
Suraj Gupta 95166796cd Address PR comments. 2021-10-04 20:16:22 -04:00
Suraj Gupta 12870ef356 Serialize all streams. 2021-10-04 11:07:08 -04:00
Suraj Gupta 282f9d35cd Cleanup comments and debugging code. 2021-10-04 11:07:08 -04:00
Suraj Gupta 4d54669ccd Recruit the blob workers via blob manager.
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
Josh Slocum 04682c9091
Merge pull request #14 from sfc-gh-sgupta/blob-worker-process-class
Add exclusive process class for blob worker.
2021-10-04 10:03:02 -05:00
Suraj Gupta ac14856307 Add BWs to every DC in simulated cluster. 2021-09-27 11:17:37 -04:00
Josh Slocum 77b0458f9f reducing volume for large workload since there are multiple testers 2021-09-25 11:25:12 -05:00
Josh Slocum 8fb7c45e65 even more bug fixes 2021-09-25 10:30:27 -05:00
Josh Slocum 3b23d6aba5 rollback fixes 2021-09-25 09:19:00 -05:00
Josh Slocum fa1fe5f08b added blob worker rollbacks that handle (most) cases 2021-09-24 17:52:36 -05:00
Suraj Gupta 0ca00ce53b Address PR comments.
Add support for more than 1 blob worker in simulated cluster.
We now have at least 2, and up to 7 (randomly).

Also removed the ability for blob workers to onboard onto
unset processes.
2021-09-24 12:17:56 -04:00
Josh Slocum a986679b7f More bug fixes 2021-09-24 10:02:02 -05:00
Suraj Gupta a4bcd3919d Add exclusive process class for Blob Worker.
Also introduces a specific machine in the simulated cluster
to test blob worker (similar to what's done for storage cache).
2021-09-23 16:54:44 -04:00
Josh Slocum b622906b9f
Merge pull request #13 from sfc-gh-sgupta/blob-manager-recruitment-3
Recruit the blob manager.
2021-09-23 14:56:24 -05:00
Suraj Gupta 889ae3f255 Address PR comments.
Remove nukeBlobWorkerData, simplify checkManagerLock call and make
final choose...when consistent.
2021-09-23 14:09:56 -04:00
Suraj Gupta d675e6b143 Kill BM by returning, instead of throwing. 2021-09-23 11:41:57 -04:00
Suraj Gupta 8c49f1f238 Improve actor management and handling of failure in BM. 2021-09-23 10:58:38 -04:00
Suraj Gupta 5fa6c687d6 Add blob manager as a singleton. 2021-09-23 10:45:37 -04:00
Suraj Gupta 95c004f80b Add missing namespace qualifier to vector. 2021-09-22 16:57:04 -05:00
Suraj Gupta 2b9dfc1371 Simplify count increments. 2021-09-22 16:56:59 -05:00
Suraj Gupta 4530e746d6 Address PR comments.
Adds comment for constant and changes method name for `setonDb`.
2021-09-22 16:56:49 -05:00
Suraj Gupta 4a71f3d0f8 Fix recruitment bug. 2021-09-22 16:56:44 -05:00
Suraj Gupta 72edcd8d73 Address PR comments.
Revert knob name change, fix comparison between new and old
recruitments, and get rid of empty `if` block.
2021-09-22 16:56:34 -05:00
Suraj Gupta 10807ddebc Rename function to be more clear. 2021-09-22 16:56:27 -05:00
Suraj Gupta 0b6fecddbc Refactor logic for recruiting singletons.
This commit refactors the logic for recruiting singletons,
which is done by the ClusterController. This allows for far
easier additions of new singletons in the future, and also
cleans up the code.

Also, the logic for recruiting DD was changed to mirror
the logic for recruiting RK. Although the logic for RK
allows there to be many RKs existing at once, the moveKeysLock
mechanism used by DD still prevents multiple DDs existing at once.
2021-09-22 16:56:18 -05:00
Josh Slocum 5ddf08dfe5 Got basic range reassignment working 2021-09-22 16:48:44 -05:00
Josh Slocum ccb78b1ce5 more CF bug fixes 2021-09-21 13:20:27 -05:00
Josh Slocum 8e4582673c more bug fixes 2021-09-20 10:21:52 -05:00
Josh Slocum 9a92b763ae Fixing issues in known committed version tracking 2021-09-17 12:26:10 -05:00