cfb8368da6
Address PR comments.
2021-10-13 14:56:17 -04:00
d002df3b21
Clean up blob worker changes.
2021-10-13 14:40:26 -04:00
266a5b06fa
Fix infinite loop.
2021-10-13 14:40:26 -04:00
423a67f448
trying to fix infinite loop
2021-10-13 14:40:26 -04:00
dfb9655c57
Handle blob work failure
2021-10-13 14:40:26 -04:00
4e4a2534da
review comments and moving priority yield to correct spot
2021-10-06 09:24:48 -05:00
6a24ef9258
adding priorities to blob worker and fixing monitoring in blob manager
2021-10-05 16:51:19 -05:00
282f9d35cd
Cleanup comments and debugging code.
2021-10-04 11:07:08 -04:00
4d54669ccd
Recruit the blob workers via blob manager.
...
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
8fb7c45e65
even more bug fixes
2021-09-25 10:30:27 -05:00
3b23d6aba5
rollback fixes
2021-09-25 09:19:00 -05:00
fa1fe5f08b
added blob worker rollbacks that handle (most) cases
2021-09-24 17:52:36 -05:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
8e4582673c
more bug fixes
2021-09-20 10:21:52 -05:00
9a92b763ae
Fixing issues in known committed version tracking
2021-09-17 12:26:10 -05:00
c9b8bdffbe
Passes a single correctness test!
2021-09-15 17:18:04 -05:00
cefb66d64c
fixing a couple bugs
2021-09-10 12:52:33 -05:00
e2a51a4fe7
Fixing up after change feed updates
2021-09-10 11:49:41 -05:00
44a64005cf
also pipelining snapshot from blob in blob worker for maximum throughput
2021-09-09 10:43:28 -05:00
99e5ecdf9d
Doing GRVs for committed version checking in blob worker
2021-09-08 17:24:49 -05:00
2491287f98
Pipelined delta file writing to improve performance
2021-09-08 16:57:57 -05:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
5aebd77f0a
PR changes
2021-09-03 13:58:39 -04:00
a0b3446572
Add metrics for blob worker.
...
We want to add metrics for the blob worker to evaluate its
performance more concretely. We decided to track the following
information:
- s3 put requests
- s3 get requests
- S3 delete requests
- Delta files written
- Snapshot files written
- Delta bytes written
- Snapshot bytes written
- Number of current ranges assigned
- Bytes read from FDB (initial snapshot)
- Bytes read from S3 (compaction)
- Read requests count
- Read files returned
- Read deltas returned
- Read delta bytes returned
- Ranges assigned
- Ranges revoked
- Number of current ranges assigned
- Total mutation bytes buffered across all ranges // current or accumulated
- Range feed bytes input
- Range feed mutations input
- Wrong Shard Server count
2021-09-03 12:09:01 -04:00
bdbb0303f3
Add missing wait around deleteFile invocations.
2021-09-03 11:12:03 -04:00
ff96843c58
review comments
2021-09-01 10:12:53 -05:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
3b011408f8
Added sequence numbers and locks to blob worker and manager
2021-08-27 16:33:07 -05:00
8d49c98a41
Added simulation workload for blob granules and fixed some bugs
2021-08-26 13:48:05 -05:00
ad899565e9
Using local files for blob granules in simulation
2021-08-24 14:15:14 -05:00
5259af787d
Switched blob implementation to use backup container
2021-08-24 13:47:47 -05:00
fb2eef38fc
Changing API and file format to full V1 specification
2021-08-24 10:05:46 -05:00
42a781016e
Made subsequent snapshots read from blob files instead of FDB
2021-08-23 16:46:59 -05:00
2ae447eaaa
Refactored blob worker/manager to be in separate files
2021-08-23 14:16:09 -05:00
Suraj Gupta