* Cleaned up BlobGranule TODO + FIXMEs and addressed some
* popping feed at correct version
* blob worker taking over a granule will pop from where previous worker left off
* addressed fixme of blob worker not re-snapshotting from old change feed
* formatting
* more change feed popped fixes after pop updates
* Getting rid of change feed parallelism lock since it can cause deadlocks in fetching, and relying on full fetch lock
* New blob worker metric and fixing old one
* server-side popped checking still doesn't work because of pops at non-mutation versions
* format
We want to add metrics for the blob worker to evaluate its
performance more concretely. We decided to track the following
information:
- s3 put requests
- s3 get requests
- S3 delete requests
- Delta files written
- Snapshot files written
- Delta bytes written
- Snapshot bytes written
- Number of current ranges assigned
- Bytes read from FDB (initial snapshot)
- Bytes read from S3 (compaction)
- Read requests count
- Read files returned
- Read deltas returned
- Read delta bytes returned
- Ranges assigned
- Ranges revoked
- Number of current ranges assigned
- Total mutation bytes buffered across all ranges // current or accumulated
- Range feed bytes input
- Range feed mutations input
- Wrong Shard Server count