85dff214a4
Address comments.
2021-11-04 11:42:28 -07:00
f15ceb5489
Add Hostname struct, and fromHostname in NetworkAddress struct.
2021-11-04 11:42:28 -07:00
586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB"
2021-11-04 08:46:56 -07:00
6c98e35893
Rename Hop to FlatMap
2021-11-03 13:32:01 -07:00
0853661d13
Introduce getRangeAndHop to push computations down to FDB
2021-11-03 13:21:16 -07:00
e3839e198a
Cleanup
2021-11-03 13:41:41 -05:00
a5b2f464e3
Refactored granule materialization code to be called by c client as well as internal granule management
2021-11-03 13:19:07 -05:00
276dbd4d37
MVC2.0: Refactoring - remove dependency of MVC on Native API
2021-11-03 16:30:18 +01:00
5b2617a524
Added local granule file reading to mako
2021-11-03 09:33:30 -05:00
382882f1c1
mako successfully calls read_blob_granules and gets stuff back
2021-11-02 13:43:42 -05:00
d6a31078fe
C API for blob granules
2021-11-02 10:01:23 -05:00
c7b28abaf0
Enable unknown-pragmas warning for clang
2021-11-01 14:18:31 -07:00
d0c9cf4fb0
Enable mismatched-tags clang warning
2021-11-01 14:18:31 -07:00
ebcc023b6f
Enable missing-field-initializers clang warning
2021-11-01 14:18:31 -07:00
25257f6f87
Enable unused-function warning for clang
2021-11-01 14:18:31 -07:00
b0cec29849
Enable unused-local-typedef clang warning
2021-11-01 14:18:31 -07:00
13bb7838aa
Enable clang -Wformat warning
2021-10-30 21:07:38 -07:00
1e7338b6c3
Merge branch 'master' into bit-flipping-workload
2021-10-28 14:24:49 -07:00
875824b186
MVC2.0: Notify clients about relevant changes of client libraries
2021-10-27 23:43:40 +02:00
4f0991eb67
MVC2.0: Introducing client library status values for instructing clients to download and activate a library; Operations to read and change client library status
2021-10-27 23:43:40 +02:00
6174229a1b
Merge pull request #5694 from sfc-gh-vgasiunas/multi-version-client-2
...
Operations to upload and manage client binaries in the system keyspace
2021-10-27 14:28:10 -07:00
f503ba6a7c
Move getConnectionString() to IClusterConnectionRecord.
...
ClusterConnectionString was moved to IClusterConnectionRecord in PR #5853 .
2021-10-27 12:59:52 -07:00
c8794ae993
MVC2.0: Adding a comment explaining buffer alignment in download & upload operations; checking additional details in testExpectedError
2021-10-27 19:40:22 +02:00
1a2a838df3
add knob
2021-10-27 09:08:37 -07:00
69190ed04e
format
2021-10-27 09:08:37 -07:00
0053b4793e
change knob and delete redundant doBuildTeam
2021-10-27 09:08:37 -07:00
40da5a80f9
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 19:29:10 +02:00
2208b04174
Merge pull request #5855 from sfc-gh-etschannen/blob_full_clean
...
Blob Granules V0
2021-10-26 09:57:35 -07:00
37bc41abbb
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-26 18:51:43 +02:00
298b5192d4
MVC2.0: Fix in getPlatformByName
2021-10-26 18:51:23 +02:00
cd52b2022e
Fix formatting
2021-10-26 08:45:59 -05:00
0e094ef727
Move ClusterConnectionString to IClusterConnectionRecord.
2021-10-25 19:39:39 -07:00
c96f560cbe
Verify rollback of a single version in simulation, other small fixes
2021-10-25 12:03:22 -07:00
92998fd20b
Merge rollback message into rollforward message
2021-10-25 12:03:22 -07:00
7357d7714c
Retry with well known endpoints, move last committed check to consumer
2021-10-25 12:03:22 -07:00
48dc91dd7f
Add rollback and rollforward logic to ConfigBroadcaster
2021-10-25 12:03:22 -07:00
0ff8ddc2b6
Merge branch 'master' into blob_full_clean
2021-10-25 13:38:48 -05:00
7273d352a2
Add extra debug traces and ignore "IMMEDIATE" priority txns for rk throttling
2021-10-25 13:44:45 -04:00
6f7558b8ea
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# tests/CMakeLists.txt
2021-10-24 21:06:33 -07:00
0e327d3d0a
fix: do not duplicate lastEpochEnd from different servers
2021-10-24 19:17:11 -07:00
4039dbd8da
fix: combine mutations from the same version into a single MutationsAndVesionRef
2021-10-24 13:46:45 -07:00
020b02ea4c
Add a comment about the limitations of the URI-based encoding.
2021-10-22 11:05:18 -07:00
4f64f98746
Update the toString method to return URIs. Persisting a cluster connection key makes sure that its not overwriting other changes.
2021-10-22 11:05:18 -07:00
9358adcf49
Address some review comments.
2021-10-22 11:05:18 -07:00
e882eb33fc
Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem.
2021-10-22 11:05:18 -07:00
6d74239760
Track throttling by measuring time spent left in queue on the proxy
2021-10-22 13:55:01 -04:00
d153519188
Merge pull request #5813 from sfc-gh-jslocum/ss_ebrake_streaming_fix
...
Fixes to ss e-brake, tlog streaming, and their interaction
2021-10-22 10:46:17 -07:00
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
dff95dc143
MVC2.0: Fix whitespace alignment
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-10-22 09:57:56 +02:00
d778cea9b5
fix: prevent infinite loop
2021-10-21 15:16:22 -07:00
39b2cb8125
Merge remote-tracking branch 'apple/master' into multi-version-client-2
2021-10-21 17:45:32 +02:00
e4ade0831a
MVC2.0: Addressing code review comments for client lib management operations
...
- Delete clientlib metadata and all chunks in one transaction
- Read all chunks of a download transaction in concurrently
- Configure number of chunks per transaction instead of transaction size
- Make all transactions lock-aware
2021-10-21 11:25:50 +02:00
f1158371a7
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# flow/error_definitions.h
2021-10-21 00:55:12 -07:00
4e79296a9f
fixed a few bugs with fetching change feeds
2021-10-21 00:44:51 -07:00
3ebabb6edc
fixed incorrect use of change feed errors
2021-10-20 22:37:31 -07:00
0cf829ef91
Reduce restore error message
2021-10-20 14:02:48 -07:00
f1c8d3fbc8
Add code to disable cache when ratekeeper begins throttling
2021-10-20 15:52:43 -04:00
8dd7f8f447
Fixes to ss e-brake, tlog streaming, and their interaction
2021-10-20 10:48:29 -05:00
ec307b3f2c
MVC2.0: Addressing code review comments for client lib management operations
2021-10-20 17:19:12 +02:00
3f7df58a77
fixed a number of issues
2021-10-19 13:56:52 -07:00
abab45760d
Add some additional logging if the network thread finishes, fails with an error, gets stopped, or is blocked.
2021-10-19 10:05:29 -07:00
85f64bf42c
more cleanup before merge
2021-10-18 17:11:14 -05:00
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
316a60b8cf
Cleanup comments for BlobWorkerInterface.
2021-10-18 17:45:51 -04:00
ff0d687704
Cleanup comments for server knobs.
2021-10-18 17:45:30 -04:00
8d0594a69d
Cleanup TODOs for backup, blob code refactor.
2021-10-18 17:37:19 -04:00
0fd1440e53
Remove comment from SystemData
2021-10-18 17:03:52 -04:00
f75c8b25ad
add small random delay before returning a cached read version
2021-10-18 13:59:04 -04:00
e2e852e515
Mitigate transitive includes.
2021-10-18 10:49:25 -04:00
507a09893c
Add ClientCount to ClusterControllerMetrics ( #5748 )
2021-10-17 20:47:11 -07:00
5688672b50
Merge pull request #5747 from liquid-helium/cc-interface-changes
...
Added new endpoints in Cluster Controller interface.
2021-10-14 17:03:36 -07:00
6184c12bda
fixing file identifier
2021-10-14 11:25:16 -05:00
b5074fd597
Reworked all of the system data to encode granule data more efficiently for persistence
2021-10-13 16:28:04 -05:00
f3c44c568f
fixing merge conflicts
2021-10-13 16:26:44 -05:00
5f0ec0612a
Merge branch 'feature-range-feed' into blob_full
2021-10-13 15:44:35 -05:00
dfb9655c57
Handle blob work failure
2021-10-13 14:40:26 -04:00
2ec8781224
Merge knobs into one.
2021-10-13 14:00:37 -04:00
bde90f058b
Gate the client API.
2021-10-13 13:26:27 -04:00
a163619fbc
Change default val for knob.
2021-10-13 09:58:09 -04:00
5a6a052c55
Add a knob to gate blob-related work.
2021-10-13 09:48:02 -04:00
ddd08a99d7
fmt
2021-10-12 16:27:51 -07:00
422d8a041d
Added comments.
2021-10-12 15:43:18 -07:00
e543570343
Add locality cache size to TransactionMetrics. ( #5758 )
...
Co-authored-by: Yao Xiao <yaoxiao@Yaos-MacBook-Pro.local>
2021-10-12 13:10:15 -07:00
282f84c807
added handling for broken promise
2021-10-12 10:36:10 -07:00
f913dfed97
Merge branch 'master' into bit-flipping-workload
2021-10-11 16:34:57 -07:00
5b6f8b2abb
added the known committed version to change feeds
2021-10-11 13:53:36 -07:00
5066d67dec
Merge pull request #5715 from sfc-gh-ljoswiak/fixes/tracing-various
...
Add knob to control distributed trace recording percentage
2021-10-11 14:03:53 -06:00
cd6049f178
Merge pull request #5708 from sfc-gh-ljoswiak/fixes/sampling-crashes
...
Fix some common sampling crashes at high sampling frequencies
2021-10-11 14:02:11 -06:00
2ebaddcc1e
Bug fix: CommitQuorum::addRequestActor() accesses self after destruction due to ignoring actor_cancelled error. ( #5744 )
2021-10-11 12:17:09 -07:00
2e03dd02cd
Added new endpoints in Cluster Controller interface.
...
moveShard is for manual data move;
repairSystemData is for repairing system metadata when some or all of
storage servers hosting system metadata are lost.
The complete implementation can be fund in #5718 .
2021-10-11 10:41:46 -07:00
effeb1dcd7
fix: must serialize the sequence
2021-10-09 21:18:13 -07:00
d51edf18dc
fixed merge conflicts
2021-10-09 19:47:24 -07:00
5c642f706e
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
efc4cec53f
fixed a variety of bugs with change feeds
2021-10-09 19:24:01 -07:00
4fbc193e19
fix priority
2021-10-09 16:40:24 -06:00
9077e15b11
Address review comments
2021-10-09 16:40:24 -06:00
4e723ba3d9
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-10-09 16:40:24 -06:00
ffdba4a133
assert we never get a stale read version
2021-10-09 16:40:24 -06:00
2b2d059039
Specify option to specifically skip grv cache for the background updater when using debug knob
2021-10-08 17:32:44 -04:00
b770936cde
move background updater actor initialization
2021-10-08 16:38:21 -04:00
ecdafba336
add debug traces to background grv updater
2021-10-08 15:17:28 -04:00
efd75ccc74
Add histograms to client GRV batcher. ( #5727 )
2021-10-07 13:30:25 -07:00
c07a07dbbe
Take uptime into account when making failover decision
2021-10-07 11:19:34 -07:00
62197faa46
Add more comments to the code
2021-10-07 11:19:34 -07:00
c0fbe5471f
Implement the core logic of grey failure triggered failover
2021-10-07 11:19:34 -07:00
61fb967484
MVC2.0: Add a clientlib metadata attribute for checksum algorithm
2021-10-07 18:06:22 +02:00
bd838217ba
MVC2.0: Testing client lib operations with random file sizes
...
- Adding test parameters to control file size range
- Disabling AIO as it does not support non-page-aligned reads and writes
- Fixing bugs for the cases of an empty file and an incomplete last chunk
- Use hexadecimal representation for checksum in JSON and document keys
2021-10-07 17:23:17 +02:00
31664ddb78
Fixing change feed error retry in pop
2021-10-07 10:07:18 -05:00
ed72067b24
MVC2.0: Refactoring - declare state variables at the beginning of actors
2021-10-07 13:34:46 +02:00
6300447e9d
MVC2.0: Use member initializers in ClientLibManagement structs
2021-10-07 11:38:52 +02:00
2a98b38a56
MVC2.0: Use anonymous namespaces instead of static declarations in ClientLibManagement
2021-10-07 11:24:30 +02:00
a150018dc6
MVC2.0: Refactoring - hide global variables for client lib status and platform names
2021-10-07 11:08:48 +02:00
114d8438fa
MVC2.0: Refactoring client lib management
...
- Move all declarations into ClientLibManagement namespace
- Rename source files for more consistent naming
- Use constant declarations instead of defines for client lib attribute names
2021-10-07 10:30:37 +02:00
54d5e8366d
fixing another change feed hang error
2021-10-06 16:04:01 -05:00
cc38c48123
Add debug knob to use grv cache for performance testing
2021-10-06 14:35:42 -04:00
bbe7c7ca9f
found change feed liveness bug
2021-10-06 12:32:16 -05:00
436ed7e497
MVC2.0: Check byte sum on client lib uploads and downloads, rollback upload in case of an error
2021-10-06 18:01:46 +02:00
39017c3b41
MVC2.0: Use page-aligned buffers and offsets, enable async IO for reading & writing client lib files
2021-10-06 18:01:46 +02:00
5360abb238
MVC2.0: Operation to list uploaded libraries with various filters;
...
Introducing constants for attribute names and platform values
2021-10-06 18:01:46 +02:00
abcef299e1
MVC2.0: Operation to delete a client library; enum for client lib status
2021-10-06 18:01:46 +02:00
816e8703d6
MVC2.0: Operation to download a client library from the system keyspace to a file
2021-10-06 18:01:46 +02:00
6ad6874f73
MVC2.0: Test client lib upload error paths
2021-10-06 18:01:46 +02:00
6c103c72d9
MVC2.0: Use ClientKnobs to configure chunk and transaction size for client lib upload
2021-10-06 18:01:46 +02:00
cda0a5f931
Operation to upload client library binary in to system keyspace
2021-10-06 18:01:46 +02:00
9a18cc8f41
Change debug time bound update logic to be located in masterserver
2021-10-05 17:31:31 -04:00
44a854772f
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-10-05 12:55:02 -04:00
ae31f01dc4
remove cache update from extractreadversion and set lower time bound to 0
2021-10-05 12:54:22 -04:00
95166796cd
Address PR comments.
2021-10-04 20:16:22 -04:00
d560eb1fea
debug time bounds using sim_validation
2021-10-04 14:12:31 -04:00
12870ef356
Serialize all streams.
2021-10-04 11:07:08 -04:00
4d54669ccd
Recruit the blob workers via blob manager.
...
In this PR, the blob manager now recruits blob workers
(via communication with the cluster controller). Blob workers
are onboarded as blob worker processes enter the cluster.
2021-10-04 11:07:08 -04:00
04682c9091
Merge pull request #14 from sfc-gh-sgupta/blob-worker-process-class
...
Add exclusive process class for blob worker.
2021-10-04 10:03:02 -05:00
e034f66fc2
Add knob to control trace recording percentage
2021-10-01 16:30:46 -07:00
4cd642be7a
Fix some common sampling crashes at high sampling frequencies
2021-09-30 17:25:09 -07:00
ee6d9b33e9
Merge pull request #5691 from sfc-gh-satherton/redwood-reporting-changes
...
Redwood now reports space used to store metadata about free space as free space
2021-09-30 11:06:45 -07:00
f27c6ef57c
addressing review comments.
2021-09-30 10:00:33 -07:00
3ea7209013
Simulation changes to randomly wiggle with locality filter and review comments.
2021-09-30 10:00:33 -07:00
3e79299898
Locality filter support to perpetual storage wiggler feature.
2021-09-30 10:00:33 -07:00
d7a420a8d1
Added Redwood pager and BTree overview statistics to trace events logged on creation and recovery. Changed getStorageBytes() to treat internal space used by free list queues as reusable.
2021-09-29 21:01:21 -07:00
a3d53c57b1
Fix a thread-safety issue with timeoutTsav
2021-09-29 11:07:47 -07:00
ce785281cc
fix: use DEFAULT_COMMIT_GRV_PROXIES_RATIO to pick the number of grv_proxies and commit_proxies after an upgrade
2021-09-28 09:36:25 -07:00
cb154c12d4
disable ENABLE_DETAILED_TLOG_POP_TRACE by default
2021-09-26 21:46:57 -07:00
fa1fe5f08b
added blob worker rollbacks that handle (most) cases
2021-09-24 17:52:36 -05:00
c2885ab70d
The BACKUP_LOCK_BYTES knob could be buggified to a value that was too low, resulting in backup getting stuck.
2021-09-24 09:15:30 -07:00
a986679b7f
More bug fixes
2021-09-24 10:02:02 -05:00
a4bcd3919d
Add exclusive process class for Blob Worker.
...
Also introduces a specific machine in the simulated cluster
to test blob worker (similar to what's done for storage cache).
2021-09-23 16:54:44 -04:00
c99ce19ad7
Merge pull request #5524 from liquid-helium/improve-exlude-failed-servers
...
Check if the src server list will be empty before removing a failed SS
2021-09-23 10:22:57 -07:00
32a834b0c2
The computation of tag throttling backoff was taking a minimum with a knob for how long a throttle lasts. When this knob was buggified to 0 in simulation, the backoff would then be 0, and this could cause some transactions to get stuck in a retry loop that failed to advance time. Eventually the test would timeout or potentially run out of memory.
2021-09-23 10:03:57 -07:00
5fa6c687d6
Add blob manager as a singleton.
2021-09-23 10:45:37 -04:00
72edcd8d73
Address PR comments.
...
Revert knob name change, fix comparison between new and old
recruitments, and get rid of empty `if` block.
2021-09-22 16:56:34 -05:00
0b6fecddbc
Refactor logic for recruiting singletons.
...
This commit refactors the logic for recruiting singletons,
which is done by the ClusterController. This allows for far
easier additions of new singletons in the future, and also
cleans up the code.
Also, the logic for recruiting DD was changed to mirror
the logic for recruiting RK. Although the logic for RK
allows there to be many RKs existing at once, the moveKeysLock
mechanism used by DD still prevents multiple DDs existing at once.
2021-09-22 16:56:18 -05:00
5ddf08dfe5
Got basic range reassignment working
2021-09-22 16:48:44 -05:00
a1f6dcc2d5
merge from master
2021-09-22 13:07:38 -07:00
4cd5d138ff
Fixed segfault involving missing futures and infinite looping in background actor
2021-09-22 13:13:35 -04:00
1730d75f73
change configure test
...
add store type check
add test file
2021-09-21 18:11:04 -07:00
1d36c18a68
solve cmake configure error when -DOPEN_FOR_IDE=ON
2021-09-21 12:43:22 -07:00
65956ae6b7
Refactor configure command; refactor changeConfig to template code to reuse existing tests
2021-09-21 10:06:04 -07:00
16171d8252
Refactoring well-known endpoint registration
...
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
47ccc75270
added a sequence number to streaming replies as a safety backstop against out of order delivery
2021-09-20 13:58:05 -07:00
f9a871218f
fixing db config parenthesis
2021-09-20 14:36:37 -05:00
6533678f0d
Address PR comments.
...
Revert knob name change, fix comparison between new and old
recruitments, and get rid of empty `if` block.
2021-09-20 14:26:42 -05:00
fe098b3b11
Refactor logic for recruiting singletons.
...
This commit refactors the logic for recruiting singletons,
which is done by the ClusterController. This allows for far
easier additions of new singletons in the future, and also
cleans up the code.
Also, the logic for recruiting DD was changed to mirror
the logic for recruiting RK. Although the logic for RK
allows there to be many RKs existing at once, the moveKeysLock
mechanism used by DD still prevents multiple DDs existing at once.
2021-09-20 14:26:42 -05:00
914b1ee960
fmt
2021-09-20 12:01:47 -07:00
95f64f97de
resolved some fmt issues
2021-09-20 11:58:30 -07:00
110d6f2d24
Enable support for integers in environment variable network options
...
Description
Description
Previously, the environment setting via shell cannot be extracted correctly, due to the code is't ablle to
determine input parameter type.
Now the logic to determien parameter and give it different processing is added.
Resolves #4127
Testing
1. Pass an invalid env var:
FDB_NETWORK_OPTION_TRACE_ROLL_SIZE=xyz - error
FDB_NETWORK_OPTION_TRACE_ROLL_SIZE=123xyz) - error
2. Pass an out-of-ranged env var:
FDB_NETWORK_OPTION_TRACE_ROLL_SIZE=9999999999999999999999999999 -error
FDB_NETWORK_OPTION_TRACE_ROLL_SIZE=-9999999999999999999999999999 -error
3. Pass a valid env var
FDB_NETWORK_OPTION_TRACE_ROLL_SIZE=1000 -ok
4. Pass a float env var(not support)
FDB_NETWORK_OPTION_TRACE_ROLL_SIZE=3.1415926535 - error
2021-09-20 11:48:07 -07:00
3ece92b93d
Enable support for integers in environment variable network options
2021-09-20 11:48:07 -07:00
1e16ab446f
Enable support for integers in environment variable network options
2021-09-20 11:48:07 -07:00
c36758c08a
Enable support for integers in environment variable network options
2021-09-20 11:48:07 -07:00
72e97775be
Enable support for integers in environment variable network options
2021-09-20 11:48:07 -07:00
d78e53893d
Update for the env variable type check function
...
Description
Testing
2021-09-20 11:48:07 -07:00
19cfc12378
Enable multiple environment type input
...
Description
Previously, the environment setting via shell cannot be extracted correctly, due to the code is't ablle to
determine input parameter type.
Now the logic to determien parameter and give it different processing is added.
Testing
2021-09-20 11:48:07 -07:00
d51b93390d
Proper handling of reset and transaction destruction. Add some comments.
2021-09-17 18:10:56 -06:00
b5fbd07155
Transactions would not honor the transaction timeout option if the MVC did not have an active database.
2021-09-17 18:10:56 -06:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
96fcde45c2
Minor leader election code improvements.
...
1. Rename monitorLeaderRemotely* functions to monitorLeaderWithDelayedCandidacy*. "Remote" is not clearly describing what the functions are doing;
2. Rename monitorLeaderForProxies() to monitorLeaderAndGetClientInfo() to better describe the function;
3. Remove monitorLeaderRemotelyInternal() and monitorLeaderRemotely() in MonitorLeader.actor.cpp, to eliminate code duplication. They already exist in worker.actor.cpp;
4. Move the declaration of getLeader() from LeaderElection.actor.cpp to MonitorLeader.h;
5. Update a few comments.
2021-09-16 15:34:45 -07:00
c9b8bdffbe
Passes a single correctness test!
2021-09-15 17:18:04 -05:00
05e3190d8c
Merge branch 'master' of https://github.com/apple/foundationdb into improve-exlude-failed-servers
2021-09-15 11:27:48 -07:00
6673540553
TSS Metrics fix
2021-09-14 13:15:05 -05:00
e7d3516084
solve comments
2021-09-14 11:04:02 -07:00
1b3bae899b
update comments
2021-09-14 11:04:02 -07:00
5c5a5d249e
Update auto_coordinators special key error message
2021-09-14 11:04:02 -07:00
9992a7b33f
Added StorageMigrationType and cli commands
2021-09-14 09:55:41 -05:00
cb9dde358c
Solve comments, refatctor the error handling part
2021-09-13 13:28:48 -07:00
e81748e732
Fix all tr->get, hold the ThreadFuture for the memory
2021-09-13 13:28:48 -07:00
87fac6e301
adjust to changed function signature after merge
2021-09-13 13:04:55 -04:00
d971b63bbf
Merge branch 'master' of github.com:apple/foundationdb into jfu-grv-cache
2021-09-13 12:47:42 -04:00
cfbe5eca9c
Use notifieddouble to track GRV request time and include grvdelay in timing calculations
2021-09-13 11:57:19 -04:00
cf1626f553
Added comments.
2021-09-10 11:10:10 -07:00
cefb66d64c
fixing a couple bugs
2021-09-10 12:52:33 -05:00
078cc7ee55
Merge branch 'master' into bit-flipping-workload
2021-09-10 10:26:21 -07:00
e2a51a4fe7
Fixing up after change feed updates
2021-09-10 11:49:41 -05:00
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
ac5b580e2d
Merge branch 'master' into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/StorageServerInterface.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/ApplyMetadataMutation.cpp
# fdbserver/TLogServer.actor.cpp
# flow/error_definitions.h
2021-09-09 23:13:22 -07:00
2ea055371b
Introduced serverKeysTrueEmptyRange to indicate the SS that the new
...
shard is empty.
2021-09-09 14:15:07 -07:00
1ce2457f7e
initial commit
2021-09-09 14:01:42 -04:00
30e7469b19
Update fdbclient/MultiVersionTransaction.actor.cpp
2021-09-08 17:37:33 -07:00
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
84a2672a33
Merge branch 'master' of github.com:apple/foundationdb into refactor-fdbcli
2021-09-08 10:42:59 -07:00
af5643c998
Fix the issue in CheckStatus where it stuck as we cannot talk to any of the coordinators
2021-09-08 10:05:13 -07:00
d9782bc29d
fixed a few bugs with merging change feeds on the client
2021-09-07 21:35:00 -07:00
f53ddf6720
fix: handle the case where I change feed is added while a fetch is in progress
2021-09-06 13:51:53 -07:00
af0792b1f5
fixes that allow change feeds to pass correctness in tests with no failures
2021-09-03 17:24:33 -07:00
b02e8d99c6
added a workload to test change feeds
2021-09-03 14:19:27 -07:00
2aa0a5cf58
The lock special key has the lock UID as the value
2021-09-03 12:15:56 -07:00
8309c2ce26
filter mutations in change feeds to the requested range
2021-09-03 12:06:13 -07:00
66166839fd
Add BlobWorkerCommon to CMakeLists.
2021-09-03 15:05:35 -04:00
5aebd77f0a
PR changes
2021-09-03 13:58:39 -04:00
afd8df4bb9
Merge commit '7f2b6c2c7ec2efcd7339e3c6531e4906322477c3' into network-thread-busyness-burstiness-score
2021-09-03 09:48:11 -07:00
a0b3446572
Add metrics for blob worker.
...
We want to add metrics for the blob worker to evaluate its
performance more concretely. We decided to track the following
information:
- s3 put requests
- s3 get requests
- S3 delete requests
- Delta files written
- Snapshot files written
- Delta bytes written
- Snapshot bytes written
- Number of current ranges assigned
- Bytes read from FDB (initial snapshot)
- Bytes read from S3 (compaction)
- Read requests count
- Read files returned
- Read deltas returned
- Read delta bytes returned
- Ranges assigned
- Ranges revoked
- Number of current ranges assigned
- Total mutation bytes buffered across all ranges // current or accumulated
- Range feed bytes input
- Range feed mutations input
- Wrong Shard Server count
2021-09-03 12:09:01 -04:00
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
27151b0831
support reads from range feeds that span multiple storage servers
2021-09-01 14:35:51 -07:00
3b34423248
Merge branch 'master' into bit-flipping-workload
2021-08-31 12:14:51 -07:00
46adada5ff
Cleaned up debugging and fixed a couple bugs
2021-08-31 12:30:43 -05:00
82419d086f
Merge pull request #5484 from sfc-gh-ajbeamon/network-thread-busyness-include-primary
...
When using an external client, report the larger of the primary and external network thread busyness
2021-08-31 09:39:46 -07:00
2fa97f60fe
Update fdbclient/NativeAPI.actor.cpp
...
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2021-08-31 08:53:37 -07:00
9a6654ef23
Merge pull request #5488 from sfc-gh-clin/refactor-profile-command
...
Refactor profile/triggerddteaminfolog/tssq commands
2021-08-30 11:25:22 -07:00
b4bfd58bcb
multiple blob workers appears to work
2021-08-30 13:07:25 -05:00
86c6c2b643
Merge remote-tracking branch 'origin/master' into paxos-config-db
2021-08-27 23:24:14 -07:00
69508b980f
modify comments to make clang-format and coverage tool play nice
2021-08-27 17:18:00 -07:00
2c788c233d
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-08-27 17:07:47 -07:00
2d6fb93316
Compute a burstiness score based on the max busyness interval in a measurement period.
2021-08-27 16:29:15 -07:00
3a880b43d4
Fix and reenable PaxosConfigTransaction::onError
2021-08-27 15:06:33 -07:00
3b011408f8
Added sequence numbers and locks to blob worker and manager
2021-08-27 16:33:07 -05:00
991d61923f
Refactor tssq command
2021-08-27 11:58:18 -07:00
def5822d42
When using an external client, use the larger of the primary and external busyness.
2021-08-27 10:46:34 -07:00
29d83291a1
Add CommitUnknownResult metric for ConfigIncrement workload
2021-08-27 00:44:12 -07:00
7f8a1385d5
Merge remote-tracking branch 'origin/master' into paxos-config-db
2021-08-26 17:00:43 -07:00
d541b3804d
Make SimpleConfigTransaction more consistent with PaxosConfigTransaction
2021-08-26 16:18:24 -07:00
28daab9f5c
Use ActorCollection instead of std::vector<Future<Void>> in *Quorum classes, to listen for errors
2021-08-26 16:18:19 -07:00
be440ab954
Merge pull request #5260 from FuhengZhao/RedwoodHistogram
...
Redwood local histograms
2021-08-26 12:05:44 -07:00
8d49c98a41
Added simulation workload for blob granules and fixed some bugs
2021-08-26 13:48:05 -05:00
96726a275a
Create separate copies of commit request in PaxosConfigTransactionImpl
2021-08-25 15:21:36 -07:00
714aca4f3c
Refactored blob granule reads to actually be in NativeAPI
2021-08-25 15:01:38 -05:00
e328e2b5df
format fix
2021-08-24 18:34:44 -07:00
5eb92ae65c
reomve registry
2021-08-24 15:47:04 -07:00
868fa93d3d
Merge branch 'master' of github.com:apple/foundationdb into refactor-cache_range-command
2021-08-24 12:58:23 -07:00
5259af787d
Switched blob implementation to use backup container
2021-08-24 13:47:47 -05:00
a047782db2
Merge remote-tracking branch 'origin/master' into paxos-config-db
2021-08-24 11:08:20 -07:00
57652a0cae
Format
2021-08-24 10:00:24 -07:00
b65a66fab7
log redwood histogram seperatly
2021-08-24 09:57:39 -07:00
fb2eef38fc
Changing API and file format to full V1 specification
2021-08-24 10:05:46 -05:00
c1bb977a93
Set sampling dependencies for Windows only
2021-08-24 10:48:31 +02:00
2c5510011c
Add dependencies between sampling and non-sampling actors projects
2021-08-24 10:48:31 +02:00
5689592ac5
Handle not_committed error in SimpleConfigTransaction::onError
2021-08-23 13:11:30 -07:00
78b57629a1
Handle not_committed error in PaxosConfigTransaction::onError
2021-08-23 13:09:49 -07:00
2ae447eaaa
Refactored blob worker/manager to be in separate files
2021-08-23 14:16:09 -05:00
d554205db3
Move common knob initialization macros to header
2021-08-23 11:41:01 -07:00
8c9f99d402
Remove unnecessary function
2021-08-23 11:41:01 -07:00
338fc59673
Separate test knobs into atomic and nonatomic
2021-08-23 11:41:00 -07:00
4107c35538
Add atomic knob unit test
2021-08-23 11:41:00 -07:00
Renxuan Wang