Josh Slocum
43543b0d0e
Removing incorrect CF validation asserts
2022-02-24 12:39:15 -06:00
Josh Slocum
b6d02478cd
Adding stream uid tracking for ss feed debugging
2022-02-24 12:19:25 -06:00
Josh Slocum
86236b766d
Added print debugging for CF memory vs disk validation
2022-02-24 12:11:06 -06:00
Josh Slocum
b67930cb93
Fixing bug in new change feed fetch logic
2022-02-24 11:31:36 -06:00
Josh Slocum
1de4fb1b9e
Adding explicit stopVersion instead of stopped status to change feed durable data, several other minor fixes
2022-02-24 11:02:15 -06:00
Josh Slocum
6739d506c4
More fixes for handling change feed empty version >= ss version
2022-02-22 16:23:32 -06:00
Josh Slocum
d89c6bb3de
Fixing initial change feed stream reply to be in try/catch
2022-02-22 13:55:40 -06:00
Josh Slocum
5dca00a21d
Cleaning up change feed on move triggers when stream is cancelled
2022-02-22 13:48:57 -06:00
Josh Slocum
e75ab5f3a0
not advancing popped version of change feed sideband from another server
2022-02-22 11:22:35 -06:00
Josh Slocum
f9f676abf9
Also re-persisting change feed metadata on fetch after cleanup
2022-02-17 19:21:40 -06:00
Josh Slocum
ef25e4fdfb
Sending empty version back immediately to establish stream connection and acknowledgement queue for change feed stream requests
2022-02-17 18:10:17 -06:00
Josh Slocum
38a75a8b89
Merge branch 'main' into blob_integration
2022-02-17 17:47:38 -06:00
Josh Slocum
c87d868dd2
Fix SS restart and change feed pop races
2022-02-17 15:55:24 -06:00
Josh Slocum
f60420a8e6
Correctly updating change feed storage/durable versions after fetch
2022-02-17 10:36:32 -06:00
Josh Slocum
26480c4705
change feed popping fixes
2022-02-15 15:12:23 -06:00
Josh Slocum
53573e7648
Don't reset CF fetch version on pop
2022-02-15 15:12:05 -06:00
Josh Slocum
b5ff2006b2
Removing incorrect assert
2022-02-14 11:36:54 -06:00
Josh Slocum
9175ba354a
Adding a separate phase for the end of change feed fetching to handle changes in fetchKeys available durability
2022-02-11 17:25:56 -06:00
Josh Slocum
0db843bc56
more CF movement and cleanup races
2022-02-11 16:04:28 -06:00
Josh Slocum
3bbddce668
Fixing shard move away and back race
2022-02-11 10:14:26 -06:00
Josh Slocum
8a56349ea3
Fixing shard move away then back then away race
2022-02-11 10:12:49 -06:00
Josh Slocum
08505325d3
Fixing change feed mapping in cleanup refactor
2022-02-11 08:41:01 -06:00
Josh Slocum
f9c16ed937
misc change feed improvements
2022-02-11 08:14:00 -06:00
Josh Slocum
36742bcd7e
Fixing durability model of change feed cleanup
2022-02-10 18:52:17 -06:00
Josh Slocum
631cf0b085
minor memory optimizations for SS change feeds
2022-02-10 12:21:22 -06:00
Josh Slocum
f997b2dab4
Smarter change feed cancelling when data moved away
2022-02-10 12:21:22 -06:00
Josh Slocum
4af1b24c74
Using higher transferred version when fetching change feeds
2022-02-10 08:00:52 -06:00
Josh Slocum
9189ea0448
Fixed SS restart and mutation log race
2022-02-09 15:37:09 -06:00
Josh Slocum
d7955a9017
Much better mechanism for change feed popped checking
2022-02-08 14:23:04 -06:00
Josh Slocum
6adf119e3d
Fixing change feed durableFetchVersion not getting updated
2022-02-08 13:59:06 -06:00
Josh Slocum
90a870962d
Unblocking bounded change feed request in changeFeedClientVersions once it hit end
2022-02-08 13:22:17 -06:00
Sreenath Bodagala
72e06369a4
Merge remote-tracking branch 'apple-upstream/main' into version-vector-prototype
2022-02-08 17:47:57 +00:00
Josh Slocum
9885d2cc90
More change feed fetch and read race fixes
2022-02-07 16:21:38 -06:00
Josh Slocum
9b3f6fe452
More bugs in streaming split
2022-02-07 13:25:14 -06:00
Josh Slocum
88cab7fb67
More change feed fetching improvements and optimizations
2022-02-07 11:19:51 -06:00
Josh Slocum
ddfc301d74
Improving memory footprint of change feeds and making it configurable
2022-02-04 16:41:25 -06:00
Josh Slocum
0a3608a33b
Fixed issue with change feed fetching durability
2022-02-04 11:03:29 -06:00
Josh Slocum
03d91fd42d
Fixing change feed fetch durability semantics
2022-02-04 09:41:39 -06:00
Josh Slocum
a974773676
Handled change feed getting moved away or cancelled during move
2022-02-04 09:33:04 -06:00
Josh Slocum
d0c4c6856f
Fix pop at end of CF fetch
2022-02-03 12:38:37 -06:00
Josh Slocum
0f9b2f7c48
Change feed fetch refactor and pop during fetch improvements
2022-02-03 10:49:30 -06:00
Josh Slocum
b324cf4bc4
Fixing TSS buggify in restarting tests
2022-02-03 06:46:28 -06:00
Josh Slocum
d3f9c01120
Handling change feed fetch for feeds added during fetching a range
2022-02-02 15:21:11 -06:00
Josh Slocum
f187d01b71
Improving change feed performance and fixing durability issue
2022-02-02 11:41:29 -06:00
Josh Slocum
d0e89ecdd5
Including Change Feeds in fetch keys budget
2022-02-01 14:39:10 -06:00
Josh Slocum
2c22d07b39
Several change feed popping fixes for behind SS
2022-02-01 10:27:41 -06:00
Josh Slocum
25a0d857fa
Fixing issue with change_feed_popped
2022-01-31 17:19:52 -06:00
Josh Slocum
d4931348c1
Fix for new change feed movement
2022-01-31 11:18:47 -06:00
Josh Slocum
c3cece552c
Fixes for previous
2022-01-28 15:19:00 -06:00
Josh Slocum
cf7ed4ee40
Simplify change feed fetching to not inject anything at all during fetch and rely on reading from the remote feeds
2022-01-28 14:21:26 -06:00
A.J. Beamon
0803abedaa
Adjust the name order of constructor variable initialization to match declaration order.
2022-01-28 10:27:13 -08:00
Josh Slocum
2d3b216f60
Persisting change feed popped version in all cases to enable popped checking
2022-01-28 12:27:04 -06:00
Josh Slocum
ac1fd056dd
Added change feed popped read guard for blob workers
2022-01-28 10:45:33 -06:00
Josh Slocum
10c3cc870f
Fixing new race in change feed cleanup while still working for fetch+cleanup race
2022-01-28 09:26:36 -06:00
Josh Slocum
91df569fdd
Fetch Change Feed metadata at the start of fetchKeys to fix mutation race
2022-01-27 18:56:07 -06:00
Josh Slocum
c26e11c2c3
Adding extra debugid to change feed stream requests to track things better
2022-01-27 18:53:27 -06:00
Dan Lambright
fe1ec33d32
Fix failures in flatmap tests
2022-01-27 16:19:33 -05:00
Josh Slocum
bc7b76b407
Fix for gotAll in getChangeFeedMutations
2022-01-27 14:25:08 -06:00
Josh Slocum
ce5faa470b
Merge branch 'ss_cf_fix' into blob_integration
2022-01-26 16:49:08 -06:00
Josh Slocum
992fc8ed7c
Using non-expanded clears in change feeds
2022-01-26 15:28:58 -06:00
Andrew Noyes
c967e6246b
Fix uninitialized memory in storage server ( #6282 )
2022-01-26 13:24:35 -08:00
Josh Slocum
311a7656c0
Fixing change feed getting stuck not quite caught up, which causes empty version problems
2022-01-26 12:22:20 -06:00
He Liu
5e0b1fd13f
Resolved comments.
2022-01-21 16:58:42 -08:00
He Liu
0421562ce9
Resolved comments.
2022-01-21 16:58:42 -08:00
He Liu
b7388a714b
Added get, scan, and commit counters.
2022-01-21 16:58:42 -08:00
He Liu
da8453eb5d
Added systemClearRange.
2022-01-21 16:58:42 -08:00
He Liu
f14c832031
Added storage engine metrics in SS.
2022-01-21 16:58:42 -08:00
Josh Slocum
1180eb6e44
Fixed uncaught error in blob worker requests
2022-01-21 18:02:30 -06:00
Dan Lambright
1f67250402
Fix problem with PROXY_USE_RESOLVER_PRIVATE_MUTATIONS in resolver.
2022-01-21 10:39:06 -05:00
Josh Slocum
3be1bcd588
Fix move race in change feeds and added extra debugging to track down similar problems
2022-01-20 14:42:24 -06:00
Dan Lambright
9544379cdf
rebase
2022-01-20 11:12:33 -05:00
Josh Slocum
215f5fae93
Reworked change feed initialization to handle more types of races
2022-01-19 15:20:23 -06:00
Josh Slocum
79704aa05f
Fixing change feed private mutation handling
2022-01-18 20:17:39 -06:00
Josh Slocum
05d2eafd0c
Correctly updating change feed storage and durable versions after a fetch
2022-01-12 18:28:31 -06:00
Josh Slocum
cdb433569e
fixed range bug in filterMutationsInverted
2022-01-12 13:17:11 -06:00
Evan Tschannen
a7421e6ca2
fix: localChangeFeeds cannot get wrong_shard_server errors
2022-01-10 12:59:30 -08:00
Josh Slocum
b0aea91895
Broadening explicit disconnect handling to explicit error handling of all types
2021-12-21 14:12:09 -06:00
A.J. Beamon
d8e161f89e
Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state.
2021-12-17 11:57:39 -08:00
Josh Slocum
f6ea67120e
Adding explicit empty version back to change feeds for now
2021-12-10 10:04:05 -06:00
Evan Tschannen
9dcdda8192
fix: do not merge rollbacks when fetching
2021-12-07 20:34:01 -08:00
Andrew Noyes
def41697bf
Merge pull request #6083 from sfc-gh-tclinkenbeard/remove-temporaries
...
Avoid creating unnecessary temporary objects
2021-12-06 13:24:56 -08:00
Evan Tschannen
98b4299fb2
.error() on a trace event must be before the details
2021-12-05 14:59:46 -08:00
Tao Lin
9b0a9c4503
Return error when getRangeAndFlatMap has more & Improve simulation tests ( #6029 )
2021-12-03 12:50:07 -08:00
Josh Slocum
03f341fd3a
Merge branch 'bw_new_empty_versions' into blob_integration
2021-12-02 15:01:31 -06:00
Josh Slocum
d85eb330e0
retooling some waitForVersion stuff and adding asserts
2021-12-02 14:52:16 -06:00
Josh Slocum
9f6c8a123b
Fix: change feed update ordering
2021-12-02 13:29:30 -06:00
Josh Slocum
85c6e30aec
Fix: several empty version issues with large single-version change feed entries
2021-12-02 09:56:15 -06:00
sfc-gh-tclinkenbeard
3b5d23ef88
Remove unnecessary temporary objects when appending to vector of pairs
2021-12-01 18:15:16 -08:00
Evan Tschannen
f12ac4468e
fix: fetched change feeds did not get their durable version updated
2021-11-30 13:32:12 -08:00
Josh Slocum
efb21ca6a1
Merge branch 'master' into blob_integration
2021-11-24 13:17:54 -06:00
Evan Tschannen
b0aad44831
change feed requests with an explicit end need to get empty versions at the end boundary
2021-11-23 15:01:44 -08:00
Evan Tschannen
3360000a91
fix: an empty mutation vector did not mean that all of the data was returned
2021-11-23 15:01:44 -08:00
Evan Tschannen
e97d337ffe
fix: when a feed transitions from atLatest to not atLatest stall all updates at the blocked version
2021-11-23 15:01:44 -08:00
Evan Tschannen
10a925b7e9
do not return mutations larger than the dequeVersion
2021-11-23 15:01:44 -08:00
Evan Tschannen
f3bb1d8f51
properly handle an active change feed stream when removed
2021-11-23 15:01:44 -08:00
Jon Fu
8f6934c4d0
Merge branch 'master' of github.com:apple/foundationdb into jfu-fix-ss-segfault
2021-11-16 18:03:32 -05:00
Evan Tschannen
964d0209ca
Merge pull request #5637 from sfc-gh-ljoswiak/features/data-loss-prevention
...
Data loss protection when joining new cluster
2021-11-15 15:26:32 -08:00
Josh Slocum
f140028086
Merge branch 'master' into blob_integration
2021-11-15 15:09:40 -06:00
Evan Tschannen
6e81b83924
fix: cleanup change feeds which have been completely removed from a storage server
2021-11-15 11:47:42 -08:00
Josh Slocum
1d1f3fe747
changing assert for now
2021-11-15 12:24:51 -06:00
Josh Slocum
fd4f13fba1
Added fetch version to change feeds to avoid races and overwrites between updateStorage and fetchChangeFeed
2021-11-15 09:49:51 -06:00
Evan Tschannen
94a51e57a5
Merge branch 'master' into feature-changefeed-empty-versions
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2021-11-14 19:13:05 -08:00
Evan Tschannen
6909754b21
changefeeds now have a whenAtLeast function for efficiently learning when the version has updated but no mutations have been committed
2021-11-14 19:08:46 -08:00
Tao Lin
2d5f924278
GetKeyValuesAndFlatMap should return error if not retriable
2021-11-12 09:35:28 -08:00
Lukas Joswiak
c93052121f
Fix issue where transaction spans would not be recorded
2021-11-10 13:33:49 -08:00
Daniel Smith
499dbcdb18
Don't fail fetchKeys when server overloaded is returned
2021-11-10 14:15:42 -05:00
Daniel Smith
b50b3de5d0
Allow SS to respond with server overloaded
2021-11-10 11:52:02 -05:00
Tao Lin
fdb3b72e35
Introduce GetRangeAndFlatMap to push computations down to FDB
...
Re-introduce #5609
2021-11-09 13:52:28 -08:00
Lukas Joswiak
15e0d5b29f
Add explicit transaction options when reading cluster ID
2021-11-09 12:29:49 -08:00
Lukas Joswiak
4640045243
Fix rare simulation failures
...
When partitions appear before a cluster has fully recovered, it was
possible to have different tlogs persist different cluster IDs because
they were involved in different partitions. This would affect recovery
when a quorum was eventually reached. The solution to this is to avoid
persisting the cluster ID before a cluster has fully recovered, to make
sure all nodes agree on the cluster ID.
2021-11-09 12:29:48 -08:00
Lukas Joswiak
aa3383f0e3
Exclude when joining new cluster
2021-11-09 12:29:48 -08:00
Lukas Joswiak
3e2c65bb11
Allow tlog to join another cluster but retain its data
2021-11-09 12:29:48 -08:00
Lukas Joswiak
30867750b5
Add protection against storage and tlog data deletion when joining a new cluster
2021-11-09 12:29:47 -08:00
Jingyu Zhou
b4ed47053b
Remove toString for mutations in trace events
2021-11-08 10:04:16 -08:00
Jon Fu
396cd58b21
cancel ss core and ss actor collection after termination and before context switch
2021-11-04 16:05:23 -04:00
Tao Lin
586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB"
2021-11-04 08:46:56 -07:00
Tao Lin
6c98e35893
Rename Hop to FlatMap
2021-11-03 13:32:01 -07:00
Tao Lin
0853661d13
Introduce getRangeAndHop to push computations down to FDB
2021-11-03 13:21:16 -07:00
Jon Fu
67bd4ddea0
Add a wait(delay(0)) to storage server termination to avoid a rare segfault
2021-11-02 16:24:40 -04:00
Dan Lambright
befe1993c4
fix conflict on rebase
2021-10-29 12:25:26 -04:00
Josh Slocum
dbf46c200f
disabling long keys and adding more debugging stuff
2021-10-28 13:53:54 -05:00
Josh Slocum
6b51e149ed
Fixing bug where SS would get removed when someone else got its tag, but its TSS pair wouldn't
2021-10-27 10:11:49 -05:00
Josh Slocum
0ff8ddc2b6
Merge branch 'master' into blob_full_clean
2021-10-25 13:38:48 -05:00
Evan Tschannen
118c307b57
fixed formatting
2021-10-24 22:26:11 -07:00
Evan Tschannen
9a6384fc26
fixed merge conflicts
2021-10-24 21:18:49 -07:00
Evan Tschannen
6f7558b8ea
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# tests/CMakeLists.txt
2021-10-24 21:06:33 -07:00
A.J. Beamon
e882eb33fc
Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem.
2021-10-22 11:05:18 -07:00
Steve Atherton
d153519188
Merge pull request #5813 from sfc-gh-jslocum/ss_ebrake_streaming_fix
...
Fixes to ss e-brake, tlog streaming, and their interaction
2021-10-22 10:46:17 -07:00
Josh Slocum
773886515e
Merge branch 'feature-range-feed' into blob_full_clean
2021-10-22 11:07:51 -05:00
Evan Tschannen
f03d32f3d4
fix: handle the case where a fetch happens at an earlier read version than the commit version of the change feed registration
2021-10-21 23:04:51 -07:00
He Liu
16ae2b76e5
Merge branch 'master' of https://github.com/apple/foundationdb into clean-sim-test-data-loss
2021-10-21 09:16:53 -07:00
Evan Tschannen
f1158371a7
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# flow/error_definitions.h
2021-10-21 00:55:12 -07:00
Evan Tschannen
4e79296a9f
fixed a few bugs with fetching change feeds
2021-10-21 00:44:51 -07:00
Evan Tschannen
3ebabb6edc
fixed incorrect use of change feed errors
2021-10-20 22:37:31 -07:00
Evan Tschannen
e34d242581
fix: do not throw wrong_shard_server for local fetch keys
2021-10-20 22:04:39 -07:00
Josh Slocum
8dd7f8f447
Fixes to ss e-brake, tlog streaming, and their interaction
2021-10-20 10:48:29 -05:00
Evan Tschannen
ad3dcd6a74
fix: memory replies were not being set
2021-10-19 21:07:36 -07:00
Evan Tschannen
3f7df58a77
fixed a number of issues
2021-10-19 13:56:52 -07:00
Josh Slocum
85f64bf42c
more cleanup before merge
2021-10-18 17:11:14 -05:00
Josh Slocum
912ef76f1c
cleanup before merge
2021-10-18 17:11:14 -05:00
Sreenath Bodagala
246f035afe
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
2021-10-18 19:07:33 +00:00
Daniel Smith
9713a14ef1
Reverse order of read type and debug ID args
2021-10-18 12:23:09 -04:00
Daniel Smith
df53cc9580
Add an enum to IKeyValueStore to indicate the source/priority of the read
2021-10-15 14:35:59 -04:00
He Liu
66166e09be
Clear range before setting the moved-in empty range as available.
2021-10-15 09:30:06 -07:00
He Liu
dbfeb06c97
Reproduced user data loss incident, and tested the improved exclude tool
...
can fix the system metadata.
2021-10-14 14:08:39 -07:00
Josh Slocum
5f0ec0612a
Merge branch 'feature-range-feed' into blob_full
2021-10-13 15:44:35 -05:00
Evan Tschannen
282f84c807
added handling for broken promise
2021-10-12 10:36:10 -07:00
He Liu
9f974ef21f
Added logs for worker_removed() errors in SS.
2021-10-12 10:12:20 -07:00
Evan Tschannen
5b6f8b2abb
added the known committed version to change feeds
2021-10-11 13:53:36 -07:00
Evan Tschannen
26b6f9a3f1
fix: resize changes to source vector even though m is a copy
2021-10-10 11:32:49 -07:00
Evan Tschannen
5c642f706e
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
2021-10-09 19:34:16 -07:00
Evan Tschannen
efc4cec53f
fixed a variety of bugs with change feeds
2021-10-09 19:24:01 -07:00
Dan Lambright
f68d018a06
conflicts
2021-10-08 12:39:39 -04:00
Josh Slocum
bbe7c7ca9f
found change feed liveness bug
2021-10-06 12:32:16 -05:00
Chang Liu
bf0b8510dc
fix the problem about initialization list disorder in storageserver.actor.cpp file
...
Description
Testing
2021-09-28 10:19:51 -07:00
Chang Liu
76ab937e55
fix the warn about initialization order chaos in storageserver.actor.cpp
...
Description
Testing
2021-09-24 11:40:04 -07:00
Chang Liu
0af8aefeee
Stop rolling trace events that are associated with a process role when the role terminates
...
Description
Testing
2021-09-24 09:53:32 -07:00
Chang Liu
612477c3ed
Fix roll trace event issue
...
Description
Testing
2021-09-24 09:53:32 -07:00
Chang Liu
8427e40cbe
Fix roll trace event issue
...
Description
Testing
2021-09-24 09:53:32 -07:00
Chang Liu
48990058a3
Fix roll trace event issue
...
Description
Testing
2021-09-24 09:53:32 -07:00
He Liu
a1f6dcc2d5
merge from master
2021-09-22 13:07:38 -07:00
Josh Slocum
ccb78b1ce5
more CF bug fixes
2021-09-21 13:20:27 -05:00
Sreenath Bodagala
950de29d5a
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
2021-09-20 21:29:57 +00:00
Evan Tschannen
5f59c037af
buggify the streaming byte limit
2021-09-20 13:58:05 -07:00
Josh Slocum
8e4582673c
more bug fixes
2021-09-20 10:21:52 -05:00
He Liu
4d5bf08da8
address comments
2021-09-19 17:03:06 -07:00
Sreenath Bodagala
2aa3b44d4e
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
...
- Conflicts:
fdbserver/LogSystem.h
fdbserver/LogSystemConfig.h
fdbserver/TagPartitionedLogSystem.actor.cpp
- Files modified during merge:
modified: fdbserver/LogSystem.cpp
modified: fdbserver/LogSystemConfig.cpp
2021-09-17 19:36:18 +00:00
Xiaoge Su
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
Josh Slocum
c9b8bdffbe
Passes a single correctness test!
2021-09-15 17:18:04 -05:00
He Liu
ef7fdc0781
fmt
2021-09-15 10:32:09 -07:00
Josh Slocum
cefb66d64c
fixing a couple bugs
2021-09-10 12:52:33 -05:00
Josh Slocum
c2d1d1704f
Merge branch 'feature-range-feed' into blob_full
2021-09-10 11:21:52 -05:00
Evan Tschannen
ac5b580e2d
Merge branch 'master' into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/StorageServerInterface.cpp
# fdbclient/StorageServerInterface.h
# fdbserver/ApplyMetadataMutation.cpp
# fdbserver/TLogServer.actor.cpp
# flow/error_definitions.h
2021-09-09 23:13:22 -07:00
helium
65e6f26f05
bug fix on unwaited actor
2021-09-09 20:43:29 -07:00
helium
2ea055371b
Introduced serverKeysTrueEmptyRange to indicate the SS that the new
...
shard is empty.
2021-09-09 14:15:07 -07:00
Josh Slocum
eb76343dfb
Added blob granule reassignment and splitting
2021-09-08 14:09:14 -05:00
Evan Tschannen
590da022ee
fix: missing mutations committed at newOldestVersion
2021-09-07 16:20:49 -07:00
Evan Tschannen
f53ddf6720
fix: handle the case where I change feed is added while a fetch is in progress
2021-09-06 13:51:53 -07:00
Evan Tschannen
7b23894039
fixed a few bugs related to change feed fetches
2021-09-05 20:33:12 -07:00
Evan Tschannen
9fefd59076
Fixed a number of bugs, change feeds are correctness clean with the current workload
2021-09-05 18:17:23 -07:00
Evan Tschannen
af0792b1f5
fixes that allow change feeds to pass correctness in tests with no failures
2021-09-03 17:24:33 -07:00
Evan Tschannen
8309c2ce26
filter mutations in change feeds to the requested range
2021-09-03 12:06:13 -07:00
Evan Tschannen
4bbae59bb0
Fixed a few bugs and added a backup popping mechanism
2021-09-02 21:11:44 -07:00
Evan Tschannen
27151b0831
support reads from range feeds that span multiple storage servers
2021-09-01 14:35:51 -07:00
Dan Lambright
8689e1f106
merge with master
2021-08-30 15:29:08 -04:00
FDB Formatster
2c788c233d
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-08-27 17:07:47 -07:00
Daniel Smith
45b40addb7
Merge pull request #5379 from neethuhaneesha/eagerReadsDisable
...
Disabling option for removing eagerReads for ClearRange mutations.
2021-08-19 12:39:30 -04:00
Evan Tschannen
a278d2977a
renamed range feeds to change feeds, data distribution support almost complete
2021-08-13 14:27:15 -07:00
Josh Slocum
e444d3781c
Various TSS improvements from snowblower testing
2021-08-13 10:24:15 -05:00
Neethu Haneesha Bingi
c45daf6f51
Disabling option for removing eagerReads for ClearRange mutations.
2021-08-13 01:26:50 -07:00
Josh Slocum
1a4e70d22f
Merge branch 'feature-range-feed' into blob_full
2021-08-12 10:22:28 -05:00
Josh Slocum
76a3c47e42
Got full demo working
2021-08-12 10:18:58 -05:00
Lukas Joswiak
a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6
...
Actor sampling
2021-08-11 13:44:17 -07:00
Evan Tschannen
7891477a7c
added support for streaming range feeds
2021-08-10 21:28:25 -07:00
Evan Tschannen
db274c2431
a number of bug fixes
2021-08-10 16:47:01 -07:00
Sreenath Bodagala
cec744cebf
- Address the following issues:
...
- Sequencer should update the version vector once for a given commit
version (irrespective of the number of times that it receives and
processes the ReportRawCommittedVersionRequest message for that commit
version). Issue found by simulation tests.
- Storage server should take both its latest commit version and the
read version into account while processing a read request. This is to
address transaction_too_old error that we saw while running tests with
mako (and also in YCSB tests).
- Do not enable the tlog blocking-peek logic if ENABLE_VERSION_VECTOR
flag is set to false.
2021-08-10 19:47:18 +00:00
Josh Slocum
921a2cfca1
Merge branch 'feature-range-feed' into blob_full
2021-08-10 11:25:48 -05:00
Steve Atherton
54c7036eaf
Move role UIDs for MutationTracking TraceEvents from various inconsistent detail fields into the TraceEvent UID field.
2021-08-10 01:52:36 -07:00
Steve Atherton
faa4154a56
MutationTracking now uses a vector of keys to track. Removed "Mutation" detail from DEBUG_MUTATION() events because they are duplicates of the fields already logged in the returned MutationTracking event, which are now renamed and combined into "Mutation". Removed more toString() calls in TraceEvent detail values.
2021-08-09 23:30:45 -07:00
Evan Tschannen
a1b0053b57
do not return send a reply to a pop request until the clear has been made durable
2021-08-09 22:06:53 -07:00
Evan Tschannen
208a5790ad
fixed usage of durable version
2021-08-09 21:58:44 -07:00
Evan Tschannen
ed28aecde0
Merge branch 'master' into feature-range-feed
2021-08-09 20:40:55 -07:00
Evan Tschannen
42ae870c84
added support for querying specific range feed versions
2021-08-09 20:39:28 -07:00
Evan Tschannen
52fcf3f565
fixed a few bugs
2021-08-09 17:16:53 -07:00
Evan Tschannen
b03649d627
fixed assertion error
2021-08-09 12:55:57 -07:00
Evan Tschannen
27f87471ab
fixed compile errors
2021-08-09 12:48:38 -07:00
Evan Tschannen
bc9a0e1315
first attempt to add data distribution support for range feeds
2021-08-09 10:05:56 -07:00
Steve Atherton
2626fdc992
Don't loop over mutations with an empty loop if mutation tracking is disabled.
2021-08-08 00:44:17 -07:00
Steve Atherton
2bf7b9b7a9
Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang.
2021-08-08 00:04:54 -07:00
Josh Slocum
e5b0cf20aa
Reader can read blob worker interface and mapping from db, read files, apply mutations, and return result
2021-08-06 11:02:05 -05:00
Sreenath Bodagala
a081c0baa5
Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
2021-08-05 22:40:32 +00:00
Lukas Joswiak
5dc9a97230
Merge branch 'master' into fixes/alp6
2021-08-01 20:42:52 -07:00
Lukas Joswiak
78b5635d9f
Remove hot lineage modification
2021-08-01 20:21:15 -07:00
Evan Tschannen
125241743d
added support for removing range feeds
2021-07-30 16:01:46 -07:00
Evan Tschannen
0989c28a6b
made range feeds durable on the storage server
2021-07-30 15:23:42 -07:00
sfc-gh-tclinkenbeard
c74047c665
Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings
2021-07-28 11:51:02 -07:00
Evan Tschannen
406562b282
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
2021-07-27 17:04:58 -07:00
Lukas Joswiak
3eed4084e2
Merge branch 'master' into fixes/alp6
2021-07-27 11:26:53 -07:00
Lukas Joswiak
59d535149e
Merge branch 'master' into fixes/alp6
2021-07-27 10:07:18 -07:00
Steve Atherton
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
sfc-gh-tclinkenbeard
e5e449b340
Fix more -Wreorder-ctor warnings in storageserver.actor.cpp
2021-07-24 17:33:29 -07:00
sfc-gh-tclinkenbeard
b9a22a61ef
Fix many -Wreorder-ctor warnings
2021-07-23 17:33:18 -07:00
sfc-gh-tclinkenbeard
6f81155784
Merge remote-tracking branch 'origin/master' into const-serverdbinfo
2021-07-20 10:18:40 -07:00
Steve Atherton
f596a81073
Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs.
2021-07-17 00:11:40 -07:00
Evan Tschannen
a5e0e88df8
fix: reply promise streams could receive messages out of order if the sender and receiver were the same process
2021-07-15 15:27:49 -07:00
Sreenath Bodagala
f77093ea61
- Address compilation errors
2021-07-15 19:49:20 +00:00
Sreenath Bodagala
0dbd917012
- Address review comments
2021-07-15 19:49:20 +00:00
Sreenath Bodagala
b6f89df060
- Propagate the latest commit version of a storage server as part of read request.
...
Make storage server read at the specified version.
2021-07-15 19:48:42 +00:00
Evan Tschannen
15fe00927a
Merge branch 'master' of https://github.com/apple/foundationdb into feature-range-feed
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/StorageServerInterface.h
# fdbclient/SystemData.cpp
# fdbclient/SystemData.h
# fdbserver/storageserver.actor.cpp
# flow/ProtocolVersion.h
2021-07-12 19:10:14 -07:00
sfc-gh-tclinkenbeard
edbac4a26a
Prevent storageServer from modifying ServerDBInfo object
2021-07-11 22:04:58 -07:00
sfc-gh-tclinkenbeard
8cc40e3a2b
Expand use of BOOLEAN_PARAM
2021-07-02 21:41:50 -07:00
sfc-gh-tclinkenbeard
79ff07a071
Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters
2021-07-02 15:04:42 -07:00
Evan Tschannen
ac72120739
fixed a trace event which used fetchVersion before it was defined in the actor
2021-06-25 16:43:19 -07:00
Evan Tschannen
1cec223434
cherry pick streaming replies to master
2021-06-19 09:47:13 -07:00
Josh Slocum
d1d2ca9285
Don't inject TSS faults if speedUpSimulation is set
2021-06-18 12:41:48 -05:00
Neethu Haneesha Bingi
2f9cd71441
Histograms for storage server write path components.
2021-06-11 14:48:13 -07:00
Josh Slocum
65c49fc1e4
Merge pull request #4934 from sfc-gh-jslocum/tss_quarantine
...
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-11 11:52:42 -07:00
Tao Lin
49211fd663
More fixes to wording
2021-06-09 16:47:04 -07:00
Tao Lin
568d1a97b6
Remove duplicated logging and improve comments/style
2021-06-09 16:47:04 -07:00
Tao Lin
f64a73cc07
Consolidate the counters and add more comments
2021-06-09 16:47:04 -07:00
Tao Lin
a00fa584c0
Metrics to compare the bandwidth used by data distributions and updates
...
And more comments about how data distribution and updates work in storage server.
2021-06-09 16:47:04 -07:00
Josh Slocum
c6d96c4124
Added TSS Quarantine and fixed TSS mapping bug in CommitProxy
2021-06-09 14:42:50 +00:00
Andrew Noyes
402622ace9
Merge pull request #4909 from apple/anoyes/fix-ub
...
Fix several instances of undefined behavior
2021-06-07 08:58:45 -07:00
Lukas Joswiak
486a04659f
Lazy inititialization
2021-06-04 15:01:18 -07:00
Lukas Joswiak
153de33f57
Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
...
This reverts commit 6499fa178e
, reversing
changes made to 1512631957
.
2021-06-04 13:31:55 -07:00
Andrew Noyes
ce25a99000
Disallow conversion from float in specialCounter
2021-06-04 12:09:13 -07:00
Josh Slocum
b3e4f182ef
TSS Mapping Change
2021-06-02 17:30:09 +00:00
Josh Slocum
f6253db7dc
Addressing final PR comments
2021-05-28 18:19:42 +00:00
Josh Slocum
4257ac2b4d
More TSS Changes/Fixes
2021-05-25 20:37:48 +00:00
Josh Slocum
ce82c9653e
Testing Storage Server implementation
2021-05-25 20:28:50 +00:00