Commit Graph

17429 Commits

Author SHA1 Message Date
Jingyu Zhou 44943d3866 Fix initialization order 2021-10-18 16:47:36 -07:00
Dan Lambright f5bf1ee741
Merge pull request #5799 from sbodagala/version-vector-prototype
Version vector prototype: Rebase
2021-10-18 17:03:27 -04:00
Sreenath Bodagala 35804aa26b - Address a merge conflict 2021-10-18 19:24:39 +00:00
Sreenath Bodagala 246f035afe Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 2021-10-18 19:07:33 +00:00
Dan Lambright 26f82440c8
Merge pull request #5793 from sbodagala/version-vector-prototype
Version vector prototype: Propagate written tags from proxy to sequencer
2021-10-18 14:51:56 -04:00
Daniel Smith faf16fb29e
Merge pull request #5785 from Daniel-B-Smith/ikvs-read-type
Add an enum to IKeyValueStore to indicate the source/priority of the read
2021-10-18 13:21:20 -04:00
Sreenath Bodagala 06f5401d14 Propagate written tags from proxy to sequencer when ENABLE_VERSION_VECTOR
is enabled, not when ENABLE_VERSION_VECTOR_TLOG_UNICAST is enabled.
2021-10-18 16:31:31 +00:00
Daniel Smith 9713a14ef1 Reverse order of read type and debug ID args 2021-10-18 12:23:09 -04:00
A.J. Beamon 507a09893c
Add ClientCount to ClusterControllerMetrics (#5748) 2021-10-17 20:47:11 -07:00
Dan Lambright 30ac2fa23b
Merge pull request #5779 from dlambrig/vv-tlog-recovery
wake waiters when data made durable
2021-10-15 15:09:13 -04:00
Dan Lambright f099bb2574 comments on this PR's change 2021-10-15 15:08:25 -04:00
Dan Lambright 99f0eb369d
Merge pull request #5726 from dlambrig/counters
stats on version vector size
2021-10-15 14:44:12 -04:00
Dan Lambright 46044f544e Add todo w.r.t. adding vv delta sizes to json output 2021-10-15 14:42:55 -04:00
Daniel Smith df53cc9580 Add an enum to IKeyValueStore to indicate the source/priority of the read 2021-10-15 14:35:59 -04:00
Dan Lambright 7c8990ef09
Merge pull request #5783 from dlambrig/cherry-pick-5700
Cherry-pick moving private mutation to Resolvers to version vector branch
2021-10-15 13:45:09 -04:00
Dan Lambright dbbda24879 Address review comments 2021-10-15 12:17:39 -04:00
Jingyu Zhou dd8840837d Assert private mutation counts are the same from Resolvers 2021-10-15 12:13:20 -04:00
Dan Lambright 212e32a600 stats on version vector size 2021-10-15 11:39:35 -04:00
Dan Lambright 15dc5a3e41 wake waiters when data made durable 2021-10-15 10:58:48 -04:00
Jingyu Zhou c6a9ac11a8 fix another conflict fdbserver/ApplyMetadataMutation.cpp 2021-10-15 09:48:33 -04:00
Jingyu Zhou e288348675 Fix forceRecovery flag to control private mutations
The flag change from initial commit should be ignored, while the rest should be
honored.
2021-10-15 09:47:23 -04:00
Jingyu Zhou fc7325e923 Fix force recovery when generating private mutations
Resolver need to skip transactions after forceRecovery flag is true.
2021-10-15 09:47:23 -04:00
Jingyu Zhou 0a88d6a94c Limit resolution balancing to normal keys
This is for resolver private mutation feature if used.
2021-10-15 09:47:23 -04:00
Jingyu Zhou 34e9932778 Separate \xff keyspace from keyResolvers
This is needed to make sure all Resolvers process all metadata mutations.
For commit proxy, we add all read/write conflict ranges for metadata mutations
to all resolvers.

We also send lock_aware flag for each transaction to resolvers.
2021-10-15 09:47:23 -04:00
Jingyu Zhou b47da4f2ea Limit resolution balancing to normal keys
When using resolver generated private mutations, all resolvers must process all
metadata mutations. So there is no point of load balancing among resolvers for
the system, i.e., \xff keyspace.
2021-10-15 09:47:23 -04:00
Jingyu Zhou fddf16534d Send metadata mutations to all Resolvers
This is to make sure all Resolvers process them and make the same decision,
as well as maintain the same in-memory states, e.g., keyServers, serverKeys.
2021-10-15 09:47:23 -04:00
Jingyu Zhou eaf0a00502 Enable PROXY_USE_RESOLVER_PRIVATE_MUTATIONS for simulation tests 2021-10-15 09:47:23 -04:00
Jingyu Zhou 0eeee21a9f Fix duplicated fields in trace events 2021-10-15 09:47:23 -04:00
Jingyu Zhou 0dc9c607f4 Add knob PROXY_USE_RESOLVER_PRIVATE_MUTATIONS
To control proxy to use private mutations from resolvers or not.
2021-10-15 09:47:23 -04:00
Jingyu Zhou c1d7b03087 CommitProxy uses Resolver's private mutations
Note the commit proxy still runs applyMetadataMutations for setting up memory
states, and no longer generates private mutations.
2021-10-15 09:47:23 -04:00
Jingyu Zhou 90a27bedf5 fix conflict fdbserver/ApplyMetadataMutation.cpp 2021-10-15 09:47:20 -04:00
Jingyu Zhou 9ecdbc39ee Add private mutations to Resolver reply messages 2021-10-15 09:45:57 -04:00
Jingyu Zhou ad64ee9858 Format fix 2021-10-15 09:45:57 -04:00
Jingyu Zhou ff75d22ce8 Fix test failures
Need to be careful what values applyMetadataMutation needs.

Need to call txnStateStore->enableSnapshot() after receiving the broadcast to
allow semiCommit to happen.
2021-10-15 09:45:57 -04:00
Jingyu Zhou fbc6f45190 Generate private mutations at Resolvers 2021-10-15 09:45:57 -04:00
Jingyu Zhou f79ca8d7fb Add actor error handling in Resolvers 2021-10-15 09:45:57 -04:00
Jingyu Zhou e0184bc8aa resolve conflict 2021-10-15 09:45:52 -04:00
Jingyu Zhou 5e6fa0737a Master broadcasts txnStateStore to Resolvers 2021-10-15 09:45:13 -04:00
Jingyu Zhou 23deeb9618 Add txnState endpoint to ResoverInterface 2021-10-15 09:45:13 -04:00
Aaron Molitor 0e4a6cc921 add ycsb Dockerfile 2021-10-15 00:27:20 -07:00
He Liu 5688672b50
Merge pull request #5747 from liquid-helium/cc-interface-changes
Added new endpoints in Cluster Controller interface.
2021-10-14 17:03:36 -07:00
He Liu ddd08a99d7 fmt 2021-10-12 16:27:51 -07:00
He Liu 422d8a041d Added comments. 2021-10-12 15:43:18 -07:00
He Liu 8174c57714
Merge pull request #5722 from liquid-helium/add-logs
Added logs for worker_removed() errors in SS.
2021-10-12 15:16:11 -07:00
Yao Xiao e543570343
Add locality cache size to TransactionMetrics. (#5758)
Co-authored-by: Yao Xiao <yaoxiao@Yaos-MacBook-Pro.local>
2021-10-12 13:10:15 -07:00
He Liu 9f974ef21f Added logs for worker_removed() errors in SS. 2021-10-12 10:12:20 -07:00
Josh Slocum 0dafb95bbf Fixing tss private mutations ranges 2021-10-11 18:14:29 -07:00
Dan Lambright fdb67a2abf
Merge pull request #5754 from dlambrig/always-true
have ENABLE_VERSION_VECTOR true by default
2021-10-11 18:41:04 -04:00
Dan Lambright ca8f4dd9d0 ENABLE_VERSION_VECTOR true 2021-10-11 18:38:54 -04:00
Yao Xiao c8e6819a10
Add FastAlloc memory utilization trace. (#5739)
Co-authored-by: Yao Xiao <yaoxiao@Yaos-MacBook-Pro.local>
2021-10-11 15:06:43 -07:00