Commit Graph

204 Commits

Author SHA1 Message Date
Jingyu Zhou 0cce5feae3 Optimize LogPushData to avoid constructing LogSystemConfig
Which seems to be a CPU hot spot in our testing.
2022-05-06 12:33:19 -07:00
Ray Jenkins dc9e782ccc
OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
Ray Jenkins 1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. (#6855)" (#6941)
This reverts commit 5df3bac110.
2022-04-25 09:29:56 -05:00
Ray Jenkins 5df3bac110
Migrate to OpenTelemetry tracing. (#6855) 2022-04-20 09:26:37 -05:00
Jingyu Zhou cfcf0f152c Merge branch 'main-4a085fc84' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
2022-03-30 22:28:06 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Sreenath Bodagala 72e06369a4 Merge remote-tracking branch 'apple-upstream/main' into version-vector-prototype 2022-02-08 17:47:57 +00:00
Jingyu Zhou f361f5a96c Rename isEmptyMessage to messagesWritten
The variable name means exactly the opposite.
2022-02-03 14:55:44 -08:00
Dan Lambright 9544379cdf rebase 2022-01-20 11:12:33 -05:00
Dan Lambright 9f4ac866cd Avoid context switch between appending version list and updating dv
Port PR 6117 (Resolver saves shardChanged in recent state transactions)
2021-12-13 13:02:32 -05:00
Dan Lambright 0222d8669d fix simulation failures 2021-12-10 09:56:21 -05:00
Lukas Joswiak 30867750b5 Add protection against storage and tlog data deletion when joining a new cluster 2021-11-09 12:29:47 -08:00
Jingyu Zhou c1d7b03087 CommitProxy uses Resolver's private mutations
Note the commit proxy still runs applyMetadataMutations for setting up memory
states, and no longer generates private mutations.
2021-10-15 09:47:23 -04:00
Jingyu Zhou 9ecdbc39ee Add private mutations to Resolver reply messages 2021-10-15 09:45:57 -04:00
Dan Lambright 58e1888d8e remove network hop by getting previous commit versions in GetCommitVersionRequest 2021-09-30 11:51:57 -04:00
Sreenath Bodagala 184c134b8a - Resolve merge conflicts 2021-09-17 20:25:16 +00:00
Sreenath Bodagala 2aa3b44d4e Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype
- Conflicts:
	fdbserver/LogSystem.h
	fdbserver/LogSystemConfig.h
	fdbserver/TagPartitionedLogSystem.actor.cpp

- Files modified during merge:

modified:   fdbserver/LogSystem.cpp
modified:   fdbserver/LogSystemConfig.cpp
2021-09-17 19:36:18 +00:00
Sreenath Bodagala 6716ed0d17 Address build failure. 2021-09-16 12:00:26 -04:00
Xiaoge Su 067c1cc55b Extract methods in LogSystem.h to corresponding cpp file 2021-09-12 14:17:19 -07:00
Dan Lambright 8689e1f106 merge with master 2021-08-30 15:29:08 -04:00
FDB Formatster 2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
Xiaoxi Wang 1f6cee89ab merge master, fix conflicts 2021-08-10 10:01:45 -07:00
Sreenath Bodagala a081c0baa5 Merge remote-tracking branch 'apple-upstream/master' into version-vector-prototype 2021-08-05 22:40:32 +00:00
yao-xiao-github 8609b45354
Add histograms to CommitProxyServer. (#5299) 2021-08-05 09:17:37 -07:00
Xiaoxi Wang 2df0474fec merge master 2021-08-02 11:58:35 -07:00
Xiaoxi Wang 10c82b422f merge master branch 2021-07-28 14:19:46 -07:00
sfc-gh-tclinkenbeard c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Steve Atherton 507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
Sreenath Bodagala 82774ae62c - Address review comments 2021-07-26 19:39:08 +00:00
Sreenath Bodagala de020efa7f - Correct a comment 2021-07-26 17:13:31 +00:00
Sreenath Bodagala d398377c4a - Move the logic that populates written_tags (the tags that have been
affected by a transaction batch) to LogPushData
2021-07-26 16:27:19 +00:00
sfc-gh-tclinkenbeard b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Dan Lambright d07c8ce211 wait for prev per tlog 2021-07-15 08:44:48 -04:00
Dan Lambright 17be3dc201 fix bug related to misunderstanding of tag replicas. 2021-07-12 11:46:52 -04:00
Dan Lambright a107dd655e tpcv messaging 2021-07-12 11:46:52 -04:00
Xiaoxi Wang 569de5259f Merge branch 'master' of https://github.com/apple/foundationdb into tlog_dev 2021-07-06 23:15:20 +00:00
Xiaoxi Wang b6d5c8a091 implement tLogPeekStream 2021-07-06 23:14:58 +00:00
Xiaoxi Wang b50fda6b4b add simple streaming peek functions 2021-07-01 23:17:28 +00:00
Jingyu Zhou 9c82b22aa3 Add comments for recordEmptyMessage and getEmptyMessageRatio. 2021-06-28 16:09:07 -07:00
Jingyu Zhou c700feaa6e Address Dan's comments 2021-06-28 11:16:35 -07:00
Jingyu Zhou 66a97b76f1 Add metric to track the ratio of empty messages to tlogs
This metric is to understand the impact of batching on empty commit messages.
If the ratio is high, it means proxy sends many empty commit messages, which
can potentially be optimized for better performance. If the ratio is low, which
means the spread factor is large, thus we need to optimize the proxy to reduce
such a factor.
2021-06-24 21:26:09 -07:00
Sreenath Bodagala b0554b4554 Capture how fast an SS is catching up to its tLog-SS lag
Changes:
LogSystem.h, LogSystemPeekCursor.actor.cpp:
Add APIs to find the ID of the tLog from which an SS has fetched the latest
set of versions.

storageserver.actor.cpp:
Capture the number of latest set of versions fetched, the time (in seconds)
in which those versions were fetched, and the tLog from which they were
fetched. Add this information to a TraceLogEvent.

Capture how many versions an SS has fetched in the
2021-05-11 20:03:21 +00:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Evan Tschannen 346a4e3ecd Merge branch 'release-6.3'
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbrpc/LoadBalance.actor.h
#	fdbrpc/MultiInterface.h
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/masterserver.actor.cpp
2021-03-01 18:52:06 -08:00
Meng Xu 33eb1de00e Add some comment to log system
and resolve review comment by deleting my questions.
2021-02-19 21:44:13 -08:00
sfc-gh-tclinkenbeard 5bfa6cea98 Merge remote-tracking branch 'origin/master' into misc-changes 2020-12-26 20:47:00 -04:00
sfc-gh-tclinkenbeard 5020e3faa1 Make ILogSystem::IPeekCursor const-correct 2020-12-08 09:09:31 -08:00
sfc-gh-tclinkenbeard dd3669886e Improve IPeekCursor and ILogSystem method signatures 2020-12-08 09:09:31 -08:00
Richard Chen c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
Lukas Joswiak 7442e43dbf Add TEST and ASSERT macros 2020-10-30 13:41:56 -07:00