Commit Graph

386 Commits

Author SHA1 Message Date
Vishesh Yadav 73e77f31eb
Merge pull request #3928 from yliucode/merge
Merge release-6.3 into master
2020-10-20 15:48:46 -07:00
Young Liu 8cc3e4d3c6 Merge release-6.3 into master 2020-10-19 22:51:56 -07:00
Xin Dong 410d418c3e Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining"
This reverts commit e258dffc65, reversing
changes made to 0e96233015.

SS commit seems to be causing write performance degredation.
2020-10-19 09:28:38 -07:00
Jingyu Zhou 8f17a1a5d6 Merge branch 'release-6.2' into release-6.3 2020-10-16 15:25:39 -07:00
Evan Tschannen 1378ecba4d If a proxy is sufficiently far behind, reset network connections to attempt to fix the problem 2020-10-11 23:06:26 -07:00
Steve Atherton e258dffc65
Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining
Pipeline the commit process in SS.
2020-10-07 20:34:57 -07:00
Xin Dong b610f01c77 Fine-grained interleaving 2020-10-06 09:32:22 -07:00
Evan Tschannen f546034366 do not prevent computePerOperation from being updated for small computeDurations. Added logging for the compute per operation. Protect against erroneously large compute estimates 2020-10-04 19:19:05 -07:00
sfc-gh-tclinkenbeard f757beaac1 Remove deprecated UNFLUSHED_DATA_RATIO knob 2020-10-02 10:36:21 -07:00
Meng Xu 862336de8f Merge branch 'master' into mengxu/merge-to-master-PR 2020-09-24 17:06:00 -07:00
Meng Xu a4aad591fd FR:Remove applyStagingKeysBatchLock flow lock 2020-09-23 08:47:37 -07:00
Meng Xu 002b1bec4c FastRestore:Control write traffic at each applier
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.

This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
sfc-gh-tclinkenbeard 0814841827 Replace NULL with nullptr in fdbserver 2020-09-20 11:31:49 -07:00
Evan Tschannen 12b07d024f
Merge pull request #3775 from sfc-gh-xwang/fdbtest
add knob MIN_TAG_WRITE_PAGES_RATE for write tag-throttling
2020-09-18 10:40:28 -07:00
Young Liu cc5bc16bd8 Rename more places from proxy to commit proxy 2020-09-15 22:29:49 -07:00
XiaoxiWang 52bd86ad42 update knob 2020-09-11 20:50:19 +00:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Young Liu 87693cae81 merge master branch and resolve conflicts 2020-09-02 13:44:33 -07:00
Evan Tschannen 12edadd059 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	fdbclient/Knobs.cpp
#	fdbclient/MasterProxyInterface.h
#	fdbrpc/simulator.h
#	fdbserver/MasterProxyServer.actor.cpp
#	tests/fast/CycleAndLock.txt
#	tests/fast/TxnStateStoreCycleTest.txt
#	tests/fast/VersionStamp.txt
#	tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
#	tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
#	versions.target
2020-08-31 19:33:34 -07:00
Evan Tschannen 29eec30183 Merge branch 'release-6.2' into release-6.3
# Conflicts:
#	CMakeLists.txt
#	build/Dockerfile
#	build/Dockerfile.devel
#	documentation/sphinx/source/downloads.rst
#	fdbserver/Knobs.cpp
#	fdbserver/LogSystem.h
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
#	fdbserver/WaitFailure.actor.cpp
#	fdbserver/fdbserver.vcxproj
#	fdbserver/fdbserver.vcxproj.filters
#	packaging/msi/FDBInstaller.wxs
2020-08-31 01:10:29 -07:00
Young Liu e87327b33b Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper 2020-08-29 12:47:35 -07:00
Meng Xu 996ba2374c FastREstore:Fix:Incorrect condition in printing out FastRestoreLoaderSendMutationToApplierDoneTooLate 2020-08-24 17:27:40 -07:00
Young Liu 63b3612ad5 Merge master branch and resolve conflicts 2020-08-24 16:42:31 -07:00
Xiaoxi Wang 3afdb44c7a merge master 2020-08-23 17:09:04 +00:00
Meng Xu a32cf7d293 FastRestore:Loader:Add sanity check before remove batch data 2020-08-18 15:44:22 -07:00
Evan Tschannen 8ede143941 Track tlog push latencies and reset connections if they are above 500ms 2020-08-18 08:43:14 -07:00
Meng Xu 7b7490efe7 FastRestore:Debug trace for seg fault 2020-08-17 20:34:33 -07:00
Xiaoxi Wang 24dbb3961c Knob adjustment 2020-08-17 22:38:22 +00:00
Meng Xu cb0e44ff42 FastRestoreLoader:Sched send mutations per loadingParam requests 2020-08-16 07:38:51 -07:00
Meng Xu 015397be74 FastRestore:LoaderScheduler:Control number of load request to release at once 2020-08-15 22:18:48 -07:00
Meng Xu a035e7a872 FastRestore:LoaderScheduler:Add knobs 2020-08-15 21:44:51 -07:00
Xiaoxi Wang ffa1973242 move OPERATION_COST_BYTE_FACTOR to CLIENT_KNOB; skip the first call of refreshBusiestTag. 2020-08-14 22:49:20 +00:00
Daniel Smith 94571786bf Add RocksDB work 2020-08-14 17:16:20 +00:00
Young Liu 8953cc5f98 Merge master again 2020-08-12 16:59:07 -07:00
Xiaoxi Wang 8f1deaa47e Merge branch 'master' of https://github.com/apple/foundationdb 2020-08-12 23:32:23 +00:00
Young Liu 79ce16650d merge master branch 2020-08-11 19:22:10 -07:00
Xiaoxi Wang 7ee6ca342e merge with master 2020-08-11 01:01:15 +00:00
Young Liu 2e41391690 Merge master branch 2020-08-06 00:11:00 -07:00
Young Liu d6a23a4d6b Resolve comments to make GRV proxy a separate process class 2020-08-06 00:01:57 -07:00
Meng Xu fe5902994c
Merge pull request #3605 from apple/release-6.3
Merge Release 6.3 to master
2020-08-05 23:37:44 -07:00
Meng Xu 7992cef025 FR:Fix sample network pkg can be too big 2020-08-04 22:35:21 -07:00
Xiaoxi Wang d1cc87452c merge with master; solve conflicts; solve initialization; 2020-08-02 22:44:07 +00:00
Xiaoxi Wang 0352e8ee0b pick busiest commit tag periodically 2020-08-02 18:38:56 +00:00
Meng Xu 2f5293fcc7 Introduce knob FASTRESTORE_USE_LOG_FILE and FASTRESTORE_USE_RANGE_FILE 2020-07-31 10:40:29 -07:00
Meng Xu ad915e462e Add knob FASTRESTORE_NOT_WRITE_DB to skip writting to DB 2020-07-30 10:17:17 -07:00
Evan Tschannen a49cb41de7 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	cmake/ConfigureCompiler.cmake
#	fdbserver/Knobs.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/ThreadHelper.actor.h
#	flow/serialize.h
#	tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
Evan Tschannen 937df4f839 Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-lifetimetoken-fix
# Conflicts:
#	documentation/sphinx/source/release-notes/release-notes-630.rst
2020-07-27 10:03:02 -07:00
Young Liu 06c081c714 Merge master branch and resolve conflicts 2020-07-23 22:41:10 -07:00
Evan Tschannen be67e9cfc7 wait for the correct cluster controller interface before starting master recovery 2020-07-20 11:29:37 -07:00
Steve Atherton e646361501 Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-redwood 2020-07-20 07:25:29 -07:00