Commit Graph

889 Commits

Author SHA1 Message Date
Lukas Joswiak 43fd545f06 Remove a/ from tracing path in special-key-space 2020-12-09 13:40:16 -08:00
Trevor Clinkenbeard 9581bc0279
Merge pull request #4129 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
Merge release 6.3 master
2020-12-08 08:57:54 -08:00
Lukas Joswiak 54237f28ea Throw error on commit for late trace options 2020-12-07 14:43:44 -08:00
Lukas Joswiak 18201d6f4f Add option to enable transaction tracing 2020-12-07 11:24:14 -08:00
A.J. Beamon 75836d6114 Fix double counting of range reads in TransactionMetrics 2020-12-07 09:01:19 -08:00
Andrew Noyes 7fbc4d7391 Resolve conflicts 2020-12-04 23:58:42 +00:00
Lukas Joswiak 76142549e0 Return keys from range read in lexicographic order 2020-12-04 11:21:39 -08:00
Andrew Noyes 877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
Lukas Joswiak 3669615e4b Refactor and clean up 2020-12-03 14:06:11 -08:00
Chaoguang Lin 88e66545bb Change the api to return bool instead of int 2020-12-03 12:01:32 -08:00
Richard Chen c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
Lukas Joswiak 1f166bc183 Add options to set custom transaction ID and disable tracing 2020-11-30 14:57:17 -08:00
Andrew Noyes 1f541f02be Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
Chaoguang Lin 2fae7333c0 Add duration parameter in the api 2020-11-23 14:55:29 -08:00
sfc-gh-tclinkenbeard b1e3478267 Simplified global knobs 2020-11-21 13:27:48 -08:00
Chaoguang Lin 2320f970c1 Add reboot_worker and pythong binding 2020-11-19 16:43:39 -08:00
Chaoguang Lin aec0398fd0 Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands 2020-11-17 11:39:12 -08:00
sfc-gh-tclinkenbeard 45c9a0abc7 Revert "Revert "Add limiting health metrics""
This reverts commit 209ebcc595.
2020-11-13 17:24:57 -08:00
Trevor Clinkenbeard 209ebcc595
Revert "Add limiting health metrics" 2020-11-13 17:08:46 -08:00
Trevor Clinkenbeard 8c0b4dbe4c
Merge pull request #4067 from sfc-gh-tclinkenbeard/add-limiting-health-metrics
Add limiting health metrics
2020-11-13 16:04:44 -08:00
sfc-gh-tclinkenbeard 6c4493166f Add limiting storage queue and durability lag to health metrics 2020-11-12 20:14:41 -08:00
Markus Pilman 0f205620bf
Merge branch 'master' into features/udp 2020-11-12 13:51:17 -07:00
Markus Pilman 1343f40117 don't allow empty coments 2020-11-11 14:07:54 -07:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Meng Xu 4788544a6f Revise comments based on review suggestions
Ack. Jingyu and Xin for their suggestions.
2020-11-06 08:51:13 -08:00
Meng Xu 063700e4d6 Add comments and questions to HA and tLog code reading
The comments' correctness need to be confirmed by reviewers.
2020-10-30 12:14:57 -07:00
sfc-gh-tclinkenbeard cf4c8e375f Merge remote-tracking branch 'origin/release-6.3' into merge 2020-10-29 22:15:41 -07:00
Chaoguang Lin a72bb52eae Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands 2020-10-28 22:32:44 -07:00
Richard Chen bf7bf470d9 Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces 2020-10-26 19:36:19 +00:00
Andrew Noyes a15bc32b14
Merge pull request #3945 from sfc-gh-clin/add-consistencycheck-to-special-keys
Add consistencycheck command to special keys
2020-10-26 10:59:17 -07:00
Richard Chen 309e38dc06 add unit test and some cleanup 2020-10-23 19:34:18 +00:00
Richard Chen 055add9682 conflicts 2020-10-23 06:33:00 +00:00
Richard Chen 90caa24af1 add function to get correct checksum class. Remove IChecksum::checkSum function 2020-10-23 02:24:29 +00:00
Chaoguang Lin c8eb003ec4 Change to \xff\xff/management/in_progress_exclusion, the snake case naming style 2020-10-22 18:05:54 -07:00
Jingyu Zhou c8b3b37b4a Remove unused code for #3930 2020-10-22 17:04:08 -07:00
Chaoguang Lin 95654ce7e7 Change dbLocked to snake_case naming db_locked 2020-10-22 11:35:39 -07:00
A.J. Beamon 2b4ec39a2f Avoid passing references from an iterator into an insertion that invalidates the iterator. 2020-10-22 11:30:08 -07:00
Chaoguang Lin 1d64b2c335 clang-format 2020-10-22 11:28:03 -07:00
Chaoguang Lin b728fe473e Add consistencycheck command to special keys 2020-10-22 11:08:54 -07:00
Chaoguang Lin 8d937a56f2 Throw operation_failed() if dd not found when calling ddMetricsRequestServer
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
Young Liu 8cc3e4d3c6 Merge release-6.3 into master 2020-10-19 22:51:56 -07:00
Richard Chen c84a5f4014 change IChecksum interface and remove coordinator protocol version optional 2020-10-19 21:39:07 +00:00
Richard Chen a4ccf8c5b6 add checksum class to change checksum in the future. expose get server protocol api 2020-10-19 16:57:11 +00:00
Trevor Clinkenbeard 24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
Andrew Noyes bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
Xin Dong 5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
Chaoguang Lin c66a775d64 Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys 2020-10-08 14:23:16 -07:00
Chaoguang Lin 53d3759e73 Add lock command to special key \xff\xff/dbLocked 2020-10-08 14:23:02 -07:00
Evan Tschannen f03a355c4e Merge branch 'release-6.2' into release-6.3
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/downloads.rst
#	fdbrpc/FlowTransport.actor.cpp
#	fdbrpc/FlowTransport.h
#	flow/Arena.h
#	flow/Knobs.cpp
#	flow/Knobs.h
#	packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
Lukas Joswiak dea7000970 Merge remote-tracking branch 'upstream/master' into visibility-1 2020-10-06 18:38:15 -07:00
A.J. Beamon 76da0a7cbc Backport #3426 to release-6.2 2020-10-05 13:53:51 -07:00
Evan Tschannen b294e148c5 another compile fix 2020-10-04 16:54:43 -07:00
Evan Tschannen 614c8bc895 Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy 2020-10-04 16:20:24 -07:00
Meng Xu bbc7ce581e Resolve conflicts merging from 6.3 to master 2020-09-25 16:08:31 -07:00
Meng Xu 862336de8f Merge branch 'master' into mengxu/merge-to-master-PR 2020-09-24 17:06:00 -07:00
Evan Tschannen b37daf88d4 add jitter to the delay 2020-09-22 22:24:40 -07:00
Evan Tschannen 09ff05c629 backed out buggify of apiVersion because there are too many tests that depend on it for a patch release 2020-09-22 21:15:49 -07:00
Evan Tschannen e0fbadfeb5 Added a buggify so older API versions are run in simulation 2020-09-22 18:58:01 -07:00
Evan Tschannen c99627a7ab Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-apiverison-batch-error 2020-09-22 17:06:17 -07:00
Evan Tschannen 7dec47938e protect api versions older than 630 from seeing batch_transaction_throttled errors 2020-09-22 17:05:09 -07:00
sfc-gh-tclinkenbeard b4e209bd4e Replace NULL with nullptr in fdbclient 2020-09-20 11:33:09 -07:00
Andrew Noyes a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
Add setclass to special keys
2020-09-19 14:55:03 -07:00
Young Liu 71309a79f9 Merge branch 'release-6.3' into range 2020-09-17 17:54:33 -07:00
Young Liu cc5bc16bd8 Rename more places from proxy to commit proxy 2020-09-15 22:29:49 -07:00
Chaoguang Lin 3dfadff6d8 Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands 2020-09-14 16:45:47 -07:00
Chaoguang Lin cc32074ed7 Rename actor 2020-09-14 16:45:34 -07:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Chaoguang Lin d270b15c89 Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space 2020-09-09 16:21:19 -07:00
Lukas Joswiak 1ca7fe1a05 Add span metadata message 2020-09-04 15:36:47 -07:00
Evan Tschannen ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
Xin Dong f9624b5a1a Avoid unnecessary copy 2020-09-03 13:56:23 -07:00
A.J. Beamon b4c96cadc7 Merge branch 'release-6.3' into merge-release-6.3-into-master 2020-09-02 12:45:57 -07:00
A.J. Beamon 93da4957ad
Update fdbclient/NativeAPI.actor.cpp 2020-09-01 13:17:01 -07:00
A.J. Beamon 800ece286d Results from read hot range requests were not held by the results arena. 2020-09-01 13:14:27 -07:00
Young Liu fd7198d874 Extend backup container interface to support query restorable files set by key ranges 2020-08-29 19:58:07 -07:00
Young Liu e87327b33b Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper 2020-08-29 12:47:35 -07:00
Xin Dong cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size 2020-08-27 10:06:02 -07:00
Young Liu e8e24a8ed6 Resolve comments 2020-08-26 21:04:35 -07:00
Chaoguang Lin a07b9f234d Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/ 2020-08-25 18:18:32 -07:00
Young Liu 63b3612ad5 Merge master branch and resolve conflicts 2020-08-24 16:42:31 -07:00
Xiaoxi Wang 3afdb44c7a merge master 2020-08-23 17:09:04 +00:00
David Youngworth b4cec6577a Fix merge bugs 2020-08-22 15:04:42 -07:00
David Youngworth e1b7dd0c7d Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1 2020-08-22 12:25:19 -07:00
Chaoguang Lin 23ef23357c Test code to add c function for kill/suspend 2020-08-21 14:30:52 -07:00
Chaoguang Lin 9b096ab58e Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands 2020-08-21 12:20:35 -07:00
Xiaoxi Wang 3b63d8b01b remove FIXME; remote tagSet.reset(); trivial changes 2020-08-21 19:17:16 +00:00
Xiaoxi Wang 5aaf86206a add counter for expensive clear estimation for audit and trace 2020-08-20 23:00:45 +00:00
Chaoguang Lin 556b239057 clang-format 2020-08-20 14:51:41 -07:00
Chaoguang Lin bff20faa42 Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space 2020-08-20 13:50:54 -07:00
Xiaoxi Wang e375761581 Merge branch 'master' of https://github.com/apple/foundationdb 2020-08-20 20:29:14 +00:00
Xiaoxi Wang 9c709b33d6 scale cost 2020-08-19 19:47:41 +00:00
Xiaoxi Wang 599675cba8 modify some details to get better performance 2020-08-19 04:23:23 +00:00
sfc-gh-tclinkenbeard 7b210aee24 Use =delete to disable special member functions
This has the benefits of:
- Easier to understand compiler error messages
- Moves some errors from link-time to compile-time
- NonCopyable classes can still specify default move constructors
2020-08-18 11:07:00 -07:00
Xiaoxi Wang 24dbb3961c Knob adjustment 2020-08-17 22:38:22 +00:00
Xin Dong 130bd7ac22 Fixed a bug that would potentially cause invalid memory access, i.e. segmentation fault. 2020-08-17 13:16:17 -07:00
Xin Dong daead84fdd Merge master 2020-08-14 14:07:08 -07:00
Xiaoxi Wang 548ba06510 change map to deque; code style problem 2020-08-14 18:30:58 +00:00
Xiaoxi Wang 475d4cca72 solve some cr comments 2020-08-14 16:40:16 +00:00
Chaoguang Lin b9474055a5 Add \xff\xff/configuration/class to read process class 2020-08-14 01:01:08 -07:00
Xiaoxi Wang e94de126de merge with master 2020-08-14 01:59:09 +00:00
Young Liu 20700f75f4 Merge master 2020-08-13 14:28:27 -07:00
Daniel Smith b2bba8ccbb
Correct fix for debug ID logging 2020-08-13 13:37:15 -04:00
Meng Xu 002cca5e58 Merge branch 'master' into tmp/merge-PR
Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/StorageMetrics.actor.h
2020-08-12 20:08:13 -07:00
Xin Dong 8bc57c16f0
Merge pull request #3637 from dongxinEric/misc/fix-spammy-readhot-logging
Increased some knobs to throttle the spammy read hot logging.
2020-08-12 16:46:32 -07:00
Xiaoxi Wang 8f1deaa47e Merge branch 'master' of https://github.com/apple/foundationdb 2020-08-12 23:32:23 +00:00
Xin Dong 3ac6996844 Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful 2020-08-12 14:27:34 -07:00
Daniel Smith cc1059be75 Make logging of server trace ID consistent with other code 2020-08-12 19:05:01 +00:00
Xiaoxi Wang f3ecf14601 change midShardSize type and other details 2020-08-12 17:49:12 +00:00
Young Liu 16ef2bd3bd Pending commit 2020-08-12 10:34:07 -07:00
Young Liu 79ce16650d merge master branch 2020-08-11 19:22:10 -07:00
Markus Pilman 4782c7c4b1 Merge remote-tracking branch 'origin/master' into features/udp 2020-08-11 15:35:52 -06:00
Lukas Joswiak b66dab8816 Fix span early destruction
We want the span to stick around for the full duration of
extractReadVersion since it writes to the log from its destructor.
2020-08-11 11:19:04 -07:00
Xiaoxi Wang c0b7ac0b7d try to throttle write tag and read tag seperatedly 2020-08-10 23:43:58 +00:00
Xiaoxi Wang cae6f04070 sample on cost instead of bytes 2020-08-10 22:30:06 +00:00
Xiaoxi Wang 696e77c94e query midShardSize from proxy 2020-08-10 20:13:44 +00:00
Xiaoxi Wang df9149fea4 ignore transaction tag of immediate transactions 2020-08-07 23:36:17 +00:00
Xin Dong 98c39ad9d4 Fix the build error. 2020-08-07 14:03:42 -07:00
Xin Dong a46e620baa Merge branch 'master' into feature/range-split-points-based-on-size 2020-08-06 15:23:11 -07:00
Markus Pilman 6d84bcb568 Merge remote-tracking branch 'origin/master' into features/udp 2020-08-06 14:08:34 -06:00
Markus Pilman 8976694ba1 UDP implementation (untested) 2020-08-06 14:06:50 -06:00
Xiaoxi Wang ba66b1f668 trivial changes 2020-08-06 14:53:26 +00:00
Young Liu d6a23a4d6b Resolve comments to make GRV proxy a separate process class 2020-08-06 00:01:57 -07:00
Xiaoxi Wang 13307679c5 use median shard size" 2020-08-05 03:57:25 +00:00
Xiaoxi Wang 4997754cf8 fix format in monitorDDMetricsChanges 2020-08-04 04:14:11 +00:00
Chaoguang Lin 00f2c38e3a Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands 2020-08-03 20:55:21 -07:00
Xiaoxi Wang 48094ea17e add INCOMPLETE_SHARD_PLUS knob 2020-08-04 03:18:05 +00:00
Xiaoxi Wang 52073738c0 fix bugs; change some KNOB values 2020-08-03 22:18:34 +00:00
Xiaoxi Wang b903e60cb7 fix monitorDDMetricsChanges bugs 2020-08-03 17:12:36 +00:00
Xiaoxi Wang 6db7040f17 update DDMetrics periodically 2020-08-03 05:52:38 +00:00
Xiaoxi Wang d1cc87452c merge with master; solve conflicts; solve initialization; 2020-08-02 22:44:07 +00:00
Xiaoxi Wang 92c1112c74 consider clear single key 2020-08-01 18:20:13 +00:00
Xiaoxi Wang 4f7dab4951 sample clear op on client 2020-08-01 06:14:52 +00:00
Xiaoxi Wang fc3dfc170f Merge branch 'master' of https://github.com/apple/foundationdb 2020-07-31 19:09:33 +00:00
Xiaoxi Wang c3a629588f add client transaction tag sample 2020-07-31 19:08:42 +00:00
Chaoguang Lin a048e5e159 Change prefix to \xff\xff/management and make decode/encode as pure functions 2020-07-31 02:18:49 -07:00
Chaoguang Lin f2662a9a74 Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space 2020-07-30 19:01:11 -07:00
Chaoguang Lin 10540a3c73 Change error name and fix a bug in ExclusionInProgressRangeImpl::getRange() 2020-07-30 18:58:09 -07:00
Chaoguang Lin 11cbcffc9a Temporary code 2020-07-30 16:07:54 -07:00
Young Liu b5dc6b715f Add file comment for GrvProxyServer.actor.cpp 2020-07-29 12:10:53 -07:00
Young Liu 30ea639666 Remove debug traces 2020-07-29 07:55:05 -07:00
Young Liu f7b76a92af pass joshua 2020-07-29 07:26:55 -07:00
Trevor Clinkenbeard 602b0f9dea
Merge branch 'master' into vectorref-emplace-back 2020-07-28 17:17:11 -07:00
Chaoguang Lin a641429d63 Add an enum for implementation type and use for registeration 2020-07-27 12:29:03 -07:00
Chaoguang Lin ec9553b6c6 change \xff\xff/failure to \xff\xff/error_message 2020-07-24 11:38:56 -07:00
Young Liu 525f10e30c Merge master branch 2020-07-22 16:08:49 -07:00
Young Liu 302cf5c45f Remove debug trace events 2020-07-22 12:20:22 -07:00
Young Liu 2703cedac5 Fixed known bugs 2020-07-17 22:24:52 -07:00
Xiaoxi Wang 0df2a8d014 better code style 2020-07-18 01:48:58 +00:00
Chaoguang Lin 1c51aaeea5 Use static keys in SpecialKeySpace 2020-07-17 15:14:18 -07:00
Chaoguang Lin 96e5a4c827 Replace all \xff\xff/conf with managementApiRange.begin 2020-07-17 11:56:27 -07:00