Commit Graph

10068 Commits

Author SHA1 Message Date
Meng Xu abda13e9df FastRestoreApplier:Free memory at each VB and refactor handleApplyToDBRequest 2020-05-04 15:29:27 -07:00
Meng Xu 135f6443da FastRestoreApplier:Add trace to track applying status 2020-05-04 15:02:53 -07:00
Meng Xu 9000a4a548 FastRestoreMaster:Free samples memory immediately after use 2020-05-04 11:49:36 -07:00
Meng Xu cb52cf2dfc Remove versionBatchStart var from RestoreRoleData 2020-05-04 11:42:19 -07:00
Meng Xu 68bcecd7d4 FastRestoreLoader:Clean batchData when a version batch finishes
Add sanity check for each loader phase to ensure an older version batch
will not be executed once a version batch has been marked as finished.
2020-05-04 11:31:42 -07:00
Meng Xu 0ba1551116 FastRestore:Trace memory usage periodically 2020-05-04 11:20:53 -07:00
Meng Xu 7b5d43da9c FastRestore:Remove unused field in RestoreRequest 2020-05-03 20:59:47 -07:00
Meng Xu ae86b5bb68 FastRestoreApplier:Continue when a key not exists in DB
Although we thought all keys cached in appliers should have
a base value in DB.
2020-05-03 20:47:21 -07:00
Vishesh Yadav 939a62449f
Merge pull request #3075 from xumengpanda/mengxu/valgrind-fix
Fix Valgrind error InvalidSuppression_FastRestoreWorkerError
2020-05-02 20:33:05 -07:00
Meng Xu 528466e0e6 FastRestore:Fix Valgrind error InvalidSuppression
Trace.error() must explicitly include error_code_actor_cancelled
to handle the error.
2020-05-02 19:52:05 -07:00
Evan Tschannen 17815fb6bf
Merge pull request #3037 from ajbeamon/status-busy-use-new-field
Use the updated field name in status when fetching process busyness info
2020-05-01 23:02:54 -07:00
Jingyu Zhou 488bbcf33d
Merge pull request #2952 from xumengpanda/mengxu/fr-code-improvement-PR
Performant restore [29/XX]: Improve coding style and efficiency
2020-05-01 20:32:11 -07:00
A.J. Beamon 9294f3ef4a
Merge pull request #3067 from atn34/atn34/set-versionstamped-key-no-ryw
Test SetVersionstampedKey without ryw enabled
2020-05-01 18:37:31 -07:00
Steve Atherton 175bdc842f
Merge pull request #3066 from tclinken/remove-dead-backup-agent-code
Remove dead backup agent code
2020-05-01 17:17:29 -07:00
Meng Xu 11a40b539e AtomicOpsWorkload:Lower Severity for TxnCommitUnknownResult 2020-05-01 16:39:41 -07:00
Meng Xu f9f1ac6594 FastRestore:Revise TraceEvent for better diagnosis 2020-05-01 16:31:55 -07:00
John Brownlee 7c36301d7b
Merge pull request #3070 from sergeyhush/allow-custom-fdb-cluster-file
Allow custom fdb.cluster file content
2020-05-01 15:55:44 -07:00
Sergey Sudakovich 3814b412fe Allow custom fdb.cluster file content 2020-05-01 15:42:16 -07:00
tclinken aca995d2c5 Removed dead backup agent code 2020-05-01 14:46:59 -07:00
Andrew Noyes 7edbbb642e Test SetVersionstampedKey without ryw enabled 2020-05-01 20:58:37 +00:00
Meng Xu 134dbca0ee FastRestore:Use cannonical way to trace error 2020-05-01 13:35:13 -07:00
Meng Xu ba07d449f6 FastRestore:Add trace to measure vb time 2020-05-01 13:30:45 -07:00
Jingyu Zhou 8b733c7ca3
Merge pull request #3064 from vishesh/master
build: Remove fdbclient/fdbclient.vcxproj
2020-05-01 13:17:32 -07:00
Vishesh Yadav c7cc546d80 build: Remove fdbclient/fdbclient.vcxproj
vcxproj files are unnecessary after CMake switch.
2020-05-01 12:33:46 -07:00
Evan Tschannen 08b3c26e5c
Merge pull request #2520 from etschannen/feature-proxy-load-balance
A master proxy load balancing algorithm
2020-05-01 12:18:54 -07:00
Evan Tschannen bd699f435c fixed compiler errors 2020-05-01 11:01:09 -07:00
Meng Xu 28178f356f FastRestore:Minor knob change and revise comments 2020-05-01 10:47:44 -07:00
Meng Xu 41c0a1768f FastRestore:Make FastRestore event type more descriptive 2020-05-01 10:27:08 -07:00
Meng Xu 05ba743f96 Control number of replies on wait in getBatchReplies 2020-05-01 10:09:08 -07:00
Meng Xu 2c17fff6bb FastRestore:Add batchIndex into metric event name 2020-05-01 10:05:33 -07:00
Meng Xu 038f3834fc Merge branch 'master' into mengxu/fr-code-improvement-PR 2020-05-01 09:26:29 -07:00
Meng Xu df63e43f79 Remove finished todo comment 2020-05-01 09:22:28 -07:00
Evan Tschannen aed2d34bcb Merge branch 'master' into feature-proxy-load-balance
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
Jingyu Zhou b3698ac921
Merge pull request #3060 from xumengpanda/mengxu/fr-agent-fix-PR
Fix fastrestore_agent errors when run in non-simulation mode
2020-04-30 22:46:39 -07:00
Alex Miller 9f2c07a97c
Merge pull request #2634 from skunkwerks/feature/add-freebsd
support FreeBSD
2020-04-30 22:00:45 -07:00
Meng Xu 06935f247c FastRestore:Change getBatchReplies from waitForAny to waitForAll
waitForAny may cause busy waiting.
For best performance, we should be able to control the number of outstanding requests to wait for.
2020-04-30 20:13:19 -07:00
Meng Xu fb1c456a2d FastRestore:Change default knob value 2020-04-30 19:15:32 -07:00
Meng Xu 6bd71560f0 FastRestore:Reduce trace events in real cluster environment 2020-04-30 19:12:31 -07:00
Meng Xu 32c791a9b0 FastRestore:Fix:memory threshold is set in MB not Bytes 2020-04-30 18:51:25 -07:00
Meng Xu 37a537c2a3 Fix MacOS compilation error 2020-04-30 18:30:15 -07:00
Jingyu Zhou ee104a116f
Merge pull request #3059 from xumengpanda/mengxu/too-many-files-fix-v2
Fix:Disable sanity check backup-metadata file
2020-04-30 16:41:40 -07:00
Meng Xu 07a9a05683 FastRestore:Agent:Fix restore requests 2020-04-30 16:20:20 -07:00
Meng Xu 6ee78aa3a4 Fix:Disable sanity check backup metadata file
Which can increase the false positive rate of TooManyFiles error
2020-04-30 16:16:14 -07:00
Dave Cottlehuber 15ba7edfb8 docs: add FreeBSD build steps 2020-04-30 22:07:17 +00:00
Dave Cottlehuber 28e58c672a fdbmonitor: ifdef config path location 2020-04-30 20:11:43 +00:00
Dave Cottlehuber b2eb93f5f4 fdbmonitor: define O_EVTONLY for FreeBSD 2020-04-30 20:11:43 +00:00
Dave Cottlehuber 5064cc776b fdbmonitor: update headers & libraries 2020-04-30 20:11:43 +00:00
Dave Cottlehuber 9753013b65 fdbrpc: update cmake & headers 2020-04-30 20:11:43 +00:00
Dave Cottlehuber d7fe80b612 cmake: add dtrace toggle 2020-04-30 20:11:26 +00:00
Evan Tschannen ea59021d86
Merge pull request #2513 from etschannen/feature-efficient-single-keyrange
do not serialize the begin key for single key ranges
2020-04-30 12:28:49 -07:00