Commit Graph

199 Commits

Author SHA1 Message Date
Yi Wu 994b8c92f8
Add option to limit resident memory and remove default memory limit (#6719)
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.

Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.

closes #6671, closes #6672
2022-04-06 20:06:24 -07:00
Renxuan Wang 465ff712b6
Move Hostname to its own files. (#6759)
* Change DNS cache to use std::map.

Revert commit 90c259d84e, because if we use unordered_map, toString() can be inconsistent.

* Move ClientKnob::COORDINATOR_HOSTNAME_RESOLVE_DELAY to FlowKnob::HOSTNAME_RESOLVE_DELAY.

* Move Hostname to its own files.

Also, add resolve-related variables and functions in Hostname.
2022-04-04 19:04:51 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Xiaoge Su 99b030c2f6 Allow the TOML file assign knobs during test
In this patch, for a given test, it is possible to override the knob
values, e.g.

[[test]]

    [[test.knobs]]
    watch_timeout = 999

will set the client knob WATCH_TIMEOUT to 999 during the test. The
original value will be recovered after the test is over.
2022-03-17 16:53:10 -07:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
sfc-gh-tclinkenbeard a32547cbbe Merge remote-tracking branch 'origin/main' into flow-enhancements 2022-02-15 15:56:42 -08:00
Ray Jenkins 226236600e Preserving original LISTENER_PORT knob and adding new Knob TRACING_UDB_LISTENER_ADDR 2022-02-07 15:49:35 -08:00
Ray Jenkins 1bd084810c Refactor TRACING_UDP_LISTENER_PORT to TRACING_UDP_LISTENER_ADDR. 2022-02-07 15:49:35 -08:00
Steven Li 36c6e23700
Add TraceEvent severity knob (#6326)
* add knob for trace event severity

* add knob for TraceEvent severity

* fix format

* fix switch format

* moved intToSeverity call inside __test initialization

* updated knob name

* fix line length format

* fix format

* git clang-format
2022-02-02 11:36:17 -08:00
sfc-gh-tclinkenbeard 8076806e76 Add ITERATIONS_PER_REACTOR_CHECK 2022-01-01 20:14:57 -08:00
Steve Atherton 035e0d6e52
Merge branch 'master' into bit-flipping-workload 2021-11-16 14:42:22 -08:00
Steve Atherton 508429f30d
Redwood chunked file growth and low priority IO starvation prevention (#5936)
* Redwood files now growth in large page chunks controlled by a knob to reduce truncate() calls for expansion.   PriorityMultiLock has limit on consecutive same-priority lock release.  Increased Redwood max priority level to 3 for more separation at higher BTree levels.

* Simulation fix, don't mark certain IO timeout errors as injected unless the simulated process has been set to have an unreliable disk.

* Pager writes now truncate gradually upward, one chunk at a time, in response to writes, which wait on only the necessary truncate operations.   Increased buggified chunk size because truncate can be very slow in simulation.

* In simulation, ioTimeoutError() and ioDegradedOrTimeoutError() will wait until at least the target timeout interval past the point when simulation is sped up.

* PriorityMultiLock::toString() prints more info and is now public.

* Added queued time to PriorityMultiLock.

* Bug fix to handle when speedUpSimulation changes later than the configured time.

* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.

* Renamed updatingInPlace to updatingDeltaTree for clarity.  Inlined switchToLinearMerge() since it is only used in one place.

* Updated extendToCover to be more clear by passing in the old extension future as a parameter.  Fixed initialization warning.
2021-11-12 13:47:07 -08:00
negoyal 1e7338b6c3 Merge branch 'master' into bit-flipping-workload 2021-10-28 14:24:49 -07:00
A.J. Beamon abab45760d Add some additional logging if the network thread finishes, fails with an error, gets stopped, or is blocked. 2021-10-19 10:05:29 -07:00
negoyal f913dfed97 Merge branch 'master' into bit-flipping-workload 2021-10-11 16:34:57 -07:00
Lukas Joswiak e034f66fc2 Add knob to control trace recording percentage 2021-10-01 16:30:46 -07:00
negoyal 3b34423248 Merge branch 'master' into bit-flipping-workload 2021-08-31 12:14:51 -07:00
Lukas Joswiak d554205db3 Move common knob initialization macros to header 2021-08-23 11:41:01 -07:00
Lukas Joswiak e2d897a2c2 Clean up types 2021-08-23 11:41:01 -07:00
Lukas Joswiak 7998ec4232 Add atomicity to knobs 2021-08-23 11:41:00 -07:00
sfc-gh-tclinkenbeard 82546853c0 Rename UseConfigDB to ConfigDBType 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard cdbcb69d86 Add configuration database type to ISimulator 2021-08-09 10:04:35 -07:00
negoyal 40b4f3b2f1 Merge branch 'master' into bit-flipping-workload 2021-07-28 18:06:07 -07:00
negoyal 050c218502 New Disk Delay Logic and ChaosMetrics. 2021-07-28 16:03:37 -07:00
Josh Slocum 02f243c821
Merge pull request #5110 from sfc-gh-jslocum/tss_streaming_master
Added Streaming Read comparison for TSS
2021-07-12 10:44:15 -07:00
sfc-gh-tclinkenbeard bbce95da5f Undo changes unrelated to BOOLEAN_PARAM macro 2021-07-09 05:24:42 -07:00
sfc-gh-tclinkenbeard 020371a78f Merge remote-tracking branch 'origin/master' into add-boolean-param 2021-07-07 16:50:51 -07:00
Trevor Clinkenbeard f5ade03538
Merge pull request #4233 from sfc-gh-tclinkenbeard/encrypt-backup-files
Added AsyncFileEncrypted
2021-07-07 13:28:28 -07:00
sfc-gh-tclinkenbeard 8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
Josh Slocum 9f1afbb709 Added Streaming Read comparison for TSS 2021-07-01 19:00:05 -05:00
negoyal df39c5a44e Implement Disk Throttling Chaos workload. 2021-06-30 17:05:04 -07:00
Josh Slocum ab0ab765c8
Merge pull request #4965 from sfc-gh-jslocum/tss_mapping_changes
TSS Mismatch Changes
2021-06-29 08:24:25 -07:00
sfc-gh-tclinkenbeard 81b8292094 Merge remote-tracking branch 'origin' into encrypt-backup-files 2021-06-25 12:21:12 -07:00
Evan Tschannen fcb8bd6475
Revert "Make the sim2 run loop match the behavior of the net2 run loop." 2021-06-22 14:50:01 -07:00
Evan Tschannen 154332a94b Merge branch 'master' of https://github.com/apple/foundationdb into feature-sim-time-batching
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
2021-06-22 09:37:40 -07:00
Josh Slocum 56dadaa428 TSS Mismatch Changes 2021-06-11 23:13:16 +00:00
sfc-gh-tclinkenbeard 669ca0bb14 Add comments to configuration database code 2021-06-10 17:14:37 -07:00
sfc-gh-tclinkenbeard c272304e60 Manage global flow knobs with global knob collection 2021-06-09 22:33:00 -07:00
sfc-gh-tclinkenbeard 83a0e473e8 Refactor IKnobCollection code 2021-06-09 20:50:00 -07:00
Evan Tschannen 08a5f17660 Merge branch 'master' of https://github.com/apple/foundationdb into feature-sim-time-batching
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
2021-06-08 10:04:06 -07:00
sfc-gh-tclinkenbeard 4c059670a0 Parse configuration knob values on the client 2021-06-02 10:04:46 -07:00
sfc-gh-tclinkenbeard a775f92fca Merge remote-tracking branch 'origin/master' into config-db 2021-06-01 15:39:34 -07:00
Josh Slocum 4257ac2b4d More TSS Changes/Fixes 2021-05-25 20:37:48 +00:00
Evan Tschannen f57f0d64f4 Merge branch 'master' into feature-sim-time-batching
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
2021-05-20 09:09:35 -07:00
sfc-gh-tclinkenbeard 01eab20fc0 Remove duplicate Knobs::reset code 2021-05-14 23:02:40 -07:00
sfc-gh-tclinkenbeard 9d0843254f Store all types of knobs in LocalConfiguration 2021-05-11 13:33:39 -07:00
sfc-gh-tclinkenbeard e5d6c5ed17 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-05-03 14:46:19 -07:00
sfc-gh-tclinkenbeard 4d3a7ceb44 Added simple DynamicKnobsWorkload 2021-04-29 16:32:30 -07:00
Steve Atherton 9475b6a5dd Correctness fix, prevent AsyncFileNonDurable from always making file writes take up to 5 seconds. 2021-04-13 20:15:19 -07:00
sfc-gh-tclinkenbeard a0c49234b2 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-03-19 20:47:53 -07:00