Commit Graph

446 Commits

Author SHA1 Message Date
Renxuan Wang 13b2b455b1 HTTP proxy for backup.
Only support non-TLS blobs now.
2022-05-09 13:56:33 -07:00
sfc-gh-tclinkenbeard 06825775db Fix formatting of lines with TLS_OPTION_FLAGS 2022-05-02 22:56:06 -07:00
sfc-gh-tclinkenbeard 7f05221cfe Removed TLS_DISABLED macro 2022-05-02 22:15:27 -07:00
Yi Wu 994b8c92f8
Add option to limit resident memory and remove default memory limit (#6719)
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.

Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.

closes #6671, closes #6672
2022-04-06 20:06:24 -07:00
Mohamed Oulmahdi 5dc07c55f7 Add missing Windows headers 2022-04-06 14:43:59 +02:00
Jingyu Zhou dcfb6ee520
Added an option to fdbdecode for saving processed files and fixed a memory corruption issue in clang build (#6767)
* Add save file locally feature to fdbdecode

* Fix use-after-free in fdbdecode

This only happens in clang, though.

* Minor refactor for fdbdecode

* Fix a small memory leak in fdbdecode
2022-04-05 16:16:59 -07:00
Jingyu Zhou f68fd28d73 Refactor duplicated code into IKnobCollection::setupKnobs() 2022-04-05 02:06:38 -07:00
Renxuan Wang e1775627ab Add a check on proxy format. 2022-03-29 10:17:14 -07:00
Renxuan Wang 0a332ee1c1 Add proxy option to backup and restore params. 2022-03-28 17:10:49 -07:00
Jingyu Zhou 2532ac5882 Add knob value changes to fdbdecode
E.g., HTTP_VERBOSE_LEVEL, and HTTP_REQUEST_AWS_V4_HEADER knobs.
2022-03-24 09:26:41 -07:00
Jingyu Zhou cff1bc24fc Fix decoder for missing ClearRanges
If the ClearRange mutation happens within the given key prefix space,
previously the mutation is not printed. Fix by checking the prefix range
overlaps with the ClearRange mutation.
2022-03-22 16:04:56 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
Jingyu Zhou 1a5bf25b5c Update code base to use fmt 8.1.1 2022-03-04 15:52:06 -08:00
Renxuan Wang 233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
sfc-gh-tclinkenbeard ef711d405c Add fdbbackup tags command 2022-01-24 13:34:54 -08:00
A.J. Beamon ff1cb58174 Convert hyphens to underscores for all prefix-based arguments (e.g. --knob-, --locality-) 2021-12-14 12:01:44 -08:00
A.J. Beamon f24adc7b6a Fix a bunch of places where we used old-style arguments. Allow hyphens for profiler args. 2021-12-14 09:59:14 -08:00
A.J. Beamon f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
sfc-gh-tclinkenbeard 464d9488ef Merge remote-tracking branch 'origin/master' into fix-unused-warnings 2021-12-01 23:52:09 -08:00
sfc-gh-tclinkenbeard 90ced244eb Fix -Wunused-but-set-variable warnings 2021-12-01 18:15:53 -08:00
sfc-gh-tclinkenbeard ec64890ac1 Remove some usages of PRId64 by using fmt library 2021-11-30 23:35:36 -08:00
sfc-gh-tclinkenbeard deb0eb85e8 Initialize global flow knobs in setupNetwork 2021-11-16 00:41:34 -08:00
A.J. Beamon e882eb33fc Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem. 2021-10-22 11:05:18 -07:00
Xiaoge Su e68b131e4a fixup! Reformat source code 2021-09-16 19:40:28 -07:00
Xiaoge Su abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
FDB Formatster 2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
A.J. Beamon d4c67eb13d Fix crash when DR binaries are invoked without a source cluster file. Update the usage text for dr_agent cluster files to include the long form options. 2021-08-20 12:55:13 -07:00
Steve Atherton e5e8a56b66 Merge branch 'master' of https://github.com/apple/foundationdb into debugging-cleanup
# Conflicts:
#	fdbbackup/FileDecoder.actor.cpp
2021-08-09 23:36:57 -07:00
Steve Atherton 2bf7b9b7a9 Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang. 2021-08-08 00:04:54 -07:00
Vishesh Yadav de93e2f854
Update fdbbackup/backup.actor.cpp
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2021-08-04 20:27:43 -07:00
Vishesh Yadav 98fa70da11 backup: Show error info for RemovedDeadBackupLayerStatus 2021-08-04 13:28:47 -07:00
Jingyu Zhou 5346213e7f Fix trace event UnfishedBlocks 2021-08-02 09:26:42 -07:00
Jingyu Zhou 31a64106fb Avoid memory copies after decoding a new file block 2021-08-02 09:21:33 -07:00
Jingyu Zhou 426e906a87 Remove duplicated code 2021-08-01 22:39:28 -07:00
Jingyu Zhou 8f1946f522 Remove unused code 2021-08-01 21:55:39 -07:00
Jingyu Zhou 1a97c3942a Fix trace events 2021-08-01 19:25:39 -07:00
Jingyu Zhou 4f5bcaf050 Remove verbose stdout messages 2021-08-01 19:11:23 -07:00
Jingyu Zhou e3629ef356 Refactor decoder 2021-08-01 19:09:09 -07:00
Jingyu Zhou d7ac5830e6 Fix decoder bug of shadowing loop index 2021-08-01 13:48:45 -07:00
Jingyu Zhou de1e9000a2 Add --hex_prefix flag 2021-07-31 21:08:13 -07:00
Jingyu Zhou 2282d8455a Fix trace format json 2021-07-31 19:02:33 -07:00
Meng Xu 9e4bee9e30 Handle ascii special characters 2021-08-01 01:50:33 +00:00
Jingyu Zhou b930faf520 Log prefix filter 2021-07-31 12:52:39 -07:00
Jingyu Zhou 1145087543 Add UID to the events for decoders 2021-07-31 12:45:47 -07:00
Jingyu Zhou 0a98c76747 Fix trace event throttling by using different types 2021-07-31 12:24:59 -07:00
Jingyu Zhou 7054bf7054 Set max field length for long mutations 2021-07-31 11:28:13 -07:00
Jingyu Zhou 0ea432a638 Add traceevent for all printed mutations 2021-07-30 18:34:13 -07:00