Commit Graph

76 Commits

Author SHA1 Message Date
Renxuan Wang 233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
Mohamed Oulmahdi 5b6098b5be Fix Windows build broken by #6449 2022-02-25 16:25:45 +01:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
vikasgupta8 edfff755bf added support for ppc64le 2022-02-11 06:17:15 +00:00
Xiaoxi Wang 6dc5921575
createdTime based storage wiggler (#6219)
* add storagemetadata

* add StorageWiggler;

* fix serverMetadataKey bug

* add metadata tracker in storage tracker

* finish StorageWiggler

* update next storage ID

* change pid to server id

* write metadata when seed SS

* add status json fields

* remove pid based ppw iteration

* fix time expression

* fix tss metadata nonexistence; fix transaction retry when retrieving metadata

* fix checkMetadata bug when store type is wrong

* fix remove storage status json

* format code

* refactor updateNextWigglingStoragePID

* seperate storage metadata tracker and store type tracker

* rename pid

* wiggler stats

* fix completion between waitServerListChange and storageRecruiter

* solve review comments

* rename system key

* fix database lock timeout by adding lock_aware

* format code

* status json

* resolve code format/naming comments

* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>

* fix omit start rount

* format code

* status json reset

* solve status json format

* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions

* status timestamp
2022-02-04 15:04:30 -08:00
Ray Jenkins dd45805312
Merge branch 'apple:main' into threadname-issue-6064 2022-02-01 17:40:07 -06:00
Ata E Husain Bohra 591ef57857
Upgrade AES 128 GCM -> AES 256, StreamCipher code refactor (#6314)
* Upgrade AES 128 GCM -> AES 256, StreamCipher code refactor

Major changes proposed are:
1. Refactor StreamCipher code to enable instantiation of
   multiple encryption keys. However, code still retains
   a globalEncryption key semantics used in Backup file
   encryption usecase.
2. Enhance StreamCipher to provide HMAC signature digest
   generation. Further, the class implements HMAC encryption
   key derivation function.
3. Upgrade StreamCipher to use AES 256 GCM mode from currently
   supported AES 128 GCM mode.
   Note: The code changes the encryption key size, however, the
         feature is NOT currently in use, hence, should be OK.
3. Add EncryptionOps validation and benchmark toml supported
   workload, it does the following:
   a. Allow user to configure encrypt-decrypt of a fixed size
      buffer or variable size buffer [100, 512K]
   b. Allow user to configure number of interactions of the runs,
      in each iteration: generate random data, derive an encryption
      key using HMAC SHA256 method, encrypt data and
      then decrypt data. It collects following metrics:
    i) time taken to derive encryption key.
    ii) time taken to encrypt the buffer.
    iii) time taken to decrypt the buffer.
    iv) total bytes encrypted and/or decrypted
   c. Along with stats it basic basic validations on the encrypted
      and decrypted buffer
   d. On completion for test, records the above mentioned metrics
      in trace files.
2022-01-31 19:52:44 -06:00
Ray Jenkins 6a1fe2489a format fixes 2022-01-28 18:21:42 -06:00
Ray Jenkins ef4e61ee33 Use GetLastError to include errno data in trace logs. 2022-01-28 17:27:41 -06:00
Ray Jenkins f9f0fb0781 Better error handling for pthread_setname_np.
In unit and simulation testing calls to pthread_setname_np may return errors,
as the threads may complete before calls to setname can be executed. This change
adds better error handling for cases where ENOENT or ESRCH is returned during testing.
Previously the ASSERT_EQ would cause tests to fail if a non-zero return value was encountered.

This change will trace log with a SevWarn when ENOENT or ESRCH is encountered. Otherwise
it will trace with SevError and throw a platform_error.
2022-01-28 16:18:22 -06:00
Renxuan Wang 4a8e2a80e6 Improve/fix disk metrics.
1. Introduce processDiskReadSeconds and processDiskWriteSeconds, which stands for disk read/write times `since the last logging`. They can only be obtained on Linux and macOS, and will be 0 on Windows and FreeBSD;
2. Rename `busyTicks` to `IOMilliSecs`;
3. On FreeBSD, the metrics should be collected among all devices.
2022-01-27 14:40:32 -08:00
Ray Jenkins 0d34ec0880 Add RunLoopProfiler thread name. 2022-01-25 13:22:22 -06:00
He Liu 2fb5c59440 Removed deprioritizeThread(). 2022-01-18 11:10:56 -08:00
He Liu 2c0c51dd6d Enabled setting thread poll priorities. 2022-01-10 17:50:56 -08:00
A.J. Beamon 264c75b9a6 Add some extra client logging details:
1. Add a trace event when a database is created and move the cluster file / connection string from ClientStart to the new trace event
2. Add a detail for the path to the image being loaded
3. Add a detail for whether a client library is primary or not
4. Set a thread name for each external client thread that includes the release version
2021-11-29 09:57:10 -08:00
A.J. Beamon 6cf731d5aa Add comment to run loop blocked message about timestamp order 2021-10-19 10:05:29 -07:00
A.J. Beamon abab45760d Add some additional logging if the network thread finishes, fails with an error, gets stopped, or is blocked. 2021-10-19 10:05:29 -07:00
Xiaoge Su e68b131e4a fixup! Reformat source code 2021-09-16 19:40:28 -07:00
Xiaoge Su abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
Lukas Joswiak 5dc9a97230 Merge branch 'master' into fixes/alp6 2021-08-01 20:42:52 -07:00
sfc-gh-tclinkenbeard c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Lukas Joswiak 3eed4084e2 Merge branch 'master' into fixes/alp6 2021-07-27 11:26:53 -07:00
Lukas Joswiak 59d535149e Merge branch 'master' into fixes/alp6 2021-07-27 10:07:18 -07:00
Steve Atherton 507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
sfc-gh-tclinkenbeard e006e4fed4 Fix -Wreorder-ctor warnings in LogSystemPeekCursor.actor.cpp and several other files 2021-07-24 00:48:13 -07:00
sfc-gh-tclinkenbeard 64dc1dc185 Fix -Wreorder-ctor warnings in NativeAPI.actor.cpp and several other files 2021-07-24 00:23:06 -07:00
sfc-gh-tclinkenbeard b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Trevor Clinkenbeard f5ade03538
Merge pull request #4233 from sfc-gh-tclinkenbeard/encrypt-backup-files
Added AsyncFileEncrypted
2021-07-07 13:28:28 -07:00
sfc-gh-tclinkenbeard a73570e538 Added comment for crashHandler 2021-06-29 11:00:05 -07:00
sfc-gh-tclinkenbeard 8855c7ee8d Set error kind to BugDetected for Crash trace events 2021-06-27 16:47:54 -07:00
sfc-gh-tclinkenbeard 81b8292094 Merge remote-tracking branch 'origin' into encrypt-backup-files 2021-06-25 12:21:12 -07:00
Trevor Clinkenbeard 0120a6ba72
Merge pull request #4936 from sfc-gh-tclinkenbeard/remove-string-copies
Remove unnecessary std::string copies from flow
2021-06-14 13:49:20 -07:00
sfc-gh-tclinkenbeard 399c2c96f0 Remove unnecessary std::string copies from flow 2021-06-09 11:40:01 -07:00
sfc-gh-tclinkenbeard 371a38e6e5 Merge remote-tracking branch 'origin/master' into remove-extra-copies 2021-06-07 10:26:06 -07:00
Lukas Joswiak 153de33f57 Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
This reverts commit 6499fa178e, reversing
changes made to 1512631957.
2021-06-04 13:31:55 -07:00
Lukas Joswiak 4ea760b2a9 Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
This reverts commit da41534618, reversing
changes made to e6300905d6.
2021-05-10 20:26:12 -07:00
sfc-gh-tclinkenbeard f28ac955c3 Remove unnecessary temporary objects while growing objects of type std::vector<std::pair<A, B>> 2021-05-10 16:32:50 -07:00
sfc-gh-tclinkenbeard e5d6c5ed17 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-05-03 14:46:19 -07:00
Markus Pilman 09ddcb3bae remove old sample thread 2021-04-19 11:55:35 -06:00
Lukas Joswiak 551268b0f2 Add well known endpoint for worker communication 2021-04-15 13:50:50 -07:00
Markus Pilman 2efcf8efec Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-serialization 2021-04-12 09:34:03 -06:00
Markus Pilman 2064903705 collect and serialize 2021-04-09 14:25:11 -06:00
Lukas Joswiak 83cf965875 Add global variable to fetch each type of sample 2021-04-07 15:38:01 -07:00
Lukas Joswiak d6c4aa67d7 Sample actors waiting on network 2021-04-07 13:27:31 -07:00
Lukas Joswiak c90be2003f Profile running actor 2021-04-01 10:34:59 -07:00
Lukas Joswiak f2d3687110 Print stack 2021-03-29 16:06:26 -07:00
Lukas Joswiak 2dfd420882 Add sampling profiler thread 2021-03-24 14:52:42 -07:00
sfc-gh-tclinkenbeard a0c49234b2 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-03-19 20:47:53 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00