Commit Graph

18793 Commits

Author SHA1 Message Date
Steve Atherton 73a81a86da Added Redwood metrics printing during set unit test using new generic repeatEvery() actor. 2022-02-26 23:37:30 -08:00
Steve Atherton 3a666610ad Tweaked Redwood unit test ops limits and added total record read count to prevent test instances that run too long. 2022-02-26 23:36:37 -08:00
Steve Atherton 6f16702042 Merge branch 'main' of https://github.com/apple/foundationdb into redwood-bug-fixes-and-memory-leak 2022-02-26 18:00:18 -08:00
sfc-gh-tclinkenbeard 90927cffe9 Set disableHostname=true in CycleTestRestart downgrade test 2022-02-26 17:56:56 -08:00
Steve Atherton c122cf1ce0 Bug fix: Avoid having a local temporary Cursor destructed after the Reference<Page> containing its DecodeCache has been dropped by using the BTreeCursor's leaf cursor by reference. 2022-02-26 02:03:43 -08:00
Steve Atherton a47f481bae Bug fix, the extent cache used during recovery is not allowed to evict anything so it now uses a private Evictor with no size limit. 2022-02-25 19:21:11 -08:00
Renxuan Wang 233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
Markus Pilman 079de5ba57 Enable rocksdb unit tests 2022-02-25 15:38:16 -08:00
Markus Pilman a61dd41b45 Add an empty test to make sure noSim tests succeed 2022-02-25 15:38:16 -08:00
Vaidas Gasiunas 3967f9ed14 ApiTester: Introduce workload manager 2022-02-26 00:09:37 +01:00
Vaidas Gasiunas c3e6eea41a ApiTester: In memory key-value store 2022-02-26 00:08:21 +01:00
Andrew Noyes 276cf3a402
Fix minor bugs turned up by _GLIBCXX_DEBUG (#6375)
* Fix a benign bug turned up by _GLIBCXX_DEBUG

Just calling std::vector::operator[] with an out of bounds index at all
is technically undefined behavior.

* Fix compilation issue with _GLIBCXX_DEBUG

For some reason std::max with an initializer list isn't constexpr with
_GLIBCXX_DEBUG set
2022-02-25 13:50:41 -08:00
Andrew Noyes b58739093e
Account for possibility that fopen might fail in mako (#6442)
Fix an issue in mako where the stats process would hang if the worker process
was unable to open a file. Previously, the worker process would segfault when
trying to dereference a null FILE* and never increment the stopcount variable
in shared memory. Now we check for a null FILE* and increment stopcount when
the worker process stops regardless of success or failure.

Also don't ignore the return value from mkdir
2022-02-25 13:18:59 -08:00
Andrew Noyes 7a9217a392
Add contrib/debug_determinism (#6389)
* Add contrib/debug_determinism

Add an instrumentation-based technique for debugging unseen mismatches. Also guard a few existing sources of nondeterminism that don't affect unseen with the DEBUG_DETERMINISM macro.

Also change the simulated run loop to not run as the only task inside the real run loop, since that was a source of nondeterminism.

Also fix nondeterminism from calling timer_int

* Add StorageMetadataType::currentTime

Basically a deterministic-in-simulation version of timer_int that we can
use instead of timer_int for StorageMetadataType::createdTime
2022-02-25 12:54:31 -08:00
Zhe Wu 57e931b489 Update 6.3.24 release notes 2022-02-25 12:12:17 -08:00
A.J. Beamon d4e6123971
Merge pull request #6457 from Doxense/fix-6449-windows-build
Fix Windows build broken by #6449
2022-02-25 09:07:20 -08:00
Mohamed Oulmahdi 5b6098b5be Fix Windows build broken by #6449 2022-02-25 16:25:45 +01:00
Steve Atherton a32c3a2891 Redwood now uses one global page cache eviction order and size limit for all page sizes, for real or virtual processes, and DecodeCache memory is counted against the global page cache budget. 2022-02-25 03:20:53 -08:00
Vaidas Gasiunas 64873b6873 ApiTester: Rename files and namespace 2022-02-25 11:35:57 +01:00
Renxuan Wang 06b1d06d38 Support hostname in coordinators commands. 2022-02-24 23:02:29 -08:00
Renxuan Wang 70d723f70c Move the construction of ConnectionString in changeQuorumChecker() to coordinatorsCommitActor(). 2022-02-24 23:02:29 -08:00
Renxuan Wang 3e761aef3b Add resetConnectionString() function to ClusterConnectionString. 2022-02-24 23:02:29 -08:00
Renxuan Wang 3a3b44efe2 Fix macOS CI broken by #6449.
foundationdb/fdbserver/CoroFlow.actor.cpp:262:8: error: no member named 'errorUnsuppressed' in 'BaseTraceEvent'
2022-02-24 20:26:13 -08:00
Zhe Wu 0bdae35a2c Address comments and move updateFailedEndpointRefreshTime 2022-02-24 16:35:30 -08:00
Zhe Wu 9c99c59b0c Fix get key location overloading proxies 2022-02-24 16:35:30 -08:00
A.J. Beamon 7275e312d8
Merge pull request #6449 from sfc-gh-ajbeamon/trace-enforce-suppression-order
Enforce that trace event suppression calls happen first when using trace event call chaining
2022-02-24 14:39:08 -08:00
A.J. Beamon d3be393246 Add comments to BaseTraceEvent and TraceEvent. Also make BaseTraceEvent contstructor protected. 2022-02-24 13:29:44 -08:00
A.J. Beamon e7d2abd764 Fix formatting issue 2022-02-24 12:28:13 -08:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Aaron Molitor 4ff60db82b Revert "Use full version of fdb to support incompatible release candidates"
This reverts commit f379cf4d05.
2022-02-24 11:55:48 -06:00
Leonidas Tsampros f379cf4d05 Use full version of fdb to support incompatible release candidates 2022-02-24 11:03:42 -06:00
Zhe Wang 541fc171f7
Merge pull request #6409 from kakaiu/log-thread-local-perfcontext-counter
Add RocksDB PerfContextMetrics
2022-02-24 08:43:24 -05:00
Markus Pilman f1189e5947
Merge pull request #6384 from vikasgupta8/ppc64le_support
added support for ppc64le
2022-02-24 14:31:31 +01:00
Vaidas Gasiunas de46144af7 SysTester: Use reference counting for transactions and futures; Define transaction workflow by lambdas 2022-02-24 13:59:12 +01:00
Zhe Wang f14e08a991 addRocksDBPerfContextMetrics 2022-02-23 22:29:07 -05:00
Jon Fu ffbe10696a updated generated go bindings 2022-02-23 17:59:37 -05:00
Jon Fu 9ab21fe873 remove unused knob 2022-02-23 13:49:53 -05:00
Bharadwaj V.R 8cb554a006 Fix formatting issues in DDTeamCollection.actor.cpp 2022-02-23 10:21:34 -08:00
Bharadwaj V.R 102dcb30c6
Merge branch 'main' into fdb#4271 2022-02-23 10:04:19 -08:00
Jon Fu ce1d71472d Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache 2022-02-23 12:35:25 -05:00
Vaidas Gasiunas b61adc10be SysTester: adding missing includes 2022-02-23 17:51:58 +01:00
Vaidas Gasiunas 2fb8d6ac97 SysTester: use unique_ptr instead of plain pointers 2022-02-23 16:51:48 +01:00
Vaidas Gasiunas e4311ae661 SysTester: remove direct dependencies on flow 2022-02-23 16:38:35 +01:00
Vaidas Gasiunas f5d722b65b SysTester: use boost io_context for scheduling 2022-02-23 15:43:01 +01:00
Ata E Husain Bohra 344a14b010
Update StreamCipher ctx/cipher management to respect determinism (#6427)
* Update StreamCipher ctx/cipher management to respect determinism

StreamCipher keeps record of CipherKeys created
(including globalCipherKey) to ensure the sensitive data gets
ZERO-OUT and not recorded as part of FDB process dump. However,
in current code it is maintained as an unordered_set indexed
by the object itself. Approach adds non determinism given
object pointer based indexing scheme.

Patch addresses the concern by updating the recording to use
a map indexed by UID.
2022-02-22 17:34:29 -08:00
Renxuan Wang 3c1394578b Address comments. 2022-02-22 16:29:59 -08:00
Renxuan Wang ea8b2bd314 Resolve hostnames synchronously when calling actor is impossible. 2022-02-22 16:29:59 -08:00
Renxuan Wang 622d89b552 Rebase on main.
Since we changed ClusterConnectionString's status flag from boolean to enum in #6422, we need to update this PR correspondingly.
2022-02-22 16:29:59 -08:00
Renxuan Wang 8eb7a10404 Address comments. 2022-02-22 16:29:59 -08:00