Commit Graph

138 Commits

Author SHA1 Message Date
Josh Slocum 4257ac2b4d More TSS Changes/Fixes 2021-05-25 20:37:48 +00:00
Josh Slocum ce82c9653e Testing Storage Server implementation 2021-05-25 20:28:50 +00:00
Markus Pilman 64608fe86b allow simulation properties to be overwritten 2021-05-20 13:48:41 -06:00
Markus Pilman a445e09c2d
Merge pull request #4617 from sfc-gh-satherton/unit-test-params
Added UnitTest parameters concept and ability to run unit tests directly from command line, with parameters
2021-04-15 14:59:15 -06:00
Jon Fu 2a64c227fb Added options to test config that specify maxtlogversion and array of excluded storage engine types 2021-04-07 15:59:51 -04:00
Steve Atherton b4e42476b7 Unit test parameters are no longer global, they are accessible via a parameter to the unit test and initialized from otherwise unconsumed test options for the UnitTests workload in the test spec or from the fdbserver command line when using the unittests role. 2021-04-06 02:36:10 -07:00
Steve Atherton eec119e0d0 Added an fdbserver role to run unit tests directly without a cluster or test spec file, and added a unit test parameters concept for passing options into unit tests.
Updated p2p network test to use unit test parameters instead of the environment.
2021-04-04 23:27:34 -07:00
Trevor Clinkenbeard 5cbb6421d2
Merge pull request #4573 from sfc-gh-jfu/jfu-test-parser
Updated test parser to read new option correctly
2021-03-25 20:25:51 -07:00
Jon Fu 1541f4234a Updated test parser with new option and also added option to test file with default value 2021-03-25 14:40:06 -04:00
Markus Pilman af869ac265 minor formatting fix 2021-03-23 10:59:01 -06:00
Markus Pilman 219a0b98d7 Add documentation 2021-03-23 10:51:16 -06:00
Markus Pilman 0ba99c8e83 Pretty-print topology when starting simulation 2021-03-23 09:38:07 -06:00
Vishesh Yadav 2c316db94d Merge remote-tracking branch 'apple/master' into master-format-final
Also format the changes
2021-03-11 12:54:26 -08:00
sfc-gh-ngoyal 800c37bbb3
Merge pull request #4434 from sfc-gh-clin/cache-correctness
Add consistencycheck for cache in consistencycheck workload
2021-03-11 12:00:18 -08:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Chaoguang Lin 425531060e Add consistencycheck for cache in consistencycheck workload 2021-03-04 14:30:35 -08:00
Andrew Noyes 79cec09255 Apply clang-tidy's performance-inefficient-vector-operation fix
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.

    $ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
Andrew Noyes 4ee97c0784 Use clang-tidy to automatically fix missing overrides
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.

Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.

- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 5bfa6cea98 Merge remote-tracking branch 'origin/master' into misc-changes 2020-12-26 20:47:00 -04:00
sfc-gh-tclinkenbeard f3c0d26806 Make ISimulator::BackupAgentType an enum class 2020-12-08 09:09:30 -08:00
Richard Chen c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Richard Chen 9c6c2704d0 Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces 2020-10-30 23:59:32 +00:00
sfc-gh-tclinkenbeard ff07dd5591 Add tracing to runWorkloadAsync for check phase 2020-10-21 20:23:19 -07:00
Richard Chen 545ee4269d master conflicts 2020-10-19 01:03:54 +00:00
sfc-gh-tclinkenbeard c7694e7b50 Merge remote-tracking branch 'origin/master' into refactor-workloads 2020-10-14 09:21:47 -07:00
Richard Chen 2f5b0bef08 switch to test newer incompatible version. Fix PR comments. Modify schema 2020-10-12 18:29:16 +00:00
Richard Chen bbf5bdf6da fix stable interfaces test and corresponding changes in simulator 2020-10-12 18:25:12 +00:00
Richard Chen 5488ff1d81 draft diff protocol 2020-10-12 18:24:03 +00:00
sfc-gh-tclinkenbeard 012886a8d8 Fix Windows CMake build 2020-10-07 12:30:19 -07:00
Jon Fu c0f857cd64 attempt to have logAntiQuorum as an option preset prior to starting simulation and running tests 2020-10-06 14:47:13 -04:00
sfc-gh-tclinkenbeard 7f0d14c8e4 Modernize/refactor workloads directory 2020-10-04 22:29:07 -07:00
Alex Miller eb7ad07df0 Final cleanups. 2020-07-12 19:53:44 -07:00
Alex Miller 2876763623 Add errors if unknown params are used in a global or test section. 2020-07-12 19:41:09 -07:00
Alex Miller 0131f0dc48 Make simulation able to load and run .toml files 2020-07-12 14:47:39 -07:00
Alex Miller 464e9332e4 Refactor testspec parsing.
This separates the file format and how to read it from how to apply a key/value
to a TestSpec.  This will allow us to reuse the same code when implementing a
TOML parser later.
2020-07-12 14:47:39 -07:00
Alex Miller 17570b5b10 Make the testspec more restrictive in terms of what can be set where.
Testspec is currently very permissive in very misleading ways.  In particular,
the tester parser itself will swallow K=V settings and apply them at the test
level, which breaks how a person would expect the scoping to work.  Other
settings apply to the entire simulation run globally, but appear to be workload
specific.  Even further, others affect simulation cluster creation or test
harness behavior, but can again be set anywhere in a testspec.

This changes testspec parsing to error if a setting that applies globally is
anywhere but the top of the file, or if a setting that applies test-wide is
applied to a workload instead of a test.
2020-07-06 02:03:30 -07:00
Markus Pilman 5f9b127e56 Emit traces regularly about role assignment
We are currently emitting Role transition traces when a role starts and
when it ends. While this is useful for debugging, it doesn't work well
with tools that inject data and might potentially miss some trace lines.

We do decorate each trace lines with the roles assigned to that
particular process, however, this is not sufficient for tools that can
make use of the UID -> Role mapping
2020-05-08 16:27:57 -07:00
Markus Pilman 20d6f2a3c3 Merge remote-tracking branch 'origin/master' into bugfixes/fix-clang10-compiler-warnings 2020-05-06 15:50:39 -07:00
Markus Pilman f967209ea0 Fix compiler warnings with clang 2020-04-22 14:14:56 -07:00
Evan Tschannen 1476057996 properly cache serialization of serverDBInfo 2020-04-11 19:30:05 -07:00
Evan Tschannen 07cc0a8d74 code cleanup 2020-04-10 17:02:11 -07:00
Evan Tschannen 2a1bd97120 fix compilation errors 2020-04-06 20:58:43 -07:00
Xin Dong 0b0414fb94 Addressded review comments. Change the issue reporting from 'ITraceLogWriter' to be a more generic way. 2020-02-25 15:37:53 -08:00
Xin Dong 034dfe5e42 Now the inability to flush trace logs will be reported to both 'stderr' and also the status json object.
- Since the first flush failure, if the accumulated consecutive failure count exceeds the value defined in knobs, it will trigger the current worker process to report this issue via the 'GetServerDBInfo' interface of the cluster controler
    - A successful flush will reset the accumulated counter.
    Notice that the current solution does not take the time into consideration. The assumption is that flush failures tend to only happen in a clustered manner. The intermittent, but short, periods of flush failures are not considered as a problem since the memory pressure built by them should be negligible.
2020-02-25 15:37:32 -08:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Andrew Noyes 17660fb18d Fix simulation test 2020-02-11 13:49:19 -08:00
Meng Xu 3b57bf1781 Merge branch 'master' into mengxu/fast-restore-agent-PR 2020-02-03 17:23:54 -08:00
Meng Xu 559b95c61a FastRestore:RestoreRole:Mimic how fdbd starts 2020-02-01 10:23:48 -08:00
Vishesh Yadav 6e6cfaff16 Cleanup old Failure Monitoring code 2020-01-07 15:53:32 -08:00