Commit Graph

128 Commits

Author SHA1 Message Date
Markus Pilman 219a0b98d7 Add documentation 2021-03-23 10:51:16 -06:00
Markus Pilman 0ba99c8e83 Pretty-print topology when starting simulation 2021-03-23 09:38:07 -06:00
Vishesh Yadav 2c316db94d Merge remote-tracking branch 'apple/master' into master-format-final
Also format the changes
2021-03-11 12:54:26 -08:00
sfc-gh-ngoyal 800c37bbb3
Merge pull request #4434 from sfc-gh-clin/cache-correctness
Add consistencycheck for cache in consistencycheck workload
2021-03-11 12:00:18 -08:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Chaoguang Lin 425531060e Add consistencycheck for cache in consistencycheck workload 2021-03-04 14:30:35 -08:00
Andrew Noyes 79cec09255 Apply clang-tidy's performance-inefficient-vector-operation fix
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.

    $ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
Andrew Noyes 4ee97c0784 Use clang-tidy to automatically fix missing overrides
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.

Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.

- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 5bfa6cea98 Merge remote-tracking branch 'origin/master' into misc-changes 2020-12-26 20:47:00 -04:00
sfc-gh-tclinkenbeard f3c0d26806 Make ISimulator::BackupAgentType an enum class 2020-12-08 09:09:30 -08:00
Richard Chen c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Richard Chen 9c6c2704d0 Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces 2020-10-30 23:59:32 +00:00
sfc-gh-tclinkenbeard ff07dd5591 Add tracing to runWorkloadAsync for check phase 2020-10-21 20:23:19 -07:00
Richard Chen 545ee4269d master conflicts 2020-10-19 01:03:54 +00:00
sfc-gh-tclinkenbeard c7694e7b50 Merge remote-tracking branch 'origin/master' into refactor-workloads 2020-10-14 09:21:47 -07:00
Richard Chen 2f5b0bef08 switch to test newer incompatible version. Fix PR comments. Modify schema 2020-10-12 18:29:16 +00:00
Richard Chen bbf5bdf6da fix stable interfaces test and corresponding changes in simulator 2020-10-12 18:25:12 +00:00
Richard Chen 5488ff1d81 draft diff protocol 2020-10-12 18:24:03 +00:00
sfc-gh-tclinkenbeard 012886a8d8 Fix Windows CMake build 2020-10-07 12:30:19 -07:00
Jon Fu c0f857cd64 attempt to have logAntiQuorum as an option preset prior to starting simulation and running tests 2020-10-06 14:47:13 -04:00
sfc-gh-tclinkenbeard 7f0d14c8e4 Modernize/refactor workloads directory 2020-10-04 22:29:07 -07:00
Alex Miller eb7ad07df0 Final cleanups. 2020-07-12 19:53:44 -07:00
Alex Miller 2876763623 Add errors if unknown params are used in a global or test section. 2020-07-12 19:41:09 -07:00
Alex Miller 0131f0dc48 Make simulation able to load and run .toml files 2020-07-12 14:47:39 -07:00
Alex Miller 464e9332e4 Refactor testspec parsing.
This separates the file format and how to read it from how to apply a key/value
to a TestSpec.  This will allow us to reuse the same code when implementing a
TOML parser later.
2020-07-12 14:47:39 -07:00
Alex Miller 17570b5b10 Make the testspec more restrictive in terms of what can be set where.
Testspec is currently very permissive in very misleading ways.  In particular,
the tester parser itself will swallow K=V settings and apply them at the test
level, which breaks how a person would expect the scoping to work.  Other
settings apply to the entire simulation run globally, but appear to be workload
specific.  Even further, others affect simulation cluster creation or test
harness behavior, but can again be set anywhere in a testspec.

This changes testspec parsing to error if a setting that applies globally is
anywhere but the top of the file, or if a setting that applies test-wide is
applied to a workload instead of a test.
2020-07-06 02:03:30 -07:00
Markus Pilman 5f9b127e56 Emit traces regularly about role assignment
We are currently emitting Role transition traces when a role starts and
when it ends. While this is useful for debugging, it doesn't work well
with tools that inject data and might potentially miss some trace lines.

We do decorate each trace lines with the roles assigned to that
particular process, however, this is not sufficient for tools that can
make use of the UID -> Role mapping
2020-05-08 16:27:57 -07:00
Markus Pilman 20d6f2a3c3 Merge remote-tracking branch 'origin/master' into bugfixes/fix-clang10-compiler-warnings 2020-05-06 15:50:39 -07:00
Markus Pilman f967209ea0 Fix compiler warnings with clang 2020-04-22 14:14:56 -07:00
Evan Tschannen 1476057996 properly cache serialization of serverDBInfo 2020-04-11 19:30:05 -07:00
Evan Tschannen 07cc0a8d74 code cleanup 2020-04-10 17:02:11 -07:00
Evan Tschannen 2a1bd97120 fix compilation errors 2020-04-06 20:58:43 -07:00
Xin Dong 0b0414fb94 Addressded review comments. Change the issue reporting from 'ITraceLogWriter' to be a more generic way. 2020-02-25 15:37:53 -08:00
Xin Dong 034dfe5e42 Now the inability to flush trace logs will be reported to both 'stderr' and also the status json object.
- Since the first flush failure, if the accumulated consecutive failure count exceeds the value defined in knobs, it will trigger the current worker process to report this issue via the 'GetServerDBInfo' interface of the cluster controler
    - A successful flush will reset the accumulated counter.
    Notice that the current solution does not take the time into consideration. The assumption is that flush failures tend to only happen in a clustered manner. The intermittent, but short, periods of flush failures are not considered as a problem since the memory pressure built by them should be negligible.
2020-02-25 15:37:32 -08:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Andrew Noyes 17660fb18d Fix simulation test 2020-02-11 13:49:19 -08:00
Meng Xu 3b57bf1781 Merge branch 'master' into mengxu/fast-restore-agent-PR 2020-02-03 17:23:54 -08:00
Meng Xu 559b95c61a FastRestore:RestoreRole:Mimic how fdbd starts 2020-02-01 10:23:48 -08:00
Vishesh Yadav 6e6cfaff16 Cleanup old Failure Monitoring code 2020-01-07 15:53:32 -08:00
tclinken 8f84fbc4b9 Only print 'waiting for DD to end...' if test actually waits 2019-11-03 16:13:32 -08:00
Evan Tschannen 58b984b846
Merge pull request #2047 from tclinken/lock-aware-db-ping
Use lock aware transaction for pingDatabase
2019-10-31 10:24:20 -07:00
Meng Xu 0b785e5c1c DD:getTeam may fail to get a team when it can
Due to randomness, when unhealthy teams are majority while there still
exists healthy teams, getTeam function may be unlucky to find
any feasible (ok) team, which leads to BestTeamStuck situation.

This commit increases the tries from 10 to 20.

A long-term solution may first find all feasible teams and choose a random
one from them. Since This can affect the statistics of which team is picked.
So it is not included in this commit.

Non-functional change: This commit removes unneeded printf introduced by
fast restore PR 1404.
2019-09-07 20:08:58 -07:00
Trevor Clinkenbeard 2f92cf8c96 Use lock aware transaction for pingDatabase 2019-09-07 12:25:44 -07:00
Meng Xu c2355f721e Merge branch 'master' into mengxu/performant-restore-PR 2019-09-04 17:11:42 -07:00
Andrew Noyes 6aa0ada7b1 Replace scalar root types with proper messages 2019-08-28 14:40:50 -07:00
Meng Xu 3b54363780 FastRestore:Apply Clang-format 2019-08-01 18:09:12 -07:00
Meng Xu 7ccaeddf05 Merge branch 'master' into mengxu/performant-restore-PR 2019-08-01 13:23:17 -07:00
Evan Tschannen 90e3b50213 Merge branch 'master' into feature-coordinator-connection
# Conflicts:
#	fdbclient/DatabaseContext.h
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/workloads/KillRegion.actor.cpp
2019-07-26 15:05:02 -07:00
Meng Xu 45083edf74 Merge branch 'master' into mengxu/performant-restore-PR
Fix conflicts as well.
2019-07-25 10:46:11 -07:00