Commit Graph

43 Commits

Author SHA1 Message Date
Meng Xu 477fd152c0 FastRestore:Refactor code
1) Use the runRYWTransaction for simple DB access
2) Replace some printf with TraceEvent
3) Remove printf not used in debugging
4) Avoid wait inside the condition in loop-choose-when for
   the core routine of restore worker, loader and applier.
5) Rename Restore.actor.cpp to RestoreWorker.actor.cpp since
   the file only has functionalities related to restore worker.

Passed correctness test
2019-06-04 11:22:47 -07:00
Meng Xu fac63a83c4 FastRestore:Use NotifiedVersion to deduplicate requests
Add a NotifiedVersion into an applier data which represents
the smallest version the applier is at.

When a loader sends mutation vector to appliers, it sends
the request that contains prevVersion and commitVersion.

This commits also put actor into an actorCollector for
loop-choose-when situation.
2019-05-22 22:09:54 -07:00
Meng Xu 54f4df604a CMake:Add FastRestore files 2019-05-16 19:43:19 -07:00
Meng Xu 25c75f4222 FastRestore: Add new empty files for restore roles
Add .h and .cpp files for RestoreLoader and RestoreApplier roles.
We will split the code for each restore role into a separate file.

This commit also fixes the bug in including RestoreCommon.actor.h, and
remove the unused code.
2019-05-06 16:59:41 -07:00
Andrew Noyes 6207d724f8 Fix all -Wunused-variable warnings 2019-04-15 18:13:00 -07:00
A.J. Beamon 058d028099
Merge pull request #1301 from mpilman/features/cheaper-traces
Defer formatting in traces to make them cheaper
2019-04-09 10:11:04 -07:00
mpilman 789cd67bcd Don't compile JavaWorkload by default 2019-04-08 13:06:29 -07:00
mpilman 4287b1d2a1 resolved minor merge issues 2019-04-05 13:12:19 -07:00
mpilman 02e3b634fb Compile sqlite with NDEBUG so we can debug 2019-04-05 13:12:19 -07:00
Markus Pilman 101a05ae77
Merge branch 'master' into features/client-simulator 2019-04-03 10:03:56 -08:00
Jingyu Zhou 56a1128a9b Enhance cmake's gperftools support
Add compiler flags and link flags for gperftools.
2019-04-02 17:34:29 -07:00
mpilman e23e63c6ac Implemented JavaWorkload
This change allows a user to write a workload in Java.

The way this is implemented is by creating a JVM within the
simulator and calling the corresponding workload class. A
workload can then run in the simulator or on a testing cluster.

If the workload is executed within the simulator, the resulting
test will not be deterministic anymore as it will execute in a
different thread (and even without that it is not clear, whether
we could get determinism as the JVM does a lot of stuff that are
not deterministic).

This is intendet to get better testing of the Java client and
layer authors can use the simulator to test their layers on a single
machine but they can still simulate failing machines etc.
2019-03-31 17:57:43 -07:00
Evan Tschannen a7e45cff91
Merge pull request #1176 from jzhou77/ratekeeper
Make Ratekeeper a separate role
2019-03-12 15:58:59 -07:00
mpilman 2537f26de6 First implementaion of more user-friendly cpack
Up unto here this code is only very rudiemantery tested.

This is a firest attempt of making cpack more user-friendly.
The basic idea is to generate a component for package type so
that we can have different paths depending on whether we build
an RPM, a DEB, a TGZ, or a MacOS installer. The cpack package
config file will then chose the correct components to use.

In a later point this should make it possible to build these
with `make packages` and the ugly iteration with calling cmake
between each package would be obsolete. While this solution is
a bit more bloated, it is also much more flexible and it will be
much easier to use.

Another benefit is, that this will get rid of all warnings during
a cpack run
2019-03-07 16:49:29 -08:00
Jingyu Zhou 36a51a7b57 Fix a segfault bug due to uncopied ratekeeper interface 2019-03-07 13:16:20 -08:00
Jingyu Zhou 3c86643822 Separate Ratekeeper from data distribution.
Add a new role for ratekeeper.

Remove StorageServerChanges from data distribution.
Ratekeeper monitors storage servers, which borrows the idea from
DataDistribution.
2019-03-07 13:16:20 -08:00
Andrew Noyes 27d199409e Add KillRegion.actor.cpp workload to cmake 2019-03-07 12:14:42 -08:00
anoyes 981426bac9 More ide fixes 2019-03-05 18:03:57 -08:00
Trevor Clinkenbeard d2bde4e55b Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-27 16:30:25 -08:00
Trevor Clinkenbeard fa96b8dd33 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-20 16:56:16 -08:00
Alex Miller df61bd07db Save an (unused) copy of the previous TLog. 2019-02-19 22:18:10 -08:00
Alex Miller 91e05575a2 Rename OldTLogServer -> OldTLogServer_4_6 2019-02-19 22:18:10 -08:00
mpilman 999ea09bfd Use correct fwd decls in TesterInterface
Also TesterInterface.h -> TesterInterface.actor.h
2019-02-19 15:16:59 -08:00
mpilman 699216f713 Use fwd decls in workloads
Also workloads.h -> workloads.actor.h
2019-02-19 15:16:59 -08:00
mpilman 3f0fd2a20c Use fwd decls in WorkerInterface
Also WorkerInterface.h -> WorkerInterface.actor.h
2019-02-19 15:16:59 -08:00
mpilman 27a3153719 Use ACTOR forward declarations in MoveKeys
Also MoveKeys.h -> MoveKeys.actor.h
2019-02-19 15:16:59 -08:00
mpilman 9b14aeb156 Tell cmake not to link/install on ide build 2019-02-19 15:16:59 -08:00
Evan Tschannen 83060c6e56
Merge pull request #1062 from jzhou77/PR
Add a new DataDistributor role.
2019-02-15 13:51:27 -08:00
mpilman 75f692b931 simplify actorcompiler and target to compile coveragetool 2019-02-15 00:01:42 -08:00
Jingyu Zhou c35d1bf2ef Fix according Alex's comment 2019-02-14 16:30:13 -08:00
A.J. Beamon 8a17905621 Add a couple new files to CMakeLists 2019-02-14 08:08:44 -08:00
Trevor Clinkenbeard a09afe5906 Added Throttling workload to test native health metrics API 2019-02-04 13:04:25 -08:00
mpilman 58964af7e1 ctest improvements - #1058
- A set of CMake variables controls whether to keep
  the simfdb directory and the traces and whether we
  want to aggregate the traces into a single file
- Test labels now contain the directory they are in
  so that one can now run `ctest -R fast/`
- A different binary can be used for restart tests. CMake
  will automatically look for an installed fdb and use that
  by default. If none is found, it will use the built one
  but it will also print a warning
- CMake will throw an error if there are any text files in
  the tests directory that are not associated with a test.
- Moved testing from fdbserver/CMakeLists.txt to
  tests/CMakeLists.txt
- Moved fdb testing functions to its own cmake module
2019-01-22 14:34:51 -08:00
Alex Miller b4a446756a Remove more top-level tests that were out of order. 2019-01-14 20:28:40 -08:00
Alex Miller 0d579b4730 Top level tests aren't run by default, and some fail. 2019-01-14 19:14:25 -08:00
Alex Miller b3e977d7c1 Apply test directory as a label.
This isn't ideal, as it makes `restarting/from_5.2.0/potato.txt` have
the label "from_5.2.0" instead of "restarting", but it does make the
fast label work right.
2019-01-14 19:14:25 -08:00
mpilman 414fb0b6c8 made TestRunner work with XML traces 2019-01-14 19:14:25 -08:00
Markus Pilman 14f0a6958b added all tests to ctest 2019-01-14 19:14:25 -08:00
Markus Pilman b096b8e3f8 First tests working with ctest 2019-01-14 19:14:25 -08:00
Alex Miller 550daa05f8 Default configuration now builds. 2018-12-13 15:52:27 -08:00
Markus Pilman df0f491c29 Some more improvements to the build and preparations for packaging 2018-12-13 15:04:13 -08:00
Alex Miller e70e59a895 Change some file locations. 2018-12-13 14:53:19 -08:00
Markus Pilman dce290909d fdbserver now compiling 2018-12-13 14:13:47 -08:00