Commit Graph

3447 Commits

Author SHA1 Message Date
Lukas Joswiak f275fd3c32 Fix gcc compilation 2021-05-02 17:27:18 -07:00
Lukas Joswiak 8dcd779fc4
Merge branch 'master' into features/actor-lineage 2021-05-02 14:11:42 -07:00
Lukas Joswiak c016e154a7 Remove global config fdbserver fix
This is causing problems with the 5.2.0 restarting test. Removing this
line disables fdbserver processes from receiving global config updates,
instead requiring a restart to see them.
2021-05-02 11:03:07 -07:00
Lukas Joswiak 637699be32 Fix issue with fdbserver not receiving global config change
notifications
2021-05-01 21:41:10 -07:00
Lukas Joswiak cf4218dfd1 Fixes simulation failures
Fixes the following issues:

1. Use the right index when initializing the WriteOnlySet's vector of
   atomics. Also switch to std::atomic_init to initialize each atomic in
   the vector (cannot default construct the atomics in the vector
   because std::atomic does not have a copy constructor).
2. Add failure check for when items cannot be inserted into the
   WriteOnlySet due to capacity constraints. This situation occurs when
   `copy` is not called on the WriteOnlySet, such as when sampling is
   disabled. The `copy` function is what clears the WriteOnlySet.
3. Remove a global config feature I added to update the ClientDBInfo
   object used by the global config listener function. This needs more
   investigation, but the effect of this change could be that global
   config changes are not correctly recognized on fdbserver processes.
4. Add various ASSERTs to verify data in WriteOnlySet.
2021-05-01 15:26:28 -07:00
A.J. Beamon e6300905d6
Merge pull request #4733 from sfc-gh-ajbeamon/fix-mvc-thread-future-cancellation
Fix: explicitly cancel MVC thread futures for version monitors
2021-04-30 16:05:17 -07:00
sbodagala f7e28c50d4
Merge pull request #4735 from sbodagala/master
Expose CommitBatchingWindowSize metric to fdbcli status
2021-04-30 15:52:29 -04:00
A.J. Beamon ab3f96f16d Fix: simulation doesn't have a protocol version monitor and can't cancel it. 2021-04-30 11:56:35 -07:00
A.J. Beamon eaf1e0f64e Be more defensive with cancellation by not capturing the this pointer in lambdas, instead capturing a full Reference. 2021-04-30 11:11:02 -07:00
A.J. Beamon cb3d2bfec7 Add cancellation of the protocol monitor when the database is destroyed. Avoid using any state when cancelled. Fix race between setting up the protocol version monitor and destroying the database. 2021-04-30 10:35:44 -07:00
Renxuan Wang 652c5c4e84
Merge pull request #4668 from RenxuanW/worker
Improve logging on worker joining cluster
2021-04-30 10:24:02 -07:00
A.J. Beamon f61f13a0ba Explicitly cancel thread futures for the protocol version monitors in MVC 2021-04-29 22:02:38 -07:00
Steve Atherton fbd59d310a
Merge pull request #4725 from RenxuanW/firstConsistentVersion-fix
Fix the logic of getting firstConsistentVersion.
2021-04-29 18:04:57 -07:00
RenxuanW 5e045bd21c Move the read of logsOnly and inconsistentSnapshotOnly to a loop where other RestoreConfig vars are fetched. 2021-04-29 17:00:34 -07:00
Sreenath Bodagala f151df3203 Expose CommitBatchingWindowSize metric to fdbcli status
Changes:

Schemas.cpp:
- Extend JSON schema to include aggregated information about
CommitBatchingWindowSize samples.

Status.actor.cpp:
- Extend getStorageServersAndMetrics() to gather metrics about
CommitBatchingWindowSize.
- Extend CommitProxy AddRole() to populate the status-JSON object
with the metrics about CommitBatchingWindowSize.
2021-04-29 22:11:09 +00:00
RenxuanW 44197644be Reset transaction after a commit and before start a read. 2021-04-29 12:19:58 -07:00
Lukas Joswiak 539a419f29
Merge pull request #19 from sfc-gh-mpilman/features/actor-lineage-fluentd
fix special keyspace register
2021-04-28 15:43:51 -07:00
Lukas Joswiak 045d20ab7a Check validity of ActorLineage 2021-04-28 15:40:13 -07:00
Markus Pilman f513543305 fix special keyspace register 2021-04-28 16:35:09 -06:00
Markus Pilman b256c6822d add default constructor to Sample 2021-04-28 15:44:06 -06:00
Andrew Noyes 904a39e473
Merge pull request #4667 from sfc-gh-ajbeamon/feature-mvc-monitor-protocol-version
Use fewer connections in the multi-version client
2021-04-28 14:13:17 -07:00
Markus Pilman 32ee206675 delete copy constructor of Sample 2021-04-28 14:11:09 -06:00
Markus Pilman d946e90d75 Use new Sample interface 2021-04-28 14:10:45 -06:00
Markus Pilman 05dba91dd4 fix OPEN_FOR_IDE 2021-04-28 10:36:41 -06:00
Markus Pilman 868ec17223 Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-fluentd 2021-04-28 10:27:28 -06:00
Markus Pilman 3400ab5b36
Merge pull request #15 from sfc-gh-ljoswiak/features/actor-lineage-interface
Add API to read samples from worker
2021-04-28 10:26:22 -06:00
A.J. Beamon 8eb78fd317
Merge pull request #4714 from sfc-gh-anoyes/anoyes/more-thread-safety-ctest
Add /fdbclient/multiversionclient/ to ctest, and fix thread safety
2021-04-28 08:54:16 -07:00
Markus Pilman f5b0008346 Merge branch 'features/actor-lineage' into features/actor-lineage-fluentd 2021-04-28 09:31:20 -06:00
Markus Pilman 6f71a811b6 fix memory leak 2021-04-28 09:27:11 -06:00
Markus Pilman 0ee0b8a76f
Fixed typo
Co-authored-by: Lukas Joswiak <lukas.joswiak@snowflake.com>
2021-04-28 09:25:28 -06:00
Markus Pilman 54919d4f3b Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage 2021-04-28 09:22:14 -06:00
Markus Pilman 4fab2ecd30 Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-04-28 09:20:54 -06:00
Markus Pilman 1e665044fe bugfix 2021-04-28 09:08:17 -06:00
RenxuanW 0145eea684 Make `MonitorLeaderForwarding` and `LeaderForwarding` trackLatest events. 2021-04-27 15:17:20 -07:00
A.J. Beamon 16dfb2b2f2 Keep connections older than 6.2 open indefinitely to avoid weird bugs around quickly closing the database. 2021-04-27 15:00:56 -07:00
Lukas Joswiak 5d0eaac3ea Remove old code 2021-04-27 11:40:02 -07:00
RenxuanW 2f3d70c084 Fix the logic of getting firstConsistentVersion.
First consistent version should be:

- In a logs-only restore, it is the begin version the user said to start applying logs for;
- In an inconsistent-snapshot-only restore, if all range files have the same version, then it is that version, otherwise unknown (use -1);
- If using both range files and logs, then it is the highest version of any range file in the RestoreSet’s ranges vector.
2021-04-27 11:27:57 -07:00
Lukas Joswiak d964b5ded0 clang-format 2021-04-27 10:41:48 -07:00
Lukas Joswiak 0ba5a8e9d1 Fix return key when sorting by time 2021-04-27 10:39:26 -07:00
Lukas Joswiak e163432303 Add filtering by wait state 2021-04-27 10:20:25 -07:00
Lukas Joswiak 10d5007e1a Cleanup 2021-04-27 09:59:10 -07:00
Markus Pilman 2d6fafde64 Implemented configuration 2021-04-27 10:26:42 -06:00
Markus Pilman 340f012e1a
Merge pull request #4695 from sfc-gh-etschannen/fix-rewrite-bme
rewrote tlog recruitment logic so that it is deterministic
2021-04-27 10:19:25 -06:00
Lukas Joswiak 7f9ee224a4 Refactor samples to include wait state 2021-04-26 22:50:44 -07:00
Lukas Joswiak 76acb0fcb9 Update date format to ISO 8601 2021-04-26 17:42:15 -07:00
A.J. Beamon 823873a9aa Address review comments:
Use nullptr instead of NULL
Use const& for a parameter
Add some comments
2021-04-26 14:39:27 -07:00
RenxuanW 719f810676 Rename incrementalBackupOnly to onlyAppyMutationLogs in all restore configs and functions. 2021-04-26 12:30:46 -07:00
Lukas Joswiak 6b81b7a04b Remove current lineage validity check 2021-04-26 11:04:36 -07:00
Andrew Noyes 656c9a6c47 Add benchmark and document entities touched 2021-04-26 17:46:35 +00:00
Evan Tschannen f1559a2203 use the stateless process class instead of master or resolution in simulation because it is the recommended process class, and the others are not deterministic when recruited in a constrained process situation 2021-04-26 09:49:26 -07:00