6499fa178e
Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage
...
Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-l…
2021-05-11 18:50:42 -07:00
4ea760b2a9
Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
...
This reverts commit da41534618e6300905d6
2021-05-10 20:26:12 -07:00
cdf98f987d
Revert "Fix global config not triggering changes on server processes"
2021-05-10 12:18:28 -07:00
3f6ef14384
Fix OOM
2021-05-07 16:28:39 -07:00
71ba6f4501
Switch std::function to AsyncTrigger
2021-05-07 16:05:03 -07:00
0fc1552683
Fix global config updates not triggering on fdbserver processes
2021-05-06 15:46:34 -07:00
7da5004690
Merge branch 'master' of github.com:apple/foundationdb into refactor-fdbcli
2021-05-06 15:44:26 -07:00
336a9bff66
Provide "time since last full recovery" in fdbcli status
...
Changes:
Schemas.cpp: Extend the JSON schema to include a new field that reports
the number of seconds since last full recovery.
Status.actor.cpp: Extend recoveryStateStatusFetcher() to populate the
new field that has been added to Schemas.cpp.
mr-status-json-schemas.rst.inc: Update the schema to reflect the change
made in Schemas.cpp.
2021-05-05 19:43:44 +00:00
ed464d3514
Deduplicate
2021-05-05 16:30:59 +00:00
946e3f989f
Use a deterministic iteration order for committing in the special key space
2021-05-05 16:30:59 +00:00
1537019237
Merge pull request #4729 from sfc-gh-etschannen/feature-bypass-unreadable
...
Added a new option to bypass unreadable protection in read your writes for calls to get
2021-05-04 11:35:44 -07:00
f5f9de48e7
Update fdbclient/vexillographer/fdb.options
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-05-04 09:31:24 -07:00
3175e82393
updated documentation
2021-05-04 09:26:09 -07:00
e7d7b39f12
Merge pull request #4744 from sfc-gh-tclinkenbeard/add-rangeresult-type-alias
...
Create RangeResult type alias
2021-05-03 16:29:33 -07:00
56e719533e
Move all Standalone<*Ref> type alias definitions to the same location
2021-05-03 15:47:55 -07:00
d49e0091ce
Merge pull request #4727 from sfc-gh-etschannen/fix-rewrite-bme
...
Simulation could still stall writes for 10 seconds even when speedUpSimulation was on
2021-05-03 13:37:04 -07:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
63d82b9b63
Merge pull request #4616 from sfc-gh-satherton/redwood-improvements
...
Redwood page splitting/building refactor
2021-05-03 13:07:41 -07:00
f275fd3c32
Fix gcc compilation
2021-05-02 17:27:18 -07:00
8dcd779fc4
Merge branch 'master' into features/actor-lineage
2021-05-02 14:11:42 -07:00
c016e154a7
Remove global config fdbserver fix
...
This is causing problems with the 5.2.0 restarting test. Removing this
line disables fdbserver processes from receiving global config updates,
instead requiring a restart to see them.
2021-05-02 11:03:07 -07:00
637699be32
Fix issue with fdbserver not receiving global config change
...
notifications
2021-05-01 21:41:10 -07:00
cf4218dfd1
Fixes simulation failures
...
Fixes the following issues:
1. Use the right index when initializing the WriteOnlySet's vector of
atomics. Also switch to std::atomic_init to initialize each atomic in
the vector (cannot default construct the atomics in the vector
because std::atomic does not have a copy constructor).
2. Add failure check for when items cannot be inserted into the
WriteOnlySet due to capacity constraints. This situation occurs when
`copy` is not called on the WriteOnlySet, such as when sampling is
disabled. The `copy` function is what clears the WriteOnlySet.
3. Remove a global config feature I added to update the ClientDBInfo
object used by the global config listener function. This needs more
investigation, but the effect of this change could be that global
config changes are not correctly recognized on fdbserver processes.
4. Add various ASSERTs to verify data in WriteOnlySet.
2021-05-01 15:26:28 -07:00
e6300905d6
Merge pull request #4733 from sfc-gh-ajbeamon/fix-mvc-thread-future-cancellation
...
Fix: explicitly cancel MVC thread futures for version monitors
2021-04-30 16:05:17 -07:00
f7e28c50d4
Merge pull request #4735 from sbodagala/master
...
Expose CommitBatchingWindowSize metric to fdbcli status
2021-04-30 15:52:29 -04:00
ab3f96f16d
Fix: simulation doesn't have a protocol version monitor and can't cancel it.
2021-04-30 11:56:35 -07:00
eaf1e0f64e
Be more defensive with cancellation by not capturing the this pointer in lambdas, instead capturing a full Reference.
2021-04-30 11:11:02 -07:00
cb3d2bfec7
Add cancellation of the protocol monitor when the database is destroyed. Avoid using any state when cancelled. Fix race between setting up the protocol version monitor and destroying the database.
2021-04-30 10:35:44 -07:00
652c5c4e84
Merge pull request #4668 from RenxuanW/worker
...
Improve logging on worker joining cluster
2021-04-30 10:24:02 -07:00
f61f13a0ba
Explicitly cancel thread futures for the protocol version monitors in MVC
2021-04-29 22:02:38 -07:00
fbd59d310a
Merge pull request #4725 from RenxuanW/firstConsistentVersion-fix
...
Fix the logic of getting firstConsistentVersion.
2021-04-29 18:04:57 -07:00
5e045bd21c
Move the read of logsOnly and inconsistentSnapshotOnly to a loop where other RestoreConfig vars are fetched.
2021-04-29 17:00:34 -07:00
f151df3203
Expose CommitBatchingWindowSize metric to fdbcli status
...
Changes:
Schemas.cpp:
- Extend JSON schema to include aggregated information about
CommitBatchingWindowSize samples.
Status.actor.cpp:
- Extend getStorageServersAndMetrics() to gather metrics about
CommitBatchingWindowSize.
- Extend CommitProxy AddRole() to populate the status-JSON object
with the metrics about CommitBatchingWindowSize.
2021-04-29 22:11:09 +00:00
44197644be
Reset transaction after a commit and before start a read.
2021-04-29 12:19:58 -07:00
283b774c2f
Merge branch 'master' of github.com:apple/foundationdb into refactor-fdbcli
2021-04-29 12:04:54 -07:00
709df795c0
Added a new option to bypass unreadable protection in read your writes for calls to get
2021-04-28 16:22:34 -07:00
539a419f29
Merge pull request #19 from sfc-gh-mpilman/features/actor-lineage-fluentd
...
fix special keyspace register
2021-04-28 15:43:51 -07:00
045d20ab7a
Check validity of ActorLineage
2021-04-28 15:40:13 -07:00
f513543305
fix special keyspace register
2021-04-28 16:35:09 -06:00
b256c6822d
add default constructor to Sample
2021-04-28 15:44:06 -06:00
904a39e473
Merge pull request #4667 from sfc-gh-ajbeamon/feature-mvc-monitor-protocol-version
...
Use fewer connections in the multi-version client
2021-04-28 14:13:17 -07:00
32ee206675
delete copy constructor of Sample
2021-04-28 14:11:09 -06:00
d946e90d75
Use new Sample interface
2021-04-28 14:10:45 -06:00
65fcf4014e
Fix: simulation could still stall writes for 10 seconds even when speedUpSimulation was on
...
Fix: disable connection failures in simulation when there are too many generations outstanding
2021-04-28 12:41:48 -07:00
05dba91dd4
fix OPEN_FOR_IDE
2021-04-28 10:36:41 -06:00
868ec17223
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-fluentd
2021-04-28 10:27:28 -06:00
3400ab5b36
Merge pull request #15 from sfc-gh-ljoswiak/features/actor-lineage-interface
...
Add API to read samples from worker
2021-04-28 10:26:22 -06:00
8eb78fd317
Merge pull request #4714 from sfc-gh-anoyes/anoyes/more-thread-safety-ctest
...
Add /fdbclient/multiversionclient/ to ctest, and fix thread safety
2021-04-28 08:54:16 -07:00
f5b0008346
Merge branch 'features/actor-lineage' into features/actor-lineage-fluentd
2021-04-28 09:31:20 -06:00
6f71a811b6
fix memory leak
2021-04-28 09:27:11 -06:00
0ee0b8a76f
Fixed typo
...
Co-authored-by: Lukas Joswiak <lukas.joswiak@snowflake.com>
2021-04-28 09:25:28 -06:00
54919d4f3b
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage
2021-04-28 09:22:14 -06:00
4fab2ecd30
Merge remote-tracking branch 'origin/master' into features/actor-lineage
2021-04-28 09:20:54 -06:00
1e665044fe
bugfix
2021-04-28 09:08:17 -06:00
0145eea684
Make `MonitorLeaderForwarding` and `LeaderForwarding` trackLatest events.
2021-04-27 15:17:20 -07:00
16dfb2b2f2
Keep connections older than 6.2 open indefinitely to avoid weird bugs around quickly closing the database.
2021-04-27 15:00:56 -07:00
5d0eaac3ea
Remove old code
2021-04-27 11:40:02 -07:00
2f3d70c084
Fix the logic of getting firstConsistentVersion.
...
First consistent version should be:
- In a logs-only restore, it is the begin version the user said to start applying logs for;
- In an inconsistent-snapshot-only restore, if all range files have the same version, then it is that version, otherwise unknown (use -1);
- If using both range files and logs, then it is the highest version of any range file in the RestoreSet’s ranges vector.
2021-04-27 11:27:57 -07:00
d964b5ded0
clang-format
2021-04-27 10:41:48 -07:00
0ba5a8e9d1
Fix return key when sorting by time
2021-04-27 10:39:26 -07:00
e163432303
Add filtering by wait state
2021-04-27 10:20:25 -07:00
10d5007e1a
Cleanup
2021-04-27 09:59:10 -07:00
2d6fafde64
Implemented configuration
2021-04-27 10:26:42 -06:00
340f012e1a
Merge pull request #4695 from sfc-gh-etschannen/fix-rewrite-bme
...
rewrote tlog recruitment logic so that it is deterministic
2021-04-27 10:19:25 -06:00
7f9ee224a4
Refactor samples to include wait state
2021-04-26 22:50:44 -07:00
76acb0fcb9
Update date format to ISO 8601
2021-04-26 17:42:15 -07:00
823873a9aa
Address review comments:
...
Use nullptr instead of NULL
Use const& for a parameter
Add some comments
2021-04-26 14:39:27 -07:00
719f810676
Rename incrementalBackupOnly to onlyAppyMutationLogs in all restore configs and functions.
2021-04-26 12:30:46 -07:00
6b81b7a04b
Remove current lineage validity check
2021-04-26 11:04:36 -07:00
656c9a6c47
Add benchmark and document entities touched
2021-04-26 17:46:35 +00:00
f1559a2203
use the stateless process class instead of master or resolution in simulation because it is the recommended process class, and the others are not deterministic when recruited in a constrained process situation
2021-04-26 09:49:26 -07:00
e45faa3534
Fix a bug where deleting a key invalidated its memory which was later
...
read
2021-04-23 16:38:01 -07:00
a794fca932
Support 5.0 (and earlier) client versions by adding GRV probing for old versions. Update the C bindings implementation of get_server_protocol to convert the ProtocolVersion object into a uint64_t. Rename a misleading protocol version alias.
2021-04-23 15:00:21 -07:00
6fc59379d8
Add /fdbclient/multiversionclient/ to ctest, and fix thread safety
2021-04-23 21:17:41 +00:00
9adce8456a
Add invalid reference check
2021-04-23 14:06:23 -07:00
3cf2dd0fbe
Remove TODO
2021-04-23 14:06:23 -07:00
25fb85a64c
Add API to read samples from worker
2021-04-23 14:06:21 -07:00
52bba82e8e
Add window size configuration key
2021-04-23 14:05:05 -07:00
039a7dc482
Merge branch 'master' of github.com:apple/foundationdb into refactor-fdbcli
2021-04-23 12:04:21 -07:00
5489de985c
Merge pull request #4582 from sfc-gh-clin/add-dd-and-maintenance
...
Add dd and maintenance
2021-04-23 11:43:35 -07:00
185d08b5b8
Add comments for added actors
2021-04-23 11:13:08 -07:00
3e18b857a8
add command line args to configure profile ingestor
2021-04-23 11:02:53 -06:00
de4753a5db
Add a workaround to temporily use the ryw to create a ThreadTransaction; Make sure we are using the same underlying ryw object
2021-04-23 01:32:30 -07:00
adb0ce9776
address review comments
2021-04-22 17:52:27 -06:00
a6a8a97e1f
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-instrument
2021-04-22 17:50:36 -06:00
99c1edf87e
Implemented fluentd functionality
2021-04-22 17:48:09 -06:00
41ca11c3e5
Implement restoring an inconsistent snapshot as a real feature.
2021-04-22 13:53:37 -07:00
4ff714efbd
Merge pull request #4691 from RenxuanW/first-consistent-version
...
Add first consistent version in restore status.
2021-04-21 17:46:13 -07:00
bc43fa99ac
Move commit to its own try loop.
2021-04-21 17:37:58 -07:00
8a00c6cdf8
Add -Wshift-sign-overflow
...
This catches the bug fixed in #4656 at compile time
2021-04-21 23:49:26 +00:00
b90f61d740
Move commit to its own try loop.
2021-04-21 15:50:25 -07:00
f0a236c544
Merge branch 'master' of github.com:apple/foundationdb into refactor-fdbcli
2021-04-21 15:36:11 -07:00
7c4b5b0337
Add first consistent version in restore status.
...
First consistent version is the max of versions in RestoreFileSet.
2021-04-21 14:32:13 -07:00
3fcbed1fbd
Merge remote-tracking branch 'sfc/features/actor-lineage' into features/actor-lineage-fluentd
2021-04-21 14:58:11 -06:00
80e15e8768
started implementation
2021-04-21 14:56:02 -06:00
f485d7fa5e
Fix comment typo
2021-04-21 12:25:03 -07:00
b9d8302ac7
Merge pull request #4540 from dlambrig/batch
...
Log latency metrics for batch GRV requests
2021-04-21 10:23:37 -07:00
ba60f18ebf
Merge remote-tracking branch 'upstream/master' into backup-agent
2021-04-20 22:54:47 -07:00
36b1ab7ba5
Detach profiler thread instead of joining it
2021-04-20 22:05:16 -07:00
8b280f5be6
Remove old includes
2021-04-20 17:55:27 -07:00
Lukas Joswiak