42430e8f5e
Add epochBegin version to OldTLogCoreData/OldLogData/OldTLogConf
...
This is to simplify the backup process so that whenever there is an old epoch
in the log system, we always know its begin version and can backup from that
version if no progress is known for that old epoch.
2020-01-22 19:38:46 -08:00
1e0753a327
Remove backup workers from DBCoreState
...
This is no longer needed.
2020-01-22 19:38:46 -08:00
a1095c8250
Remove epoch from DBCoreState
...
Use existing recoveryCount if needed.
2020-01-22 19:38:45 -08:00
19d6a889ff
Recruit backup workers for old epochs
...
If there are unfinished ranges in the old epochs, the new master will recruit
backup workers responsible for finishing these ranges. These workers remains in
the cluster until the next epoch, when it will remove itself.
2020-01-22 19:38:45 -08:00
ac851619bb
Fix merge errors with master
2020-01-22 19:38:45 -08:00
17002740bb
Add epoch and backup workers to DBCoreState
...
This enables backup workers to know the end version of the epoch. Additionally,
the master recovery only needs to deal with crashed backup workers by
recruiting new workers to backup the unfinished version range.
2020-01-22 19:38:45 -08:00
2c0bf99ef2
Update fdbserver/DBCoreState.h
...
Co-Authored-By: Alex Miller <35046903+alexmiller-apple@users.noreply.github.com>
2019-07-12 18:17:30 -07:00
2278e905ce
Update fdbserver/DBCoreState.h
...
Co-Authored-By: Alex Miller <35046903+alexmiller-apple@users.noreply.github.com>
2019-07-12 18:15:59 -07:00
7f4586ad49
the number of txsTags needs to be tracked separately from the number of transaction logs because of forced recoveries
2019-06-28 12:33:24 -07:00
68ce9a5e75
ProtocolVersion type - second try
2019-06-18 17:55:27 -07:00
8b67da57bb
Fix upgrade test failure
...
Serialize pseudoLocalities if protocol version is larger than 0x0FDB00B061060001LL.
Note this version may need to be changed to "currentProtocolVersion" when merging
into the master, and "currentProtocolVersion" should be incremented.
2019-04-21 10:41:07 -07:00
010f825aff
Remove pseudoLocalities from LogSet, TLogSet, and CoreTLogSet
2019-04-21 10:41:07 -07:00
7befce6bf1
More pseudoLocalities and refactors.
2019-04-21 10:41:07 -07:00
966ec30fcc
Add pseudoLocalities for special tag consumers
2019-04-21 10:41:07 -07:00
ec1bc5cfca
Add LogSystemType enum
2019-04-21 10:41:07 -07:00
a2108047aa
removed LocalitySetRef and IRepPolicyRef typedefs, because for clarity the Ref suffix is reserved for arena allocated objects instead of reference counted objects.
2019-03-13 13:14:39 -07:00
d4fe9f905c
Let log_spill/log_version default in config, and clean up serialization.
...
We don't need to abide by object serializer rules yet, and the minor
change to Simulation config lets us test config being the default at the
start of the test.
2019-02-26 17:14:41 -08:00
2dc57568cb
Change many things about log_version.
...
* log_version in the database (`/conf/log_version`) is now a hint that gets
rounded to the nearest supported version.
* fdbcli and FDB enforce that only a valid log_version can be configured to
* TLogVersion is persisted in CoreTLogSet (and LogSet and TLogSet)
* Some comments here and there
* Add an assert on filename length to make sure KV-pairs in filename
don't exceed a maximum length.
2019-02-26 16:47:04 -08:00
6a4d87802b
Replace & operator with variadic function
2018-12-28 11:33:42 -08:00
b1fe069165
fix: during forced recovery logs can be removed from the logSystemConfig. We need to avoid killing the removed logs as unneeded until we actually complete the recovery
2018-09-28 12:13:46 -07:00
6931a00993
satellite log push locations are static per tag, which will reduce the number of tags each satellite log has to index, and reduce the proxy cpu when calculating push locations
2018-06-16 17:39:02 -07:00
f694f7c9ca
removed hasBestPolicy
2018-06-15 12:36:19 -07:00
feb8578c06
fix: only flush and exit in simulation
2018-06-14 13:48:30 -07:00
eded5631e6
fix: epoch end was already known committed version + 1, and did not need an additional + 1.
2018-04-30 22:03:11 -07:00
4248fbec61
fix: must set startVersion when upgrading
2018-04-11 17:33:17 -07:00
7af892f50b
first working version of non-copying recovery working with fearless configurations
2018-04-08 21:24:05 -07:00
1a4ded1c99
support upgrades by merging tags associated with the different peek requests
2018-03-29 17:54:08 -07:00
b36e08f08f
first version of non-copying recovery. Upgrades are broken, and it has not been tested using fearless configurations yet
2018-03-29 15:12:38 -07:00
37a6a81634
Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs
...
# Conflicts:
# fdbserver/workloads/RestartRecovery.actor.cpp
2018-02-23 12:33:28 -08:00
0bae9880f1
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-21 10:25:11 -08:00
6b54d56ca7
gracefully exit if attempting to upgrade from 4.X versions
2018-01-30 17:10:50 -08:00
af97a512f5
to support more complicated policies in the future for determining the best location for a tag within a set of tlogs, use an integer instead of a bool
2018-01-29 17:48:18 -08:00
22e5a0b257
formatting
2018-01-11 14:44:09 -08:00
173a8de3ed
DBCoreState supports upgrades from 3.0 versions
2018-01-11 14:39:51 -08:00
ea26bc1c43
passed first tests which kill entire datacenters
...
added configuration options for the remote data center and satellite data centers
updated cluster controller recruitment logic
refactors how master writes core state
updated log recovery, and log system peeking
2017-09-07 15:32:08 -07:00
81ae263ad9
implemented setPeekCursor
...
removed oldTLogServer
first compiling version
2017-07-10 17:41:32 -07:00
979ebcef6c
changed to using a vector of logSets instead of a duplicate set of logs for remote servers
...
finished porting changes to the tlog
everything but peeking is finished in the TagPartitionedLogSystem
2017-07-09 14:46:16 -07:00
0906250e78
merged everything from feature-remote-logs besides the tlog and tagpartitionedlogsystem
...
re-included tags in messages to the tlog
previously never committed the LogRouter
2017-06-29 15:50:19 -07:00
a674cb4ef4
Initial repository commit
2017-05-25 13:48:44 -07:00
Jingyu Zhou