Commit Graph

56 Commits

Author SHA1 Message Date
Xiaoxi Wang ce33366396
only add mutations can change configuration (#6760) 2022-04-05 17:05:51 -07:00
Josh Slocum f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Josh Slocum e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
A.J. Beamon 5fa9d3e1b7 Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode. 2022-03-06 21:54:21 -08:00
Suraj Gupta fc3376fe8f Move client knob to database config for blob granules. 2021-12-10 14:00:34 -06:00
Ata E Husain Bohra 0962fcb243 Override commit/grv proxies_count if mutation supplied new value is -1
Patch improves on handling scenarios where either commit or grv proxies
value is update to -1 OR `proxies_count` is being reset.
The code splits the proxies between two proxies by ensuring for invalid
input configuration, the min (read as 1) proxies gets provisioned, otherwise,
the split is done based on input values

Patch handles the scenario where mutation supplied values to update grv_proxies
and/or commit_proxies is -1, however, the total proxy count > 1,
uses DEFAULT_COMMIT_GRV_PROXIES_RATIO to split proxies between
grv_proxies & commit_proxies.
2021-11-24 12:52:31 -08:00
Neethu Haneesha Bingi 3e79299898 Locality filter support to perpetual storage wiggler feature. 2021-09-30 10:00:33 -07:00
Josh Slocum 9992a7b33f Added StorageMigrationType and cli commands 2021-09-14 09:55:41 -05:00
Neethu Haneesha Bingi 62355571d0 exclude servers based on locality match 2021-06-23 18:03:27 -07:00
Josh Slocum ce82c9653e Testing Storage Server implementation 2021-05-25 20:28:50 +00:00
Xiaoxi Wang 8b1f9f7337 Add command line support 2021-05-17 20:57:54 +00:00
Zhe Wu 2648d19edd Update comment 2021-03-29 15:10:57 -07:00
Zhe Wu 27060cf4a9 Populate min_replicas_remaining field for all regions during cluster status computation. This provides more information about the number of available replicas currently in the database. 2021-03-17 15:45:12 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes b0f61fb74f Resolve the simple-looking conflicts 2021-01-15 19:35:10 +00:00
Andrew Noyes ff7d306b09 Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
Vishesh Yadav 6957e64886 fix: Calculate fault tolerance when keys are not fully replicated
Whem the database is not fully replicated to remote, fault tolerance
can be calculated by couting total number of tLogs replicas in primary
and satellite - 1, as we can lose all but one of those, but we were
subtracting 2 instead of 1.
2020-12-22 16:26:19 -08:00
Vishesh Yadav a1809f7d86 doc: Fault Tolerance and Region Configuration 2020-12-22 16:25:17 -08:00
Markus Pilman 268ba0bddc Merge remote-tracking branch 'origin/release-6.3' into merge-6.3 2020-10-01 14:14:06 -06:00
Evan Tschannen 8c729ca8e6 only add additional fault tolerance for availability if automatic failover is enabled 2020-09-30 18:04:23 -07:00
Evan Tschannen fe5c30e778 fault tolerance was not being properly increased when usable regions was 2 and satellites are configured. 2020-09-30 16:41:00 -07:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Young Liu 8953cc5f98 Merge master again 2020-08-12 16:59:07 -07:00
Evan Tschannen 2f52c5f79b Merge branch 'release-6.3'
# Conflicts:
#	fdbserver/RestoreLoader.actor.cpp
2020-08-11 22:45:43 -07:00
Xiaoge Su ceeade7334 Address review comments, improve performance 2020-08-08 23:52:38 -07:00
Xiaoge Su 6b8843ad18 When a configuration K/V pair is modified, always output the K/V pair values
With this patch, if there is a configuration special key in the
database, even if the value is reverted to default/hidden, it will still
be forwarded when constructing a JSON string. This should address the
requirements described in https://github.com/apple/foundationdb/issues/3551.
2020-08-08 23:24:30 -07:00
Young Liu d6a23a4d6b Resolve comments to make GRV proxy a separate process class 2020-08-06 00:01:57 -07:00
Young Liu 525f10e30c Merge master branch 2020-07-22 16:08:49 -07:00
Young Liu 302cf5c45f Remove debug trace events 2020-07-22 12:20:22 -07:00
Young Liu 5b06d69d25 Pass watches test 2020-07-15 00:37:41 -07:00
Andrew Noyes f470ba8316 Remove using namespace std::rel_ops
This causes the following to not compile anymore

\#include <utility>
\#include <vector>

using namespace std::rel_ops;

int main() {
    std::vector<int> xs;
    return xs.rbegin() != xs.rend();
}

See https://godbolt.org/z/s1977n
2020-07-10 22:58:15 +00:00
Evan Tschannen ce4493f679 many bug fixes 2020-04-10 13:45:16 -07:00
Evan Tschannen e08f0201f1 merge release 6.2 into master 2020-03-17 12:51:47 -07:00
Evan Tschannen 4640edf5d6 do not recruit satellite tlogs when usable regions=1 2020-03-13 10:24:52 -07:00
Jingyu Zhou 52c6737411 Rename backupLoggingEnabled as backupWorkerEnabled
To highlight the changes for 7.0 backup changes. By default,
backup_worker_enabled flag is set for 7.0 version.
2020-02-04 10:09:16 -08:00
Jingyu Zhou 0db03f1d3c Use backup_logging_enabled flag
The default is to enable new backup workers. Users can disable this flag to
turn off the backup worker feature.
2020-02-03 20:03:22 -08:00
Jingyu Zhou 38aa1903fd Add a DB configuration option for backup workers
Right now, the default is to keep the old backup behavior, i.e., do NOT use
backup workers. Specifically, if BackupType is not set (or is set to default),
the master will not recruit backup workers and will not add pseudo locality for
backup workers.

The StartFullBackupTaskFunc is updated to check if backup worker is enabled.
Only when it is not enabled, starting a backup will wait on all backup workers
to be started.
2020-01-31 19:29:09 -08:00
Evan Tschannen 85bc5f6b8b
Update fdbclient/DatabaseConfiguration.h
Co-Authored-By: Markus Pilman <markus@pilman.ch>
2019-10-16 15:11:24 -07:00
Evan Tschannen 1af44afad3
Update fdbclient/DatabaseConfiguration.h
Co-Authored-By: Markus Pilman <markus@pilman.ch>
2019-10-16 14:55:02 -07:00
Evan Tschannen 35e816e9ad added the ability to configure satellite_logs by satellite location, this will overwrite the region configure if both are present 2019-10-14 18:30:15 -07:00
A.J. Beamon 438bc636d5 Rename max_machine_failures_without_losing_X to max_zone_failures_without_losing_X in status. 2019-07-30 14:02:31 -07:00
Evan Tschannen a2108047aa removed LocalitySetRef and IRepPolicyRef typedefs, because for clarity the Ref suffix is reserved for arena allocated objects instead of reference counted objects. 2019-03-13 13:14:39 -07:00
Evan Tschannen 8afb7fbb9d
Merge pull request #1160 from alexmiller-apple/tstlog-fork
Spill-By-Reference TLog Part 2: New and Old TLogServers co-exist harmoniously
2019-02-26 18:00:04 -08:00
Alex Miller 6d23eb2d1a Implement log_version.
This mega-commit introduces a new configuration setting, `log_version`,
that controls the TLog implementations and features that are available
within FDB, so that users can opt in to new features if they're willing
to sacrifice backwards compatibility.
2019-02-22 12:15:23 -08:00
Alex Miller 0cf3ee9f99 Add a way to configure log spill type. 2019-02-19 22:18:10 -08:00
Evan Tschannen 8f2af8bed1 fix: forced recoveries now require a target dcid which will become the new primary location. During the forced recovery, the configuration will be changed to make that location primary, and usable_regions will be set to 1. If the target dcid is already the primary location, the forced recovery will do nothing. This makes forced recoveries idempotent, so it is safe to the client to re-send forced recovery commands to the cluster controller.
fix: the cluster controller attempts to do a commit to determine if the cluster is alive, since its own internal recoveryState might not be up-to-date.

fix: forceMasterFailure on the cluster controller did not always cause the current master to be re-recruited
2019-02-18 14:54:28 -08:00
anoyes 6a4d87802b Replace & operator with variadic function 2018-12-28 11:33:42 -08:00
Evan Tschannen 866ccfe344 added the ability to allow the master to finish recovery before all storage servers in both regions have their mutations. This allows you to recover from scenarios where you lose all your tlogs in one dc. 2018-07-04 01:59:04 -04:00
Evan Tschannen a288d5b9a9 added a fallback satellite configuration, so that we can use two satellites if available, but do not have to failover to the remote datacenter if one satellite is down 2018-06-28 23:15:32 -07:00