Commit Graph

1018 Commits

Author SHA1 Message Date
Evan Tschannen 0010d500c5
Merge pull request #1303 from ajbeamon/fix-compiler-warnings
Fix various compiler warnings
2019-03-15 17:58:49 -07:00
Evan Tschannen b5d2debda7
Merge pull request #1298 from ajbeamon/fix-ryow-onerror-reset
RYOW onError no longer resets a transaction when it gets a non-retrya…
2019-03-15 10:46:28 -07:00
A.J. Beamon 19d27b6271 Merge branch 'master' into fix-compiler-warnings 2019-03-15 10:35:36 -07:00
A.J. Beamon 85b3f11e71 Fix various compiler warnings 2019-03-15 10:34:57 -07:00
A.J. Beamon adbdbf91a3 Merge branch 'master' into fix-ryow-onerror-reset
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2019-03-14 15:45:34 -07:00
A.J. Beamon 59b41b03ed Merge branch 'master' into fix-threadsafedatabase-safety
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
2019-03-14 14:50:14 -07:00
A.J. Beamon e12d242d18 Address review comments. 2019-03-14 13:42:03 -07:00
Balachandar Namasivayam 2ac07fe7e0
Merge pull request #1248 from satherton/feature-backup-json
JSON output options for fdbbackup status and describe
2019-03-14 13:41:28 -07:00
A.J. Beamon bf170493c6 RYOW onError no longer resets a transaction when it gets a non-retryable error and the transaction is not already in an error state (as of API version 610). 2019-03-14 11:40:51 -07:00
Meng Xu 5a10bf5dfc Merge branch 'master' into mengxu/tls-switch-status-PR 2019-03-14 10:35:12 -07:00
A.J. Beamon cab284027d Be more careful about accessing ThreadSafeDatabase's db member. Preallocate the db on the calling thread so that other callers can access the pointer immediately. 2019-03-14 10:26:22 -07:00
Stephen Atherton c6edcc7f06 Added schema version string to backup JSON status docs. Bug fix in backup status JSON, the document was being created outside the transaction retry loop so retries would combine partial element sets across all tries into the result. 2019-03-14 02:10:14 -07:00
Meng Xu e30e2af1f3 ClientKnobs: Add CHECK_CONNECTED_COORDINATOR_NUM_DELAY 2019-03-13 16:54:56 -07:00
Steve Atherton dbacfcbc82
Merge branch 'master' into feature-backup-json 2019-03-13 13:30:45 -07:00
Evan Tschannen a2108047aa removed LocalitySetRef and IRepPolicyRef typedefs, because for clarity the Ref suffix is reserved for arena allocated objects instead of reference counted objects. 2019-03-13 13:14:39 -07:00
Evan Tschannen e068c478b5 merge master 2019-03-12 18:31:25 -07:00
Steve Atherton 8aab719c22
Merge branch 'master' into feature-backup-json 2019-03-12 18:23:16 -07:00
Evan Tschannen a7e45cff91
Merge pull request #1176 from jzhou77/ratekeeper
Make Ratekeeper a separate role
2019-03-12 15:58:59 -07:00
Meng Xu 85c24b0067 Merge branch 'master' into mengxu/tls-switch-status-PR 2019-03-12 15:20:54 -07:00
Meng Xu 22f5624494 TLS Status: Reduce cluster controller load
When the coordinator changes, we use delayedAsyncVar() to reduce
the frequency for cluster controller to send the updated connectedCoordinatorsNumDelayed
to clients.
This help reduce the cluster controllers workload
2019-03-12 15:08:10 -07:00
Evan Tschannen c5a18945b6
Merge pull request #1260 from vishesh/task/tls-upgrade
Allows cluster string to contain coordinators with different TLS states
2019-03-12 13:45:08 -07:00
A.J. Beamon a25e224cda
Merge pull request #1213 from etschannen/feature-metadata-version
Added a metadata version key
2019-03-12 13:36:33 -07:00
Evan Tschannen 32cf742c1d api versioning is unnecessary for the metadataVersionKey 2019-03-12 11:08:40 -07:00
Stephen Atherton bc0b2aa040 Merge branch 'release-6.0' of https://github.com/apple/foundationdb
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/BlobStore.actor.cpp
2019-03-12 04:49:12 -07:00
Stephen Atherton f0eae0295f Merge branch 'master' of https://github.com/apple/foundationdb into feature-backup-json 2019-03-12 03:35:03 -07:00
Stephen Atherton c1745b9061 Bug fix, backup snapshot dispatch shardsBehind calculation would divide by zero if the snapshot scheduled interval is 0. 2019-03-11 19:43:59 -07:00
Balachandar Namasivayam 880e8643d1 Fix Windows link errors 2019-03-11 17:49:03 -07:00
Meng Xu 46f4b02807 TLS Status: Resolve review comments
Use connectedCoordinatorsNumDelayed to reduce the load on cluster controller;
Set connectedCoordinatorsNum to null by default for monitorLeader()
2019-03-11 17:10:08 -07:00
Alec Grieser 1ac8ecf55e
Merge pull request #1187 from ryanworl/bindings/go-tuple-layer-versionstamps
Add Versionstamp support to the Go Tuple layer
2019-03-11 12:36:47 -07:00
Ryan Worl d90da27ee5 Add Go to list of supported bindings for `set_versionstamped_key` in the Tuple layer. 2019-03-11 14:33:32 -04:00
Meng Xu 435e515985 Merge branch 'master' into mengxu/tls-switch-status-PR 2019-03-11 11:17:40 -07:00
Stephen Atherton f0024c876e Changed backup paused JSON field name to be more precise and match the human readable status version. 2019-03-11 01:42:19 -07:00
Stephen Atherton f2953db7d8 Added updating of backup snapshot shards behind in snapshot dispatcher so status can determine if a snapshot is lagging the configured speed. 2019-03-11 01:25:51 -07:00
Evan Tschannen 80c3f2f8e2 added status fields detailing which processes are degraded, and also the total number of degraded processes 2019-03-10 22:58:15 -07:00
Evan Tschannen 98bf4ddf28 watching the metadataVersionKey does not require ACCESS_SYSTEM_KEYS, this is not api versioned, as it is harmless to allow this for older api versions 2019-03-10 21:19:35 -07:00
Evan Tschannen 2627bcd35e Merge branch 'master' into feature-metadata-version 2019-03-10 21:13:28 -07:00
Stephen Atherton 023bbb566f Renamed backup state enums for clarity, added backup state names. Changed Epochs to EpochSeconds in backup JSON along with some other renaming/moving of fields, and added information about snapshot dispatch. Changed timestamp format for input/output in all backup/restore contexts to be a fully qualified time with timezone offset. Added information about the last snapshot dispatch to backup config and status (not yet populated). 2019-03-10 16:00:01 -07:00
Balachandar Namasivayam b9acc9a0e8 Code refactor to fix windows msvc14 compiler errors. 2019-03-08 15:13:11 -08:00
Steve Atherton dcb3ce7848
Merge branch 'release-6.0' into fdbbackup-modify 2019-03-07 23:07:17 -08:00
Meng Xu 7f405130bb Change a typo in Cluster init function 2019-03-07 16:06:19 -08:00
Jingyu Zhou cdfe906c30 Data distributor pulls batch limited info from proxy
Add a flag in HealthMetrics to indicate that batch priority is rate limited.
Data distributor pulls this flag from proxy to know roughly when rate limiting
happens.

DD uses this information to determine when to do the rebalance in the background,
i.e., moving data from heavily loaded servers to lighter ones. If the cluster is
currently rate limited for batch commits, then the rebalance will use longer
time intervals, otherwise use shorter intervals. See BgDDMountainChopper() and
BgDDValleyFiller() in DataDistributionQueue.actor.cpp.
2019-03-07 13:16:20 -08:00
Jingyu Zhou 7340998261 Fix status message for ratekeeper 2019-03-07 13:16:20 -08:00
Balachandar Namasivayam 9e4c780baa
Merge pull request #1249 from xumengpanda/mengxu/status/teamcollection-info
Status:healthy: Add optimizing_team_collections
2019-03-07 11:44:24 -08:00
Stephen Atherton 06c11a316d Normalized timestamp to text format across backup and restore tooling. Added epochs field to JSON objects describing versions and timestamps in backup status and describe output, renamed some fields for clarity. 2019-03-06 22:34:25 -08:00
Stephen Atherton 1399aee532 Added --json option to fdbbackup status. 2019-03-06 21:32:46 -08:00
Balachandar Namasivayam 43c290b546 Address review comments. 2019-03-06 17:39:29 -08:00
Balachandar Namasivayam f3391ea413
Merge pull request #1240 from satherton/feature-restore-by-timestamp
Restore by timestamp
2019-03-06 16:21:06 -08:00
Vishesh Yadav ed49d603a0 Allows cluster string to contain coordinators with different TLS states
During live TLS upgrades, we can hence switch one coordinator at a time
to TLS than all of them together.
2019-03-06 16:05:10 -08:00
Meng Xu 845f8fdcbc Status:healthy: Add optimizing_team_collections
Change removing_redundant_teams status name to
optimizing_team_collections.
The new name is more general and can be applied in the future
when we switch storage engines.
2019-03-06 15:05:23 -08:00
Stephen Atherton ca8bbad657 Added --json option to fdbbackup describe. Also added expired percentage indicator to snapshot details. 2019-03-06 14:14:06 -08:00
Meng Xu 04880e3d4d Merge branch 'master' into mengxu/tls-switch-status-PR 2019-03-06 13:41:16 -08:00
Stephen Atherton 7778112f6a Bug fix, restore was using the destination cluster to look up timestamps when printing the backup description instead of (optionally) the original cluster which generated the backup. Made missing cluster file errors more clear. 2019-03-06 02:45:55 -08:00
Alex Miller c6a65389ae Remove noexcept macro and replace with BOOST_NOEXCEPT.
BOOST_NOEXCEPT does what the noexcept macro was supposed to do, but in a
way that is correctly maintained over time.
2019-03-05 22:06:12 -08:00
Alex Miller af617d68e6 boost 1.52.0 -> 1.67.0 in all vcxproj files 2019-03-05 22:06:12 -08:00
Meng Xu 820548223a Status: connected_coordinators misc minor changes
Change the rst document file;
Change the coding style to be consistent with the nearby code;
Ensure we always initilize the connectedCoordinatesNum to 0
even when the variable is not used.
2019-03-05 21:45:18 -08:00
Meng Xu b7a52e81e2 Status: Count connected coordinators per client
A client will always try to connect all coordinators.
This commit let Status track the number of connected coordinators
for each client.

This allows us to do canary in coordinators. For example,
when we switch from non-TLS to TLS, we can switch 1 coordinator
from non-TLS to TLS. This can help check if a client has the ability
to connect through TLS.
We can make the non-TLS to TLS switch for each coordinators
one by one. This avoid the risk of losing connection in the switch.
2019-03-05 21:21:23 -08:00
Stephen Atherton 87d0c5bae0 Bug/usability fix: The output URLs of fdbbackup list were meant to be directly usable for backup management operations but they were missing the bucket URL parameter. 2019-03-05 21:14:21 -08:00
Evan Tschannen 82d957e0bb
Merge pull request #1178 from vishesh/task/issue-963-IPv6
IPv6 Support
2019-03-05 17:14:16 -08:00
Stephen Atherton 5980276aea An HTTP RequestID/ResponseID mismatch is logged as only a warning if the response code is 5xx and the responseID is empty. 2019-03-05 16:51:52 -08:00
Steve Atherton 21f55e1878
Merge pull request #1190 from bnamasivayam/restore-multiple-ranges
Add support for restoring multiple ranges.
2019-03-05 10:15:55 -08:00
Stephen Atherton f6559bbb11 Added help and documentation for fdbbackup modify. Add BackupUID and more parseable BackupURL to fdbbackup status output. 2019-03-05 05:13:48 -08:00
Stephen Atherton d3377722d5 Added blob store Backup URL parameter 'header' which enables addition of custom HTTP header fields to blob store HTTP requests. Added 'fdbbackup modify' command line tool for changing the backup URL and parameters, default snapshot interval, and/or current snapshot interval of a running backup. 2019-03-05 04:00:11 -08:00
Evan Tschannen f1897f3eb6 Merge branch 'master' into feature-metadata-version
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
2019-03-04 21:06:16 -08:00
Evan Tschannen 6d8aa20ae3
Merge pull request #1220 from bnamasivayam/dr-switchover-checks
Add some basic checks before doing an atomic switchover.
2019-03-04 20:57:50 -08:00
Evan Tschannen 988add9fb5 cache the metadataVersion for commits, so that doing setVersion with the commit version of a different transaction will 2019-03-04 16:48:34 -08:00
Trevor Clinkenbeard cbc5df5e93 Removed unnecessary Trace line 2019-03-04 16:31:40 -08:00
Balachandar Namasivayam a258df32f6 Skip switchover checks for force option. 2019-03-04 15:58:36 -08:00
Trevor Clinkenbeard 56ae46f89e Client lazily fetches health metrics from proxies 2019-03-04 14:16:39 -08:00
Vishesh Yadav e93cd0ff21 Add some checks and comments to IPv6 changes #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 41d18db7b9 fix: update the encoding of AddressExclusion in SystemData #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 592e224155 net: add/use formatIpPort to format IP:PORT pairs #963 2019-03-04 14:12:45 -08:00
Vishesh Yadav 57832e625d net: Support IPv6 #963
- NetworkAddress now contains IPAddress object which can be either
IPv4 or IPv6 address. 128bits are used even for IPv4 addresses,
however only 32bits are used when using/serializing IPv4 address.

- ConnectPacket is updated to store IPv6 address. Backward compatible
with old format since the first 32bits of IP address field is used
for serialization of IPv4.

- Mainly updates rest of the code to use IPAddress structure instead
of plain uint32_t.

- IPv6 address/pair ports should be represented as `[ip]:port` as per
convention. This applies to both cluster files and command line
arguments.
2019-03-04 14:12:41 -08:00
Evan Tschannen 075fdef31a Merge branch 'master' into feature-metadata-version
# Conflicts:
#	fdbclient/DatabaseContext.h
2019-03-03 22:58:45 -08:00
Evan Tschannen 5231ee6da0 make sure the version of a cluster is larger than the restore version, so future version stamps are larger than restored version stamps 2019-03-03 21:34:21 -08:00
Trevor Clinkenbeard 1576d50ee9 Removed unnecessary Trace line 2019-03-03 17:09:03 -08:00
Trevor Clinkenbeard 015997f2f9 Simplified updateHealthMetricsActor 2019-03-02 18:01:23 -08:00
Trevor Clinkenbeard 39f612d132 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-03-02 17:07:00 -08:00
Trevor Clinkenbeard 6c45ffc56f Added exception handling to setNetworkOption
Handle non-integer option values for sendDetailedHealthMetrics option by
throwing invalid_option_value
2019-03-02 15:51:51 -08:00
Evan Tschannen c1de93f467 fix: binary search could get stuck in an infinite loop
fix: avoid picking a read version which could be before the last real commit
fix: we must wait on metadataVersionKey in case it is not already cached
fixed review comments
2019-03-02 13:55:41 -08:00
Evan Tschannen 9d53305516
Make GetHealthMetricsRequest constructor explicit
Co-Authored-By: tclinken <trevorclinkenbeard@gmail.com>
2019-03-02 08:30:10 -08:00
Evan Tschannen de577a6ca9
Modified GetHealthMetricsReply constructor
Co-Authored-By: tclinken <trevorclinkenbeard@gmail.com>
2019-03-02 08:29:16 -08:00
Evan Tschannen 3a91fb287b
Use const HealthMetrics& parameter in update
Co-Authored-By: tclinken <trevorclinkenbeard@gmail.com>
2019-03-02 08:17:20 -08:00
Meng Xu 94385447bc Status: Get if client configured TLS
To understand if all clients have configured TLS,
we check the tlsoption when a client tries to open database.
This is similar to how we track the versions of multi-version clients.
2019-03-01 15:17:01 -08:00
Balachandar Namasivayam 7cf71b0931 Add some basic checks before doing an atomic switchover. 2019-03-01 14:49:04 -08:00
A.J. Beamon 2af99080ac Merge branch 'master' into ratekeeper-batch-priority-limits 2019-03-01 13:02:07 -08:00
Evan Tschannen 2168b14834 Merge branch 'master' into feature-metadata-version 2019-02-28 17:45:55 -08:00
Evan Tschannen 3da85f3acd implemented the \xff/metadataVersion key, which can be used by layers to help them cheaply cache metadata and know when their cache is invalid 2019-02-28 17:45:00 -08:00
Stephen Atherton 7d287c6999 Merge branch 'release-6.0'
# Conflicts:
#	fdbclient/FileBackupAgent.actor.cpp
2019-02-28 14:01:00 -08:00
A.J. Beamon e2bcecb08f Merge branch 'master' into ratekeeper-batch-priority-limits 2019-02-28 12:52:44 -08:00
A.J. Beamon 3e6a6a6569 Update status schema for correctness. Send the count of batch transactions started back to ratekeeper so that it can be logged with other ratekeeper metrics. 2019-02-28 12:00:58 -08:00
A.J. Beamon af69ba035a
Merge pull request #1200 from bnamasivayam/master
User provided Client Transaction id merging
2019-02-28 11:16:44 -05:00
Stephen Atherton 887856b6b0 Bug fix in AsyncFileReadAhead where a file size that is an integer multiple of the read chunk size will cause a crash when reading the file's final block. BackupContainerLocalDirectory now uses AsyncFileReadAhead in simulation to get simulation coverage of that class, and FileBackup will generate file sizes which expose the bug. 2019-02-28 00:22:38 -08:00
Trevor Clinkenbeard d2bde4e55b Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-27 16:30:25 -08:00
Trevor Clinkenbeard 3f59f82670 Calculate durabilityLag instead of NDV for health metrics 2019-02-27 16:30:01 -08:00
Evan Tschannen 934e0457e9 Merge branch 'master' of github.com:apple/foundationdb
# Conflicts:
#	fdbclient/DatabaseConfiguration.cpp
2019-02-27 15:21:06 -08:00
Evan Tschannen fb76cf6738 fix: prevent log_anti_quorum from being set to an invalid number so that we can better handle upgrades from configurations which may have set the value incorrectly 2019-02-27 15:16:49 -08:00
Evan Tschannen 51e6b1dda0
Merge pull request #1197 from ajbeamon/avoid-transient-errors-in-incorrect-cluster-file-contents-detection
Improve avoidance of transient issues when logging IncorrectClusterFileContents...
2019-02-27 14:44:45 -08:00
Evan Tschannen fafb66b0a8
Merge pull request #1126 from bnamasivayam/ratelimit-consistencycheck
Dynamically rate limit consistency check.
2019-02-27 14:43:09 -08:00
Balachandar Namasivayam cc0aac588a Merge branch 'client-transaction-id'
* client-transaction-id:
  Add user provided transaction id as part of the format in ClientTransactionProfileCorrectness workload.
  Addressed review comments.
  Apply suggestions from code review
  Add support for client provided transaction identifier to be logged as part of trace logging or transaction profiling.
2019-02-27 13:11:15 -08:00
Balachandar Namasivayam 7d7db54515 Addressed review comments. 2019-02-27 12:42:45 -08:00
A.J. Beamon 81e7c4ea4e
Apply suggestions from code review
Co-Authored-By: bnamasivayam <36455962+bnamasivayam@users.noreply.github.com>
2019-02-27 11:41:33 -08:00
A.J. Beamon 999ee68609 Improve avoidance of transient issues when logging IncorrectClusterFileContents SevWarnAlways events by making it time based. 2019-02-27 10:08:24 -08:00
Evan Tschannen 8afb7fbb9d
Merge pull request #1160 from alexmiller-apple/tstlog-fork
Spill-By-Reference TLog Part 2: New and Old TLogServers co-exist harmoniously
2019-02-26 18:00:04 -08:00
Alex Miller 2dc57568cb Change many things about log_version.
* log_version in the database (`/conf/log_version`) is now a hint that gets
  rounded to the nearest supported version.
* fdbcli and FDB enforce that only a valid log_version can be configured to
* TLogVersion is persisted in CoreTLogSet (and LogSet and TLogSet)
* Some comments here and there
* Add an assert on filename length to make sure KV-pairs in filename
  don't exceed a maximum length.
2019-02-26 16:47:04 -08:00
Balachandar Namasivayam 7eba50b086 Add support for restoring multiple ranges. 2019-02-25 18:00:28 -08:00
Trevor Clinkenbeard abfe057805 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-25 13:47:16 -08:00
Trevor Clinkenbeard ff9a7cb2f1 Combined proxy health metrics replies into single message type 2019-02-23 10:13:43 -08:00
Evan Tschannen b8910ba7cd Merge branch 'master' into feature-fix-force-recovery
# Conflicts:
#	fdbclient/ManagementAPI.actor.h
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KillRegion.actor.cpp
2019-02-22 14:38:13 -08:00
Evan Tschannen d008de576e
Merge pull request #1139 from xumengpanda/mengxu/machine-team-upgrade-PR
Add background actor to remove redundant teams
2019-02-22 14:22:07 -08:00
Trevor Clinkenbeard edc0c5bf2b Client resends health metrics request when proxies change 2019-02-22 12:30:36 -08:00
Alex Miller 6d23eb2d1a Implement log_version.
This mega-commit introduces a new configuration setting, `log_version`,
that controls the TLog implementations and features that are available
within FDB, so that users can opt in to new features if they're willing
to sacrifice backwards compatibility.
2019-02-22 12:15:23 -08:00
Meng Xu 64db109f20 Status: Add schema for the new data distributor role 2019-02-22 10:05:12 -08:00
Trevor Clinkenbeard 2a20ea6ac5 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-22 09:41:47 -08:00
Meng Xu 9445ac0b0c Status: Use new data distributor worker to publish status
After we add a new data distributor role, we publish the data
related to data distributor and rate keeper through the new
role (and new worker).

So the status needs to contact the data distributor, instead of master,
to get the status information.
2019-02-21 18:05:50 -08:00
Alex Miller b6f7821ae4 Actually compile ManagementAPI.actor.cpp on windows.
<EnableCompile>false should only be applied to .actor.h files, so that
we don't try to compile the resulting .h file.

Apparently.
2019-02-21 16:28:14 -08:00
Trevor Clinkenbeard fb925f8ca6 Improved Throttling workload
Test now fails if client health metrics stop updating. Added SevError
trace lines for different failure cases. Also fixed bug so that
(detailedWorstDiskUsage == 0) causes test failure when detailed health
metrics are sent.
2019-02-21 15:50:17 -08:00
Meng Xu db19b08762 TeamRemover: Add new status to fdbcli
Add the healthy_removing_redundant_teams status to fdbcli
2019-02-21 15:03:32 -08:00
Evan Tschannen 329ab766f1 factored out a duplicate code block
attempted to fix a compiler error
2019-02-20 18:20:10 -08:00
Trevor Clinkenbeard fa96b8dd33 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-20 16:56:16 -08:00
Trevor Clinkenbeard 1bb384db4d Merge branch 'master' of https://github.com/apple/foundationdb into add-no-assign-class 2019-02-20 13:13:12 -08:00
Trevor Clinkenbeard a20f5482bc Created StorageStats struct to combine health metrics for storage servers 2019-02-20 11:57:41 -08:00
Alex Miller d48cf3ec57 Make TLogSpillType cleaner now that it's not used in worker. 2019-02-19 22:30:15 -08:00
Alex Miller bf8bfb8137 Set log_spill in SimulationConfig.
Which also revealed that it needed to be added to the schema.
2019-02-19 22:30:15 -08:00
Alex Miller 0cf3ee9f99 Add a way to configure log spill type. 2019-02-19 22:18:10 -08:00
Evan Tschannen 3a572b010f fix: a forced recovery needed to force the data distributor to restart 2019-02-19 16:04:52 -08:00
mpilman 4be90c91d6 Moved THIS and THIS_ADDR to actorcompiler.h 2019-02-19 15:16:59 -08:00
mpilman 62bd677cc9 fix old build system 2019-02-19 15:16:59 -08:00
Alex Miller f3cd9f15ca Update fdbclient/FileBackupAgent.actor.cpp
Co-Authored-By: mpilman <markus@pilman.ch>
2019-02-19 15:16:59 -08:00
Alex Miller 82cfb94f4a Update fdbclient/FileBackupAgent.actor.cpp
Co-Authored-By: mpilman <markus@pilman.ch>
2019-02-19 15:16:59 -08:00
mpilman da67067a07 Make unnecessary state variable a local variable 2019-02-19 15:16:59 -08:00
mpilman f79a9594c1 Several bugfixes to make fdb build on non-ide 2019-02-19 15:16:59 -08:00
mpilman 7fa7db3251 Minor bug fix - fdbclient now compiling 2019-02-19 15:16:59 -08:00
mpilman 0bb60e5a3b Use proper fwd decl in NativeAPI
Also NativeAPI.h -> NativeAPI.actor.h
2019-02-19 15:16:59 -08:00
mpilman c1592b4c3a Several minor fixes within fdbclient 2019-02-19 15:16:59 -08:00
mpilman 70e089e791 Define THIS_ADDR for use with intellisense
this is undefined within ACTORs if compiled with
intellisense. We use a macro here to make sure just
replaces this with 0
2019-02-19 15:16:59 -08:00
mpilman 78dd80ea8a Proper fwd decl in BackupAgent
Also BackupAgent.h -> BackupAgent.actor.h
2019-02-19 15:16:59 -08:00
mpilman 3cb2391b58 use proper fwd declarations in ManagementAPI
Also ManagementAPI.h -> ManagementAPI.actor.h
2019-02-19 15:16:59 -08:00
mpilman 479a4d7c22 Minor fixes in fdbclient for intellisense 2019-02-19 15:16:59 -08:00
Evan Tschannen ed9e20ce17 forgot to fix merge conflicts 2019-02-18 17:09:55 -08:00
Evan Tschannen 065a45e05f Merge branch 'master' into feature-fix-force-recovery
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/workloads/KillRegion.actor.cpp
2019-02-18 17:09:06 -08:00
Evan Tschannen 562b315744 fix: The anti quorum cannot be more than half of the replication factor, or the log system will continue to accept commits when a recovery is impossible 2019-02-18 15:22:32 -08:00
Evan Tschannen 8f2af8bed1 fix: forced recoveries now require a target dcid which will become the new primary location. During the forced recovery, the configuration will be changed to make that location primary, and usable_regions will be set to 1. If the target dcid is already the primary location, the forced recovery will do nothing. This makes forced recoveries idempotent, so it is safe to the client to re-send forced recovery commands to the cluster controller.
fix: the cluster controller attempts to do a commit to determine if the cluster is alive, since its own internal recoveryState might not be up-to-date.

fix: forceMasterFailure on the cluster controller did not always cause the current master to be re-recruited
2019-02-18 14:54:28 -08:00
Evan Tschannen 4c35ebdcc6 fix: because of forced recoveries, storage servers in remote regions cannot update their durable version to (lastLogVersion - 5e6), because the lastLogVersion might have jumped due to an epoch end and the recovery version after the forced recovery could be before the epoch end, causing the storage server to want to rollback to a version it does not have on disk 2019-02-18 14:40:30 -08:00
Evan Tschannen 05ca0a10d8 fix: kill all storage servers which are not in the safe locality after a forced recovery 2019-02-18 14:30:51 -08:00
Vishesh Yadav e05b53d755 Merge remote-tracking branch 'apple/master' into task/tls-upgrade 2019-02-15 20:37:07 -08:00
Evan Tschannen 83060c6e56
Merge pull request #1062 from jzhou77/PR
Add a new DataDistributor role.
2019-02-15 13:51:27 -08:00
mpilman 75f692b931 simplify actorcompiler and target to compile coveragetool 2019-02-15 00:01:42 -08:00
Evan Tschannen abc3c01fb2 Update fdbclient/SystemData.cpp
Co-Authored-By: jzhou77 <jingyuzhou@gmail.com>
2019-02-14 16:37:16 -08:00
Evan Tschannen 107b361396 Update fdbclient/SystemData.h
Co-Authored-By: jzhou77 <jingyuzhou@gmail.com>
2019-02-14 16:37:16 -08:00
Jingyu Zhou 6a655143e8 A follow-on fix for config key usage
And some trace event cleanups.
2019-02-14 16:37:16 -08:00