0d08fcf1d3
removed old node.js bindings
...
issue: Node.js Bindings #129
2018-04-06 17:16:17 -07:00
8d424a5097
javadoc prose fixes
2018-03-21 19:25:36 -07:00
fd5763efa4
fixed all the warnings making javadocs
2018-03-19 18:45:21 -07:00
91fa01a5a6
Updates URLs and email addresses in documentation and package config.
2018-03-15 16:30:29 -07:00
3cac430f6f
Merge pull request #58 from satherton/more-precise-backup-expire-safety
...
Clarity improvement, resetting backup description variable because it…
2018-03-09 12:06:24 -08:00
9c901983f0
Clarity improvement, resetting backup description variable because it's no longer valid due to some of its contents being std::move'd.
2018-03-09 12:03:10 -08:00
f5d85288eb
Merge pull request #42 from satherton/more-precise-backup-expire-safety
...
Backup expiration safety logic improvement
2018-03-09 11:45:49 -08:00
3a7288924a
Bug fixes. During expiration, the backup container's log range metadata could be updated incorrectly if force was required and not specified or if a backup had no log begin metadata and an expire was done which covered 1 or more log file. In the latter case a backup could be left in a state where the container metadata suggests the backup has more log coverage than it actually does.
2018-03-09 11:29:23 -08:00
ae7d8e90b2
Merge branch 'release-5.1' of github.com:apple/foundationdb into release-5.1
2018-03-09 09:56:09 -08:00
5390af8be4
suppress spammy logs
2018-03-09 09:40:36 -08:00
1aa32291f3
Fix line endings of backported changes.
2018-03-08 13:14:15 -08:00
bb8f48313f
Declare each state variable on its own line...
2018-03-08 12:43:44 -08:00
d136fc8eea
Improve fdbbackup status
...
Additional details added are
log bytes written
range bytes written
last snapshot version and its corresponding timestamp
last log version and its corresponding timestamp
If backup is supposed to stop at the next snapshot completion (stopWhenDone)
version/timestamp of snapshot start and expected end
Backup error reporting format changed to
Recent Errors (since latest restore point TIME ago)
<TIME> ago: <error description> on <task name>
Older Errors
<TIME> ago: <error description> on <task name>
TIME format:
<= 59 seconds ago
<= 59.99 minutes ago
<= 23.99 hours ago
N.NN days ago
2018-03-08 12:43:33 -08:00
4f58bca66a
Simple refactor of code...
2018-03-08 11:34:25 -08:00
1c1a497ea2
Refactor getKeyServers to be more readable.
...
Fix possible memory corruption by returning KeyRange instead of KeyRangeRef in getKeyServers.
Simplify getMasterProxies on DatabaseContext class.
2018-03-08 11:34:18 -08:00
cb68885328
If backup expiration determines that force is required but the force parameter is not set, it will no longer throw an error unless the backup contains data from prior to the expire_before_version.
2018-03-08 11:27:15 -08:00
dcf5b2e35d
All readCommitted() functions now use Transaction instead of ReadYourWritesTransaction to reduce memory consumption in Backup and DR. Also removed one readCommitted() variant as it is just a special case of another definition.
2018-03-07 13:56:34 -08:00
0bae9880f1
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-21 10:25:11 -08:00
a9a9590058
Merge branch 'release-5.1' of github.com:apple/foundationdb into release-5.1
2018-02-16 19:44:36 -08:00
54fc81b260
Improved backup error reporting in backup status. The most recent error for each error type is reported along with how long ago the error occurred, and errors are divided into two categories based on whether or not they occurred since the most recent backup progress.
2018-02-16 19:38:31 -08:00
8c53483838
fix: log ranges were not being cleared correctly
2018-02-16 10:27:10 -08:00
d8879dc3f3
HTTP::doRequest() now reads responses in parallel with sending requests, so if the server responds before receiving all of the the request the client can stop sending the remainder of the request. For PUT requests which upload files, this prevents sending potentially several megabytes of unnecessary bytes if the server responds with an error (such as 429) before the request is completely sent. Updated the backup container unit test to use more parallelism in order to test this new behavior.
2018-02-07 10:38:31 -08:00
0792d5e3dd
Fix: last restorable version for a backup tag name (a separate value from the latest restorable version for a configured backup) was not being updated.
...
Fix: backup blob speed was sometimes an error because the JSON $sum merge operator did not support mixed numeric types.
Fix: JSON merge operator handling was squashing errors in some cases, which was generally obscuring the backup speed metric issue.
Cleaned up some of the JSON object merging logic.
Improved error messages in JSON merge operators. Added JSON merge operator tests for mixed numeric math and improved readability of test output.
2018-02-06 13:44:04 -08:00
080a454051
fix: getVersionstamp would return broken promise if a transaction was disposed before being set. getAddressesForKey would not return when resetPromise was set.
2018-01-31 13:47:36 -08:00
84f7565d04
MultiVersionApi::createCluster was throwing network_not_setup rather than returning it.
2018-01-31 10:03:09 -08:00
2f291d8955
Bug fix in blob backup container deletion. The list/delete loop could end before deleting all of the files, but the index entry would still be deleted. Also preemptively made the same code change in listBucket() - Although it is technically correct as written it is a dangerous style because it is not obvious that the addition of a wait() call in the second 'when' block would create a bug. Consolidated deleteContainer() and deleteBucket() as they differ by only 1 line.
2018-01-29 00:32:41 -08:00
4dec5423f7
Optimization in backup snapshot dispatching. If the next dispatch version is not ahead of the recently known current version then do not set a scheduled time for new range tasks in order to avoid the overhead and delay of task timeout handling. Adjusted task timeout knobs to avoid large transaction warnings. Removed backtrace() from LargeTransaction trace event. Tweaked suppression in backup trace events.
2018-01-24 17:40:02 -08:00
95d4e5520b
Added TraceEvent.
2018-01-24 12:44:37 -08:00
aebbe1dcfd
Changed core_versionspersecond knob to int64_t type to avoid integer overflow. Cleaned up backup TraceEvent suppression. Added backtrace to LargeTransaction TraceEvent to make it easier to find the source of large commits in applications using NativeAPI directly.
2018-01-24 11:59:37 -08:00
83409fb067
Bug fix, versionFolderString() was not reducing the precision of the number in the output string. Not technically a 'bug' as the scheme will still work but produces an overly deep and sparse folder structure.
2018-01-24 10:29:37 -08:00
40d38880fe
Changed version-based folder naming scheme to something simpler, a fixed width 0-padded 19 digit number (the longest a Version can be) with /'s inserted to limit the size of each folder level. Comparisons using these folder names ignore the /'s so any future change to the splitting scheme would still be compatible with the current listing/reading logic.
2018-01-23 15:02:15 -08:00
7db7a51440
Changes to backup folder structure in BackupContainerBlobStore at the top level inside the backup bucket. All data files now live under data/<backup_name> and there is an 'index' at backups/<backup_name> which indicates what backups exist. Backup names can now contain '/' characters.
2018-01-23 11:46:16 -08:00
7f0b7311b9
Corrected function name to timeKeeperVersionFromDatetime(). 'Fdbbackup expire' now allows an expire_before version of 0 if explicitly passed by version or by timestamp.
2018-01-23 00:19:51 -08:00
51a1bd9327
Timekeeper lookup improvements, moved both function declartions to BackupContainer.h. VersionFromEpochs() now uses versions/sec to adjust the lookup result to improve accuracy. Conversions in both directions look for the latest record less than the target conversion value, but failing that they will now fall back on any available data point and adjust from there using versions/sec.
2018-01-22 23:57:01 -08:00
f086ba9d9d
Improved version to timestamp lookup - if there are no older versioned records in the database then the next available record, if any, will be used to calculate a result.
2018-01-22 22:47:57 -08:00
1f59f9ee5b
Reduce restore dispatch transaction size.
2018-01-22 15:04:14 -08:00
16cd0c8f75
Change WaitStorageMetricsPenalty to not be SevWarnAlways unless it happens multiple times in a row (about 2.5 minutes with current knobs).
2018-01-19 16:41:15 -08:00
0ca8b612c7
fix: do not set onDone until all addTaskFutures have had a chance to join that future
2018-01-19 16:29:42 -08:00
9d5f7bd5ab
Aborting an old incompatible backup did not work if the old backup was never actually initialized, started, or was already aborted.
2018-01-19 15:08:03 -08:00
6e96d3c30c
Bug fix, backup snapshots could take unexpectedly long if the desired snapshot interval is less than the configured snapshotDispatch interval.
2018-01-19 12:14:04 -08:00
da02099c4c
Merge branch 'release-5.1' of github.com:apple/foundationdb into release-5.1
2018-01-19 11:02:48 -08:00
fc16bb94ab
Discontinuing a backup that is already restorable now stops immediately and aborts (via validation key) any tasks scheduled to run later.
2018-01-19 11:02:43 -08:00
570f72ba40
fix: nextDispatchVersion was being set too large if the snapshot interval was small
2018-01-19 10:53:58 -08:00
b6dd06d945
Bug fix in version to timestamp conversion.
2018-01-18 02:54:12 -08:00
307e04c0ad
Updated backup container unit test to match new safer behavior of expireData(). Rewrote BackupContainerLocalDirectory::deleteContainer() to actually delete the whole directory but only if it appears to be a backup with either log or snapshot data.
2018-01-18 00:36:28 -08:00
cdd1e784dc
Added yields to writing backup snapshot manifests to avoid slow tasks.
2018-01-17 13:28:56 -08:00
f6f0816bc1
Merge branch 'release-5.1' of github.com:apple/foundationdb into release-5.1
2018-01-17 12:12:12 -08:00
8fece71662
Bug fix in backup metadata handling if logEnd becomes less than logBegin, which can happen if an expire is done without logEnd being updated.
2018-01-17 12:12:04 -08:00
d7f8fe218a
Bug fix, resolving versions to timestamps for use in backup descriptions did not work on a locked database. Added some trace events to AtomicRestore to show progress.
2018-01-17 12:03:19 -08:00
4bfbdbf454
Extract getLocalTime to platform.cpp
2018-01-17 11:35:34 -08:00
Alec Grieser