278e7de6f5
added more documentation
2021-01-04 16:59:08 -05:00
a0d33c5d03
added more comments
2021-01-04 16:00:12 -05:00
53d23d95d8
initial commit in adding more documentation to special keys
2021-01-04 15:26:48 -05:00
e097c98956
Remove headed added by IDE, add comments
2021-01-04 00:18:01 -08:00
da7b65cd71
Change the interface back to return int64_t
2021-01-03 22:45:09 -08:00
e7e2439abe
Remove unnecessary copy from BackupContainerAzureBlobStoreImpl::read
2021-01-01 09:43:03 -04:00
70e62d34ca
Merge remote-tracking branch 'origin/master' into misc-changes
2020-12-28 01:58:56 -04:00
86c7c1e946
Fix IAsyncFileSystem method signatures
2020-12-28 01:57:42 -04:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
5bfa6cea98
Merge remote-tracking branch 'origin/master' into misc-changes
2020-12-26 20:47:00 -04:00
33ec968d5a
Mark expectedSize methods const
2020-12-26 18:30:44 -04:00
4e460c0b30
Remove BlobStore.actor.cpp
2020-12-26 13:30:16 -04:00
bbb56e4089
Merge branch 'release-6.2' of https://github.com/apple/foundationdb into release-6.3
2020-12-23 14:26:59 -08:00
6957e64886
fix: Calculate fault tolerance when keys are not fully replicated
...
Whem the database is not fully replicated to remote, fault tolerance
can be calculated by couting total number of tLogs replicas in primary
and satellite - 1, as we can lose all but one of those, but we were
subtracting 2 instead of 1.
2020-12-22 16:26:19 -08:00
a1809f7d86
doc: Fault Tolerance and Region Configuration
2020-12-22 16:25:17 -08:00
8d4cbfbb86
Merge branch 'release-6.2' into feature/allow-manually-trigger-dd-teams-info-logging
2020-12-22 10:00:24 -08:00
0ecfaf138e
Merge pull request #4150 from sfc-gh-ljoswiak/fixes/sps-tracing
...
Remove "a/" from tracing keys in special-key-space
2020-12-21 13:48:01 -08:00
9db9a84798
Ignore negative duration value
2020-12-21 13:29:16 -08:00
b68d84aaea
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-12-21 12:32:31 -08:00
7b4f7ce4e0
update comments
2020-12-20 22:47:55 -08:00
adc916121e
Copy the network address stringref
2020-12-20 22:44:04 -08:00
fb62ce4f64
Removed the override
2020-12-14 08:34:09 -05:00
24cdbc5cee
Fixed error with Windows CMake build
2020-12-14 08:23:24 -05:00
9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces
...
Stable interfaces
2020-12-11 09:34:27 -08:00
5f57d72a59
remove print statements and format protocol version workload
2020-12-11 04:46:20 +00:00
4cb821cd63
Merge remote-tracking branch 'upstream/release-6.2' into merge-6.2-to-6.3
2020-12-09 15:47:44 -08:00
43fd545f06
Remove a/ from tracing path in special-key-space
2020-12-09 13:40:16 -08:00
b305e71ece
Merge pull request #4141 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
...
Merge release 6.3 master
2020-12-09 08:50:57 -08:00
c8786a65c3
Remove unnecessary uses of virtual
2020-12-08 09:09:33 -08:00
276f80cfce
Mark Tag::toTagDataIndex const
2020-12-08 09:09:33 -08:00
9581bc0279
Merge pull request #4129 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
...
Merge release 6.3 master
2020-12-08 08:57:54 -08:00
54237f28ea
Throw error on commit for late trace options
2020-12-07 14:43:44 -08:00
cc669f399e
Merge remote-tracking branch 'upstream/release-6.3' into anoyes/merge-release-6.3-master
2020-12-07 22:26:11 +00:00
18201d6f4f
Add option to enable transaction tracing
2020-12-07 11:24:14 -08:00
75836d6114
Fix double counting of range reads in TransactionMetrics
2020-12-07 09:01:19 -08:00
d97523a966
Fix typo
2020-12-06 14:04:16 -08:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
e5cad0f077
Fix the bug if boundary same as range and add comments
2020-12-04 14:58:40 -08:00
76142549e0
Return keys from range read in lexicographic order
2020-12-04 11:21:39 -08:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
ac02329d7d
Added a command in fdbcli to allow user to manually trigger the detailed teams info loggings in data distributor
2020-12-03 14:42:30 -08:00
3669615e4b
Refactor and clean up
2020-12-03 14:06:11 -08:00
8d59dae26d
Add check for existence of databaseRebootWorker in c api
2020-12-03 13:47:59 -08:00
88e66545bb
Change the api to return bool instead of int
2020-12-03 12:01:32 -08:00
c77d9e4abe
merge conflicts
2020-12-02 21:53:19 +00:00
1f166bc183
Add options to set custom transaction ID and disable tracing
2020-11-30 14:57:17 -08:00
dc2bac5670
Resolve conflicts
2020-11-24 19:09:42 +00:00
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
2fae7333c0
Add duration parameter in the api
2020-11-23 14:55:29 -08:00
b46384e1b8
Initialize client and flow knobs
2020-11-22 09:56:18 -08:00
b1e3478267
Simplified global knobs
2020-11-21 13:27:48 -08:00
615029a393
Merge pull request #4082 from sfc-gh-dyoungworth/dyoungworth/merge_6_3_master
...
merge 6 3 master
2020-11-19 20:53:51 -07:00
2320f970c1
Add reboot_worker and pythong binding
2020-11-19 16:43:39 -08:00
cc4b282236
Remove the incorrect(unnecessary) versioning stuff, since the error is thrown by the server
2020-11-19 13:17:38 -08:00
da39b05620
Add comments about error type change
2020-11-18 12:14:59 -08:00
a101d3d706
Add versioning
2020-11-18 12:06:31 -08:00
a4174d3108
Change operation_failed to dd_not_found
2020-11-18 00:18:42 -08:00
aec0398fd0
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-11-17 11:39:12 -08:00
d64cf8b9e3
Merge branch 6.3 into master
2020-11-17 11:22:45 -08:00
fdf41110e5
Merge branch 'release-6.3' into dyoungworth/merge_6_2_to_6_3
2020-11-16 14:49:51 -08:00
d0391db862
Merge branch 'release-6.2' into release-6.3
2020-11-16 10:15:23 -08:00
12a6205d99
Move MetricLogger from fdbclient to fdbserver
2020-11-15 11:41:57 -08:00
82a50ea157
Improve type safety of ClientLogEvents
2020-11-14 19:22:19 -08:00
eab75d4ee1
Make enums automatically binary serializable
2020-11-14 19:22:04 -08:00
45c9a0abc7
Revert "Revert "Add limiting health metrics""
...
This reverts commit 209ebcc595
2020-11-13 17:24:57 -08:00
209ebcc595
Revert "Add limiting health metrics"
2020-11-13 17:08:46 -08:00
8c0b4dbe4c
Merge pull request #4067 from sfc-gh-tclinkenbeard/add-limiting-health-metrics
...
Add limiting health metrics
2020-11-13 16:04:44 -08:00
6c4493166f
Add limiting storage queue and durability lag to health metrics
2020-11-12 20:14:41 -08:00
222da17558
Merge branch 'release-6.2' into mengxu/ha-code-read
2020-11-12 13:39:27 -08:00
cc13ef08bd
Sort the failed sets before modifying them in attempts to make changes consistent
2020-11-12 16:26:34 -05:00
0f205620bf
Merge branch 'master' into features/udp
2020-11-12 13:51:17 -07:00
c2dd7d1d38
Remove unresolved questions
2020-11-11 22:39:11 -08:00
1343f40117
don't allow empty coments
2020-11-11 14:07:54 -07:00
bdd3dbfa7d
remove duplicates
2020-11-10 14:01:07 -07:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
51db9a7e0a
add static method to access backup pause key instead of constructing it manually
2020-11-06 14:03:29 -05:00
4788544a6f
Revise comments based on review suggestions
...
Ack. Jingyu and Xin for their suggestions.
2020-11-06 08:51:13 -08:00
2909e35415
When moving dispaching shard relocations, log the source and destination storage servers.
2020-11-05 16:13:18 -08:00
c50e997f60
Make status tests deterministic
...
This change seems to be incorrect since afaict INetwork::timer isn't
guaranteed to be monotonic. Maybe we can make that guarantee or add an
INetwork::timer_monotonic symbol?
2020-11-05 17:05:34 +00:00
1664e2ff7f
Add more comments and questions to LR tLog and loadbalance
2020-11-01 21:22:23 -08:00
9c6c2704d0
Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces
2020-10-30 23:59:32 +00:00
063700e4d6
Add comments and questions to HA and tLog code reading
...
The comments' correctness need to be confirmed by reviewers.
2020-10-30 12:14:57 -07:00
ca26cb57a2
Reapply BlobStore bug fix to S3BlobStore.actor.cpp
...
Because BlobStore.actor.cpp was renamed, this commit was lost in the
merge, so reapplying here
2020-10-30 10:06:35 -07:00
09350c760e
Merge remote-tracking branch 'origin/master' into merge
2020-10-30 08:36:19 -07:00
035708f43a
Post-merge cleanup
2020-10-29 22:35:27 -07:00
0faaecc780
Merge branch 'master' into merge
2020-10-29 22:27:40 -07:00
cf4c8e375f
Merge remote-tracking branch 'origin/release-6.3' into merge
2020-10-29 22:15:41 -07:00
55e86c1336
Disambiguate between S3BlobStore and other blob stores
2020-10-29 20:42:23 -07:00
96cc700440
Merge pull request #3961 from sfc-gh-tclinkenbeard/azure-backup
...
Add support for writing backup files directly to Azure blob storage
2020-10-29 00:30:40 -07:00
a72bb52eae
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-10-28 22:32:44 -07:00
99c1880a83
Merge commit 'f9581de2005e6b085776e81b9fcaa16442b32589' into merge-6.2-to-6.3
...
# Conflicts:
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
2020-10-27 12:21:26 -07:00
bf7bf470d9
Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces
2020-10-26 19:36:19 +00:00
5b546c4854
change protocol version to hex encoded string in status json. Move constant from flow transport header to cpp
2020-10-26 19:35:38 +00:00
a15bc32b14
Merge pull request #3945 from sfc-gh-clin/add-consistencycheck-to-special-keys
...
Add consistencycheck command to special keys
2020-10-26 10:59:17 -07:00
9501edb960
Bug fix, blob client was not correctly using keys longer than 64 bytes in request signing.
2020-10-25 21:11:04 -07:00
2a849dc52c
Update BackupContainerAzureBlobStoreImpl::WriteFile::sync
2020-10-24 16:48:06 -07:00
195873e1cc
Cleaned up unnecessary backup container includes
2020-10-24 16:48:05 -07:00
195ca30a20
Azure backup cleanup
2020-10-24 16:48:05 -07:00
e389a5dabe
Fix issues with Azure credentials
2020-10-24 16:48:05 -07:00
c6739f7075
Add skeleton for BackupContainerAzureBlobStore::listURLs
2020-10-24 16:48:05 -07:00
bb09f0e657
Improve AsyncTaskThread encapsulation
2020-10-24 16:48:05 -07:00
7cba81316a
Fix some race conditions in AsyncTaskThread (some more remain)
2020-10-24 16:48:04 -07:00
4d127a29cc
Added testing and fixed some bugs in AsyncTaskThread
2020-10-24 16:48:04 -07:00
456a7c77a2
Add random delay to AsyncTaskThread::execAsync in simulation
2020-10-24 16:48:04 -07:00
f3c52ed41e
Guard Azure backup compilation behind CMake variable
2020-10-24 16:48:04 -07:00
e3f94c80fc
Added BackupContainerAzureBlobStore::getURLFormat
2020-10-24 16:48:03 -07:00
dc3738b97f
Improve encapsulation for BackupContainerFileSystem
2020-10-24 16:48:03 -07:00
6a27619544
Remove unnecessary copies in BackupContainer code
2020-10-24 16:48:03 -07:00
1d28285cc5
Made BackupContainer.h header guard consistent with other header guards
2020-10-24 16:48:03 -07:00
ae91cf1d44
Make BackupContainerFileSystem.h a non-actor file
2020-10-24 16:48:02 -07:00
9242456d71
Improve encapsulation for BackupContainerAzureBlobStore
2020-10-24 16:48:02 -07:00
226b3f6bf2
Improve encapsulation for BackupContainerS3BlobStore
2020-10-24 16:48:02 -07:00
e1d58d3c66
Moved BackupContainerS3BlobStore into its own files
2020-10-24 16:48:02 -07:00
e15a268548
Use anonymous namespace for BackupContainerLocalDirectory helper functions
2020-10-24 16:48:02 -07:00
961aeaecac
Moved BackupContainerAzureBlobStore into its own files
2020-10-24 16:48:01 -07:00
e5338d213b
Moved BackupContainerLocalDirectory into its own files
2020-10-24 16:48:01 -07:00
3c6de425a1
Moved BackupContainerFileSystem implementation to cpp file
2020-10-24 16:48:01 -07:00
cd50d53b1e
Created BackupContainerFileSystem.actor.h
2020-10-24 16:48:01 -07:00
29d6fc5dff
Clarify scope of IBackupFile_impl namespace
2020-10-24 16:48:00 -07:00
0879e529d2
Add error handling to AsyncTaskThread::execAsync
2020-10-24 16:48:00 -07:00
0a99438d84
Update pNumDeleted in BackupContainerAzureBlobStore::deleteContainer
2020-10-24 16:48:00 -07:00
2049789a99
Remove BackupContainerAzure.actor.cpp
2020-10-24 16:47:59 -07:00
ad911f4d61
Encapsulate some methods in BackupContainerAzureBlobStore
2020-10-24 16:47:59 -07:00
72b24c36e7
Fix header for AsyncTaskThread.h
2020-10-24 16:47:59 -07:00
82b6daa16b
First draft of Azure blob storage backup container
2020-10-24 16:47:51 -07:00
309e38dc06
add unit test and some cleanup
2020-10-23 19:34:18 +00:00
9043357934
Merge pull request #3360 from sfc-gh-tclinkenbeard/use-vector-for-tagset
...
Use std::vector for TagSet
2020-10-23 10:34:20 -07:00
055add9682
conflicts
2020-10-23 06:33:00 +00:00
90caa24af1
add function to get correct checksum class. Remove IChecksum::checkSum function
2020-10-23 02:24:29 +00:00
bd578a0d95
clang-format
2020-10-22 18:27:58 -07:00
3cbed7b0d1
Add checker to make sure registered special key range following snake case naming
2020-10-22 18:25:51 -07:00
c8eb003ec4
Change to \xff\xff/management/in_progress_exclusion, the snake case naming style
2020-10-22 18:05:54 -07:00
c8b3b37b4a
Remove unused code for #3930
2020-10-22 17:04:08 -07:00
0060eeb6f8
Merge pull request #3930 from sfc-gh-clin/fix-FuzzApiCorrectness
...
Fix FuzzApiCorrectness test failures
2020-10-22 16:24:37 -07:00
4c42917b1d
Fix typos in comments
2020-10-22 15:36:10 -07:00
1d081401c6
Fix Tag::load bug
2020-10-22 14:13:14 -07:00
95654ce7e7
Change dbLocked to snake_case naming db_locked
2020-10-22 11:35:39 -07:00
2b4ec39a2f
Avoid passing references from an iterator into an insertion that invalidates the iterator.
2020-10-22 11:30:08 -07:00
1d64b2c335
clang-format
2020-10-22 11:28:03 -07:00
b728fe473e
Add consistencycheck command to special keys
2020-10-22 11:08:54 -07:00
dec643987f
Add wait time after dd not found error
2020-10-21 16:58:23 -07:00
8d937a56f2
Throw operation_failed() if dd not found when calling ddMetricsRequestServer
...
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
e183a10c8c
Add "waitForDestUID" flag for abortBackup
2020-10-20 22:55:31 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
c84a5f4014
change IChecksum interface and remove coordinator protocol version optional
2020-10-19 21:39:07 +00:00
a4ccf8c5b6
add checksum class to change checksum in the future. expose get server protocol api
2020-10-19 16:57:11 +00:00
944f30484a
Merge pull request #3759 from dongxinEric/misc/3739/expose-time-since-last-recovery
...
This resolves issue #3739 by exposing time since last full recovery.
2020-10-19 09:03:31 -07:00
47a9bc9d3d
Fix commit_unknown_result issue
2020-10-18 23:11:19 -07:00
bfd3328448
Fix a race between submit and abort backup
...
After submit a backup, immediately abort the backup may cause a rare race
condition, which results in BackupCorrectnessLeftoverVersionKey error.
Specifically, in the StartFullBackupTaskFunc:
1st Txn sets the destUid at the source database and the 2nd Txn writes the dest
DB.
An abort can come after the 1st Txn succeeds, and clears the config rage so
that the 2nd Txn above would fail. Because 2nd Txn didn't write destUid, the
3rd Txn of abort can't read the correct source DB for latestVersionKey, which
contains the destUid value.
The fix is to let the 1st Txn of abort to wait until destUid becomes valid.
2020-10-18 23:11:15 -07:00
545ee4269d
master conflicts
2020-10-19 01:03:54 +00:00
1c533e7363
fix flowtransport conflicts
2020-10-19 01:00:03 +00:00
c1d88b39ab
Merge remote-tracking branch 'origin/master' into add-upgrade-and-backup-restore-test
2020-10-17 11:35:12 -07:00
c1b4e62bde
Change the trace from SevInfo to SevDebug
2020-10-15 13:55:31 -07:00
0718329705
Make getDesiredClusterKeyName const
...
This fixes a virtual function overload in NameQuorumChange
2020-10-15 10:39:38 -07:00
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
...
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
bf00369576
getRange only enters special key space codepath when both begin key and end key are in (\xff\xff, \xff\xff\xff)
2020-10-14 16:57:38 -07:00
e47e0108a7
Add new TLogVersion
2020-10-14 15:41:10 -07:00
a1e868a569
Merge pull request #3862 from sfc-gh-tclinkenbeard/use-override-more
...
Add uses of override keyword, remove unnecessary uses of virtual
2020-10-14 15:06:45 -07:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
4eb20a1283
merge anoyes/stable-interface and add back in isCompatible
2020-10-12 20:39:37 +00:00
2f5b0bef08
switch to test newer incompatible version. Fix PR comments. Modify schema
2020-10-12 18:29:16 +00:00
a4e1efbc0e
revert MultiVersionTransaction
2020-10-12 18:25:14 +00:00
5488ff1d81
draft diff protocol
2020-10-12 18:24:03 +00:00
b233f44d2d
remove some print statements and spin lock that was used for debugging
2020-10-12 18:20:37 +00:00
41843f07e6
add simulator support for different process versions and ProtocolVersion test
2020-10-12 18:19:31 +00:00
76d0027fa2
merge anoyes/stable-interface and add back in isCompatible
2020-10-12 18:18:30 +00:00
d3b57d9873
Merge pull request #3823 from sfc-gh-jfu/jfu-snapshot-record-version
...
Have snapshot record to system keys and write lastEpochEnd on recovery
2020-10-09 07:27:19 -07:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
ccf2ca73f3
Fix Windows CMake build
2020-10-08 23:58:58 -07:00
76440018fb
Remove unnecessary error thrown when calling clear range on the lock special key range
2020-10-08 21:39:13 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
c66a775d64
Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys
2020-10-08 14:23:16 -07:00
53d3759e73
Add lock command to special key \xff\xff/dbLocked
2020-10-08 14:23:02 -07:00
2def07a18d
Apply suggestions from code review
2020-10-08 13:31:47 -07:00
a9607bdcec
Explicitly seal classes that inherit but aren't inherited from
2020-10-07 21:58:24 -07:00
a3de8c333b
Use override where applicable in fdbclient
2020-10-07 16:57:38 -07:00
f03a355c4e
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/FlowTransport.h
# flow/Arena.h
# flow/Knobs.cpp
# flow/Knobs.h
# packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
7218586f2f
adjust test spec and backup describe
2020-10-06 17:03:09 -04:00
76da0a7cbc
Backport #3426 to release-6.2
2020-10-05 13:53:51 -07:00
c622a11c70
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-05 13:15:56 -04:00
43ac7f21a4
Mark toString methods const
2020-10-04 23:27:10 -07:00
d944e13e5b
Make TaskBucket::getAvailableSpace const
2020-10-04 23:27:09 -07:00
8ae0679112
Remove unnecessary separation of declaration and definition of version field
2020-10-04 23:27:05 -07:00
b294e148c5
another compile fix
2020-10-04 16:54:43 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
6326de052b
Remove unused LogFileWriter::FFs
2020-10-02 17:03:08 -07:00
cacf4be449
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-02 12:57:17 -04:00
268ba0bddc
Merge remote-tracking branch 'origin/release-6.3' into merge-6.3
2020-10-01 14:14:06 -06:00
13d1978c19
Merge branch 'release-6.3' into feature-fix-fault-tolerance
2020-09-30 19:00:31 -07:00
8c729ca8e6
only add additional fault tolerance for availability if automatic failover is enabled
2020-09-30 18:04:23 -07:00
c7a55fe15f
Merge pull request #3829 from yliucode/process-class
...
Support specifying proxy as process class and converting it to commit proxy
2020-09-30 16:46:24 -07:00
fe5c30e778
fault tolerance was not being properly increased when usable regions was 2 and satellites are configured.
2020-09-30 16:41:00 -07:00
0c579674e6
Add more help messages for configuring 'proxies'
2020-09-30 13:29:01 -07:00
e2705668be
Lower WATCH_TIMEOUT default value to 30.0
2020-09-30 11:52:51 -07:00
8da8c6ff0d
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-30 14:25:29 -04:00
44cd3b0999
add stopBackup to incrementalBackup workload
2020-09-30 14:24:52 -04:00
3705f5a6fe
Merge pull request #3820 from sfc-gh-tclinkenbeard/improve-type-safety
...
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-30 10:46:59 -07:00
c34385ef7b
Support proxy process class
2020-09-30 01:23:10 -07:00
21071b6214
Merge pull request #3813 from xumengpanda/mengxu/merge-to-master-PR
...
Merge 6.3 to master and resolve conflicts
2020-09-28 15:24:18 -07:00
cfaa207f88
fix spacing and newlines at ends of files
2020-09-28 16:32:20 -04:00
3aa4070adb
remove trace events used for debugging
2020-09-28 16:27:30 -04:00
e02caac2d9
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-28 16:20:33 -04:00
6c726ba8dd
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-27 15:29:15 -07:00
1fb11858e6
Fix failure caused by merge
2020-09-27 11:03:16 -07:00
8d9e7adac7
Merge pull request #3817 from yliucode/proxy-doc
...
Mention GRV & Commit proxy in docs and add backward compatibility for configuring proxies
2020-09-25 22:58:55 -07:00
480fc82779
Resolve review comments
2020-09-25 16:58:54 -07:00
bbc7ce581e
Resolve conflicts merging from 6.3 to master
2020-09-25 16:08:31 -07:00
3278e21b0b
Change description of GRV proxy and Commit proxy in docs and add backward compatibility support for configuring proxies in fdbcli
2020-09-25 10:50:48 -07:00
b6b87cd8d4
fileconfigure fails with Assertion false failed
2020-09-25 20:20:00 +03:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
a96d6f85c5
Removed redundant field number_of_old_generations_of_tlogs from status json
2020-09-24 09:44:51 -07:00
6ed330572e
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-24 12:09:17 -04:00
feb3bda79e
Merge pull request #3797 from xumengpanda/mengxu/fr-write-traffic-control-PR
...
Fast Restore: Add write rate control
2020-09-23 15:50:08 -07:00
3a03c9bd74
Remove deprecated enumActionResult
2020-09-23 15:14:09 -07:00
d13a9d3c91
Make EBackupState an enum class
2020-09-23 15:14:09 -07:00
69580593dd
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-23 15:35:05 -04:00
50f681cd32
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-09-23 10:54:49 -07:00
b37daf88d4
add jitter to the delay
2020-09-22 22:24:40 -07:00
09ff05c629
backed out buggify of apiVersion because there are too many tests that depend on it for a patch release
2020-09-22 21:15:49 -07:00
e0fbadfeb5
Added a buggify so older API versions are run in simulation
2020-09-22 18:58:01 -07:00
c99627a7ab
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-apiverison-batch-error
2020-09-22 17:06:17 -07:00
7dec47938e
protect api versions older than 630 from seeing batch_transaction_throttled errors
2020-09-22 17:05:09 -07:00
002b1bec4c
FastRestore:Control write traffic at each applier
...
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.
This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
9b7d75b6dd
WIP
2020-09-21 13:41:35 -04:00
7d606ebff4
Fix some accidental s/NULL/nullptr replacements
2020-09-20 11:33:18 -07:00
b4e209bd4e
Replace NULL with nullptr in fdbclient
2020-09-20 11:33:09 -07:00
a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
...
Add setclass to special keys
2020-09-19 14:55:03 -07:00
d4e3e98957
Use new getRestoreSet in RestoreController
2020-09-17 17:56:36 -07:00
71309a79f9
Merge branch 'release-6.3' into range
2020-09-17 17:54:33 -07:00
cf69f455a9
Merge pull request #3785 from apple/release-6.3
...
Merge Release 6.3 to master
2020-09-17 14:43:56 -07:00
4df0f60729
Instead of using fully_recovered, use accepting_commits as a singal of DB turned available. Also add the number of old generations into status
2020-09-17 09:55:25 -07:00
ea27e4bd9c
Merge pull request #3772 from yliucode/rename
...
Rename master proxy as commit proxy
2020-09-16 20:06:50 -07:00
1fe31aa9e8
Merge pull request #3706 from yliucode/tlogs
...
Report missing old tlogs in recovery before storage servers are fully recovered
2020-09-16 14:07:06 -07:00
cc5bc16bd8
Rename more places from proxy to commit proxy
2020-09-15 22:29:49 -07:00
8224e17a08
Integrate extended getRestoreSet API into fast restore
...
The extended getRestoreSet provides a much smaller set of backup files for small keyrange restore;
This commit integrate it into fast restore so that fast restore does not have to filter out unneeded backup files.
2020-09-15 09:38:43 -07:00
3dfadff6d8
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-09-14 16:45:47 -07:00
cc32074ed7
Rename actor
2020-09-14 16:45:34 -07:00
f9a5b727a6
Add comments questions and TODOs
2020-09-14 16:28:01 -07:00
a3c861b59d
Merge pull request #3777 from apple/release-6.3
...
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
0282a1745a
Address more review comments
2020-09-11 16:44:09 -07:00
0a7d2d31f1
Review some review comments from AJ
2020-09-11 16:17:23 -07:00
6dbcd42ebd
No need for a new protocol version since the serialization won't change. Added code for deserializing from old binary.
2020-09-11 16:16:56 -07:00
1b923477f7
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-09-11 14:28:56 -07:00
260c8d9568
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-11 15:05:58 -04:00
22996284c7
added changes to allow writing of last epoch end version to special keys when performing recovery due to snapshot
2020-09-11 15:00:11 -04:00
537d6ffad2
change count
2020-09-11 17:46:14 +00:00
9947b416ee
Merge branch 'master' of https://github.com/apple/foundationdb into fdbtest
2020-09-11 15:27:35 +00:00
da468df628
fix unthrottle signal bug
2020-09-11 07:22:54 +00:00
084c69b2ad
fix reason serialization bug
2020-09-11 07:21:57 +00:00
35bef73a1c
Rename proxy to commit proxy
2020-09-10 17:44:15 -07:00
d270b15c89
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-09-09 16:21:19 -07:00
2803e6be52
Remove enabling ACCESS_SYSTEM_KEYS
2020-09-09 16:17:43 -07:00
f10e9ea679
FastRestoreApplier:Add write conflict range
2020-09-09 12:12:14 -07:00
1155d015c9
fetch current log generation as well
2020-09-09 11:54:58 -07:00
e69ee87f33
Refactor blob failure to simulateBlobFailure function
2020-09-09 10:54:21 -07:00
989299dc74
Merge pull request #3751 from sfc-gh-tclinkenbeard/fix-tb-traces
...
Use consistent prefix for task bucket traces
2020-09-09 10:26:54 -07:00
5880bd9fa7
BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error
2020-09-09 08:47:43 -07:00
20733ae1eb
Simulate throwing blob http error immediately after read blob file
2020-09-09 06:47:19 -07:00
cffb54ba0b
Buggify decodeLogFileBlock with blob http error
2020-09-09 06:44:02 -07:00
f3986d201d
Buggify backup decodeRangeFileBlock with blob connection error
2020-09-09 06:40:41 -07:00
62dd1f7234
Merge pull request #3696 from sfc-gh-xwang/tag-report
...
report busiest write tag of each storage server
2020-09-08 15:21:14 -07:00
2935d3d4f6
change workload; solve some comments
2020-09-08 21:47:49 +00:00
4363dd0f25
This resolves issue #3739 by exposing time since last full recovery.
2020-09-08 14:26:01 -07:00
23e1ff694c
Report missing old tlogs in recovery between accepting commits and storage recovered
2020-09-08 13:35:42 -07:00
223c419f35
Merge pull request #3676 from sfc-gh-jfu/jfu-incremental-backup-only
...
incremental backup and restore
2020-09-08 12:06:25 -07:00
6c1b09829a
Replace TB_ trace prefix with TaskBucket
2020-09-08 11:38:53 -07:00
43e3e320e3
Refactor getrange for read-write module and add a test to make sure we have consistent results
2020-09-08 11:08:48 -07:00
a188f91fe6
Merge pull request #3738 from dongxinEric/misc/remove-unused-client-knob
...
The actual BYTE_LIMIT_UNLIMITED and ROW_LIMIT_UNLIMITED are defined i…
2020-09-08 10:46:27 -07:00
63a5f26a23
Remove implicitly setting read_system_keys when set SPECIAL_KEY_SPACE_ENABLE_WRITES, we can achieve read/set system keys through transaction object
2020-09-08 10:46:19 -07:00
0eb7092faa
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only
2020-09-08 12:53:29 -04:00
fda6e0c275
Use consistent prefix for task bucket traces
2020-09-05 21:42:51 -07:00
3728ed03dd
Resolve comments
2020-09-05 18:55:09 -07:00
1ad5e17458
add support for comparing original and current impls
2020-09-05 11:14:59 -07:00
53b7721d6c
Add additional trace information
2020-09-04 15:36:47 -07:00
1ca7fe1a05
Add span metadata message
2020-09-04 15:36:47 -07:00
443e9a251d
Igore value in set in excluding(fail) implementation
2020-09-04 15:21:47 -07:00
ee2ce6e758
Refactor ryw function which updating read result with writes in the transaction
2020-09-04 14:54:32 -07:00
ecf2c0109c
more concise status json
2020-09-04 18:40:45 +00:00
fb758bf937
update Schemas.cpp
2020-09-04 16:34:05 +00:00
ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
...
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
b9bdd6c0ff
The actual BYTE_LIMIT_UNLIMITED and ROW_LIMIT_UNLIMITED are defined inside the struct GetRangeLimits and these two unused declarations can be confusing.
2020-09-03 14:55:16 -07:00
f9624b5a1a
Avoid unnecessary copy
2020-09-03 13:56:23 -07:00
2fa8697796
Merge pull request #3726 from satherton/taskbucket-comments
...
Added some overview comments to TaskBucket
2020-09-02 13:58:44 -07:00
cac1b8a6e5
Update fdbclient/TaskBucket.h
2020-09-02 13:55:18 -07:00
87693cae81
merge master branch and resolve conflicts
2020-09-02 13:44:33 -07:00
430921f25f
Update local kv's contruction using result.arena()
2020-09-02 13:38:24 -07:00
b4c96cadc7
Merge branch 'release-6.3' into merge-release-6.3-into-master
2020-09-02 12:45:57 -07:00
d334b6484e
attempt to write to system keys with snapshot
2020-09-02 15:17:54 -04:00
9c130b5ea2
added new system keys
2020-09-02 14:13:58 -04:00
8ac3b21459
changed occurrences of getOrThrow to get
2020-09-02 13:45:15 -04:00
c6477bbc61
Merge branch 'master' into feature/range-split-points-based-on-size
2020-09-02 09:41:17 -07:00
5ca42899aa
Added some overview comments to TaskBucket
2020-09-01 17:29:56 -07:00
93da4957ad
Update fdbclient/NativeAPI.actor.cpp
2020-09-01 13:17:01 -07:00
800ece286d
Results from read hot range requests were not held by the results arena.
2020-09-01 13:14:27 -07:00
b9baa996e3
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only
2020-09-01 14:38:02 -04:00
12edadd059
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# fdbclient/Knobs.cpp
# fdbclient/MasterProxyInterface.h
# fdbrpc/simulator.h
# fdbserver/MasterProxyServer.actor.cpp
# tests/fast/CycleAndLock.txt
# tests/fast/TxnStateStoreCycleTest.txt
# tests/fast/VersionStamp.txt
# tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
# tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
# versions.target
2020-08-31 19:33:34 -07:00
4c6fac058b
Merge pull request #3709 from etschannen/release-6.3
...
Disabled two features using knobs for performance reasons
2020-08-31 10:39:59 -07:00
5c97461d18
Merge pull request #3653 from etschannen/feature-proxy-busy-loadbalance
...
Changed proxy load balancing to balance on CPU usage
2020-08-31 10:39:08 -07:00
8994719e46
Merge branch 'master' into grv-proxy
2020-08-31 10:21:32 -07:00
b6c0299d09
Add help message in backup CLI for added options
2020-08-31 09:31:57 -07:00
808f8309b1
disable tag_encode_key_servers because it does not help recovery times (the CPU on the proxies decoding the tags is more than the saving for the smaller size)
2020-08-31 09:29:46 -07:00
29eec30183
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# build/Dockerfile
# build/Dockerfile.devel
# documentation/sphinx/source/downloads.rst
# fdbserver/Knobs.cpp
# fdbserver/LogSystem.h
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/TagPartitionedLogSystem.actor.cpp
# fdbserver/WaitFailure.actor.cpp
# fdbserver/fdbserver.vcxproj
# fdbserver/fdbserver.vcxproj.filters
# packaging/msi/FDBInstaller.wxs
2020-08-31 01:10:29 -07:00
33aa10b461
Minor optimizations
2020-08-29 20:10:45 -07:00
fd7198d874
Extend backup container interface to support query restorable files set by key ranges
2020-08-29 19:58:07 -07:00
e87327b33b
Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper
2020-08-29 12:47:35 -07:00
00c77ba2b4
Added beginVersion cmd line option and addressed code review comments
2020-08-28 14:29:22 -04:00
4831f11e2a
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only
2020-08-28 12:47:19 -04:00
0270ee13c8
change targetversion from maxlogend to contiguouslogend. also adjusted test workload and specs
2020-08-28 12:46:34 -04:00
0dc5736f54
Allow directly read \xff\xff/management/* without setting option
2020-08-28 01:01:37 -07:00
caeeea365e
Revert "Disable support for pattern match of a whole machine address"
...
This reverts commit 94221f1e91
2020-08-27 23:07:22 -07:00
ca9b1f5b34
Merge branch 'release-6.3' into mengxu/fr-sched-PR
...
Resolve conflict at BackupContainer.actor.cpp
2020-08-27 16:54:00 -07:00
94221f1e91
Disable support for pattern match of a whole machine address
2020-08-27 11:53:30 -07:00
939f59d989
Merge pull request #3672 from sfc-gh-tclinkenbeard/make-iasyncfile-const-correct
...
Make IAsyncFile const-correct
2020-08-27 11:22:46 -07:00
cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-27 10:06:02 -07:00
e8e24a8ed6
Resolve comments
2020-08-26 21:04:35 -07:00
369000a125
BackupContainer:Remove link to filename with random string
2020-08-26 15:55:36 -07:00
a2ab709a0c
BackupContainer:Use processId as the process filename
...
instead of using a randomly generated string which change every time
when a file is open.
Having too many files will trigger TOO_MANY_FILES error
2020-08-26 15:54:34 -07:00
a07b9f234d
Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/
2020-08-25 18:18:32 -07:00
6d17e996fb
Remove override
2020-08-25 14:59:43 -07:00
6256bedf8d
BackupContainer:Use processId as the process filename
...
instead of using a randomly generated string which change every time
when a file is open.
Having too many files will trigger TOO_MANY_FILES error
2020-08-25 12:25:09 -07:00
bd7c07436b
FastRestore:Add batchIndex to RestoreAsset for better performance tracking
2020-08-25 09:34:18 -07:00
9564171463
Merge branch 'master' into grv-proxy
2020-08-24 22:45:01 -07:00
9011ee6f57
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-25 00:42:04 +00:00
3e39f01496
Merge pull request #3689 from sfc-gh-xwang/tag-report
...
add throttle objects into Schemas.cpp
2020-08-24 17:23:00 -07:00
63b3612ad5
Merge master branch and resolve conflicts
2020-08-24 16:42:31 -07:00
4e627691a9
add throttle objects into Schemas.cpp
2020-08-24 23:37:58 +00:00
4e4fa0fded
merge with master
2020-08-24 21:04:53 +00:00
e7662eecda
Merge pull request #3669 from sfc-gh-xwang/tag-report
...
Report recommended tx tag to be throttled to status json
2020-08-24 12:06:25 -07:00
0d65e1e0e0
update ProtocolVersion
2020-08-23 21:03:26 +00:00
3afdb44c7a
merge master
2020-08-23 17:09:04 +00:00
b4cec6577a
Fix merge bugs
2020-08-22 15:04:42 -07:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
23ef23357c
Test code to add c function for kill/suspend
2020-08-21 14:30:52 -07:00
9b096ab58e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-08-21 12:20:35 -07:00
3b63d8b01b
remove FIXME; remote tagSet.reset(); trivial changes
2020-08-21 19:17:16 +00:00
b8475ebb4b
Fix network address order bug in ExclusionInProgressActor
2020-08-21 01:20:14 -07:00
cf19c5dac9
Move validation of network address and class type before get worker list
2020-08-21 00:56:12 -07:00
476faa6521
change reason type
2020-08-21 04:07:25 +00:00
5aaf86206a
add counter for expensive clear estimation for audit and trace
2020-08-20 23:00:45 +00:00
556b239057
clang-format
2020-08-20 14:51:41 -07:00
e54f728d42
update comments
2020-08-20 14:41:17 -07:00
4a8a356ffb
throw special_keys_api_failure if clear called on \xff\xff/configuration/class
2020-08-20 14:38:07 -07:00
1815cc24a3
Use default set and clear implementations for exclude and failed
2020-08-20 14:26:41 -07:00
bff20faa42
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-08-20 13:50:54 -07:00
707b88583a
Add default encode, decode methods. Add test for setclass special keys
2020-08-20 13:50:35 -07:00
e375761581
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-20 20:29:14 +00:00
7dcce6979e
format
2020-08-20 07:00:37 +00:00
0945959a35
add fdbcli support to list recommended tags
2020-08-20 06:26:31 +00:00
9cf3041393
Add read conflict for metadata txs that aren't lock-aware
2020-08-20 02:52:37 +00:00
c8b774a30a
Explicitly mark IAsyncFile functions as overrided
2020-08-19 18:17:32 -07:00
5660de9c09
Add \xff\xff/configuration/class to change process class
2020-08-19 17:54:38 -07:00
157700e5b6
Make IAsyncFile const-correct
2020-08-19 17:34:56 -07:00
ae999aa118
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only
2020-08-19 16:36:47 -04:00
9c709b33d6
scale cost
2020-08-19 19:47:41 +00:00
bc6e42c634
add status json report for recommended throttled tags
2020-08-19 19:22:16 +00:00
35aa1034cd
added new test spec and workload to verify correctness of incremental backup and restore
2020-08-19 13:45:51 -04:00
aa253742ec
add getRecommendedTags to ThrottleApi
2020-08-19 17:39:43 +00:00
7bfb0a4cc5
Merge pull request #3662 from sfc-gh-tclinkenbeard/modernize-copy-disable
...
Use =delete to disable special member functions
2020-08-19 00:13:12 -07:00
599675cba8
modify some details to get better performance
2020-08-19 04:23:23 +00:00
33f5ce98c7
Merge branch 'release-6.3' into mengxu/fr-sched-PR
2020-08-18 17:41:31 -07:00
6729b4143b
FastRestore:Requests:Add const for toString function
2020-08-18 16:05:23 -07:00
7b210aee24
Use =delete to disable special member functions
...
This has the benefits of:
- Easier to understand compiler error messages
- Moves some errors from link-time to compile-time
- NonCopyable classes can still specify default move constructors
2020-08-18 11:07:00 -07:00
b8c06b3d15
Ensure FlowLock::Releaser is always constructed before AsyncFileBlobStore upload
2020-08-17 23:19:51 -07:00
7b7490efe7
FastRestore:Debug trace for seg fault
2020-08-17 20:34:33 -07:00
24dbb3961c
Knob adjustment
2020-08-17 22:38:22 +00:00
130bd7ac22
Fixed a bug that would potentially cause invalid memory access, i.e. segmentation fault.
2020-08-17 13:16:17 -07:00
7e302b5910
FastRestore:Loader:Add scheduler to dispatch requests based on resource utilization and request priority
2020-08-15 21:21:51 -07:00
421f44f4cc
Merge pull request #3640 from sfc-gh-tclinkenbeard/fix-asyncfileblobstore-lock
...
Fix potential memory leak in AsyncFileBlobStore
2020-08-14 18:09:34 -07:00
ffa1973242
move OPERATION_COST_BYTE_FACTOR to CLIENT_KNOB; skip the first call of refreshBusiestTag.
2020-08-14 22:49:20 +00:00
daead84fdd
Merge master
2020-08-14 14:07:08 -07:00
548ba06510
change map to deque; code style problem
2020-08-14 18:30:58 +00:00
475d4cca72
solve some cr comments
2020-08-14 16:40:16 +00:00
b9474055a5
Add \xff\xff/configuration/class to read process class
2020-08-14 01:01:08 -07:00
e94de126de
merge with master
2020-08-14 01:59:09 +00:00
19df032aec
Change some formatting issues
2020-08-13 15:30:21 -07:00
20700f75f4
Merge master
2020-08-13 14:28:27 -07:00
6f98ed3d24
Merge branch 'release-6.3' into feature-proxy-busy-loadbalance
2020-08-13 11:30:28 -07:00
b2bba8ccbb
Correct fix for debug ID logging
2020-08-13 13:37:15 -04:00
633565aa1a
Use std::make_shared for AsyncFileBlobStore lock
2020-08-12 20:30:03 -07:00
002cca5e58
Merge branch 'master' into tmp/merge-PR
...
Conflicts:
fdbclient/NativeAPI.actor.cpp
fdbserver/StorageMetrics.actor.h
2020-08-12 20:08:13 -07:00
c72068d6b5
clients load balance across proxies based on process busyness instead of number of requests
2020-08-12 17:17:21 -07:00
8953cc5f98
Merge master again
2020-08-12 16:59:07 -07:00
8bc57c16f0
Merge pull request #3637 from dongxinEric/misc/fix-spammy-readhot-logging
...
Increased some knobs to throttle the spammy read hot logging.
2020-08-12 16:46:32 -07:00
8f1deaa47e
Merge branch 'master' of https://github.com/apple/foundationdb
2020-08-12 23:32:23 +00:00
ddec93c504
Update fdbclient/StorageServerInterface.h
...
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2020-08-12 15:47:32 -07:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
cc1059be75
Make logging of server trace ID consistent with other code
2020-08-12 19:05:01 +00:00
f3ecf14601
change midShardSize type and other details
2020-08-12 17:49:12 +00:00
16ef2bd3bd
Pending commit
2020-08-12 10:34:07 -07:00
3f82c815f9
changed another file identifier
2020-08-11 22:53:34 -07:00
610e1e2525
change file identifier
2020-08-11 22:51:03 -07:00
2f52c5f79b
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/RestoreLoader.actor.cpp
2020-08-11 22:45:43 -07:00
79ce16650d
merge master branch
2020-08-11 19:22:10 -07:00
850a21dea3
Merge pull request #3624 from sfc-gh-tclinkenbeard/rename-threadsafetransaction
...
Rename ThreadSafeTransaction.cpp
2020-08-11 16:39:13 -07:00
2a72b1527d
Rename ThreadSafeTransaction.cpp
...
This file doesn't contain actors, so it doesn't need the actor.cpp
suffix
2020-08-11 15:48:23 -07:00
4782c7c4b1
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-11 15:35:52 -06:00
f240999cd2
Merge pull request #3621 from sfc-gh-ljoswiak/fix-destructing-span
...
Fix span early destruction
2020-08-11 12:52:35 -07:00
7dce3a9187
fixed issue with mutations not applying and allow backup to non-empty db
2020-08-11 15:39:21 -04:00
b66dab8816
Fix span early destruction
...
We want the span to stick around for the full duration of
extractReadVersion since it writes to the log from its destructor.
2020-08-11 11:19:04 -07:00
842a5aa925
Merge pull request #3458 from yliucode/clean-grv-broadcast
...
Clean up code paths serving GRV through broadcasting to other proxies
2020-08-11 11:17:20 -07:00
7ee6ca342e
merge with master
2020-08-11 01:01:15 +00:00
c0b7ac0b7d
try to throttle write tag and read tag seperatedly
2020-08-10 23:43:58 +00:00
cae6f04070
sample on cost instead of bytes
2020-08-10 22:30:06 +00:00
c5c6906a3d
Merge pull request #3600 from xumengpanda/mengxu/fr-fix-large-pkg-PR
...
Fast Restore: Fix the sampling packet size at sampling phase
2020-08-10 13:53:24 -07:00
7c05e9c1d3
Merge pull request #3610 from xis19/issue_3551
...
When a configuration K/V pair is modified, always output the K/V pair values
2020-08-10 13:52:44 -07:00
696e77c94e
query midShardSize from proxy
2020-08-10 20:13:44 +00:00
5456b01c3a
Avoid the extra copy in RYW APIs
2020-08-10 09:34:21 -07:00
7f422105fe
Merge pull request #3612 from dongxinEric/misc/avoid-additional-copy
...
Avoid additional copying of the key range.
2020-08-10 09:31:24 -07:00
8e50b51fbd
If mutableConfiguration exists, skip checking rawConfiguration in DatabaseConfiguration
2020-08-09 22:12:42 -07:00
0e97851451
Use reference to avoid extra copy.
2020-08-09 22:11:29 -07:00
ceeade7334
Address review comments, improve performance
2020-08-08 23:52:38 -07:00
49968b8200
Cleanup DatabaseConfiguration::toJSON
2020-08-08 23:24:30 -07:00
6b8843ad18
When a configuration K/V pair is modified, always output the K/V pair values
...
With this patch, if there is a configuration special key in the
database, even if the value is reverted to default/hidden, it will still
be forwarded when constructing a JSON string. This should address the
requirements described in https://github.com/apple/foundationdb/issues/3551 .
2020-08-08 23:24:30 -07:00
df9149fea4
ignore transaction tag of immediate transactions
2020-08-07 23:36:17 +00:00
98c39ad9d4
Fix the build error.
2020-08-07 14:03:42 -07:00
a46e620baa
Merge branch 'master' into feature/range-split-points-based-on-size
2020-08-06 15:23:11 -07:00
6d84bcb568
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-06 14:08:34 -06:00
8976694ba1
UDP implementation (untested)
2020-08-06 14:06:50 -06:00
ba66b1f668
trivial changes
2020-08-06 14:53:26 +00:00
2e41391690
Merge master branch
2020-08-06 00:11:00 -07:00
d6a23a4d6b
Resolve comments to make GRV proxy a separate process class
2020-08-06 00:01:57 -07:00
b00b1bb30f
Add a test for management api options
2020-08-05 13:20:12 -07:00
b682ad75f9
Fix a memory bug, change \xff\xff/management/options/exclude/force to .../excluded/force
2020-08-05 13:18:39 -07:00
9640ba42d3
merge master
2020-08-05 16:23:10 +00:00
879998d01b
FR:fix compilation error
2020-08-04 23:04:38 -07:00
7992cef025
FR:Fix sample network pkg can be too big
2020-08-04 22:35:21 -07:00
13307679c5
use median shard size"
2020-08-05 03:57:25 +00:00
21635f8a28
update backup restore for local testing
2020-08-04 15:48:43 -04:00
4997754cf8
fix format in monitorDDMetricsChanges
2020-08-04 04:14:11 +00:00
00f2c38e3a
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-08-03 20:55:21 -07:00
48094ea17e
add INCOMPLETE_SHARD_PLUS knob
2020-08-04 03:18:05 +00:00
52073738c0
fix bugs; change some KNOB values
2020-08-03 22:18:34 +00:00
e9adec421d
FR:Add controller interface and send to roles
2020-08-03 15:06:15 -07:00
b903e60cb7
fix monitorDDMetricsChanges bugs
2020-08-03 17:12:36 +00:00
6db7040f17
update DDMetrics periodically
2020-08-03 05:52:38 +00:00
d1cc87452c
merge with master; solve conflicts; solve initialization;
2020-08-02 22:44:07 +00:00
0352e8ee0b
pick busiest commit tag periodically
2020-08-02 18:38:56 +00:00
da83cfbce1
Explain unit for waitForDuration in comment
2020-08-02 17:28:26 +00:00
92c1112c74
consider clear single key
2020-08-01 18:20:13 +00:00
29f4381c01
Deprecate \xff\xff/suspend in 7.0
2020-08-01 01:09:45 -07:00
d9e5d65aaf
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-08-01 00:31:26 -07:00
4f7dab4951
sample clear op on client
2020-08-01 06:14:52 +00:00
fbe36ced23
Rewrite all \xff\xff/reboot_worker call
2020-07-31 16:33:16 -07:00
372f876673
Add note for deprecating \xff\xff/reboot_worker in 7.0
2020-07-31 15:59:46 -07:00
5c7b4c7acf
Merge branch 'master' of github.com:apple/foundationdb
2020-07-31 15:58:13 -07:00
6dc8b4c1e9
Fix \xff\xff in error message
2020-07-31 13:31:36 -07:00
df4beb629e
fix the bug where we call old \xff\xff/conf special keys
2020-07-31 13:28:24 -07:00
fc3dfc170f
Merge branch 'master' of https://github.com/apple/foundationdb
2020-07-31 19:09:33 +00:00
c3a629588f
add client transaction tag sample
2020-07-31 19:08:42 +00:00
a048e5e159
Change prefix to \xff\xff/management and make decode/encode as pure functions
2020-07-31 02:18:49 -07:00
c8a2e7ab07
Add operation instruction in error message for failure of exclusion safety check
2020-07-30 22:38:35 -07:00
05a0a47824
throw actor_cancelled error
2020-07-30 22:15:22 -07:00
db8bb69196
Solve conflicts from master
2020-07-30 19:02:19 -07:00
f2662a9a74
Merge remote-tracking branch 'upstream/master' into add-management-api-into-special-key-space
2020-07-30 19:01:11 -07:00
10540a3c73
Change error name and fix a bug in ExclusionInProgressRangeImpl::getRange()
2020-07-30 18:58:09 -07:00
96b1327568
change to const std::string&
2020-07-30 16:27:40 -07:00
11cbcffc9a
Temporary code
2020-07-30 16:07:54 -07:00
bfa4eb9ab2
Resolve comments
2020-07-30 14:45:03 -07:00
db30943fbf
Merge pull request #3399 from sfc-gh-tclinkenbeard/vectorref-emplace-back
...
Add VectorRef::emplace_back
2020-07-30 13:01:22 -07:00
1bd6ea51ac
Merge pull request #3572 from xumengpanda/mengxu/master-dd-failure-PR
...
Fix master branch failure on DDBalanceAndRemove.toml test
2020-07-30 10:07:41 -07:00
dea915f2d7
Merge pull request #3177 from alexmiller-apple/bump-tlog-version
...
Walk TLogVersion forward for 7.0 now that 6.3 has been branched.
2020-07-29 13:53:41 -07:00
b5dc6b715f
Add file comment for GrvProxyServer.actor.cpp
2020-07-29 12:10:53 -07:00
cb0808a170
Resolve comments
2020-07-29 11:19:29 -07:00
86148f1277
Merge branch 'master' into grv-proxy
2020-07-29 11:14:51 -07:00
fad18ad85a
Remove more debug traces
2020-07-29 08:06:15 -07:00
30ea639666
Remove debug traces
2020-07-29 07:55:05 -07:00
f7b76a92af
pass joshua
2020-07-29 07:26:55 -07:00
a49cb41de7
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# cmake/ConfigureCompiler.cmake
# fdbserver/Knobs.cpp
# fdbserver/StorageCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/ThreadHelper.actor.h
# flow/serialize.h
# tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
a2089b354a
RemoveServersSafely:Safety check toKill1 to avoid cluster getting stuck
...
toKill1 and toKill2 are a random subset of all processes. If simply kill all processes in toKill1 or toKill2,
we may kill too many processes to make the cluster unavailable and stuck.
Similar as what toKill2 were modified if it can cause cluster unavailable,
we should do the same thing for toKill1
2020-07-28 21:07:31 -07:00
602b0f9dea
Merge branch 'master' into vectorref-emplace-back
2020-07-28 17:17:11 -07:00
2fc3bfc3bd
Remove \xff\xff/reboot_worker from special keys
2020-07-28 13:45:24 -07:00
27a7a28fc5
Change to ryw in checkForExcludingServers
2020-07-28 13:40:14 -07:00
70aecfa2d6
refine ManagementCommandsOptionsImpl::clear(..., range)
2020-07-28 13:32:31 -07:00
419a187ee8
reinitilize specialKeySpaceWriteMap in resertRyow()
2020-07-28 13:20:13 -07:00
a219ea853a
using const string&
2020-07-28 12:05:43 -07:00
8a5fde1b85
change SPECIAL_KEY_SPACE_CHANGE_CONFIGURATION to SPECIAL_KEY_SPACE_ENABLE_WRITES
2020-07-28 12:03:00 -07:00
3ba940a63d
Change json string to snake_case
2020-07-28 11:42:03 -07:00
726eb18119
Fix typo
2020-07-28 11:39:29 -07:00
d2cf700bd4
Fix compiler warnings
2020-07-28 18:30:26 +00:00
fbdbe0f052
adjusted locations in which restore pipeline is modified
2020-07-28 12:39:35 -04:00
2d09d2f68e
re-ordered parameters to fit existing test workloads
2020-07-28 12:39:35 -04:00
71a1b2cb0d
first attempt at making restore pipeline incremental only
2020-07-28 12:39:35 -04:00
3c62651ad6
initial commit to introduce incremental backup only (ignore base snapshot)
2020-07-28 12:39:35 -04:00
f065cbbd2d
Merge pull request #3534 from sfc-gh-xwang/master
...
Throttling related workload improvements
2020-07-28 00:40:06 -07:00
52178f9eae
Add json schema for the management api error message in special key space
2020-07-27 17:37:19 -07:00
4930b9f8a6
Update comments
2020-07-27 12:36:59 -07:00
a641429d63
Add an enum for implementation type and use for registeration
2020-07-27 12:29:03 -07:00
c3348be3a0
changed knobs to improve master recovery speeds
2020-07-27 10:59:06 -07:00
e0db748fb3
Merge pull request #3403 from satherton/tls-background-handshake
...
TLS handshaking in background threads
2020-07-27 10:55:00 -07:00
92b2894311
Allow in-place arena construction for MutationRef
2020-07-26 14:27:39 -07:00
2ac7df8a18
Merge pull request #3563 from xumengpanda/mengxu/fr-filesize-PR
...
FastRestore:Add trace for file size and bc progress
2020-07-25 20:08:57 -07:00
99d8399f4e
FastRestore:Add trace for file size and bc progress
2020-07-25 19:12:25 -07:00
1826ac75d5
Add some trace events to debug
2020-07-25 18:16:08 -07:00
9d3b1cc4e4
Merge pull request #3562 from sfc-gh-anoyes/anoyes/memory-error
...
Fix memory error in report conflicting keys special keys
2020-07-24 21:11:42 -07:00
57b29fd4cd
Fix memory error
2020-07-24 23:31:16 +00:00
b495c8a664
Remove unnecessary code
2020-07-24 12:25:03 -07:00
d3093f50bb
attempt to get rid of compiler warnings
2020-07-24 12:24:42 -07:00
e1dedff7b3
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# cmake/ConfigureCompiler.cmake
# documentation/sphinx/source/downloads.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/FileBackupAgent.actor.cpp
# packaging/msi/FDBInstaller.wxs
# versions.target
2020-07-24 12:10:44 -07:00
ec9553b6c6
change \xff\xff/failure to \xff\xff/error_message
2020-07-24 11:38:56 -07:00
Jon Fu