e9c9bd5299
Merge commit 'e2d3ddee2cb4da0bba65346d29ad2212fbce16f0' into file-backup-async-rename
...
# Conflicts:
# fdbclient/BackupContainer.actor.cpp
# fdbclient/FileBackupAgent.actor.cpp
# fdbclient/Knobs.cpp
# fdbrpc/AsyncFileNonDurable.actor.h
# fdbrpc/sim2.actor.cpp
2021-03-08 02:00:40 -08:00
318b862aa3
Apply clang-format to backup changes.
2021-03-08 01:49:29 -08:00
a397d8625f
AsyncFileNonDurable now has a flag for whether or not to emulate AIO mode, so the previous behavior limits are preserved when this flag is set. AsyncFileDurable writes which are selected to be durable are again done in a single write, and the logic to handle non-aligned writes for non-AIO mode is more clear and commented. In simulation, backup files on local filesystems are written with random buffer sizes. Backup container unit test now uses more random file sizes and limits memory consumption for all files being written in parallel.
2021-03-07 22:21:10 -08:00
8a4aca3f47
Bug fix: In simulation only, RangeFileWriter was not waiting on a write future in one location.
2021-03-07 22:19:04 -08:00
1d00c41385
Bug fixes: Avoid empty writes, avoid 0 length truncation, and make AsyncFileNonDurable support writes at arbitrary offsets and lengths. Increase randomness of IBackupFile appends in backup container unit test.
2021-03-05 17:45:36 -08:00
8a8c488ede
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-05 18:13:38 -06:00
0c214d71da
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-05 16:31:35 -06:00
1025a0da37
File-based backup now uses async rename to prevent blocking the network thread in the event of slow filesystem metadata operations. Backup now opens its write-only files without a block cache so that writes to disk will be larger (1MB by default instead of 4k).
2021-03-03 21:45:33 -08:00
79cec09255
Apply clang-tidy's performance-inefficient-vector-operation fix
...
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.
$ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
37d9e975e9
Fix multiple compiler warnings
2021-03-03 10:18:03 -07:00
346a4e3ecd
Merge branch 'release-6.3'
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbrpc/LoadBalance.actor.h
# fdbrpc/MultiInterface.h
# fdbserver/ClusterController.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/masterserver.actor.cpp
2021-03-01 18:52:06 -08:00
25069a9444
Fix compile error: Related to pr 3961
2021-02-27 17:10:21 -08:00
5b4e77bb99
Update comments, add asssertions
2021-02-25 13:13:34 -08:00
424aae71cc
Solve comments. Disable clear for ClientProfilingImpl and set the upperbound value for advanceversion
2021-02-25 13:02:56 -08:00
d8932f142a
Merge pull request #4350 from sears/multithreaded-pr-comments
...
Multithreaded pr comments
2021-02-23 15:09:52 -07:00
1ee9c3e45d
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-02-22 20:18:01 -07:00
d2440b5986
Merge pull request #4313 from xumengpanda/mengxu/ha-code
...
Add comments to explain loadBalance algorithm
2021-02-22 18:24:06 -08:00
e9b7e22e02
Merge branch 'rel-6.3'
2021-02-22 08:47:22 -08:00
e49cc68bbc
Merge branch 'rel-6.2' into rel-6.3
2021-02-21 00:15:31 -08:00
ef0bf2728e
Merge branch 'release-6.3' into mengxu/ha-code
...
Resolve Conflicts:
fdbserver/LogRouter.actor.cpp: Only conflicts at comments
2021-02-19 21:47:09 -08:00
9ce9cba0e5
Merge branch 'master' of github.com:apple/foundationdb into add-advanceversion
2021-02-19 14:29:08 -08:00
703107332a
Add special keys for profile client get/set
2021-02-19 12:22:00 -08:00
9fb396e372
Merge pull request #4255 from sfc-gh-clin/add-coordinators-into-special-keys
...
Add coordinators into special keys
2021-02-18 18:26:07 -08:00
fa3834f115
fix bug
2021-02-18 18:23:28 -05:00
70098b752a
Fix typo, solve comments
2021-02-18 14:23:51 -08:00
af77fae131
Fix bug in code I recently added where the process name memory was not long-lived and was later used.
2021-02-18 18:33:23 +00:00
ac860b3c22
Add advanceversion into special keys
2021-02-18 01:27:14 -08:00
971b1aa1e6
Add txnRejectedForQueuedTooLong in ProxyStats
...
Added transactions.rejected_for_queued_too_long for bookkeeping the
number of transactions rejected by commit proxy because its queuing
time exceeds MVCC window.
2021-02-17 21:40:46 -08:00
5324f127c9
Address skipped PR comments from multithreaded client PR
2021-02-17 21:10:42 +00:00
c075415a51
Merge pull request #4344 from sfc-gh-ljoswiak/fixes/remove-async-tracer
...
Remove async tracer
2021-02-17 09:43:24 -08:00
f3a849de10
Update ChangeConfig test for auto coordinators change
2021-02-16 23:55:58 -08:00
30d2d9ff82
Remove async tracer
2021-02-16 16:36:33 -08:00
bcd9a00a50
Backport #4285 and 1513df22f3 to release-6.2
...
Fixes #4342
2021-02-17 00:35:25 +00:00
fca7f753f8
Remove special key for coordinators' 'auto' option
2021-02-16 13:46:45 -08:00
ab1c51618a
Merge pull request #4269 from sears/multithreaded_client
...
Multithreaded client
2021-02-16 13:11:14 -08:00
9dd94499f2
Update traces
2021-02-16 13:06:25 -08:00
731ee8a121
Update ChangeConfig to test coordinators special keys
2021-02-16 13:01:37 -08:00
82cda5448d
Merge pull request #4266 from sfc-gh-nwijetunga/optimize-watches-client
...
Optimize Watches Client
2021-02-16 13:28:51 -07:00
a59f07e6fe
address pr comment
2021-02-16 13:32:03 -05:00
614d1584cc
Update documentation from https://github.com/apple/foundationdb/pull/4278
2021-02-15 23:56:55 -08:00
76a4293d1d
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/fdbserver.actor.cpp
2021-02-15 02:13:06 -08:00
bf3e6db9e9
Fix build issues; move to new assert style
2021-02-12 23:52:55 +00:00
36915e848a
clang format
2021-02-12 22:42:57 +00:00
555f3ebfaa
Implement + test corner case configurations for mutlithreaded client.
2021-02-12 22:37:24 +00:00
fb9632297e
Add txnRejectedForQueuedTooLong in ProxyStats
2021-02-12 13:04:58 -08:00
13e4f6d7e1
small formatting fix
2021-02-12 12:57:33 -05:00
6e483b6da8
Remove sendfile.h to fix non-Linux builds
2021-02-12 09:31:22 -08:00
058557a3b1
fix pr issues
2021-02-11 15:51:37 -05:00
4938aee113
Address PR comments, remove some old cruft + comments
2021-02-11 01:27:57 +00:00
9122be4d81
Add comments to HA code and loadBalance code
2021-02-10 13:51:36 -08:00
1dac117543
Merge pull request #4278 from dongxinEric/misc/document-two-new-fields-in-storage-metrics
...
Document two fields in storage metrics
2021-02-10 10:57:38 -08:00
5b74556733
Fix bug in code I recently added where the process name memory was not long-lived and was later used.
2021-02-10 10:46:02 -08:00
270a18f089
formatting fixes
2021-02-10 13:19:59 -05:00
e4f6a39e48
change client thread count from knob to fdb.option (to be more consistent with other multiversion client options)
2021-02-10 16:29:27 +00:00
7acfeda955
cleanup
2021-02-09 20:57:16 -05:00
53739c9da5
Merge pull request #4241 from sfc-gh-clin/add-c-function-snapshot
...
Add c function for snapshot
2021-02-09 15:49:20 -08:00
386e3b0838
minor change
2021-02-09 18:36:45 -05:00
4982679ef2
change function name
2021-02-09 18:06:36 -05:00
3b48942f9a
resolve pr comments
2021-02-09 17:56:30 -05:00
d2233ffa5a
whitespace change
2021-02-09 13:38:24 -05:00
82e9f54938
small change
2021-02-09 13:36:44 -05:00
2d46cfcb80
remove traces
2021-02-09 13:33:19 -05:00
2560dc0227
add test workload
2021-02-09 12:32:37 -05:00
90686a95a0
format code
2021-02-08 22:17:16 -08:00
7c1ef37bcb
format code
2021-02-08 21:42:40 -08:00
61082e70e1
format code, add comments
2021-02-08 19:11:30 -08:00
e3c1530a15
format code
2021-02-08 18:56:12 -08:00
3def9731a4
Merge branch 'master' of github.com:apple/foundationdb into add-coordinators-into-special-keys
2021-02-08 18:52:45 -08:00
b10959be0f
format code, add trace
2021-02-08 18:52:36 -08:00
0947b87b60
Update documentation
2021-02-08 18:47:49 -08:00
e8e11d8f6f
Fix compiler warning now being treated as error
2021-02-08 14:38:41 -08:00
be57b70dbb
Fix function call where the signature changed
2021-02-08 14:00:47 -08:00
9e9bc4abe5
Create one client thread for each one that was requested (assumes local client is disabled)
2021-02-08 21:20:37 +00:00
aaf0a9aa7b
Merge branch 'release-6.3' into merge-release-6.3-into-master
...
# Conflicts:
# build/docker-compose.yaml
# cmake/ConfigureCompiler.cmake
# fdbclient/FileBackupAgent.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/IAsyncFile.h
# fdbrpc/IRateControl.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
a78cb179e8
Merge pull request #4291 from sfc-gh-ajbeamon/merge-release-6.2-into-release-6.3
...
Merge release 6.2 into release 6.3
2021-02-08 12:47:50 -08:00
67e783acf8
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# cmake/FDBComponents.cmake
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/storageserver.actor.cpp
# flow/Knobs.h
# flow/network.h
2021-02-08 09:20:28 -08:00
84be77958e
Add a simulator guard
2021-02-05 22:47:49 -08:00
53d8bd5799
If a quorum change has to retry, it's possible that one of our previously chosen coordinators could be rebooting.
2021-02-05 22:36:54 -08:00
5c4863b33e
Fix unit test that set coordinators to fake processes that the simulator didn't know about
2021-02-05 22:07:40 -08:00
83af53b1bb
correctness fixes
2021-02-05 19:23:37 -05:00
618585f9dd
Fix a bug in simulated coordinator selection that could allow an unreliable process to be chosen as coordinator despite the intent that this not happen.
2021-02-05 15:59:14 -08:00
a85458dd1e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-snapshot
2021-02-05 14:14:52 -08:00
97925de9d4
Update fdb_database_create_snapshot to take a UID as input
2021-02-05 14:14:38 -08:00
d185ff3752
update \xff\xff/configuration/coordinators/processes
2021-02-05 00:55:34 -08:00
10c1bd475e
Improve comments based on review
2021-02-04 17:31:05 -08:00
1095dba821
Document two fields in storage metrics
2021-02-04 16:20:18 -08:00
b1fd8a439e
revert NativeAPI
2021-02-04 12:54:18 -05:00
6c687067a6
refactor
2021-02-04 12:51:53 -05:00
675e567305
Merge pull request #4218 from sfc-gh-etschannen/feature-low-priority-reads-6.2
...
Reduce read priority when durability lag is high
2021-02-03 18:09:39 -08:00
ec6226bde9
change logic
2021-02-03 20:01:26 -05:00
ad3de5c9e0
WIP: seems to work?
2021-02-03 23:19:26 +00:00
1032a5d005
WIP
2021-02-03 21:45:35 +00:00
b2ffdf47f0
added low priority reads to status
2021-02-03 13:24:34 -08:00
f50206aabb
fdbclient: Copy external libraries MULTI_VERSION_CLIENT_THREADS_PER_VERSION times
...
To support multiple threads in for each multi-version clients, we make a copy of
external library for each thread in `/tmp/` directory and load that.
(cherry-picked by Rusty; there were some conflicts, so this has commented out cruft in it)
2021-02-03 20:47:15 +00:00
87aeafb294
partial refactor to support multiple threads per client library
2021-02-03 19:58:37 +00:00
b6e7961b78
change to references
2021-02-02 20:29:03 -05:00
c67aac8212
remove the use of pointers
2021-02-02 13:26:12 -05:00
6f38105e48
fix issue with case 4
2021-02-01 19:07:14 -05:00
6bee8ac0aa
Inital implementation
2021-02-01 18:42:13 -05:00
3fc8870ce3
winbuild: fix duplicate definition of Codec<ERestoreState>::pack and unpack in Debug build
...
- defined twice in FileBackupAgent.actor.cpp and BackupAgent.actor.h
- only fails when building in Debug ?
2021-01-30 19:03:25 +01:00
c2aedb0b94
clang-format, remove comments
2021-01-29 18:20:09 -08:00
766de6574a
Add trace
2021-01-29 15:32:53 -08:00
5a62e0c084
Fix repeated code usage
2021-01-29 12:20:30 -08:00
7d32045506
Merge branch 'master' of github.com:apple/foundationdb into add-coordinators-into-special-keys
2021-01-29 11:46:01 -08:00
e1182f2f86
Add coordinators to special keys
2021-01-29 11:45:52 -08:00
5b5dd0a0d4
Merge pull request #4204 from sfc-gh-ljoswiak/features/network-tracer
...
Add network option to set tracer in fdbclient
2021-01-27 20:00:49 -08:00
cef53b2b40
clang-format
2021-01-26 12:06:40 -08:00
5f912f0deb
Add snapshot c function
2021-01-26 11:53:35 -08:00
7558cd8e06
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-26 10:00:25 -08:00
074531309b
Merge pull request #3769 from sfc-gh-ljoswiak/binary-info
...
Add CLI argument to print build information
2021-01-26 10:29:57 -07:00
564c70acb4
Merge branch 'master' into anoyes/modernize-use-override
2021-01-25 18:26:18 -08:00
a51de2f73d
Merge pull request #4220 from sfc-gh-clin/add-c-function-force-recovery
...
Add c function fdb_database_force_recovery_with_data_loss
2021-01-25 18:22:41 -08:00
cec050a1f8
Apply clang-tidy's modernize-use-override fix
2021-01-26 02:04:12 +00:00
4ee97c0784
Use clang-tidy to automatically fix missing overrides
...
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.
Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.
- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
c3775ede37
add override keyword properly to squash compiler warnings
2021-01-25 13:24:15 -05:00
6e619e9157
Update the returned future when local client is disabled
2021-01-25 00:41:50 -08:00
29c0e05ea6
Clarify naming of distributed tracing options
2021-01-22 17:30:33 -08:00
e74fa1238b
Merge pull request #4181 from sfc-gh-tclinkenbeard/azure-read-remove-copy
...
Remove unnecessary copy from BackupContainerAzureBlobStoreImpl::read
2021-01-22 14:13:23 -08:00
500eabf6dd
Add network option to set tracer in fdbclient
2021-01-21 12:04:28 -08:00
fb9b4b7626
Merge remote-tracking branch 'origin/master' into misc-changes
2021-01-21 10:45:50 -08:00
ad99bf0471
Merge remote-tracking branch 'origin' into misc-changes
2021-01-21 10:03:07 -08:00
9b5fc3b638
Update code
2021-01-21 01:22:53 -08:00
116f034933
update var names
2021-01-21 01:11:40 -08:00
9cd7c43a2c
Add c function fdb_database_force_recovery_with_data_loss
2021-01-21 00:42:23 -08:00
fd20011276
Remove tags
2021-01-20 17:05:58 -08:00
4d6fbf099c
Use StringRef for span tags
2021-01-20 13:57:36 -08:00
7ae5e98c5a
Add metadata to a few spans
2021-01-20 13:57:36 -08:00
7c9070df39
Fix macOS compilation issue
2021-01-19 11:21:21 -08:00
88f8145dec
Add CLI option to get build information
2021-01-19 11:21:21 -08:00
6fb61ca2bc
Merge pull request #4207 from sfc-gh-anoyes/anoyes/merge-6.3-to-master
...
Merge 6.3 into master
2021-01-19 10:43:34 -08:00
b8ff724232
Merge pull request #4173 from sfc-gh-anoyes/anoyes/fdb-c89-test
...
Make sure that fdb_c.h is c90 compliant
2021-01-19 10:42:40 -08:00
c101c6ec0c
Resolve last conflict
2021-01-15 21:51:25 +00:00
5743fe724c
Restore buffering logic lost to botched merge
2021-01-15 20:10:39 +00:00
dadc4d1b21
Initialize m_offset
2021-01-15 19:56:28 +00:00
b0f61fb74f
Resolve the simple-looking conflicts
2021-01-15 19:35:10 +00:00
ff7d306b09
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Include conflict markers for now. Will resolve.
2021-01-15 18:04:09 +00:00
13afadfae4
Use c90 compatibility for clarity
2021-01-15 02:50:34 +00:00
8ce9bd4e8e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2021-01-14 17:31:26 -08:00
e32a0fb742
Resolve conflicts
2021-01-14 20:13:29 +00:00
6d6b2843da
Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-release-6.2
...
I'm including the conflict markers for now and I'll resolve them in a
subsequent commit
2021-01-14 17:20:11 +00:00
95eaa5e866
Merge remote-tracking branch 'origin/master' into misc-changes
2021-01-13 21:14:36 -08:00
87b4463305
Solve comments: update c api description; refine code
2021-01-13 14:56:41 -08:00
8ff14878fe
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-13 14:39:35 -08:00
e29ed3bf99
Remove createGlobal*Knobs functions
2021-01-13 12:14:04 -08:00
fe71ef88c3
Merge pull request #4199 from sfc-gh-satherton/file-backup-write-buffering
...
Add write buffering to BackupContainerLocalDirectory::BackupFile
2021-01-13 12:23:57 -07:00
2609c3d619
Merge pull request #4072 from sfc-gh-tclinkenbeard/improve-type-safety
...
Make enums automatically binary serializable
2021-01-12 10:31:34 -07:00
a860778b51
Add write buffering to BackupContainerLocalDirectory::BackupFile to greatly reduce the number of IAsyncFile::write() calls made when writing backup data. The buffer size is controlled by a knob.
2021-01-09 07:57:48 -08:00
fbcc85843f
clarify comment for serverKeys possible values
2021-01-06 17:15:31 -05:00
43a79a34ff
Merge pull request #4175 from vishesh/task/document
...
Docs, fix and hack for Fault Tolerance
2021-01-05 15:46:45 -08:00
58360a192f
Add a counter for the number of status requests made by a client
2021-01-05 10:18:43 -08:00
278e7de6f5
added more documentation
2021-01-04 16:59:08 -05:00
a0d33c5d03
added more comments
2021-01-04 16:00:12 -05:00
53d23d95d8
initial commit in adding more documentation to special keys
2021-01-04 15:26:48 -05:00
e097c98956
Remove headed added by IDE, add comments
2021-01-04 00:18:01 -08:00
da7b65cd71
Change the interface back to return int64_t
2021-01-03 22:45:09 -08:00
e7e2439abe
Remove unnecessary copy from BackupContainerAzureBlobStoreImpl::read
2021-01-01 09:43:03 -04:00
70e62d34ca
Merge remote-tracking branch 'origin/master' into misc-changes
2020-12-28 01:58:56 -04:00
86c7c1e946
Fix IAsyncFileSystem method signatures
2020-12-28 01:57:42 -04:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
5bfa6cea98
Merge remote-tracking branch 'origin/master' into misc-changes
2020-12-26 20:47:00 -04:00
33ec968d5a
Mark expectedSize methods const
2020-12-26 18:30:44 -04:00
4e460c0b30
Remove BlobStore.actor.cpp
2020-12-26 13:30:16 -04:00
bbb56e4089
Merge branch 'release-6.2' of https://github.com/apple/foundationdb into release-6.3
2020-12-23 14:26:59 -08:00
6957e64886
fix: Calculate fault tolerance when keys are not fully replicated
...
Whem the database is not fully replicated to remote, fault tolerance
can be calculated by couting total number of tLogs replicas in primary
and satellite - 1, as we can lose all but one of those, but we were
subtracting 2 instead of 1.
2020-12-22 16:26:19 -08:00
a1809f7d86
doc: Fault Tolerance and Region Configuration
2020-12-22 16:25:17 -08:00
8d4cbfbb86
Merge branch 'release-6.2' into feature/allow-manually-trigger-dd-teams-info-logging
2020-12-22 10:00:24 -08:00
0ecfaf138e
Merge pull request #4150 from sfc-gh-ljoswiak/fixes/sps-tracing
...
Remove "a/" from tracing keys in special-key-space
2020-12-21 13:48:01 -08:00
9db9a84798
Ignore negative duration value
2020-12-21 13:29:16 -08:00
b68d84aaea
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-12-21 12:32:31 -08:00
7b4f7ce4e0
update comments
2020-12-20 22:47:55 -08:00
adc916121e
Copy the network address stringref
2020-12-20 22:44:04 -08:00
fb62ce4f64
Removed the override
2020-12-14 08:34:09 -05:00
24cdbc5cee
Fixed error with Windows CMake build
2020-12-14 08:23:24 -05:00
9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces
...
Stable interfaces
2020-12-11 09:34:27 -08:00
5f57d72a59
remove print statements and format protocol version workload
2020-12-11 04:46:20 +00:00
4cb821cd63
Merge remote-tracking branch 'upstream/release-6.2' into merge-6.2-to-6.3
2020-12-09 15:47:44 -08:00
43fd545f06
Remove a/ from tracing path in special-key-space
2020-12-09 13:40:16 -08:00
b305e71ece
Merge pull request #4141 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
...
Merge release 6.3 master
2020-12-09 08:50:57 -08:00
c8786a65c3
Remove unnecessary uses of virtual
2020-12-08 09:09:33 -08:00
276f80cfce
Mark Tag::toTagDataIndex const
2020-12-08 09:09:33 -08:00
9581bc0279
Merge pull request #4129 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
...
Merge release 6.3 master
2020-12-08 08:57:54 -08:00
54237f28ea
Throw error on commit for late trace options
2020-12-07 14:43:44 -08:00
cc669f399e
Merge remote-tracking branch 'upstream/release-6.3' into anoyes/merge-release-6.3-master
2020-12-07 22:26:11 +00:00
18201d6f4f
Add option to enable transaction tracing
2020-12-07 11:24:14 -08:00
75836d6114
Fix double counting of range reads in TransactionMetrics
2020-12-07 09:01:19 -08:00
d97523a966
Fix typo
2020-12-06 14:04:16 -08:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
e5cad0f077
Fix the bug if boundary same as range and add comments
2020-12-04 14:58:40 -08:00
76142549e0
Return keys from range read in lexicographic order
2020-12-04 11:21:39 -08:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
ac02329d7d
Added a command in fdbcli to allow user to manually trigger the detailed teams info loggings in data distributor
2020-12-03 14:42:30 -08:00
3669615e4b
Refactor and clean up
2020-12-03 14:06:11 -08:00
8d59dae26d
Add check for existence of databaseRebootWorker in c api
2020-12-03 13:47:59 -08:00
88e66545bb
Change the api to return bool instead of int
2020-12-03 12:01:32 -08:00
c77d9e4abe
merge conflicts
2020-12-02 21:53:19 +00:00
1f166bc183
Add options to set custom transaction ID and disable tracing
2020-11-30 14:57:17 -08:00
dc2bac5670
Resolve conflicts
2020-11-24 19:09:42 +00:00
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
2fae7333c0
Add duration parameter in the api
2020-11-23 14:55:29 -08:00
b46384e1b8
Initialize client and flow knobs
2020-11-22 09:56:18 -08:00
b1e3478267
Simplified global knobs
2020-11-21 13:27:48 -08:00
615029a393
Merge pull request #4082 from sfc-gh-dyoungworth/dyoungworth/merge_6_3_master
...
merge 6 3 master
2020-11-19 20:53:51 -07:00
2320f970c1
Add reboot_worker and pythong binding
2020-11-19 16:43:39 -08:00
cc4b282236
Remove the incorrect(unnecessary) versioning stuff, since the error is thrown by the server
2020-11-19 13:17:38 -08:00
da39b05620
Add comments about error type change
2020-11-18 12:14:59 -08:00
a101d3d706
Add versioning
2020-11-18 12:06:31 -08:00
a4174d3108
Change operation_failed to dd_not_found
2020-11-18 00:18:42 -08:00
aec0398fd0
Merge remote-tracking branch 'upstream/master' into add-c-function-for-management-commands
2020-11-17 11:39:12 -08:00
d64cf8b9e3
Merge branch 6.3 into master
2020-11-17 11:22:45 -08:00
fdf41110e5
Merge branch 'release-6.3' into dyoungworth/merge_6_2_to_6_3
2020-11-16 14:49:51 -08:00
d0391db862
Merge branch 'release-6.2' into release-6.3
2020-11-16 10:15:23 -08:00
12a6205d99
Move MetricLogger from fdbclient to fdbserver
2020-11-15 11:41:57 -08:00
82a50ea157
Improve type safety of ClientLogEvents
2020-11-14 19:22:19 -08:00
eab75d4ee1
Make enums automatically binary serializable
2020-11-14 19:22:04 -08:00
45c9a0abc7
Revert "Revert "Add limiting health metrics""
...
This reverts commit 209ebcc595
2020-11-13 17:24:57 -08:00
209ebcc595
Revert "Add limiting health metrics"
2020-11-13 17:08:46 -08:00
8c0b4dbe4c
Merge pull request #4067 from sfc-gh-tclinkenbeard/add-limiting-health-metrics
...
Add limiting health metrics
2020-11-13 16:04:44 -08:00
6c4493166f
Add limiting storage queue and durability lag to health metrics
2020-11-12 20:14:41 -08:00
222da17558
Merge branch 'release-6.2' into mengxu/ha-code-read
2020-11-12 13:39:27 -08:00
cc13ef08bd
Sort the failed sets before modifying them in attempts to make changes consistent
2020-11-12 16:26:34 -05:00
0f205620bf
Merge branch 'master' into features/udp
2020-11-12 13:51:17 -07:00
c2dd7d1d38
Remove unresolved questions
2020-11-11 22:39:11 -08:00
1343f40117
don't allow empty coments
2020-11-11 14:07:54 -07:00
bdd3dbfa7d
remove duplicates
2020-11-10 14:01:07 -07:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
51db9a7e0a
add static method to access backup pause key instead of constructing it manually
2020-11-06 14:03:29 -05:00
4788544a6f
Revise comments based on review suggestions
...
Ack. Jingyu and Xin for their suggestions.
2020-11-06 08:51:13 -08:00
2909e35415
When moving dispaching shard relocations, log the source and destination storage servers.
2020-11-05 16:13:18 -08:00
c50e997f60
Make status tests deterministic
...
This change seems to be incorrect since afaict INetwork::timer isn't
guaranteed to be monotonic. Maybe we can make that guarantee or add an
INetwork::timer_monotonic symbol?
2020-11-05 17:05:34 +00:00
1664e2ff7f
Add more comments and questions to LR tLog and loadbalance
2020-11-01 21:22:23 -08:00
9c6c2704d0
Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces
2020-10-30 23:59:32 +00:00
063700e4d6
Add comments and questions to HA and tLog code reading
...
The comments' correctness need to be confirmed by reviewers.
2020-10-30 12:14:57 -07:00
ca26cb57a2
Reapply BlobStore bug fix to S3BlobStore.actor.cpp
...
Because BlobStore.actor.cpp was renamed, this commit was lost in the
merge, so reapplying here
2020-10-30 10:06:35 -07:00
09350c760e
Merge remote-tracking branch 'origin/master' into merge
2020-10-30 08:36:19 -07:00
035708f43a
Post-merge cleanup
2020-10-29 22:35:27 -07:00
0faaecc780
Merge branch 'master' into merge
2020-10-29 22:27:40 -07:00
cf4c8e375f
Merge remote-tracking branch 'origin/release-6.3' into merge
2020-10-29 22:15:41 -07:00
55e86c1336
Disambiguate between S3BlobStore and other blob stores
2020-10-29 20:42:23 -07:00
96cc700440
Merge pull request #3961 from sfc-gh-tclinkenbeard/azure-backup
...
Add support for writing backup files directly to Azure blob storage
2020-10-29 00:30:40 -07:00
a72bb52eae
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-for-management-commands
2020-10-28 22:32:44 -07:00
99c1880a83
Merge commit 'f9581de2005e6b085776e81b9fcaa16442b32589' into merge-6.2-to-6.3
...
# Conflicts:
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
2020-10-27 12:21:26 -07:00
bf7bf470d9
Merge branch 'master' of https://github.com/apple/foundationdb into stable_interfaces
2020-10-26 19:36:19 +00:00
5b546c4854
change protocol version to hex encoded string in status json. Move constant from flow transport header to cpp
2020-10-26 19:35:38 +00:00
a15bc32b14
Merge pull request #3945 from sfc-gh-clin/add-consistencycheck-to-special-keys
...
Add consistencycheck command to special keys
2020-10-26 10:59:17 -07:00
9501edb960
Bug fix, blob client was not correctly using keys longer than 64 bytes in request signing.
2020-10-25 21:11:04 -07:00
2a849dc52c
Update BackupContainerAzureBlobStoreImpl::WriteFile::sync
2020-10-24 16:48:06 -07:00
195873e1cc
Cleaned up unnecessary backup container includes
2020-10-24 16:48:05 -07:00
195ca30a20
Azure backup cleanup
2020-10-24 16:48:05 -07:00
e389a5dabe
Fix issues with Azure credentials
2020-10-24 16:48:05 -07:00
c6739f7075
Add skeleton for BackupContainerAzureBlobStore::listURLs
2020-10-24 16:48:05 -07:00
bb09f0e657
Improve AsyncTaskThread encapsulation
2020-10-24 16:48:05 -07:00
7cba81316a
Fix some race conditions in AsyncTaskThread (some more remain)
2020-10-24 16:48:04 -07:00
4d127a29cc
Added testing and fixed some bugs in AsyncTaskThread
2020-10-24 16:48:04 -07:00
456a7c77a2
Add random delay to AsyncTaskThread::execAsync in simulation
2020-10-24 16:48:04 -07:00
f3c52ed41e
Guard Azure backup compilation behind CMake variable
2020-10-24 16:48:04 -07:00
e3f94c80fc
Added BackupContainerAzureBlobStore::getURLFormat
2020-10-24 16:48:03 -07:00
dc3738b97f
Improve encapsulation for BackupContainerFileSystem
2020-10-24 16:48:03 -07:00
6a27619544
Remove unnecessary copies in BackupContainer code
2020-10-24 16:48:03 -07:00
1d28285cc5
Made BackupContainer.h header guard consistent with other header guards
2020-10-24 16:48:03 -07:00
ae91cf1d44
Make BackupContainerFileSystem.h a non-actor file
2020-10-24 16:48:02 -07:00
9242456d71
Improve encapsulation for BackupContainerAzureBlobStore
2020-10-24 16:48:02 -07:00
226b3f6bf2
Improve encapsulation for BackupContainerS3BlobStore
2020-10-24 16:48:02 -07:00
e1d58d3c66
Moved BackupContainerS3BlobStore into its own files
2020-10-24 16:48:02 -07:00
e15a268548
Use anonymous namespace for BackupContainerLocalDirectory helper functions
2020-10-24 16:48:02 -07:00
961aeaecac
Moved BackupContainerAzureBlobStore into its own files
2020-10-24 16:48:01 -07:00
e5338d213b
Moved BackupContainerLocalDirectory into its own files
2020-10-24 16:48:01 -07:00
3c6de425a1
Moved BackupContainerFileSystem implementation to cpp file
2020-10-24 16:48:01 -07:00
cd50d53b1e
Created BackupContainerFileSystem.actor.h
2020-10-24 16:48:01 -07:00
29d6fc5dff
Clarify scope of IBackupFile_impl namespace
2020-10-24 16:48:00 -07:00
0879e529d2
Add error handling to AsyncTaskThread::execAsync
2020-10-24 16:48:00 -07:00
0a99438d84
Update pNumDeleted in BackupContainerAzureBlobStore::deleteContainer
2020-10-24 16:48:00 -07:00
2049789a99
Remove BackupContainerAzure.actor.cpp
2020-10-24 16:47:59 -07:00
ad911f4d61
Encapsulate some methods in BackupContainerAzureBlobStore
2020-10-24 16:47:59 -07:00
72b24c36e7
Fix header for AsyncTaskThread.h
2020-10-24 16:47:59 -07:00
82b6daa16b
First draft of Azure blob storage backup container
2020-10-24 16:47:51 -07:00
309e38dc06
add unit test and some cleanup
2020-10-23 19:34:18 +00:00
9043357934
Merge pull request #3360 from sfc-gh-tclinkenbeard/use-vector-for-tagset
...
Use std::vector for TagSet
2020-10-23 10:34:20 -07:00
055add9682
conflicts
2020-10-23 06:33:00 +00:00
90caa24af1
add function to get correct checksum class. Remove IChecksum::checkSum function
2020-10-23 02:24:29 +00:00
bd578a0d95
clang-format
2020-10-22 18:27:58 -07:00
3cbed7b0d1
Add checker to make sure registered special key range following snake case naming
2020-10-22 18:25:51 -07:00
c8eb003ec4
Change to \xff\xff/management/in_progress_exclusion, the snake case naming style
2020-10-22 18:05:54 -07:00
c8b3b37b4a
Remove unused code for #3930
2020-10-22 17:04:08 -07:00
0060eeb6f8
Merge pull request #3930 from sfc-gh-clin/fix-FuzzApiCorrectness
...
Fix FuzzApiCorrectness test failures
2020-10-22 16:24:37 -07:00
4c42917b1d
Fix typos in comments
2020-10-22 15:36:10 -07:00
1d081401c6
Fix Tag::load bug
2020-10-22 14:13:14 -07:00
95654ce7e7
Change dbLocked to snake_case naming db_locked
2020-10-22 11:35:39 -07:00
2b4ec39a2f
Avoid passing references from an iterator into an insertion that invalidates the iterator.
2020-10-22 11:30:08 -07:00
1d64b2c335
clang-format
2020-10-22 11:28:03 -07:00
b728fe473e
Add consistencycheck command to special keys
2020-10-22 11:08:54 -07:00
dec643987f
Add wait time after dd not found error
2020-10-21 16:58:23 -07:00
8d937a56f2
Throw operation_failed() if dd not found when calling ddMetricsRequestServer
...
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
e183a10c8c
Add "waitForDestUID" flag for abortBackup
2020-10-20 22:55:31 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
c84a5f4014
change IChecksum interface and remove coordinator protocol version optional
2020-10-19 21:39:07 +00:00
a4ccf8c5b6
add checksum class to change checksum in the future. expose get server protocol api
2020-10-19 16:57:11 +00:00
944f30484a
Merge pull request #3759 from dongxinEric/misc/3739/expose-time-since-last-recovery
...
This resolves issue #3739 by exposing time since last full recovery.
2020-10-19 09:03:31 -07:00
47a9bc9d3d
Fix commit_unknown_result issue
2020-10-18 23:11:19 -07:00
bfd3328448
Fix a race between submit and abort backup
...
After submit a backup, immediately abort the backup may cause a rare race
condition, which results in BackupCorrectnessLeftoverVersionKey error.
Specifically, in the StartFullBackupTaskFunc:
1st Txn sets the destUid at the source database and the 2nd Txn writes the dest
DB.
An abort can come after the 1st Txn succeeds, and clears the config rage so
that the 2nd Txn above would fail. Because 2nd Txn didn't write destUid, the
3rd Txn of abort can't read the correct source DB for latestVersionKey, which
contains the destUid value.
The fix is to let the 1st Txn of abort to wait until destUid becomes valid.
2020-10-18 23:11:15 -07:00
545ee4269d
master conflicts
2020-10-19 01:03:54 +00:00
1c533e7363
fix flowtransport conflicts
2020-10-19 01:00:03 +00:00
c1d88b39ab
Merge remote-tracking branch 'origin/master' into add-upgrade-and-backup-restore-test
2020-10-17 11:35:12 -07:00
c1b4e62bde
Change the trace from SevInfo to SevDebug
2020-10-15 13:55:31 -07:00
0718329705
Make getDesiredClusterKeyName const
...
This fixes a virtual function overload in NameQuorumChange
2020-10-15 10:39:38 -07:00
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
...
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
bf00369576
getRange only enters special key space codepath when both begin key and end key are in (\xff\xff, \xff\xff\xff)
2020-10-14 16:57:38 -07:00
e47e0108a7
Add new TLogVersion
2020-10-14 15:41:10 -07:00
a1e868a569
Merge pull request #3862 from sfc-gh-tclinkenbeard/use-override-more
...
Add uses of override keyword, remove unnecessary uses of virtual
2020-10-14 15:06:45 -07:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
4eb20a1283
merge anoyes/stable-interface and add back in isCompatible
2020-10-12 20:39:37 +00:00
2f5b0bef08
switch to test newer incompatible version. Fix PR comments. Modify schema
2020-10-12 18:29:16 +00:00
a4e1efbc0e
revert MultiVersionTransaction
2020-10-12 18:25:14 +00:00
5488ff1d81
draft diff protocol
2020-10-12 18:24:03 +00:00
b233f44d2d
remove some print statements and spin lock that was used for debugging
2020-10-12 18:20:37 +00:00
41843f07e6
add simulator support for different process versions and ProtocolVersion test
2020-10-12 18:19:31 +00:00
76d0027fa2
merge anoyes/stable-interface and add back in isCompatible
2020-10-12 18:18:30 +00:00
d3b57d9873
Merge pull request #3823 from sfc-gh-jfu/jfu-snapshot-record-version
...
Have snapshot record to system keys and write lastEpochEnd on recovery
2020-10-09 07:27:19 -07:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
ccf2ca73f3
Fix Windows CMake build
2020-10-08 23:58:58 -07:00
76440018fb
Remove unnecessary error thrown when calling clear range on the lock special key range
2020-10-08 21:39:13 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
c66a775d64
Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys
2020-10-08 14:23:16 -07:00
53d3759e73
Add lock command to special key \xff\xff/dbLocked
2020-10-08 14:23:02 -07:00
2def07a18d
Apply suggestions from code review
2020-10-08 13:31:47 -07:00
a9607bdcec
Explicitly seal classes that inherit but aren't inherited from
2020-10-07 21:58:24 -07:00
a3de8c333b
Use override where applicable in fdbclient
2020-10-07 16:57:38 -07:00
f03a355c4e
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/FlowTransport.h
# flow/Arena.h
# flow/Knobs.cpp
# flow/Knobs.h
# packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
7218586f2f
adjust test spec and backup describe
2020-10-06 17:03:09 -04:00
76da0a7cbc
Backport #3426 to release-6.2
2020-10-05 13:53:51 -07:00
c622a11c70
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-05 13:15:56 -04:00
43ac7f21a4
Mark toString methods const
2020-10-04 23:27:10 -07:00
d944e13e5b
Make TaskBucket::getAvailableSpace const
2020-10-04 23:27:09 -07:00
8ae0679112
Remove unnecessary separation of declaration and definition of version field
2020-10-04 23:27:05 -07:00
b294e148c5
another compile fix
2020-10-04 16:54:43 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
6326de052b
Remove unused LogFileWriter::FFs
2020-10-02 17:03:08 -07:00
cacf4be449
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-02 12:57:17 -04:00
268ba0bddc
Merge remote-tracking branch 'origin/release-6.3' into merge-6.3
2020-10-01 14:14:06 -06:00
13d1978c19
Merge branch 'release-6.3' into feature-fix-fault-tolerance
2020-09-30 19:00:31 -07:00
8c729ca8e6
only add additional fault tolerance for availability if automatic failover is enabled
2020-09-30 18:04:23 -07:00
c7a55fe15f
Merge pull request #3829 from yliucode/process-class
...
Support specifying proxy as process class and converting it to commit proxy
2020-09-30 16:46:24 -07:00
fe5c30e778
fault tolerance was not being properly increased when usable regions was 2 and satellites are configured.
2020-09-30 16:41:00 -07:00
0c579674e6
Add more help messages for configuring 'proxies'
2020-09-30 13:29:01 -07:00
e2705668be
Lower WATCH_TIMEOUT default value to 30.0
2020-09-30 11:52:51 -07:00
8da8c6ff0d
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-30 14:25:29 -04:00
44cd3b0999
add stopBackup to incrementalBackup workload
2020-09-30 14:24:52 -04:00
3705f5a6fe
Merge pull request #3820 from sfc-gh-tclinkenbeard/improve-type-safety
...
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-30 10:46:59 -07:00
c34385ef7b
Support proxy process class
2020-09-30 01:23:10 -07:00
21071b6214
Merge pull request #3813 from xumengpanda/mengxu/merge-to-master-PR
...
Merge 6.3 to master and resolve conflicts
2020-09-28 15:24:18 -07:00
Steve Atherton