2e1d03cbe7
Addressed AJ's review comments
2020-02-12 14:57:40 -08:00
d20ce99774
Resolved the review comment and renamed the functions
2020-02-12 14:57:40 -08:00
d934aed1d7
Because when the user issue 'getStorageByteSample' on a large key range, which can be as large as the whole DB, we need to change the behavior of 'waitStorageMetricsMultipleLocation' to avoid the case where a target key range got moved/splited by DD and thus the call to 'waitMetircs' on the corresponding storage server will return 'wrong_shard_server' error and thus the whole 'waitStorageMetricsMultipleLocation' will be retried on the large key range. What we want is to do the retry only for the key range that caused the error.
2020-02-12 14:57:40 -08:00
0c16d43c2f
Added necessary plumbings to expose byte sample collected by storage servers to fdb_c library
2020-02-12 14:57:40 -08:00
73ad702d14
Clients which fetch status should not disconnect from the coordinators and cluster controller between each retrieval
2020-01-22 15:41:22 -08:00
7224807d1d
fix typo in var name
2020-01-21 15:38:48 -08:00
18e6e36a8e
Fill in the versionstamp, but still leave the position.
...
Using range.begin() stripped off the last 4 bytes that specified where
the versionstamp is, so when the mutation got to the proxy, it wouldn't
get versionstamped correctly.
2020-01-21 15:24:57 -08:00
5386b4ecdf
Adding prefix to returned keys, keeping consistency with query range
2020-01-21 15:18:16 -08:00
5b62d672dc
update comments
2020-01-17 15:18:07 -08:00
17721a00f1
change vars names
2020-01-17 14:26:45 -08:00
d17d1c88cd
add const to local KeyRef
2020-01-17 11:31:43 -08:00
eb64eede8d
Make a smaller range inaccessable after writing a versionstamped key
...
A transaction's read version is the lower bound of what a transaction's
commit version could be. Thus, we can narrow the conflict range of a
versionstamped key, and thus reduce the amount of the keyspace that is
rendered inaccessable, by filling in the read version on the
versionstamped key and using that as the lower bound of the conflict
range.
This allows reads to still be done to versionstampled keys lower than
the read version of the current transaction.
2020-01-16 21:41:59 -08:00
1cf3b5b807
Change to general prefix '\xff\xff/transaction/conflicting_keys/'
2020-01-13 20:54:16 -08:00
1f633d3e5f
change local var from Key to KeyRef
2020-01-10 15:53:06 -08:00
1cdb22c4a4
refine code according to comments
2020-01-09 15:42:21 -08:00
b8ffc72cca
Use a RYW in transactionInfo to track conflicting keys
2020-01-07 13:55:05 -08:00
57fb28af2c
Explicitly set read-version and clear the whole keyspace to make sure the getRange happens locally
2020-01-07 13:54:33 -08:00
10719200c3
A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>").
2020-01-06 11:22:11 -08:00
b376b947d5
Encode conflicting keys using base64.
2019-12-06 16:21:19 -08:00
eacfdf6da3
Add a simple workload, ReportConflictingKeysWorkload, to test correctness of the API and performance overhead added to the resovler.
2019-12-06 16:21:03 -08:00
edf52e8c97
First version for reporting conflicting keys
2019-12-02 13:45:39 -08:00
e3644d9c00
Add a little extra time to the timeout delay so that we don't end up in a situation where we get stuck in a loop due to floating point math.
2019-11-13 13:13:36 -08:00
5bd4bf357f
Limit length of delays in timeout, repeating them as necessary.
2019-11-13 12:49:07 -08:00
bc5c65e5ab
Merge pull request #1756 from jzhou77/db-option
...
Add transaction getApproximateSize() API
2019-07-19 08:33:24 -07:00
7c0aca5b0c
Fix review comments: release notes and formatting
2019-07-18 10:26:39 -07:00
4b36099097
Update fdbclient/ReadYourWrites.actor.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-18 10:10:36 -07:00
e8e48e0dbd
Fix size calculation
...
Mutations in writeRangeToNativeTransaction() is already counted, so there is no
need to count them again.
2019-07-16 15:21:13 -07:00
d5cc2beb5f
Update fdbclient/ReadYourWrites.actor.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-16 10:33:25 -07:00
562bf6511a
Fix approximate size calculation
2019-07-12 16:53:37 -07:00
f4366e69ca
Unknown options should not be used internally (i.e. underneath thread-safe API). This commit removes various checks that options exist and replaces them with an ASSERT.
2019-07-11 11:25:39 -07:00
bbef631872
fix: do not access optionInfo unless the option already exists in the map
2019-07-10 18:48:54 -07:00
c70a426f04
Update approximate size calculation
2019-07-10 15:00:50 -07:00
5d1437c8e0
Push int directly to stack for getApproximateSize
2019-07-10 15:00:50 -07:00
8ef8b59fcc
Use ThreadFuture for getApproximateSize
...
Change return type to int64_t and fix C and Python binding to use the correct
type.
2019-07-10 14:58:07 -07:00
7e5b5a0536
Apply suggestions from code review
...
Use emplace_back instead of push_back
Co-Authored-By: Jingyu Zhou <jingyuzhou@gmail.com>
2019-07-02 11:09:46 -07:00
aa1bc0087e
Address some review comments
2019-06-28 14:17:25 -07:00
2035b36257
Make default and persistent options specifyable via annotations to fdb.options. Fix some issues with persisting these options in the multi-version client. Make size limit option not persistent.
2019-06-28 13:24:32 -07:00
603721e125
Merge branch 'master' into thread-safe-random-number-generation
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/genericactors.actor.cpp
# fdbrpc/sim2.actor.cpp
# fdbserver/DiskQueue.actor.cpp
# fdbserver/workloads/BulkSetup.actor.h
# flow/ActorCollection.actor.cpp
# flow/Net2.actor.cpp
# flow/Trace.cpp
# flow/flow.cpp
2019-05-23 08:35:47 -07:00
f4fbaac6b0
Merge branch 'release-6.1'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# versions.target
2019-05-19 10:27:59 -07:00
f3897238f8
added the ability to add a read conflict range on the metadata version key without the READ_SYSTEM_KEYS option
2019-05-15 10:13:38 -07:00
5f55f3f613
Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used.
2019-05-10 14:01:52 -07:00
d01cbf3455
Addressed code review comments
2019-04-05 13:12:20 -07:00
1c16f87a4e
Remove trace-calls to printable (in non-workloads)
2019-04-05 13:12:19 -07:00
64e45e6826
retry limit and max delay transaction options are no longer reset after onError
2019-03-21 18:50:02 -04:00
22f592ce6e
reset the timeout only if the API version is less than 610 to allow transactions with longer timeouts than the database default
2019-03-21 16:47:12 -04:00
7c8a1c8db7
Revert "start the timeout actor only after the first read to allow transaction timeouts longer than the default db timeout"
...
This reverts commit df8826115d
2019-03-21 14:45:43 -04:00
df8826115d
start the timeout actor only after the first read to allow transaction timeouts longer than the default db timeout
2019-03-21 10:24:02 -04:00
148a21a31c
set the timeout actor upon transaction creation if necessary
...
initialize lockaware and maxbackoff in options constructor rather than NativeAPI transaction constructor
2019-03-20 11:13:38 -06:00
5f73b7d6ae
Resolves #775 : Support setting Transaction options at the Database level
...
This selects a few useful transaction options that users may want to set for every transaction and adds them to the database options.
2019-03-20 11:13:38 -06:00
bf170493c6
RYOW onError no longer resets a transaction when it gets a non-retryable error and the transaction is not already in an error state (as of API version 610).
2019-03-14 11:40:51 -07:00
32cf742c1d
api versioning is unnecessary for the metadataVersionKey
2019-03-12 11:08:40 -07:00
98bf4ddf28
watching the metadataVersionKey does not require ACCESS_SYSTEM_KEYS, this is not api versioned, as it is harmless to allow this for older api versions
2019-03-10 21:19:35 -07:00
2627bcd35e
Merge branch 'master' into feature-metadata-version
2019-03-10 21:13:28 -07:00
c6a65389ae
Remove noexcept macro and replace with BOOST_NOEXCEPT.
...
BOOST_NOEXCEPT does what the noexcept macro was supposed to do, but in a
way that is correctly maintained over time.
2019-03-05 22:06:12 -08:00
c1de93f467
fix: binary search could get stuck in an infinite loop
...
fix: avoid picking a read version which could be before the last real commit
fix: we must wait on metadataVersionKey in case it is not already cached
fixed review comments
2019-03-02 13:55:41 -08:00
3da85f3acd
implemented the \xff/metadataVersion key, which can be used by layers to help them cheaply cache metadata and know when their cache is invalid
2019-02-28 17:45:00 -08:00
c532d5c277
Implements CompareAndClear AtomicOp
...
Adds CompareAndClear mutation. If the given parameter is equal to the
current value of the key, the key is cleared. At client, the mutation
is added to the operation stack. Hence if the mutation evaluates to
clear, we only get to know so when `read()` evaluates the stack in
`RYWIterator::kv()`, which is unlike what we currently do for typical
ClearRange.
2019-02-04 14:59:56 -08:00
04b94e74c7
Merge pull request #942 from ajbeamon/remove-cluster-from-bindings
...
Remove cluster from bindings
2019-01-24 14:39:09 -08:00
d4d5740282
* Add Optional.map and ErrorOr.map.
...
* Rename Optional/ErrorOr cast_to to castTo.
* Make printable(Optional<T>) templated rather than restricted to StringRef types.
* Fixes bug in (unused) ErrorOr.castTo where an ErrorOr that was not set would lose its error.
2019-01-11 09:03:38 -08:00
890a0f3e82
Merge branch 'master' into remove-cluster-from-bindings
...
# Conflicts:
# bindings/flow/fdb_flow.actor.cpp
2018-11-26 12:01:03 -08:00
a654183f63
Merge pull request #791 from ajbeamon/remove-cluster-from-iclientapi
...
Remove cluster from IClientApi (phase 2 of removing DB names)
2018-11-10 10:16:18 -08:00
268093a96d
Adjust all includes to be relative to the root.
...
Remove the use of relative paths. A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h". Adjust so that every include references such a header with the
latter form.
Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
d5357d2439
Update all of the bindings to use the new C API and deprecate or remove unneeded parts of their APIs.
2018-10-02 15:28:46 -07:00
c831051474
This removes the idea of clusters from IClientApi.
2018-09-21 15:58:14 -07:00
fb31a6999f
Rewrite all files to have #include actorcompiler.h as the last include.
2018-08-14 15:50:26 -07:00
535b5701e5
Rewrite all `Void _ = wait(...)` -> `wait(...)`.
...
This takes advantage of the new actorcompiler functionality to avoid
having duplicate definitions of `Void _` when trying to feed the
un-actorompiled source through clang.
2018-08-14 15:50:26 -07:00
3535ddad80
Merge pull request #674 from alexmiller-apple/glibcxx-debug-fixes
...
Fix bugs uncovered by -D_GLIBCXX_DEBUG
2018-08-09 08:18:51 -07:00
1a7cda4149
Stop performing self-moves. (e.g. a = std::move(a))
...
self-moves are frowned upon in C++, and in our code this generally happens from
calls to swap as part of trying to implement a "unordered erase" function via
swap-to-the-end-and-pop_back. For convenience, a swapAndPop() function is now
offered that performs this, while disallowing self-moves.
2018-08-01 18:09:54 -07:00
1c29275672
call all methods which could disable a trace event before it is initialized. In practice this means calling .error first, then .suppressFor, then all your details.
2018-08-01 14:30:57 -07:00
e5488419cc
Attempt to normalize trace events:
...
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.
Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.
This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
026458baf3
Merge release-5.2 into master
2018-05-23 15:32:56 -07:00
464e2cdbf0
change SetVersionstampedKey and SetVersionstampedValue behavior based on API version to make them consistent
2018-05-08 08:57:09 -07:00
14cca75429
server components of version of alternative versionstamp op that writes to an arbitrary place in the value
2018-05-08 08:57:08 -07:00
218b7a41e2
add APPEND_IF_FITS to workload and remove guard ; add command to vexillographer
2018-03-02 17:43:39 -08:00
0bae9880f1
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-21 10:25:11 -08:00
080a454051
fix: getVersionstamp would return broken promise if a transaction was disposed before being set. getAddressesForKey would not return when resetPromise was set.
2018-01-31 13:47:36 -08:00
2d5a3a07e4
Avoid copies and comparisons in RYW get range
2017-11-02 10:51:30 -07:00
0d68db0dac
Merge branch 'master' into getrange-perf-improvements
2017-10-26 09:25:04 -07:00
9dd588dcce
Addressed review comments.
...
Changed naming for NewMin and NewAnd to MinV2 and AndV2
2017-10-25 14:48:05 -07:00
8c3bdc5b3b
Make atomic ops differentiate between unset and empty values.
2017-10-23 16:48:13 -07:00
0167c1e7e8
Remove unintentionally committed line
2017-10-20 09:47:10 -07:00
39a43aeb95
Eliminate another copy
2017-10-20 09:46:35 -07:00
312f614133
Add the new ops and AND to NON_ASSOCIATIVE_MASK.
...
In the storage server, read the entire value if the op is ByteMin or ByteMax.
2017-10-16 11:06:31 -07:00
eeebf10030
Modified existing behavior of MIN and AND atomic ops. The new behavior results in a 'SET' if the atomic op is performed on a non -existing key.
...
Added new atomic ops ByteMin and ByteMax that does lexicographic comparison of byte strings.
2017-10-10 13:02:22 -07:00
e5e7f8a081
When using setKey() on Standalone<KeySelectorRef> in RYW, make sure that the key is part of the key selector's arena.
2017-09-25 15:52:45 -07:00
a674cb4ef4
Initial repository commit
2017-05-25 13:48:44 -07:00
Xin Dong