f23392ec5a
Don't use O_DIRECT in EIO by default
2019-10-24 11:39:55 -07:00
7ad0e20e48
Added knob to disable O_DIRECT
2019-10-24 11:20:14 -07:00
f41f19b5f6
Introduced knob to set eio parallelism
2019-10-24 11:20:14 -07:00
0e51a248b4
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-redwood
2019-10-23 10:12:54 -07:00
a79757a788
Fix compiler errors on Catalina
...
Fixes #2263
2019-10-21 11:15:37 -07:00
42b7acf7b7
Merge pull request #2202 from etschannen/feature-share-mutations
...
Backup and DR would not share mutations if started on different versions of FDB
2019-10-16 20:28:39 -07:00
35ef9b32de
fix: if establishing a TLS connection took longer than 10ms, we could spend all our CPU establishing new connections instead of pinging to maintain existing connections, leading to an infinite loop
2019-10-16 17:26:01 -07:00
b88b1614c1
bool knobs would improperly report a SevWarnAlways that they were unrecognized
2019-10-01 18:53:04 -07:00
9ec9f41d34
backup and DR would not share mutations if they were started on different versions of FDB
2019-10-01 18:52:07 -07:00
09be680105
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-redwood
2019-09-27 21:51:17 -07:00
2bbaa917f8
Switched out C style cast to more favorable type
2019-09-24 11:22:24 -07:00
6354e63269
Added parenthesis around ntohl value to resolve problem with CLANG compiler
2019-09-24 11:22:24 -07:00
8f755f0f3a
Added parenthesis around macro condition which could cause problems with interpretation since being used in 'or' statement.
...
This is currently a problem for CLANG compiler
2019-09-24 11:22:24 -07:00
cba5866152
added a fix
2019-09-12 16:30:29 -07:00
a5314e2cee
ObjectSerializer fixes for pr/2086
2019-09-11 13:31:07 -07:00
7ecd31da5a
Fix EnsureTable<T> where T has a LoadSaveHelper specialization
2019-09-11 13:11:58 -07:00
dc668d1fc9
attempt to fix the serialization of CachedSerialization
2019-09-11 11:45:07 -07:00
945cff1e5b
the cluster controller caches the serialization of serverDBInfo, to avoid regenerating it many times
2019-09-10 14:27:22 -07:00
a7237c4302
Merge pull request #2045 from atn34/disallow-scalar-network-messages
...
Disallow scalar network messages
2019-08-30 13:38:54 -07:00
84e2c9e1a5
Merge pull request #2041 from senthil-ram/snap_error_reporting
...
improved error msgs for snapshot cmd
2019-08-30 12:58:41 -07:00
c59168fd07
error msg: Snapshot error -> Disk Snapshot error
2019-08-30 08:46:36 -07:00
6aa0ada7b1
Replace scalar root types with proper messages
2019-08-28 14:40:50 -07:00
26a3672751
Disallow scalars from being root types
...
By not specializing FileIdentifierFor for them
2019-08-28 14:40:06 -07:00
fa6e45a852
Separate AsioReactor sleep and react into two different functions. Track slow tasks and time spent in react, track time spent in launch. Don't track react time at priority 0.
2019-08-28 14:35:48 -07:00
5d87443323
improved error msgs for snapshot cmd
2019-08-27 16:43:52 -07:00
41b908752e
increased move keys parallelism to be less of a decrease just in case lowering this could effect normal data distribution
...
raised target durability lag versions to give more time for batch limiting to come into play before this limit is hit
changed max_bad_options to better reflect the name
2019-08-21 14:55:21 -07:00
54282288cb
disabled zone_id load balancing, because it can cause hot read shards
2019-08-19 14:04:21 -07:00
51cedd24c8
load balance will send reads to remote servers if more than one alternative is failed or overloaded
2019-08-19 13:59:49 -07:00
da8163fd5a
allow one requests every second to skip there all_alteratives_failed delay, because if a client has a timeout longer than the delay we will never invalidate the key servers cache
2019-08-09 13:03:40 -07:00
36b7b98d3c
Buggified the idle_timeout so that it happens in simulation
2019-08-09 12:35:55 -07:00
5f7d3498ea
The delay for all_alteratives_failed can scale all the way up to 30.0 at a much slow time ratio
2019-08-09 12:35:19 -07:00
ca78e35f31
fixed minor bug in flow readme
2019-08-07 08:34:20 -07:00
10be4b8127
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-08-07 04:20:14 -07:00
da9c4e97d3
Added new pager interface, IPager2, whose write interface enables forcing the user to handle a page update causing a copy to a new Page ID. Implemented FIFOQueue<T> which uses pages of T stored in a Pager2 instance to implement a FIFO queue. Implemented COWPager, a copy-on-write Pager2 in which all page writes cause a change of Page ID. VersionedBTree, still only operating in single-version mode, now uses Pager2.
2019-08-07 02:36:33 -07:00
370ba8b841
Remove --object-serializer flag from executables
2019-08-06 09:25:40 -07:00
476641a087
Merge pull request #1929 from jzhou77/fix-warning
...
Fix compiler warnings
2019-08-01 11:15:41 -07:00
2e0e5a27bb
Use invoke_result_t for all
2019-08-01 09:56:49 -07:00
7d247af500
Two minor bug fixes from recent optimizations
2019-07-31 19:14:11 -07:00
1bad0fd44e
Make requestTime private
2019-07-31 17:59:35 -07:00
dabe516320
Avoid unnecessary timer calls
2019-07-31 17:59:35 -07:00
3a0514851a
Fix a signed and unsigned mismatch
2019-07-31 17:01:23 -07:00
7592e129d2
Use friend struct for _IncludeVersion
2019-07-31 16:36:08 -07:00
7775ab7892
Use std::invoke_result_t for type names
2019-07-31 15:45:35 -07:00
c5cc8c5180
Change deprecated std::result_of to std::invoke_result
...
std::result_of is deprecated in C++17.
2019-07-31 14:18:41 -07:00
e8a9931dbe
Merge pull request #1918 from atn34/at-what-cost
...
Avoid memcpy for small types
2019-07-31 11:39:38 -07:00
fcc0f7a681
Merge pull request #1934 from etschannen/feature-bool-knobs
...
Added a knob to always use CAUSAL_READ_RISKY for GRV
2019-07-30 18:38:20 -07:00
058f21b0fc
fixed merge conflicts
2019-07-30 18:36:10 -07:00
aaef83e122
Added first parser functionality for sequences
2019-07-30 18:36:10 -07:00
6b8b666ef6
Write object address in probes
2019-07-30 18:36:10 -07:00
75e78f106e
Added more run_loop probes and added documentation
2019-07-30 18:36:10 -07:00
884628bc0a
moved actor probes to function
2019-07-30 18:36:10 -07:00
b7df7f3549
compatability with older cmake versions
2019-07-30 18:36:10 -07:00
32d141ad3a
Probes with strings
2019-07-30 18:36:10 -07:00
9b96d8c166
a step further to uniqueness
2019-07-30 18:36:10 -07:00
8c73fa556c
Reduced number of macros to 1
2019-07-30 18:36:10 -07:00
497c0aa456
fixed typo
2019-07-30 18:36:10 -07:00
8eb06f7ab4
Only generate probes where it is supported
2019-07-30 18:36:10 -07:00
13e101c441
Added d-trace probes for actors
2019-07-30 18:36:10 -07:00
b81a4ef6a2
clang+libc++ support on linux + dtrace probes
2019-07-30 18:36:10 -07:00
2d136af2bd
bool knobs can now be set with the words “true” or “false” instead of just a number
2019-07-30 18:21:24 -07:00
ea1858953b
Merge branch 'master' into merge-release-6.1-into-master
2019-07-30 17:40:15 -07:00
d8a11a704e
Add gotcha about arena ordering
2019-07-30 17:33:21 -07:00
43e86ff819
Address review comments
2019-07-30 17:33:21 -07:00
8145b45ad5
Update documentation
2019-07-30 17:33:21 -07:00
b5d2234a13
Merge branch 'release-6.1' into merge-release-6.1-into-master
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbserver/MoveKeys.actor.cpp
# flow/FastAlloc.h
# versions.target
2019-07-30 16:23:42 -07:00
d0ecdb4fb1
Replace std::map with sorted std::vector
2019-07-30 16:17:14 -07:00
06fc8cb904
Merge pull request #1919 from etschannen/feature-buffered-popped
...
Implement popped on bufferedCursor
2019-07-30 15:56:41 -07:00
14648e20f9
Merge pull request #1901 from ajbeamon/data-distribution-receives-bytes-input-rate
...
Send bytes input rate to data distribution
2019-07-30 15:01:36 -07:00
7a932479dd
throw away state if we ever read popped data from the disk queue adapter
2019-07-30 10:14:39 -07:00
9f32edf4df
Avoid memcpy for small types
...
This is undefined behavior, since it's potentially a misaligned access.
But it's _probably_ not worse than the status quo
2019-07-29 17:11:45 -07:00
997da6882a
Explain test subtlety
2019-07-29 16:00:14 -07:00
9d32cbcf50
Deserialize Arena after VectorRef in flat_buffers unit test
2019-07-29 15:45:54 -07:00
1a4ca05a04
Merge pull request #1889 from ajbeamon/add-cache-memory-parameter
...
Add cache_memory parameter to fdbserver
2019-07-26 13:34:24 -07:00
98c3b24036
Merge pull request #1869 from alexmiller-apple/sharded-txs-performance
...
Raise the priority of TLogRejoin above TLogPeekReply
2019-07-26 13:30:13 -07:00
fc7f9e7c69
Fix Void flatbuffers serialization
...
Before this fix, the encoding of Void happens to 'work' in that this
implementation can both read and write it, but it is not a flatbuffers
message since it contains a invalid offset (the implementation 'works'
because it does not dereference that offset).
2019-07-25 17:30:43 -07:00
b91795d288
Send bytes input rate to DD.
2019-07-25 16:27:32 -07:00
a92b6cd3d1
Merge branch 'master' into add-cache-memory-parameter
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
2019-07-25 13:41:57 -07:00
a65c9f92ed
get rid of all timeouts and other changes
2019-07-24 15:36:28 -07:00
a2f2ad96ff
code review comments and merge to master changes
2019-07-24 15:36:28 -07:00
806a0822f1
timeoutError that takes Future<Error<T>> as input
2019-07-24 15:36:28 -07:00
94be9560ea
Add cache_memory parameter to fdbserver to control the size of the (4K) page cache. Change the default slighty from 2000 MiB to 2GiB.
2019-07-23 15:05:21 -07:00
6e078a41a7
Merge pull request #1828 from mpilman/features/lexicographical-ordered-traces
...
Make trace files lexicographically ordered
2019-07-23 08:31:59 -07:00
e98cee016d
Fix unsafe usage of now() function from multiple threads in trace logging.
2019-07-22 22:31:38 -07:00
5cef65b6c4
Update flow/FileTraceLogWriter.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-22 16:38:18 -07:00
de4fe4d640
Remove dead code and revert back to base 10
...
there can be at most 2^32 files with a distinct index. Therefore
`log10(index) < 10` - this means base 10 is good enough.
2019-07-22 16:20:03 -07:00
aa93f8411a
Merge branch 'features/lexicographical-ordered-traces' of github.com:mpilman/foundationdb into features/lexicographical-ordered-traces
2019-07-22 14:22:51 -07:00
001bfaf80c
made trace index unsigned
2019-07-22 14:20:09 -07:00
a48bdc0095
Update flow/FileTraceLogWriter.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-22 14:16:34 -07:00
e29a6ea280
Merge pull request #1871 from bnamasivayam/tr-priority-add-client-log
...
Track the priority of sampled Transaction as part of GetReadVersion e…
2019-07-22 13:04:52 -07:00
c26c68d6d2
Address review comments
2019-07-22 11:45:05 -07:00
a66dc937fc
Make trace files lexicographically ordered
...
This resolves #1825
The basic idea is that we prefix every sequence number with its with,
separated by a dot. That way the filename length isn't increased
drastically (unlike using a fixed whidth number with 0-padding)
and the file names will be lexicographically ordered.
2019-07-22 10:10:47 -07:00
af267ba053
Track the priority of sampled Transaction as part of GetReadVersion event.
2019-07-19 17:31:49 -07:00
df7f0cffa1
Raise the priority of TLogRejoin above the default work priority.
...
With sharded txs tags, the master now receives data from transaction
logs at an order of magnitude higher rate. This is the intentional
desires result of sharding the txs tag. With a sufficient number of
TLogs, the master will saturate its CPU time handling the peek
responses.
Performance tests revealed some unstable oddities in how long a recovery
would take, which was eventually root caused to a priority inversion
between TLogRejoin requests and TLog peek replies.
Once peek replies saturate the CPU, the master would proceed to ignore
further TLogRejoin messages. TLogRejoin is what marks a TLog as
available to the failure monitor, which is also what decides between a
ServerPeekCursor and a MergePeekCursor for a SetPeekCursor. Ignoring
TLogRejoins meant that the sharded txs locality tags for those servers
would be merge peeked over all TLogs. This is much less efficient than
just peeking one copy of data from the one preferred server.
Depending on the race between TLogPeek replies saturating the CPU and
TLogRejoin requests being submitted, a variable number of tags would be
affected, and thus the performance test would have some variance in its
results.
2019-07-19 16:55:04 -07:00
3045826e3c
Merge pull request #1819 from mpilman/flatbuffers-fixes2
...
Flatbuffers fixes2
2019-07-19 16:33:50 -07:00
c3a8ae4752
Merge pull request #1791 from fzhjon/fetch-keys-requests-priority
...
Introduce priority to fetchKeys requests from data distribution
2019-07-19 14:54:51 -07:00
8e9aaa767e
Merge pull request #1860 from alexmiller-apple/comma-operator
...
Remove `operator , (vector<T>, T)` as an append operator.
2019-07-19 12:53:32 -07:00
9f0a9da7c3
Remove `operator , (vector<T>, T)` as an append operator.
...
Because no one expects the comma overload.
2019-07-18 21:16:08 -07:00
1ac2d01b03
Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2
2019-07-18 09:50:08 -07:00
d981de18e4
Restrict huge arena sampling to the network thread. Revert removal of thread_local definitions.
2019-07-17 16:23:17 -07:00
b9e938972c
Merge pull request #1719 from mpilman/features/static-libc++
...
statically link libstdc++ on Linux and remove std::variant
2019-07-16 21:00:02 -07:00
mpilman