Jingyu Zhou
3a0514851a
Fix a signed and unsigned mismatch
2019-07-31 17:01:23 -07:00
Jingyu Zhou
7592e129d2
Use friend struct for _IncludeVersion
2019-07-31 16:36:08 -07:00
Jingyu Zhou
7775ab7892
Use std::invoke_result_t for type names
2019-07-31 15:45:35 -07:00
Jingyu Zhou
c5cc8c5180
Change deprecated std::result_of to std::invoke_result
...
std::result_of is deprecated in C++17.
2019-07-31 14:18:41 -07:00
Balachandar Namasivayam
e8a9931dbe
Merge pull request #1918 from atn34/at-what-cost
...
Avoid memcpy for small types
2019-07-31 11:39:38 -07:00
Evan Tschannen
fcc0f7a681
Merge pull request #1934 from etschannen/feature-bool-knobs
...
Added a knob to always use CAUSAL_READ_RISKY for GRV
2019-07-30 18:38:20 -07:00
mpilman
058f21b0fc
fixed merge conflicts
2019-07-30 18:36:10 -07:00
mpilman
aaef83e122
Added first parser functionality for sequences
2019-07-30 18:36:10 -07:00
mpilman
6b8b666ef6
Write object address in probes
2019-07-30 18:36:10 -07:00
mpilman
75e78f106e
Added more run_loop probes and added documentation
2019-07-30 18:36:10 -07:00
mpilman
884628bc0a
moved actor probes to function
2019-07-30 18:36:10 -07:00
mpilman
b7df7f3549
compatability with older cmake versions
2019-07-30 18:36:10 -07:00
mpilman
32d141ad3a
Probes with strings
2019-07-30 18:36:10 -07:00
mpilman
9b96d8c166
a step further to uniqueness
2019-07-30 18:36:10 -07:00
mpilman
8c73fa556c
Reduced number of macros to 1
2019-07-30 18:36:10 -07:00
mpilman
497c0aa456
fixed typo
2019-07-30 18:36:10 -07:00
mpilman
8eb06f7ab4
Only generate probes where it is supported
2019-07-30 18:36:10 -07:00
mpilman
13e101c441
Added d-trace probes for actors
2019-07-30 18:36:10 -07:00
mpilman
b81a4ef6a2
clang+libc++ support on linux + dtrace probes
2019-07-30 18:36:10 -07:00
Evan Tschannen
2d136af2bd
bool knobs can now be set with the words “true” or “false” instead of just a number
2019-07-30 18:21:24 -07:00
Evan Tschannen
ea1858953b
Merge branch 'master' into merge-release-6.1-into-master
2019-07-30 17:40:15 -07:00
Andrew Noyes
d8a11a704e
Add gotcha about arena ordering
2019-07-30 17:33:21 -07:00
Andrew Noyes
43e86ff819
Address review comments
2019-07-30 17:33:21 -07:00
Andrew Noyes
8145b45ad5
Update documentation
2019-07-30 17:33:21 -07:00
A.J. Beamon
b5d2234a13
Merge branch 'release-6.1' into merge-release-6.1-into-master
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbserver/MoveKeys.actor.cpp
# flow/FastAlloc.h
# versions.target
2019-07-30 16:23:42 -07:00
Andrew Noyes
d0ecdb4fb1
Replace std::map with sorted std::vector
2019-07-30 16:17:14 -07:00
Evan Tschannen
06fc8cb904
Merge pull request #1919 from etschannen/feature-buffered-popped
...
Implement popped on bufferedCursor
2019-07-30 15:56:41 -07:00
A.J. Beamon
14648e20f9
Merge pull request #1901 from ajbeamon/data-distribution-receives-bytes-input-rate
...
Send bytes input rate to data distribution
2019-07-30 15:01:36 -07:00
Evan Tschannen
7a932479dd
throw away state if we ever read popped data from the disk queue adapter
2019-07-30 10:14:39 -07:00
Andrew Noyes
9f32edf4df
Avoid memcpy for small types
...
This is undefined behavior, since it's potentially a misaligned access.
But it's _probably_ not worse than the status quo
2019-07-29 17:11:45 -07:00
Andrew Noyes
997da6882a
Explain test subtlety
2019-07-29 16:00:14 -07:00
Andrew Noyes
9d32cbcf50
Deserialize Arena after VectorRef in flat_buffers unit test
2019-07-29 15:45:54 -07:00
Evan Tschannen
1a4ca05a04
Merge pull request #1889 from ajbeamon/add-cache-memory-parameter
...
Add cache_memory parameter to fdbserver
2019-07-26 13:34:24 -07:00
Evan Tschannen
98c3b24036
Merge pull request #1869 from alexmiller-apple/sharded-txs-performance
...
Raise the priority of TLogRejoin above TLogPeekReply
2019-07-26 13:30:13 -07:00
Andrew Noyes
fc7f9e7c69
Fix Void flatbuffers serialization
...
Before this fix, the encoding of Void happens to 'work' in that this
implementation can both read and write it, but it is not a flatbuffers
message since it contains a invalid offset (the implementation 'works'
because it does not dereference that offset).
2019-07-25 17:30:43 -07:00
A.J. Beamon
b91795d288
Send bytes input rate to DD.
2019-07-25 16:27:32 -07:00
A.J. Beamon
a92b6cd3d1
Merge branch 'master' into add-cache-memory-parameter
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
2019-07-25 13:41:57 -07:00
sramamoorthy
a65c9f92ed
get rid of all timeouts and other changes
2019-07-24 15:36:28 -07:00
sramamoorthy
a2f2ad96ff
code review comments and merge to master changes
2019-07-24 15:36:28 -07:00
sramamoorthy
806a0822f1
timeoutError that takes Future<Error<T>> as input
2019-07-24 15:36:28 -07:00
A.J. Beamon
94be9560ea
Add cache_memory parameter to fdbserver to control the size of the (4K) page cache. Change the default slighty from 2000 MiB to 2GiB.
2019-07-23 15:05:21 -07:00
A.J. Beamon
6e078a41a7
Merge pull request #1828 from mpilman/features/lexicographical-ordered-traces
...
Make trace files lexicographically ordered
2019-07-23 08:31:59 -07:00
A.J. Beamon
e98cee016d
Fix unsafe usage of now() function from multiple threads in trace logging.
2019-07-22 22:31:38 -07:00
Andrew Noyes
5cef65b6c4
Update flow/FileTraceLogWriter.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-22 16:38:18 -07:00
mpilman
de4fe4d640
Remove dead code and revert back to base 10
...
there can be at most 2^32 files with a distinct index. Therefore
`log10(index) < 10` - this means base 10 is good enough.
2019-07-22 16:20:03 -07:00
mpilman
aa93f8411a
Merge branch 'features/lexicographical-ordered-traces' of github.com:mpilman/foundationdb into features/lexicographical-ordered-traces
2019-07-22 14:22:51 -07:00
mpilman
001bfaf80c
made trace index unsigned
2019-07-22 14:20:09 -07:00
Markus Pilman
a48bdc0095
Update flow/FileTraceLogWriter.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-22 14:16:34 -07:00
A.J. Beamon
e29a6ea280
Merge pull request #1871 from bnamasivayam/tr-priority-add-client-log
...
Track the priority of sampled Transaction as part of GetReadVersion e…
2019-07-22 13:04:52 -07:00
mpilman
c26c68d6d2
Address review comments
2019-07-22 11:45:05 -07:00
mpilman
a66dc937fc
Make trace files lexicographically ordered
...
This resolves #1825
The basic idea is that we prefix every sequence number with its with,
separated by a dot. That way the filename length isn't increased
drastically (unlike using a fixed whidth number with 0-padding)
and the file names will be lexicographically ordered.
2019-07-22 10:10:47 -07:00
Balachandar Namasivayam
af267ba053
Track the priority of sampled Transaction as part of GetReadVersion event.
2019-07-19 17:31:49 -07:00
Alex Miller
df7f0cffa1
Raise the priority of TLogRejoin above the default work priority.
...
With sharded txs tags, the master now receives data from transaction
logs at an order of magnitude higher rate. This is the intentional
desires result of sharding the txs tag. With a sufficient number of
TLogs, the master will saturate its CPU time handling the peek
responses.
Performance tests revealed some unstable oddities in how long a recovery
would take, which was eventually root caused to a priority inversion
between TLogRejoin requests and TLog peek replies.
Once peek replies saturate the CPU, the master would proceed to ignore
further TLogRejoin messages. TLogRejoin is what marks a TLog as
available to the failure monitor, which is also what decides between a
ServerPeekCursor and a MergePeekCursor for a SetPeekCursor. Ignoring
TLogRejoins meant that the sharded txs locality tags for those servers
would be merge peeked over all TLogs. This is much less efficient than
just peeking one copy of data from the one preferred server.
Depending on the race between TLogPeek replies saturating the CPU and
TLogRejoin requests being submitted, a variable number of tags would be
affected, and thus the performance test would have some variance in its
results.
2019-07-19 16:55:04 -07:00
Evan Tschannen
3045826e3c
Merge pull request #1819 from mpilman/flatbuffers-fixes2
...
Flatbuffers fixes2
2019-07-19 16:33:50 -07:00
Alex Miller
c3a8ae4752
Merge pull request #1791 from fzhjon/fetch-keys-requests-priority
...
Introduce priority to fetchKeys requests from data distribution
2019-07-19 14:54:51 -07:00
Jingyu Zhou
8e9aaa767e
Merge pull request #1860 from alexmiller-apple/comma-operator
...
Remove `operator , (vector<T>, T)` as an append operator.
2019-07-19 12:53:32 -07:00
Alex Miller
9f0a9da7c3
Remove `operator , (vector<T>, T)` as an append operator.
...
Because no one expects the comma overload.
2019-07-18 21:16:08 -07:00
mpilman
1ac2d01b03
Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2
2019-07-18 09:50:08 -07:00
A.J. Beamon
d981de18e4
Restrict huge arena sampling to the network thread. Revert removal of thread_local definitions.
2019-07-17 16:23:17 -07:00
Alvin Moore
b9e938972c
Merge pull request #1719 from mpilman/features/static-libc++
...
statically link libstdc++ on Linux and remove std::variant
2019-07-16 21:00:02 -07:00
Evan Tschannen
5d3e69b6fc
Merge pull request #1820 from fzhjon/load-balance-locality
...
Introduced a knob that can turn locality on/off
2019-07-16 16:40:43 -07:00
A.J. Beamon
7d6dcc92d0
I accidentally omitted the important change here.
2019-07-16 15:32:06 -07:00
A.J. Beamon
0bfd474e21
Fix huge arena tracking thread-safety issues. Fixes #1846 .
2019-07-16 15:22:19 -07:00
mpilman
6a4a129cf5
fixed silly boost visitor bug
2019-07-16 15:10:55 -07:00
mpilman
b18666d942
statically link libstdc++ on Linux and remove std::variant
...
this will hopefully fix #1610
2019-07-16 14:53:16 -07:00
mpilman
d5caf0c1b4
Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2
2019-07-16 14:47:40 -07:00
Andrew Noyes
e7e48a40ce
Fix a few IncludeVersion bugs
2019-07-16 13:28:29 -07:00
Jon Fu
7d37040725
split the single knob into two for finer-grained control
2019-07-16 12:46:02 -07:00
Alec Grieser
a89d09f1a6
Merge pull request #1844 from ajbeamon/configurable-trace-field-length
...
Make trace event field lengths default knobified and individually configurable.
2019-07-15 15:53:22 -07:00
mpilman
54416f46fd
Pass type as param to VectorRef instead of bool
2019-07-15 15:08:49 -07:00
A.J. Beamon
562e41d25d
Apply suggestions from code review
...
Co-Authored-By: Alec Grieser <alloc@apple.com>
2019-07-15 15:02:55 -07:00
mpilman
707b29cbcc
Fixed compilation error
2019-07-15 13:33:22 -07:00
Alex Miller
c8e94e601a
Merge pull request #1729 from etschannen/feature-fast-txs-recovery
...
Improve the recovery speed of the txnStateStore
2019-07-15 13:27:41 -07:00
mpilman
6c6a1ca8f4
Expose serialization context too all traits
2019-07-15 12:58:31 -07:00
mpilman
1ab36cafbc
Fixed memory bug
2019-07-15 09:53:26 -07:00
mpilman
cb7b941452
expose protocolVersion to serialize function
...
The main idea here is that we now have a context for
serialization and deserialization. This can be used
to access the protocolVersion.
The context still needs to be exposed to the
deserailization traits
2019-07-12 17:16:54 -07:00
A.J. Beamon
d5051b08dd
Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated.
2019-07-12 16:12:35 -07:00
mpilman
75d4b612cf
Make object serializer versioned
2019-07-12 11:53:14 -07:00
Andrew Noyes
984b603c33
Avoid saving each item twice
2019-07-12 11:20:01 -07:00
Andrew Noyes
11c969511c
Update TODOs
2019-07-12 11:17:11 -07:00
mpilman
74f992b2cc
fixed wrong assertion
2019-07-11 23:03:31 -07:00
mpilman
b68f2d925f
Serialize range result to string for speed
2019-07-11 23:03:31 -07:00
Andrew Noyes
c63ff98535
Add some documentation todos
2019-07-11 23:03:31 -07:00
Andrew Noyes
eb87b1fc61
Remove dead code
2019-07-11 23:03:31 -07:00
Andrew Noyes
137936762d
Eliminate unnecessary memcpy
2019-07-11 23:03:31 -07:00
Andrew Noyes
70f0726185
Support PacketBuffer's of arbitrary size
2019-07-11 23:03:31 -07:00
Andrew Noyes
70eac949e6
Zero-initialize memory more precisely
...
Also fix two alignment bugs
2019-07-11 23:03:31 -07:00
Andrew Noyes
a34f8d70d6
Re-use writeToOffsets vector
2019-07-11 23:03:31 -07:00
Andrew Noyes
f8584d0df8
Implement new dynamic_size_traits
2019-07-11 23:03:31 -07:00
Andrew Noyes
71119975f5
Documentation WIP
2019-07-11 23:03:31 -07:00
Jon Fu
f707d186fe
added new priority for fetchkeys requests and adjusted ddmetrics workload to run parallel with mako
2019-07-11 09:56:58 -07:00
A.J. Beamon
b4dbc6d7fa
Change the way cache hits and misses are tracked to avoid counting blind page writes as misses and count the results of partial page writes. Report cache hit rate in status.
2019-07-10 14:43:20 -07:00
Evan Tschannen
d8948c8be1
Merge branch 'master' into feature-fast-txs-recovery
...
# Conflicts:
# fdbserver/TagPartitionedLogSystem.actor.cpp
2019-07-10 13:59:52 -07:00
A.J. Beamon
38ae352fc5
Fix a merge issue
2019-07-10 09:46:23 -07:00
A.J. Beamon
69d7c4f79c
Merge branch 'master' into track-run-loop-busyness
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# flow/Net2.actor.cpp
# flow/network.h
2019-07-09 18:39:23 -07:00
Meng Xu
cce00bb413
Merge pull request #1808 from ajbeamon/improved-transaction-metrics
...
Improve TransactionMetrics
2019-07-09 16:46:17 -07:00
Jon Fu
19a91765f6
introduced a knob that can turn locality on/off
2019-07-09 16:16:15 -07:00
Vishesh Yadav
983343978e
fdbrpc: ConnectionMonitor should close unreferenced after delay
...
Potentially for cases, where it goes up to 1 immediately.
2019-07-09 14:24:16 -07:00
Vishesh Yadav
1f9c80f633
fdbrpc: Instead of tracking last sent data, track last sent non-ping data
...
* This will allow client to continue monitoring peer connections while
connection stays open, so that there is no period of "uncertainity"
without previous no-monitoring approach.
* Use multiplier for incoming connection idle timeout
* Update idle connection timeout values and leaked connection timeout in
simulator.
2019-07-09 14:24:16 -07:00
Vishesh Yadav
867986cdea
fdbrpc: Reduced connection monitoring from clients
...
This patch does two changes to connection monitoring:
1. Connection monitoring at client side will check if the connection
has been stayed idle for some time. If connection is unused for a
while, we close the connection. There is some weirdness involved here
as ping messages are by themselves are connection traffic. We get over
this by making it two-phase process, first being checking idle
reliable traffic, followed by disabling pings and then checking for
idle unreliable traffic.
2. Connection monitoring of clients from server will no longer send
pings to clients. Instead, it keep monitor the received bytes and
close after certain period of inactivity.
2019-07-09 14:24:16 -07:00
Vishesh Yadav
705059dea1
Trace: Add support to print pointers
2019-07-09 14:24:16 -07:00
A.J. Beamon
a5a6f8431c
Add a random UID to TransactionMetrics in case a client opens multiple connections and also a field to indicate whether the connection is internal. Convert some of the metrics to our Counter object instead of running totals.
2019-07-08 14:01:04 -07:00
Evan Tschannen
15e894c724
Merge in master
2019-07-05 15:49:24 -07:00
Andrew Noyes
6d74af93d3
Use true instead of 1
2019-07-05 14:07:02 -07:00
Andrew Noyes
15c6f2b864
Explain SFINAE for has_serialization_done
2019-07-05 14:07:02 -07:00
Andrew Noyes
e2ed56fa56
Convert ownedPtr to unownedPtr for IReplicationPolicy
...
Remove WriteRawMemory feature
Remove deserialization_done
2019-07-05 14:07:02 -07:00
Andrew Noyes
9894d928a1
Re-use identical vtables
2019-07-05 14:07:02 -07:00
Andrew Noyes
4c5ebd7609
Avoid assert when collecting vtables
2019-07-05 14:07:02 -07:00
Alex Miller
888f4f92e0
Fix errors and TaskPriority more priorities.
2019-07-03 21:03:58 -07:00
Alex Miller
ea6898144d
Merge remote-tracking branch 'upstream/master' into flowlock-api
2019-07-03 20:44:15 -07:00
Jingyu Zhou
5ea2e69016
Remove a fdbprc header from flow library
...
Flow should be an independent library.
2019-07-03 19:56:38 -07:00
Evan Tschannen
3fb0999e10
revert storage server priority changes
2019-07-02 16:54:47 -07:00
Alex Miller
8e1ab6e7db
Merge remote-tracking branch 'upstream/master' into flowlock-api
2019-06-28 17:32:54 -07:00
Evan Tschannen
4cef1d3937
Experimental change of storage write priority
2019-06-28 16:54:22 -07:00
Evan Tschannen
1b939d5208
Merge pull request #1749 from satherton/feature-redwood
...
Update redwood storage engine to latest correctness-passing version
2019-06-28 16:22:06 -07:00
Evan Tschannen
5ccffe3cb3
Merge pull request #1684 from jzhou77/large-packet
...
Better handling for large packets
2019-06-28 16:19:01 -07:00
Evan Tschannen
7f4586ad49
the number of txsTags needs to be tracked separately from the number of transaction logs because of forced recoveries
2019-06-28 12:33:24 -07:00
Evan Tschannen
52efcfd136
fix: properly create the right number for txsTags when changing between different numbers of logs
2019-06-27 15:15:05 -07:00
mpilman
7efdf492ce
Fixed two silly compilation bugs
2019-06-27 10:29:59 -07:00
Alex Miller
a2a5bcc73d
Merge remote-tracking branch 'upstream/master' into flowlock-api
2019-06-26 20:31:09 -07:00
A.J. Beamon
7f23814841
Track run loop busyness and report it in status.
2019-06-26 14:03:02 -07:00
Alec Grieser
e5d02b7ad4
use _WIN32 to determine platform instead of __WIN32__
2019-06-26 11:55:26 -07:00
Alex Miller
bf883d7055
Merge remote-tracking branch 'upstream/master' into flowlock-api
2019-06-25 14:26:50 -07:00
Evan Tschannen
0fe6edc254
Merge pull request #1678 from mpilman/features/external-workload
...
Features/external workload
2019-06-25 13:53:19 -07:00
Jingyu Zhou
e6ff23c420
Allow next packet size to be read when receiving data
...
For large packet, allocate sizeof(uint32_t) more bytes for next packet size.
Also add knob MIN_PACKET_BUFFER_FREE_BYTES, which is used to trigger allocation
of a new arena when free bytes are lower than this threshold.
2019-06-25 10:18:56 -07:00
Jingyu Zhou
c2c82c89a5
Remove knob DEFAULT_PACKET_BUFFER_BYTES
2019-06-25 10:18:56 -07:00
Jingyu Zhou
2363326ecb
Add knobs for various packet buffer sizes
2019-06-25 10:18:56 -07:00
Alex Miller
7a500cd37f
A giant translation of TaskFooPriority -> TaskPriority::Foo
...
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
Stephen Atherton
f1f1081202
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-06-24 20:17:49 -07:00
Evan Tschannen
54c2b8d5bf
pseudoLocalities was introduced in the current protocol version, which is 6107
2019-06-24 17:04:52 -07:00
mpilman
a812b93fbf
Fixed version for PseudoLocalities
2019-06-23 09:46:18 -07:00
Evan Tschannen
37c1df2491
Merge pull request #1705 from bnamasivayam/suspend-process
...
Extend RebootRequest API to include time to suspend the process befor…
2019-06-20 17:36:25 -07:00
Alex Miller
df0baa0066
Merge pull request #1720 from mpilman/features/protocol-version
...
Make protocol version a type
2019-06-19 13:46:35 -07:00
mpilman
2eff2b7e21
First simple test is working (but very buggy)
2019-06-19 13:03:41 -07:00
mpilman
c8957d93f8
Implementation code complete
2019-06-19 13:03:41 -07:00
mpilman
68ce9a5e75
ProtocolVersion type - second try
2019-06-18 17:55:27 -07:00
Alex Miller
0e5773106c
Platform.cpp is a mess, and I was far too optimistic with that push.
2019-06-18 16:50:08 -07:00
Alex Miller
ed118bac17
Fix copy-paste error.
2019-06-18 16:40:17 -07:00
Alex Miller
4fa5dc0502
Merge remote-tracking branch 'upstream/master' into cloexec
2019-06-18 16:35:18 -07:00
Alex Miller
5da8746803
Move fnctl calls into Platform function, and add traceevent on error.
2019-06-18 16:34:35 -07:00
mpilman
8576665a90
Revert "Revert "Make protocol version a type""
...
This reverts commit 455bf3b3ec
.
2019-06-18 14:49:04 -07:00
Alex Miller
455bf3b3ec
Revert "Make protocol version a type"
2019-06-18 10:59:17 -07:00
A.J. Beamon
c3aa5819f2
Merge pull request #1417 from mpilman/features/client-buggify
...
Overall framework and first buggify entries
2019-06-18 09:10:11 -07:00
mpilman
4a14117549
Fixed typo
2019-06-17 09:30:28 -07:00
mpilman
52da31bfb3
Simplifications and addressed reviews
2019-06-16 11:07:10 -07:00
Markus Pilman
882fa39e80
fixed typo
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-06-16 09:59:15 -07:00
Markus Pilman
23fb403216
Fix typo
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-06-16 09:59:15 -07:00
mpilman
dc9522bd86
Address code-review comments
2019-06-16 09:59:15 -07:00
mpilman
9fd8dac767
Added license header to new file
2019-06-16 09:59:15 -07:00
mpilman
da53a92bec
Make protocol version a type
...
This fixes #1214
The basic idea is that ProtocolVersion is now its own type. This
alone is an improvement as it makes many things more typesafe. For
each version, we can now add breaking features (for example Fearless).
After that, there's no need to test against actual (confusing) version
numbers. Instead a developer can simply test
`protocolVersion->hasFearless()` and this will return true iff the
protocolVersion is newer than the newest version that didn't support
fearless.
2019-06-16 09:59:15 -07:00
mpilman
7483cba26e
Address review comments and added documentation
2019-06-16 09:52:40 -07:00
mpilman
6ea75713cb
Overall framework and first buggify entries
2019-06-16 09:09:09 -07:00
Balachandar Namasivayam
5eb833759e
Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures.
2019-06-14 11:35:38 -07:00
Evan Tschannen
6ececa94ce
Merge pull request #1640 from vishesh/task/client-failmon
...
Clients will no longer get failure monitoring info from cluster controller
2019-06-13 17:31:17 -07:00
Trevor Clinkenbeard
1e8f7e5b82
Refactor NextFastAllocatedSize to be constexpr function
2019-06-11 15:55:23 -07:00
mpilman
4c62458172
Make valgrind work on Fedora 30
2019-06-11 10:27:08 -07:00
Vishesh Yadav
a8e408e268
run clang-format on changes
2019-06-10 14:10:24 -07:00
Alex Miller
0b07dbd05c
Remove a horrible hack aiming at Ubuntu 14.04's libstdc++_pic.
...
Because our supported build environments no longer include Ubuntu 14.04,
and this is causing problems with libc++-based builds.
2019-06-10 13:16:03 -07:00
Vishesh Yadav
6fa7081a21
net: Don't make FailureMonitoring requests from client
...
This patch removes the need for clients to continuously contact
cluster coordinator for failure monitoring information. Instead, it
uses the FlowTransport to monitor the statuses of peers and update
FailureMonitor accordingly.
2019-06-09 00:43:38 -07:00
Vishesh Yadav
6b4d30c3ae
failmon: Identify client vs server when starting failure monitoring client
2019-06-09 00:43:12 -07:00
Alex Miller
526468becb
Fix printable() being incorrect for bytes >127.
...
The use of `auto` hid that TraceableString<T>::begin(value) was a signed
char, and therefore `*iter` was -1, and -1/16 is 0.
Thus, a 0xFF byte was printed as \x0f. Forcing it to an unsigned type
makes it correctly become \xff.
2019-06-05 14:41:31 -07:00
A.J. Beamon
c027792d0f
Make the default seed of the generator returned by deterministicRandom() random. This means that the random number sequence won't be deterministic unless you set a seed.
2019-06-05 11:48:29 -07:00
Parallels
773f52d0a1
Merge remote-tracking branch 'upstream/master' into cloexec
2019-06-03 15:43:32 -07:00
sramamoorthy
a8c9432ced
windows build fix : del sys/wait.h in Platform.cpp
2019-05-31 14:59:53 -07:00
Evan Tschannen
362c2bf1e6
improved the cpu efficiency of printable
2019-05-29 14:55:45 -07:00
Stephen Atherton
02882dbf00
Checkpointing progress, RedwoodRecordRef and DeltaTree tests pass but BTree test does not. RedwoodRecordRef::Delta rewritten to actually do prefix compression on key and integer fields. Added related unit tests and benchmarks. Some improvements to DeltaTree and requirements on its T and Delta types to avoid repeated common prefix discovery.
2019-05-29 06:23:32 -07:00
sramamoorthy
b17ad85497
exec op not supported when log_anti_quorum > 0
2019-05-28 22:07:46 -07:00
sramamoorthy
bb474dc323
if recovery < fully_recovered then fail the exec
...
Will do more cleanup, pushing it for a test run in CI
2019-05-28 22:07:46 -07:00
sramamoorthy
925499954b
New status cluster_not_fully_recovered
2019-05-28 22:07:46 -07:00
sramamoorthy
d68a229772
makefile changes to accommodate boost/process.hpp
2019-05-28 22:07:46 -07:00
sramamoorthy
b6e037ffbc
Replace fork with boost::process::child
2019-05-28 22:07:46 -07:00
sramamoorthy
898bed66c1
Allow only whitelisted binary path for exec op
2019-05-28 22:07:46 -07:00
sramamoorthy
aa79480d69
changes to make fdbfork asynchronous
2019-05-28 22:07:46 -07:00
sramamoorthy
4bc4c615da
exec op to all tlog, restore change in test &other
...
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
sramamoorthy
72dd067173
Trace message changes and fix few FIXMEs
2019-05-28 22:07:46 -07:00
sramamoorthy
69edefe68b
Snapshot based backup and resotre implementation
2019-05-28 22:07:46 -07:00
A.J. Beamon
ec80ff1e8d
Remove unneeded redefinition of random functions.
2019-05-24 11:55:25 -07:00
A.J. Beamon
fc204c846a
Make IRandom reference counted. Simplify the logic around deterministicRandom() to require less state and branching.
2019-05-23 18:51:59 -07:00
A.J. Beamon
764ca2971c
Merge branch 'master' into thread-safe-random-number-generation
2019-05-23 18:49:49 -07:00
A.J. Beamon
296adada93
Use c++11 thread_local instead of redefining it to platform specific variants.
2019-05-23 18:35:45 -07:00
A.J. Beamon
f417e60264
Merge branch 'merge-release-6.1-into-master' into thread-safe-random-number-generation
...
# Conflicts:
# fdbserver/QuietDatabase.actor.cpp
2019-05-23 09:52:00 -07:00
A.J. Beamon
d29c7e4c9b
Merge branch 'release-6.1' into merge-release-6.1-into-master
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbserver/QuietDatabase.actor.cpp
# versions.target
2019-05-23 09:28:45 -07:00
A.J. Beamon
e5381e0612
Fix some new usages of g_random
2019-05-23 09:23:27 -07:00
A.J. Beamon
603721e125
Merge branch 'master' into thread-safe-random-number-generation
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/genericactors.actor.cpp
# fdbrpc/sim2.actor.cpp
# fdbserver/DiskQueue.actor.cpp
# fdbserver/workloads/BulkSetup.actor.h
# flow/ActorCollection.actor.cpp
# flow/Net2.actor.cpp
# flow/Trace.cpp
# flow/flow.cpp
2019-05-23 08:35:47 -07:00
Stephen Atherton
ebc96a7e0e
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-05-21 23:49:27 -07:00
A.J. Beamon
d3c1f9afbd
Suppress 'TraceEventFieldNotFound' event. Don't suppress simultaneous FDBLibTLSVerifyFailure events (this requires including flow.h and removing the copied version of Reference/ReferenceCounted in FDBLibTLS).
2019-05-20 15:12:28 -07:00
Jingyu Zhou
d660ffb654
Merge pull request #1605 from mpilman/remove-traceevent-benchmark
...
remove memory-hungry benchmark
2019-05-17 17:04:45 -07:00
A.J. Beamon
a8b9d8e34b
Merge pull request #1336 from tclinken/fast-allocate-ptree-nodes
...
Create 96-byte fast allocator for storage queue PTree nodes
2019-05-17 14:22:46 -07:00
mpilman
a0fa81aa1c
remove memory-hungry benchmark
...
This test was used as a benchmark to prove that the
new `IsTraceable` feature improves performance. However
it will cause smaller machines to OOM and it doesn't
help for correctness testing. Therefore this test should
likely be removed
2019-05-17 14:12:44 -07:00
Steve Atherton
5a8c97480a
Merge pull request #1506 from nikolas-ioannou/feature-pagecache-lru
...
AsyncFileCached: switch from a random to an LRU cache eviction policy
2019-05-17 13:42:21 -07:00
Alvin Moore
f598d0a614
Merge pull request #1593 from AlvinMooreSr/variant-missing
...
Switched Variant Header
2019-05-17 10:42:21 -07:00
Alvin Moore
3acaa7343e
Enabled C++17 for all Windows projects
...
Set Visual Studio version to 2017 (first version to support C++17)
2019-05-16 17:44:13 -07:00
Alvin Moore
94aed513c7
Switched Windows tools within projects to 2017
2019-05-16 15:05:11 -07:00
Alvin Moore
17a2983943
Switch boost variant to C++17 variant
2019-05-16 13:05:49 -07:00
Alvin Moore
6389a12931
Switched variant to use boost variant for source consistency
2019-05-16 10:37:33 -07:00
Alex Miller
69fb852ee0
Add more CLOEXEC-like things.
...
From missed call sites found during/after code review.
2019-05-14 20:30:58 -10:00
Alex Miller
1f02cd30e2
Mark all opened files as close-on-exec.
2019-05-13 16:05:49 -10:00
Alex Miller
19165dc873
Mark all network sockets as close-on-exec.
2019-05-13 16:05:49 -10:00
Evan Tschannen
a9b03ec505
Merge pull request #1090 from mpilman/features/protocol-version-flags
...
Features/protocol version flags
2019-05-13 16:33:06 -07:00
A.J. Beamon
60ce790a2b
Consistent use of #if VALGRIND
2019-05-13 12:44:13 -10:00