0239d612d1
Verbose printf cleanup
2020-08-23 13:35:00 +05:30
6a8da71e80
Issue 1647 Resolution
2020-08-21 06:12:39 -05:00
6e0df3b9ac
Fix AsyncFileWinASIO build
2020-08-19 21:05:56 -07:00
c8b774a30a
Explicitly mark IAsyncFile functions as overrided
2020-08-19 18:17:32 -07:00
157700e5b6
Make IAsyncFile const-correct
2020-08-19 17:34:56 -07:00
9e2ee1ed4c
fixed lastedZeroBusy; added a knob
2020-08-17 23:16:59 -07:00
7c84e04028
fixed compiler errors
2020-08-12 17:22:37 -07:00
c72068d6b5
clients load balance across proxies based on process busyness instead of number of requests
2020-08-12 17:17:21 -07:00
8953cc5f98
Merge master again
2020-08-12 16:59:07 -07:00
79ce16650d
merge master branch
2020-08-11 19:22:10 -07:00
ba803a5ea3
Fixed formatting issues and removed GRV related code in MasterProxy
2020-08-11 18:54:54 -07:00
3192984309
Remove unused zlib library
2020-08-11 17:15:07 -07:00
a88d2f72e4
Simulation test now working
2020-08-11 15:34:59 -06:00
104bac3cbd
Add trace to debug
2020-08-07 13:02:41 -07:00
6d84bcb568
Merge remote-tracking branch 'origin/master' into features/udp
2020-08-06 14:08:34 -06:00
8976694ba1
UDP implementation (untested)
2020-08-06 14:06:50 -06:00
d6a23a4d6b
Resolve comments to make GRV proxy a separate process class
2020-08-06 00:01:57 -07:00
5c7b4c7acf
Merge branch 'master' of github.com:apple/foundationdb
2020-07-31 15:58:13 -07:00
ea2837dd5e
Remove bogus function that is never used
2020-07-30 09:24:08 -07:00
a49cb41de7
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# cmake/ConfigureCompiler.cmake
# fdbserver/Knobs.cpp
# fdbserver/StorageCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/ThreadHelper.actor.h
# flow/serialize.h
# tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
a2089b354a
RemoveServersSafely:Safety check toKill1 to avoid cluster getting stuck
...
toKill1 and toKill2 are a random subset of all processes. If simply kill all processes in toKill1 or toKill2,
we may kill too many processes to make the cluster unavailable and stuck.
Similar as what toKill2 were modified if it can cause cluster unavailable,
we should do the same thing for toKill1
2020-07-28 21:07:31 -07:00
229ab0d5f1
Fix some conflicts and remote debugging trace events
2020-07-22 23:35:46 -07:00
525f10e30c
Merge master branch
2020-07-22 16:08:49 -07:00
302cf5c45f
Remove debug trace events
2020-07-22 12:20:22 -07:00
ab0d8b0626
Merge pull request #3509 from sfc-gh-anoyes/anoyes/remove-using-relops
...
Remove using namespace std::rel_ops
2020-07-22 11:58:25 -07:00
abaae40453
Modified ComposedIdentifier to prevent file identifier conflicts at
...
compile-time
2020-07-20 18:52:44 -07:00
38cade1146
Merged tag release-6.3.3 into tls-background-handshake.
2020-07-17 23:39:57 -07:00
2703cedac5
Fixed known bugs
2020-07-17 22:24:52 -07:00
d375ee7939
Rolled back the knob and logic for doing multiple write() calls in write loops before yielding, because they are not necessary. The testing that suggested otherwise was in error.
2020-07-16 22:54:14 -07:00
c4f3553826
TLS handshake limit changed to regular flow lock, added new knob for simultaneous handshakes limit and removed old knobs. Added knob to control how much data the FlowTransport connectionWriter() can write using multiple write() calls before yielding to other tasks, and updated p2p network test to use the same logic. Changed knob defaults for send limit on single writes and TLS handshake thread count.
2020-07-16 02:31:17 -07:00
5b06d69d25
Pass watches test
2020-07-15 00:37:41 -07:00
47ae66bd61
Merge branch 'master' into mengxu/tmp-minor-comment-PR
...
Resolve conflict at waitFailureClient
2020-07-13 16:17:50 -07:00
8ae0d9b73d
Resolve merge conflict
2020-07-13 10:21:53 -07:00
ef8c1060a2
Merge branch 'master' into mengxu/tmp-merge-6.3
2020-07-13 10:15:56 -07:00
6f2e12be42
Minor improvement on comments
2020-07-12 18:32:47 -07:00
a3245d5cc2
Address review comments
2020-07-10 19:04:35 -07:00
f470ba8316
Remove using namespace std::rel_ops
...
This causes the following to not compile anymore
\#include <utility>
\#include <vector>
using namespace std::rel_ops;
int main() {
std::vector<int> xs;
return xs.rbegin() != xs.rend();
}
See https://godbolt.org/z/s1977n
2020-07-10 22:58:15 +00:00
b09dddc07e
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/ConfigureCompiler.cmake
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/fdbrpc.vcxproj
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogSystemPeekCursor.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/Status.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/flow.vcxproj
2020-07-10 15:06:34 -07:00
2e5ae8eed5
Change iterator name to small case
2020-07-10 10:56:41 -07:00
1a57d88179
Add const iterator for RangeMap
2020-07-10 10:56:41 -07:00
dd10dbe7c7
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-ha-fixes
2020-07-09 23:09:14 -07:00
f6163d0a79
fix compile errors
2020-07-09 22:53:02 -07:00
717242a0ee
reset WAN network connections every 5 minutes is responses take more than 500ms
2020-07-09 22:50:47 -07:00
693595f4e5
Fix make build, fix GRV schema
2020-07-09 16:50:08 -07:00
04d1217941
Track statistics about server-side request latency on each process, to include min, max, mean, and various percentiles.
2020-07-09 16:39:15 -07:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
2f602b42e9
Revert "Merge pull request #3467 from jzhou77/release-6.3"
...
This reverts commit 32d76fcff15e8eeadd7e
2020-07-08 18:51:54 -07:00
d6dd88c2b9
Move building of KeyRangeMap out of the loop
...
So that it is only done once.
2020-07-08 14:16:10 -07:00
e32458ebd0
Merge pull request #3464 from sfc-gh-tclinkenbeard/fix-valgrind
...
Fix Valgrind build
2020-07-08 11:10:39 -07:00
f5118c42c0
Fix valgrind build
2020-07-08 10:23:32 -07:00
d972e072b7
Remove unused batcher.actor.h
2020-07-07 21:54:56 -07:00
f1cd466a6b
Fix infinite recursion caused by implicit conversion
...
The keyText and keyValue functions taking an optional parameter are
never used, but if they were it would cause and infinite recursion,
because the AttribKey and AttribValue constructors are marked explicit,
so the same keyText or valueText overload keeps getting called recursively.
2020-07-04 12:50:41 -07:00
dd5cd762a8
Removed dead code
2020-07-04 12:50:40 -07:00
fcaaf11678
Merge pull request #3402 from sfc-gh-tclinkenbeard/improve-const-correctness
...
Added more const-correctness improvements
2020-07-02 16:43:06 -07:00
22f7f804b8
Merge branch 'release-6.3' into mengxu/merge-6.3-PR
2020-06-28 11:19:39 -07:00
648c870fec
Merge branch 'release-6.3' of https://github.com/apple/foundationdb into tls-background-handshake
2020-06-26 00:21:57 -07:00
0da4c91ad5
Performance improvements in socket use and buffering which results in TLS network connections using around 3.5x less CPU.
...
PacketBuffers are 16k minimum sizes, using FastAlloc. All calling paths to PacketBuffer creation now pass a size hint if known. P2PNetworkTest improved in many ways, errors go to TraceEvents instead of stdout, error counts are reported in stats, range min/max parameters are combined into "min:max" strings, new options to "stutter" connections by delaying randomly before socket read or write, message size to send is now configured separately for incoming vs outgoing sessions.
2020-06-25 20:44:43 -07:00
10e043da9d
Format sim2 code a bit
2020-06-23 10:07:39 -07:00
d7cb54d2b2
Fixed Windows CMake build
2020-06-20 23:03:18 -07:00
a59925dd73
Added more const-correctness improvements
2020-06-20 22:15:19 -07:00
d268b6c055
Sim2:Remove an useless line
2020-06-18 12:12:15 -07:00
6865923329
Removed deprecated NetworkMessageReceiver::receive overrides
2020-06-13 23:20:21 -07:00
99bf993815
Replace BOOST_NOEXCEPT with noexcept
2020-06-09 22:39:19 -07:00
693d9e8b89
Merge branch 'master' into fdb_cache_wo_allocator
2020-06-09 15:09:58 -07:00
cf13e00a8f
Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator
2020-06-01 17:38:31 -07:00
414206015d
Merge pull request #3273 from apple/release-6.3
...
Merge release-6.3 into master
2020-05-29 16:58:36 -07:00
8329a242d2
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# documentation/sphinx/source/downloads.rst
# documentation/sphinx/source/release-notes.rst
2020-05-29 15:51:56 -07:00
0e557e0832
fixed compiler error
2020-05-29 09:09:09 -07:00
92b10accd3
randomize which tlogs are recruited by the cluster controller
2020-05-29 09:03:42 -07:00
fa08384bc3
Merge pull request #3239 from apple/release-6.3
...
Merge release 6.3 into master
2020-05-27 08:29:28 -07:00
755267a7db
Merge pull request #3229 from ajbeamon/calling-run-network-twice-is-error
...
Eliminate the undefined behavior of calling run_network twice, instead returning an error
2020-05-26 21:24:52 -07:00
1c35ad884f
Merge branch 'master' into mengxu/release-6.3-conflict-PR
...
Has conflict with master;
Next commit will fix the conflicts.
2020-05-25 12:01:49 -07:00
ced65cd30b
finished explicitly versioning everything stored in the database
2020-05-22 17:14:21 -07:00
72ce997d22
explicitly versioned every key in systemData, so we only will update the associated protocolVersion when the serialization actually changes
2020-05-22 16:35:01 -07:00
86f712657e
Eliminate the undefined behavior of calling run_network twice, instead returning an error.
2020-05-22 13:31:06 -07:00
d128252e90
Merge release-6.3 into master
2020-05-22 09:25:32 -07:00
f1dca05cf3
Merge pull request #3194 from etschannen/feature-small-endpoint
...
Optimized failure monitor performance
2020-05-20 14:48:13 -07:00
6a006c4c1f
use the first RequestStream as the base endpoint
2020-05-20 13:52:22 -07:00
459186dad0
revert previous change to bit shift
2020-05-20 12:32:50 -07:00
5f979c0178
another compiler fix, reduced the size of the endpointNotFound map
2020-05-20 12:30:26 -07:00
b999b93fa2
added a hash function for endpoint
2020-05-20 12:21:57 -07:00
0f21e9ccf4
keep a second unordered_set for failed endpoints
2020-05-20 12:15:30 -07:00
dfc152ebb3
shift the bits farther
2020-05-20 11:34:24 -07:00
405350ce56
Merge pull request #3167 from tclinken/fix-requeststream-send
...
Remove duplicate code from RequestStream::send
2020-05-20 10:35:44 -07:00
ed9d5dd857
cannot change sort order of endpoints
2020-05-20 00:00:06 -07:00
c09a28b0db
speed up comparison of endpoints
2020-05-19 20:41:56 -07:00
bf9f24beb5
fixed mismatch of Endpoint UIDs
2020-05-19 17:16:41 -07:00
9313e490aa
change the first part of the UID to make comparisons cheaper
2020-05-19 16:48:20 -07:00
c1fc78489d
Merge pull request #3169 from tclinken/remove-clockticks-and-errorcount
...
Removed uses of outdated cpuTicks and errorCounts
2020-05-15 11:56:33 -07:00
7003a68ba1
Removed outdated comments and errorCounts() declaration
2020-05-15 11:04:26 -07:00
3ff2fa9c2a
Removed uses of outdated cpuTicks and errorCounts
2020-05-13 23:27:44 -07:00
ae2b2700fc
Remove duplicate code from RequestStream::send
2020-05-13 23:24:21 -07:00
cc17484cac
Fixed Windows build
2020-05-13 19:59:35 -07:00
641fd37e71
Remove const_cast from AsyncFileCached::write_impl
2020-05-13 19:59:34 -07:00
4eaf91da14
Remove comma operator from actor fuzz
2020-05-13 19:59:34 -07:00
afe6bf19e2
Fix typos and whitespace
2020-05-11 12:53:19 -07:00
9dc65547ad
Avoid setting failure status of local addresses in main thread
2020-05-11 11:24:19 -07:00
f0f52fb2be
Merge branch 'master' into feature-small-endpoint
...
# Conflicts:
# fdbclient/StorageServerInterface.h
2020-05-08 16:37:35 -07:00
749fcd13b0
Merge branch 'master' into fdb_cache_wo_allocator
2020-05-08 16:23:29 -07:00
62882f5552
Merge pull request #3088 from sears/lb_ub_perf_2
...
unrevert lb_ub_perf pull request
2020-05-07 18:23:17 -07:00
0be453de0a
Merge pull request #3026 from atn34/atn34/tsan
...
Add USE_TSAN and fix some data races
2020-05-07 18:06:18 -07:00
3964fe34b8
Use c11 atomics in libeio
2020-05-07 23:43:50 +00:00
ff992060cd
Merge pull request #3073 from tclinken/fix-open-for-ide-build
...
Fix non-boost-related OPEN_FOR_IDE build errors
2020-05-07 14:47:59 -07:00
574914640a
Merge pull request #2999 from tclinken/more-move-optimizations
...
Generate rvalue reference overloads for actor callback functions
2020-05-07 14:32:27 -07:00
5e1c43f87b
replace double invocations of operator<() with three way compare().
2020-05-07 12:08:29 -07:00
ad900135dd
Simulation did not properly track exclusions of tls processes
2020-05-07 10:53:13 -07:00
51d3aaf4ae
fixed a few rare correctness bugs
2020-05-06 23:24:58 -07:00
8125096e9b
Revert "reduce the number of calls to operator< made by lower_bound and upper…"
2020-05-05 08:17:10 -07:00
dd033736ed
Merge branch 'master' into fdb_cache_subfeature2
2020-05-04 17:29:43 -07:00
4ae356b237
Merge pull request #2882 from sears/lb_ub_perf
...
reduce the number of calls to operator< made by lower_bound and upper…
2020-05-04 15:46:22 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
bc19f70455
Move delayedHealthUpdateF declaration
2020-05-01 18:14:51 -07:00
c126422296
fix: firstFree was not reset after using all available space
2020-05-01 17:46:20 -07:00
4d131bdd4a
Merge branch 'master' into feature-small-endpoint
2020-05-01 13:16:15 -07:00
aed2d34bcb
Merge branch 'master' into feature-proxy-load-balance
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
9753013b65
fdbrpc: update cmake & headers
2020-04-30 20:11:43 +00:00
7a8cb3ffcf
add headers
2020-04-30 18:11:23 +00:00
dd90eff171
Added additional assertion to /flow/flow/PromiseStream/move2
2020-04-30 07:55:13 -07:00
fd0ee72293
Merge branch 'master' into feature-small-endpoint
2020-04-29 18:43:10 -07:00
c4dda9195c
Fixed Windows CMake build
2020-04-29 13:22:01 -07:00
c205bc902f
Added SingleCallback<T>::fire(T&&)
2020-04-29 12:28:17 -07:00
dcdda88794
replace double invocations of operator<() with three way compare().
2020-04-29 10:10:30 -07:00
76fb345dd1
Merge branch 'master' into feature-tree-broadcast
...
# Conflicts:
# fdbrpc/FailureMonitor.actor.cpp
2020-04-29 09:51:22 -07:00
2b2e0b9f75
Merge pull request #2932 from vishesh/task/issue-1017-slow-machine-poisoning
...
FailureMonitoring: Mark unstable connections as failed [1/X]
2020-04-28 13:57:49 -07:00
b7f5f3be48
merge in master
2020-04-28 13:11:47 -07:00
2e6990119b
Fix some data races
2020-04-25 00:26:54 +00:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
76d4570d5d
Merge pull request #3008 from etschannen/release-6.2
...
Added logging for parallel peeks from TLogs
2020-04-23 10:27:49 -07:00
c30010f5b8
FlowTransport: Increase delay for connecting to failed connections
2020-04-23 09:44:30 -07:00
a835123680
exit fdbserver after a ReceiverError, because a packet which should be delivered will never be received
2020-04-22 23:38:46 -07:00
810bba2067
cleanup calls to FlowTransport::isClient()
2020-04-22 23:36:40 -07:00
3a5315d10c
FlowTransport: Don't immediately mark connections failed
...
In connectionKeeper(), when a connection is failed for
FAILURE_DETECTION_DELAY, then only mark connection as failed.
This is much closer to the original centralized behaviour, and also
adds more confidence on whether the connection is actually failed.
2020-04-22 19:46:04 -07:00
8b004fe8e3
Move stop callbacks to be called after run() in sim2.
2020-04-21 20:22:16 -07:00
2628afe463
Improved PromiseStream move unit tests
2020-04-21 15:18:59 -07:00
885a2e020e
Generate rvalue reference overloads with actor compiler
2020-04-21 15:18:59 -07:00
500a265d20
FlowTransport: Refactor and clear FailureStatus on destroy
2020-04-21 14:11:19 -07:00
92c86a7799
Merge remote-tracking branch 'apple/master' into task/issue-1017-slow-machine-poisoning
2020-04-21 03:36:19 -07:00
2ce539ef6d
Respect flow<->fdbrpc module boundaries.
...
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a. However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
3f3a728bb3
Merge pull request #2970 from xumengpanda/mengxu/fr-range-versions-PR
...
Fix:New backup and restore: Mutations before min range version may not be complete
2020-04-19 09:49:07 -07:00
a442565e13
more work towards shrinking locality
2020-04-18 21:29:38 -07:00
1398e9a82e
Stop background eio threads on Net2::stop().
...
This will stop eio threads for both the client (`fdb_stop_network()`)
and the server. This change is being done more for the former, but I
don't see any harm in doing the latter as well.
2020-04-18 19:40:55 -07:00
99a58f8ee5
fix compiler errors
2020-04-17 17:47:50 -07:00
0ee62badcd
Merge branch 'feature-tree-broadcast' into feature-small-endpoint
2020-04-17 17:16:04 -07:00
ba3e2af473
Merge commit '5288033bcfe40c3ade97c8bf2d04cf31b3f16cb1' into feature-tree-broadcast
2020-04-17 15:17:37 -07:00
992002cf34
FlowTransport: don't start connectionKeeper for local peer
...
getOrOpenPeer() is used for local addresses sometimes, which ends up
starting connectionKeeper() which is unnecessary.
2020-04-16 14:04:32 -07:00
43f19bd463
Don't skip filesystem check on when KAIO is disabled
...
WSL the known Linux system where KAIO is not supported, can run these checks.
2020-04-16 10:56:01 -07:00
8c8f23bff2
Merge remote-tracking branch 'apple/master' into task/issue-1017-slow-machine-poisoning
2020-04-16 00:45:35 -07:00
1901f49b97
Net2FileSystem: Add guards to honor DISABLE_POSIX_KERNEL_AIO
...
- Adds some asserts in KAIO to ensure that when knob is set, we don't
end up using KAIO in any case.
- Fixes a bug where we initialize AsyncFileKAIO on Linux builds even
when KAIO is disabled. This can cause problems in systems such as
Windows Subsystem for Linux where KAIO is not supported.
FIXES #2382
2020-04-15 23:47:37 -07:00
da7d0093ee
Cleanup unused code
2020-04-15 19:48:25 -07:00
023372a226
FailMon: Mark peer failed after retrying
2020-04-15 19:47:24 -07:00
Monsij Biswal