wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,901 Commits 33 Branches 308 Tags 276 MiB
Commit Graph

16901 Commits

Author SHA1 Message Date
Steve Atherton 2de47a9792
Merge pull request #5364 from sfc-gh-satherton/debugging-cleanup 
Added range tracking support to MutationTracking.
 2021-08-10 15:20:32 -07:00
Lukas Joswiak b112560c94 Reorder registerWorker to prevent potential conflict 2021-08-10 15:09:35 -07:00
Steve Atherton 5482c86d31 Added range tracking support to MutationTracking. 2021-08-10 14:38:02 -07:00
Lukas Joswiak f018af6ee4
Update fdbserver/ConfigBroadcaster.actor.cpp 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-08-10 13:24:41 -07:00
Lukas Joswiak d27c9e2520 Revert error check 2021-08-10 12:41:41 -07:00
Lukas Joswiak a838a47b0b Use ActorCollection for consumer future 2021-08-10 12:27:19 -07:00
Lukas Joswiak 9f4ea02784 Reremove old TODO 2021-08-10 12:14:50 -07:00
Lukas Joswiak 598b23f8d4 Merge branch 'features/broadcaster-push' of github.com:sfc-gh-ljoswiak/foundationdb into features/broadcaster-push 2021-08-10 12:08:16 -07:00
Lukas Joswiak 2ed2294ccb Remove old TODO 2021-08-10 12:00:43 -07:00
Lukas Joswiak 5dfd7c4b1a Remove redundant dead worker check 2021-08-10 11:56:58 -07:00
Lukas Joswiak cf81b0650d Only register consumer once on the broadcaster 2021-08-10 11:56:16 -07:00
Lukas Joswiak 72e55ef72e Add broadcaster error check to unit tests 2021-08-10 11:39:29 -07:00
Lukas Joswiak 564a3d69b7 Rename config broadcast interface messages 2021-08-10 11:39:29 -07:00
Lukas Joswiak 833e53f8f0 Add tests back 2021-08-10 11:39:29 -07:00
Lukas Joswiak 85fa264a16 Remove move constructor and assignment operator 2021-08-10 11:39:29 -07:00
Lukas Joswiak 305a17c811 Improve config broadcaster logic, fix unit tests 2021-08-10 11:39:29 -07:00
Lukas Joswiak 72e63db856 Send ConfigBroadcastInterface to ConfigBroadcaster instead of entire worker interface 2021-08-10 11:39:29 -07:00
Lukas Joswiak 3946cf94ff Push updates to workers (clang-formatted files) 2021-08-10 11:39:29 -07:00
Lukas Joswiak 092ab4302b Push updates to workers 2021-08-10 11:39:29 -07:00
sfc-gh-tclinkenbeard d8bdc972ba Only test simple configuration database in ConfigIncrement.toml 
There are still some issues with the Paxos implementation that are
causing test failures at the moment.
 2021-08-10 11:18:56 -07:00
Xiaoxi Wang 1f6cee89ab merge master, fix conflicts 2021-08-10 10:01:45 -07:00
Lukas Joswiak 3a607d9a38
Update fdbserver/ConfigBroadcaster.actor.cpp 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-08-10 09:36:39 -07:00
Josh Slocum dcef866e18
Merge pull request #5360 from sfc-gh-satherton/debugging-cleanup 
TraceEvent detail formatting cleanup and MutationTracking improvements
 2021-08-10 07:13:50 -07:00
Steve Atherton 54c7036eaf Move role UIDs for MutationTracking TraceEvents from various inconsistent detail fields into the TraceEvent UID field. 2021-08-10 01:52:36 -07:00
Steve Atherton e5e8a56b66 Merge branch 'master' of https://github.com/apple/foundationdb into debugging-cleanup 
# Conflicts:
#	fdbbackup/FileDecoder.actor.cpp
 2021-08-09 23:36:57 -07:00
Steve Atherton faa4154a56 MutationTracking now uses a vector of keys to track. Removed "Mutation" detail from DEBUG_MUTATION() events because they are duplicates of the fields already logged in the returned MutationTracking event, which are now renamed and combined into "Mutation". Removed more toString() calls in TraceEvent detail values. 2021-08-09 23:30:45 -07:00
Trevor Clinkenbeard 428c2f3bfa
Merge pull request #5322 from apple/anoyes/fix-tsan 
Fix TSAN warnings
 2021-08-09 23:16:22 -07:00
Lukas Joswiak ec78fb3060 Add tests back 2021-08-09 17:42:27 -07:00
Chaoguang Lin eac7ffde74
Merge pull request #5358 from sfc-gh-clin/reenable-exclude-test 
Fix the exclude command ctest
 2021-08-09 17:09:11 -07:00
helium 751fc59d54 revert unintended changes 2021-08-09 16:29:57 -07:00
helium 15d050ece5 Added unit test, and bug fixes. 2021-08-09 16:26:29 -07:00
helium 9a9346afaf Added unit test. 2021-08-09 16:21:43 -07:00
Lukas Joswiak c97a1b3b4d Remove move constructor and assignment operator 2021-08-09 15:33:01 -07:00
Chaoguang Lin 3b9cb1a85a Re-enable exclude command ctest 2021-08-09 21:18:27 +00:00
Andrew Noyes 7706b47085 Fix another TSAN warning 
```
  Read of size 1 at 0x7f02082ac578 by main thread:
    #0 TraceLog::close() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:487:7 (libfdb_c.so+0xf4928c)
    #1 closeTraceFile() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:759:13 (libfdb_c.so+0xf3992a)
    #2 stopNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2052:2 (libfdb_c.so+0x58315c)
    #3 ThreadSafeApi::stopNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:466:2 (libfdb_c.so+0xce8ad3)
    #4 MultiVersionApi::stopNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1756:20 (libfdb_c.so+0x4f7da3)
    #5 fdb_stop_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:134:2 (libfdb_c.so+0x4c84e5)
    #6 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/trace_partial_file_suffix_test.cpp:93:12 (trace_partial_file_suffix_test+0x40b3d0)

  Previous write of size 1 at 0x7f02082ac578 by thread T1 (mutexes: write M100):
    #0 TraceLog::open(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, unsigned long, Optional<NetworkAddress>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:344:10 (libfdb_c.so+0xf48aaa)
    #1 openTraceFile(NetworkAddress const&, unsigned long, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Trace.cpp:741:13 (libfdb_c.so+0xf3828c)
    #2 Database::createDatabase(Reference<ClusterConnectionFile>, int, IsInternal, LocalityData const&, DatabaseContext*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:1695:4 (libfdb_c.so+0x57d57e)
    #3 ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7::operator()() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:124:8 (libfdb_c.so+0xcf018c)
    #4 internal_thread_helper::DoOnMainThreadVoidActorState<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7, internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7> >::a_body1cont1(Void const&, int) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/ThreadHelper.actor.h:45:4 (libfdb_c.so+0xcf018c)
    #5 internal_thread_helper::DoOnMainThreadVoidActorState<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7, internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7> >::a_body1when1(Void const&, int) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/flow/ThreadHelper.actor.g.h:148:15 (libfdb_c.so+0xcf018c)
    #6 internal_thread_helper::DoOnMainThreadVoidActorState<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7, internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7> >::a_callback_fire(ActorCallback<internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7>, 0, Void>*, Void const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/flow/ThreadHelper.actor.g.h:169:4 (libfdb_c.so+0xcefdef)
    #7 ActorCallback<internal_thread_helper::DoOnMainThreadVoidActor<ThreadSafeDatabase::ThreadSafeDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int)::$_7>, 0, Void>::fire(Void const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/flow.h:1074:78 (libfdb_c.so+0xcefdef)
    #8 void SAV<Void>::send<Void>(Void&&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/flow.h:462:23 (libfdb_c.so+0x4dd542)
    #9 void Promise<Void>::send<Void>(Void&&) const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/flow.h:699:8
    ```
 2021-08-09 14:06:38 -07:00
Lukas Joswiak 5249105b04 Improve config broadcaster logic, fix unit tests 2021-08-09 13:20:06 -07:00
Andrew Noyes ecc078a42f Fix TSAN warnings 
Avoid doing thread-unsafe things in the ReadYourWritesTransaction
default constructor since we now will be calling it in a foreign thread.

Here's the warning we're fixing:

Write of size 8 at 0x7b6800000398 by thread T1:
    #0 N2::Net2::updateNow() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:192:33 (libfdb_c.so+0xe74568)
    #1 N2::Net2::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:1459:3 (libfdb_c.so+0xe74568)
    #2 runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2036:13 (libfdb_c.so+0x575004)
    #3 ThreadSafeApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:445:3 (libfdb_c.so+0xcda9cc)
    #4 MultiVersionApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1741:20 (libfdb_c.so+0x4eab8d)
    #5 fdb_run_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:130:2 (libfdb_c.so+0x4bb425)
    #6 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3899:1 (fdb_c_unit_tests+0x389816)
    #7 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:280:5 (fdb_c_unit_tests+0x389816)
    #8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (fdb_c_unit_tests+0x389816)

  Previous read of size 8 at 0x7b6800000398 by main thread:
    #0 N2::Net2::now() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:160:39 (libfdb_c.so+0xe7ea5d)
    #1 Span::Span(UID, Location, std::initializer_list<UID> const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Tracing.h:38:41 (libfdb_c.so+0x5837b4)
    #2 Transaction::Transaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:4092:64 (libfdb_c.so+0x5837b4)
    #3 ReadYourWritesTransaction::ReadYourWritesTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ReadYourWrites.h:134:2 (libfdb_c.so+0x7cb4a8)
    #4 ISingleThreadTransaction::allocateOnForeignThread(ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ISingleThreadTransaction.cpp:29:17 (libfdb_c.so+0xcf785f)
    #5 ThreadSafeTransaction::ThreadSafeTransaction(DatabaseContext*, ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:149:23 (libfdb_c.so+0xcc9540)
    #6 ThreadSafeDatabase::createTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:50:37 (libfdb_c.so+0xcc4b1c)
    #7 fdb_database_create_transaction /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:332:2 (libfdb_c.so+0x4bce1d)
    #8 fdb::Transaction::Transaction(FDB_database*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/fdb_api.cpp:111:24 (fdb_c_unit_tests+0x3925d0)
    #9 _DOCTEST_ANON_FUNC_10() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:250:19 (fdb_c_unit_tests+0x32a7a8)
    #10 doctest::Context::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/doctest/src/doctest/doctest/doctest.h:6112:21 (fdb_c_unit_tests+0x3262e4)
    #11 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:2286:20 (fdb_c_unit_tests+0x36303d)
 2021-08-09 11:16:52 -07:00
helium 32b1bead62 Introduced ThreadReturnPromisStream, the stream version of ThreadReturnPromise. 2021-08-09 10:26:48 -07:00
sfc-gh-tclinkenbeard c126e6239b Weaken ConfigIncrement.toml test 
This test is not yet stable, so simplify the test in order to make
debugging easier.
 2021-08-09 10:18:18 -07:00
sfc-gh-tclinkenbeard a5b916cd8d PaxosConfigTransaction should only send read requests to valid replicas 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard 79ba9c4e3a Add GetGenerationQuorum to get generation from a quorum of config nodes 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard bd5a87e0e3 Fix TestConfig::ConfigBuilder::trace_visitor::operator()(Optional<std::vector<int>> const* val) 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard f6b6e11d6a Support randomizing configuration database type in simulation 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard 82546853c0 Rename UseConfigDB to ConfigDBType 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard cdbcb69d86 Add configuration database type to ISimulator 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard cd8a82b1b8 Rename GetGenerationRequests metric 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard 42a45ebfc4 Temporarily comment out configuration database code breaking simulation tests 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard 6ac88a3cf9 Fix localIncrementActors calculation in ConfigIncrement workload 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard 6524dd1567 Add random sleep to ConfigIncrement workload 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard 7deb339a0c Added ConfigIncrement.toml test 2021-08-09 10:04:35 -07:00