wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,087 Commits 34 Branches 312 Tags 293 MiB
Commit Graph

180 Commits

Author SHA1 Message Date
Sreenath Bodagala 852fc96200 Address simulation test failures caused by: 
- Assertion failures in MoveKeys.actor.cpp
- Wrong results returned by getRange()

Changes:

DatabaseContext.h, NativeAPI.actor.[h,cpp]:
- Introduce a new flag, TransactionInfo::readVersionObtainedFromGrvProxy.
- Set this flag to true by default, and clear it when the read version of a
transaction is explicitly set (by using setVersion()).
- Modify getLatestCommitVersions() to not populate "latestCommitVersions" if
this flag is not set. (This will cause storage server to read at the specified
read version.)

- Modify getRange() actor to always use the specified version as the read
version (except when the specified version is latestVersion).

- Modify waitForCommittedVersion(), getRawVersion(), and getConsistentReadVersion()
to update local version vector cache after receiving GetReadVersionReply.

IClientApi.h, IConfigTransaction.h, ISingleThreadTransaction.h,
MultiVersionTransaction[.actor].[h,cpp], ThreadSafeTransaction.[h,cpp],
ApiWorkload.h:
- Add methods to get the spanID of a transaction and also the version vector
cached in a transaction. (Likely to be useful for debugging simulation test
failures.)

VersionVector.h:
- Update "maxVersion" when populating/applying a delta. (Note that empty
mutation messages only update VersionVector::maxVersion.)

BackupWorker.actor.cpp:
- Update local version vector cache after receiving GetReadVersionReply message.

Status.actor.cpp:
- Update local version vector cache and
TransactionInfo::info.readVersionObtainedFromGrvProxy after setting the
read version.
 2021-09-16 12:00:26 -04:00
Chaoguang Lin fbed117b21 Solve comments; Add a method to return DatabaseContext Reference 2021-08-19 19:34:25 +00:00
Andrew Noyes ecc078a42f Fix TSAN warnings 
Avoid doing thread-unsafe things in the ReadYourWritesTransaction
default constructor since we now will be calling it in a foreign thread.

Here's the warning we're fixing:

Write of size 8 at 0x7b6800000398 by thread T1:
    #0 N2::Net2::updateNow() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:192:33 (libfdb_c.so+0xe74568)
    #1 N2::Net2::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:1459:3 (libfdb_c.so+0xe74568)
    #2 runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:2036:13 (libfdb_c.so+0x575004)
    #3 ThreadSafeApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:445:3 (libfdb_c.so+0xcda9cc)
    #4 MultiVersionApi::runNetwork() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/MultiVersionTransaction.actor.cpp:1741:20 (libfdb_c.so+0x4eab8d)
    #5 fdb_run_network /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:130:2 (libfdb_c.so+0x4bb425)
    #6 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3899:1 (fdb_c_unit_tests+0x389816)
    #7 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:280:5 (fdb_c_unit_tests+0x389816)
    #8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (fdb_c_unit_tests+0x389816)

  Previous read of size 8 at 0x7b6800000398 by main thread:
    #0 N2::Net2::now() const /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/flow/Net2.actor.cpp:160:39 (libfdb_c.so+0xe7ea5d)
    #1 Span::Span(UID, Location, std::initializer_list<UID> const&) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../flow/Tracing.h:38:41 (libfdb_c.so+0x5837b4)
    #2 Transaction::Transaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/fdbclient/NativeAPI.actor.cpp:4092:64 (libfdb_c.so+0x5837b4)
    #3 ReadYourWritesTransaction::ReadYourWritesTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ReadYourWrites.h:134:2 (libfdb_c.so+0x7cb4a8)
    #4 ISingleThreadTransaction::allocateOnForeignThread(ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ISingleThreadTransaction.cpp:29:17 (libfdb_c.so+0xcf785f)
    #5 ThreadSafeTransaction::ThreadSafeTransaction(DatabaseContext*, ISingleThreadTransaction::Type) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:149:23 (libfdb_c.so+0xcc9540)
    #6 ThreadSafeDatabase::createTransaction() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../fdbclient/ThreadSafeTransaction.cpp:50:37 (libfdb_c.so+0xcc4b1c)
    #7 fdb_database_create_transaction /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/fdb_c.cpp:332:2 (libfdb_c.so+0x4bce1d)
    #8 fdb::Transaction::Transaction(FDB_database*) /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/fdb_api.cpp:111:24 (fdb_c_unit_tests+0x3925d0)
    #9 _DOCTEST_ANON_FUNC_10() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:250:19 (fdb_c_unit_tests+0x32a7a8)
    #10 doctest::Context::run() /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/doctest/src/doctest/doctest/doctest.h:6112:21 (fdb_c_unit_tests+0x3262e4)
    #11 main /home/jenkins/fdb/extra/long/path/to/work/around/strange/cpack/debug/rpm/behavior/_build/../bindings/c/test/unit/unit_tests.cpp:2286:20 (fdb_c_unit_tests+0x36303d)
 2021-08-09 11:16:52 -07:00
Andrew Noyes 39eff8c569 Add trace_partial_file_suffix network option 2021-08-03 09:43:42 -07:00
sfc-gh-tclinkenbeard b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Steve Atherton f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00
sfc-gh-tclinkenbeard 475abe301c Merge remote-tracking branch 'origin/master' into fix-ub 2021-07-14 10:47:02 -07:00
sfc-gh-tclinkenbeard d84b2a95e2 Make function declarations less verbose 2021-07-05 13:27:01 -07:00
sfc-gh-tclinkenbeard 8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
sfc-gh-tclinkenbeard 79ff07a071 Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters 2021-07-02 15:04:42 -07:00
Andrew Noyes d69fda1d2e Remove preinitializeOnForeignThread 2021-06-23 14:33:41 -07:00
Evan Tschannen 244e8f3527 Merge branch 'master' of https://github.com/apple/foundationdb into feature-streaming-reply-master 
# Conflicts:
#	fdbclient/CMakeLists.txt
#	fdbclient/Knobs.h
#	fdbserver/Knobs.h
 2021-06-21 16:35:21 -07:00
Evan Tschannen 1cec223434 cherry pick streaming replies to master 2021-06-19 09:47:13 -07:00
sfc-gh-tclinkenbeard 41c790b299 Merge remote-tracking branch 'origin/master' into config-db 2021-06-10 22:31:23 -07:00
Xiaoxi Wang 5be65fab5e add comment 2021-06-04 18:40:18 +00:00
Xiaoxi Wang 365b0bc4d8 better api 2021-06-03 19:57:44 +00:00
sfc-gh-tclinkenbeard a775f92fca Merge remote-tracking branch 'origin/master' into config-db 2021-06-01 15:39:34 -07:00
Xiaoxi Wang 24c0c3361a change quietdatabase 2021-05-25 21:20:24 +00:00
sfc-gh-tclinkenbeard bb61a20851 Improve const-correctness of ISingleThreadTransaction 2021-05-15 00:23:21 -07:00
sfc-gh-tclinkenbeard 5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
sfc-gh-tclinkenbeard ec58f0f0cf Improve const-correctness of ISingleThreadTransaction 2021-04-30 11:32:53 -07:00
A.J. Beamon b2d6930103 The multi-version client monitors the cluster's protocol version and only activates the client library that can connect. 2021-04-15 11:45:14 -07:00
A.J. Beamon 3ed0d614d2 Move fdb_get_server_protocol to be a function on the database object. Add an argument for expected_version that can be used to signal that the function shouldn't return unless the protocol version is different. 2021-04-14 12:50:30 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes 9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces 
Stable interfaces
 2020-12-11 09:34:27 -08:00
Lukas Joswiak 3669615e4b Refactor and clean up 2020-12-03 14:06:11 -08:00
Richard Chen a4ccf8c5b6 add checksum class to change checksum in the future. expose get server protocol api 2020-10-19 16:57:11 +00:00
Xin Dong 5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size 
Feature/range split points based on size
 2020-10-08 14:25:11 -07:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Xin Dong cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size 2020-08-27 10:06:02 -07:00
Xiaoxi Wang 24dbb3961c Knob adjustment 2020-08-17 22:38:22 +00:00
Xin Dong daead84fdd Merge master 2020-08-14 14:07:08 -07:00
Xiaoxi Wang e94de126de merge with master 2020-08-14 01:59:09 +00:00
Meng Xu 002cca5e58 Merge branch 'master' into tmp/merge-PR 
Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/StorageMetrics.actor.h
 2020-08-12 20:08:13 -07:00
Xin Dong 3ac6996844 Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful 2020-08-12 14:27:34 -07:00
Xiaoxi Wang f3ecf14601 change midShardSize type and other details 2020-08-12 17:49:12 +00:00
Xiaoxi Wang cae6f04070 sample on cost instead of bytes 2020-08-10 22:30:06 +00:00
Xiaoxi Wang 696e77c94e query midShardSize from proxy 2020-08-10 20:13:44 +00:00
Xin Dong a46e620baa Merge branch 'master' into feature/range-split-points-based-on-size 2020-08-06 15:23:11 -07:00
Xiaoxi Wang 6db7040f17 update DDMetrics periodically 2020-08-03 05:52:38 +00:00
Xiaoxi Wang bcb858288b eliminate cost estimation when tagSet is empty 2020-07-14 23:07:21 +00:00
Xiaoxi Wang d512170cd8 add clear cost estimation 2020-07-14 00:18:52 +00:00
Xiaoxi Wang 6ec2f92a8d merge master 2020-07-13 17:22:37 +00:00
Xiaoxi Wang 6374cd96c9 add commit cost estimation 2020-07-10 23:07:29 +00:00
Markus Pilman 69864c9f96 Make Spans not allocate heap memory 2020-07-09 11:49:33 -06:00
Markus Pilman 0fbe7101c3 Revert "Revert "Request tracing"" 
This reverts commit 327cc31e35.
 2020-07-07 10:06:13 -06:00
Xin Dong eb28492900 Fixed a bug which leads to inaccurate metrics being reported. Added tests for the new API. 2020-07-02 14:05:11 -07:00
Xin Dong ea08bc5462 Added server side code for range split support 2020-07-02 14:05:11 -07:00
Meng Xu 22f7f804b8 Merge branch 'release-6.3' into mengxu/merge-6.3-PR 2020-06-28 11:19:39 -07:00
Jingyu Zhou 5cd3dad986
Merge pull request #3409 from sears/assert_strings 
Print out the text (to stderr and as trace) of the failed condition o…
 2020-06-24 14:58:12 -07:00