wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
23,587 Commits 34 Branches 308 Tags 276 MiB
Commit Graph

23587 Commits

Author SHA1 Message Date
Xiaoxi Wang cc61ea6a01 finish the clearrange ops 2022-11-01 14:56:55 -07:00
Xiaoxi Wang 334fced572 add data api implementations; add more realistic fetchKey implementation; finish randomKeyBetween implementation 2022-11-01 14:56:55 -07:00
Xiaoxi Wang 8a59bc276d data operation api (not finished) 2022-11-01 14:56:54 -07:00
Josh Slocum e48135086e refactoring check for summarize op to use existing validation 2022-11-01 13:56:11 -07:00
Josh Slocum 55acc27bd3 fixing bug for tenant-aware reads 2022-11-01 13:56:11 -07:00
Josh Slocum f02fd406b4 bg api tester: better debugging on errors, and added verbose debugging mode 2022-11-01 13:56:11 -07:00
Jingyu Zhou 0135d9cee1
Merge pull request #8643 from jzhou77/fix 
Remove unnecessary decodeServerTagValue calls
 2022-11-01 12:28:08 -07:00
Xiaoge Su f1eca6d672 fixup! Fix compile error 2022-11-01 10:36:25 -07:00
Xiaoge Su 9ee99ad058 fixup! Reformat source 2022-11-01 10:36:25 -07:00
Xiaoge Su 520a14cd72 fixup! update code per comments 2022-11-01 10:36:25 -07:00
Xiaoge Su 769bc28ea3 Reformat source 2022-11-01 10:36:25 -07:00
Xiaoge Su 12ea86efb6 Print parsed log lines via vsnprintf 2022-11-01 10:36:25 -07:00
Xiaoge Su bf84b7a216 Reformat source 2022-11-01 10:36:25 -07:00
Xiaoge Su c4b083f22d Move the periodic logger to RocksDBLogger 2022-11-01 10:36:25 -07:00
Xiaoge Su 74de1aaa8d Redirect RocksDB Log data to FDB TraceEvent 
RocksDB supports injecting a custom logger. By introducing
RocksDBLogForwarder, it is possible to retrieve RocksDB log lines and
log them as TraceEvents.

Note that RocksDB is multi-threaded while FDB is designed to be
single threaded, in the FDB event loop there are certain variables
defined as thread_local. This causes that only FDB main thread could
generate TraceEvent, since information like Machine are only available
in the main thread. The current design is to cache all log lines from
the other threads, and let the main thread report the cached log, when:

  * The main thread received log
  * Every some time (this should be a Knob but at this stage a magic
    number is used for convenience.)

It is possible to parse the log lines into Key/Value pairs for
TraceEvent output, but at this stage only the format string is printed
out. Also, RocksDB may log some errors that are interpreted as SevError,
which will cause simulation failures. To prevent this, the maximum error
level is set to SevWarn, which should be reconsidered carefully.

The code is tested by 1) random simulations to ensure the log lines are
included in the XML files 2) force the simulator uses RocksDB and run a
10k correctness run. Both tests passed.
 2022-11-01 10:36:25 -07:00
Xiaoge Su 7c4063e32c dfasdf 2022-11-01 10:36:25 -07:00
Jingyu Zhou d1adc067d6
Merge pull request #8646 from sfc-gh-satherton/diskqueue-write-bug 
File write() buffer lifetime fix in DiskQueue.
 2022-11-01 10:23:12 -07:00
Jingyu Zhou bc098ff8d7 Save a copy for param2 2022-11-01 10:14:58 -07:00
Josh Slocum 178e3bb700
Changing priority for not-at-latest change feed reads (#8651) 2022-11-01 10:59:04 -05:00
Trevor Clinkenbeard 39abc712b0
Merge pull request #8549 from sfc-gh-tclinkenbeard/expose-txn-cost 
Create `fdb_transaction_get_total_cost` function
 2022-11-01 08:14:57 -07:00
Steve Atherton d12bb31233 File write() buffer lifetime fix in DiskQueue. 
Both push() and pushAndCommit() now require reference-counted pageData and file->write() is wrapped in uncancellable(holdWhile(pageData, ..)).  To support this with no additional overhead, StringBuffer returns a Standalone reference to its contents.
 2022-10-31 22:06:48 -07:00
Dennis Zhou aab0276969 blob: fix bad continuation actor after shutdown 
bin/fdbserver -r simulation -f tests/slow/BlobGranuleVerifyBalance.toml -s 68505494 -b off --crash --trace_format json
 2022-10-31 21:28:18 -07:00
Nim Wijetunga 24ce8c0fd0
Commit Proxy Encryption Code Probes (#8618) 
* add commit proxy encryption code probes

* fix comment

* address pr comments

* address pr comments

* address pr comments

* address pr comments

* Trigger Build
 2022-10-31 20:04:42 -07:00
Trevor Clinkenbeard d385fc5af1
Merge pull request #8626 from sfc-gh-satherton/flowbench-delay-yield 
Added benchmark for delay(0) and yield().
 2022-10-31 19:06:05 -07:00
sfc-gh-tclinkenbeard 69f669f36b Change type of value in QuotaCommand.actor.cpp 2022-10-31 18:58:55 -07:00
Jingyu Zhou d76a003351 Remove unnecessary decodeServerTagValue calls 2022-10-31 16:58:08 -07:00
Josh Slocum fa80d7bfde
Blobbifiyng tenants in mako so that blob granule reads in tenant mode work (#8587) 
* Blobbifiyng tenants in mako so that blob granule reads in tenant mode work

* removing unecessary comment
 2022-10-31 18:08:29 -05:00
Jingyu Zhou 8cb2d5e73c
Merge pull request #8594 from yao-xiao-github/fix-deadline 
Use a different error code for KV timeout and add timeout to ReadRange.
 2022-10-31 15:54:39 -07:00
Trevor Clinkenbeard fdd2f24174
Fix comment in NativeAPI.actor.h 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2022-10-31 14:48:13 -07:00
Dennis Zhou 27f9bc1fbc
Merge pull request #8638 from sfc-gh-dzhou/blob-2 
blob: fix transaction reset in getBlobRanges() and refactor blob get tenant code
 2022-10-31 14:22:22 -07:00
Yao Xiao e7e8c7ea19 Merge branch 'main' of github.com:apple/foundationdb into fix-deadline 2022-10-31 13:26:31 -07:00
Steve Atherton 70e078718c Bug fix in items processed count, changed delay/yield test to be a template. 2022-10-31 12:49:38 -07:00
Ankita Kejriwal fcdf1ed6ff
Merge pull request #8603 from sfc-gh-tclinkenbeard/ignore-worst-zone 
Make `GlobalTagThrottler` ignore worst zone
 2022-10-31 12:46:44 -07:00
A.J. Beamon e44a7a105b
Merge pull request #8636 from LukasMoll/pep-8-tests 
Applied flake8
 2022-10-31 12:31:09 -07:00
He Liu 8d76636a7f
Don't unassign ranges from dest servers if they are skipped. (#8611) 
Co-authored-by: He Liu <heliu@apple.com>
 2022-10-31 12:07:42 -07:00
Yao Xiao 891a5bee10
Add timeout for long running test. (#8634) 2022-10-31 11:19:58 -07:00
Lukas Molleman 76197540dc Applied flake8 2022-10-31 19:19:17 +01:00
Dennis Zhou f7b608e53f blob: refactor blob get tenant code 2022-10-31 10:37:36 -07:00
Dennis Zhou 1ab432e49d blob: fix error propagation in getBlobRanges() 
Fixes: 48d6e725c2 ("blob: convert listBlobbifiedRangesActor() to take a Transaction")
 2022-10-31 10:37:36 -07:00
A.J. Beamon 0091abd02b
Merge pull request #8491 from LukasMoll/pep-8 
Applied flake8 rules to 2 python files
 2022-10-31 09:53:08 -07:00
Jingyu Zhou 766d7a6159
Merge pull request #8581 from sfc-gh-tclinkenbeard/add-tag-throttling-latency-bands 
Add latency bands for proxy-enforced tag throttling
 2022-10-31 09:23:46 -07:00
Jingyu Zhou af7b434e51
Merge pull request #8593 from sfc-gh-jshim/fix-tls-circular-reference 
Fix circular reference in SSLConnection
 2022-10-31 09:21:43 -07:00
Steve Atherton 34c61580a0 Added benchmark for delay(0) and yield(). 2022-10-30 20:50:58 -07:00
sfc-gh-tclinkenbeard 317f17df89 Added fdb_transaction_get_total_cost test 2022-10-30 14:32:58 -07:00
sfc-gh-tclinkenbeard 0eb1598afa Merge remote-tracking branch 'origin/main' into expose-txn-cost 2022-10-30 09:36:37 -07:00
sfc-gh-tclinkenbeard 1d6bd0057b Merge remote-tracking branch 'origin/main' into add-tag-throttling-latency-bands 2022-10-29 09:49:37 -07:00
sfc-gh-tclinkenbeard c3c2435228 Addressed review comments 2022-10-28 20:48:18 -07:00
Chaoguang Lin 65aeeff2a2
Update long running tests configs and add more debugging info in the summary (#8615) 
* Disable simulation speedup knob and increase the trace limit for long running tests; Add some debugging info for the summary of long running tests

* Solve comments
 2022-10-28 18:03:22 -07:00
Jingyu Zhou 22293ebac5
Merge pull request #8614 from neethuhaneesha/clearRanges 
Enable clear range eager reads knob for rocksdb.
 2022-10-28 17:03:39 -07:00
Steve Atherton b3017ae330
Merge pull request #8577 from sfc-gh-satherton/storageserver-pml 
Unrevert #7578 - new PriorityMultiLock and StorageServer read prioritization.
 2022-10-28 16:04:44 -07:00