Commit Graph

27117 Commits

Author SHA1 Message Date
Zhe Wang ebb05f54c3
Add storage interface for checksum (#11144)
* add-storage-interface-for-check-sum

* address comments
2024-01-24 14:34:35 -08:00
Jingyu Zhou ddee332d87
Merge pull request #11145 from brownleej/backup-typo-fix
Fixes a typo in the output from the restore command.
2024-01-24 12:56:24 -08:00
John Brownlee 04ef1be92a Fixes a typo in the output from the restore command. 2024-01-24 09:00:15 -08:00
Jingyu Zhou 888423f8f4
Merge pull request #11132 from chuang-he/main-dev
Fix healthyzone is not modified when ignoressfailuers is set
2024-01-23 09:12:34 -08:00
Jingyu Zhou 734dcce738
Merge pull request #11143 from FartSimps0n/patch-1
A minor grammar and spelling suggestion
2024-01-23 09:03:29 -08:00
David Sugarman 43870df35e
A minor grammar and spelling suggestion
Hello FoundationDB team,

This PR contains a minor grammar suggestion for you to consider.

Best regards,
David
2024-01-23 13:33:26 +02:00
Johannes Scheuermann ea5f521f15
Refactor code to make use of the controller-runtime and allow delayed shutdown of the monitor process (#11108) 2024-01-18 08:00:26 +01:00
Yao Xiao 55d88c17c9
Rename knobs and clean up some trace events. (#11135) 2024-01-17 16:26:26 -08:00
Yao Xiao 2329e8327a
Add log cleaner for rocksdb logs. (#11134)
Co-authored-by: yaoxiao-github <yaoxiao@Yaos-MacBook-Pro-14.local>
2024-01-17 14:51:15 -08:00
Jingyu Zhou 590e5b8673
Merge pull request #11130 from jen20/jen20/crc32-sse
Do not opt into sse4.2 for arm64 CPUs in crc32 library
2024-01-17 14:48:47 -08:00
Jingyu Zhou 14febc1f36
Merge pull request #11126 from apple/dependabot/pip/documentation/sphinx/jinja2-3.1.3
Bump jinja2 from 3.1.2 to 3.1.3 in /documentation/sphinx
2024-01-17 14:48:16 -08:00
Jingyu Zhou e2fc5f3f1c
Merge pull request #10935 from drTr0jan/msgpack
GetMsgpack: add 4+ versions of upstream
2024-01-17 13:02:00 -08:00
Dan Lambright 0bfc99bf1f
ensure synthetic data is written to existing shards (#11128)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-01-16 10:22:07 -05:00
chuang he 64b01298b2 Fix healthyzone is not modified when ignoressfailuers is set
Signed-off-by: chuang he <hechuang@xsky.com>
2024-01-15 16:27:07 +08:00
Jingyu Zhou da1ad45200
Merge pull request #11125 from yao-xiao-github:fix
Retry when Ratekeeper failed to get StorageQueueInfo.
2024-01-12 14:11:38 -08:00
James Nugent 3813e08129 Do not opt into sse4.2 for arm64 CPUs in crc32 library
The fork of the crc32 library in contrib makes use of streaming SIMD
instructions where possible. This is achieved via a preprocessor directive, but
short-circuiting of the evaluation previously meant that any use of clang or
GCC opted into SSE regardless.

This commit adds additional parentheses to ensure a warning-free build of the
CRC32 library using clang on ARM64 machines.

On macOS on Apple Silicon:

```
$ uname -m
arm64
```

Before:

```
$ ninja crc32
[0/2] Re-checking globbed directories...
[3/4] Building CXX object contrib/crc32/CMakeFiles/crc32.dir/crc32c.cpp.o
'+sse4.2' is not a recognized feature for this target (ignoring feature)
'+sse4.2' is not a recognized feature for this target (ignoring feature)
[4/4] Linking CXX static library lib/libcrc32.a
```

After:

```
$ ninja -t clean
Cleaning... 5 files.

$ ninja crc32
[0/2] Re-checking globbed directories...
[4/4] Linking CXX static library lib/libcrc32.a
```
2024-01-11 19:40:15 -06:00
dependabot[bot] 49e354beb2
Bump jinja2 from 3.1.2 to 3.1.3 in /documentation/sphinx
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.2...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-11 20:04:39 +00:00
yaoxiao-github f4af16ecf7 Retry when Ratekeeper failed to get StorageQueueInfo. 2024-01-10 16:07:27 -08:00
Jingyu Zhou 947dc32963
Merge pull request #11118 from hfu94/fixm
Fix checkall when shard is large
2024-01-10 09:24:39 -08:00
hao fu e8cdfc5a0c Fix checkall when shard is large
the begin key has to be updated for checkall command when a shard
is large, this PR makes this change.
2024-01-09 18:19:49 -08:00
Jingyu Zhou e458651a04
Merge pull request #10740 from sfc-gh-jslocum/fix_tenant_watching
Fix tenant watching
2024-01-09 11:22:48 -08:00
Dan Lambright 54ebcde97b
Fix bug in synthetic data creation (#11115)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-01-08 12:56:41 -05:00
neethuhaneesha b9166aa164
Removed latencySamples in kvstorerockddb to avoid DDSketch race access. (#11113) 2024-01-05 10:46:58 -08:00
Jingyu Zhou 75f7814ad1
Merge pull request #11112 from sfc-gh-jslocum/stuck_watch_fix_main
Stuck watch bug fix
2024-01-05 10:41:10 -08:00
Jingyu Zhou 78174692d3
Merge pull request #11107 from apple/synthesize-data 2024-01-04 09:17:39 -08:00
Yao Xiao 3af49dc07e
read knob (#11111)
Co-authored-by: yaoxiao-github <yaoxiao@Yaos-MacBook-Pro-14.local>
2024-01-03 14:34:59 -08:00
Josh Slocum 611eb00fe1 stuck watch bug fix
* buggify watch version retry and fix multiple watch race after retry

* watch debugging improvements
2024-01-03 16:05:42 -06:00
Dan Lambright 86a2301faa updated per review comments 2024-01-03 12:21:54 -05:00
Dan Lambright 20882507f4 sanity checks, fix knob 2024-01-02 12:09:32 -05:00
Dan Lambright 857e38b80b bug fixes/cleanup 2023-12-21 16:39:22 -05:00
Dan Lambright 05571c59a9 Set tags on apply metadata mutations 2023-12-21 13:20:21 -05:00
Jingyu Zhou 35efd7b048
Merge pull request #11104 from gm42/fix/network-goroutine-sigsegv
Go: simplify network start check logic
2023-12-20 14:40:46 -08:00
Johannes M. Scheuermann 8902b3ac52 Build the FDB Kubernetes monitor 2023-12-20 15:30:15 -06:00
Dan Lambright 2b4b4ae512 Synthesize data on SS based off parameters from new system transaction 2023-12-20 11:25:47 -05:00
Dan Lambright 5ebe8b0915 move data to value and parse it 2023-12-18 09:10:06 -05:00
gm42 6b54636fb4 Go: simplify network start check logic
This change attempts to address the SIGSEGV happening when network routine is started
multiple times concurrently.
It changes the network mutex to be a RW mutex, to optimize the case
of calls when network is already started.

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7f4600000011 pc=0x7f46a405678e]

runtime stack:
runtime.throw({0x1169773?, 0x7f46a479f96c?})
	/usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7f465affb790 sp=0x7f465affb760 pc=0x44e45d
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:821 +0x3e9 fp=0x7f465affb7f0 sp=0x7f465affb790 pc=0x466e49

goroutine 60 [syscall]:
runtime.cgocall(0xf4f640, 0xc0001f7f80)
	/usr/local/go/src/runtime/cgocall.go:157 +0x6e fp=0xc0001f7f58 sp=0xc0001f7f20 pc=0x41840e
github.com/apple/foundationdb/bindings/go/src/fdb._Cfunc_fdb_run_network()
	_cgo_gotypes.go:378 +0x85 fp=0xc0001f7f80 sp=0xc0001f7f58 pc=0xd19d25
github.com/apple/foundationdb/bindings/go/src/fdb.startNetwork.func1()
	/home/user/go/pkg/mod/github.com/apple/foundationdb/bindings/go@v0.0.0-20221026173525-97cc643cef69/src/fdb/fdb.go:209 +0x2a fp=0xc0001f7fe0 sp=0xc0001f7f80 pc=0xd2704a
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001f7fe8 sp=0xc0001f7fe0 pc=0x487741
created by github.com/apple/foundationdb/bindings/go/src/fdb.startNetwork
	/home/user/go/pkg/mod/github.com/apple/foundationdb/bindings/go@v0.0.0-20221026173525-97cc643cef69/src/fdb/fdb.go:208 +0x72

goroutine 1 [running]:
	goroutine running on other thread; stack unavailable
2023-12-14 12:49:48 +01:00
Dan Lambright a20f9d3475 Interfaces to synthesize data 2023-12-13 15:19:17 -05:00
Jingyu Zhou 3d3ad1ce80
Merge pull request #11098 from sbodagala/main
Bump up the number of keys that "fdbcli checkall" can fetch per GetKeyValueRequest call
2023-12-11 15:50:05 -08:00
Sreenath Bodagala 8d2feda708 - Print warning (that the check was incomplete) irrespective of
whether the first batch of keys fetched are consistent or not.
2023-12-11 20:07:37 +00:00
Sreenath Bodagala 4edd5ec435 - Print indexes of corrupted keys. Also, print a warning in case the
check was incomplete.
2023-12-11 19:55:10 +00:00
Sreenath Bodagala b9662794f4 - Print information about "cached" and "more" flags too. 2023-12-11 17:10:30 +00:00
Sreenath Bodagala 182ef6f199 - Bump up the number of keys that "fdbcli checkall" can fetch
per GetKeyValueRequest call.
2023-12-11 01:57:07 +00:00
Jingyu Zhou f8c0a21e87
Merge pull request #11094 from sfc-gh-mpilman/bugfixes/choose-cancellation
Fix cancellation bug in Choose class
2023-12-09 09:34:33 -08:00
Markus Pilman a2bb520847 Fix cancellation bug in Choose class 2023-12-08 17:46:21 +01:00
Jingyu Zhou 3a04f8729d
Merge pull request #11091 from sbodagala/main
Address issues in code related to "fdbcli checkall" command
2023-12-07 14:30:52 -08:00
Sreenath Bodagala 5429e3e24c Merge remote-tracking branch 'apple-upstream/main' 2023-12-07 18:30:52 +00:00
Sreenath Bodagala 5c31d1a8f5 - Address a review comment 2023-12-07 15:56:05 +00:00
He Liu 6610dda763
Fixed unassigned data move reasons. (#11089) 2023-12-06 18:42:37 -08:00
Sreenath Bodagala fe13f740e6 - Address issues in code related to "fdbcli checkall". 2023-12-07 00:30:34 +00:00
He Liu 29eab90528
Clean up dd traces (#11090)
* Clean up DD traces.

* clean up dd traces.
2023-12-06 15:53:04 -08:00