Commit Graph

25680 Commits

Author SHA1 Message Date
Markus Pilman b5b9cec974 Adding additional tests 2023-03-09 17:25:05 -07:00
Jingyu Zhou b755e668bf
Merge pull request #9601 from jzhou77/fix-head
Allow log router to detect slow peeks and to switch DC for peeking
2023-03-09 15:34:24 -08:00
Yanqin Jin effda73ef4 Rename a variable 2023-03-09 14:42:18 -08:00
Jingyu Zhou eb4e122787 Reduce running time for DcLag
The switch can happen quicker than the workload detection time, so need to
adjust detection time lower than LOG_ROUTER_PEEK_SWITCH_DC_TIME.
2023-03-09 14:34:23 -08:00
Yanqin Jin 2feb60cd63
Update fdbserver/workloads/FastTriggeredWatches.actor.cpp
Change `StringRef()` to `""_sr`.

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2023-03-09 14:28:04 -08:00
Jingyu Zhou abe6b40fc9 Address comments 2023-03-09 13:58:37 -08:00
Yanqin Jin 8e478b7dcc Merge remote-tracking branch 'origin/main' into deflake-test-1 2023-03-09 13:29:28 -08:00
Yanqin Jin 88d9c1f610 address comments 2023-03-09 13:29:07 -08:00
Hui Liu 6fca5b4e13 Fix asan error caused by StringRef parameter of updateRestoreState 2023-03-09 13:04:45 -08:00
Markus Pilman fbc79c001c Make process events reentrant save 2023-03-09 13:55:54 -07:00
Yanqin Jin 4822af9417 Revert "Deflake test FastTriggeredWatches"
This reverts commit c2939ba51e.
2023-03-09 12:55:48 -08:00
Markus Pilman 4b90a01f0b First successful negative run 2023-03-09 13:05:00 -07:00
sfc-gh-tclinkenbeard 1c7076c9a4 Fix argument type in ProxyCommitData::updateSSTagCost 2023-03-09 11:01:13 -08:00
Jingyu Zhou 5c97fb2c20 Use a constant for connectionFailuresDisableDuration 2023-03-09 09:50:24 -08:00
Jingyu Zhou e18ed14278 Refactor to address comments 2023-03-09 09:39:27 -08:00
Josh Slocum 0c718f7a04
Fixing double-completing of destination servers in busy map (#9629) 2023-03-09 09:43:45 -06:00
Josh Slocum dd032d7a16
fixing bytesInNewDeltaFiles calculation when a snapshot file is rolled back (#9609) 2023-03-09 09:43:27 -06:00
Ata E Husain Bohra b227007ab0
EaR: Fix knob name (#9630)
Description

Knob 'REST_KMS_ALLOW_NOT_SECURE_CONNECTION' got renamed in recent
patch, however, there are other places that needs an update too.

Testing

devRunCorrectness - 100K
RESTUtilUnits.toml
RESTKmsConnectorUnits.toml
2023-03-08 17:37:39 -08:00
Nim Wijetunga 2702665e35
Refactor GetEncryptCipherKeys (#9600)
* inital commit

* address pr comments
2023-03-08 17:05:03 -08:00
Evan Tschannen 4a17ed363a Fix: the consistency check did not properly report failed tests 2023-03-08 16:56:23 -08:00
Markus Pilman 3dc9ae18f5 fix compiler error 2023-03-08 17:15:53 -07:00
Markus Pilman fb350edb44 Add framework for writing negative simulation tests 2023-03-08 17:09:50 -07:00
Jingyu Zhou 38c1e3f603 Revert disableSimSpeedup 2023-03-08 15:54:21 -08:00
Jingyu Zhou 493e81f31d Limit connection failures to be within tests
In particular, disable connection failures when initializing the database
during the startup phase, i.e., before running with test specs.
2023-03-08 15:36:58 -08:00
Nim Wijetunga 218ed4519f
Strengthen Snapshot Backup/Restore Asserts (#9552)
strengthen backup/restore asserts for encryption
2023-03-08 15:24:02 -08:00
Jingyu Zhou 9913f5b5e1 Simplify DcLag code 2023-03-08 15:23:34 -08:00
Yanqin Jin c2939ba51e Deflake test FastTriggeredWatches
In FastTriggeredWatchesWorkload, if the randomized new value for the given
`setKey` happens to be the same as the current value, the following will hold

- `first` is true, and
- `getDuration` is 0, and
- assertion `lastReadVersion - ver >= SERVER_KNOBS->MAX_VERSIONS_IN_FLIGHT || lastReadVersion - ver < SERVER_KNOBS->VERSIONS_PER_SECOND * (25 + getDuration)` will fail

To fix this, change the assertion to
```
assert(first || lastReadVersion - ver >= SERVER_KNOBS->MAX_VERSIONS_IN_FLIGHT ||
				       lastReadVersion - ver < SERVER_KNOBS->VERSIONS_PER_SECOND * (25 + getDuration));

```

Test plan:
Apply to the fix on top of the commit reported by Joshua test. Rerun the command to make sure it passes.
2023-03-08 14:07:57 -08:00
Xiaoge Su 4373f111fb Let FDB uses Findjemalloc.cmake 2023-03-08 13:09:13 -08:00
Xiaoge Su 873b800ba8 Add Findjemalloc.cmake 2023-03-08 13:09:13 -08:00
Jingyu Zhou c8bfb40d3c
Merge pull request #9625 from sfc-gh-ahusain/ahusain-fix-test-pattern
Remove Test pattern from RandomUnit Test
2023-03-08 10:58:51 -08:00
Jingyu Zhou aab19c65ea
Merge pull request #9623 from jzhou77/clog-fix
Reduce running time for ClogTlog
2023-03-08 10:46:57 -08:00
Ata E Husain Bohra 1c997acfd2 Remove Test pattern from RandomUnit Test
Description

Remove Test pattern from RandomUnit Test

Testing
2023-03-08 10:00:14 -08:00
Ata E Husain Bohra d0eec9d0ba
EaR: REST KMS fixes - encryption integration testing (#9598)
* EaR: REST KMS fixes - encryption integration testing

Description

Major changes:
1. Multiple fixes observed while performing integration end-to-end
testing for Encryption at-rest feature.
2. Improve REST module logging. Introduced FLOW_KNOBS->REST_LOG_LEVEL
to have more granular control of feature logging disconnected from
the cluster log level.

Testing

Integration testbed:
1. Run fdbserver standalone
2. Run external KMS http-server to serve encryption key fetch requests
2023-03-08 09:49:43 -08:00
Jingyu Zhou 66b0db399c Reduce running time for ClogTlog
When the ClogTlog is running, we may already pass the 450s, i.e., SIM_SPEEDUP_AFTER_SECONDS,
and clogging is no longer effective. If that's the case, we want to finish the test quickly.
2023-03-08 09:20:33 -08:00
Markus Pilman 6937524594
Merge pull request #9599
Allow code to act on test timeouts
2023-03-08 09:26:55 -07:00
Hui Liu c43f8b3fdc
Refactor - introduce BlobRestoreController for APIs to manage restore state (#9616) 2023-03-08 07:50:30 -08:00
Johannes M. Scheuermann c6eca3f398 Format code 2023-03-08 08:33:19 +01:00
Johannes M. Scheuermann 1550f3c596 Make use of precomputed exclude check 2023-03-08 08:19:42 +01:00
Johannes M. Scheuermann bae627f016 Fix syntax 2023-03-08 08:19:42 +01:00
Johannes M. Scheuermann db8c60c80f Don't block the exclusion of stateless processes by the free capacity check 2023-03-08 08:19:41 +01:00
Xiaoxi Wang a92669f232
Merge pull request #9571 from sfc-gh-xwang/fix/main/sampleCopy
IndexSet and TransientStorageMetricSample optimization - change parameter type from KeyRef to Key to avoid extra copy in sampler
2023-03-07 20:50:31 -08:00
Jingyu Zhou 2b73a0c5c1
Fix ClogTlog valgrind error (#9588)
* Fix ClogTlog valgrind error

addr is the one we want to keep. gcc build seems to push_back a copy of it into
the vector.

* Use changeConfig that takes a string

* Remove an unused variable
2023-03-07 20:30:58 -08:00
mavenraven 6c1a0ada3d Bumps minimum cmake required version. 2023-03-07 14:24:28 -08:00
A.J. Beamon 605fdacd64 Fix misplaced link in fdbcli documentation 2023-03-07 11:48:21 -08:00
A.J. Beamon fa1f20a9ff Update the help text for some fdbcli tenant commands 2023-03-07 11:47:21 -08:00
A.J. Beamon de5f2c0fee Disallow cluster names that start with the `\xff` byte 2023-03-07 11:46:34 -08:00
Andrew Hayworth 04cbcdee6b Fix build for ruby bindings
The ruby bindings are not currently installable. We can reproduce this
with a build from source:

```
foundationdb/tmp on  main [$?] via △ v3.25.2
zsh ❯ gem install ./bindings/ruby/fdb-7.3.0.gem
ERROR:  While executing gem ... (Gem::Package::PathError)
    installing into parent path /Users/andrew/projects/foundationdb/LICENSE of /Users/andrew/.rbenv/versions/3.2.1/lib/ruby/gems/3.2.0/gems/fdb-7.3.0 is not allowed
```

The problem is that the gemspec is interpolating the source directory
into the `files` array - and while this allows the gem to build, it
prevents it from actually being installed.

To fix it, we borrow similar code from the python bindings to copy the
necessary files and license into the build directory before building the
gem. This allows the gem to be installed:

```
foundationdb/tmp on  main [!?] via △ v3.25.2
zsh ❯ gem install ./bindings/ruby/fdb-7.3.0.gem
Successfully installed fdb-7.3.0
Parsing documentation for fdb-7.3.0
Installing ri documentation for fdb-7.3.0
Done installing documentation for fdb after 0 seconds
1 gem installed
```
2023-03-07 10:57:42 -08:00
Andrew Hayworth 55d12fe1fc Allow ruby bindings on arm64
While fixing the gem build, I noticed that the gem will not run on arm64
machines, like the new spiffy M1/M2 MacBooks. On a whim, I tried just
removing that restriction... and it does seem to work:

```
foundationdb/bindings/ruby/lib on  ahayworth/fix-ruby-binding-build [?] via 💎 v3.2.1 took 2s
zsh ❯ pry
[1] pry(main)> require 'fdb'
=> true
[2] pry(main)> FDB.api_version 720
LoadError: FoundationDB API only supported on x86_64 (not arm64)
from /Users/andrew/.rbenv/versions/3.2.1/lib/ruby/gems/3.2.0/gems/fdb-7.3.0/lib/fdbimpl.rb:40:in `<module:FDBC>'
[3] pry(main)>
```
2023-03-07 10:53:37 -08:00
Markus Pilman 505642c038 Added test 2023-03-07 11:32:31 -07:00
Jingyu Zhou 684464cebb
Merge pull request #9594 from sfc-gh-jfu/fix-fdbcli-segfault
Fix fdbcli segfault
2023-03-07 08:28:14 -08:00