Commit Graph

27397 Commits

Author SHA1 Message Date
Vishesh Yadav a84026cca5
[testing] Automatically discover unit-test and register as ctest (#11612)
* [testing] Automatically discover unit-test and register as ctest

This patch adds `collect_unit_tests()` to CMake which searches over
the codebase and finds all the unit-tests written using Flow's TEST_CASE
macro and register as ctest.

The test then can be then run using ctest command or directly via Test
Explorer in VSCode.

* Update CMakeLists.txt

* Check failed tests

* Update TestDirectory.py to create more unique directory

* Put the feature behind flag
2024-09-04 11:31:49 -07:00
Yao Xiao f700a4a2cf
Convert ASSERT to error logs. (#11625) 2024-09-04 00:23:36 -07:00
Sreenath Bodagala daca092939
- Address a bug related to computing the ids of log servers (#11623) 2024-09-03 21:15:08 -07:00
Dan Lambright 4e96c3cab2
refactor to use struct rather than tuple for "uncommitedVersions" (#11619)
* refactor to use struct rather than tuple for uncommitedVersions

* Document UnknownCommittedVersions struct

---------

Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-08-30 14:28:14 -07:00
Sreenath Bodagala e490ba8c21
Address an assertion failure (#11617) 2024-08-29 11:22:30 -07:00
Syed Paymaan Raza 48064f6cf1
Make some codeprobes rare (#11607)
* Make BlobGranule code probes rare

* Make encryption related code probes rare

* fixup! Fix formatting
2024-08-26 22:33:38 -07:00
Jingyu Zhou 9381d35e62
Merge pull request #11606 from hfu94/fix1
Fix overflow
2024-08-26 16:52:46 -07:00
Jingyu Zhou 9a914d9400
Merge pull request #11600 from jzhou77/coroutine
Refactor LogRouter and Watch workload using coroutines
2024-08-26 16:52:23 -07:00
Dan Lambright bf292a6c95
Stash correct id for tlog idenfication in recovery (#11604)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-08-26 15:39:40 -04:00
hao fu 43b9f89d24 Fix overflow
uint64_t should be used instead of int
2024-08-26 12:13:37 -07:00
Jingyu Zhou 8475ad87bb
Ignore FileNotFoundError when tearing down a temp cluster (#11594)
* Ignore FileNotFoundError when tearing down a temp cluster

ctest can fail with errors like:

[2024-08-22 20:37:15,780] - fdbcli_tests.py:231 - DEBUG - killall - Old generation: 8, New generation: 10
log-dir: /codebuild/output/src3881629096/src/github.com/apple/foundationdb/build_output/tmp/ZIpl2Hm2zri6nlpu/log
etc-dir: /codebuild/output/src3881629096/src/github.com/apple/foundationdb/build_output/tmp/ZIpl2Hm2zri6nlpu/etc
data-dir: /codebuild/output/src3881629096/src/github.com/apple/foundationdb/build_output/tmp/ZIpl2Hm2zri6nlpu/data
cluster-file: /codebuild/output/src3881629096/src/github.com/apple/foundationdb/build_output/tmp/ZIpl2Hm2zri6nlpu/etc/fdb.cluster
Traceback (most recent call last):
  File "/codebuild/output/src3881629096/src/github.com/apple/foundationdb/tests/TestRunner/tmp_cluster.py", line 146, in <module>
    print(f.read())
  File "/codebuild/output/src3881629096/src/github.com/apple/foundationdb/tests/TestRunner/tmp_cluster.py", line 45, in __exit__
    shutil.rmtree(self.tmp_dir)
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/shutil.py", line 718, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/shutil.py", line 655, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/shutil.py", line 675, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/shutil.py", line 673, in _rmtree_safe_fd
    os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'fdbmonitor.lock'

* Explicitly ignore FileNotFoundError
2024-08-26 11:20:38 -07:00
Jingyu Zhou 1afb2d42da Fix crash due to invalid ReplyPromise 2024-08-23 17:41:39 -07:00
Jingyu Zhou fac874595f Fix logRouterPop 2024-08-23 16:46:01 -07:00
Jingyu Zhou 707e413d1e Log router refactor 2024-08-23 14:44:25 -07:00
Jingyu Zhou 8d3cb1eab5 Convert the last actor in LogRouter 2024-08-23 12:11:57 -07:00
Jingyu Zhou 4354c88829 Convert LogRouter actors into coroutines 2024-08-23 12:11:57 -07:00
Jingyu Zhou cf188a99d4 Convert most actors in Watches workload into coroutines
There is one left that doesn't seem to have a good way for conversion. To make
sure the converted code is behaving correctly, I added a few CodeProbes to
ensure code coverage.
2024-08-23 12:11:57 -07:00
Dan Lambright d040b9a24c
check existence of optional interface (#11593)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-08-22 23:28:30 -04:00
Dan Lambright 94fba2cce0
cluster level RV when tLogs advance at different rates (#11557)
* simulate more than one tlog

* Draft use cluster RV for tlogs in version vector

* add TestTLogRecovery2

* Respond to review comments

* Add assert

* Send clusterRV to all locked tlogs

* Fix typo on rebase

* add memory managed IdToInterf structure

---------

Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-08-22 16:17:02 -04:00
Syed Paymaan Raza bf59b5434f
[tutorial] Simplify connect() logic (#11589) 2024-08-20 22:44:17 -07:00
John Brownlee 9c4ab5e879
Fix references to the wrong DB in the backup actor initialization. (#11588)
* Fix references to the wrong DB in the backup actor initialization.

* Fix variable name.

* Apply clang-format updates.
2024-08-20 20:44:34 -07:00
Sreenath Bodagala 7efbf9f041
Recovery version computation when version vector unicast is enabled (#11565)
* - Recovery version computation when version vector unicast is enabled

* - Address a review comment

* - Modify code to not use "max(DV)" as the starting recovery version

* - Remove references to "max(DV)"

* - Address a review comment

* - Address PR review comments

* - Address a review comment
2024-08-20 13:58:18 -04:00
Zhe Wang 3305d2e3ee
fix storage engine selection (#11586) 2024-08-20 09:24:30 -07:00
Jingyu Zhou a1aa0e5b53
Merge pull request #11572 from apple/test-update
Modify tlog unit tests to work with multiple tags
2024-08-19 11:06:24 -07:00
Dan Lambright 1c3be55977 Respond to review comments 2024-08-19 09:56:21 -04:00
Jingyu Zhou 5d5f1a2dc7
Merge pull request #11575 from brownleej/backup-transaction-options
Capture default database options in fdbbackup in a local variable.
2024-08-16 14:16:38 -07:00
Jingyu Zhou cd31a8918a
Merge pull request #11582 from xis19/main
Add a new flag of USE_CUSTOM_JEMALLOC
2024-08-16 13:41:11 -07:00
Jingyu Zhou 8f3594caec
Merge pull request #11581 from jzhou77/release-notes
Add 7.3.44-7.3.49 release notes
2024-08-16 13:10:06 -07:00
Zhe Wang 6c502e9707
Solve RocksDB external timeout error and re-enable RocksDB simulation tests (#11577)
* init knob tune

* include rocksdb in tests

* probably reuse rocksdb iterator in simulation

* clear unnecessary knob change
2024-08-16 12:37:18 -07:00
John Brownlee cd2962f10c Rename fields in new trace events to match formatting standards. 2024-08-16 11:44:25 -07:00
Xiaoge Su 4c94acbd53 fixup! 2024-08-16 09:46:06 -07:00
Xiaoge Su 8197348f95 Add flag for custom jemalloc build 2024-08-16 09:45:55 -07:00
Jingyu Zhou 39d895d700 Add 7.3.44-7.3.49 release notes 2024-08-16 09:41:45 -07:00
John Brownlee 860963ba05 Reformat changes. 2024-08-15 12:31:17 -07:00
John Brownlee cd4eb794b5 Add debug logging to help validate the transaction options set in fdbbackup. 2024-08-15 12:01:56 -07:00
John Brownlee 1becf341dd Capture default database options in fdbbackup in a local variable. 2024-08-14 14:55:17 -07:00
Jingyu Zhou fc4ea65c52
Merge pull request #11573 from jzhou77/fix
Fix a cluster controller crash in status server
2024-08-14 13:05:59 -07:00
Jingyu Zhou 95adc14806 Fix a cluster controller crash in status server
The logSystemConfig could be changed after the yield(), thus the iterator can
points to invalid memory.

To reproduce with gcc build at commit 3e90644779:
   -f tests/slow/DDBalanceAndRemoveStatus.toml -s 3134372275 -b off
2024-08-13 20:55:26 -07:00
Jingyu Zhou e1781b5234
Merge pull request #11571 from jzhou77/fix 2024-08-13 19:53:40 -07:00
Syed Paymaan Raza a12b8a7ffc
[fdbcli] Add exclude in progress signal (#11569) 2024-08-13 15:37:01 -07:00
Dan Lambright 5339d9c99f Modify test to work with multiple tags 2024-08-13 17:20:40 -04:00
Jingyu Zhou d0b6430a69 Add more trace events for exclude command
Use PRIORITY_SYSTEM_IMMEDIATE for excludeServersAndLocalities() call.
2024-08-12 18:19:57 -07:00
Jingyu Zhou 3e90644779
Merge pull request #11568 from sepeth/Fix_errors_in_tlog-spilling.md.html
Fix some minor errors in design/tlog-spilling.md.html
2024-08-12 16:05:17 -07:00
Doğan Çeçen 1e5fd76d64 Fix some minor errors in design/tlog-spilling.md.html
And, add two related metrics to the empty metrics section for Spilling.
2024-08-12 23:19:19 +03:00
Zhe Wang 0bfaa7b608
fix uninitialized bulkloadmode var (#11567) 2024-08-09 16:25:17 -07:00
Sepeth f84a3987a7
Fix fdbmonitor build issues on macOS (#11561)
Without this change, the build fails with multiple identifiers not found, such
as `gettimeofday`, `sigprocmask`, `kqueue` and some more related ones.
`watch_conf_dir` and  `watch_conf_file` were also missing in the
`fdbmonitor.h`.
2024-08-09 15:23:48 -07:00
Syed Paymaan Raza bc17127e36
[fdbmonitor] Ctests for envvars key-value extraction (#11562) 2024-08-07 22:05:14 -07:00
Syed Paymaan Raza 46e985acf3
[fdbmonitor] delete-envvars bug fixes (#11559) 2024-08-07 10:48:55 -07:00
Syed Paymaan Raza ae39e18b7d
Allow exporting rocksdb compile commands (#11560) 2024-08-06 12:46:39 -07:00
Jingyu Zhou bd2e108531
Merge pull request #11555 from jzhou77/fix
Reduce chance of running rare tests
2024-08-06 10:13:03 -07:00