8f8f95931b
In the SQLite storage engine, destroy and create new cursors after SQLITE_CURSOR_MAX_LIFETIME_BYTES of KV read usage because cursor usage increases page cache memory usage in SQLite (internal page cache, not AsyncFileCached) by pinning pages which are not freed until the cursor is destroyed.
2022-03-07 11:20:59 -08:00
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
233c918ffb
Replace printf() and fprintf() with fmt::print().
2022-02-25 19:06:57 -08:00
f7eb66441d
Try eliminating warnings in macOS and Windows CI builds.
...
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
dd45805312
Merge branch 'apple:main' into threadname-issue-6064
2022-02-01 17:40:07 -06:00
f1bd0f16da
Run clang-format
2022-01-28 10:40:33 -08:00
0e6dbf434b
Fix use allKeys.end
2022-01-28 10:40:33 -08:00
1419d1aab7
Fix short hand
2022-01-28 10:40:33 -08:00
697d075e20
Add role 'kvfiledump' to dump key-values from storage file
2022-01-28 10:40:33 -08:00
aa41c04993
add back threadnames for debugging
2022-01-28 11:57:52 -06:00
f32fa3a2c8
format fix
2022-01-27 12:38:05 -06:00
f47f1dc2f0
Remove threadName to attempt to find cause of bug in Apple CI build
2022-01-27 12:26:49 -06:00
74bf23950f
Check if threadName exceeds 15 chars.
2022-01-26 12:02:54 -06:00
a57b4b355e
Update fdbserver/KeyValueStoreSQLite.actor.cpp
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-01-26 10:48:02 -06:00
15ca24dea8
Update fdbserver/KeyValueStoreSQLite.actor.cpp
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-01-26 10:47:50 -06:00
fd6597cc7a
Rename sqllite-write thread, add index for read threads.
...
Additionally make note these are actually coroutines and not threads.
2022-01-25 13:56:35 -06:00
fa1d8fc11e
Formatting fix.
2022-01-24 20:27:31 -06:00
59726928bb
Add sqlite reader thread name.
2022-01-24 19:45:09 -06:00
3d48e6cb5b
Add sqlite writer thread name.
2022-01-24 19:44:32 -06:00
9e06b6e6e3
Make IClosable interface const-correct
2021-10-18 13:40:47 -07:00
9713a14ef1
Reverse order of read type and debug ID args
2021-10-18 12:23:09 -04:00
df53cc9580
Add an enum to IKeyValueStore to indicate the source/priority of the read
2021-10-15 14:35:59 -04:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
353efe7db2
Merge pull request #5264 from sfc-gh-tclinkenbeard/fix-more-clang-warnings
...
Enable more warnings for `clang`
2021-07-29 15:43:54 -07:00
94a65865d9
Merge remote-tracking branch 'origin/master' into fix-clang-warnings
2021-07-28 12:29:27 -07:00
c74047c665
Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings
2021-07-28 11:51:02 -07:00
507c1f11e3
Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use.
2021-07-26 19:55:10 -07:00
3442ebd3b7
Fix more -Wreorder-ctor warnings across many files
2021-07-24 11:20:51 -07:00
e62e6503ac
Fix most delete-non-virtual-dtor clang warnings
2021-07-21 23:32:44 -07:00
fcb8bd6475
Revert "Make the sim2 run loop match the behavior of the net2 run loop."
2021-06-22 14:50:01 -07:00
2ae4f51d19
added a comment
2021-06-22 09:42:00 -07:00
86ebb0a646
throw non-file not found errors first to better handle injected faults
2021-05-20 16:46:00 -07:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
aaf0a9aa7b
Merge branch 'release-6.3' into merge-release-6.3-into-master
...
# Conflicts:
# build/docker-compose.yaml
# cmake/ConfigureCompiler.cmake
# fdbclient/FileBackupAgent.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/IAsyncFile.h
# fdbrpc/IRateControl.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
67e783acf8
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# cmake/FDBComponents.cmake
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/storageserver.actor.cpp
# flow/Knobs.h
# flow/network.h
2021-02-08 09:20:28 -08:00
8547abbbe4
Bug fix: In simulation, KVStoreSQLite can outlive its process during a simulated reboot so its file references must be cleared explicitly during shutdown instead of implicitly during destruction. Re-enabled a useful trace event. Some cleanup, removed unnecessary member var initializer and removed unnecessary method.
2021-02-05 21:26:39 -08:00
f6342376ab
Update the knob name
2021-02-02 13:31:00 -08:00
17671604c0
Add option to define I/O size
2021-02-02 12:39:05 -08:00
e4b953ab38
Merge pull request #4229 from sfc-gh-satherton/afcache-write-limit
...
AsyncFileCached RateControl support and bug fixes
2021-01-31 15:13:03 -07:00
d607ba3ac1
Removed merge artifact which calls functions not present in release-6.2.
2021-01-31 03:59:36 -08:00
15aa877388
Use VFSAsyncFile::checkInjectedError() to replace old injected fault detection, see pr#4212
2021-01-30 18:25:43 -08:00
bb6608ccd8
Another bug fix in SQLite file instance lifetimes. Writer can be destroyed before doClose() or stopOnError() have a chance to react, so Writer can't be used to get access to file handles to remove the rate limiter and trigger pending requests to complete. New logic is for Writer initialization to place opened file handles in KeyValueStoreSQLite instance and move disableRateControl() there, to be called from doClose().
2021-01-30 18:23:50 -08:00
4982787a80
Another attempt to fix SQLite lifetime issues. KeyValueStoreSQLite::doClose() can now access the Writer instance and call disableRateControl() on its SQLiteDB instance prior to stopping the reader and writer thread pools. This should prevent rate limiting from interfering with SQLite shutdown.
2021-01-30 18:18:36 -08:00
b0fd6bfbcf
More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent.
2021-01-30 18:18:16 -08:00
52c2695f59
Bug fixes involving use of AFCached rate control in simulation.
2021-01-30 18:13:44 -08:00
e56fe02a25
Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit
2021-01-30 18:11:54 -08:00
14af5857ed
Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why.
2021-01-30 18:06:05 -08:00
e72d486215
Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments.
2021-01-30 18:06:04 -08:00
Steve Atherton