648c870fec
Merge branch 'release-6.3' of https://github.com/apple/foundationdb into tls-background-handshake
2020-06-26 00:21:57 -07:00
0da4c91ad5
Performance improvements in socket use and buffering which results in TLS network connections using around 3.5x less CPU.
...
PacketBuffers are 16k minimum sizes, using FastAlloc. All calling paths to PacketBuffer creation now pass a size hint if known. P2PNetworkTest improved in many ways, errors go to TraceEvents instead of stdout, error counts are reported in stats, range min/max parameters are combined into "min:max" strings, new options to "stutter" connections by delaying randomly before socket read or write, message size to send is now configured separately for incoming vs outgoing sessions.
2020-06-25 20:44:43 -07:00
8f842a66e4
Merge pull request #3397 from xumengpanda/mengxu/fr-addPrefix-PR
...
Fast Restore: Support and test addPrefix and removePrefix feature
2020-06-25 14:06:39 -07:00
43d55fb73f
AtomicRestore:Fix typo in comments
2020-06-25 13:24:58 -07:00
af6b141736
Merge pull request #3407 from sfc-gh-tclinkenbeard/fix-vectorref-memory-leaks
...
Fix memory leaks in VectorRef
2020-06-25 00:54:33 -07:00
5860a5b4db
FastRestore:Suppress or mute spammy trace events
2020-06-24 22:10:54 -07:00
c13ba7aeb4
FastRestore:Cosmetic code style improvement
...
Also mute some trace events
2020-06-24 21:25:37 -07:00
94b1e17a96
FastRestore:Fix:Avoid reset txn backoff time in sanity check
2020-06-24 21:10:47 -07:00
d2b71e77c2
FastRestore:Fix:checkDatabaseLock tr should not reset
2020-06-24 20:21:49 -07:00
5cd3dad986
Merge pull request #3409 from sears/assert_strings
...
Print out the text (to stderr and as trace) of the failed condition o…
2020-06-24 14:58:12 -07:00
a0edd1d4eb
FastRestore:Disable addPrefix test
2020-06-24 14:26:33 -07:00
422003df34
Print out the text (to stderr and as trace) of the failed condition on assert failure
2020-06-24 14:21:48 -07:00
3f6222a04d
Mark DebugEntryRef trivially destructible
2020-06-24 14:08:22 -07:00
8de70432a6
Fix memory leaks in VectorRef
...
VectorRef<T> relies on T being trivially destructible, but this was not
yet enforced. By statically asserting that T is trivially destructible,
we avoid leaking memory by not calling the destructor for non-trivially
destructible types
2020-06-24 10:54:19 -07:00
b69c36f0d6
transformDatabaseContents:cosmetic change
2020-06-23 19:27:58 -07:00
198696bc1e
Move transformRestoredDatabase from server to client
...
AtomicRestore workload turns out to rely on the FileBackupAgent
client. Keeping transformRestoredDatabase in server makes linking harder.
2020-06-23 15:48:43 -07:00
0000d81595
Merge pull request #3377 from yliucode/fix-recent-requests
...
Fix GetReadVersionReply.recentRequests to reflect a proxy's number of recent requests in the commit stats.
2020-06-23 15:05:57 -07:00
6eb8dc58b6
FastRestore:addPrefixTest:Fix calling transformRestoredDatabase
2020-06-23 15:02:38 -07:00
84a640e4ee
AtomicRestore:Add addPrefix test for fast restore
2020-06-23 14:52:27 -07:00
4e27fd34e5
Refactor transformDatabaseContents into RestoreCommon
...
Prepare to enable addPrefix for atomicRestore
2020-06-23 14:33:13 -07:00
f6954ef038
FastRestore:Test both addPrefix and not add it
2020-06-23 13:09:02 -07:00
b66b8d3c74
FastRestoreTest:run writeKVs concurrently
2020-06-23 12:50:36 -07:00
142fa5477b
Add SevFRTestInfo trace level to control debug info in BackupAndParallelRestoreCorrectness
2020-06-23 10:49:05 -07:00
d0c0d06b1e
FastRestore:Avoid calling withPrefix when addPrefix is not set
...
It help improve performance when addPrefix is not set.
2020-06-23 10:43:35 -07:00
10e043da9d
Format sim2 code a bit
2020-06-23 10:07:39 -07:00
90861ead6a
Reenable all tests in BackupAndParallelRestoreWithAtomicOp
2020-06-23 08:29:31 -07:00
c20d0120ac
BackupAndParallelRestoreCorrectness:Avoid false positive in SanityCheckData
...
addPrefix can happen to be the prefix of valid backup data. So we cannot
enforce the final data in DB does not have addPrefix as prefix
2020-06-23 07:40:43 -07:00
813dbfb297
TLS handshake threads are now created with TLS initialization instead of when the network is created since they may not be needed and it avoids calling startThread() with a custom stack size too early in process execution. StartThread() now logs a warning if the given stack size can't be used. Reorganized TLS handshake thread state near the other TLS related stuff.
2020-06-23 03:09:56 -07:00
c3ce0034bf
Background threads for TLS handshakes use a stack size based on a knob. Platform::startThread() now accepts a stack size. The generic thread pool implementation takes an optional stack size override which is used for each added thread.
2020-06-23 02:08:01 -07:00
5b19233989
Implemented count of used background TLS handshake threads differently using a generic utility class that updates the count on destruction.
2020-06-23 00:45:07 -07:00
d9610fe745
BackupAndParallelRestoreCorrectness:Fix memory leak that cause nondeterminism error
2020-06-23 00:13:48 -07:00
44af9273cd
BackupAndParallelRestoreCorrectnessWorkload:Change unnecessary Key to KeyRef
2020-06-22 23:39:46 -07:00
9cff90fb5f
runRYWTransaction:Add comment to remind func should be idempodent
2020-06-22 23:04:09 -07:00
c4ac68ef1e
FastRestore:CorrectnessTest:Fix writeKVs
...
runRYWTransaction() should not modify states whose lifetime is outside of the funtion.
Otherwise, the retry in runRYWTransaction() will be wrong.
2020-06-22 23:00:57 -07:00
dd5c1ec416
FastRestore:Add debug trace
...
The bug behavior is:
In transformDatabaseContents(), we write all new kvs into the DB in 100 keys batch.
We immediately read these keys out.
There are 100 keys missing.
2020-06-22 19:47:21 -07:00
b122575dda
BackupAndParallelRestoreWithAtomicOp:Turn on debug options
2020-06-22 16:53:43 -07:00
4f29c3ba59
Updated release notes for TLS background handshakes.
2020-06-22 16:39:19 -07:00
3853860cf1
Changed default TLS handshake thread count.
2020-06-22 16:28:30 -07:00
d36f64bd09
Track session completions instead of session starts.
2020-06-22 15:40:04 -07:00
a48d005f61
Bug fixes, stats improvements.
2020-06-22 13:37:13 -07:00
f16381c7e8
FastRestore:Clean backup ranges
2020-06-22 11:43:24 -07:00
d75b4c9a69
Bug fix, accidental local variable redeclaration.
2020-06-22 05:53:37 -07:00
81b8dc30f5
Network test bug fixes. TLS still not working for unknown reason, seems to be related to initialization of something.
2020-06-22 05:15:24 -07:00
57344922a4
Added a network connection test and very hacky interface to it. The test simultaneously listens on a set of addresses, connects to a random set of addresses to maintain a configured number of connections, and for each session executes a two way data exchange followed by a random delay and close.
2020-06-22 03:37:16 -07:00
3d6f69c8e2
FastRestore:addPrefix:Transform must clear both orignal and transformed range
...
Otherwise, anything left in the range can interfer with the result.
2020-06-21 22:18:12 -07:00
d681e9d350
Follow TLS_DISABLED macro setting.
2020-06-21 11:21:24 -07:00
6330f8b458
TLS handshakes can now be done using up to a configured number of background threads if any of them are not busy.
2020-06-21 06:23:32 -07:00
9318ec033d
FastRestore:FixTest:Remove unnecessary read
2020-06-20 20:07:18 -07:00
11af1a6916
FastRestore:Transform only backup ranges when addPrefix and removePrefix exist
2020-06-20 19:21:16 -07:00
faec3cc556
FastRestore:Fix attempt:Seg fault
2020-06-19 16:07:09 -07:00
Steve Atherton