5403a29ecb
add WolfSSL support ( #6682 )
...
remove extraneous include
2022-04-28 16:53:38 -07:00
195a196392
crc32 support added for ppc64le
2022-04-28 08:35:42 -07:00
bf956f5630
Merge remote-tracking branch 'origin/main' into features/private-request-streams
2022-04-07 13:29:27 -06:00
465ff712b6
Move Hostname to its own files. ( #6759 )
...
* Change DNS cache to use std::map.
Revert commit 90c259d84e
2022-04-04 19:04:51 -07:00
017709aec6
Introduce BlobCipher interface and cipher caching interface ( #6391 )
...
* Introduce BlobCipher interface and cipher caching interface
diff-3: Update the code to avoid deriving encryption key periodically.
Implement EncyrptBuf interface to limit memcpys.
Improve both unit test and simulation to better code coverage.
diff-2: Add specific error code for OpenSSL AES call failures
diff-1: Update encryption scheme to AES-256-CTR. Minor
updates to Header to capture more information.
Major changes proposed are:
1. Introduce encyrption header format.
2. Introduce a BlobCipher cipher key representation encoding
following information: baseCipher details, derived encryption cipher
details, creationTime and random salt.
3. Introduce interface to support block cipher encrytion and decrytion
operations. Encyrption populates encryption header allowing client to
persist them on-disk, this header is then read allowing decryption
on reads.
4. Introduce interface to allow in-memory caching of cipher keys. The
cache allowing mapping of "encryption domain" -> "base cipher id" ->
"derived cipher keys" (3D hash map). This cache interface will be used
by FDB processes participating in encryption to cache recently used
ciphers (performance optimization).
Testing:
1. Unit test to validate caching interface.
2. Update EncryptionOps simulation test to validate block cipher
operations.
2022-03-24 07:31:49 -07:00
117ee637db
Merge remote-tracking branch 'origin/main' into features/private-request-streams
2022-03-15 17:17:47 +01:00
9e81945301
Enable encryption for Windows
2022-03-11 10:06:07 +01:00
dc973fb67e
Allow List and first test
2022-02-22 11:15:16 +01:00
ff1cb58174
Convert hyphens to underscores for all prefix-based arguments (e.g. --knob-, --locality-)
2021-12-14 12:01:44 -08:00
b1633b90f1
Added fmt to flow
2021-11-16 12:03:49 -07:00
05853b97c5
Merge pull request #4920 from apple/anoyes/aarch64-memcmp
...
Use custom mem{cmp,cpy} impl on Arm
2021-08-26 14:30:05 -07:00
c1bb977a93
Set sampling dependencies for Windows only
2021-08-24 10:48:31 +02:00
2c5510011c
Add dependencies between sampling and non-sampling actors projects
2021-08-24 10:48:31 +02:00
59d535149e
Merge branch 'master' into fixes/alp6
2021-07-27 10:07:18 -07:00
e9a1679467
Disable sampling everywhere except fdbserver
2021-07-27 09:53:23 -07:00
ad03a4787a
Fix non-TLS build
2021-07-09 21:06:15 -07:00
020371a78f
Merge remote-tracking branch 'origin/master' into add-boolean-param
2021-07-07 16:50:51 -07:00
f5ade03538
Merge pull request #4233 from sfc-gh-tclinkenbeard/encrypt-backup-files
...
Added AsyncFileEncrypted
2021-07-07 13:28:28 -07:00
79ff07a071
Added *BOOLEAN_PARAM macros to enforce documentation of boolean parameters
2021-07-02 15:04:42 -07:00
f58c7fd8bc
Fix IThreadPool thread naming
2021-06-29 16:27:37 -04:00
81b8292094
Merge remote-tracking branch 'origin' into encrypt-backup-files
2021-06-25 12:21:12 -07:00
9a24a57007
refactor templates to make them more readable and reusable
2021-06-16 10:46:21 -06:00
16e05fad9d
Add custom memcpy for aarch64
2021-06-16 14:35:04 +00:00
b3d83e1b2e
Use custom memcmp impl on Arm
2021-06-16 14:35:04 +00:00
153de33f57
Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
...
This reverts commit 6499fa178e1512631957
2021-06-04 13:31:55 -07:00
6499fa178e
Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage
...
Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-l…
2021-05-11 18:50:42 -07:00
4ea760b2a9
Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
...
This reverts commit da41534618e6300905d6
2021-05-10 20:26:12 -07:00
7da5004690
Merge branch 'master' of github.com:apple/foundationdb into refactor-fdbcli
2021-05-06 15:44:26 -07:00
e5d6c5ed17
Merge remote-tracking branch 'origin/master' into encrypt-backup-files
2021-05-03 14:46:19 -07:00
4fdfca0dc0
Update the thread-safe Future-to-ThreadFuture method, added two TSAN unit tests
2021-04-13 13:21:04 -07:00
a0c49234b2
Merge remote-tracking branch 'origin/master' into encrypt-backup-files
2021-03-19 20:47:53 -07:00
5c1b674815
implemented test
2021-03-19 10:31:58 -06:00
9812a49058
use consume_all to clean up after copy
2021-03-17 15:40:19 -06:00
534fa45291
Merge branch 'master' of github.com:apple/foundationdb into features/coroutine2
2021-02-01 16:31:16 -07:00
301e25334d
fixed asan build
2021-01-28 09:01:40 -07:00
7c64e3943b
compile boost for sanitizers
2021-01-27 19:13:02 -07:00
511ec43e62
Merge remote-tracking branch 'origin/master' into encrypt-backup-files
2021-01-27 11:17:58 -08:00
1612c44988
Disable StreamCipher for Windows
2021-01-25 12:36:35 -08:00
037279c843
Disable StreamCipher if TLS is disabled
2021-01-25 10:42:14 -08:00
88bc157bd0
Added AsyncFileEncrypted
2021-01-24 22:53:04 -08:00
1615977695
Added StreamCipher
2021-01-24 22:53:01 -08:00
51cbc494f1
Merge remote-tracking branch 'origin/master' into run-minio-joshua
2021-01-23 11:31:19 -08:00
1961c3fc80
Serialize spans to msgpack and send via UDP
2021-01-20 13:57:35 -08:00
61a29ecfc8
Merge remote-tracking branch 'origin/master' into run-minio-joshua
2021-01-01 09:38:36 -04:00
913401aaee
Move SendBufferIterator out of network.h
...
This class is only needed for implementation in Net2.actor.cpp and
SimExternalConnection.actor.cpp, so this class should not be included
everywhere network.h is included
2020-12-26 16:30:00 -04:00
128c758edc
Set special sanitizer annotations in stacktrace.amalgamation.cpp
...
That's the only place they're needed. This fixes at least the USE_TSAN
build and possibly other sanitizer builds
2020-12-11 02:08:02 +00:00
dae8ea24ad
Move compiler definitions into config file
2020-11-25 15:06:59 -07:00
231a7a10d0
Merge pull request #4075 from sfc-gh-tclinkenbeard/sqlite-xxhash3-checksum
...
Sqlite xxhash3 checksum
2020-11-23 10:43:35 -08:00
156a617ed8
Replace xxhash64 with xxhash3
...
The goal here is to improve performance
2020-11-21 18:25:35 -08:00
bcda617b80
Benchmark hashing algorithms in flowbench
2020-11-17 23:14:55 -08:00
d64cf8b9e3
Merge branch 6.3 into master
2020-11-17 11:22:45 -08:00
d0391db862
Merge branch 'release-6.2' into release-6.3
2020-11-16 10:15:23 -08:00
32c87bbb33
Lightweight, power of two spaced histogram implementation + automatic reporting
2020-11-02 11:13:16 -08:00
ef8c1060a2
Merge branch 'master' into mengxu/tmp-merge-6.3
2020-07-13 10:15:56 -07:00
b09dddc07e
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/ConfigureCompiler.cmake
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/fdbrpc.vcxproj
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogSystemPeekCursor.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/Status.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/flow.vcxproj
2020-07-10 15:06:34 -07:00
04d1217941
Track statistics about server-side request latency on each process, to include min, max, mean, and various percentiles.
2020-07-09 16:39:15 -07:00
cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression
2020-07-09 17:04:04 -06:00
c94f46440b
Merge pull request #3094 from sfc-gh-dyoungworth/dyoungworth/asyncFindFiles
...
Make platform::findFiles asynchronous
2020-07-08 11:47:09 -07:00
0fbe7101c3
Revert "Revert "Request tracing""
...
This reverts commit 327cc31e35
2020-07-07 10:06:13 -06:00
327cc31e35
Revert "Request tracing"
2020-06-16 12:32:42 -07:00
80334351c7
Merge pull request #3329 from sfc-gh-mpilman/features/flatbuffers-debugtx
...
Request tracing
2020-06-12 14:50:19 -07:00
87a53be22f
silence macos x linker warning
2020-06-11 13:30:29 -07:00
eb8a40b079
Put trace interfaces in separate file
2020-06-10 16:38:25 -07:00
4ab3441a95
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-06-10 15:31:29 -07:00
09c136e434
Framework for transaction tracing
2020-06-08 16:09:37 -07:00
b84fcbc828
folly_memcpy is ready for benchmarking
2020-06-02 14:51:21 -07:00
678b57c0d9
port rte_memcpy to flow; add -mavx compiler flag
2020-06-02 14:51:21 -07:00
c12c11acd3
Merge branch 'master' into dyoungworth/asyncFindFiles
2020-05-27 17:51:08 -07:00
f22a8845e4
Finish putting actors in Platform.actor.cpp
2020-05-20 18:14:29 -07:00
d748166ed3
Merge branch 'master' of github.com:apple/foundationdb into features/flatbuffers-debugtx
2020-05-18 10:15:58 -07:00
ae2d9744cb
simple benchmarks of IndexedSet, VersionedMap and std::map
2020-05-07 12:08:00 -07:00
5877994f53
Make platform::findFiles asynchronous
2020-05-06 10:40:08 -07:00
8125096e9b
Revert "reduce the number of calls to operator< made by lower_bound and upper…"
2020-05-05 08:17:10 -07:00
4ae356b237
Merge pull request #2882 from sears/lb_ub_perf
...
reduce the number of calls to operator< made by lower_bound and upper…
2020-05-04 15:46:22 -07:00
362eb41509
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-05-04 14:17:26 -07:00
d039e75ac4
Merge remote-tracking branch 'origin/master' into features/flatbuffers-debugtx
2020-05-04 14:04:52 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
95bc24de11
flow: update headers and includes
2020-04-30 18:11:23 +00:00
14836619ab
simple benchmarks of IndexedSet, VersionedMap and std::map
2020-04-29 10:10:30 -07:00
72028de14d
intermediate commit
2020-04-17 15:30:45 -07:00
9f114d96c1
Merge branch 'master' of github.com:apple/foundationdb into features/flatbuffers-debugtx
2020-04-16 12:32:19 -07:00
7dc348d077
Remove `-isystem flow/-lpthread` from INCLUDES/CXXFLAGS
...
This cmake line generated a bogus and nonsensical include path, so as
the entire line isn't necessary, just remove it.
2020-04-13 02:22:25 -07:00
8b5780c36c
don't include source and binary dir
...
This forces users to use include paths from the sources root.
So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
3e3ac9f67d
Added interface
2020-04-01 14:41:16 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
9b5ef3416e
Refactor TLSParams into TLSConfig + LoadedTLSConfig
...
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.
initTLS now just takes the in-memory bytes and applies them to the ssl
context.
This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
f29d6c3f67
Move implementation of ArenaBlock members to Arena.cpp
2020-02-28 12:33:57 -08:00
9585cd10f1
Removed duplicate CMake link request
2020-02-24 00:19:43 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
9042cab7bc
Changed ordering of link libraries
2020-02-21 08:56:52 -08:00
d716c50000
Find OpenSSL or LibreSSL in CMake
2020-02-13 21:31:36 -08:00
2a2bf945ef
Also remove FDBLibTLS from CMake
2020-02-06 21:55:13 -08:00
e390dbd36c
Add a non-FDBLibTLS verify peers framework to new TLS impl
2020-02-06 21:06:52 -08:00
da73164eda
Move crc32c from fdbrpc to flow
...
So that we can use it from a piece of flow code without breaking module
boundaries.
Also rename generated-constants to crc32c-generated-constants so that
it's more apparent that they're related files.
2020-01-13 18:19:30 -08:00
21390c493a
Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
...
Resolved merge by keeping new test file from master branch: SampleNoSimAttrition.txt adding new constraint from Release branch about existing test file: SimpleExternalTest.txt
# Conflicts:
# tests/CMakeLists.txt
2019-12-18 09:05:08 -08:00
5080b8293a
Added Windows required library for function GetProcessMemoryInfo
2019-12-16 08:09:10 -08:00
3bf971ba8b
Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbserver/storageserver.actor.cpp
2019-12-12 07:13:12 -08:00
4f943be21d
Move DeterministicRandom impl to its own translation unit
...
This will allow me to recompile faster after making changes, and should
(slightly) speed up overall compilation.
I manually verified that the unseed matched for one test before and
after this change, so I probably didn't screw up the refactor
2019-12-04 16:45:32 -08:00
ed8d3f163c
Rename hgVersion to sourceVersion.
2019-11-15 12:26:51 -08:00
e4acd2e318
Disable TLS temporarily for OPEN_FOR_IDE build
2019-10-25 10:42:22 -07:00
Sam Gwydir