wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
foundationdb/fdbrpc
History
Ata E Husain Bohra 33ae398268
REST KmsConnector implementation (#6994) 
* REST KmsConnector implementation

Description
  diff-1: Address review comments.
          Add utility interface to Platform namespace to
          create and operate on tmpfile
 diff-2: Address review comments
         Link Boost::filesystem to CMake build process

Major changes includes:
1. Implement REST based KmsConnector implementation.
2. Salient features of the connector:
 2.1. Two required configuration are:
   a. Discovery KMS URLs - enable KMS discovery on bootstrap
   b. Endpoint path configuration to construct URI to fetch/refresh
      encryption keys
   c. Configuration to provide "validationTokens" to connect with
      external KMS. Patch implements file-based token validation scheme.
 2.2. On startup, RESTKmsConnector discovers KMS Urls and caches
      them in-memory. Extracts "validationTokens" based on input config.
 2.3. Expose endpoints to allow fetch/refresh of encryption keys.
 2.4. Defines JSON format to interact with external KMS - request &
      response payload format.
3. Extend Platform namespace with an interface to create and operate on
   tmp files.
4. Update Platform 'readFileBytes' and 'writeFileBytes' to leverage
   fstream supported implementation.

NOTE: KMS URLs fetched after initial discovery will be persisted using
      DynamicKnobs. It is TODO at the moment and shall be completed
      once DynamicKnobs is feature complete

Testing

Unit test to validation following:
1. Parsing on "validation tokens" logic.
2. Construction and parsing of REST JSON request and response strings.
 		2022-05-07 13:18:35 -07:00
..
libcoroutine apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
libeio apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
ActorFuzz.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
ActorFuzz.h Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileCached.actor.cpp Use allocateFast4kAligned instead of duplicating its logic 2022-03-22 11:01:33 -07:00
AsyncFileCached.actor.h Use allocateFast4kAligned instead of duplicating its logic 2022-03-22 11:01:33 -07:00
AsyncFileChaos.actor.h Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileEIO.actor.h Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileEncrypted.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileEncrypted.h Remove ENCRYPTION_ENABLED macro 2022-05-02 22:26:31 -07:00
AsyncFileKAIO.actor.h AsyncFileKAIO: add latency histograms 2022-03-30 10:45:04 -07:00
AsyncFileNonDurable.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileNonDurable.actor.h Bug fix: Killing a machine process would not wait for AsyncFileNonDurable close operations to finish, causing a later reopen of the same file in a new process to hang forever. Renamed AsyncFileNonDurable::deleteFile to closeFile for clarity. Renamed Machine deletingFiles to deletingOrClosingFiles for clarity. (#7007) 2022-04-29 14:01:18 -07:00
AsyncFileReadAhead.actor.h Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileWinASIO.actor.h Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileWriteChecker.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
AsyncFileWriteChecker.h Update copyright header dates 2022-03-21 13:36:23 -07:00
CMakeLists.txt Remove WITH_TLS CMake variable 2022-05-02 22:45:00 -07:00
ContinuousSample.h Update copyright header dates 2022-03-21 13:36:23 -07:00
FailureMonitor.actor.cpp Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-04-07 13:29:27 -06:00
FailureMonitor.h Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-04-07 13:29:27 -06:00
FlowProcess.actor.h Remote ikvs debugging (#6465) 2022-03-31 17:08:59 -07:00
FlowTests.actor.cpp OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
FlowTransport.actor.cpp OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
FlowTransport.h OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
HTTP.actor.cpp Interface to enable clients to send/receive REST requests/responses (#6866) 2022-04-27 12:17:52 -07:00
HTTP.h Interface to enable clients to send/receive REST requests/responses (#6866) 2022-04-27 12:17:52 -07:00
HealthMonitor.actor.cpp Fix valgrind error in HealthMonitor 2022-04-07 15:48:06 -07:00
HealthMonitor.h Allow worker health monitor to report recent destroyed peers who currently have roles in transaction systems 2022-04-06 13:33:50 -07:00
IAsyncFile.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
IAsyncFile.h Update copyright header dates 2022-03-21 13:36:23 -07:00
IPAllowList.cpp Address review comments 2022-04-11 09:17:10 -06:00
IPAllowList.h Address review comments 2022-04-11 09:17:10 -06:00
IRateControl.h Update copyright header dates 2022-03-21 13:36:23 -07:00
LoadBalance.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
LoadBalance.actor.h Fix IDE build warnings and errors 2022-05-01 16:20:57 -07:00
LoadBalance.h Update copyright header dates 2022-03-21 13:36:23 -07:00
LoadPlugin.h Update copyright header dates 2022-03-21 13:36:23 -07:00
Locality.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
Locality.h Update copyright header dates 2022-03-21 13:36:23 -07:00
MultiInterface.h Update copyright header dates 2022-03-21 13:36:23 -07:00
Net2FileSystem.cpp Remove ENCRYPTION_ENABLED macro 2022-05-02 22:26:31 -07:00
Net2FileSystem.h Update copyright header dates 2022-03-21 13:36:23 -07:00
PerfMetric.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
PerfMetric.h Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-04-07 13:29:27 -06:00
QueueModel.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
QueueModel.h Update copyright header dates 2022-03-21 13:36:23 -07:00
RESTClient.actor.cpp Interface to enable clients to send/receive REST requests/responses (#6866) 2022-04-27 12:17:52 -07:00
RESTClient.h Interface to enable clients to send/receive REST requests/responses (#6866) 2022-04-27 12:17:52 -07:00
RESTUtils.actor.cpp REST KmsConnector implementation (#6994) 2022-05-07 13:18:35 -07:00
RESTUtils.h Interface to enable clients to send/receive REST requests/responses (#6866) 2022-04-27 12:17:52 -07:00
RangeMap.h Update copyright header dates 2022-03-21 13:36:23 -07:00
Replication.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
Replication.h Update copyright header dates 2022-03-21 13:36:23 -07:00
ReplicationPolicy.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
ReplicationPolicy.h Update copyright header dates 2022-03-21 13:36:23 -07:00
ReplicationTypes.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
ReplicationTypes.h Update copyright header dates 2022-03-21 13:36:23 -07:00
ReplicationUtils.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
ReplicationUtils.h Update copyright header dates 2022-03-21 13:36:23 -07:00
SimExternalConnection.actor.cpp Add DNS cache. 2022-04-04 15:08:17 -07:00
SimExternalConnection.h Add DNS cache. 2022-04-04 15:08:17 -07:00
Smoother.h Update copyright header dates 2022-03-21 13:36:23 -07:00
Stats.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
Stats.h Update copyright header dates 2022-03-21 13:36:23 -07:00
TSSComparison.h Update copyright header dates 2022-03-21 13:36:23 -07:00
TenantInfo.h Pass object as reference 2022-02-25 14:54:52 +01:00
TimedRequest.h Update copyright header dates 2022-03-21 13:36:23 -07:00
TokenSign.cpp Fix TokenSign copying and using uninitialized arena 2022-05-02 09:48:43 +02:00
TokenSign.h Use camelCase & move error code to 6xxx 2022-04-13 21:11:52 +02:00
TraceFileIO.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
TraceFileIO.h Update copyright header dates 2022-03-21 13:36:23 -07:00
actorFuzz.py Merge release-6.3 into master 2020-05-22 09:25:32 -07:00
dsltest.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
fdbrpc.h Merge remote-tracking branch 'origin/main' into features/private-request-streams 2022-04-10 14:12:37 -06:00
genericactors.actor.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
genericactors.actor.h Check in the new Hostname logic. (#6926) 2022-04-27 21:54:13 -07:00
linux_kaio.h Update copyright header dates 2022-03-21 13:36:23 -07:00
networksender.actor.h Update copyright header dates 2022-03-21 13:36:23 -07:00
sim2.actor.cpp Remove ENCRYPTION_ENABLED macro 2022-05-02 22:26:31 -07:00
sim_validation.cpp Update copyright header dates 2022-03-21 13:36:23 -07:00
sim_validation.h Update copyright header dates 2022-03-21 13:36:23 -07:00
simulator.h Bug fix: Killing a machine process would not wait for AsyncFileNonDurable close operations to finish, causing a later reopen of the same file in a new process to hang forever. Renamed AsyncFileNonDurable::deleteFile to closeFile for clarity. Renamed Machine deletingFiles to deletingOrClosingFiles for clarity. (#7007) 2022-04-29 14:01:18 -07:00