Commit Graph

8 Commits

Author SHA1 Message Date
Yi Wu 66f1c5c85a
Small BlobCipher and SimKmsConnector fixes and changes (#6936)
* SimKmsConnector fix domain id being unsigned
* SimKmsConnector fix returning cipher id 0 as latest key, which is invalid
* SimKmsConnector fix keys initialized as c-style strings with incorrect length and uninitialized bytes
* SimKmsConnector fix returning different keys for the same id after restart
* BlobCipher change APIs to return null reference when key not found
* BlobCipher insertCipherKey to return the inserted key
2022-05-04 14:09:31 -07:00
sfc-gh-tclinkenbeard 475d66084d Remove ENCRYPTION_ENABLED macro 2022-05-02 22:26:31 -07:00
Ata E Husain Bohra 04ecd8e08f
Revert "Revert "Update 'salt' details for EncryptHeader AuthToken details (#6881)" (#6902)" (#6922)
Description

Major changes proposed:
1. This reverts commit f38b2e8209.
2. Also add fix for Valgrind failure due to unintialized variables.
3. Improve checks to catch is cipherKey details cached in BlobCipherKeyCache
   isn't as expected

Testing

Overall correctness: 10K (20220421-193911-ahusain-foundationdb-a730e5cb38541e20)
EncyrptionOps correctness: 100K (20220421-194315-ahusain-foundationdb-29c598a8b9420430)
EncryptionOps Valgrind: 100 (20220421-194434-ahusain-foundationdb-7fc5f98eddc0921a)
2022-04-21 18:57:56 -07:00
Markus Pilman f38b2e8209
Revert "Update 'salt' details for EncryptHeader AuthToken details (#6881)" (#6902)
This reverts commit a38318a6ac.
2022-04-21 09:04:40 -07:00
Ata E Husain Bohra a38318a6ac
Update 'salt' details for EncryptHeader AuthToken details (#6881)
* Update 'salt' details for EncryptHeader AuthToken details

Description

Major changes:
1. Add 'salt' to BlobCipherEncryptHeader::cipherHeaderDetails.
2. During decryption it is possible that BlobKeyCacheId doesn't
    contain required baseCipherDetails. Add API to KeyCache to
    allowing re-populating of CipherDetails with a given 'salt'
3. Update BaseCipherKeyIdCache indexing using {BaseCipherKeyId, salt}
    tuple. FDB processes leverage BlobCipherKeyCache to implement
    in-memory caching of cipherKeys, given EncryptKeyProxy supplies
    BaseCipher details, each encryption participant service would
    generate its derived key by using different 'salt'. Further,
    it is possible to cache multiple {baseCipherKeyId, salt} tuples;
    for instance: CP encrypted mutations being deciphered by
    StorageServer etc.

Testing

1. Update EncyrptionOps simulation test to simulate KeyCache miss
2. Update BlobCipher unit tests to validate above mentioned changes
2022-04-18 22:01:56 -07:00
Ata E Husain Bohra 933e5bbd2e
EncryptKeyProxy server APIs for simulation runs. (#6727)
* EncryptKeyProxy server APIs for simulation runs.

Description

  diff-2: FlowSingleton util class
              Bug fixes
  diff-1: Expected errors returned to the caller

Major changes proposed are:
1. EncryptKeyProxy server APIs:
 1.1. Lookup Cipher details via BaseCipherId
 1.2. Lookup latest Cipher details via encryption domainId.
2. EncyrptKeyProxy implements caches indexed by: baseCipherId &
   encyrptDomainId
3. Periodic task to refresh domainId indexed cache to support
   'limiting cipher lifetime' abilities if supported by
   external KMS solutions.

Testing

EncyrptKeyProxyTest workload to validate the newly added code.
2022-04-11 09:08:42 -07:00
Ata E Husain Bohra 81c7834d06
Encryption header authentication tokens (#6750)
* Encryption header authentication tokens

Description

  diff-1: Allow NONE AuthTokenMode operations
          Address review comments

Major changes proposed are:
1.Encryption header support two modes of generation 'authentication tokens':
  a) SingleAuthTokenMode: the scheme generates single crypto-secure auth
     token to protect {cipherText + header} payload. Scheme is geared towards
     optimizing cost due to crypto-secure auth-token generation, however,
     on decryption client needs to be read 'header' + 'encrypted-buffer'
     to validate the 'auth-token'. The scheme is ideal for usecases where
     payload represented by the encryptionHeader is not large and it is
     desirable to minimize CPU/latency penalty due to crypto-secure ops,
     such as: CommitProxies encrypted inline transactions,
     StorageServer encrypting pages etc.
  b) MultiAuthTokenMode: Scheme generates separate authTokens for
     'encrypted buffer' & 'encryption-header'. The scheme is ideal where
     payload represented by encryptionHeader is large enough such that it
     is desirable to optimize cost of upfront reading full 'encrypted buffer',
     compared to reading only encryptionHeader and ensuring its sanity;
     for instance: backup-files
2. Leverage full crypto-secure digest as 'authentication token'

Testing

Update EncryptionOps simulation test
Update BlobCipher unit test
20220408-182229-ahusain-foundationdb-7fd2e4b19328cd44
20220408-175754-ahusain-foundationdb-5352e37e1dcabfc8
2022-04-08 11:32:05 -07:00
Ata E Husain Bohra 017709aec6
Introduce BlobCipher interface and cipher caching interface (#6391)
* Introduce BlobCipher interface and cipher caching interface

 diff-3: Update the code to avoid deriving encryption key periodically.
         Implement EncyrptBuf interface to limit memcpys.
         Improve both unit test and simulation to better code coverage.
 diff-2: Add specific error code for OpenSSL AES call failures
 diff-1: Update encryption scheme to AES-256-CTR. Minor
         updates to Header to capture more information.

Major changes proposed are:
1. Introduce encyrption header format.
2. Introduce a BlobCipher cipher key representation encoding
following information: baseCipher details, derived encryption cipher
details, creationTime and random salt.
3. Introduce interface to support block cipher encrytion and decrytion
operations. Encyrption populates encryption header allowing client to
persist them on-disk, this header is then read allowing decryption
on reads.
4. Introduce interface to allow in-memory caching of cipher keys. The
cache allowing mapping of "encryption domain" -> "base cipher id" ->
"derived cipher keys" (3D hash map). This cache interface will be used
by FDB processes participating in encryption to cache recently used
ciphers (performance optimization).

Testing:
1. Unit test to validate caching interface.
2. Update EncryptionOps simulation test to validate block cipher
operations.
2022-03-24 07:31:49 -07:00