History

Josh Slocum baa35fbc8f Blob single part upload (#8703 ) * Exposing writeEntireFile up through BackupContainerFileSystem, and using it in blob worker * Adding blob worker latency metrics * avoid writeEntireFile if object is too large * gracefully falling back to multi-part upload if the file is too big		2022-11-08 17:30:01 -06:00
..
BackupContainerAzureBlobStore.actor.cpp	Blob single part upload (#8703 )	2022-11-08 17:30:01 -06:00
README.md	Extend url support for azure blob backup (#7776 )	2022-08-09 01:37:24 -07:00

README.md

Set up the Azure Backup Testing Environment

Make sure we built FDB with -DBUILD_AZURE_BACKUP=ON

Test

If you run BackupToBlob and RestoreFromBlob workloads with the paramter backupURL starts with azure://, the workload will backup to and restore from the azure blob storage. For example, BackupAzureBlobCorrectness.toml

Url format

The code now supports the following style urls:

azure://<account_name>.blob.core.windows.net/<container_name> (The formal url format for the blob service provided by the azure storage account)
azure://<ip|hostname>:<port>/<account_name>/<container_name> (Directly providing the endpoint address for the blob service, usually for local testing)

Local test environment

We need to use the Azurite to simulate an Azure blob service locally. Please follow the turtorial to start your service locally.

For example,

docker run -p 10000:10000 -v `pwd`:<path> -w <path> mcr.microsoft.com/azure-storage/azurite azurite-blob --blobHost 0.0.0.0 --blobPort 10000 --oauth basic --cert ./<...>.pem --key ./<...>.key.pem --debug ./<log_file_path>

Notice

To use uses https, we need to provide the certificates via --cert and --key The detailed turtorial to setup HTTPS. (We tested with the mkcert method)
To use Azure SDKs, we need to pass --oauth basic option
Please take a look at the difference between Azurite and Azure Storage