Go to file
Evan Tschannen 9b3254d5f4 A corrupted processId file should be deleted in simulation, as that is the manual operation that would fix the problem in the real world 2020-02-19 15:21:42 -08:00
FDBLibTLS Establishing TLS connection through the handshake process is expensive and the fdbserver process can get easily saturated with doing repeated TLS handshakes with only a few hundreds of clients have bad certificate. Hence throttle the number of handshakes done on the server per client ip if it has a bad certificate. 2020-01-10 16:19:41 -08:00
bindings Add -no_weak_exports 2020-02-18 10:49:43 -08:00
build Merge pull request #2451 from atn34/atn34/add-ubsan 2019-12-13 12:02:35 -08:00
cmake Several build improvements - enables parallel builds on Windows (again) 2020-02-09 08:34:14 -08:00
design Design: Backup data format: Add endianness discussion 2019-05-30 17:06:53 -07:00
documentation Added release note. 2020-02-18 16:45:31 -08:00
fdbbackup Backup and DR layer status document generation now uses snapshot reads for all keys read to avoid unnecessary conflicts when read during a status update or cleanup transaction. Since many of the keys read use wrapper functions, all of the underlying functions in BackupAgentBase and its two implementations also required a snapshot mode argument. All snapshot arguments default to false to match the underlying FDB API get/getrange methods. 2019-12-19 00:29:35 -08:00
fdbcli Allow new databases to be configured as locked 2020-02-03 15:32:01 -08:00
fdbclient BackupContainerFilesystem no longer unnecessarily depends on abspath() to find the last part of a path string, since it shouldn't touch the local filesystem in the remote case. 2020-02-18 16:35:00 -08:00
fdbmonitor Backported fix for linking pthread into fdbmonitor from master 2019-09-16 12:31:42 -07:00
fdbrpc fix: rebooting an unreliable process will make it reliable again, but while unreliable the files for that process could have already been corrupted so simulation will think a process is healthy that is actually corrupted 2020-02-19 15:18:57 -08:00
fdbserver A corrupted processId file should be deleted in simulation, as that is the manual operation that would fix the problem in the real world 2020-02-19 15:21:42 -08:00
fdbservice Enabled C++17 for all Windows projects 2019-05-16 17:44:13 -07:00
flow A corrupted processId file should be deleted in simulation, as that is the manual operation that would fix the problem in the real world 2020-02-19 15:21:42 -08:00
layers Fix comments to use transaction_too_old instead of past_version 2019-04-24 18:50:57 -07:00
monitoring Added required header for std::max 2019-12-11 08:36:06 -08:00
packaging update installer WIX GUID following release 2020-01-21 21:33:56 -08:00
recipes Increase the API version to 620. 2019-07-18 10:56:05 -07:00
tests Fix newline at end of file 2020-02-11 13:52:56 -08:00
.clang-format Add a clang-format config file. 2018-03-29 12:10:12 -07:00
.gitignore ignore ccls-cache 2019-02-22 13:09:43 -08:00
ACKNOWLEDGEMENTS Add Ruby domain source for future modification (note that this is slightly modified from the original to add license details to the file itself). 2019-06-14 09:16:49 -07:00
CMakeLists.txt Make compile commands target work on Windows 2020-02-09 08:34:52 -08:00
CODE_OF_CONDUCT.md Updates markdown link to Contributor Covenant homepage in the Code of Conduct. 2018-04-18 01:08:55 -04:00
CONTRIBUTING.md Apply suggestions from code review 2019-06-12 17:39:11 -07:00
LICENSE Initial repository commit 2017-05-25 13:48:44 -07:00
Makefile Suppress warnings for clang on Linux 2019-07-30 21:43:54 -07:00
README.md Apply suggestions from code review 2019-06-12 17:39:11 -07:00
fdb.cluster.cmake Fix port to match sandbox foundationdb.conf 2019-04-03 13:49:44 -07:00
foundationdb.sln Removed dead project id from solution file 2019-05-15 23:41:19 -07:00
versions.target update version to 6.2.16 2020-01-21 21:33:56 -08:00

README.md

FoundationDB logo

FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as an ordered key-value store and employs ACID transactions for all operations. It is especially well-suited for read/write workloads but also has excellent performance for write-intensive workloads. Users interact with the database using API language binding.

To learn more about FoundationDB, visit foundationdb.org

Documentation

Documentation can be found online at https://apple.github.io/foundationdb/. The documentation covers details of API usage, background information on design philosophy, and extensive usage examples. Docs are built from the source in this repo.

Forums

The FoundationDB Forums are the home for most of the discussion and communication about the FoundationDB project. We welcome your participation! We want FoundationDB to be a great project to be a part of and, as part of that, have established a Code of Conduct to establish what constitutes permissible modes of interaction.

Contributing

Contributing to FoundationDB can be in contributions to the code base, sharing your experience and insights in the community on the Forums, or contributing to projects that make use of FoundationDB. Please see the contributing guide for more specifics.

Getting Started

Binary downloads

Developers interested in using the FoundationDB store for an application can get started easily by downloading and installing a binary package. Please see the downloads page for a list of available packages.

Compiling from source

Developers on an OS for which there is no binary package, or who would like to start hacking on the code, can get started by compiling from source.

Currently there are two build systems: a collection of Makefiles and a CMake-based build system. Both of them should currently work for most users, and CMake should be the preferred choice as it will eventually become the only build system available.

CMake

To build with CMake, generally the following is required (works on Linux and Mac OS - for Windows see below):

  1. Check out this repository.
  2. Install cmake Version 3.12 or higher CMake
  3. Download version 1.67 of Boost.
  4. Unpack boost (you don't need to compile it)
  5. Install Mono.
  6. Install a JDK. FoundationDB currently builds with Java 8.
  7. Create a build directory (you can have the build directory anywhere you like): mkdir build
  8. cd build
  9. cmake -DBOOST_ROOT=<PATH_TO_BOOST> <PATH_TO_FOUNDATIONDB_DIRECTORY>
  10. make

CMake will try to find its dependencies. However, for LibreSSL this can be often problematic (especially if OpenSSL is installed as well). For that we recommend passing the argument -DLibreSSL_ROOT to cmake. So, for example, if you LibreSSL is installed under /usr/local/libressl-2.8.3, you should call cmake like this:

cmake -DLibreSSL_ROOT=/usr/local/libressl-2.8.3/ ../foundationdb

FoundationDB will build just fine without LibreSSL, however, the resulting binaries won't support TLS connections.

Language Bindings

The language bindings that are supported by cmake will have a corresponding README.md file in the corresponding bindings/lang directory.

Generally, cmake will build all language bindings for which it can find all necessary dependencies. After each successful cmake run, cmake will tell you which language bindings it is going to build.

Generating compile_commands.json

CMake can build a compilation database for you. However, the default generated one is not too useful as it operates on the generated files. When running make, the build system will create another compile_commands.json file in the source directory. This can than be used for tools like CCLS, CQuery, etc. This way you can get code-completion and code navigation in flow. It is not yet perfect (it will show a few errors) but we are constantly working on improving the development experience.

CMake will not produce a compile_commands.json, you must pass -DCMAKE_EXPORT_COMPILE_COMMANDS=ON. This also enables the target processed_compile_commands, which rewrites compile_commands.json to describe the actor compiler source file, not the post-processed output files, and places the output file in the source directory. This file should then be picked up automatically by any tooling.

Note that if building inside of the foundationdb/foundationdb-build docker image, the resulting paths will still be incorrect and require manual fixing. One will wish to re-run cmake with -DCMAKE_EXPORT_COMPILE_COMMANDS=OFF to prevent it from reverting the manual changes.

Using IDEs

CMake has built in support for a number of popular IDEs. However, because flow files are precompiled with the actor compiler, an IDE will not be very useful as a user will only be presented with the generated code - which is not what she wants to edit and get IDE features for.

The good news is, that it is possible to generate project files for editing flow with a supported IDE. There is a CMake option called OPEN_FOR_IDE which will generate a project which can be opened in an IDE for editing. You won't be able to build this project, but you will be able to edit the files and get most edit and navigation features your IDE supports.

For example, if you want to use XCode to make changes to FoundationDB you can create a XCode-project with the following command:

cmake -G Xcode -DOPEN_FOR_IDE=ON <FDB_SOURCE_DIRECTORY>

You should create a second build-directory which you will use for building (probably with make or ninja) and debugging.

Linux

There are no special requirements for Linux. A docker image can be pulled from foundationdb/foundationdb-build that has all of FoundationDB's dependencies pre-installed, and is what the CI uses to build and test PRs.

If you want to create a package you have to tell cmake what platform it is for. And then you can build by simply calling cpack. So for debian, call:

cmake -DINSTALL_LAYOUT=DEB  <FDB_SOURCE_DIR>
make
cpack

For RPM simply replace DEB with RPM.

MacOS

The build under MacOS will work the same way as on Linux. To get LibreSSL and boost you can use Homebrew. LibreSSL will not be installed in /usr/local instead it will stay in /usr/local/Cellar. So the cmake command will look something like this:

cmake -DLibreSSL_ROOT=/usr/local/Cellar/libressl/2.8.3 <PATH_TO_FOUNDATIONDB_SOURCE>

To generate a installable package, you have to call CMake with the corresponding arguments and then use cpack to generate the package:

cmake -DINSTALL_LAYOUT=OSX  <FDB_SOURCE_DIR>
make
cpack

Windows

Under Windows, the build instructions are very similar, with the main difference that Visual Studio is used to compile.

  1. Install Visual Studio 2017 (Community Edition is tested)
  2. Install cmake Version 3.12 or higher CMake
  3. Download version 1.67 of Boost.
  4. Unpack boost (you don't need to compile it)
  5. Install Mono.
  6. Install a JDK. FoundationDB currently builds with Java 8.
  7. Set JAVA_HOME to the unpacked location and JAVA_COMPILE to $JAVA_HOME/bin/javac.
  8. Install Python if it is not already installed by Visual Studio.
  9. (Optional) Install WIX. Without it Visual Studio won't build the Windows installer.
  10. Create a build directory (you can have the build directory anywhere you like): mkdir build
  11. cd build
  12. cmake -G "Visual Studio 15 2017 Win64" -DBOOST_ROOT=<PATH_TO_BOOST> <PATH_TO_FOUNDATIONDB_DIRECTORY>
  13. This should succeed. In which case you can build using msbuild: msbuild /p:Configuration=Release foundationdb.sln. You can also open the resulting solution in Visual Studio and compile from there. However, be aware that using Visual Studio for development is currently not supported as Visual Studio will only know about the generated files. msbuild is located at c:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe for Visual Studio 15.

If you want TLS support to be enabled under Windows you currently have to build and install LibreSSL yourself as the newer LibreSSL versions are not provided for download from the LibreSSL homepage. To build LibreSSL:

  1. Download and unpack libressl (>= 2.8.2)
  2. cd libressl-2.8.2
  3. mkdir build
  4. cd build
  5. cmake -G "Visual Studio 15 2017 Win64" ..
  6. Open the generated LibreSSL.sln in Visual Studio as administrator (this is necessary for the install)
  7. Build the INSTALL project in Release mode

This will install LibreSSL under C:\Program Files\LibreSSL. After that cmake will automatically find it and build with TLS support.

If you installed WIX before running cmake you should find the FDBInstaller.msi in your build directory under packaging/msi.

Makefile

MacOS

  1. Check out this repo on your Mac.
  2. Install the Xcode command-line tools.
  3. Download version 1.67.0 of Boost.
  4. Set the BOOSTDIR environment variable to the location containing this boost installation.
  5. Install Mono.
  6. Install a JDK. FoundationDB currently builds with Java 8.
  7. Navigate to the directory where you checked out the foundationdb repo.
  8. Run make.

Linux

  1. Install Docker.

  2. Check out the foundationdb repo.

  3. Run the docker image interactively Docker Run with the directory containing the foundationdb repo mounted Docker Mounts.

    docker run -it -v '/local/dir/path/foundationdb:/docker/dir/path/foundationdb' foundationdb/foundationdb-build:latest
    
  4. Run $ scl enable devtoolset-8 python27 rh-python36 rh-ruby24 -- bash within the running container. This enables a more modern compiler, which is required to build FoundationDB.

  5. Navigate to the container's mounted directory which contains the foundationdb repo.

    cd /docker/dir/path/foundationdb
    
  6. Run make.

This will build the fdbserver binary and the python bindings. If you want to build our other bindings, you will need to install a runtime for the language whose binding you want to build. Each binding has an .mk file which provides specific targets for that binding.