# Release Notes

## 6.0.18

### Fixes

- Backup metadata could falsely indicate that a backup is not usable. (PR #1007)
- Blobstore request failures could cause backup expire and delete operations to skip some files. (PR #1007)
- Blobstore request failures could cause restore to fail to apply some files. (PR #1007)
- Storage servers with large amounts of data would pause for a short period of time after rebooting. (PR #1001)
- The client library could leak memory when a thread died. (PR #1011)

### Features

- Added the ability to specify versions as version-days ago from latest log in backup. (PR #1007)

## 6.0.17

### Fixes

- Existing backups did not make progress when upgraded to 6.0.16. (PR #962)

## 6.0.16

### Performance

- Added a new backup folder scheme which results in far fewer kv range folders. (PR #939)

### Fixes

- Blobstore REST client attempted to create buckets that already existed. (PR #923)
- DNS would fail if IPv6 responses were received. (PR #945)
- Backup expiration would occasionally fail due to an incorrect assert. (PR #926)

## 6.0.15

### Features

- Added support for asynchronous replication to a remote DC with processes in a single cluster. This improves on the asynchronous replication offered by fdbdr because servers can fetch data from the remote DC if all replicas have been lost in one DC.
- Added support for synchronous replication of the transaction log to a remote DC. This remote DC does not need to contain any storage servers, meaning you need much fewer servers in this remote DC.
- The TLS plugin is now statically linked into the client and server binaries and no longer requires a separate library. (Issue #436)
- TLS peer verification now supports verifiying on Subject Alternative Name. (Issue #514)
- TLS peer verification now supports suffix matching by field. (Issue #515)
- TLS certificates are automatically reloaded after being updated. [6.0.5] (Issue #505)
- Added the `fileconfigure` command to fdbcli, which configures a database from a JSON document. [6.0.10] (PR #713)
- Backup-to-blobstore now accepts a “bucket” URL parameter for setting the bucket name where backup data will be read/written. [6.0.15] (PR #914)

### Performance

- Transaction logs do not copy mutations from previous generations of transaction logs. (PR #339)
- Load balancing temporarily avoids communicating with storage servers that have fallen behind.
- Avoid assigning storage servers responsibility for keys they do not have.
- Clients optimistically assume the first leader reply from a coordinator is correct. (PR #425)
- Network connections are now closed after no interface needs the connection. [6.0.1] (Issue #375)
- Significantly improved the CPU efficiency of copy mutations to transaction logs during recovery. [6.0.2] (PR #595)
- Significantly improved the CPU efficiency of generating status on the cluster controller. [6.0.11] (PR #758)
- Reduced CPU cost of truncating files that are being cached. [6.0.12] (PR #816)
- Significantly reduced master recovery times for clusters with large amounts of data. [6.0.14] (PR #836)
- Reduced read and commit latencies for clusters which are processing transactions larger than 1MB. [6.0.14] (PR #851)
- Significantly reduced recovery times when executing rollbacks on the memory storage engine. [6.0.14] (PR #821)
- Clients update their key location cache much more efficiently after storage server reboots. [6.0.15] (PR #892)
- Tuned multiple resolver configurations to do a better job balancing work between each resolver. [6.0.15] (PR #911)

### Fixes

- Not all endpoint failures were reported to the failure monitor.
- Watches registered on a lagging storage server would take a long time to trigger.
- The cluster controller would not start a new generation until it recovered its files from disk.
- Under heavy write load, storage servers would occasionally pause for ~100ms. [6.0.2] (PR #597)
- Storage servers were not given time to rejoin the cluster before being marked as failed. [6.0.2] (PR #592)
- Incorrect accounting of incompatible connections led to occasional assertion failures. [6.0.3] (PR #616)
- A client could fail to connect to a cluster when the cluster was upgraded to a version compatible with the client. This affected upgrades that were using the multi-version client to maintain compatibility with both versions of the cluster. [6.0.4] (PR #637)
- A large number of concurrent read attempts could bring the database down after a cluster reboot. [6.0.4] (PR #650)
- Automatic suppression of trace events which occur too frequently was happening before trace events were suppressed by other mechanisms. [6.0.4] (PR #656)
- After a recovery, the rate at which transaction logs made mutations durable to disk was around 5 times slower than normal. [6.0.5] (PR #666)
- Clusters configured to use TLS could get stuck spending all of their CPU opening new connections. [6.0.5] (PR #666)
- A mismatched TLS certificate and key set could cause the server to crash. [6.0.5] (PR #689)
- Sometimes a minority of coordinators would fail to converge after a new leader was elected. [6.0.6] (PR #700)
- Calling status too many times in a 5 second interval caused the cluster controller to pause for a few seconds. [6.0.7] (PR #711)
- TLS certificate reloading could cause TLS connections to drop until process restart. [6.0.9] (PR #717)
- Watches polled the server much more frequently than intended. [6.0.10] (PR #728)
- Backup and DR didn’t allow setting certain knobs. [6.0.10] (Issue #715)
- The failure monitor will become much less reactive after multiple successive failed recoveries. [6.0.10] (PR #739)
- Data distribution did not limit the number of source servers for a shard. [6.0.10] (PR #739)
- The cluster controller did not do locality aware reads when measuring status latencies. [6.0.12] (PR #801)
- Storage recruitment would spin too quickly when the storage server responded with an error. [6.0.12] (PR #801)
- Restoring a backup to the exact version a snapshot ends did not apply mutations done at the final version. [6.0.12] (PR #787)
- Excluding a process that was both the cluster controller and something else would cause two recoveries instead of one. [6.0.12] (PR #784)
- Configuring from `three_datacenter` to `three_datacenter_fallback` would cause a lot of unnecessary data movement. [6.0.12] (PR #782)
- Very rarely, backup snapshots would stop making progress. [6.0.14] (PR #837)
- Sometimes data distribution calculated the size of a shard incorrectly. [6.0.15] (PR #892)
- Changing the storage engine configuration would not effect which storage engine was used by the transaction logs. [6.0.15] (PR #892)
- On exit, fdbmonitor will only kill its child processes instead of its process group when run without the daemonize option. [6.0.15] (PR #826)
- HTTP client used by backup-to-blobstore now correctly treats response header field names as case insensitive. [6.0.15] (PR #904)
- Blobstore REST client was not following the S3 API in several ways (bucket name, date, and response formats). [6.0.15] (PR #914)
- Data distribution could queue shard movements for restoring replication at a low priority. [6.0.15] (PR #907)

### Fixes only impacting 6.0.0+

- A cluster configured with usable_regions=2 did not limit the rate at which it could copy data from the primary DC to the remote DC. This caused poor performance when recovering from a DC outage. [6.0.5] (PR #673)
- Configuring usable_regions=2 on a cluster with a large amount of data caused commits to pause for a few seconds. [6.0.5] (PR #687)
- On clusters configured with usable_regions=2, status reported no replicas remaining when the primary DC was still healthy. [6.0.5] (PR #687)
- Clients could crash when passing in TLS options. [6.0.5] (PR #649)
- Databases with more than 10TB of data would pause for a few seconds after recovery. [6.0.6] (PR #705)
- Configuring from usable_regions=2 to usable_regions=1 on a cluster with a large number of processes would prevent data distribution from completing. [6.0.12] (PR #721) (PR #739) (PR #780)
- Fixed a variety of problems with `force_recovery_with_data_loss`. [6.0.12] (PR #801)
- The transaction logs would leak memory when serving peek requests to log routers. [6.0.12] (PR #801)
- The transaction logs were doing a lot of unnecessary disk writes. [6.0.12] (PR #784)
- The master will recover the transaction state store from local transaction logs if possible. [6.0.12] (PR #801)
- A bug in status collection led to various workload metrics being missing and the cluster reporting unhealthy. [6.0.13] (PR #834)
- Data distribution did not stop tracking certain unhealthy teams, leading to incorrect status reporting. [6.0.15] (PR #892)
- Fixed a variety of problems related to changing between different region configurations. [6.0.15] (PR #892) (PR #907)
- fdbcli protects against configuration changes which could cause irreversible damage to a cluster. [6.0.15] (PR #892) (PR #907)
- Significantly reduced both client and server memory usage in clusters with large amounts of data and usable_regions=2. [6.0.15] (PR #892)

### Status

- The replication factor in status JSON is stored under `redundancy_mode` instead of `redundancy.factor`. (PR #492)
- The metric `data_version_lag` has been replaced by `data_lag.versions` and `data_lag.seconds`. (PR #521)
- Additional metrics for the number of watches and mutation count have been added and are exposed through status. (PR #521)

### Bindings

- API version updated to 600. See the API version upgrade guide for upgrade details.
- Several cases where functions in go might previously cause a panic now return a non-`nil` error. (PR #532)
- C API calls made on the network thread could be reordered with calls made from other threads. [6.0.2] (Issue #518)
- The TLS_PLUGIN option is now a no-op and has been deprecated. [6.0.10] (PR #710)
- Java: the Versionstamp::getUserVersion() method did not handle user versions greater than `0x00FF` due to operator precedence errors. [6.0.11] (Issue #761)
- Python: bindings didn’t work with Python 3.7 because of the new `async` keyword. [6.0.13] (Issue #830)
- Go: `PrefixRange` didn’t correctly return an error if it failed to generate the range. [6.0.15] (PR #878)
- Go: Add Tuple layer support for `uint`, `uint64`, and `*big.Int` integers up to 255 bytes. Integer values will be decoded into the first of `int64`, `uint64`, or `*big.Int` in which they fit. (PR #915) [6.0.15]
- Ruby: Add Tuple layer support for integers up to 255 bytes. (PR #915) [6.0.15]
- Python: bindings didn’t work with Python 3.7 because of the new `async` keyword. [6.0.13] (Issue #830)
- Go: `PrefixRange` didn’t correctly return an error if it failed to generate the range. [6.0.15] (PR #878)

### Other Changes

- Does not support upgrades from any version older than 5.0.
- Normalized the capitalization of trace event names and attributes. (PR #455)
- Various stateless processes now have a higher affinity for running on processes with unset process class, which may result in those roles changing location upon upgrade. See Version-specific notes on upgrading for details. (PR #526)
- Increased the memory requirements of the transaction log by 400MB. [6.0.5] (PR #673)

### Earlier release notes

- 5.2 (API Version 520)
- 5.1 (API Version 510)
- 5.0 (API Version 500)
- 4.6 (API Version 460)
- 4.5 (API Version 450)
- 4.4 (API Version 440)
- 4.3 (API Version 430)
- 4.2 (API Version 420)
- 4.1 (API Version 410)
- 4.0 (API Version 400)
- 3.0 (API Version 300)
- 2.0 (API Version 200)
- 1.0 (API Version 100)
- Beta 3 (API Version 23)
- Beta 2 (API Version 22)
- Beta 1 (API Version 21)
- Alpha 6 (API Version 16)
- Alpha 5 (API Version 14)