From f36be44599a951cb3343b91a485011e20810d59e Mon Sep 17 00:00:00 2001 From: Evan Tschannen Date: Sat, 30 Mar 2019 21:22:17 -0700 Subject: [PATCH 01/13] Updated release notes for 6.1.0 --- documentation/sphinx/source/release-notes.rst | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index 32c5ef26d6..50bbddfbe7 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -15,7 +15,7 @@ Features * Added configuration option to choose log spilling implementation `(PR #1160) `_ * Added configuration option to choose log system implementation `(PR #1160) `_ * Batch priority transactions are now limited separately by ratekeeper and will be throttled at lower levels of cluster saturation. This makes it possible to run a more intense background load at saturation without significantly affecting normal priority transactions. It is still recommended not to run excessive loads at batch priority. `(PR #1198) `_ -* Restore now requires the destnation cluster to be specified explicitly to avoid confusion. `(PR #1240) `_ +* Restore now requires the destination cluster to be specified explicitly to avoid confusion. `(PR #1240) `_ * Restore target version can now be specified by timestamp if the original cluster is available. `(PR #1240) `_ * Backup status and describe commands now have a --json output option. `(PR #1248) `_ * Separate data distribution out from master as a new role. `(PR #1062) `_ @@ -29,10 +29,21 @@ Features * Deprecated transaction option ``TRANSACTION_LOGGING_ENABLE``. Added two new transaction options ``DEBUG_TRANSACTION_IDENTIFIER`` and ``LOG_TRANSACTION`` that sets an identifier for the transaction and logs the transaction to the trace file respectively. `(PR #1200) `_ * Clients can now specify default transaction timeouts and retry limits to all transactions through a database option. `(Issue #775) `_ * The "timeout", "max retry delay", and "retry limit" transaction options are no longer reset when the transaction is reset after a call to ``onError`` (as of API version 610). `(Issue #775) `_ +* Added the ``force_recovery_with_data_loss`` command to fdbcli. When a cluster is configured with usable_regions=2, this command will force the database to recover in the remote region. `(PR #1168) `_ +* Added a limit to the number of status requests the cluster controller will handle. `(PR #1093) `_ (submitted by tclinken) +* Added a ``coordinator`` process class. Processes with this class can only be used as a coordinator, and ``coordinators auto`` will prefer to chose processes of this class. `(PR #1069) `_ (submitted by tclinken) +* The ``consistencycheck`` fdbserver role will check the entire database at most once every week. `(PR #1126) `_ +* Added the metadata version key (``\xff/metadataVersion``). The value of this key is sent with every read version. It is intended to help clients cache rarely changing metadata. `(PR #1213) `_ +* The ``fdbdr switch`` command verifies a ``dr_agent`` exists in both directions. `(Issue #1220) `_ +* Transaction logs that cannot commit to disk for more than 5 seconds are marked as degraded. The cluster controller will prefer to recruit transaction logs on other processes before using degraded processes. `(Issue #690) `_ +* The ``memory`` storage engine configuration now uses the ssd engine for transaction log spilling. Transaction log spilling only happens when the transaction logs are using too much memory, so using the memory storage engine for this purpose can cause the process to run out of memory. `(PR #1314) `_ +* Trace logs can be output as JSON instead of XML using the ``--trace_format`` command line option. `(PR #976) `_ (by mpilman) Performance ----------- +* Increased the get read version batch size in the client. This change reduces the load on the proxies when doing many transactions with only a few operations per transaction. `(PR #1311) `_ +* Clients no longer attempt to connect to the master during recovery. `(PR #1317) `_ * Java: Succesful commits and range reads no longer create ``FDBException`` objects to reduce memory pressure. `(Issue #1235) `_ Fixes @@ -42,10 +53,16 @@ Fixes * In some cases, calling ``OnError`` with a non-retryable error would partially reset a transaction. As of API version 610, the transaction will no longer be reset in these cases and will instead put the transaction into an error state. `(PR #1298) `_ * Standardized datetime string format across all backup and restore command options and outputs. `(PR #1248) `_ * Read workload status metrics would disappear when a storage server was missing. `(PR #1348) `_ +* The ``coordinators auto`` command could recruit multiple coordinators with the same zone ID. `(Issue #988) `_ +* The version of a cluster after a restore could have been lower than the restore version, making version stamp operations get smaller. `(PR #1213) `_ +* If a process changed its process class while being rebooted, it could recruited for the cluster controller using its previous process class. `(PR #1350) `_ +* Fixed a few thread safety issues. `(PR #1085) `_ Status ------ +* Degraded processes are reported in ``status json``. `(Issue #690) `_ + Bindings -------- From 74ca0a672d3088536d6e2c5ed16e843c407e2a67 Mon Sep 17 00:00:00 2001 From: Evan Tschannen Date: Sat, 30 Mar 2019 21:28:39 -0700 Subject: [PATCH 02/13] Release notes from 6.0.19 were moved to 6.1.0 --- documentation/sphinx/source/release-notes.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index 50bbddfbe7..da66725957 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -38,13 +38,14 @@ Features * Transaction logs that cannot commit to disk for more than 5 seconds are marked as degraded. The cluster controller will prefer to recruit transaction logs on other processes before using degraded processes. `(Issue #690) `_ * The ``memory`` storage engine configuration now uses the ssd engine for transaction log spilling. Transaction log spilling only happens when the transaction logs are using too much memory, so using the memory storage engine for this purpose can cause the process to run out of memory. `(PR #1314) `_ * Trace logs can be output as JSON instead of XML using the ``--trace_format`` command line option. `(PR #976) `_ (by mpilman) +* Added ``modify`` command to fdbbackup for modifying parameters of a running backup. `(PR #1237) `_ +* Added 'header' parameter to blobstore backup URLs for setting custom HTTP headers. `(PR #1237) `_ Performance ----------- * Increased the get read version batch size in the client. This change reduces the load on the proxies when doing many transactions with only a few operations per transaction. `(PR #1311) `_ * Clients no longer attempt to connect to the master during recovery. `(PR #1317) `_ -* Java: Succesful commits and range reads no longer create ``FDBException`` objects to reduce memory pressure. `(Issue #1235) `_ Fixes ----- @@ -57,6 +58,12 @@ Fixes * The version of a cluster after a restore could have been lower than the restore version, making version stamp operations get smaller. `(PR #1213) `_ * If a process changed its process class while being rebooted, it could recruited for the cluster controller using its previous process class. `(PR #1350) `_ * Fixed a few thread safety issues. `(PR #1085) `_ +* The Go bindings reported an incorrect required version when trying to load an incompatible fdb_c library. `(PR #1053) `_ +* The ``include`` command in fdbcli would falsely include all machines with IP addresses that + have the included IP address as a prefix (for example ``include 1.0.0.1`` would also include + ``1.0.0.10``). `(PR #1121) `_ +* Restore could crash when reading a file that ends on a block boundary (1MB default). `(PR #1205) `_ +* Java: Successful commits and range reads no longer create ``FDBException`` objects to reduce memory pressure. `(Issue #1235) `_ Status ------ From 2d4d219ef283e46591c6c46e83627195073c2498 Mon Sep 17 00:00:00 2001 From: Chris Donati Date: Sun, 31 Mar 2019 11:54:36 -0700 Subject: [PATCH 03/13] Improve Python KeySelector representation Python 2 resolves escape characters when printing byte strings, which doesn't seem useful in this context. I replaced __str__ since it defaults to __repr__ and the change should be suitable for both cases. --- bindings/python/fdb/impl.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bindings/python/fdb/impl.py b/bindings/python/fdb/impl.py index fb248eb44a..77c7a74d13 100644 --- a/bindings/python/fdb/impl.py +++ b/bindings/python/fdb/impl.py @@ -1159,8 +1159,8 @@ class KeySelector(object): def first_greater_or_equal(cls, key): return cls(key, False, 1) - def __str__(self): - return 'KeySelector(%s, %r, %d)' % (self.key, self.or_equal, self.offset) + def __repr__(self): + return 'KeySelector(%r, %r, %r)' % (self.key, self.or_equal, self.offset) class KVIter(object): From 1ba0b4e6820dd3467bb06a5a6766d97f053a822d Mon Sep 17 00:00:00 2001 From: Vishesh Yadav Date: Mon, 1 Apr 2019 11:40:26 -0700 Subject: [PATCH 04/13] fix: Parse IPv6 addresses correctly in status details --- fdbcli/fdbcli.actor.cpp | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/fdbcli/fdbcli.actor.cpp b/fdbcli/fdbcli.actor.cpp index 0bd785b7bd..09b38d0dd5 100644 --- a/fdbcli/fdbcli.actor.cpp +++ b/fdbcli/fdbcli.actor.cpp @@ -40,6 +40,7 @@ #include "fdbcli/FlowLineNoise.h" +#include #include #ifdef __unixish__ @@ -1311,7 +1312,7 @@ void printStatus(StatusObjectReader statusObj, StatusClient::StatusLevel level, outputStringCache = outputString; try { // constructs process performance details output - std::map workerDetails; + std::map workerDetails; for (auto proc : processesMap.obj()){ StatusObjectReader procObj(proc.second); std::string address; @@ -1319,27 +1320,24 @@ void printStatus(StatusObjectReader statusObj, StatusClient::StatusLevel level, std::string line; - // Windows does not support the "hh" width specifier so just using unsigned int to be safe. - unsigned int a, b, c, d, port; - int tokens = sscanf(address.c_str(), "%u.%u.%u.%u:%u", &a, &b, &c, &d, &port); - - // If did not get exactly 5 tokens, or one of the integers is too large, address is invalid. - if (tokens != 5 || a & ~0xFF || b & ~0xFF || c & ~0xFF || d & ~0xFF || port & ~0xFFFF) - { + NetworkAddress parsedAddress; + try { + parsedAddress = NetworkAddress::parse(address); + } catch (Error& e) { + // Groups all invalid IP address/port pair in the end of this detail group. line = format(" %-22s (invalid IP address or port)", address.c_str()); - std::string &lastline = workerDetails[std::numeric_limits::max()]; + IPAddress::IPAddressStore maxIp; + for (int i = 0; i < maxIp.size(); ++i) { + maxIp[i] = std::numeric_limits::type>::max(); + } + std::string& lastline = + workerDetails[NetworkAddress(IPAddress(maxIp), std::numeric_limits::max())]; if (!lastline.empty()) lastline.append("\n"); lastline += line; continue; } - // Create addrNum as a 48 bit number {A}{B}{C}{D}{PORT} - uint64_t addrNum = 0; - for (auto i : { a, b, c, d }) - addrNum = (addrNum << 8) | i; - addrNum = (addrNum << 16) | port; - try { double tx = -1, rx = -1, mCPUUtil = -1; int64_t processTotalSize; @@ -1401,12 +1399,12 @@ void printStatus(StatusObjectReader statusObj, StatusClient::StatusLevel level, } } - workerDetails[addrNum] = line; + workerDetails[parsedAddress] = line; } catch (std::runtime_error& e) { std::string noMetrics = format(" %-22s (no metrics available)", address.c_str()); - workerDetails[addrNum] = noMetrics; + workerDetails[parsedAddress] = noMetrics; } } for (auto w : workerDetails) From bae815a777fbf93a3bdb0b88e6d3531bff9937be Mon Sep 17 00:00:00 2001 From: Stephen Atherton Date: Mon, 1 Apr 2019 15:00:24 -0700 Subject: [PATCH 05/13] Bug fix, starting a restore on a tag already in-use would spinloop forever and eventually run out of memory. --- fdbclient/FileBackupAgent.actor.cpp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fdbclient/FileBackupAgent.actor.cpp b/fdbclient/FileBackupAgent.actor.cpp index 9f607f0806..57e61d661e 100644 --- a/fdbclient/FileBackupAgent.actor.cpp +++ b/fdbclient/FileBackupAgent.actor.cpp @@ -3772,7 +3772,7 @@ public: printf("%s\n", details.c_str()); } - ERestoreState status_ = wait(restore.stateEnum().getD(tr)); + ERestoreState status_ = wait(restore.stateEnum().getD(tr)); status = status_; state bool runnable = wait(restore.isRunnable(tr)); @@ -4251,9 +4251,10 @@ public: wait(tr->commit()); break; } catch(Error &e) { - if(e.code() != error_code_restore_duplicate_tag) { - wait(tr->onError(e)); + if(e.code() == error_code_restore_duplicate_tag) { + throw; } + wait(tr->onError(e)); } } From b750d2a0a17d5de5ffeab452c5a577fae6c20516 Mon Sep 17 00:00:00 2001 From: "A.J. Beamon" Date: Mon, 1 Apr 2019 15:42:20 -0700 Subject: [PATCH 06/13] Update documentation/sphinx/source/release-notes.rst Co-Authored-By: etschannen <36455792+etschannen@users.noreply.github.com> --- documentation/sphinx/source/release-notes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index da66725957..038b3b42bd 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -31,7 +31,7 @@ Features * The "timeout", "max retry delay", and "retry limit" transaction options are no longer reset when the transaction is reset after a call to ``onError`` (as of API version 610). `(Issue #775) `_ * Added the ``force_recovery_with_data_loss`` command to fdbcli. When a cluster is configured with usable_regions=2, this command will force the database to recover in the remote region. `(PR #1168) `_ * Added a limit to the number of status requests the cluster controller will handle. `(PR #1093) `_ (submitted by tclinken) -* Added a ``coordinator`` process class. Processes with this class can only be used as a coordinator, and ``coordinators auto`` will prefer to chose processes of this class. `(PR #1069) `_ (submitted by tclinken) +* Added a ``coordinator`` process class. Processes with this class can only be used as a coordinator, and ``coordinators auto`` will prefer to choose processes of this class. `(PR #1069) `_ (submitted by tclinken) * The ``consistencycheck`` fdbserver role will check the entire database at most once every week. `(PR #1126) `_ * Added the metadata version key (``\xff/metadataVersion``). The value of this key is sent with every read version. It is intended to help clients cache rarely changing metadata. `(PR #1213) `_ * The ``fdbdr switch`` command verifies a ``dr_agent`` exists in both directions. `(Issue #1220) `_ From d84fcd79c6e31276d1c7147ac333b27851d1b664 Mon Sep 17 00:00:00 2001 From: "A.J. Beamon" Date: Mon, 1 Apr 2019 15:43:29 -0700 Subject: [PATCH 07/13] Update documentation/sphinx/source/release-notes.rst Co-Authored-By: etschannen <36455792+etschannen@users.noreply.github.com> --- documentation/sphinx/source/release-notes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index 038b3b42bd..f2b00b5509 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -55,7 +55,7 @@ Fixes * Standardized datetime string format across all backup and restore command options and outputs. `(PR #1248) `_ * Read workload status metrics would disappear when a storage server was missing. `(PR #1348) `_ * The ``coordinators auto`` command could recruit multiple coordinators with the same zone ID. `(Issue #988) `_ -* The version of a cluster after a restore could have been lower than the restore version, making version stamp operations get smaller. `(PR #1213) `_ +* The data version of a cluster after a restore could have been lower than the restore version, making versionstamp operations get smaller. `(PR #1213) `_ * If a process changed its process class while being rebooted, it could recruited for the cluster controller using its previous process class. `(PR #1350) `_ * Fixed a few thread safety issues. `(PR #1085) `_ * The Go bindings reported an incorrect required version when trying to load an incompatible fdb_c library. `(PR #1053) `_ From 310fe6d4d843819f62bfa5d86af3f53b11bbc42f Mon Sep 17 00:00:00 2001 From: "A.J. Beamon" Date: Mon, 1 Apr 2019 16:29:19 -0700 Subject: [PATCH 08/13] Update documentation/sphinx/source/release-notes.rst Co-Authored-By: etschannen <36455792+etschannen@users.noreply.github.com> --- documentation/sphinx/source/release-notes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index f2b00b5509..f0cb93f10d 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -57,7 +57,7 @@ Fixes * The ``coordinators auto`` command could recruit multiple coordinators with the same zone ID. `(Issue #988) `_ * The data version of a cluster after a restore could have been lower than the restore version, making versionstamp operations get smaller. `(PR #1213) `_ * If a process changed its process class while being rebooted, it could recruited for the cluster controller using its previous process class. `(PR #1350) `_ -* Fixed a few thread safety issues. `(PR #1085) `_ +* Fixed a few thread safety issues with slow task profiling. `(PR #1085) `_ * The Go bindings reported an incorrect required version when trying to load an incompatible fdb_c library. `(PR #1053) `_ * The ``include`` command in fdbcli would falsely include all machines with IP addresses that have the included IP address as a prefix (for example ``include 1.0.0.1`` would also include From b9e782c6b6f50af79b3f578ebcbf07880200c83c Mon Sep 17 00:00:00 2001 From: "A.J. Beamon" Date: Mon, 1 Apr 2019 16:29:26 -0700 Subject: [PATCH 09/13] Update documentation/sphinx/source/release-notes.rst Co-Authored-By: etschannen <36455792+etschannen@users.noreply.github.com> --- documentation/sphinx/source/release-notes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index f0cb93f10d..c692ef6415 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -63,7 +63,7 @@ Fixes have the included IP address as a prefix (for example ``include 1.0.0.1`` would also include ``1.0.0.10``). `(PR #1121) `_ * Restore could crash when reading a file that ends on a block boundary (1MB default). `(PR #1205) `_ -* Java: Successful commits and range reads no longer create ``FDBException`` objects to reduce memory pressure. `(Issue #1235) `_ +* Java: Successful commits and range reads no longer create ``FDBException`` objects, which avoids wasting resources and reduces memory pressure. `(Issue #1235) `_ Status ------ From aa389c2a5c78fba3950cf3aee5126fad241734cd Mon Sep 17 00:00:00 2001 From: Evan Tschannen Date: Mon, 1 Apr 2019 16:35:18 -0700 Subject: [PATCH 10/13] updated release notes based on review feedback --- documentation/sphinx/source/release-notes.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/documentation/sphinx/source/release-notes.rst b/documentation/sphinx/source/release-notes.rst index c692ef6415..f0c49d3f91 100644 --- a/documentation/sphinx/source/release-notes.rst +++ b/documentation/sphinx/source/release-notes.rst @@ -36,8 +36,8 @@ Features * Added the metadata version key (``\xff/metadataVersion``). The value of this key is sent with every read version. It is intended to help clients cache rarely changing metadata. `(PR #1213) `_ * The ``fdbdr switch`` command verifies a ``dr_agent`` exists in both directions. `(Issue #1220) `_ * Transaction logs that cannot commit to disk for more than 5 seconds are marked as degraded. The cluster controller will prefer to recruit transaction logs on other processes before using degraded processes. `(Issue #690) `_ -* The ``memory`` storage engine configuration now uses the ssd engine for transaction log spilling. Transaction log spilling only happens when the transaction logs are using too much memory, so using the memory storage engine for this purpose can cause the process to run out of memory. `(PR #1314) `_ -* Trace logs can be output as JSON instead of XML using the ``--trace_format`` command line option. `(PR #976) `_ (by mpilman) +* The ``memory`` storage engine configuration now uses the ssd engine for transaction log spilling. Transaction log spilling only happens when the transaction logs are using too much memory, so using the memory storage engine for this purpose can cause the process to run out of memory. Existing clusters will NOT automatically change their configuration. `(PR #1314) `_ +* Trace logs can be output as JSON instead of XML using the ``--trace_format`` command line option. `(PR #976) `_ (by atn34) * Added ``modify`` command to fdbbackup for modifying parameters of a running backup. `(PR #1237) `_ * Added 'header' parameter to blobstore backup URLs for setting custom HTTP headers. `(PR #1237) `_ @@ -56,8 +56,8 @@ Fixes * Read workload status metrics would disappear when a storage server was missing. `(PR #1348) `_ * The ``coordinators auto`` command could recruit multiple coordinators with the same zone ID. `(Issue #988) `_ * The data version of a cluster after a restore could have been lower than the restore version, making versionstamp operations get smaller. `(PR #1213) `_ -* If a process changed its process class while being rebooted, it could recruited for the cluster controller using its previous process class. `(PR #1350) `_ * Fixed a few thread safety issues with slow task profiling. `(PR #1085) `_ +* Changing the class of a process would not change its preference for becoming the cluster controller. `(PR #1350) `_ * The Go bindings reported an incorrect required version when trying to load an incompatible fdb_c library. `(PR #1053) `_ * The ``include`` command in fdbcli would falsely include all machines with IP addresses that have the included IP address as a prefix (for example ``include 1.0.0.1`` would also include From f5de52de91e91411719e2b5af6838df76586f837 Mon Sep 17 00:00:00 2001 From: Evan Tschannen Date: Mon, 1 Apr 2019 16:38:25 -0700 Subject: [PATCH 11/13] fix: cancel the previous log system recruitment before calling newEpoch, to avoid multiple actors attempting to modify oldLogSystem at the same time --- fdbserver/masterserver.actor.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fdbserver/masterserver.actor.cpp b/fdbserver/masterserver.actor.cpp index 6554e01032..b628d2b313 100644 --- a/fdbserver/masterserver.actor.cpp +++ b/fdbserver/masterserver.actor.cpp @@ -326,10 +326,12 @@ ACTOR Future newTLogServers( Reference self, RecruitFromConfig Future fRemoteWorkers = brokenPromiseToNever( self->clusterController.recruitRemoteFromConfiguration.getReply( RecruitRemoteFromConfigurationRequest( self->configuration, remoteDcId, recr.tLogs.size() * std::max(1, self->configuration.desiredLogRouterCount / std::max(1, recr.tLogs.size())), exclusionWorkerIds) ) ); self->primaryLocality = self->dcId_locality[recr.dcId]; + self->logSystem = Reference(); Reference newLogSystem = wait( oldLogSystem->newEpoch( recr, fRemoteWorkers, self->configuration, self->cstate.myDBState.recoveryCount + 1, self->primaryLocality, self->dcId_locality[remoteDcId], self->allTags, self->recruitmentStalled ) ); self->logSystem = newLogSystem; } else { self->primaryLocality = tagLocalitySpecial; + self->logSystem = Reference(); Reference newLogSystem = wait( oldLogSystem->newEpoch( recr, Never(), self->configuration, self->cstate.myDBState.recoveryCount + 1, self->primaryLocality, tagLocalitySpecial, self->allTags, self->recruitmentStalled ) ); self->logSystem = newLogSystem; } From 45c466e26993c9cbd76c3dd292009dd308a409bc Mon Sep 17 00:00:00 2001 From: Alex Miller Date: Mon, 1 Apr 2019 17:25:08 -0700 Subject: [PATCH 12/13] Open incrementalDelete files with OPEN_UNBUFFERED This fixes crashes from AsyncFileWinASIO refusing to open a file that didn't have OPEN_UNBUFFERED. --- fdbrpc/IAsyncFile.actor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fdbrpc/IAsyncFile.actor.cpp b/fdbrpc/IAsyncFile.actor.cpp index 64b88bc14c..7a4854e89d 100644 --- a/fdbrpc/IAsyncFile.actor.cpp +++ b/fdbrpc/IAsyncFile.actor.cpp @@ -88,7 +88,7 @@ ACTOR static Future incrementalDeleteHelper( std::string filename, bool mu state bool exists = fileExists(filename); if(exists) { - Reference f = wait(IAsyncFileSystem::filesystem()->open(filename, IAsyncFile::OPEN_READWRITE | IAsyncFile::OPEN_UNCACHED, 0)); + Reference f = wait(IAsyncFileSystem::filesystem()->open(filename, IAsyncFile::OPEN_READWRITE | IAsyncFile::OPEN_UNCACHED | IAsyncFile::OPEN_UNBUFFERED, 0)); file = f; int64_t fileSize = wait(file->size()); From 8714394d42d4532d14f40a787ce264bbe32f90f3 Mon Sep 17 00:00:00 2001 From: Evan Tschannen Date: Mon, 1 Apr 2019 18:37:40 -0700 Subject: [PATCH 13/13] =?UTF-8?q?increase=20the=20priority=20of=20the=20cl?= =?UTF-8?q?ient=E2=80=99s=20version=20batch=20timeout,=20so=20that=20we=20?= =?UTF-8?q?prefer=20issuing=20the=20batch=20over=20other=20possible=20work?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- fdbclient/NativeAPI.actor.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fdbclient/NativeAPI.actor.cpp b/fdbclient/NativeAPI.actor.cpp index 6c1d1a9164..42581e9527 100644 --- a/fdbclient/NativeAPI.actor.cpp +++ b/fdbclient/NativeAPI.actor.cpp @@ -2945,7 +2945,7 @@ ACTOR Future readVersionBatcher( DatabaseContext *cx, FutureStream< std::p if (requests.size() == CLIENT_KNOBS->MAX_BATCH_SIZE) send_batch = true; else if (!timeout.isValid()) - timeout = delay(batchTime, cx->taskID); + timeout = delay(batchTime, TaskProxyGetConsistentReadVersion); } when(wait(timeout.isValid() ? timeout : Never())) { send_batch = true;