ea12a54946
Rename DISK_QUEUE_MAX_TRUNCATE_EXTENTS -> ..._BYTES
...
So as to not make filesystem assumptions. This knob did technically
appear in (only the) 6.1.5 release, but this feature was broken 6.1.5,
so thus impossible to use anyway.
2019-05-10 18:26:22 -10:00
c95d09f9fd
Convert truncate(0) to truncate(4KB) on Windows.
...
Blindly, in case Windows doesn't like 0 length truncates too.
2019-05-10 14:55:11 -10:00
c502ed3d15
Fix a variety of problems stemming from a wait() being added to push().
...
And that this code was previously insufficiently tested.
2019-05-10 14:55:11 -10:00
fbc4e7b351
Add a release note
2019-05-08 23:10:18 -10:00
510b0b2fcd
Fix DiskQueue not replaceFile'ing frequently enough for the final time.
2019-05-08 23:08:25 -10:00
c6c33a4daa
Make replaceFile more likely to be tested.
2019-05-08 21:23:42 -10:00
0d0f54d1e6
Fix IAsyncFileSystem::open() flags to stop a crash.
...
OPEN_ATOMIC_WRITE_AND_CREATE was missing a required OPEN_CREATE.
I'm honestly baffled how this was missed in testing.
2019-05-08 21:22:40 -10:00
b50926c792
replaceFile is truncate(0) on windows
2019-05-08 21:22:14 -10:00
e4ba2f5788
Add an ending TraceEvent.
2019-05-08 12:35:12 -10:00
c093017c2f
Add a TraceEvent and release note.
2019-05-08 12:34:25 -10:00
0685e6c1c7
Avoid large truncates in the DiskQueue.
...
And instead create a new file while incrementally truncating the old one
down. This avoids queueing up a massive number of filesystem metadata
operations in one call, thus flooding the disk with requests and
stalling out all other filesystem operations.
This sets the knobs so that a truncate of >10GB causes us to create a
new file rather than trying to truncate the old one.
2019-05-08 12:33:31 -10:00
36dfbf4fb3
Only truncate DiskQueues down to TLOG_HARD_LIMIT*2.
...
DiskQueue shrinking was implemented for spill-by-reference, as now
a DiskQueue could grow "unboundedly" large.
Without a minimum file size, write burst workloads would cause the
DiskQueue to shrink down to 100MB, and then grow back to its usual ~4GB
size in a cycle. File growth means filesystem metadata mutations, which
we'd prefer to avoid if possible since they're more unpredicatble in
terms of latency.
In a healthy cluster, the TLog never spills, so the disk of a single
DiskQueue file should stay less than 2*TLOG_SPILL_THRESHOLD. In the
worst case of spill-by-value, the DiskQueue could grow to
2*TLOG_HARD_LIMIT. Therefore, having this limit will cause DiskQueue
shrinking to never behave sub-optimally for spill-by-value, and will
cause the DiskQueue files to return to the optimal size with
spill-by-reference.
2019-05-08 12:33:31 -10:00
a269a784cc
Convert push() into an actor.
2019-05-08 12:33:31 -10:00
68c773987c
Merge pull request #1544 from etschannen/release-6.1
...
The team tracker does not provide data movement priority information for non-failure related data movement
2019-05-08 11:39:17 -07:00
f7b74be520
Merge pull request #1523 from AlvinMooreSr/cmake-vendor
...
Fixed CMake Vendor Config
2019-05-08 08:21:42 -07:00
0d4fcd526b
Merge pull request #1543 from bnamasivayam/release-6.1
...
Add a workload to trigger repeated recoveries.
2019-05-07 18:39:29 -07:00
d45e7bf0b1
Addressed review comments
2019-05-07 17:19:59 -07:00
30ebdc3b47
updated release notes
2019-05-07 17:15:45 -07:00
d9a4553270
fix: The team tracker does not provide data movement priority information for non-failure related data movement
2019-05-07 17:06:54 -07:00
5d824f5fbc
Address review comments
2019-05-07 17:06:52 -07:00
adc0800b93
Merge pull request #1541 from etschannen/feature-trace-safety
...
Fixed a crash when memory tracking trace events are called recursively
2019-05-07 17:06:28 -07:00
c1a58bd27a
updated release notes
2019-05-07 17:04:46 -07:00
a0cc3d98a1
Add a workload to trigger repeated recoveries.
2019-05-06 18:16:44 -07:00
aefd68e1e7
The memory tracking trace events can crash if the memory that is being allocated is coming from a trace event itself. Specifically TDMetrics within trace events uses fast allocated memory. Trace events are supposed to be short lived, so this commits adds a global count of outstanding trace events, and disables memory tracking when a trace event exists.
2019-05-06 17:41:32 -07:00
61018dae71
Merge branch 'release-6.1' of github.com:apple/foundationdb into release-6.1
2019-05-06 17:37:35 -07:00
93eb2a9395
Merge pull request #1527 from alexmiller-apple/tstlog-6.1
...
Spill-by-reference knob + TLog6.0 Spilled Peek deprioritization
2019-05-03 17:19:45 -07:00
c918b21137
Deprioritize spilled peeks in spill-by-value, and improve its logic.
...
This deprioritizes before calling peekMessagesFromMemory, which should
improve the memory usage of the TLog, and makes sure to keep txsTag
peeks at a high priority to help recoveries stay fast.
2019-05-03 15:27:11 -07:00
4052f3826a
Add a knob to limit the number of commits indexed per key.
...
Theoretically, we could spill 20MB of 22B mutations for one key, which
would generate a very long value being stored in SQLite, and very
inefficiently read back. This stops that from being a problem, at the
cost of some extra write calls.
2019-05-03 15:27:10 -07:00
12088119d2
Merge pull request #1517 from alexmiller-apple/tstlog-6.1
...
Add a knob to limit amount of data read from sqlite for one PeekRequest.
2019-05-03 11:01:11 -07:00
085a9242e4
Changed specification of the path to the License.txt file since previous value resulted in search at top level (/) directory.
2019-05-03 00:12:07 -07:00
f308549be7
Commented out Readme file for mac os until working within packaging
2019-05-02 23:45:16 -07:00
345e8496ab
Fixed the cmake package vendor and email information in order to prevent cpack errors
2019-05-02 22:14:18 -07:00
f4e48c3851
Add a knob to limit amount of data read from sqlite for one PeekRequest.
...
This prevents peeking from degrading over time if there are a very large
number of SpilledData entries for one particular tag.
2019-05-02 17:26:45 -07:00
c91ac03ec6
LogRouterStats did not need to be a separate struct
2019-05-02 17:24:39 -07:00
8590b710bf
added additional logging on the logs and log routers
2019-05-02 17:24:39 -07:00
e6407806af
LogRouterStats did not need to be a separate struct
2019-05-02 16:43:20 -07:00
f0fe0d7858
added additional logging on the logs and log routers
2019-05-02 16:16:25 -07:00
d54ba497f0
Merge pull request #1514 from etschannen/post-release-cleanup-6.1.4
...
Post release cleanup 6.1.4
2019-04-30 18:23:50 -07:00
3835fb8e41
update installer WIX GUID following release
2019-04-30 18:23:16 -07:00
cf5e2759d0
update versions target to 6.1.5
2019-04-30 18:23:16 -07:00
0cbbd5070a
Merge pull request #1513 from etschannen/release-6.1
...
updated documentation for 6.1.4
2019-04-30 17:01:49 -07:00
31657b91e0
updated documentation for 6.1.4
2019-04-30 17:00:58 -07:00
2adff860ed
Merge pull request #1512 from etschannen/prepare-release-6.1.4
...
update installer WIX GUID following release
2019-04-30 16:59:36 -07:00
97bb037d2f
update installer WIX GUID following release
2019-04-30 16:59:03 -07:00
1d3d411094
Merge pull request #1511 from etschannen/post-release-cleanup-6.1.3
...
Post release cleanup 6.1.3
2019-04-30 16:55:05 -07:00
926e8d2cc6
update installer WIX GUID following release
2019-04-30 16:54:31 -07:00
f781582ea3
update versions target to 6.1.4
2019-04-30 16:54:31 -07:00
ef2b2cd585
Merge branch 'release-6.1' of github.com:etschannen/foundationdb into release-6.1
2019-04-30 16:32:49 -07:00
743b2f51ad
Merge branch 'release-6.1' of github.com:apple/foundationdb into release-6.1
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
2019-04-30 16:31:35 -07:00
6124a7e145
Update fdbclient/ManagementAPI.actor.cpp
2019-04-30 16:30:19 -07:00
Alex Miller