And instead create a new file while incrementally truncating the old one
down. This avoids queueing up a massive number of filesystem metadata
operations in one call, thus flooding the disk with requests and
stalling out all other filesystem operations.
This sets the knobs so that a truncate of >10GB causes us to create a
new file rather than trying to truncate the old one.
DiskQueue shrinking was implemented for spill-by-reference, as now
a DiskQueue could grow "unboundedly" large.
Without a minimum file size, write burst workloads would cause the
DiskQueue to shrink down to 100MB, and then grow back to its usual ~4GB
size in a cycle. File growth means filesystem metadata mutations, which
we'd prefer to avoid if possible since they're more unpredicatble in
terms of latency.
In a healthy cluster, the TLog never spills, so the disk of a single
DiskQueue file should stay less than 2*TLOG_SPILL_THRESHOLD. In the
worst case of spill-by-value, the DiskQueue could grow to
2*TLOG_HARD_LIMIT. Therefore, having this limit will cause DiskQueue
shrinking to never behave sub-optimally for spill-by-value, and will
cause the DiskQueue files to return to the optimal size with
spill-by-reference.
dmcs was noted as deprecated in its startup message script many years
ago (2016), and dmcs itself was *actually* deprecated even years before
that -- 'dmcs' itself only executes 'mcs' with the .NET 4.0 SDK enabled
anyway.
Since CMake will be the new primary build system, let's go ahead and
drop support for dmcs entirely in this setup.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
When using the CMake build system with tools such as Ninja (instead of
Make), they have very concise output, and the general "UX flow" involves
only outputting warnings that should be fixed or errors that should stop
the build. Furthermore, when building things in CI systems and looking
at build logs, etc, it's often vastly easier to to hunt down errors when
only relevant stuff is output.
With this change, the output from coveragetool.exe is suppressed, which
accounts for a lot of spurious output that isn't necessary when running
these builds.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This deprioritizes before calling peekMessagesFromMemory, which should
improve the memory usage of the TLog, and makes sure to keep txsTag
peeks at a high priority to help recoveries stay fast.
Theoretically, we could spill 20MB of 22B mutations for one key, which
would generate a very long value being stored in SQLite, and very
inefficiently read back. This stops that from being a problem, at the
cost of some extra write calls.
chaoguang