Commit Graph

2581 Commits

Author SHA1 Message Date
Tao Lin 9b0a9c4503
Return error when getRangeAndFlatMap has more & Improve simulation tests (#6029) 2021-12-03 12:50:07 -08:00
Evan Tschannen b11ae4dae8
Merge pull request #5910 from sfc-gh-jslocum/bg_bindings
Blob Granule C bindings
2021-12-02 11:40:26 -08:00
sfc-gh-tclinkenbeard 464d9488ef Merge remote-tracking branch 'origin/master' into fix-unused-warnings 2021-12-01 23:52:09 -08:00
FoundationDB CI ca5d5ac942
apply formatting with clang 13
Signed-off-by: FoundationDB CI <foundationdb_ci@apple.com>
2021-12-02 05:13:59 +00:00
sfc-gh-tclinkenbeard 90ced244eb Fix -Wunused-but-set-variable warnings 2021-12-01 18:15:53 -08:00
Josh Slocum 9cb6fb5114 fixing unrelated code formatting 2021-12-01 17:20:11 -06:00
Josh Slocum a82845af43 Merge branch 'master' into bg_bindings 2021-12-01 16:55:28 -06:00
Josh Slocum 0f2f5bc0b6 Cleanup of ThreadResult 2021-12-01 16:24:28 -06:00
Josh Slocum 7f4fcc8c2c Added FDBResult and made readBlobGranules use it 2021-12-01 16:22:05 -06:00
sfc-gh-tclinkenbeard ec64890ac1 Remove some usages of PRId64 by using fmt library 2021-11-30 23:35:36 -08:00
A.J. Beamon c47535245b
Merge pull request #6033 from sfc-gh-ajbeamon/improved-client-db-logging
Client logging improvements
2021-11-29 13:23:10 -08:00
Trevor Clinkenbeard 6429b82796
Merge pull request #6053 from RenxuanW/fromHostname
Change member variable fromHostname to type bool.
2021-11-29 13:13:21 -08:00
A.J. Beamon b8bd89f88d Shorten the name of external client threads. Add a thread name for trace logging threads. 2021-11-29 09:57:10 -08:00
A.J. Beamon 264c75b9a6 Add some extra client logging details:
1. Add a trace event when a database is created and move the cluster file / connection string from ClientStart to the new trace event
2. Add a detail for the path to the image being loaded
3. Add a detail for whether a client library is primary or not
4. Set a thread name for each external client thread that includes the release version
2021-11-29 09:57:10 -08:00
Renxuan Wang 09fedc429a Remove unnecessary boost/bind.hpp.
Complement of #6026.
2021-11-24 16:33:05 -08:00
Ata E Husain Bohra 0962fcb243 Override commit/grv proxies_count if mutation supplied new value is -1
Patch improves on handling scenarios where either commit or grv proxies
value is update to -1 OR `proxies_count` is being reset.
The code splits the proxies between two proxies by ensuring for invalid
input configuration, the min (read as 1) proxies gets provisioned, otherwise,
the split is done based on input values

Patch handles the scenario where mutation supplied values to update grv_proxies
and/or commit_proxies is -1, however, the total proxy count > 1,
uses DEFAULT_COMMIT_GRV_PROXIES_RATIO to split proxies between
grv_proxies & commit_proxies.
2021-11-24 12:52:31 -08:00
Renxuan Wang 46d17d748f Change member variable fromHostname to type bool.
So that it can be serialized.
2021-11-23 14:25:02 -10:00
Renxuan Wang 22e34bd6b9 Replace <boost/bind.hpp> with <boost/bind/bind.hpp>.
This eliminates many useless warnings when compiling.
`#pragma message: The practice of declaring the Bind placeholders (_1, _2, ...) in the global namespace is deprecated. Please use <boost/bind/bind.hpp> + using namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to retain the current behavior.`
2021-11-18 14:00:13 -08:00
Lukas Joswiak 18243351e7 Fix possible data race
Transactions (created on a separate thread) can read the `globals` field
at the same time as `setGlobal` is called on the main thread, causing a
potential race. TSAN surfaced this issue.
2021-11-18 10:16:20 -08:00
negoyal 8b0938c7b3 Added a TODO 2021-11-17 09:45:50 -08:00
Steve Atherton 3caca74ac2 Merge commit 'fd707c6d7ee80de6d9fda5796da2d0add10abd79' into bit-flipping-workload 2021-11-16 21:54:27 -08:00
Markus Pilman b2019cd4f2
Merge pull request #5992 from sfc-gh-mpilman/features/fmt
added fmt dependency to flow
2021-11-16 18:58:55 -07:00
Steve Atherton 035e0d6e52
Merge branch 'master' into bit-flipping-workload 2021-11-16 14:42:22 -08:00
Markus Pilman b1633b90f1 Added fmt to flow 2021-11-16 12:03:49 -07:00
Steve Atherton 867999a41a Rename wrong_format_version to unsupported_format_version. 2021-11-16 03:25:54 -08:00
Renxuan Wang 4630b0ccea Move DNS mock from SimExternalConnection to Sim2.
This is a revise PR of #5934. In simulation, we don't have direct access to SimExternalConnection.
2021-11-15 17:02:51 -08:00
Evan Tschannen 964d0209ca
Merge pull request #5637 from sfc-gh-ljoswiak/features/data-loss-prevention
Data loss protection when joining new cluster
2021-11-15 15:26:32 -08:00
Jingyu Zhou 02d0c43bc2
Merge pull request #5982 from sfc-gh-tclinkenbeard/improve-error-descriptions
Make snapshot errors more descriptive
2021-11-15 13:18:19 -08:00
Evan Tschannen a546fb63ea
Merge pull request #5985 from sfc-gh-etschannen/feature-changefeed-empty-versions
Added a whenAtLeast function to change feeds to efficiently learn about empty versions
2021-11-15 10:51:28 -08:00
Markus Pilman daf6dc22d4
Merge pull request #5959 from mpilman/features/apple-silicon-3
FDB compiles on Apple Sillicon
2021-11-15 11:21:28 -07:00
Evan Tschannen 94a51e57a5 Merge branch 'master' into feature-changefeed-empty-versions
# Conflicts:
#	fdbclient/StorageServerInterface.h
2021-11-14 19:13:05 -08:00
Evan Tschannen 6909754b21 changefeeds now have a whenAtLeast function for efficiently learning when the version has updated but no mutations have been committed 2021-11-14 19:08:46 -08:00
sfc-gh-tclinkenbeard dc756228f2 Make snapshot errors more descriptive 2021-11-14 13:46:17 -08:00
Steve Atherton 508429f30d
Redwood chunked file growth and low priority IO starvation prevention (#5936)
* Redwood files now growth in large page chunks controlled by a knob to reduce truncate() calls for expansion.   PriorityMultiLock has limit on consecutive same-priority lock release.  Increased Redwood max priority level to 3 for more separation at higher BTree levels.

* Simulation fix, don't mark certain IO timeout errors as injected unless the simulated process has been set to have an unreliable disk.

* Pager writes now truncate gradually upward, one chunk at a time, in response to writes, which wait on only the necessary truncate operations.   Increased buggified chunk size because truncate can be very slow in simulation.

* In simulation, ioTimeoutError() and ioDegradedOrTimeoutError() will wait until at least the target timeout interval past the point when simulation is sped up.

* PriorityMultiLock::toString() prints more info and is now public.

* Added queued time to PriorityMultiLock.

* Bug fix to handle when speedUpSimulation changes later than the configured time.

* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.

* Renamed updatingInPlace to updatingDeltaTree for clarity.  Inlined switchToLinearMerge() since it is only used in one place.

* Updated extendToCover to be more clear by passing in the old extension future as a parameter.  Fixed initialization warning.
2021-11-12 13:47:07 -08:00
Josh Slocum 329091e14f Merge branch 'master' into bg_bindings 2021-11-11 10:13:37 -06:00
Markus Pilman 28dde27cb1 Fix Linux compiler errors 2021-11-11 08:49:51 -07:00
Markus Pilman 0e6dd46f5a Correct code formatting 2021-11-11 08:38:25 -07:00
Josh Slocum b8ac4213a1 Switched BG APIs to transaction instead of database 2021-11-11 08:59:06 -06:00
Markus Pilman 5af465aa29 FDB compiles on Apple Sillicon 2021-11-10 20:05:38 -07:00
Lukas Joswiak 069a04c5e5 Removed outdated definition 2021-11-10 13:33:49 -08:00
Lukas Joswiak 1da288822f Remove distributed trace database option 2021-11-10 13:33:49 -08:00
Lukas Joswiak 8bc0c3e8a2 Convert distributed trace sampling rate to a database option 2021-11-10 13:33:49 -08:00
Lukas Joswiak c93052121f Fix issue where transaction spans would not be recorded 2021-11-10 13:33:49 -08:00
Lukas Joswiak 5ed7abdc05 Fix bug where spans were not being properly constructed 2021-11-10 13:33:49 -08:00
Josh Slocum 2e5699c6d3 bump protocol version on master to 7.1 2021-11-10 09:38:51 -06:00
Tao Lin fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
Lukas Joswiak 3988b11fd6 Cleanup 2021-11-09 12:29:48 -08:00
Lukas Joswiak 30867750b5 Add protection against storage and tlog data deletion when joining a new cluster 2021-11-09 12:29:47 -08:00
Renxuan Wang 85dff214a4 Address comments. 2021-11-04 11:42:28 -07:00
Renxuan Wang f15ceb5489 Add Hostname struct, and fromHostname in NetworkAddress struct. 2021-11-04 11:42:28 -07:00