Commit Graph

348 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard 2391e58fb2 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-21 10:09:15 -07:00
Josh Slocum 34e6a8f942
Merge pull request #7399 from sfc-gh-jslocum/bg_tenant_improvements
Bg tenant improvements
2022-06-17 11:19:41 -05:00
Josh Slocum b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00
Andrew Noyes 207e0bc105
Fix a few places we weren't doing exponential backoff (#7349)
* Fix a few places we weren't doing exponential backoff

We re-create the transaction every iteration of each of these retry
loops, so we need to manage exponential backoff here ourselves.

Closes #7301

* Remove former Backoff definition
2022-06-13 13:18:58 -07:00
sfc-gh-tclinkenbeard df71a49bf6 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-13 10:03:10 -07:00
sfc-gh-tclinkenbeard 75c858eb2c Differentiate between different quotas in GlobalTagThrottling workload 2022-06-13 10:01:55 -07:00
Lukas Joswiak 49e4305760 Initialize GlobalConfig on Database clone 2022-05-27 10:25:29 -07:00
Vishesh Yadav 9173e2e19b Move GlobalConfig to DatabaseContext 2022-05-09 14:54:51 -07:00
Sreenath Bodagala 2102ed1eaa - Remove "stale_version_vector" error code. 2022-05-03 21:56:11 +00:00
Ray Jenkins dc9e782ccc
OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
Ray Jenkins 1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. (#6855)" (#6941)
This reverts commit 5df3bac110.
2022-04-25 09:29:56 -05:00
A.J. Beamon 1352083d4c
Merge pull request #6884 from sfc-gh-clin/deprecate-speical-keys
Remove the client profiling special keys and update related documentations
2022-04-21 21:51:16 -07:00
Chaoguang Lin 4b1d22d684 Polish the documentation 2022-04-21 19:28:50 -07:00
Chaoguang Lin 879eb98dec Rename registerSpecialKeySpaceModule to registerSpecialKeysImpl 2022-04-21 12:54:34 -07:00
Ray Jenkins 5df3bac110
Migrate to OpenTelemetry tracing. (#6855) 2022-04-20 09:26:37 -05:00
Chaoguang Lin c0264a8522 Remove the client profiling special keys and update related documentations 2022-04-18 17:54:50 -07:00
Dan Lambright 1b3b4166c6
Merge branch 'main' into vv 2022-04-08 17:18:13 -04:00
Josh Slocum 6276cebad9
Blob integration (#6808)
* Fixing leaked stream with explicit notify failed before destructor

* better logic to prevent races in change feed fetching

* Found new race that makes assert incorrect

* handle server overloaded in initial read from fdb

* Handling more blob error types in granule retry

* Fixing rollback metadata problem, added better debugging

* Fixing version race when fetching change feed metadata

* Better racing split request handling

* fixing assert

* Handle change feed popped check in the blob worker

* fix: do not use a RYW transaction for a versionstamp because of randomize API version (#6768)

* more merge conflict issues

* Change feed destroy fixes

* Fixing change feed destroy and move race

* Check error condition in BG file req

* Using relative endpoints for blob worker interface

* Fixing bug in previous fix

* More destroy and move race fixes

* Don't update empty version on destroy in case it gets rolled back. moved() and removing will take care of ensuring it is not read

* Bug fix (#6796)

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* Fixing durability issue with moving and destroying change feeds

* Adding fix for not fully deleting files for a granule that child granules need to re-snapshot

* More destroy and move races

* Fixing change feed destroy and pop races

* Renaming bg prune to purge, and adding a C api and unit test for it

* more cleanup

* review comments

* Observability for granule purging

* better handling for change feed not registered

* Fixed purging bugs (#6815)

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* fixed a few purging bugs

Co-authored-by: Evan Tschannen <evan.tschannen@snowflake.com>
2022-04-08 14:15:25 -07:00
Dan Lambright 5bdc525353
Merge branch 'main' into vv 2022-04-08 13:16:04 -04:00
Jingyu Zhou 4fd414a8ed Merge branch 'main' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
2022-03-31 09:38:36 -07:00
Jingyu Zhou cfcf0f152c Merge branch 'main-4a085fc84' into vv
Fix Conflicts:
	fdbclient/NativeAPI.actor.cpp
	fdbserver/ClusterRecovery.actor.cpp
	fdbserver/MasterInterface.h
	fdbserver/masterserver.actor.cpp
	flow/error_definitions.h
2022-03-30 22:28:06 -07:00
Jingyu Zhou b34f4052cd Merge branch 'main-f28dfc12b' into vv
Fix Conflicts:
	fdbclient/MultiVersionTransaction.actor.cpp
	fdbclient/NativeAPI.actor.cpp
	fdbclient/NativeAPI.actor.h
	fdbserver/storageserver.actor.cpp
2022-03-30 21:01:25 -07:00
Jingyu Zhou 00b57d4cce Merge branch 'main-67eba5ec7' into vv
Fix Conflicts:
	fdbclient/CommitProxyInterface.h
	fdbclient/NativeAPI.actor.cpp
	fdbclient/StorageServerInterface.h
	fdbserver/CommitProxyServer.actor.cpp
	fdbserver/storageserver.actor.cpp
2022-03-30 20:05:55 -07:00
Jingyu Zhou e9659b5dd4 Merge branch 'master-PR-6500' into vv
Fix Conflicts:
	fdbclient/CommitProxyInterface.h
	fdbclient/NativeAPI.actor.cpp
	fdbserver/masterserver.actor.cpp
2022-03-30 14:53:49 -07:00
Jon Fu f266695a76 clear shared state map entries on version upgrade and use currentProtocolVersion instead of piping version through function parameters 2022-03-30 15:28:55 -04:00
Jon Fu 9d0a520fe0 add ProtocolVersion to shared state and state cluster map 2022-03-30 13:28:06 -04:00
Jon Fu 53d252fda0 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-29 17:23:08 -04:00
Jon Fu bdbe78e48f address code review comments 2022-03-29 17:20:22 -04:00
Josh Slocum b92c6ea92c granule parallelism, metrics, and other improvements to bg client bindings 2022-03-24 13:44:09 -05:00
Jon Fu 7c3151142e Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-23 16:41:33 -04:00
Jon Fu 4466e00716 Fix multithreaded GRV cache code and enable grv caching in multithreaded ctest suite 2022-03-23 16:09:12 -04:00
Josh Slocum f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Josh Slocum 37e7c80f26 Merge branch 'main' into blob_integration 2022-03-17 18:45:42 -05:00
A.J. Beamon 05495908b8 Implement some tenant tests 2022-03-17 12:10:18 -07:00
Trevor Clinkenbeard 765e018afb
Merge pull request #6580 from sfc-gh-tclinkenbeard/const-smoother
Mark `Smoother::smooth*` methods `const`
2022-03-16 13:43:25 -07:00
Jon Fu 9731d96797 Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-03-16 14:32:33 -04:00
Jon Fu 7ab205391b Revert "remove all multi-version client code related to grv cache"
This reverts commit 6d05ecffc2.
2022-03-16 14:26:36 -04:00
Jingyu Zhou 2acc222d9a Detect stale GRV proxy replies
Each GetReadVersionReply now has the GRV proxy ID that sent it so that the
client can detect old GRV proxy's replies and avoid apply stale version vector
changes.
2022-03-15 14:23:07 -07:00
Josh Slocum ce5b567971 Cleanup the cleanup 2022-03-15 15:02:28 -05:00
A.J. Beamon 0536e56725 The tenant cache uses a simpler eviction policy. Some various improvement around the use of futures. Add documentation for C API and fix inaccurate statement in special keys documentation. 2022-03-15 09:23:30 -07:00
A.J. Beamon 68069c9784 Undo some unused changes. Fix tenant cache eviction. 2022-03-15 09:23:30 -07:00
A.J. Beamon ecccfd0868 Add cache invalidation to tenant cache. Send tenant ID along with tenant name in requests to validate that the tenant hasn't changed. Fix a few bugs. 2022-03-15 09:23:30 -07:00
A.J. Beamon 2a21126028 Don't apply read prefixes on the client. Cache tenant data locally. 2022-03-15 09:23:30 -07:00
A.J. Beamon c635dcd3ad Add tenant support in the FDB native client 2022-03-15 09:21:27 -07:00
Jingyu Zhou 800478ad5e Add a new client error code stale_version_vector 2022-03-14 20:50:12 -07:00
Josh Slocum 98eeaac1b7 More debugging and fixing a stuck merge cursor 2022-03-14 11:57:56 -05:00
sfc-gh-tclinkenbeard 9aca5f8011 Mark several methods const 2022-03-11 00:47:35 -04:00
Tao Lin e2c7c30faf
GetMappedRange support serializable & check RYW & continuation (#6181) 2022-03-10 10:05:44 -08:00
Josh Slocum b21d0943b9 client-focused cleanup 2022-03-09 10:01:25 -06:00