Commit Graph

1965 Commits

Author SHA1 Message Date
Jon Fu 631538537f Merge branch 'main' of github.com:apple/foundationdb into tenant-list-filter 2022-11-09 10:45:45 -08:00
Jingyu Zhou ec471aa63c
Merge pull request #8684 from jzhou77/fix
Fix ASan failure heap use after free in status json generation
2022-11-03 18:48:26 -07:00
Josh Slocum d68cd3493e
making read caching configurable for blob store (#8607) 2022-11-03 16:42:46 -05:00
Jingyu Zhou 2a5ff5bba5 Update 7.1 release notes 2022-11-03 12:45:01 -07:00
Jon Fu 8ef0411b32 address code review comments and introduce offset parameter 2022-11-03 11:39:39 -07:00
Jingyu Zhou 80afd5e449 Add 6.3.25 release notes
Documentation only change.
2022-11-02 15:52:43 -07:00
Jingyu Zhou 81e0e7c4df Add 7.1.24, 7.1.25 release notes 2022-11-02 15:16:19 -07:00
Jon Fu d95eb4dd71 Merge branch 'main' of github.com:apple/foundationdb into tenant-list-filter 2022-10-28 10:00:42 -07:00
Alex Moundalexis 67049518b9
updated copyright year on web site 2022-10-27 15:05:52 -04:00
Jon Fu f21fcf67ce initial commit to allow tenant list filtering in metacluster 2022-10-25 12:01:29 -07:00
Vaidas Gasiunas feee5ddb80 Updating API Tester documentation 2022-10-21 14:21:55 +02:00
Nim Wijetunga 025359a974
Add Options to restore user/system keys in backup agent (#8497)
* add options to restore user/system keys

* format help commands

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update documentation/sphinx/source/backups.rst

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbbackup/backup.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbbackup/backup.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* address pr comments

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-10-20 09:58:00 -07:00
Zhe Wu 550e1e86e8 Also add fetch_consistency_scan_info_timeout to mr-status-json-schemas.rst.inc 2022-10-17 15:35:32 -07:00
Kevin Hoxha 2f898fb006 fdbcli: Add documentation for setknob/getknob commands 2022-10-11 15:32:01 -07:00
Jon Fu 461e42bfe1 restructure updater code and add capacity check in metacluster management workload 2022-09-29 16:24:02 -07:00
Jon Fu a29b853979 add assigned_cluster to tenant command hints in fdbcli 2022-09-29 12:18:30 -07:00
Sam Gwydir 61ce598aef Specify cert and key files separately 2022-09-26 11:59:42 -07:00
Sam Gwydir a5a0028e65 document TLS cert and key file behavior change. 2022-09-26 11:37:24 -07:00
A.J. Beamon 97a325adab Add an fdbcli command to get tenant group metadata 2022-09-22 13:24:21 -07:00
A.J. Beamon 64c8e0e2b1 Convert fdbcli tenant commands to a single command with subcommands. Update the structure of fdbcli tenant tests. 2022-09-21 16:22:58 -07:00
Markus Pilman 9c062a220e clarify statistics of TestHarness 2022-09-21 09:29:49 -06:00
Markus Pilman a8cec4f05c clarify how code probes are reported 2022-09-21 09:27:28 -06:00
Markus Pilman 90b48e862e
Apply suggestions from code review
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
Co-authored-by: Bharadwaj V.R <bharadwaj.vr@snowflake.com>
2022-09-21 08:28:01 -06:00
Markus Pilman 24aced6d4a Upgrade sphinx and document test harness and code probes 2022-09-20 16:43:02 -06:00
Jingyu Zhou 890681a52f Added 7.1.22, 7.1.23 release notes 2022-09-19 20:35:32 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Junhyun Shim dcdb347ff1 Address review comments 2022-09-15 21:59:54 +02:00
Junhyun Shim f57c5f1f16 Further formatting and phrasing changes 2022-09-15 08:13:21 +02:00
Junhyun Shim e6e67a271c Rephrase client auth requirement 2022-09-14 19:49:59 +02:00
Junhyun Shim 29b89eddae Add description of ALLOW_TOKENLESS_TENANT_ACCESS knob
Also fix typos and formatting
2022-09-14 19:46:05 +02:00
Junhyun Shim bb3e8e9e5e Add documentation for Authz 2022-09-14 16:23:43 +02:00
A.J. Beamon 726d5215a0
Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (#8108)
* Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (no need to guard)

* Enable the relaxed special key access in transactions that need to use special key-space APIs introduced in 7.2
2022-09-08 17:22:36 +02:00
Ata E Husain Bohra 28e608e717
Encryption data at-rest db-config (#7929)
* Encryption data at-rest db-config

Description

 diff-1: Handle 'force' updates to encryption_at_rest db-config

Major changes proposed:
1. Introduce 'encryption_data_at_rest_mode" 'configure new'
option to enable Encryption data at-rest. The feature is disabled
by default.
2. The configuration is meant to be set at the time of database
creation, addition checks will be done to avoid updating the config
in subsequent PR.
3. DatabaseConfiguration validity check to account for "tenant_mode"
set to `required` if Encryption data at-rest is selected given
EncryptionDomain matches Tenant boundaries.

Testing

devCorrectness - 100K
2022-09-02 14:11:38 -07:00
Jingyu Zhou 7abf487c50 Address a comment 2022-08-30 16:01:41 -07:00
Jingyu Zhou c982f9a370 Add 7.1.20, 7.1.21 release notes 2022-08-30 15:54:08 -07:00
Ata E Husain Bohra 7a056997c7
Improve FDB architecture documentation related to cluster recruitment (#8012)
Description

Improve FDB architecture documentation to explicitly mention that
Stateless processes may get recruited on a single cluster process

Testing
2022-08-29 09:24:13 -07:00
Lukas Joswiak e705b055b0 Improve wording
Co-authored-by: Bharadwaj V.R <bharadwaj.vr@snowflake.com>
2022-08-23 10:46:59 -07:00
Lukas Joswiak 4a7122c802 Document the `three_data_hall_fallback` datacenter mode 2022-08-23 10:46:59 -07:00
Chaoguang Lin 3fed0456ca
Add an verify option for \xff\xff/worker_interfaces special keys (#7873)
* Add the verify option for \xff\xff/worker_interfaces

* Remove unused code

* update documentations

* update documentations

* solve comments from review

* update some of the comments to be more clear
2022-08-15 14:05:07 -04:00
Vaidas Gasiunas f04ab582dd
Add documentation for API Tester (#7869)
* Add documentation for API Tester

* API Tester documentation: addressing review comments
2022-08-15 15:10:15 +02:00
Evan Tschannen a9d3c9f9b3
Added throttling when a blob worker falls behind (#7751)
* throttle the cluster when blob workers fall behind

* do not throttle on blob workers if they are not enabled

* remove an unnecessary actor

* fixed a compile error

* fetch blob worker metrics at the same interval as the rate is updated, avoid fetching the complete blob worker list too frequently

* fixed another compilation bug

* added a 5 second delay before bw throttling to prevent false positives caused by the 100e6 version jump during recovery. Lower the throttling thresholds to react much quicker to bw lag.

* fixed a number of problems

* changed the minBlobVersionRequest to look at storage server versions since this will be a lot more efficient

* fix: do not let desired go backwards

* fix: track the version of notAtLatest changefeeds for throttling

* ratekeeper now throttled blob workers by estimating the transaction per second throughput of the blob workers

* added metrics for blob worker change feeds

* added a knob to disable bw throttling

* fixed the transaction options in blob manager
2022-08-12 13:15:56 -07:00
Xiaoge Su 0326d53965 Split proxy_memory_limit_exceeded to commit/grv specific exceptions
Currently GRV is reporting proxy_memory_limit_exceeded error which has
error message claiming Commit proxy failing. This split should remove
such confusion.
2022-08-12 00:45:57 -07:00
Jingyu Zhou 7ee6d1719d Added 7.1.18, 7.1.19 release notes 2022-08-09 11:28:05 -07:00
A.J. Beamon ff23d5994e
Merge pull request #7729 from sfc-gh-ajbeamon/feature-metacluster
Metacluster
2022-08-04 15:29:44 -07:00
Chaoguang Lin 48e46cbc81
Add test coverage for SpecialKeyRangeAsyncImpl::getRange (#7671)
* Add getRange test coverage for SpecialKeyRangeAsyncImpl

* Fix the bug in SpecialKeyRangeAsyncImpl found by the test

* Refactor ConflictingKeysImpl::getRange to use containedRanges to simplify the code

* Fix file format

* Initialize SpecialKeyRangeAsyncImpl cache with correct end key

* Add release notes

* Revert "Refactor ConflictingKeysImpl::getRange to use containedRanges to simplify the code"

This reverts commit fdd298f469.
2022-08-02 12:04:40 -07:00
A.J. Beamon 7f53b340f5 Merge branch 'main' into feature-metacluster
# Conflicts:
#	bindings/python/tests/fdbcli_tests.py
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/ClientKnobs.cpp
#	fdbclient/include/fdbclient/ClientKnobs.h
#	fdbclient/include/fdbclient/Tenant.h
#	fdbserver/BlobManager.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/TenantCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
#	flow/include/flow/error_definitions.h
2022-07-28 11:43:21 -07:00
A.J. Beamon 9b060aacf6
Merge pull request #7715 from sfc-gh-jfu/jfu-tenant-rename-special-keys
Tenant renaming special key-space support
2022-07-27 16:12:03 -07:00
Jingyu Zhou 38d2d39001 Add 7.1.16, 7.1.17 release notes 2022-07-27 09:53:02 -07:00
A.J. Beamon 7c6b3fb0b8 Merge branch 'main' into feature-metacluster 2022-07-27 08:55:10 -07:00
Jon Fu fbf566ac4a
Update documentation/sphinx/source/special-keys.rst
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-26 15:53:21 -04:00
Jon Fu 0950d82fe5 add documentation and fix merge errors 2022-07-26 11:58:49 -07:00
Jingyu Zhou 88cfaf8793
Merge pull request #7681 from jzhou77/main
Add timeout during Status generation
2022-07-25 11:04:00 -07:00
Jingyu Zhou 16519a9e5f Update status json doc with fetch_storage_wiggler_stats_timeout error 2022-07-24 15:24:21 -07:00
A.J. Beamon 3e1763e7ac Merge branch 'main' into feature-metacluster 2022-07-21 14:48:53 -07:00
A.J. Beamon 190ad8c7e9 Convert existing tuple usages to use Tuple::makeTuple() 2022-07-19 13:45:59 -07:00
A.J. Beamon 1b81e72604 Add a Tuple::makeTuple function to easily construct a tuple. Update Tuple to allow all types to be passed via .append() so they can be used with makeTuple. 2022-07-19 11:50:58 -07:00
A.J. Beamon c7c7bd5f17 Undo change to mark tenant modes non-experimental. Other minor cleanup. 2022-07-19 09:18:53 -07:00
A.J. Beamon 0fe6f6ad36 Merge remote-tracking branch 'origin/update-prefix-encoding-in-tenant-metadata-json' into feature-metacluster
# Conflicts:
#	bindings/python/tests/fdbcli_tests.py
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/include/fdbclient/TenantManagement.actor.h
#	fdbclient/include/fdbclient/TenantSpecialKeys.actor.h
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-14 14:32:29 -07:00
A.J. Beamon cb499fa5db Merge branch 'main' into feature-metacluster 2022-07-13 15:28:34 -07:00
A.J. Beamon 91ccdbcb03 Use a more standard encoding in the JSON metadata reported for tenants. 2022-07-13 15:22:19 -07:00
A.J. Beamon a3f970cf3a Parse integer options in fdbcli rather than passing the bytes unparsed. This allows specifing the string "1" instead of "\x01\x00\x00\x00\x00\x00\x00\x00". 2022-07-11 16:39:59 -07:00
Jingyu Zhou ef37bf3fca Add 7.1.14 and 7.1.15 release notes 2022-07-11 10:29:38 -07:00
Jingyu Zhou 5b3521f7f0 Add 7.1.12, 7.1.13 release notes 2022-07-07 15:13:01 -07:00
A.J. Beamon b9cc5389b1 Undo some changes that aren't needed 2022-07-07 14:48:36 -07:00
A.J. Beamon 1519f24f77 Merge branch 'main' into feature-metacluster 2022-07-07 09:35:40 -07:00
A.J. Beamon c4b0f6eaae
Add an internal C API to support connection to a cluster using a connection string (#7438)
* Add an internal C API to support memory connection records

* Track shared state in the client using a unique and immutable cluster ID from the cluster

* Add missing code to store the clusterId in the database state object

* Update some arguments to pass by const&
2022-07-07 10:12:49 +02:00
A.J. Beamon c9b553663e Merge branch 'main' into feature-metacluster
# Conflicts:
#	bindings/java/src/main/com/apple/foundationdb/TenantManagement.java
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/TenantSpecialKeys.actor.cpp
#	fdbclient/include/fdbclient/KeyBackedTypes.h
#	fdbclient/include/fdbclient/RunTransaction.actor.h
#	fdbclient/include/fdbclient/SpecialKeySpace.actor.h
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-06 15:44:21 -07:00
Jon Fu df1f108b78 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename 2022-07-06 14:10:22 -07:00
A.J. Beamon 2f67328a0c Update the tenant special keys submodule to support multiple sub-ranges. This will enable future work that allows configuring tenants at the same time as creating them. 2022-06-30 15:03:37 -07:00
A.J. Beamon e1a93988ef Merge branch 'main' into feature-metacluster 2022-06-28 14:58:07 -07:00
Jon Fu f431ac58a8 address code review comments 2022-06-28 11:35:55 -07:00
Jon Fu ddfdbdcb09 initial commit introducing renameTenant function 2022-06-28 11:34:44 -07:00
A.J. Beamon 309f244f78
Merge pull request #7449 from sfc-gh-ajbeamon/get_tenant_json
Add support for getting tenant metadata as a JSON document
2022-06-27 14:21:06 -07:00
Jingyu Zhou a7e71cbac2 Add 7.1.10, 7.1.11 release notes 2022-06-27 15:52:03 -04:00
A.J. Beamon f990fba44d Add support for getting tenant metadata as a JSON document. 2022-06-27 12:34:40 -07:00
Markus Pilman d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
A.J. Beamon 9f3819752f Change the command to create a metacluster from using 'configure tenant_mode=management' to 'metacluster create <NAME>'. Distribute this name to all processes in a metacluster. Eliminate the tenant mode entirely from metacluster clusters, instead relying on a metacluster registration key. 2022-06-22 12:15:43 -07:00
Hao Fu 9cee4c94e7
Safely remove fdb_transaction_get_range_and_flat_map (#7314) 2022-06-13 19:05:00 -07:00
Zhanwei Wang e632aef1c7
Make backup work with s3 compatible service (#6355)(#6382) (#7324)
1. Support virtual hosting endpoint.

2. On-premise s3 compatible storage service may use IP instead of s3 form domain name,
especially for development/test environment.

Instead of parsing service and region from domain name,

1). Hard code "s3" as service name in v4 signature
2). Add new parameter to allow pass region name from url

3. Fix creating bucket issue on aws, adding request body.
2022-06-13 13:33:05 -07:00
Jingyu Zhou d0c5449d5c Add 7.1.8 and 7.1.9 release notes 2022-06-10 15:09:10 -07:00
A.J. Beamon 739fc9ce6b Merge branch 'main' into feature-metacluster 2022-06-09 12:39:34 -07:00
Jingyu Zhou 217ba24b6f Add rss_bytes to process memory and fix available_bytes calculation
Since memory is now limited with RSS size, add RSS size in status json for
reporting. Also change how available_bytes is calculated from:
  (available + virtual memory) * process_limit / machine_limit
to:
  (available memory) * process_limit / machine_limit
2022-06-07 16:44:14 -07:00
A.J. Beamon 2907aafa60 Merge branch 'main' into feature-metacluster 2022-06-02 12:18:42 -07:00
Josh Slocum 07adc873a6 Add documentation for aws sdk auth 2022-05-26 13:18:25 -05:00
Jingyu Zhou 47f2118cb0 Update 7.1.6 and 7.1.7 release notes. 2022-05-25 13:30:48 -07:00
A.J. Beamon 804bd8d3e2 Remove documentation of the tenant prefix configuration key. 2022-05-23 13:28:40 -07:00
A.J. Beamon 69261f9f10 Merge branch 'main' into feature-metacluster
# Conflicts:
#	fdbclient/MultiVersionTransaction.actor.cpp
#	fdbclient/MultiVersionTransaction.h
#	fdbserver/tester.actor.cpp
2022-05-19 09:11:30 -07:00
Sagar Vemuri ebf11d5d48 Update schemas with the tenants information 2022-05-16 11:09:10 -07:00
Jingyu Zhou e4ac7ab1d9 Update release notes 7.1.0 through 7.1.5 2022-05-12 22:27:19 -07:00
A.J. Beamon 85019612d4 Remove subordinate mode and instead use a key in the system key-space to track metacluster membership. Use this key to determine if a cluster is already part of a metacluster or can be configured away from required mode. Disallow configuring to or from a management cluster that has data. 2022-05-11 17:10:55 -07:00
A.J. Beamon 7e860dc92c Some initial metacluster implementation 2022-05-03 12:53:35 -07:00
greid 508ee73f86 Fix compile errors in Java tutorial code
Use CompletableFuture::join instead of CompletableFuture::get
when blocking on futures in java tutorial snippets. This avoids
the unreported checked exceptions which prevent the snippets from
compiling (and is in line with the full tutorial code).
2022-04-26 14:10:05 -07:00
A.J. Beamon 1352083d4c
Merge pull request #6884 from sfc-gh-clin/deprecate-speical-keys
Remove the client profiling special keys and update related documentations
2022-04-21 21:51:16 -07:00
Zhe Wang 6c9ff6ee5e
Add sharded rocksdb type (#6862)
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Chaoguang Lin 4b1d22d684 Polish the documentation 2022-04-21 19:28:50 -07:00
Chaoguang Lin 879eb98dec Rename registerSpecialKeySpaceModule to registerSpecialKeysImpl 2022-04-21 12:54:34 -07:00
Chaoguang Lin 588041f977 Polish words according to comments 2022-04-21 12:40:32 -07:00
Chaoguang Lin 9f110b0f51 Update documentations and comments 2022-04-21 10:19:44 -07:00
Xiaoxi Wang 2fd640b0d0 fix doc compilation 2022-04-20 21:27:22 -07:00
Xiaoxi Wang 5d5dae9a0c update release note 2022-04-20 21:27:22 -07:00
Chaoguang Lin 6bfa4f6ac4 Fix typos; Polish documentations according to comments 2022-04-19 13:57:11 -07:00
Chaoguang Lin 36c876b769 Fix a typo; Fix a format error in special-keys.rst 2022-04-18 18:10:17 -07:00
Chaoguang Lin 4af36fa9ea Update documentations 2022-04-18 17:59:28 -07:00
Chaoguang Lin c0264a8522 Remove the client profiling special keys and update related documentations 2022-04-18 17:54:50 -07:00
Ray Jenkins 865b5f1ff8 Substitute other with author. 2022-04-18 14:50:30 -05:00
Ray Jenkins 28ee6fd7b2 Fix typo, expose a function... 2022-04-18 14:45:03 -05:00
Aaron Molitor 63a2e317fb update version to 7.2.0 -- pr comment 2022-04-11 23:23:27 -05:00
Aaron Molitor cbaef8f03b update version to 7.2.0 2022-04-11 23:23:27 -05:00
Tao Lin 73ed34d2fb Fixes #6793: Upgrade from 7.0.0 to 7.1.0 fails because of missing function fdb_transaction_get_mapped_range 2022-04-08 09:32:18 -07:00
Yi Wu 994b8c92f8
Add option to limit resident memory and remove default memory limit (#6719)
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.

Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.

closes #6671, closes #6672
2022-04-06 20:06:24 -07:00
Ray Jenkins 0c56de19c8 Merge branch 'main' into opentracing-dep-710 2022-04-06 18:02:42 -05:00
Ray Jenkins 7e773fe13c OpenTracing deprecation notice. 2022-04-06 16:16:41 -05:00
Jon Fu 08838fada6 remove duplicate label of release notes 2022-04-06 16:52:59 -04:00
Jon Fu e12d623e94 Add release notes for grv cache in 7.1 2022-04-06 16:26:28 -04:00
Steve Atherton d9ca18effa Add release notes template for 7.1. 2022-04-06 12:18:10 -07:00
A.J. Beamon 5469b57a2b Add a note that opening a tenant does not check whether that tenant exists in the cluster 2022-03-31 11:39:50 -07:00
A.J. Beamon 15b3465b49
Merge pull request #6647 from sfc-gh-ajbeamon/fdb-tenant-documentation
Add some basic documentation for tenants
2022-03-31 10:33:01 -07:00
A.J. Beamon b6350a2535 Add a note that use of special keys may implicitly enable raw access on a transaction. 2022-03-31 08:29:31 -07:00
A.J. Beamon 860ede0c2e Add some basic documentation for tenants 2022-03-31 08:29:31 -07:00
Xiaoxi Wang d6d4596c35
Merge pull request #6709 from sfc-gh-ajbeamon/fix-fdbcli-bugs
Fix issues with command completion for exclude and storage_migration_type
2022-03-30 19:53:49 -07:00
A.J. Beamon fc68cdf45f
Merge pull request #6693 from sfc-gh-ajbeamon/python-tenant-management-module
Move Python tenant management to its own module
2022-03-30 12:03:08 -07:00
A.J. Beamon 16cc74e910 Remove documenation for Database.delete_tenant. 2022-03-30 11:01:54 -07:00
He Liu dd15489605 rename ssd-rocksdb-experimental as ssd-rocksdb-v1. 2022-03-29 10:53:38 -07:00
A.J. Beamon 23fcd8c076 Fix issues with command completion for exclude and storage_migration_type. Add missing documentation for tenant_mode in one spot. 2022-03-29 09:07:33 -07:00
Jingyu Zhou 4de05a8a8d
Merge pull request #6692 from sfc-gh-tclinkenbeard/fix-tutorial-typo
Fix typo in SimpleKeyValueStoreInterface class name
2022-03-28 13:33:34 -07:00
A.J. Beamon 4d277fe19a Fix typo 2022-03-28 13:06:17 -07:00
A.J. Beamon b4cfcc10d3 Move Python tenant management to its own module 2022-03-25 11:36:35 -07:00
sfc-gh-tclinkenbeard c8bac58536 Fix typo in SimpleKeyValueStoreInterface class name 2022-03-25 09:10:19 -07:00
Aaron Molitor ad6a63c16d used fixed Jinja2 version for documentation 2022-03-24 15:39:29 -05:00
A.J. Beamon 1b735593eb
Merge pull request #6643 from sfc-gh-ajbeamon/fdb-tenant-python
Python support for tenants
2022-03-24 10:34:47 -07:00
Balachandar Namasivayam 831d2f8e2c
Merge pull request #6660 from bnamasivayam/release-7.1
Add backup documentation to select between v2 and v4 signature protocol.
2022-03-24 09:41:06 -07:00
A.J. Beamon ded4d046f0 Update transactional decorator documentation to reference tenants 2022-03-23 14:46:56 -07:00
A.J. Beamon 1d0a3669df Add documentation for the new Python tenant APIs. 2022-03-23 14:46:56 -07:00
Bala Namasivayam 930cd7f9c7 Add backup documentation to select between v2 and v4 signature protocol. 2022-03-23 11:03:49 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon a23add6bc4 Add fdbcli test for tenants. Add documentation for new fdbcli tenant commands. Various output cleanup. Fix limit parsing bug in listtenants command. Update gettenant output format. 2022-03-17 12:10:39 -07:00
A.J. Beamon 14c5c78f8f
Merge pull request #6594 from sfc-gh-almiller/monitored-metrics
[doc] Add "Monitored Metrics" page to the docs.
2022-03-16 15:47:00 -07:00
A.J. Beamon e8077b65e1
Merge pull request #6559 from sfc-gh-ajbeamon/fdb-tenant-client
Add client support for tenants
2022-03-15 12:40:17 -07:00
A.J. Beamon 445754d7cb Fix documentation substitutions 2022-03-15 09:23:30 -07:00
A.J. Beamon 0536e56725 The tenant cache uses a simpler eviction policy. Some various improvement around the use of futures. Add documentation for C API and fix inaccurate statement in special keys documentation. 2022-03-15 09:23:30 -07:00
A.J. Beamon d6247e32ef Fix documentation error 2022-03-15 09:23:30 -07:00
A.J. Beamon 68069c9784 Undo some unused changes. Fix tenant cache eviction. 2022-03-15 09:23:30 -07:00
A.J. Beamon 8bc2b283e1 Create a special keys API to create, delete, and read tenants. Remove the C API to create/delete tenants. 2022-03-15 09:23:30 -07:00
Alex Miller a7f6a9e8f7 [doc] Add "Monitored Metrics" page to the docs.
This page contains a description of suggested alerts and dashboard graphs to have to support a production FDB cluster.

(Credit to AJ for the actual text.)
2022-03-15 00:04:22 +00:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
A.J. Beamon 07169b1f11 Add unknown_tenant error to documentation. Avoid doing an extra tenant lookup in the clampRangeToTenant function. 2022-03-07 10:37:22 -08:00
A.J. Beamon 3ae98189af Server processes send unknown_tenant responses back to clients, which is meant to be retried after updating the tenant cache. Fix bug where key selectors could be truncated after applying the tenant prefix. 2022-03-06 21:54:21 -08:00
A.J. Beamon 72a34945ce Add the ability to disable tenants. Server processes verify the ID of tenants being read or written. 2022-03-06 21:54:21 -08:00
A.J. Beamon ea273291c7 Add new tenant mode configuration field to status documentation 2022-03-06 21:54:21 -08:00
Jingyu Zhou 1a5bf25b5c Update code base to use fmt 8.1.1 2022-03-04 15:52:06 -08:00
Renxuan Wang 233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
Zhe Wu 57e931b489 Update 6.3.24 release notes 2022-02-25 12:12:17 -08:00
Bala Namasivayam a420098fc5 Support AWS v4 header for s3 backup and restore 2022-02-07 17:53:05 -08:00
sfc-gh-tclinkenbeard ef711d405c Add fdbbackup tags command 2022-01-24 13:34:54 -08:00
Jingyu Zhou 2793b2613f Replace master branch with main in documentations 2022-01-18 11:37:11 -08:00
Aaron Molitor 6e31821bf5 update download links in documentation #6154 2022-01-06 17:53:35 -08:00
A.J. Beamon 16fb079c2d Undo some changes that aren't command line flags. 2021-12-14 12:35:49 -08:00
A.J. Beamon f24adc7b6a Fix a bunch of places where we used old-style arguments. Allow hyphens for profiler args. 2021-12-14 09:59:14 -08:00
A.J. Beamon f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
Zhe Wu e95ed4e9e0 Release notes for 6.3.23 2021-12-07 16:35:35 -08:00
Lukas Joswiak fc08313925 Use path relative to Python binary 2021-11-16 16:36:44 -08:00
Lukas Joswiak 8d4ec8562d Move generated versions.target file to build directory 2021-11-16 16:36:44 -08:00
Steve Atherton c53f5aa110 Renamed redwood to redwood-1-experimental and file extension to .redwood-v1. 2021-11-16 02:15:22 -08:00
Lukas Joswiak 1da288822f Remove distributed trace database option 2021-11-10 13:33:49 -08:00
Lukas Joswiak 8bc0c3e8a2 Convert distributed trace sampling rate to a database option 2021-11-10 13:33:49 -08:00
Tao Lin fdb3b72e35 Introduce GetRangeAndFlatMap to push computations down to FDB
Re-introduce #5609
2021-11-09 13:52:28 -08:00
sfc-gh-tclinkenbeard e08de9e304 Update transaction-tagging.rst documentation 2021-11-04 12:57:50 -07:00
Tao Lin 586cc3b102
Revert "Introduce GetRangeAndFlatMap to push computations down to FDB" 2021-11-04 08:46:56 -07:00
Tao Lin 6c98e35893 Rename Hop to FlatMap 2021-11-03 13:32:01 -07:00
Tao Lin 0853661d13 Introduce getRangeAndHop to push computations down to FDB 2021-11-03 13:21:16 -07:00
Yao Xiao 7ce29dd153 fix CI error 2021-11-01 16:26:37 -07:00
Yao Xiao 3a6a9bdba5
Update documentation/sphinx/source/api-common.rst.inc
apply fix

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-11-01 16:07:36 -07:00
Yao Xiao 648bd336b5 resolve comments 2021-11-01 14:47:36 -07:00
Yao Xiao 90b231e96e Add link to client knobs. 2021-11-01 14:47:36 -07:00
Yao Xiao af51de902f Add documation about network options. 2021-11-01 14:47:36 -07:00
He Liu 784576f214
Merge pull request #5713 from liquid-helium/clean-sim-test-data-loss
Reproduced user data loss incident, and tested the improved exclude tool
2021-10-21 14:14:10 -07:00
Yao Xiao a58140c1bd Update release-notes-630.rst 2021-10-21 12:40:10 -07:00
Yao Xiao 2a1a5af939 Update release-notes-630.rst
Resolve comments
2021-10-21 12:40:10 -07:00
Yao Xiao 1a24ad33dd Update documentation/sphinx/source/release-notes/release-notes-630.rst
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-10-21 12:40:10 -07:00
Yao Xiao ac23751608 Update release-notes-630.rst 2021-10-21 12:40:10 -07:00
He Liu f20f43a85a Added warning in the `exlucde` cli command help about potential dataloss, as well as in command-line-interface.rst. 2021-10-18 12:10:34 -07:00
Markus Pilman 5066d67dec
Merge pull request #5715 from sfc-gh-ljoswiak/fixes/tracing-various
Add knob to control distributed trace recording percentage
2021-10-11 14:03:53 -06:00
Lukas Joswiak e034f66fc2 Add knob to control trace recording percentage 2021-10-01 16:30:46 -07:00
A.J. Beamon cd72ae21fd Updating release notes for 6.3.21 2021-09-30 16:36:08 -06:00
Neethu Haneesha Bingi 3ea7209013 Simulation changes to randomly wiggle with locality filter and review comments. 2021-09-30 10:00:33 -07:00
Neethu Haneesha Bingi 3e79299898 Locality filter support to perpetual storage wiggler feature. 2021-09-30 10:00:33 -07:00
Zhe Wu 7035b53d2c Update documentation/sphinx/source/release-notes/release-notes-630.rst
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-09-29 15:58:12 -07:00
Zhe Wu a3232d2cd2 Adding missing release notes from 6.3.19 to 6.3.21 2021-09-29 15:58:12 -07:00
Markus Pilman 49d196e471 Complete release notes for 6.3.20 2021-09-22 12:14:22 -06:00
Xiaoxi Wang 1730d75f73 change configure test
add store type check
add test file
2021-09-21 18:11:04 -07:00
Vaidas Gasiunas 16171d8252 Refactoring well-known endpoint registration
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
Jingyu Zhou 2061fc6dbb Fix more links 2021-09-21 09:35:32 -07:00
Jingyu Zhou 154573c1bb Fix a documentation link in ha-write-path 2021-09-21 09:35:32 -07:00
Jingyu Zhou 566ee3f3b2 Fix image links in read-write-path documentation 2021-09-21 09:35:32 -07:00
Evan Tschannen 47ccc75270 added a sequence number to streaming replies as a safety backstop against out of order delivery 2021-09-20 13:58:05 -07:00
A.J. Beamon df5966ac85 Update release notes. 2021-09-17 18:10:56 -06:00
Markus Pilman 1cf98785a0 updated release notes 2021-09-16 15:15:06 -07:00
Mohamed Oulmahdi f83d97ee39 Enable documentation for Windows 2021-09-15 16:31:25 -06:00
Hari Bhaskaran ee1056cacd Remove incorrect comment
As described in this comment, https://forums.foundationdb.org/t/questions-on-status-json/2843/3?u=harikb , removing the comment that will make reader think this is about RAM. No new comment is necessary since the key is already "disk"
2021-09-14 09:39:23 -07:00