Commit Graph

141 Commits

Author SHA1 Message Date
Xiaoge Su 5b50634b62 fixup! Add notes for AcAC 2023-09-27 13:02:11 -07:00
Xiaoge Su ba264a26cc Add a function to dump the actor call backtrace to stdout 2023-09-26 18:03:05 -07:00
Xiaoge Su b5e40c5598 fixup! Update per review 2023-09-21 14:55:56 -07:00
Xiaoge Su 7d982a16c1 Add documentation 2023-09-19 21:53:28 -07:00
Lukas Joswiak bfb1c51299 Add `clearknob` fdbcli command
The `clearknob` command clears the value that a knob has been set to in
the configuration database. Note that this does not mean the knob value
itself gets cleared - only the value in the configuration database is
cleared. The value of the knob will revert to whatever is hardcoded in
the corresponding `*Knobs.cpp` file.

Sample `fdbcli` session:

```
Welcome to the fdbcli. For help, type `help'.
fdb> getknob min_trace_severity
`min_trace_severity' is not found
fdb> setknob min_trace_severity 20
Please set a description for the change. Description must be non-empty
description: test
Committed (2)
fdb> getknob min_trace_severity
`min_trace_severity' is `20'
fdb> clearknob min_trace_severity
Please set a description for the change. Description must be non-empty
description: clear
Committed (4)
fdb> getknob min_trace_severity
`min_trace_severity' is not found
```

Transactions are also supported with the new `clearknob` command:

```
Welcome to the fdbcli. For help, type `help'.
fdb> begin
Transaction started
fdb> setknob min_trace_severity 20
fdb> clearknob min_trace_severity
fdb> commit
Please set a description for the change. Description must be non-empty.
description: test
Committed (16)
fdb> getknob min_trace_severity
`min_trace_severity' is not found
```
2023-08-31 17:36:05 -07:00
Ankita Kejriwal 2e6aed58b4
Update Storage Quota Documentation (#10839) 2023-08-30 13:40:01 -07:00
sfc-gh-tclinkenbeard f647a1289e Split GLOBAL_TAG_THROTTLING_FOLDING_TIME into several knobs 2023-05-31 17:20:32 -07:00
Zhe Wu e3490a1af8 Update API version to 730 2023-05-10 11:26:46 -07:00
Ata E Husain Bohra 03dc8e07a5
EaR: Update encryption at-rest design with recent changes (#10168)
Description

Updates FDB Encryption at-rest design document as per the
recent changes

Testing
2023-05-09 09:52:08 -07:00
Xiaoxi Wang f89a483f3d add informal classification of priority 2023-03-20 09:46:36 -07:00
Xiaoxi Wang c73577de7d Add team priority comments and document. 2023-03-20 09:46:36 -07:00
Ankita Kejriwal 58a4c4a16e
Update design/storage-quota.md as per review comment
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2023-02-07 11:34:20 -08:00
Ankita Kejriwal 92ae21eb44 Design for the storage quota feature 2023-01-31 16:42:36 -08:00
Andrew Noyes 3a514d3451
Add design for avoiding storage servers for idempotency ids (#9113)
* Add design for avoiding storage servers for idempotency ids

* Update design/idempotency_ids.md
2023-01-20 11:26:54 -08:00
Andrew Noyes f0cb73a941 Address review comments 2022-12-28 13:29:57 -08:00
Andrew Noyes 4fb8e6d090 Update design doc, and make AUTOMATIC_IDEMPOTENCY visible 2022-12-19 10:24:19 -08:00
Jon Fu 3c08901d2c
Kill datacenter in simulation as part of repair process (#8914)
* kill datacenter in simulation as part of repair process

* fix typo
2022-12-05 18:09:51 -05:00
sfc-gh-tclinkenbeard 7ed8838094 Update testing section of global-tag-throttling.md 2022-11-15 12:58:34 -08:00
sfc-gh-tclinkenbeard ca572f58fb Combine *_COST_BYTE_FACTOR knobs into a single TAG_THROTTLING_PAGE_SIZE knob 2022-11-08 07:44:01 -08:00
sfc-gh-tclinkenbeard 0eb1598afa Merge remote-tracking branch 'origin/main' into expose-txn-cost 2022-10-30 09:36:37 -07:00
Andrew Noyes 0a15f081a1
Proactively clean up idempotency ids for successful commits (#8578)
* Proactively clean up idempotency ids for successful commits

This change also includes some minor changes from my branch working on
an idempotency ids cleaner, that I'd like to get merged sooner rather
than later.

- Adding a timestamp to idempotency values
- Making IdempotencyId an actor file
- Adding commit_unknown_result_fatal
- Checking idempotencyIdsExpiredVersion in determineCommitStatus
- Some testing QOL changes

* Factor out decodeIdempotencyKey logic

* Fix formatting

* Update flow/include/flow/error_definitions.h

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Use KeyBackedObjectProperty for idempotencyIdsExpiredVersion

* Add IDEMPOTENCY_ID_IN_MEMORY_LIFETIME knob

* Rename ExpireIdempotencyKeyValuePairRequest

Also add a code probe for the case where an ExpireIdempotencyIdRequest is
received before the count is known, and add an assert

* Fix formatting and add TODO for nwijetunga

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-10-28 09:07:54 -07:00
sfc-gh-tclinkenbeard fef4c9c5b2 Remove SERVER_KNOBS->GLOBAL_TAG_THROTTLING_RW_FUNGIBILITY_RATIO 2022-10-23 20:18:21 -07:00
sfc-gh-tclinkenbeard 4fbaa816ea Fix several bugs in TransactionCostWorkload 2022-10-23 13:57:24 -07:00
sfc-gh-tclinkenbeard 32ae7bb529 Merge remote-tracking branch 'origin/main' into expose-txn-cost 2022-10-23 12:59:07 -07:00
sfc-gh-tclinkenbeard b442705dc7 Change units for tag quota enforcement from pages to bytes 2022-10-23 12:57:19 -07:00
sfc-gh-tclinkenbeard 3827b1ee62 Merge remote-tracking branch 'origin/main' into add-quota-clear-command 2022-10-14 11:55:45 -07:00
sfc-gh-tclinkenbeard d0be1930f0 Addressed review comments 2022-10-14 11:54:38 -07:00
Jingyu Zhou df5825ff65
Merge pull request #8398 from sfc-gh-anoyes/anoyes/idempotency-id2
Initial work for automatic idempotency
2022-10-13 13:07:14 -07:00
Andrew Noyes cd01ee8dcb Describe new design for cleaning idempotency ids 2022-10-12 10:21:25 -07:00
Kevin Hoxha 2f898fb006 fdbcli: Add documentation for setknob/getknob commands 2022-10-11 15:32:01 -07:00
Andrew Noyes 13c5e7f240 Mention batch index is a 2 byte int 2022-10-11 14:15:04 -07:00
Andrew Noyes 7fc5f03b81 Update design/idempotency_ids.md
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2022-10-11 14:04:29 -07:00
Andrew Noyes f0f2b944a2 Apply suggestions from code review
Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2022-10-11 13:46:40 -07:00
Andrew Noyes 64eaeb22a7 Add design doc 2022-10-11 13:46:39 -07:00
sfc-gh-tclinkenbeard d67f3ca3b9 Update documentation for quota clear command 2022-10-09 19:43:44 -07:00
Nim Wijetunga 232e23c87e
Snapshot Backup Encryption (#8095)
* inital commit

* add encryption support

* refactor passing encryption keys

* disable encryption codepath

* pass enable encryption cli arg for fdbbackup

* change encryption key passing

* only enable encryption for backup if tenant mode is required

* revert server knobs

* ignore json trace files

* --amend

* integrate ekp

* add comments

* comments

* comments

* revert knobs

* fix bug

* modify encrypted backup logic

* use common prefix

* check boundaries between begin and end key

* return system encrypt domain

* modify

* add tenant entry cache to backup

* slight revert

* format

* Trigger Build

* format

* use tenant name to fetch keys

* Trigger Build

* Trigger Build

* merge

* fix getting ekp

* fix bug

* address pr comments

* Trigger Build

* Trigger Build

* remove faulty check
2022-09-29 14:45:47 -07:00
sfc-gh-tclinkenbeard 16fe88948e Update setQuota function in QuotaCommand.actor.cpp to round up to nearest page size 2022-09-27 09:48:42 -07:00
sfc-gh-tclinkenbeard c70ec0ce1e Fix writeCost documentation in global-tag-throttling.md 2022-09-26 23:28:27 -07:00
sfc-gh-tclinkenbeard 6b094c122f Make throttling cost functions round up to nearest page size 2022-09-26 23:24:32 -07:00
Trevor Clinkenbeard bd94743acd
Update design/global-tag-throttling.md
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-09-26 13:24:26 -07:00
sfc-gh-tclinkenbeard 7fc5c196c4 Make read and write quotas fungible 2022-09-25 21:00:11 -07:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
sfc-gh-tclinkenbeard 9df990e375 Remove global_tag_throttler status section 2022-08-29 23:17:20 -07:00
Lukas Joswiak d710c1fce5 Add dynamic knobs design doc
Co-authored-by: Markus Pilman <markus.pilman@snowflake.com>
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-08-26 09:28:57 -07:00
Xiaoxi Wang 9133d4e16d
Merge pull request #7803 from sfc-gh-xwang/feature/main/ddvisibility
Add server selection counter in DDQueue
2022-08-12 15:10:25 -07:00
Xiaoxi Wang 2d94128140
Apply suggestions from code review
Co-authored-by: Bharadwaj V.R <bharadwaj.vr@snowflake.com>
2022-08-12 13:33:12 -07:00
Xiaoxi Wang 4819707af3 solve invalid detail name 2022-08-11 12:28:18 -07:00
Xiaoxi Wang fe45b4f0b2 update document 2022-08-11 12:14:24 -07:00
Xiaoxi Wang 2e467c51e0 update design doc 2022-08-10 09:53:10 -07:00
Xiaoxi Wang f6dce5dcee add knob to control summarize 2022-08-10 09:44:01 -07:00