Commit Graph

24317 Commits

Author SHA1 Message Date
Nim Wijetunga 08b2123c1c Merge branch 'main' into nim/fix-encrypted-backup-workload
* main:
  Moving rocksdb read iterator destruction from commit path to actor. (#8970)
  Nit fix
  apply format in fdbclient/FDBTypes.cpp
  Update physical shard tracking document.
  Fix issue with KeyRangeRef original data gone.
  Add code document.
  During initialization, keyRangePhysicalShardIDMap may not contain full range. Adjust assertion in checkKeyRangePhysicalShardMapping
  Create per physical shard tracking data stucture. Not populate yet
  Do not compress debug sections in release builds (#9005)
  Fix assertion failure from changeFeedTSSValidator()
  metrics: Fix windows build
  Fix feed metadata reset (#8751)
  MultiVersionTransaction: Improve Comments
  MultiVersionTransaction: Add comments
  MultiverstionTransaction: Remove redundant setOption calls
  The pair name will only match when we are deleting the source tenant entry
  Add an assert in deleteTenantFromManagementCluster that the tenant's pair name matches the locally stored pair name
  DeleteTenantImpl::markTenantInRemovingState needs to record the pair name even if the tenant is already in a removing state (it could be a pair in the process of being removed). Use better asserts in tenant consistency.
2022-12-12 16:32:09 -08:00
Sreenath Bodagala a4ceb8ff63 Merge remote-tracking branch 'apple-upstream/main' 2022-12-12 22:35:00 +00:00
neethuhaneesha 3a4d79cb8b
Moving rocksdb read iterator destruction from commit path to actor. (#8970) 2022-12-12 13:46:43 -08:00
Zhe Wu 9f8f19d0ef Nit fix 2022-12-12 13:25:52 -08:00
Zhe Wu 68972c039b apply format in fdbclient/FDBTypes.cpp 2022-12-12 13:25:52 -08:00
Zhe Wu 1b1eb58972 Update physical shard tracking document. 2022-12-12 13:25:52 -08:00
Zhe Wu 8dc87f7171 Fix issue with KeyRangeRef original data gone. 2022-12-12 13:25:52 -08:00
Zhe Wu a30208548b Add code document. 2022-12-12 13:25:52 -08:00
Zhe Wu 75b2c14d0f During initialization, keyRangePhysicalShardIDMap may not contain full range. Adjust assertion in checkKeyRangePhysicalShardMapping 2022-12-12 13:25:52 -08:00
Zhe Wu a6cd33134a Create per physical shard tracking data stucture. Not populate yet 2022-12-12 13:25:52 -08:00
Sreenath Bodagala 774fc1168e - Log FoundationDB version as part of "ProcessMetrics". 2022-12-12 21:16:30 +00:00
Jingyu Zhou 6e47d85b77
Merge pull request #8901 from sfc-gh-ajbeamon/fix-tenant-rename-delete-concurrency-bug
Fix concurrency bug with tenant renames and deletes
2022-12-12 13:04:11 -08:00
A.J. Beamon 8d1aeb4ee9 Merge branch 'main' into commit-proxy-tenant-name-refactor 2022-12-12 09:13:23 -08:00
A.J. Beamon 70abb457ce Merge branch 'main' into fix-tenant-rename-delete-concurrency-bug 2022-12-12 09:12:18 -08:00
Vaidas Gasiunas 585996a88b
Do not compress debug sections in release builds (#9005) 2022-12-12 16:19:57 +01:00
Christian Wende 6d29eb71d1
Merge pull request #8980 from sfc-gh-cwende/client-timout-optimizations
MultiversionTransaction: Remove redundant setOption calls
2022-12-12 14:18:57 +01:00
Christian Wende 396acf25f2 Merge remote-tracking branch 'origin/main' into client-timout-optimizations 2022-12-12 13:00:56 +01:00
Xiaoxi Wang c12de23824 Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/wiggleDelay 2022-12-11 14:27:22 -08:00
Jingyu Zhou 1c4f389671
Merge pull request #9022 from jzhou77/fix
Fix assertion failure from changeFeedTSSValidator()
2022-12-10 11:36:34 -08:00
Jingyu Zhou fee561921a Fix assertion failure from changeFeedTSSValidator() 2022-12-10 10:02:14 -08:00
Junhyun Shim 9ddc8e076d Fix clang format 2022-12-09 17:07:17 -08:00
Junhyun Shim 4882fa590f Merge remote-tracking branch 'origin/main' into mako-authz-token-generation 2022-12-09 16:48:59 -08:00
Junhyun Shim 5da9b5e10d Apply review suggestions 2022-12-09 16:48:42 -08:00
Jingyu Zhou 2240254cde
Merge pull request #9013 from sfc-gh-khoxha/khoxha/metric-windows-fix
metrics: Fix windows build
2022-12-09 15:32:25 -08:00
Kevin Hoxha f2b781407f metrics: Fix windows build 2022-12-09 13:41:46 -08:00
Junhyun Shim 691f0cd709 Mako: exempt CLEAN mode from generating tokens
In CLEAN mode, admin server uses trusted client mode (i.e. connect w/ TLS cert)
to remove clear tenant keyspaces and delete tenants.
2022-12-09 13:32:45 -08:00
Junhyun Shim d24df289e3 Remove debug code + fix broken switch-case block 2022-12-09 13:25:59 -08:00
Josh Slocum 23206257ef
Fix feed metadata reset (#8751)
* re-making feed metadata durable on private mutation reset

* cleaning up unrelated code
2022-12-09 12:03:41 -08:00
A.J. Beamon bfc664a814 Merge branch 'main' into fix-tenant-rename-delete-concurrency-bug 2022-12-09 11:25:28 -08:00
A.J. Beamon c33efe5ed9 Merge branch 'main' into commit-proxy-tenant-name-refactor 2022-12-09 11:24:50 -08:00
Nim Wijetunga dbf6fc0fb8 Merge branch 'main' into nim/fix-encrypted-backup-workload
* main: (68 commits)
  Fix clang 15 warning in mako
  Fix more clang 15 warnings
  Fix clang 15 compiler warnings
  metrics: Break down OTELSum into smaller chunks to avoid UDP max packet size
  metrics: Make NumberDataPoint contain a variant<int, double> of the value
  ddsketch: Fix undefined behaviour in DDSketch::getValue
  metrics: Make DDSketch buckets uint32_t but double precision
  metrics: Add OTEL metric definitions
  metrics: Add simulation testing and fix incorrect TraceEvent names - Added a background actor that listens on METRICS_EMISSION_UDP_PORT for incoming metrics (and verifies they are in the correct format) - TraceEvent details have certain requirements for naming. This commit makes a seperate name for Counter/LatencySample and its underlying IMetric to avoid those issues
  metrics: Add verifyStatsdMessage() for testing
  metrics: Avoid name conflicts in MetricCollection - In Counter constructor, make sure that we combine CounterCollection name - In IMetric constructor, add a trace event and assertion in case of collision - Rename ClusterRecoveryData CounterCollection name from master to ClusterRecoveryData
  metrics: Fix bug in LatencySample::flush() - Make sure that we check batch.statsd_message to be non-empty before adding a newline - Make sure to actually add the message to the batch in flush()
  metrics: Added runMetrics() to workerServer
  metrics: Add IMetricClient and StatsdMetric to send batches over UDP
  metrics: Add MetricsLogger loop and more knobs
  metrics: Make sure LatencyBand counters are unique
  metrics: Add knobs and make Counter, LatencySample implement flush() method
  metrics: Add flush() definitions to Counter and LatencySample
  metrics: Make LatencySample and Counter inherit from IMetric
  metrics: Initial code - Define new IMetric interface - Rename MetricUpdateBatch to MetricBatch - Move previous fields of MetricUpdateBatch into a new struct called FDBScope - Define global MetricCollection class and enMetric enumerate
  ...
2022-12-09 10:17:12 -08:00
Jingyu Zhou 10070fd7f2
Merge pull request #9007 from sfc-gh-tclinkenbeard/fix-clang-15-warnings
Fix clang 15 compiler warnings
2022-12-09 09:07:09 -08:00
Xiaoxi Wang 3e4966d5bd persistent perpetual wiggle delay 2022-12-08 23:46:26 -05:00
Nim Wijetunga eb1c0e11a2 fix test 2022-12-08 15:33:43 -08:00
sfc-gh-tclinkenbeard e008bca899 Fix clang 15 warning in mako 2022-12-08 14:25:41 -08:00
sfc-gh-tclinkenbeard 49ab88e912 Fix more clang 15 warnings 2022-12-08 14:25:36 -08:00
sfc-gh-tclinkenbeard 68f14f017c Fix clang 15 compiler warnings 2022-12-08 13:59:37 -08:00
Xiaoxi Wang ccc494319c perpetual wiggle key functions 2022-12-08 16:46:05 -05:00
Junhyun Shim e4c83c8913 Merge remote-tracking branch 'origin/main' into mako-authz-token-generation 2022-12-08 10:33:38 -08:00
Kevin Hoxha f32281bdce metrics: Break down OTELSum into smaller chunks to avoid UDP max packet size 2022-12-08 10:07:11 -08:00
Kevin Hoxha 5dc40d04fe metrics: Make NumberDataPoint contain a variant<int, double> of the value
Gauges tend to be double while counters are int, so we need to support both.
2022-12-08 10:07:11 -08:00
Kevin Hoxha 8e248529ae ddsketch: Fix undefined behaviour in DDSketch::getValue
There was unsigned integer underflow occuring in DDSketch::getValue, resulting in percentile sometimes returning inf as a value.
2022-12-08 10:07:11 -08:00
Kevin Hoxha 866679ed6d metrics: Make DDSketch buckets uint32_t but double precision 2022-12-08 10:07:11 -08:00
Kevin Hoxha 3cea754ba3 metrics: Add OTEL metric definitions 2022-12-08 10:07:11 -08:00
Kevin Hoxha 407c371635 metrics: Add simulation testing and fix incorrect TraceEvent names
- Added a background actor that listens on METRICS_EMISSION_UDP_PORT for incoming metrics (and verifies they are in the correct format)
- TraceEvent details have certain requirements for naming. This commit makes a seperate name for Counter/LatencySample and its underlying IMetric to avoid
those issues
2022-12-08 10:07:11 -08:00
Kevin Hoxha 3055f0d7b7 metrics: Add verifyStatsdMessage() for testing 2022-12-08 10:07:11 -08:00
Kevin Hoxha 188e635b2e metrics: Avoid name conflicts in MetricCollection
- In Counter constructor, make sure that we combine CounterCollection name
- In IMetric constructor, add a trace event and assertion in case of collision
- Rename ClusterRecoveryData CounterCollection name from master to ClusterRecoveryData
2022-12-08 10:07:11 -08:00
Kevin Hoxha 1d16bd9580 metrics: Fix bug in LatencySample::flush()
- Make sure that we check batch.statsd_message to be non-empty before adding a newline
- Make sure to actually add the message to the batch in flush()
2022-12-08 10:07:11 -08:00
Kevin Hoxha 8d151ee9b6 metrics: Added runMetrics() to workerServer 2022-12-08 10:07:11 -08:00
Kevin Hoxha 5a9d3343cc metrics: Add IMetricClient and StatsdMetric to send batches over UDP 2022-12-08 10:07:11 -08:00