Commit Graph

4294 Commits

Author SHA1 Message Date
Balachandar Namasivayam 7eba50b086 Add support for restoring multiple ranges. 2019-02-25 18:00:28 -08:00
Trevor Clinkenbeard abfe057805 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-25 13:47:16 -08:00
Trevor Clinkenbeard 78aad255b0 updateProcessStats gets process stats directly
Storage servers no longer parse ProcessMetrics trace lines to get their
own cpuUsage and diskUsage statistics.
2019-02-25 13:45:53 -08:00
A.J. Beamon ad8bfc1b8f
Merge pull request #1166 from alecgrieser/tuple-performance-improvements
Java Tuples: Hash and equality improvements
2019-02-25 15:10:45 -05:00
Trevor Clinkenbeard 07f800eeee Got rid of detailed field in GetRateInfoReply message 2019-02-23 17:52:11 -08:00
Trevor Clinkenbeard f3a73963b4 Got rid of detailedLeaseDuration in GetRateInfoReply message 2019-02-23 16:42:11 -08:00
Trevor Clinkenbeard ff9a7cb2f1 Combined proxy health metrics replies into single message type 2019-02-23 10:13:43 -08:00
Evan Tschannen fb9b5e05fd
Merge pull request #1168 from etschannen/feature-fix-force-recovery
fixed a number of bugs with forced recoveries
2019-02-22 15:35:08 -08:00
Evan Tschannen b8910ba7cd Merge branch 'master' into feature-fix-force-recovery
# Conflicts:
#	fdbclient/ManagementAPI.actor.h
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KillRegion.actor.cpp
2019-02-22 14:38:13 -08:00
Evan Tschannen d008de576e
Merge pull request #1139 from xumengpanda/mengxu/machine-team-upgrade-PR
Add background actor to remove redundant teams
2019-02-22 14:22:07 -08:00
Evan Tschannen d600b0a51c fix: getLocalityForDcId was returning counts rather than actual localities 2019-02-22 14:02:34 -08:00
mpilman 7316dbf458 ignore ccls-cache 2019-02-22 13:09:43 -08:00
Trevor Clinkenbeard edc0c5bf2b Client resends health metrics request when proxies change 2019-02-22 12:30:36 -08:00
Alex Miller 6d23eb2d1a Implement log_version.
This mega-commit introduces a new configuration setting, `log_version`,
that controls the TLog implementations and features that are available
within FDB, so that users can opt in to new features if they're willing
to sacrifice backwards compatibility.
2019-02-22 12:15:23 -08:00
Meng Xu 64db109f20 Status: Add schema for the new data distributor role 2019-02-22 10:05:12 -08:00
Trevor Clinkenbeard 2a20ea6ac5 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-22 09:41:47 -08:00
Meng Xu e5e594b648 TeamRemover: Cosmetic minor change
No functional change.
Remove 1 printf in simulation test.
2019-02-21 22:15:35 -08:00
Meng Xu 9445ac0b0c Status: Use new data distributor worker to publish status
After we add a new data distributor role, we publish the data
related to data distributor and rate keeper through the new
role (and new worker).

So the status needs to contact the data distributor, instead of master,
to get the status information.
2019-02-21 18:05:50 -08:00
Alec Grieser b3ed541632
Merge pull request #1177 from alexmiller-apple/missing-managementapi
Actually compile ManagementAPI.actor.cpp on windows.
2019-02-21 17:05:28 -08:00
Evan Tschannen 0e19b5a935 fix: allow the txnStateStore to be recovered from a process in a down datacenter, so that the cluster controller can know to switch to the other region 2019-02-21 16:52:27 -08:00
Alex Miller b6f7821ae4 Actually compile ManagementAPI.actor.cpp on windows.
<EnableCompile>false should only be applied to .actor.h files, so that
we don't try to compile the resulting .h file.

Apparently.
2019-02-21 16:28:14 -08:00
John Brownlee 655be7f32b
Merge pull request #1169 from alexmiller-apple/readme-build-docker
Change the build instructions to point to the published build image.
2019-02-21 16:24:25 -08:00
Trevor Clinkenbeard 0d7f26beb1 Removed unnecessary code from Throttling.actor.cpp 2019-02-21 16:20:10 -08:00
Meng Xu 3e703dc2d1 TeamRemover: Fix bug that may not remove all teams needed 2019-02-21 15:54:16 -08:00
Trevor Clinkenbeard fb925f8ca6 Improved Throttling workload
Test now fails if client health metrics stop updating. Added SevError
trace lines for different failure cases. Also fixed bug so that
(detailedWorstDiskUsage == 0) causes test failure when detailed health
metrics are sent.
2019-02-21 15:50:17 -08:00
Meng Xu db19b08762 TeamRemover: Add new status to fdbcli
Add the healthy_removing_redundant_teams status to fdbcli
2019-02-21 15:03:32 -08:00
Andrew Noyes 1292f5c59a Add TEST_LOG_FORMAT to cmake 2019-02-21 14:36:01 -08:00
Andrew Noyes 0503471ac1 Rename ../src in build/cmake/build.sh
Before this change:
$ sudo --preserve-env docker-compose run build
Starting cmake_common_1 ... done
Num commands 1
Command: build
CMake Error: The source directory "/foundationdb/deep/directory/as/debuginfo/doesnt/work/otherwise/src" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
2019-02-21 14:35:14 -08:00
Meng Xu 7cca439e00 TeamRemover: Add status to show redundant team removing
Distinguish the removal of unhealthy team and redundant team.
Change status report to include redundant team removal report.
2019-02-21 14:16:46 -08:00
Meng Xu 0ac7014142 TeamRemover: Resolve minor comments from code review 2019-02-21 13:18:11 -08:00
Evan Tschannen 171d119d95 another attempt to fix compiler errors 2019-02-20 18:35:08 -08:00
Evan Tschannen 329ab766f1 factored out a duplicate code block
attempted to fix a compiler error
2019-02-20 18:20:10 -08:00
Evan Tschannen 038a952939
Merge pull request #1069 from tclinken/add-no-assign-class
Add Coordinator class
2019-02-20 17:46:36 -08:00
Trevor Clinkenbeard 25b397977c Never assign DataDistributor role to process of class CoordinatorClass 2019-02-20 17:22:01 -08:00
Trevor Clinkenbeard fa96b8dd33 Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics 2019-02-20 16:56:16 -08:00
Meng Xu d86ba0e811 TeamRemover: Change it to run periodically
This simplifies the problem of when we should invoke the teamRemover
2019-02-20 16:08:34 -08:00
Alex Miller 7bd63cf5ea Added missing switch case. 2019-02-20 15:26:41 -08:00
Evan Tschannen 27e3617548 fix: remove bad teams needed to use dd_stall_check delay, because in simulation the buggified delay time could make us remove bad teams before they submit their ranges to the queue 2019-02-20 14:18:36 -08:00
Trevor Clinkenbeard 1bb384db4d Merge branch 'master' of https://github.com/apple/foundationdb into add-no-assign-class 2019-02-20 13:13:12 -08:00
Trevor Clinkenbeard a20f5482bc Created StorageStats struct to combine health metrics for storage servers 2019-02-20 11:57:41 -08:00
Trevor Clinkenbeard 1bb08b6e14 Minor bug fix in Throttling.actor.cpp 2019-02-20 11:46:24 -08:00
Trevor Clinkenbeard 7594606ee2 Use DETAILED_METRIC_UPDATE_RATE knob to determine GetRateInfoReply lease duration 2019-02-20 11:40:17 -08:00
Alex Miller d48cf3ec57 Make TLogSpillType cleaner now that it's not used in worker. 2019-02-19 22:30:15 -08:00
Alex Miller b725d841ea Restore a hash check as an ASSERT_WE_THINK 2019-02-19 22:30:15 -08:00
Alex Miller bf8bfb8137 Set log_spill in SimulationConfig.
Which also revealed that it needed to be added to the schema.
2019-02-19 22:30:15 -08:00
Alex Miller fa1bfbc0c5 Replace TLogSpillType with TLogVersion in worker and filenames. 2019-02-19 22:30:15 -08:00
Alex Miller 7b1afdc71e Hacky plumbing of spill type and file renaming. 2019-02-19 22:18:10 -08:00
Alex Miller 0cf3ee9f99 Add a way to configure log spill type. 2019-02-19 22:18:10 -08:00
Alex Miller df61bd07db Save an (unused) copy of the previous TLog. 2019-02-19 22:18:10 -08:00
Alex Miller 91e05575a2 Rename OldTLogServer -> OldTLogServer_4_6 2019-02-19 22:18:10 -08:00