Commit Graph

3626 Commits

Author SHA1 Message Date
Markus Pilman df0f491c29 Some more improvements to the build and preparations for packaging 2018-12-13 15:04:13 -08:00
Markus Pilman f8eac70700 Fixed C bindings DLLEXPORT and Java bindings 2018-12-13 14:59:36 -08:00
Markus Pilman 5fafeaf219 added bindings to cmake build 2018-12-13 14:54:38 -08:00
Markus Pilman 4833afea53 CMake compiles fdb c bindings 2018-12-13 14:54:13 -08:00
Alex Miller e70e59a895 Change some file locations. 2018-12-13 14:53:19 -08:00
Markus Pilman c06bb33730 CMake compiles all binaries except for bindings 2018-12-13 14:15:44 -08:00
Markus Pilman dce290909d fdbserver now compiling 2018-12-13 14:13:47 -08:00
A.J. Beamon bd52f8cbd6
Merge pull request #959 from bluk/fix-go-bindings-warning
Fix Go bindings possible nested comment warnings
2018-12-13 17:09:40 -05:00
mpilman 51beb8b48c fdbrpc compiling with cmake 2018-12-13 14:02:16 -08:00
mpilman f5fcb666c5 flow compiling with cmake on OS X 2018-12-13 13:58:06 -08:00
Bryant Luk 2379552aea
Fix Go bindings possible nested comment warnings
When using the Go bindings in a project, you can get compilation
warnings for the `/*` in the comment blocks. These warnings seem
unnecessary.

Warning is:

\# github.com/apple/foundationdb/bindings/go/src/fdb
In file included from _cgo_export.c:4:
cgo-gcc-export-header-prolog:43:1: warning: '/*' within block comment [-Wcomment]
cgo-gcc-export-header-prolog:44:1: warning: '/*' within block comment [-Wcomment]
2018-12-13 15:41:01 -06:00
Evan Tschannen d9626895b1
Merge pull request #964 from xumengpanda/mengxu/teamcollection-release
TeamCollection: Use machine teams to create server teams to increase availability at scale when a machine has multiple servers
2018-12-13 13:18:54 -08:00
Meng Xu 79d94f78f1 TeamCollection: Improve code efficiency
Further improve code efficiency by

1) Avoid rebuild machine locality map when machine locality is changed.
This may leave the global machine locality map stale.
This is ok as long as we do not use the global map to validate
the machine team follows the locality policy.

2) Use ASSERT_WE_THINK instead of ASSERT to avoid runtime overhead.
ASSERT_WE_THINK will only validate the condition in simulation mode.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-12 22:38:38 -08:00
Meng Xu e197926c80 TeamCollection: Remove a duplicate function
Remove a duplicate function that has different signature.
No functionality change.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-12 15:21:37 -08:00
Alec Grieser ed7f03de0d
Merge pull request #973 from atn34/patch-1
Make polling_loop yield agree with usage in docs
2018-12-12 14:59:17 -08:00
Henry 34956b4266
Merge pull request #971 from aarongreenlee/master
Removed typo in go binding `fdb-go-install.sh` file
2018-12-12 14:40:27 -08:00
Meng Xu ad7040efcd TeamCollection: Bug fix in handle server locality change
Make sure the link between server and machine is updated
in both server and machine.
Rename function name to better reflect its functionality.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-12 14:03:29 -08:00
Andrew Noyes 1a7d7ab068
Make polling_loop yield agree with usage in docs 2018-12-11 14:41:49 -08:00
Aaron Greenlee a1a88b2f47
Fixed apparent typo in help response 2018-12-08 17:27:32 -05:00
Meng Xu e069b5c31c TeamCollection: Use clang format
No functional change.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-06 11:39:35 -08:00
Meng Xu 5d47b9c884 TeamCollection: Handle server locality change
A server locality may change from one machine to another.
This affects the old machine and machine team the server is on, and
the new machine the server moves to.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-05 22:23:14 -08:00
Meng Xu c5047bc8c3 TeamCollection: All machine teams are correct size
We only create correct size machine teams.
When configuration (e.g., team size) is changed,
the DDTeamCollection will be destroyed and rebuilt
so that the invariant will not be violated.

Based on the invariant, we can count the number of
machine teams more quickly.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-05 15:09:38 -08:00
Meng Xu 57eab1f283 DataDistribution: Remove addAllTeams function
The addAllTeams function can be replaced with the new addTeamsBestOf
function by passing a large enough number of teams to build.
Remove addAllTeams function and update the related unit tests.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-05 15:03:16 -08:00
Meng Xu 38c5c2562b DataDistribution: Update NotEnoughServers unit test
The buggify option may set 1 to the knob parameters
(DESIRED_TEAMS_PER_SERVER and MAX_TEAMS_PER_SERVER).
When this happens, the number of machine teams to build will be
less than what we want, which prevents us from building enough
server teams.

To avoid this problem, we build machine teams before
we call addTeamsBestOf to build server teams.

We also add the ASSERT to ensure we build enough machine teams and
server teams in the test case.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-05 14:36:48 -08:00
Meng Xu f32c04c834 DataDistribution: Update NotEnoughServers unit test
Change the test condition for the NotEnoughServers unit test.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-03 23:14:01 -08:00
Evan Tschannen 62d4378109
Merge pull request #969 from etschannen/master
Merge 6.0 into master
2018-12-03 18:27:43 -08:00
Evan Tschannen d2d68aa171 Merge branch 'release-6.0'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/ManagementAPI.actor.cpp
#	versions.target
2018-12-03 18:26:52 -08:00
Evan Tschannen f78a5a4d1a
Merge pull request #968 from etschannen/post-release-cleanup-6.0.17
Post release cleanup 6.0.17
2018-12-03 18:22:10 -08:00
Evan Tschannen f6c9cf89bf update installer WIX GUID following release 2018-12-03 18:21:35 -08:00
Evan Tschannen decfe23408 update versions target to 6.0.18 2018-12-03 18:21:35 -08:00
Evan Tschannen 8e0db56449
Merge pull request #967 from etschannen/prepare-release-6.0.17
update installer WIX GUID following release
2018-12-03 16:33:12 -08:00
Evan Tschannen c41ccb687d update installer WIX GUID following release 2018-12-03 16:32:42 -08:00
Evan Tschannen 37991d40cb
Merge pull request #966 from etschannen/release-6.0
updated documentation for 6.0.17
2018-12-03 16:31:13 -08:00
Evan Tschannen 9935223818 updated documentation for 6.0.17 2018-12-03 16:30:22 -08:00
Evan Tschannen 55a9c4a0f0
Merge pull request #955 from ajbeamon/fix-bad-error-creation-and-whitespace
throw platform_error; -> throw platform_error();. Convert some spaces to tabs.
2018-12-03 15:12:37 -08:00
Evan Tschannen 9b09102f37
Merge pull request #962 from satherton/fix-backup-rangefile-count
Bug fix, after upgrade to 6.0.16 backup snapshot progress will halt
2018-12-03 15:11:50 -08:00
John Brownlee 33d2ad5753
Merge pull request #887 from brownleej/packaged-dockerfile
Add an official dockerfile
2018-12-03 13:47:51 -08:00
Alex Miller 2bc00fe5f4
Merge pull request #958 from etschannen/release-6.0
added safety check to configure commands
2018-12-03 13:32:19 -08:00
A.J. Beamon f7c7becf33
Merge pull request #960 from satherton/fix-typo-macos
Fixed typo causing mac build failure.
2018-12-03 11:22:20 -05:00
Stephen Atherton e974ee367d Bug fix, backups in progress during upgrade to 6.0.16 or later do not have have range file count initialized so reading it must default to 0. 2018-12-02 22:50:26 -08:00
Stephen Atherton e2673b2bbc Fixed typo causing mac build failure. 2018-12-02 22:03:21 -08:00
Meng Xu 54a4d6b308 TeamCollection: Improve code efficiency
Improve code efficiency with the following changes:
1) Change always-true if-statement to ASSERT;
2) Return when we are confident we will not find more machine teams.

No functionality change.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-01 17:10:50 -08:00
Meng Xu 8d6c6e000b DataDistribution: Mute the NotEnoughServers test
Due to the randomness in choosing a server, we cannot gurantee to
find all teams. The NotEnoughServers test case may create false positive
bug report in the correctness test.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-01 13:29:45 -08:00
Meng Xu 68dcec2240 DataDistribution: Change a unit test
Try multiple times of addTeamsBestOf() when we cannot find an available team
due to the pure randomness in choosing the server teams.

The changes for the unit test reduces the false positive in the simulation test results.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-01 13:12:55 -08:00
Meng Xu a43f579f66 TeamCollection: Change 1 unit test
Relax the assert condition on the random unit test.
Due to the randomness in choosing the machine team and
the server team from the machine team, it is possible that
we may not find the remaining several (e.g., 1 or 2) available teams.
For example, there are at most 10 teams available, and we have found
9 teams, the chance of finding the last one is low
when we do pure random selection.

It is ok to not find every available team because
1) In reality, we only create a small fraction of available teams, and
2) In practical system, this situation only happens when most of servers
   are *temporarily* unhealthy. When this situation happens, we will
   abandon all existing teams and restart the build team from scratch.

In simulation test, the situation happens 100 times out of 128613 test cases
when we run RandomUnitTests.txt only.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-12-01 13:11:19 -08:00
Evan Tschannen 6e94d7d71a changing the database configuration now checks that enough processes exist to support the new configuration 2018-11-30 18:52:24 -08:00
Alex Miller 3bf3fbed74
Merge pull request #957 from ajbeamon/throw-io-errors-in-platform
Errors that occur in platform that are the result of IO issues are now…
2018-11-30 18:11:11 -08:00
Meng Xu f311455c45 TeamCollection: Cleanup code and add checks
Remove unnecessary sanity checks and remove the dead code.
Add some necessary sanity checks.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-11-30 17:40:21 -08:00
A.J. Beamon 50c9dfdd01 Errors that occur in platform that are the result of IO issues are now raised as io_error rather than platform_error. 2018-11-30 10:55:19 -08:00
Meng Xu ea3bd1502d TeamCollection: Calculate machine team number
Calculate the number of machine teams in the same way
as we calculate the number of server teams.

Only count the machine teams that has the correct size and is healthy.

Simplify code by removing unnecessary check.

Signed-off-by: Meng Xu <meng_xu@apple.com>
2018-11-29 15:38:23 -08:00