Commit Graph

828 Commits

Author SHA1 Message Date
Alvin Moore 3bf971ba8b Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/storageserver.actor.cpp
2019-12-12 07:13:12 -08:00
Alvin Moore bdc16def8e
Merge pull request #2437 from AlvinMooreSr/publish_cmake
CMake Publishing Changes
2019-12-12 06:50:04 -08:00
Xin Dong 8016d02699 Added documents explaining the at-most-once guarantee provided by add callbacks function 2019-12-11 14:45:39 -08:00
Kao Makino b8ce7dde5f
Update bindings/c/test/mako/mako.rst
Co-Authored-By: Andrew Noyes <anoyes34@gmail.com>
2019-12-10 17:56:44 -08:00
Kao Makino 71f4e70a5d
Update mako.rst 2019-12-10 14:34:10 -08:00
A.J. Beamon d09e3e1d96 Apply go fmt. 2019-12-10 13:42:30 -08:00
A.J. Beamon bdbc97c452 Add docuentation about opening connections to multiple clusters 2019-12-10 12:38:37 -08:00
Alvin Moore 53ea319ec7 Added support for publishing jni library to the packages/lib directory 2019-12-10 11:44:45 -08:00
Kao Makino 38c3adbb84 Mako: fix minor bugs 2019-12-07 00:31:14 +00:00
chaoguang eacfdf6da3 Add a simple workload, ReportConflictingKeysWorkload, to test correctness of the API and performance overhead added to the resovler. 2019-12-06 16:21:03 -08:00
chaoguang edf52e8c97 First version for reporting conflicting keys 2019-12-02 13:45:39 -08:00
Evan Tschannen 4de60fc437 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/TLogServer.actor.cpp
2019-11-01 15:48:04 -07:00
Evan Tschannen 0953bf376d fixed Javadoc headings 2019-10-25 13:38:04 -07:00
Kao Makino 48aa55699a Cleanup mako c-binding benchmark 2019-10-24 18:14:55 +00:00
canardleteer d715e2909c Use the python module to guide PackWithVersionstamp's documentation. 2019-10-21 19:57:01 -07:00
canardleteer 7503f2f46c Remove unnecessary test 2019-10-19 14:11:05 -07:00
canardleteer 2e5e168d01 Add PackWithVersionstamp to Go Subpace & Directory bindings. 2019-10-15 14:19:04 -07:00
chaoguang e7b97c393d added zipfian distribution to mako workload 2019-10-15 01:14:21 -07:00
Vishesh Yadav b9a8e60318
Merge pull request #2091 from ajbeamon/binding-tester-python3
Update binding tester to use Python 3
2019-10-08 15:54:07 -07:00
Vishesh Yadav 62972b523c
Merge pull request #2133 from kaomakino/kaomakino/mako
Mako benchmark:  Add variable throttling
2019-10-08 15:26:52 -07:00
Balachandar Namasivayam 493d39be7a Revert "Removed unnecessary and ununed libraries from compilation command. Inclusion of this will produce an error with certain compilers such as Clang"
This reverts commit b10f3ad7a1.
2019-10-03 18:24:43 -07:00
Meng Xu d0147e5e5d Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3
Resolved Conflicts:
	documentation/sphinx/source/release-notes.rst
	fdbserver/DataDistribution.actor.cpp
	versions.target
2019-10-02 13:22:56 -07:00
A.J. Beamon 0bf7fdb450 Merge branch 'release-6.1' into merge-release-6.1-into-release-6.2
# Conflicts:
#	bindings/c/fdb_c.cpp
#	documentation/sphinx/source/release-notes.rst
2019-09-30 13:37:46 -07:00
A.J. Beamon 88c63637d3 Remove DLLEXPORT from various functions. Don't call public functions from the C bindings, as that has a bad interaction with the multi-version API. 2019-09-27 15:33:44 -07:00
A.J. Beamon e2f4d19c34 Don't export the fdb_future_get_version_v619 symbol. Mark fdb_future_get_version as a removed function. 2019-09-27 15:31:09 -07:00
Kao Makino 45e7406ddc Add variable throttling 2019-09-23 20:02:23 +00:00
Evan Tschannen b495cc697b Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-09-13 09:25:08 -07:00
Alvin Moore b10f3ad7a1 Removed unnecessary and ununed libraries from compilation command. Inclusion of this will produce an error with certain compilers such as Clang
Added the `-c` option to specify the source files being compiled. Exclusion of this will produce and error with certain compilers such as gcc 9.1
2019-09-12 11:08:01 -07:00
A.J. Beamon a84b6b8570 Fix versionstamp ordering in Python3 2019-09-11 10:39:01 -07:00
A.J. Beamon aa0d99c15e Update binding tester to use python3 2019-09-11 10:36:02 -07:00
Kao Makino e5bbaa293a Add FLOW_TCP_NODELAY and FLOW_TCP_QUICKACK (Linux-only) knobs 2019-09-10 20:13:15 +00:00
Andrew Noyes 95c6c9618e Update documentation and testers for new options 2019-09-05 16:40:12 -07:00
A.J. Beamon 5cf05214fd Update generated.go with new options 2019-07-29 18:41:28 -07:00
Evan Tschannen b509a441e7 Merge branch 'master' into feature-skip-confirm
# Conflicts:
#	bindings/flow/tester/Tester.actor.cpp
#	bindings/go/src/_stacktester/stacktester.go
#	bindings/java/src/test/com/apple/foundationdb/test/AsyncStackTester.java
#	bindings/java/src/test/com/apple/foundationdb/test/StackTester.java
#	bindings/python/tests/tester.py
#	bindings/ruby/tests/tester.rb
#	documentation/sphinx/source/api-c.rst
#	documentation/sphinx/source/api-python.rst
#	documentation/sphinx/source/api-ruby.rst
#	documentation/sphinx/source/data-modeling.rst
#	documentation/sphinx/source/developer-guide.rst
#	fdbclient/vexillographer/fdb.options
#	fdbserver/MasterProxyServer.actor.cpp
2019-07-27 15:08:13 -07:00
A.J. Beamon cbc913f902 When run through external means (such as the binding tester), the size limits tests should not change db level defaults. 2019-07-19 09:03:11 -07:00
A.J. Beamon bc5c65e5ab
Merge pull request #1756 from jzhou77/db-option
Add transaction getApproximateSize() API
2019-07-19 08:33:24 -07:00
Alec Grieser a26980682d
Merge pull request #1835 from TWarszawski/master
Add more information to the java binding jar's MANIFEST.
2019-07-18 14:01:46 -07:00
Jingyu Zhou 7fd772803e Better handle API version in mako.c
Only use fdb_future_get_int64 for API version >= 620.
2019-07-18 13:47:43 -07:00
A.J. Beamon 93be69b6a3 Increase the API version to 620. 2019-07-18 10:56:05 -07:00
Todd Warszawski 24c7960c12 Removing uneccessary attributes from MANIFEST 2019-07-16 12:46:40 -07:00
TWarszawski b0a6e2d2c4
Update bindings/java/CMakeLists.txt
Fix Specification-Title

Co-Authored-By: Alec Grieser <alloc@apple.com>
2019-07-16 11:22:10 -07:00
Jingyu Zhou 14cb21285f Remove futureGetVersion in C binding and FutureVersion in Python binding 2019-07-16 10:46:07 -07:00
Jingyu Zhou 94563464cb
Update bindings/c/foundationdb/fdb_c.h
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-16 10:15:47 -07:00
Evan Tschannen 7390df35a1 updated binding testers to set causalReadRisky 2019-07-15 11:39:03 -07:00
Todd Warszawski af35061200 Addressing reviewer comments. 2019-07-15 08:51:22 -07:00
Evan Tschannen c4c9e6cee7 fixed compiler errors 2019-07-12 18:28:41 -07:00
Evan Tschannen 02de53160d only skip confirm epoch live if CAUSAL_READ_RISKY is enabled
time checked on the proxy should be less than the time waited by the master to account for clock speed differences
setting REQUIRED_MIN_RECOVERY_DURATION and ENFORCED_MIN_RECOVERY_DURATION to 0 will go back to the old behavior
2019-07-12 17:58:16 -07:00
Jingyu Zhou 562bf6511a Fix approximate size calculation 2019-07-12 16:53:37 -07:00
A.J. Beamon d5051b08dd Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated. 2019-07-12 16:12:35 -07:00
Jingyu Zhou 990c205f70 Push a string to stack for getApproximateSize in binding testers
Because different bindings may issue different limit for get_range calls, it is
impossible to return the same size value for getApproximateSize API. So we just
push a string to make sure binding test results are the same. Use another unit
test to make sure the sizes got back are monotonically increasing.
2019-07-12 14:15:20 -07:00
Jingyu Zhou 9c3591ff43 Fix python3 test failure
Both key and value has to be of type bytes.
2019-07-12 10:15:40 -07:00
Jingyu Zhou 2dcc3cfd0a Deprecate fdb_future_get_version for version 620
Use fdb_future_get_int64 in all bindings.
2019-07-11 21:17:31 -07:00
Jingyu Zhou b2a89c8b77 Address review comments for PR #1756
Use fdb_future_get_int64 for language bindings and get rid of using Version
with getApproximateSize API.
2019-07-11 16:41:29 -07:00
Jingyu Zhou a82662a3bc
Update bindings/c/foundationdb/fdb_c.h
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-07-11 14:00:47 -07:00
Todd Warszawski ada31e7dcc Add more information to MANIFEST in jar. 2019-07-11 13:20:15 -07:00
Jingyu Zhou a8cd25bc46 Fix python3 test failure
Both key and value has to be of type bytes.
2019-07-10 21:55:24 -07:00
Jingyu Zhou 55d1d62ee3 Fix python tests and use 620 version for C test 2019-07-10 15:19:49 -07:00
Jingyu Zhou 20ed0bf802 Add get_approximate_size python unit test 2019-07-10 15:03:24 -07:00
Jingyu Zhou c70a426f04 Update approximate size calculation 2019-07-10 15:00:50 -07:00
Jingyu Zhou 5d1437c8e0 Push int directly to stack for getApproximateSize 2019-07-10 15:00:50 -07:00
Jingyu Zhou 0802df2c8f Convert size from int to string before pushing onto stack
Using int is troublesome because the size of int can be different from the
desired 64 bits. So, using a string representation seems to be more consistent.
2019-07-10 14:58:35 -07:00
Jingyu Zhou 9d12843a26 Push size as tuple to stack 2019-07-10 14:58:35 -07:00
Jingyu Zhou 78a5838312 Address reivew comments for PR #1756 2019-07-10 14:58:07 -07:00
Jingyu Zhou 14f7082428 Update binding API test documentation for GET_APPROXIMATE_SIZE 2019-07-10 14:58:07 -07:00
Jingyu Zhou d5aaba3b15 Minor code fix 2019-07-10 14:58:07 -07:00
Jingyu Zhou 7d6a317f2c Fix a type error 2019-07-10 14:58:07 -07:00
Jingyu Zhou 0ad2d2d16e Add binding test for getApproximateSize API 2019-07-10 14:58:07 -07:00
Jingyu Zhou cc47641bd3 Fix flow binding to use int64_t for get_approximate_size 2019-07-10 14:58:07 -07:00
Jingyu Zhou c17b314c3b Add a C binding unit test for fdb_transaction_get_approximate_size 2019-07-10 14:58:07 -07:00
Jingyu Zhou 8ef8b59fcc Use ThreadFuture for getApproximateSize
Change return type to int64_t and fix C and Python binding to use the correct
type.
2019-07-10 14:58:07 -07:00
Jingyu Zhou c91e712576 Add Python txn get_approximate_size API and test 2019-07-10 14:57:39 -07:00
Jingyu Zhou 4c0e824456 Include unactorcompiler.h at the end of *.actor.h 2019-07-10 14:51:52 -07:00
Jingyu Zhou 4a15be3643 Code format for fdbJNI.cpp 2019-07-10 14:51:52 -07:00
Jingyu Zhou c50a675bf0 Add transaction getApproximateSize() API
The size is the summation of expected size of mutations, read conflict ranges,
and write conflict ranges.
2019-07-10 14:51:52 -07:00
A.J. Beamon 15ecba59db Merge branch 'master' into fix-mvc-default-options
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2019-07-09 15:28:12 -07:00
Evan Tschannen ec11ef024b
Merge pull request #1798 from ajbeamon/merge-release-6.1-into-master
Merge release 6.1 into master
2019-07-08 09:02:56 -07:00
A.J. Beamon 2a56e011ea Merge branch 'release-6.1' into merge-release-6.1-into-master
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/DataDistribution.actor.cpp
2019-07-05 13:52:29 -07:00
Alex Miller 8e1ab6e7db Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-28 17:32:54 -07:00
A.J. Beamon c6df30deb3 Move size_limit.py to size_limit_tests.py 2019-06-28 13:24:56 -07:00
A.J. Beamon 2035b36257 Make default and persistent options specifyable via annotations to fdb.options. Fix some issues with persisting these options in the multi-version client. Make size limit option not persistent. 2019-06-28 13:24:32 -07:00
mpilman 7bfda1faaa Fixed three more Windows issues
This is now compiling on my Windows machine
2019-06-27 11:39:36 -07:00
Alex Miller a2a5bcc73d Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-26 20:31:09 -07:00
Andrew Noyes 231577adc6 Fix gcc9 warning 2019-06-25 14:35:32 -07:00
Alex Miller bf883d7055 Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-25 14:26:50 -07:00
Evan Tschannen 0fe6edc254
Merge pull request #1678 from mpilman/features/external-workload
Features/external workload
2019-06-25 13:53:19 -07:00
Alex Miller 7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
Alvin Moore 8ff5150470
Merge pull request #1636 from atn34/export-libfdb_c
Export libfdb c
2019-06-24 13:08:41 -07:00
Alec Grieser e8c75505d3
Merge pull request #1725 from jzhou77/db-option
Add transaction size option
2019-06-21 08:25:34 -07:00
Jingyu Zhou 16765c6b67 Fix a comment 2019-06-20 20:52:29 -07:00
mpilman 923a89748c removed dead code 2019-06-20 14:34:23 -07:00
mpilman ab019fbe41 More minor fixes, removed snapshots 2019-06-20 14:28:31 -07:00
Jingyu Zhou 77d5932465 Add a transaction size limit test 2019-06-20 14:14:09 -07:00
Jingyu Zhou 3a63d053e9 Address review comments for PR#1725 2019-06-20 14:06:32 -07:00
mpilman 844dd60202 FDB compiling with intel compiler 2019-06-20 09:29:01 -07:00
mpilman ab7562160c Made JavaWorkload an external workload 2019-06-19 13:03:41 -07:00
mpilman 5150782e86 SimpleWorkload now succeeding 2019-06-19 13:03:41 -07:00
mpilman 2eff2b7e21 First simple test is working (but very buggy) 2019-06-19 13:03:41 -07:00
mpilman 1707f068e0 started implementation first c workload 2019-06-19 13:03:41 -07:00
mpilman c8957d93f8 Implementation code complete 2019-06-19 13:03:41 -07:00
Jingyu Zhou 9c2257a0e5 Add transaction size option 2019-06-19 07:45:23 -07:00
A.J. Beamon 88e765b9e6 Fix: the binding tester was taking the min() of a list of tuples, but that could fail if the tuple contained incomparable types. Instead, use fdb.tuple.compare() to do the comparison. 2019-06-17 11:43:58 -07:00
Kao Makino 899db09a87 Fix retry logic 2019-06-12 12:38:07 -07:00
Evan Tschannen 566f546e04
Merge pull request #1646 from kaomakino/kaomakino/mako
mako (C API) benchmark
2019-06-11 13:36:06 -07:00
Andrew Noyes f5e7b8b165 Support pkg-config 2019-06-09 15:21:36 -07:00
Andrew Noyes 9af6326e1b Make find_package work 2019-06-09 15:21:36 -07:00
Andrew Noyes 882523330c include(/path/to/cmake/file) now works for fdb_c 2019-06-09 15:21:36 -07:00
Andrew Noyes cabe41626c Prepare for exporting fdb_c 2019-06-09 15:21:36 -07:00
Andrew Noyes 2b07a0559e Install headers in foundationdb dir 2019-06-09 15:21:36 -07:00
Kao Makino 431513c654 Make mako sources a variable 2019-06-06 16:58:11 -07:00
Kao Makino db300f71c5 Build mako automatically 2019-06-06 14:53:26 -07:00
Evan Tschannen 5bdf5aaeb6
Merge pull request #1662 from etschannen/master
Merge 6.1 into master
2019-06-06 13:57:34 -07:00
Evan Tschannen cfa46ef48e fix compiler errors 2019-06-06 12:57:46 -07:00
Kao Makino c812224ddb Adjust mako.rst formatting 2019-06-05 17:26:40 -07:00
Kao Makino d2d5d52dca add mako.rst 2019-06-05 17:21:26 -07:00
Kao Makino 8a3facbb77 Add the error handling for commits 2019-06-05 13:19:53 -07:00
A.J. Beamon dbfa746494 Don't set the deterministic random seed with platform::getRandomSeed since that is what it does by default. 2019-06-05 11:48:29 -07:00
Kao Makino 1c7fa2bc7a Update build instruction 2019-06-04 17:38:05 -07:00
Kao Makino 42c14dcdc2 Fix build stuff 2019-06-04 17:35:47 -07:00
Evan Tschannen 29b96414e2 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/NativeAPI.actor.cpp
#	fdbserver/Coordination.actor.cpp
#	flow/Arena.h
#	versions.target
2019-06-03 18:49:35 -07:00
A.J. Beamon b30fe29c9f
Merge pull request #1633 from pjvds/directory-errors
change common directory errors for easy comparison
2019-05-31 15:19:21 -07:00
Pieter Joost van de Sande b5773d2f6b
fix spelling error
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-05-30 18:16:29 +02:00
Kao Makino ae0a5388c9 mako creation 2019-05-30 09:01:22 -07:00
Evan Tschannen 2166464c3b changed how printable was called to fix compile errors 2019-05-29 15:58:00 -07:00
Evan Tschannen 362c2bf1e6 improved the cpu efficiency of printable 2019-05-29 14:55:45 -07:00
Pieter Joost 19de1ae229 include errors in the dir layer documentation 2019-05-29 11:15:00 +02:00
A.J. Beamon 764ca2971c Merge branch 'master' into thread-safe-random-number-generation 2019-05-23 18:49:49 -07:00
A.J. Beamon 296adada93 Use c++11 thread_local instead of redefining it to platform specific variants. 2019-05-23 18:35:45 -07:00
A.J. Beamon 603721e125 Merge branch 'master' into thread-safe-random-number-generation
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
2019-05-23 08:35:47 -07:00
Pieter Joost van de Sande 7debb23cf6 change common errors for easy comparison 2019-05-23 13:40:00 +02:00
A.J. Beamon 250955d18c Update required Go version in the Go binding's README.md 2019-05-22 08:54:42 -07:00
A.J. Beamon 2e09a214d9
Merge pull request #1451 from segmentio/master
bindings/go: use keepalive(f) on futures when they're used
2019-05-21 13:21:50 -07:00
Alec Grieser 76333ad87a
Resolves #1612: Go bindings: Use single line comments in CGO
This converts the multi-line comments for C imports to single line comments, which at the least is more idiomatic.
2019-05-20 13:49:22 -07:00
Alvin Moore 3acaa7343e Enabled C++17 for all Windows projects
Set Visual Studio version to 2017 (first version to support C++17)
2019-05-16 17:44:13 -07:00
Alvin Moore 94aed513c7 Switched Windows tools within projects to 2017 2019-05-16 15:05:11 -07:00
Alvin Moore bfee87bff6
Merge pull request #1567 from AlvinMooreSr/build-docker
Updated Build Docker
2019-05-15 15:01:22 -07:00
Alvin Moore 2ed1176d6a Add support for statically linking stdc++ to libfdbc_c on Centos while using custom hack for Ubuntu
Set compilationn flag to c++17
2019-05-15 02:37:16 -07:00
Alvin Moore b0362a654e Removed the debug compilation command display 2019-05-13 23:30:41 -07:00
Alvin Moore 7e4da69343 Added stdc++ library to link libraries 2019-05-13 23:25:00 -07:00
Alvin Moore 4d32909837 Moved standard libraries to LDFLAGS rather than LIBS 2019-05-10 15:22:41 -07:00
A.J. Beamon 5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Colin 9a85c19063
bindings/go: fix generated int64ToBytes calls 2019-05-10 15:20:59 -05:00
Colin Adler 93c1dced16
bindings/go: add labels for futures 2019-05-10 15:18:44 -05:00
Colin Adler 0d7748d59f
bindings/go: use binary.LittleEndian.PutUint64 instead of binary.Write
Benchmarks can be found in bindings/go/bench
2019-05-06 15:48:28 -05:00
Colin Adler 8033a0e717
bindings/go: format lines over 120 characters long 2019-05-06 15:37:58 -05:00
Austin Seipp ca83a80168 bindings/c: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Austin Seipp aab32eb855 bindings/flow: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Alvin Moore 648fc8ec7c
Merge pull request #1472 from mpilman/fixes/no-libfdbc-in-jar
Remove fdb_c from fat-jar
2019-05-01 14:23:56 -07:00
Evan Tschannen 2d5043c665 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-04-30 18:27:04 -07:00
Alvin Moore 8badc2e505 Merge pull request #1338 from mpilman/features/stripped-binaries
Strip debug symbols with `make packages`
2019-04-25 12:02:10 -07:00
Trevor Clinkenbeard 9fea60675f Fix comments to use transaction_too_old instead of past_version 2019-04-24 18:50:57 -07:00