Commit Graph

357 Commits

Author SHA1 Message Date
A.J. Beamon e4b1489a0e
Merge pull request #3202 from ajbeamon/cmake-use-stripped-binaries-for-packages
cmake: use stripped binaries for packages unless generating debug packages
2020-05-27 11:36:38 -07:00
A.J. Beamon 1f1a8df7ef Auto completion for "throttle disable auto" were wrong because of logical operator grouping problems. 2020-05-22 09:48:53 -07:00
A.J. Beamon 2f9cd383ea Use stripped executables in packages when not building with debug symbols. 2020-05-21 13:39:55 -07:00
tclinken 3a8efb2d0b Added advanceversion command to fdbcli 2020-05-20 15:47:31 -07:00
tclinken 40aec61d30 Added fdbcli getversion command 2020-05-20 15:47:30 -07:00
tclinken 4f5bf41ed8 Print explanation when fdbcli unlock fails 2020-05-20 15:47:30 -07:00
tclinken 9c6bec1af6 Add unlock command to fdbcli 2020-05-20 15:47:30 -07:00
tclinkenbeard b3aa54e7eb Add lock command to fdbcli 2020-05-20 15:47:30 -07:00
A.J. Beamon cc4874918a Merge branch 'release-6.3' into tag-throttling-by-priority
# Conflicts:
#	fdbserver/Ratekeeper.actor.cpp
2020-05-20 14:26:35 -07:00
Evan Tschannen a3d4f3f4de
Merge pull request #3175 from ajbeamon/fdbcli-throttle-improve-hints-and-completion
Better hint and completion help with throttle command in fdbcli
2020-05-20 14:25:03 -07:00
A.J. Beamon b589c3c940 Remove IncludeVersions.h, which is no longer needed for cmake-only builds 2020-05-20 13:23:02 -07:00
A.J. Beamon 14b23c146f Support throttling and unthrottling tags by priority and their auto/manual state in fdbcli. 2020-05-15 12:47:55 -07:00
A.J. Beamon f1477b09e9 Better hint and completion help with throttle command in fdbcli. 2020-05-14 16:04:03 -07:00
Andrew Noyes 8bd5dcaff8 Merge branch 'master' into atn34/special-key-versioning 2020-05-09 15:34:20 -07:00
Andrew Noyes a7c21cdadd Use new worker_interfaces key range in fdbcli 2020-05-08 05:37:37 +00:00
A.J. Beamon b1055a8501 Merge branch 'master' into transaction-tagging 2020-05-05 16:03:39 -07:00
Alex Miller 78d644f673
Merge branch 'master' into bugfixes/cmake-includes 2020-05-04 02:11:46 -07:00
A.J. Beamon decf3e82b0 Fix various bugs and make sure to cleanup throttles from the database when they expire 2020-05-01 21:36:28 -07:00
A.J. Beamon 41c517a5dd Merge branch 'master' into transaction-tagging
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
2020-04-27 13:05:24 -07:00
A.J. Beamon 239876351b Add some initial auto-throttling. Move the definition of the priority enum to a more global place and use it for all transaction priorites (except in ClientLogEvents, because of serialization incompatibilites). 2020-04-24 11:31:16 -07:00
A.J. Beamon 35c18ac60a Improvements to expiration. Encode throttles with auto/manual and priority in the key to support throttling the same tag with different values in these parameters. 2020-04-23 20:50:40 -07:00
A.J. Beamon f1dd0ee298 Protect against a ratekeeper starting up with a clock set in the past (compared to old ratekeeper) extending the duration of throttles excessively. 2020-04-21 16:35:25 -07:00
A.J. Beamon 78d48a0dad Merge branch 'master' into transaction-tagging
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/Knobs.h
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/fdbclient.vcxproj
#	fdbserver/MasterProxyServer.actor.cpp
2020-04-17 09:23:18 -07:00
A.J. Beamon 2b66dcd24a Some more refactoring. Reduce what is sent from RK->MP->clients 2020-04-17 08:07:01 -07:00
A.J. Beamon 0fba8c47be Checkpoint: Ratekeeper sets absolute limits for tag throttles and enforces them by distributing requests to proxies, who distribute them to clients.
A few refactorings.
2020-04-16 14:43:22 -07:00
tclinken 93400d25c8 Added advanceversion command to fdbcli 2020-04-15 20:01:01 -07:00
A.J. Beamon bcb0a4cc78
Merge pull request #2890 from tclinken/expose-lock-fdbcli
Add lock and unlock commands to fdbcli
2020-04-10 13:54:47 -07:00
A.J. Beamon 7f3fa00897 Reorganize a bit of code. 2020-04-10 13:53:23 -07:00
A.J. Beamon 55a0d00ad4 Encoding of tags in the database now supports multiple tags per throttle. Remove throttle prefix search. 2020-04-10 10:12:26 -07:00
tclinken b44105b54c Print explanation when fdbcli unlock fails 2020-04-08 16:38:30 -07:00
A.J. Beamon af4e0088ba
Merge pull request #2896 from tclinken/atomically-update-dependent-knobs
Atomically update dependent knobs
2020-04-08 15:00:49 -07:00
tclinken 52860043c9 Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs 2020-04-08 12:26:21 -07:00
A.J. Beamon 36da61dd9c Merge branch 'master' into transaction-tagging
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/vexillographer/fdb.options
2020-04-07 21:12:14 -07:00
A.J. Beamon a1d8623e5f Various changes to the throttling scheme, including most notably that clients now enforce the throttles with info they receive from the proxy. 2020-04-07 16:28:09 -07:00
Markus Pilman d4542dbb5a Delete old build system 2020-04-07 11:03:45 -07:00
Markus Pilman e4611e8ae4 fix versions.h stupidity 2020-04-06 10:28:55 -07:00
Markus Pilman 8b5780c36c don't include source and binary dir
This forces users to use include paths from the sources root.

So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
A.J. Beamon 2336f073ad Checkpointing a bunch of work on throttles. Rudimentary implementation of auto-throttling. Support for manual throttling via fdbcli. Throttles are stored in the system keyspace. 2020-04-03 15:24:14 -07:00
A.J. Beamon 638369492f
Merge pull request #2863 from dongxinEric/feature/2290/expose-region-config-via-status-default-or-details
Expose region related configurations in status
2020-04-03 11:42:43 -07:00
Xin Dong eaae9397e5 Address review comments 2020-04-02 11:04:58 -07:00
Markus Pilman bbd2fe62cc Merge branch 'master' of github.com:apple/foundationdb into features/boost70 2020-04-02 09:21:01 -07:00
tclinken aa7d5d8816 Add unlock command to fdbcli 2020-04-01 17:39:16 -07:00
tclinken 884e92bb49 Atomically update dependent knobs 2020-04-01 15:18:49 -07:00
Xin Dong e755583c07 Address review comments. 2020-04-01 15:13:04 -07:00
Xin Dong 8e489e7031
Merge pull request #2870 from dongxinEric/feature/2399/provide-comprehensive-info-about-exclude-in-fdbcli
Instead of providing partial information about exclude results in dif…
2020-03-31 11:58:41 -07:00
Xin Dong 18356ec4e6
Apply suggestions from code review
Co-Authored-By: John Brownlee <mail@johnbrownlee.com>
2020-03-31 11:11:24 -07:00
tclinken bee6acb717 Added fdbcli getversion command 2020-03-30 17:12:52 -07:00
tclinkenbeard bb6a8bf7ab Add lock command to fdbcli 2020-03-30 16:19:18 -07:00
Xin Dong 012d41548e Address review comments 2020-03-30 13:55:59 -07:00
Xin Dong 52b6754598 Align up the outputs 2020-03-27 15:45:51 -07:00
Markus Pilman 28cd38913d Merge branch 'master' of github.com:apple/foundationdb into features/boost70 2020-03-27 13:44:10 -07:00
Xin Dong 248a097c58 Instead of providing partial information about exclude results in different scenarios, provide comprehensive result for all cases. 2020-03-26 13:15:47 -07:00
Xin Dong d7c7bbbea5 Fix macOS build failures 2020-03-25 16:39:41 -07:00
Xin Dong 040c953f74 Expose region related configurations in status 2020-03-25 16:34:24 -07:00
Evan Tschannen e08f0201f1 merge release 6.2 into master 2020-03-17 12:51:47 -07:00
Meng Xu b44747fe12
Merge pull request #2726 from alexmiller-apple/fdbcli-hints
Update LineNoise, and use hints to provide inline argument descriptions.
2020-03-17 11:24:44 -07:00
Alex Miller 8e96e5a525
Expand comment about ignoring malformed escape sequences 2020-03-16 18:25:56 -07:00
Alex Miller 6ee992eb35 Add that debug tls exits after running. 2020-03-13 15:47:16 -07:00
Alex Miller a5568b2fc6 Rewrite tlsinfo into --debug-tls, and print out configuration. 2020-03-13 15:46:03 -07:00
Alex Miller 0c558efcfe Add a `tlsinfo` command to fdbcli that prints the certificate chain.
This requires the certificate chain to load successfully, otherwise
fdbcli will error out at an earlier point due to Net2 not being able to
configure TLS.
2020-03-13 00:11:53 -07:00
Evan Tschannen 303df197cf Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/flow.vcxproj
#	flow/flow.vcxproj.filters
#	versions.target
2020-03-06 18:22:46 -08:00
Alex Miller 595dd77ed1 Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh 2020-03-04 20:25:42 -08:00
Alex Miller 9b5ef3416e Refactor TLSParams into TLSConfig + LoadedTLSConfig
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.

initTLS now just takes the in-memory bytes and applies them to the ssl
context.

This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
A.J. Beamon 58e621eca1 Invalid knobs or knob values are treated as warnings rather than errors. Apply this change to backup as well. 2020-03-04 15:50:04 -08:00
A.J. Beamon 9862aa8bed Add support for setting knobs in fdbcli 2020-03-04 11:15:32 -08:00
Alex Miller 243a4c4480 Reduce code indentation 2020-03-03 18:14:57 -08:00
Alex Miller 6d14aacb40 I forgot to add the last change before committing and pushing the last commit. 2020-03-01 02:00:45 -08:00
Alex Miller e997be3e2d Make fdbcli act like redis-cli, and a couple fixes.
Hints now show up as a darker gray text.

Hint components now only show for pieces of the command string that
hasn't yet been typed.

The code now correctly handles offering hints if multiple commands are
issued at once seperated by ';'.

A hint of "{malformed escape sequence}" is displayed if a bad escape
sequence was typed.  e.g. \e
2020-02-29 03:07:39 -08:00
Evan Tschannen c11c24b79d removed the fdbrpc version of platform.h 2020-02-28 14:56:10 -08:00
Evan Tschannen 6054c05963 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/fdbserver.actor.cpp
#	versions.target
2020-02-28 12:11:05 -08:00
A.J. Beamon 0f5c999d4b Better containment of boost errors related to TLS. 2020-02-26 12:26:43 -08:00
Alex Miller b76f68d0e2 Add a --no-hints flag to fdbcli to disable linenoise hints 2020-02-24 16:08:04 -08:00
Alex Miller 9545091899 Merge remote-tracking branch 'upstream/master' into fdbcli-hints 2020-02-24 16:02:30 -08:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Alex Miller 390330176f Update LineNoise, and use hints to provide inline argument descriptions.
This makes it so that if you type:

    fdbcli> setclass

Then fdbcli shows

    fdbcli> setclass <ADDRESS> <unset|storage|transaction|default>
                    ^

With the cursor located where the ^ is pointing.
2020-02-21 17:15:40 -08:00
Evan Tschannen 3a3f6afc4c
Update fdbcli/fdbcli.actor.cpp 2020-02-20 18:04:44 -08:00
Evan Tschannen f04e311a1e Merge commit 'b46d6e25e24993ab5a5f04091fd3235050b7cd09' into feature-boost-ssl
# Conflicts:
#	fdbserver/SimulatedCluster.actor.cpp
#	flow/Net2.actor.cpp
2020-02-20 17:36:38 -08:00
Alex Miller 927cff3317 Report errors on TLS misconfigurations ... or at least try to. 2020-02-20 16:57:29 -08:00
A.J. Beamon 4f1301b2dd
Merge pull request #2583 from etschannen/feature-keep-status-connected
Clients should not disconnect from the CC after fetching status
2020-02-20 13:12:30 -08:00
Evan Tschannen 08c318d28a re-added the connect lock in the fdbcli so that the timeout is not spent before a connection has been initiated (because of the handshake lock) 2020-02-20 10:43:34 -08:00
Evan Tschannen 761da5a059 code cleanup 2020-02-19 17:59:45 -08:00
mpilman 3a1e878a9b Upgrade to boost 1.72 2020-02-14 18:10:13 -08:00
Evan Tschannen 53d0867a17 limit the number of connections a process can attempt to establish in parallel 2020-02-04 18:15:10 -08:00
mpilman d09e07f1f5 Merge remote-tracking branch 'upstream/master' into features/icc 2020-02-04 10:26:18 -08:00
Andrew Noyes 61a727d701 Allow new databases to be configured as locked 2020-02-03 15:32:01 -08:00
Alex Miller ee6490c9d1
Merge pull request #2314 from mengranwo/memory-engine
New Radix-Tree based Memory Storage Engine
2020-01-30 16:20:13 -08:00
Evan Tschannen 73ad702d14 Clients which fetch status should not disconnect from the coordinators and cluster controller between each retrieval 2020-01-22 15:41:22 -08:00
Evan Tschannen 3f9d9d8b84 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
2020-01-16 18:37:47 -08:00
mengranwo 227edd4248 change memory storage engine name from memory-radixtree to memory-radixtree-beta 2020-01-15 13:49:45 -08:00
mengranwo f597aa7e18 WIP : deployable/stable version since Nov 3. Start rebase to master branch 2020-01-15 13:49:45 -08:00
Alvin Moore c46aebb718 Added bigobj compilation option in order to fix problem with too many sections in object file for Windows 2020-01-13 00:54:28 -08:00
Evan Tschannen 9a3dfec7c5 open a connection with processes before attempting to kill them to improve the reliability of the kill process
secondaryAddresses are included in the list of processes which can be killed
2020-01-03 16:10:44 -08:00
Jon Fu 92892d4168 Merge branch 'master' of https://github.com/apple/foundationdb into include-failed-ss 2019-11-25 11:05:35 -08:00
A.J. Beamon ed8d3f163c Rename hgVersion to sourceVersion. 2019-11-15 12:26:51 -08:00
Andrew Noyes d4de608bb6 Fix OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
Jon Fu bedb38d3e8 fixed compilation error 2019-10-24 13:05:28 -07:00
Jon Fu 8930172e4a fixed includeall case 2019-10-23 11:12:10 -07:00
Jon Fu 9db95bd976 initial commit to allow re-inclusion of servers marked as failed 2019-10-23 11:05:48 -07:00
Jon Fu f38d52b82d re-introduced coordinator quorum and added some comments 2019-10-18 14:52:07 -07:00
Jon Fu 896701006f addressed code review changes 2019-10-16 11:30:20 -07:00