Commit Graph

50 Commits

Author SHA1 Message Date
Richard Low 39894ea798 Merge remote-tracking branch 'apple/release-5.2' 2018-06-12 18:31:20 -07:00
Alex Miller cfa7fe8866 Identify processes with host:port regardless of if TLS is enabled or not.
This makes `kill` and `profile` behave like how `exclude` functions, and means
commands don't have to change depending on TLS status.

Verified via starting a TLS cluster and killing a process before and after this change.
2018-06-11 16:49:20 -07:00
A.J. Beamon e5488419cc Attempt to normalize trace events:
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.

Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.

This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
A.J. Beamon 026458baf3 Merge release-5.2 into master 2018-05-23 15:32:56 -07:00
Richard Low 086700aeb1 Plumb through TLS key password to CLI and from environment 2018-05-21 10:56:10 -07:00
Evan Tschannen 8f984cb2c9 Merge branch 'release-5.2'
# Conflicts:
#	fdbrpc/TLSConnection.h
2018-05-10 09:13:22 -07:00
Balachandar Namasivayam e8b7f4b190 Add password support for tls. 2018-05-08 20:46:31 -07:00
Balachandar Namasivayam d3b5cfb93c Support latest TLS plugin.
Add support for https in backup.
2018-05-08 16:28:13 -07:00
Meng Wang 8d1bd95f36 Fix the printStatus redundancy mode output
The key used to get the redundancy mode should match what's in JSON
2018-05-08 11:40:47 -07:00
Alec Grieser 42c8527f43
Merge remote-tracking branch 'upstream/release-5.2' into merge-release-5.2 2018-04-11 18:35:32 -07:00
Yichi Chiang d8175471bc Merge release-5.1.5 2018-04-11 17:55:10 -07:00
Evan Tschannen 44c0eaf4d7 fix: transactions created in fdbcli did not wait on the result of committing 2018-04-11 11:24:24 -07:00
Evan Tschannen 19762b847d Merge branch 'release-5.2'
# Conflicts:
#	fdbserver/DatabaseConfiguration.cpp
#	fdbserver/SimulatedCluster.actor.cpp
2018-04-10 17:02:43 -07:00
Evan Tschannen c1ba16b3c8 Merge branch 'release-5.1' into release-5.2
# Conflicts:
#	bindings/java/src/test/com/apple/foundationdb/test/AbstractTester.java
#	bindings/java/src/test/com/apple/foundationdb/test/VersionstampSmokeTest.java
#	bindings/nodejs/lib/fdb.js
#	bindings/nodejs/src/Version.h
#	bindings/nodejs/tests/tuple_test.js
2018-04-10 16:50:47 -07:00
Evan Tschannen 82ed956c65 renamed the multi_dc configuration to three_datacenter. The old three_datacenter configuration was not a useful configuration. 2018-03-26 18:31:26 -07:00
Alec Grieser 551ea9c7f8
Merge remote-tracking branch 'upstream/release-5.2' into master-release-5.2-merge 2018-03-19 12:34:50 -07:00
Alec Grieser 0853fcb052
switch to using zu for some size_t variables in printf 2018-03-14 18:07:05 -07:00
A.J. Beamon f2c804e14f Reverting changes from merge of master into release-5.2 (b25810711c). Note that we never intend to release master into release-5.2, but if we did we would need to revert this commit. 2018-03-06 10:15:04 -08:00
Evan Tschannen 37a6a81634 Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs
# Conflicts:
#	fdbserver/workloads/RestartRecovery.actor.cpp
2018-02-23 12:33:28 -08:00
Alec Grieser 0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
A.J. Beamon ceec580d2f fdbcli considered a backup running if the 'running_backup' field was present, not if it was true. 2018-02-16 10:09:03 -08:00
Evan Tschannen 5ac4f73978 Merge branch 'release-5.1' into feature-remote-logs
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
#	fdbrpc/Locality.h
#	fdbrpc/simulator.h
#	fdbserver/ApplyMetadataMutation.h
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/SimulatedCluster.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
#	fdbserver/WorkerInterface.h
#	fdbserver/masterserver.actor.cpp
#	flow/Net2.actor.cpp
#	tests/fast/SidebandWithStatus.txt
#	tests/rare/LargeApiCorrectnessStatus.txt
#	tests/slow/DDBalanceAndRemoveStatus.txt
2018-01-05 11:33:42 -08:00
Balachandar Namasivayam 486d089e98 Better message is displayed to the user. 2017-11-09 13:55:19 -08:00
Balachandar Namasivayam 9809e84806 Added a counter to keep track of active outgoing incompatible connections.
This counter is used to print a warning in fdbcli if there are incompatible peers.

Example Output:

./fdbcli
Using cluster file `fdb.cluster'.

WARNING: Incompatible peers exist.

The database is unavailable; type `status' for more information.

Welcome to the fdbcli. For help, type `help'.
fdb> status

WARNING: Incompatible peers exist.

Using cluster file `fdb.cluster'.

Could not communicate with a quorum of coordination servers:
  127.0.0.1:4000  (unreachable)
2017-11-09 11:20:35 -08:00
A.J. Beamon fefbf22b6f Remove redundant tagCount 2017-11-06 14:00:08 -08:00
A.J. Beamon 7d63c2cfb7 Formatting fixes 2017-11-06 09:20:31 -08:00
A.J. Beamon 3ae3200cc5 Untested implementation of feature to log basic backup and DR info to fdbcli status. 2017-11-03 14:02:03 -07:00
Balachandar Namasivayam 988bc0207f Reset Client Transaction profiling parameters when the config keys are cleared. 2017-10-31 15:40:57 -07:00
Evan Tschannen 54d82c0d92 Merge pull request #194 from cie/alexmiller/valgrind
Fix valgrind errors
2017-10-27 17:25:12 -07:00
Alex Miller e0d33ef8d7 Preemptively fix profiler-related valgrind errors/straight out bugs.
I forgot to initialize some fields in requests.
2017-10-27 17:20:19 -07:00
Evan Tschannen edfb839ad9 added a message to exclude to suggest retrying the exclude after 30 seconds 2017-10-25 13:32:25 -07:00
Evan Tschannen 48901a9223 added a list of tlog IDs that are missing to status 2017-10-24 16:28:50 -07:00
Evan Tschannen df74e2a373 re-added support for non-copying tlog recovery 2017-10-24 15:09:31 -07:00
Alex Miller cf646d4a99 Address review comments.
* Fixed fdbcli to be more idiomatic.
* Removed is_binary_serializable in favor of std::is_pod<>
* Removed custom enable_if<> in favor of std::enable_if<>
* Removed HEY REVIEWER comments
* Removed print from prof.py
* Added FLOW_PROFILER_ENABLED=yes to circus components that wished to enable the flow profiler.
2017-10-16 16:46:52 -07:00
Alex Miller 91a26a170c Add toggleable profiling support to fdbserver+fdbcli.
This adds the fdbcli commands:
* profile list -- Lists all workers in a way that doesn't fill `kill`'s list.
* profile flow run -- Allows starting flow profiling on a set of hosts for a specified interval.

And threads through all the support for enabling and disabling profiling as an RPC.
2017-10-16 16:05:02 -07:00
Alex Miller 2e662b6bb6 Fixing review comments.
* parse_with_units found a proper home in flow.h while this was pending
* atof->strtod for error checking
2017-10-04 14:00:38 -07:00
Alex Miller e55cc447d2 Address code review comments.
* Fixed memory corruption with SystemData key constants
* Removed duplication in ClusterController
* Reworked fdbcli actions to better represent explicit vs default assignments
2017-10-04 13:36:18 -07:00
Alex Miller 80fa597422 Allow client profiling to be configured from fdbcli.
This adds the following commands:
* profile client status
* profile client on 0.001 100MB
* profile client off
2017-10-04 13:36:18 -07:00
Evan Tschannen 6ea9903c82 Merge branch 'release-5.0'
# Conflicts:
#	fdbbackup/backup.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	versions.target
2017-10-01 18:46:44 -07:00
Evan Tschannen e2b65e86ed added configurable memory limits for backup and dr executables
added a default memory limit of 8GB for fdbcli
2017-09-29 10:35:40 -07:00
Evan Tschannen 6e26ae2bb3 added a new multi_dc configuration 2017-09-01 15:45:27 -07:00
Alex Miller 0c7fd0a23d Unify and clarify interactive and --exec command error and parse error handling.
Previously, interactive execution and --exec used two models for how a failed
command would impact other commands in the execution.

As an example, consider:

    fdb> set foo bar ; set bar f\00 ; set baz foo

In interactive mode, this had the effect of {foo=bar, baz=foo}.
In --exec mode, this had the effect of {foo=bar}.

With this change, both now have the effect of {foo=bar}.  This is achieved by
prefixing the last parsed command, which is the one that had the error, with a
fake "parse_error" token.

The execution of this would now look like:

    ERROR: malformed escape sequence
    >>> set foo bar
    ERROR: Command failed to completely parse.
    ERROR: Not running partial or malformed command: set bar

Which indicates how much execution occurred and where it halted, identically in
both modes of execution.
2017-08-21 13:42:01 -07:00
Alex Miller d78b29625c Make fdbcli gracefully handle malformed and partial command errors.
Previously, running a command like `set \xffx\02abcded/` would cause a crash.
The `x\02` is a malformed typo of `\x02`, and the previously existing code to
handle this case looks like

    loop {
      err = parse_command
      if (err)  continue;
      // do things
    }

Thus, if we hit an error, we'd go back to the top of the loop, and try again.
This should be an infinite loop.  However, the actor compiler implementation of
loops involves function calls, so this actually turns into a series of the loop
head calling the loop body calling the loop head calling ... and we eventually
crash due to running out of stack.

This is now fixed by simply letting the code continue on to the check later
that does

    if (there was an error) {
      print nasty message
      return error
    }

With output that looks like

    ERROR: malformed escape sequence
    WARNING: the previous command failed, the remaining commands will not be executed.

And therefore the world becomes a happy place.
2017-08-15 17:49:39 -07:00
John King f6d282e66a change exclusions so that they calculate free space based on storage servers only 2017-08-11 17:20:28 -07:00
A.J. Beamon 9ce8d3ae4f Merge branch 'release-5.0' 2017-08-09 10:37:43 -07:00
A.J. Beamon c7402eb10c fix: call initSignalSafeUnwind in fdbcli and backup to avoid hangs in slow task profiling. 2017-08-07 15:55:08 -07:00
Yichi Chiang 6a8a5c41b0 Add a switch to turn off data distribution in CLI 2017-07-28 18:14:55 -07:00
Evan Tschannen 15cb498aa7 removed fast_recovery_double and fast_recovery_triple from the fdbcli 2017-06-23 16:18:23 -07:00
Alvin Moore b28ed397a2 Fixed printf field width specifier to reduce compilation warnings within OS X 2017-05-26 14:51:34 -07:00
FDB Dev Team a674cb4ef4 Initial repository commit 2017-05-25 13:48:44 -07:00