Commit Graph

583 Commits

Author SHA1 Message Date
Yichi Chiang f39cce9b8d Use processId instead of address for comparison 2017-10-25 11:35:29 -07:00
Yichi Chiang 5fcef911f0 Take cluster controller role into consideration when recruiting workers 2017-10-25 10:35:46 -07:00
Evan Tschannen 48901a9223 added a list of tlog IDs that are missing to status 2017-10-24 16:28:50 -07:00
Bhaskar Muppana eba8151a09 Merge pull request #190 from bmuppana/backup-joshua-fix
Fixing Backup wrong key range issue.
2017-10-24 15:28:18 -07:00
Bhaskar Muppana 3848f7808c Fixing Backup wrong key range issue. 2017-10-24 15:26:55 -07:00
Yichi Chiang c2a117fe07 Merge pull request #189 from cie/enable-check-desired-class
Enable checkUsingDesiredClasses() in consistency check
2017-10-24 15:18:21 -07:00
Yichi Chiang defdc6550d Exclude excluded processses when getting testers 2017-10-24 15:16:34 -07:00
Yichi Chiang 3865c5ae0e Enable checkUsingDesiredClasses() in consistency check 2017-10-24 12:58:54 -07:00
A.J. Beamon ffda3a3431 Merge pull request #187 from cie/bindings-java-subspace-hash
<rdar://problem/35125601> Java Bindings: Add hashCode method to subspace
2017-10-24 10:44:33 -07:00
Alec Grieser 49fb135c5a Merge remote-tracking branch 'origin/release-5.0' 2017-10-24 08:29:19 -07:00
Alec Grieser cc37288933 Merge pull request #179 from cie/java-docs-update
Update our Java sample code
2017-10-24 08:28:40 -07:00
Bhaskar Muppana 2348c69586 Merge pull request #188 from bmuppana/backup-joshua-fix
Fixing memory corruption in KeyBackedTypes.h
2017-10-23 18:04:48 -07:00
Bhaskar Muppana 7799aef549 Fixing memory corruption in KeyBackedTypes.h 2017-10-23 18:03:53 -07:00
Balachandar Namasivayam 8c3bdc5b3b Make atomic ops differentiate between unset and empty values. 2017-10-23 16:48:13 -07:00
Alec Grieser 9baa8c30b5 added hashCode implementation to subspace 2017-10-23 09:12:45 -07:00
Stephen Atherton 42955012e9 Merge branch 'release-5.0'
# Conflicts:
#	fdbrpc/BlobStore.actor.cpp
#	flow/error_definitions.h
2017-10-20 21:16:55 -07:00
Stephen Atherton 0b26e5af42 Merge pull request #185 from cie/improve-backup-error-handling
Improve backup error handling
2017-10-20 18:43:04 -07:00
Stephen Atherton efe857fef6 Fixed inconsistent styles of recently changed error messages. 2017-10-20 12:56:00 -07:00
Stephen Atherton 9f151314b3 Changed some trace event severities. Also fixed a weird casing of “retryable”. 2017-10-19 17:47:42 -07:00
Stephen Atherton 5718d32553 Bug fix, file backup on Windows has been broken for a long time because it was constructing a path using / and then checking that it matches a normalized absolute path which would have the / converted to \. 2017-10-19 13:33:12 -07:00
Bhaskar Muppana f6822a4f6b Merge pull request #186 from bmuppana/backup-joshua-fix
Backup joshua fixes
2017-10-19 08:17:38 -07:00
Stephen Atherton caad691ae2 Added comments for how to handle HTTP 400 errors gracefully in certain instances should the need arise. 2017-10-18 23:47:59 -07:00
Bhaskar Muppana 360b777b78 Fail with correct error code in case of abort or discontinue of
non-existing backups.
2017-10-18 23:17:48 -07:00
Stephen Atherton 09e97e1e7e TaskBucket now logs a trace event for any task execution failures. Previously only external timeouts were logged, but now timeouts or any other error from inside the task is logged as well. 2017-10-18 17:26:18 -07:00
Alec Grieser dd6d8f3b0e Merge branch 'master' into add-new-atomic-ops 2017-10-18 16:36:44 -07:00
Alex Miller d3df8469dd Upgrade protocol version as ClientWorkerInterface was changed. 2017-10-18 14:56:31 -07:00
Bhaskar Muppana 2007f3799f Don't ignore TimeKeeper failures. 2017-10-18 14:31:31 -07:00
Bhaskar Muppana 314511f4d7 Fixing spaces in BackupCorrectness TraceEvents. 2017-10-18 14:27:52 -07:00
A.J. Beamon 795f178b11 Merge pull request #178 from cie/bindings-java-repackage
<rdar://problem/33271641> "cie" should be removed from Java binding package path
2017-10-18 13:34:07 -07:00
A.J. Beamon a5c2373dbb Spaces->tabs 2017-10-18 09:04:35 -07:00
A.J. Beamon 050c1bcba6 Update transaction_too_old error text. 2017-10-18 08:49:17 -07:00
Stephen Atherton ef84e52127 Improved error handling and memory usage in AsyncFileBlobStoreWrite. Writes will now fail if any upload has already failed, rather than buffering unboundedly until sync() is called to complete the file. There is also a configurable limit on how many uploads can be pending before writes will stall waiting for one to finish. 2017-10-18 05:51:30 -07:00
Stephen Atherton ebd0234514 Rewrote most error handling in BlobStoreEndpoint to fix several shortcomings in error handling and logging. The request loop now logs but rate limits all errors, and the exceptions thrown are more appropriate. HTTP 503 is now treated as retryable. Callers of BlobStoreEndpoint::doRequest() now specify which codes they consider to be successful so that more error handling can take place in the main request loop. 2017-10-18 02:52:09 -07:00
Alec Grieser 479c9fab1b added experimental warning to external callbacks docs 2017-10-17 13:18:58 -07:00
Alec Grieser 191aca0a3c removed reference to cie in vexillographer.csproj 2017-10-17 09:30:53 -07:00
Alec Grieser fc2ff719d0 fixed some capitalization issues that slid in through the case-insenstive filesystems 2017-10-17 09:07:05 -07:00
Alex Miller 7b9bc1d715 Merge pull request #170 from cie/alexmiller/flowprofile
Add support for profiling a running fdb cluster to fdbcli, fix security issues, and add an improved backtrace.
2017-10-16 16:51:53 -07:00
Alex Miller cf646d4a99 Address review comments.
* Fixed fdbcli to be more idiomatic.
* Removed is_binary_serializable in favor of std::is_pod<>
* Removed custom enable_if<> in favor of std::enable_if<>
* Removed HEY REVIEWER comments
* Removed print from prof.py
* Added FLOW_PROFILER_ENABLED=yes to circus components that wished to enable the flow profiler.
2017-10-16 16:46:52 -07:00
Yichi Chiang a6ae89af1a Merge pull request #176 from cie/add-cluster-controller-process-class
Add cluster controller process class
2017-10-16 16:27:54 -07:00
Yichi Chiang af2aa41136 Downgrade Transaction process class for cluster controller 2017-10-16 16:27:01 -07:00
Yichi Chiang 76c5488421 Add cluster controller process class 2017-10-16 16:21:25 -07:00
Alex Miller 16e5b50685 Replace backtrace with absl::GetStackTrace on non-MacOS platforms.
backtrace() gives a list of return addresses, which means that addr2line will
print out the line after the caller. GetStackTrace returns the list of caller
addresses, so the addr2line results should be accurate.  The flow profiler was
also changed to use the new backtracing code, so flow profiles will now be
accurate as well.  Unfortunately, the abseil code doesn't work on MacOS, so we
still fall back to backtrace() in this case.

For the stack unwinder to work, we must disable -fomit-frame-pointer.  This can
result in a small performance penalty, as it effectively reduces the number of
general purpose registers available by one.  (I'm also curious if this has
anything to do with the overly frequent "<value optimized out>" messages from
gdb.)  If this shows up as a problem, we can make release builds still have
-fomit-frame-pointer, and fall back to backtrace when it's enabled then as
well.
2017-10-16 16:05:02 -07:00
Alex Miller 1a91aab1d7 Import //base/debugging:stacktrace from abseil.
This code is all Apache 2 licensed, and all headers were maintained when
concatinated, so we should be completely fine from a legal standpoint.

I've scriptified the steps that I took so that if we need to update this code
in the future, it hopefully shouldn't be too much of a hassle.
2017-10-16 16:05:02 -07:00
Alex Miller f997cb9038 Add a string knob to hold the Log directory, and write profiles to it.
This is the combination of two small changes.

1. Add support for a string knob type.
2. Change profiles to be written to the log directory instead of the working
   directory.

We have three options of where to write files: the working directory, the data
directory, and the log directory.

The working directory may be set to a non-writable location, and likely
contains the fdb binaries.  Allowing these files to be overwritten would likely
not be a wise idea.

The data directory hosts our sqlite b-trees.  It would also be very unfortunate
if these were ever overwritten by an unfortunate profile name.

The log directory contains logs.  Out of the three, these matter the least if
they disappear or become corrupted.

Thus, we write to the log directory.
2017-10-16 16:05:02 -07:00
Alex Miller c5fbe33df6 Disallow arbitrary paths for storing profiles.
Previously, one could request profiles to be stored at
"../../../../../../etc/passwd".  Now we expand the paths, including symlinks,
and ensure that the target is a child of the targetted subdirectory.  This was
the least convoluted way I could figure out to handle paths.
2017-10-16 16:05:02 -07:00
Alex Miller 91a26a170c Add toggleable profiling support to fdbserver+fdbcli.
This adds the fdbcli commands:
* profile list -- Lists all workers in a way that doesn't fill `kill`'s list.
* profile flow run -- Allows starting flow profiling on a set of hosts for a specified interval.

And threads through all the support for enabling and disabling profiling as an RPC.
2017-10-16 16:05:02 -07:00
Alvin Moore 770d9663d8 Prepared for new release 5.0.6
Updated WIX Product Id
Updated Artifactory links
2017-10-16 11:25:14 -07:00
Balachandar Namasivayam 312f614133 Add the new ops and AND to NON_ASSOCIATIVE_MASK.
In the storage server, read the entire value if the op is ByteMin or ByteMax.
2017-10-16 11:06:31 -07:00
Alec Grieser e0be1ef1e0 Merge branch 'release-5.0' 2017-10-16 10:08:11 -07:00
Alec Grieser 432726ba2d Merge branch 'release-4.6' into release-5.0 2017-10-16 09:54:21 -07:00