Commit Graph

1239 Commits

Author SHA1 Message Date
mpilman 7e26b4ef0d Address comments from PR 2019-02-07 15:37:04 -08:00
mpilman 5737349676 Fix weird bug with boost interprocess
Strangely, boost interprocess didn't compile with VS 2017.
However, it does compile if it is included as the first thing.
I don't quite know what is happening here, but for now this fix
makes it that I am not blocked
2019-02-07 15:37:04 -08:00
mpilman 8a94d80deb fdbservice and fdbrpc now compiling 2019-02-07 15:37:04 -08:00
Evan Tschannen 7e0e0a7673
Merge pull request #1105 from vishesh/task/issue-218-compare-and-clear
Implements CompareAndClear AtomicOp
2019-02-05 18:11:28 -08:00
Evan Tschannen 486e0e13c3
Merge pull request #1116 from alexmiller-apple/tstlog
Random cleanups that prepare for Spill-By-Reference TLog
2019-02-05 18:09:06 -08:00
A.J. Beamon 882f8d70b7
Merge pull request #1066 from etschannen/master
fix: coordinators auto could put two coordinators in the same zone
2019-02-05 11:52:04 -08:00
Alex Miller 22a08b2b4e Change mutable ref to pointer so outparams are obvious. 2019-02-04 18:04:22 -08:00
Alex Miller 0efcccc06f Fix a long standing minor bug in disk queue that could lead to unnecessary commits.
If the disk queue is called with the following series of operations:

Push(a) -> 1
Commit()
Pop(1)
Push(b)
Commit()
Commit()

Then the last Commit() should be a no-op, and not actually run accordingly.

However, anyPopped was only set to `false` if no pages were pushed, and thus
we'd falsely think that an extra empty page commit needed to happen to log to
record the new popped position, but there actually was no new popped page
position to record.

Aside from the extra commit, it maybe makes getCommitOverhead slightly
inaccurate, but that's only used for some accounting inside of the memory
storage engine and at a quick glance doesn't look like it should have caused
any bad effects.

I dug through history, and this code has been this way since the initial commit
by Dave, and then no one has touched the anyPopped logic since.
2019-02-04 18:04:22 -08:00
Vishesh Yadav 5985566a8e Don't issue a second read in storageserver if possible for CompareAndClear
If the previous eager read request is equal to the CompareAndClear op
key, do not issue a read again.
2019-02-04 16:10:59 -08:00
Vishesh Yadav c532d5c277 Implements CompareAndClear AtomicOp
Adds CompareAndClear mutation. If the given parameter is equal to the
current value of the key, the key is cleared. At client, the mutation
is added to the operation stack. Hence if the mutation evaluates to
clear, we only get to know so when `read()` evaluates the stack in
`RYWIterator::kv()`, which is unlike what we currently do for typical
ClearRange.
2019-02-04 14:59:56 -08:00
Evan Tschannen a678f778fa
Increase severity to SevWarnAlways for TooManyStatusRequests trace
Co-Authored-By: tclinken <trevorclinkenbeard@gmail.com>
2019-01-28 17:50:50 -08:00
Trevor Clinkenbeard 5b89db811a Throttle status requests with MAX_STATUS_REQUESTS_PER_SECOND knob, whenever status batching is used. 2019-01-28 15:37:30 -08:00
Alex Miller ec32d3308b
Merge pull request #1086 from mpilman/features/c++-compiler-errors
Fixed several minor code issues
2019-01-24 15:24:33 -08:00
mpilman 79637f07ac Fixed several minor code issues
These will become a problem as soon as we
switch to C++17
2019-01-24 14:43:12 -08:00
mpilman 58964af7e1 ctest improvements - #1058
- A set of CMake variables controls whether to keep
  the simfdb directory and the traces and whether we
  want to aggregate the traces into a single file
- Test labels now contain the directory they are in
  so that one can now run `ctest -R fast/`
- A different binary can be used for restart tests. CMake
  will automatically look for an installed fdb and use that
  by default. If none is found, it will use the built one
  but it will also print a warning
- CMake will throw an error if there are any text files in
  the tests directory that are not associated with a test.
- Moved testing from fdbserver/CMakeLists.txt to
  tests/CMakeLists.txt
- Moved fdb testing functions to its own cmake module
2019-01-22 14:34:51 -08:00
Evan Tschannen 699f8dd617 fix: coordinators auto could put two coordinators in the same zone
simulation now tests two machines in the same zone
2019-01-18 15:42:48 -08:00
Alex Miller b4a446756a Remove more top-level tests that were out of order. 2019-01-14 20:28:40 -08:00
Alex Miller 0d579b4730 Top level tests aren't run by default, and some fail. 2019-01-14 19:14:25 -08:00
Alex Miller b3e977d7c1 Apply test directory as a label.
This isn't ideal, as it makes `restarting/from_5.2.0/potato.txt` have
the label "from_5.2.0" instead of "restarting", but it does make the
fast label work right.
2019-01-14 19:14:25 -08:00
mpilman 414fb0b6c8 made TestRunner work with XML traces 2019-01-14 19:14:25 -08:00
Markus Pilman 14f0a6958b added all tests to ctest 2019-01-14 19:14:25 -08:00
Markus Pilman b096b8e3f8 First tests working with ctest 2019-01-14 19:14:25 -08:00
Evan Tschannen 7dbf06162e
Update fdbserver/ClusterController.actor.cpp
Co-Authored-By: bnamasivayam <36455962+bnamasivayam@users.noreply.github.com>
2019-01-14 16:57:00 -08:00
Balachandar Namasivayam ff661bca22 Fix a minor bug in the RoleFitness Class. 2019-01-14 14:54:54 -08:00
Evan Tschannen 9912d17c35
Merge pull request #1030 from bnamasivayam/master
Add some sanity checks to tester.actor.cpp
2019-01-11 17:15:40 -08:00
Evan Tschannen 4eb11d74af
Merge pull request #1029 from bnamasivayam/reenable-check_desired_classes
Re-enable CheckDesiredClasses after making necessary changes for mult…
2019-01-11 17:15:05 -08:00
A.J. Beamon d4d5740282 * Add Optional.map and ErrorOr.map.
* Rename Optional/ErrorOr cast_to to castTo.
* Make printable(Optional<T>) templated rather than restricted to StringRef types.
* Fixes bug in (unused) ErrorOr.castTo where an ErrorOr that was not set would lose its error.
2019-01-11 09:03:38 -08:00
Balachandar Namasivayam baeaa490e4 Add some sanity checks to tester.actor.cpp 2019-01-10 11:05:50 -08:00
Balachandar Namasivayam a8e2e75cd5 Re-enable CheckDesiredClasses after making necessary changes for multi-region setup.
Fixed a couple of bugs
1) A rare race condition where a worker is being roles even after it died.
2) Fix how RoleFitness is calculated for TLog and LogRouter. Only worst fitness is compared to see if a better fit is available.
2019-01-10 10:28:32 -08:00
Evan Tschannen 684a22a52b Merge branch 'release-6.0'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/BackupContainer.actor.cpp
#	fdbclient/HTTP.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/BackupCorrectness.actor.cpp
#	versions.target
2019-01-09 16:14:46 -08:00
A.J. Beamon 7c5b2ab330
Merge pull request #976 from alexmiller-apple/jsonlogs
Allow trace logs to be output as JSON instead of XML
2019-01-09 17:04:50 -05:00
Evan Tschannen 5d2b11cba9
Merge pull request #1019 from satherton/http-request-id
Backup usability and sanity check improvements
2019-01-09 13:29:37 -08:00
Stephen Atherton 604ad062d5 Updated backup correctness test to new behavior. WaitBackup() can now return the UID and BackupContainer atomically with the status code for a backup tag. 2019-01-08 18:12:15 -08:00
A.J. Beamon d265517156 Fix: fast allocator would not cleanup memory for a thread if that thread never called getMagazine. This could happen if the first thing the thread did was to release memory.
Added a new metric for the number of threads that hold memory for each size and improve some existing metrics.
Fix: a failed ASSERT would crash if done early in the program lifetime.
2019-01-08 14:36:01 -08:00
Evan Tschannen 57293a2db0 byte sample recovery did not use limits for its range reads, leading to slow tasks 2019-01-04 10:32:31 -08:00
Markus Pilman dbe9baff1f Several small compilation fixes for new versions of gcc
There are several missing includes for cmath in the code, I added those.

Next, Coro returns a reference to a stack variable and this causes a
warning. As this is probably ok for Coro, I disabled the warning in
that file for GCC. I want to have this warning in the build system as
it is generally a very useful warning to have.

Another change is that major and minor are deprecated for a while now.
I replaced those with gnu_dev_major and gnu_dev_minor.

ErrorOr currently implements operators ==, !=, and <. These do not
compile because Error does not implement ==. This compiles on older
versions of gcc and clang because ErrorOr<T>::operator== is not used
anywhere. It is still wrong though and newer gcc versions complain.
I simply removed these methods.

The most interesting fix is that TraceEvent::~TraceEvent is currently
throwing exceptions. This is illegal behavior in C++11 and a idea in
older versions of C++. For now I simply removed the throw, but this
might need some more thought.
2019-01-03 12:44:19 -08:00
Evan Tschannen 4901e37b8f
Merge pull request #983 from alexmiller-apple/compilationfixes
Various minor fixes
2019-01-03 10:01:05 -08:00
Andrew Noyes 7eb6765698 Mention that xml is the default 2019-01-03 08:48:31 -08:00
Bhaskar Muppana aa2a76ef4c
Merge pull request #981 from alexmiller-apple/cmake
Add a CMake build system
2019-01-02 18:50:15 -08:00
Andrew Noyes bce5b03340 Fix whitespace 2019-01-02 15:24:11 -08:00
Alex Miller 73f53f9861
Merge pull request #991 from atn34/replace-ampersand-operator
Replace & operator with variadic function
2018-12-30 22:51:59 -06:00
Simon Zhou 7edf221986 Avoid null check 2018-12-28 13:09:04 -08:00
anoyes 6a4d87802b Replace & operator with variadic function 2018-12-28 11:33:42 -08:00
anoyes 1bca665b29 Document --trace_format flag 2018-12-20 16:22:41 -08:00
anoyes b8df5acc15 Add --trace_format flag to fdbserver 2018-12-20 15:02:01 -08:00
Alex Miller bfab7c150a Require PageHeader to be 36 bytes, and don't use magic numbers. 2018-12-17 13:37:44 -08:00
Alex Miller b4b7f382a7 Fix issues that a newer compiler warned about. 2018-12-14 14:43:50 -08:00
Meng Xu 486a7b04fa TeamCollection: Fix build in osX
In osX, we cannot adding unsigned long to a string to append to the string.
2018-12-14 13:44:11 -08:00
Alex Miller 550daa05f8 Default configuration now builds. 2018-12-13 15:52:27 -08:00
Markus Pilman df0f491c29 Some more improvements to the build and preparations for packaging 2018-12-13 15:04:13 -08:00