Commit Graph

596 Commits

Author SHA1 Message Date
mpilman b06fe39b47 renamed flatbuffers tests for consistency 2019-05-13 14:15:22 -07:00
mpilman 1df3d8f0c7 disable rlimit when ASAN is used 2019-05-13 14:15:22 -07:00
mpilman 6afce01744 Implementation complete (not yet working) 2019-05-13 14:15:22 -07:00
mpilman 1d5572a078 removed unused knob 2019-05-13 14:15:22 -07:00
mpilman 9eeb48c43d Allow to turn on object serializer
This commit includes functionality to turn on
the object serializer for network communication.
This is done the following way:

- On incoming connections, a process will detect
  whether the client supports the object serializer
  and will only serialize responses with it, if it does
- On outgoing connections, the command line flag is used
  to determine whether the object serializer should be used
  to send data.

This way, a cluster can run in mixed mode. To upgrade one
can upgrade one process at a time and set the flag one process
at a time.

This is how this is tested on the simulator:
- The command line flag can take three options: on, off,
  and random.
- For off, the object serializer will never we used.
- For on, the object serializer will be always used.
- For random, the simulator will flip a coin for each
  process it starts up.
2019-05-13 14:15:22 -07:00
mpilman ba83c458a6 types implemented 2019-05-13 14:15:22 -07:00
mpilman fe81454ec2 basic functionality for object serializer
This commit includes:
- The flatbuffers implementation
- A draft on how it should be used for network messages
- A serializer that can be used independently

What is missing:
- All root objects will need a file identifier
- Many special classes can not be serialized yet as the
  corresponding traits are not yet implemented
- Object serialization can not yet be turned on (this will
  need a network option)
2019-05-13 14:15:22 -07:00
mpilman 8bbd28315e Added Object serializer (does not yet compile) 2019-05-13 14:15:22 -07:00
A.J. Beamon 8e1927a5cb Rename trace event field to avoid duplicating 'Time' field. 2019-05-13 13:25:01 -07:00
A.J. Beamon ab4618c3f9 Don't access TraceEvent::eventCounts outside the network thread. Make net2liveness an atomic counter. 2019-05-10 12:35:11 -10:00
Evan Tschannen 22499666d0 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/LogRouter.actor.cpp
#	flow/Trace.cpp
#	versions.target
2019-05-08 18:19:35 -07:00
Austin Seipp a67d47844b coveragetool: suppress output unless VERBOSE is set
When using the CMake build system with tools such as Ninja (instead of
Make), they have very concise output, and the general "UX flow" involves
only outputting warnings that should be fixed or errors that should stop
the build. Furthermore, when building things in CI systems and looking
at build logs, etc, it's often vastly easier to to hunt down errors when
only relevant stuff is output.

With this change, the output from coveragetool.exe is suppressed, which
accounts for a lot of spurious output that isn't necessary when running
these builds.

Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-07 16:54:05 -10:00
Evan Tschannen aefd68e1e7 The memory tracking trace events can crash if the memory that is being allocated is coming from a trace event itself. Specifically TDMetrics within trace events uses fast allocated memory. Trace events are supposed to be short lived, so this commits adds a global count of outstanding trace events, and disables memory tracking when a trace event exists. 2019-05-06 17:41:32 -07:00
Austin Seipp 7a10199a35 flow: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Jingyu Zhou 41ae9cd0d8 Remove ; for namespace declarations 2019-05-06 13:21:28 -07:00
Jingyu Zhou e193cac5ef Merge remote-tracking branch 'apple/master' into tlog
Resolve Conflicts: fdbserver/MasterProxyServer.actor.cpp
2019-05-01 17:18:00 -07:00
Evan Tschannen 2d5043c665 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-04-30 18:27:04 -07:00
Evan Tschannen 4fa1c008f9 Highly prioritize storageServerRejoin messages on the proxy, so that storage servers can rejoin the cluster even when a proxy is CPU saturated 2019-04-23 20:56:01 -07:00
A.J. Beamon e0f76edf77
Merge pull request #1471 from AlvinMooreSr/release-6.1-merge
Merge Release 6.1 Into Master
2019-04-23 11:08:21 -07:00
Alex Miller c25fac9421 Lower the priority of spilled peeks to below that of spilling.
This prevents peeking from starving the TLog of CPU time to spill, and
thus impacting write throughput at prolonged saturation.
2019-04-22 18:39:21 -07:00
Jingyu Zhou 8b67da57bb Fix upgrade test failure
Serialize pseudoLocalities if protocol version is larger than 0x0FDB00B061060001LL.
Note this version may need to be changed to "currentProtocolVersion" when merging
into the master, and "currentProtocolVersion" should be incremented.
2019-04-21 10:41:07 -07:00
Jingyu Zhou e7ee61660c Fix unit test failure due to trace event names
Capitalize the first letter.
2019-04-20 10:19:53 -07:00
Alvin Moore 2bea99591e Merge branch 'release-6.1' of copy of master
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2019-04-17 15:51:48 -07:00
Andrew Noyes 6888baf1c3 Fix compile error 2019-04-16 15:44:55 -07:00
Andrew Noyes 08e66d1764 Prefer success to cast-to-void 2019-04-16 15:29:03 -07:00
Andrew Noyes 781b6ece77 Fix OPEN_FOR_IDE -Wunused-variable warnings
CC #1255, #1173
2019-04-16 15:28:01 -07:00
Andrew Noyes baa3e806ef Address review comments from #1446 2019-04-16 09:48:15 -07:00
Andrew Noyes 6207d724f8 Fix all -Wunused-variable warnings 2019-04-15 18:13:00 -07:00
Andrew Noyes 6c7fd1593f Avoid potential index-out-of-bounds 2019-04-15 15:20:09 -07:00
Andrew Noyes dced9232d0 ASSERT_WE_THINK trailing byte is '\0' 2019-04-15 09:43:26 -07:00
Andrew Noyes 369c1a0230 Exclude trailing \0 byte for string literals
This will "do the right thing" as long as no one expects something like

char a[] = {'a'};
TraceEvent("Event").detail("a", a);

to trace `{..., "a": "a"}`.

It will trace `{..., "a": ""}`
2019-04-11 10:39:48 -07:00
Andrew Noyes 1135331ce6 Include space as a printable character 2019-04-11 10:01:06 -07:00
Evan Tschannen cd5c9d91fa
Merge pull request #1443 from etschannen/master
Merge 6.1 into master
2019-04-10 17:43:07 -07:00
Evan Tschannen 8e05713a5d do not log a SevError trace event if we cannot deserialize the connect packet 2019-04-10 17:41:02 -07:00
Trevor Clinkenbeard a9b1b32fde Handle negative input to base16Char
base16Char was being called with (c < 0), causing a failed assertion.
2019-04-09 18:14:09 -07:00
Evan Tschannen 6220a5ce0f
Merge pull request #1370 from jzhou77/fix-unreferenced
Remove unused functions
2019-04-09 11:49:45 -07:00
A.J. Beamon 058d028099
Merge pull request #1301 from mpilman/features/cheaper-traces
Defer formatting in traces to make them cheaper
2019-04-09 10:11:04 -07:00
Jingyu Zhou af17babb9a Restore a deleted comment line 2019-04-09 09:38:45 -07:00
mpilman e4746888eb Removed dead code 2019-04-09 09:08:45 -07:00
Evan Tschannen 21c0ba555c Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-04-08 18:38:42 -07:00
Evan Tschannen 05869a8383 do not log a degraded reset message if the previous reset was more than a week ago 2019-04-07 23:00:58 -07:00
Evan Tschannen 19d9b8fcd8 don’t log MachineLoadDetails from getSystemStatistics() 2019-04-07 22:55:19 -07:00
Andrew Noyes d7612a4426 Fix OPEN_FOR_IDE build errors 2019-04-05 16:30:42 -07:00
mpilman d01cbf3455 Addressed code review comments 2019-04-05 13:12:20 -07:00
mpilman 4287b1d2a1 resolved minor merge issues 2019-04-05 13:12:19 -07:00
mpilman b0877e7dd0 Don't quote spaces in std::strings 2019-04-05 13:12:19 -07:00
mpilman 1c6c73af6d Fixed infinite recursion 2019-04-05 13:12:19 -07:00
mpilman c6bc23791d Fixed format error in microbench 2019-04-05 13:12:19 -07:00
mpilman ea67b742c7 Implemented Traceable for printable types 2019-04-05 13:12:19 -07:00
mpilman c9ee7f54e7 Add compound type to micro-benchmark 2019-04-05 13:12:19 -07:00