Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
/*
|
|
|
|
* MutationTracking.cpp
|
|
|
|
*
|
|
|
|
* This source file is part of the FoundationDB open source project
|
|
|
|
*
|
|
|
|
* Copyright 2013-2020 Apple Inc. and the FoundationDB project authors
|
|
|
|
*
|
|
|
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
|
|
* you may not use this file except in compliance with the License.
|
|
|
|
* You may obtain a copy of the License at
|
|
|
|
*
|
|
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
*
|
|
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
* See the License for the specific language governing permissions and
|
|
|
|
* limitations under the License.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <vector>
|
|
|
|
#include "fdbserver/MutationTracking.h"
|
|
|
|
#include "fdbserver/LogProtocolMessage.h"
|
2020-08-28 06:11:16 +08:00
|
|
|
#include "fdbserver/SpanContextMessage.h"
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
|
|
|
|
#if defined(FDB_CLEAN_BUILD) && MUTATION_TRACKING_ENABLED
|
|
|
|
#error "You cannot use mutation tracking in a clean/release build."
|
|
|
|
#endif
|
|
|
|
|
2020-05-14 09:48:43 +08:00
|
|
|
// Track up to 2 keys in simulation via enabling MUTATION_TRACKING_ENABLED and setting the keys here.
|
2020-05-14 09:44:22 +08:00
|
|
|
StringRef debugKey = LiteralStringRef( "" );
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
StringRef debugKey2 = LiteralStringRef( "\xff\xff\xff\xff" );
|
|
|
|
|
|
|
|
TraceEvent debugMutationEnabled( const char* context, Version version, MutationRef const& mutation ) {
|
|
|
|
if ((mutation.type == mutation.ClearRange || mutation.type == mutation.DebugKeyRange) &&
|
|
|
|
((mutation.param1<=debugKey && mutation.param2>debugKey) || (mutation.param1<=debugKey2 && mutation.param2>debugKey2))) {
|
|
|
|
return std::move(TraceEvent("MutationTracking").detail("At", context).detail("Version", version).detail("MutationType", typeString[mutation.type]).detail("KeyBegin", mutation.param1).detail("KeyEnd", mutation.param2));
|
|
|
|
} else if (mutation.param1 == debugKey || mutation.param1 == debugKey2) {
|
|
|
|
return std::move(TraceEvent("MutationTracking").detail("At", context).detail("Version", version).detail("MutationType", typeString[mutation.type]).detail("Key", mutation.param1).detail("Value", mutation.param2));
|
|
|
|
} else {
|
2020-07-22 03:49:52 +08:00
|
|
|
return TraceEvent();
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
TraceEvent debugKeyRangeEnabled( const char* context, Version version, KeyRangeRef const& keys ) {
|
|
|
|
if (keys.contains(debugKey) || keys.contains(debugKey2)) {
|
2020-07-22 03:49:52 +08:00
|
|
|
return debugMutation(context, version, MutationRef(MutationRef::DebugKeyRange, keys.begin, keys.end));
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
} else {
|
2020-07-22 03:49:52 +08:00
|
|
|
return TraceEvent();
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-05-14 05:28:04 +08:00
|
|
|
TraceEvent debugTagsAndMessageEnabled( const char* context, Version version, StringRef commitBlob ) {
|
2020-03-27 18:31:04 +08:00
|
|
|
BinaryReader rdr(commitBlob, AssumeVersion(currentProtocolVersion));
|
|
|
|
while (!rdr.empty()) {
|
|
|
|
if (*(int32_t*)rdr.peekBytes(4) == VERSION_HEADER) {
|
|
|
|
int32_t dummy;
|
|
|
|
rdr >> dummy >> version;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
TagsAndMessage msg;
|
|
|
|
msg.loadFromArena(&rdr, nullptr);
|
|
|
|
bool logAdapterMessage = std::any_of(
|
|
|
|
msg.tags.begin(), msg.tags.end(), [](const Tag& t) { return t == txsTag || t.locality == tagLocalityTxs; });
|
|
|
|
StringRef mutationData = msg.getMessageWithoutTags();
|
|
|
|
uint8_t mutationType = *mutationData.begin();
|
|
|
|
if (logAdapterMessage) {
|
|
|
|
// Skip the message, as there will always be an idential non-logAdapterMessage mutation
|
|
|
|
// that we can match against in the same commit.
|
|
|
|
} else if (LogProtocolMessage::startsLogProtocolMessage(mutationType)) {
|
|
|
|
BinaryReader br(mutationData, AssumeVersion(rdr.protocolVersion()));
|
|
|
|
LogProtocolMessage lpm;
|
|
|
|
br >> lpm;
|
|
|
|
rdr.setProtocolVersion(br.protocolVersion());
|
2020-08-28 06:11:16 +08:00
|
|
|
} else if (SpanContextMessage::startsSpanContextMessage(mutationType)) {
|
|
|
|
BinaryReader br(mutationData, AssumeVersion(rdr.protocolVersion()));
|
|
|
|
SpanContextMessage scm;
|
|
|
|
br >> scm;
|
2020-03-27 18:31:04 +08:00
|
|
|
} else {
|
|
|
|
MutationRef m;
|
|
|
|
BinaryReader br(mutationData, AssumeVersion(rdr.protocolVersion()));
|
|
|
|
br >> m;
|
|
|
|
TraceEvent&& event = debugMutation(context, version, m);
|
|
|
|
if (event.isEnabled()) {
|
|
|
|
return std::move(event.detail("MessageTags", msg.tags));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2020-07-22 03:49:52 +08:00
|
|
|
return TraceEvent();
|
2020-03-27 18:31:04 +08:00
|
|
|
}
|
|
|
|
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
#if MUTATION_TRACKING_ENABLED
|
|
|
|
TraceEvent debugMutation( const char* context, Version version, MutationRef const& mutation ) {
|
|
|
|
return debugMutationEnabled( context, version, mutation );
|
|
|
|
}
|
|
|
|
TraceEvent debugKeyRange( const char* context, Version version, KeyRangeRef const& keys ) {
|
|
|
|
return debugKeyRangeEnabled( context, version, keys );
|
|
|
|
}
|
2020-05-14 05:28:04 +08:00
|
|
|
TraceEvent debugTagsAndMessage( const char* context, Version version, StringRef commitBlob ) {
|
|
|
|
return debugTagsAndMessageEnabled( context, version, commitBlob );
|
2020-03-27 19:04:13 +08:00
|
|
|
}
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
#else
|
2020-07-22 03:49:52 +08:00
|
|
|
TraceEvent debugMutation(const char* context, Version version, MutationRef const& mutation) {
|
|
|
|
return TraceEvent();
|
|
|
|
}
|
|
|
|
TraceEvent debugKeyRange(const char* context, Version version, KeyRangeRef const& keys) {
|
|
|
|
return TraceEvent();
|
|
|
|
}
|
|
|
|
TraceEvent debugTagsAndMessage(const char* context, Version version, StringRef commitBlob) {
|
|
|
|
return TraceEvent();
|
|
|
|
}
|
Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 16:35:26 +08:00
|
|
|
#endif
|