wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
foundationdb/fdbclient/ClientWorkerInterface.h

96 lines
2.8 KiB
C
Raw Normal View History

Initial repository commit
2017-05-26 04:48:44 +08:00
/*
* ClientWorkerInterface.h
*
* This source file is part of the FoundationDB open source project
*
* Copyright 2013-2018 Apple Inc. and the FoundationDB project authors
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-22 02:25:11 +08:00
*
Initial repository commit
2017-05-26 04:48:44 +08:00
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-22 02:25:11 +08:00
*
Initial repository commit
2017-05-26 04:48:44 +08:00
* http://www.apache.org/licenses/LICENSE-2.0
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-22 02:25:11 +08:00
*
Initial repository commit
2017-05-26 04:48:44 +08:00
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef FDBCLIENT_CLIENTWORKERINTERFACE_H
#define FDBCLIENT_CLIENTWORKERINTERFACE_H
#pragma once
Adjust all includes to be relative to the root. Remove the use of relative paths. A header at foo/bar.h could be included by files under foo/ with "bar.h", but would be included everywhere else as "foo/bar.h". Adjust so that every include references such a header with the latter form. Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-20 01:30:13 +08:00
#include "fdbclient/FDBTypes.h"
Initial repository commit
2017-05-26 04:48:44 +08:00
#include "fdbrpc/FailureMonitor.h"
Adjust all includes to be relative to the root. Remove the use of relative paths. A header at foo/bar.h could be included by files under foo/ with "bar.h", but would be included everywhere else as "foo/bar.h". Adjust so that every include references such a header with the latter form. Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-20 01:30:13 +08:00
#include "fdbclient/Status.h"
#include "fdbclient/ClientDBInfo.h"
Initial repository commit
2017-05-26 04:48:44 +08:00
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
// Streams from WorkerInterface that are safe and useful to call from a client.
// A ClientWorkerInterface is embedded as the first element of a WorkerInterface.
Initial repository commit
2017-05-26 04:48:44 +08:00
struct ClientWorkerInterface {
Implementation complete (not yet working)
2019-01-31 05:53:23 +08:00
constexpr static FileIdentifier file_identifier = 12418152;
Initial repository commit
2017-05-26 04:48:44 +08:00
RequestStream< struct RebootRequest > reboot;
Add toggleable profiling support to fdbserver+fdbcli. This adds the fdbcli commands: * profile list -- Lists all workers in a way that doesn't fill `kill`'s list. * profile flow run -- Allows starting flow profiling on a set of hosts for a specified interval. And threads through all the support for enabling and disabling profiling as an RPC.
2017-10-12 05:13:16 +08:00
RequestStream< struct ProfilerRequest > profiler;
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
Initial repository commit
2017-05-26 04:48:44 +08:00
bool operator == (ClientWorkerInterface const& r) const { return id() == r.id(); }
bool operator != (ClientWorkerInterface const& r) const { return id() != r.id(); }
UID id() const { return reboot.getEndpoint().token; }
Listen to multiple addresses and start using vector<NetworkAdddress> in Endpoint - This patch will make FDB listen to multiple addresses given via command line. Although, we'll still use first address in most places, this patch starts using vector<NetworkAddress> in Endpoint at some basic places. - When sending packets to an endpoint, pick a random network address in endpoints - Renames Endpoint::address to Endpoint::addresses since it now holds a vector of addresses.
2018-10-31 04:44:37 +08:00
NetworkAddress address() const { return reboot.getEndpoint().getPrimaryAddress(); }
Initial repository commit
2017-05-26 04:48:44 +08:00
template <class Ar>
void serialize( Ar& ar ) {
Replace & operator with variadic function
2018-12-29 02:49:26 +08:00
serializer(ar, reboot, profiler);
Initial repository commit
2017-05-26 04:48:44 +08:00
}
};
struct RebootRequest {
Implementation complete (not yet working)
2019-01-31 05:53:23 +08:00
constexpr static FileIdentifier file_identifier = 11913957;
Initial repository commit
2017-05-26 04:48:44 +08:00
bool deleteData;
bool checkData;
Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures.
2019-06-15 02:35:38 +08:00
uint32_t waitForDuration;
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures.
2019-06-15 02:35:38 +08:00
explicit RebootRequest(bool deleteData = false, bool checkData = false, uint32_t waitForDuration = 0)
: deleteData(deleteData), checkData(checkData), waitForDuration(waitForDuration) {}
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
Initial repository commit
2017-05-26 04:48:44 +08:00
template <class Ar>
void serialize(Ar& ar) {
Extend RebootRequest API to include time to suspend the process before reboot. This is intended to be used for testing purposes to simulate failures.
2019-06-15 02:35:38 +08:00
serializer(ar, deleteData, checkData, waitForDuration);
Initial repository commit
2017-05-26 04:48:44 +08:00
}
};
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
struct ProfilerRequest {
Implementation complete (not yet working)
2019-01-31 05:53:23 +08:00
constexpr static FileIdentifier file_identifier = 15437862;
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
ReplyPromise<Void> reply;
Add toggleable profiling support to fdbserver+fdbcli. This adds the fdbcli commands: * profile list -- Lists all workers in a way that doesn't fill `kill`'s list. * profile flow run -- Allows starting flow profiling on a set of hosts for a specified interval. And threads through all the support for enabling and disabling profiling as an RPC.
2017-10-12 05:13:16 +08:00
enum class Type : std::int8_t {
GPROF = 1,
Add manually triggered heap profiling At client side: fdb> profile ERROR: Usage: profile <client|list|flow|heap> fdb> profile heap 127.0.0.1:4500 On the server side: $ HEAPPROFILE=/tmp/fdbserver bin/fdbserver -C ../test.cluster -p 127.0.0.1:4500 Starting tracking the heap FDBD joined cluster. Dumping heap profile to /tmp/fdbserver.0001.heap (1024 MB allocated cumulatively, 13 MB currently in use) Dumping heap profile to /tmp/fdbserver.0002.heap (User triggered heap dump)
2019-04-04 06:57:16 +08:00
FLOW = 2,
GPROF_HEAP = 3,
Add toggleable profiling support to fdbserver+fdbcli. This adds the fdbcli commands: * profile list -- Lists all workers in a way that doesn't fill `kill`'s list. * profile flow run -- Allows starting flow profiling on a set of hosts for a specified interval. And threads through all the support for enabling and disabling profiling as an RPC.
2017-10-12 05:13:16 +08:00
};
enum class Action : std::int8_t {
DISABLE = 0,
ENABLE = 1,
RUN = 2
};
Type type;
Action action;
int duration;
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
Standalone<StringRef> outputFile;
Update fdbclient/ClientWorkerInterface.h Co-Authored-By: jzhou77 <jingyuzhou@gmail.com>
2019-04-04 11:05:19 +08:00
ProfilerRequest() = default;
Add manually triggered heap profiling At client side: fdb> profile ERROR: Usage: profile <client|list|flow|heap> fdb> profile heap 127.0.0.1:4500 On the server side: $ HEAPPROFILE=/tmp/fdbserver bin/fdbserver -C ../test.cluster -p 127.0.0.1:4500 Starting tracking the heap FDBD joined cluster. Dumping heap profile to /tmp/fdbserver.0001.heap (1024 MB allocated cumulatively, 13 MB currently in use) Dumping heap profile to /tmp/fdbserver.0002.heap (User triggered heap dump)
2019-04-04 06:57:16 +08:00
explicit ProfilerRequest(Type t, Action a, int d) : type(t), action(a), duration(d) {}
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
template<class Ar>
void serialize( Ar& ar ) {
Replace & operator with variadic function
2018-12-29 02:49:26 +08:00
serializer(ar, reply, type, action, duration, outputFile);
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
}
};
Fix a massive amount of valgrind errors and make them easier to debug in the future. std::is_pod<> being less restrictive than is_binary_serializable<> meant that structs that both were POD and had a serialize method defined would be binary serialized instead of using the defined serialize(). This means that it would also serialize any padding that the struct contained, which would cause mass waves of valgrind failures from uninitialized memory. Included in this change is additional uses of valgrind client requests so that attempts to send uninitialized memory are reported at the sending site, versus as part of checksum calculation in sending the packet.
2017-10-28 07:54:44 +08:00
BINARY_SERIALIZABLE( ProfilerRequest::Type );
BINARY_SERIALIZABLE( ProfilerRequest::Action );
Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface. A way to access this stream is required if we wish to be able to toggle profiling from fdbcli. There's two ways to do this: 1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use `getWorkers` from there to get a list of `WorkerInterface`s, from which we can access cpuProfilerRequest. 2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code in the client that can fetch a list of all `ClientWorkerInterface`s. The split between WorkerInterface and ClientWorkerInterface appears to be what a client might have a need to call versus what is fdbserver-internal (and thus no client should even want to call). Thus, it seems to make more sense to acknowledge that profiling is useful to be able to toggle from a client, and go with option (2).
2017-07-13 15:27:56 +08:00
#endif