Summary:
This is the third patch to improve module loading in a series that started here (where I explain the motivation and solution): D62499
Add functions to read the r_debug location to know where the linked list of loaded libraries are so I can generate the `xfer:libraries-svr4` packet.
I'm also using this function to implement `GetSharedLibraryInfoAddress` that was "not implemented" for linux.
Most of this code was inspired by the current ds2 implementation here: https://github.com/facebook/ds2/blob/master/Sources/Target/POSIX/ELFProcess.cpp.
Reviewers: clayborg, xiaobai, labath
Reviewed By: clayborg, labath
Subscribers: emaste, krytarowski, mgorny, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D62501
llvm-svn: 363458
Summary:
This is the first of a few patches I have to improve the performance of dynamic module loading on Android.
In this first diff I'll describe the context of my main motivation and will then link to it in the other diffs to avoid repeating myself.
## Motivation
I have a few scenarios where opening a specific feature on an Android app takes around 40s when lldb is attached to it. The reason for that is because 40 modules are dynamicly loaded at that point in time and each one of them is taking ~1s.
## The problem
To learn about new modules we have a breakpoint on a linker function that is called twice whenever a module is loaded. One time just before it's loaded (so lldb can check which modules are loaded) and another right after it's loaded (so lldb can check again which ones are loaded and calculate the diference).
It's figuring out which modules are loaded that is taking quite some time. This is currently done by traversing the linked list of loaded shared libraries that the linker maintains in memory. Each item in the linked list requires its own `x` packet sent to the gdb server (this is android so the network also plays a part). In my scenario there are 400+ loaded libraries and even though we read 0x800 worth of bytes at a time we still make ~180 requests that end up taking 150-200ms.
We also do this twice, once before the module is loaded (state = eAdd) and another right after (state = eConsistent) which easly adds up to ~400ms per module.
## A solution
**Implement `xfer:libraries-svr4` in lldb-server:**
I noticed in the code that loads the new modules that it had support for the `xfer:libraries-svr4` packet (added ~4 years ago to support the ds2 debug server) but we didn't support it in lldb-server. This single packet returns an xml list of all the loaded modules by the process. The advantage is that there's no more need to make 180 requests to read the linked list. Additionally this new requests takes around 10ms.
**More efficient usage of the `xfer:libraries-svr4` packet in lldb:**
When `xfer:libraries-svr4` is available the Process class has a `LoadModules` function that requests this packet and then loads or unloads modules based on the current list of loaded modules by the process.
This is the function that is used by the DYLDRendezvous class to get the list of loaded modules before and after the module is loaded. However, this is really not needed since the LoadModules function already loaded or unloaded the modules accordingly. I changed this strategy to call LoadModules only once (after the process has loaded the module).
**Bugs**
I found a few issues in lldb while implementing this and have submitted independent patches for them.
I tried to devide this into multiple logical patches to make it easier to review and discuss.
## Tests
I wanted to put these set of diffs up before having all the tests up and running to start having them reviewed from a techical point of view. I'm also having some trouble making the tests running on linux so I need more time to make that happen.
# This diff
The `xfer` packages follow the same protocol, they are requested with `xfer:<object>:<read|write>:<annex>:<offset,length>` and a return that starts with `l` or `m` depending if the offset and length covers the entire data or not. Before implementing the `xfer:libraries-svr4` I refactored the `xfer:auxv` to generically handle xfer packets so we can easly add new ones.
The overall structure of the function ends up being:
* Parse the packet into its components: object, offset etc.
* Depending on the object do its own logic to generate the data.
* Return the data based on its size, the requested offset and length.
Reviewers: clayborg, xiaobai, labath
Reviewed By: labath
Subscribers: mgorny, krytarowski, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D62499
llvm-svn: 362982
Summary:
According to [C128] "Virtual functions should specify exactly one
of `virtual`, `override`, or `final`", I've added override where a
virtual function is overriden but the explicit `override` keyword
was missing. Whenever both `virtual` and `override` were specified,
I removed `virtual`. As C.128 puts it:
> [...] writing more than one of these three is both redundant and
> a potential source of errors.
I anticipate a discussion about whether or not to add `override` to
destructors but I went for it because of an example in [ISOCPP1000].
Let me repeat the comment for you here:
Consider this code:
```
struct Base {
virtual ~Base(){}
};
struct SubClass : Base {
~SubClass() {
std::cout << "It works!\n";
}
};
int main() {
std::unique_ptr<Base> ptr = std::make_unique<SubClass>();
}
```
If for some odd reason somebody removes the `virtual` keyword from the
`Base` struct, the code will no longer print `It works!`. So adding
`override` to destructors actively protects us from accidentally
breaking our code at runtime.
[C128]: https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#c128-virtual-functions-should-specify-exactly-one-of-virtual-override-or-final
[ISOCPP1000]: https://github.com/isocpp/CppCoreGuidelines/issues/1000#issuecomment-476951555
Reviewers: teemperor, JDevlieghere, davide, shafik
Reviewed By: teemperor
Subscribers: kwk, arphaman, kadircet, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D61440
llvm-svn: 359868
The tests reading the untouched module list are now not using any lldb
code (as module list loading lives in llvm now), so they can be removed.
The "filtering" of the module list remains (and probably will remain) an
lldb concept, so I keep those tests, but replace the checked-in binaries
with their yaml equivalents.
The binaries which are no longer referenced by any tests have been
removed.
llvm-svn: 358850
In this patch, I just remove the structure definitions for the
ModuleList stream and the associated parsing code. The rest of the code
is converted to work with the definitions in llvm. NFC.
llvm-svn: 358070
I also update the tests for SystemInfo parsing to use the yaml2minidump
capabilities in llvm instead of relying on checked-in binaries.
llvm-svn: 357896
This patch removes the lower layers of the minidump parsing code from
the MinidumpParser class, and replaces it with the minidump parser in
llvm.
Not all functionality is already avaiable in the llvm class, but it is
enough for us to be able to stop enumerating streams manually, and rely
on the minidump directory parsing code from the llvm class.
This also removes some checked-in binaries which were used to test error
handling in the parser, as the error handling is now done (and tested)
in llvm. Instead I just add one test that ensures we correctly propagate
the errors reported by the llvm parser. The input for this test can be
written in yaml instead of a checked-in binary.
llvm-svn: 357748
remove the Initialize function, move the things that can fail into the
static factory function. The factory function now returns
Expected<Parser> instead of Optional<Parser> so that it can give a
reason why creation failed.
llvm-svn: 354668
The tests were doing two somewhat independent things:
- checking that the registers can be retrieved from the minidump file
- checking that they can be converted into a form suitable for
consumption by lldb
The first thing requires a minidump file (but it's independent of other
lldb structures), while the second one does not require a minidump file
(but it needs lldb register info structures).
Splitting this into two tests gives an opportunity to write more
detailed tests, and allows the two pieces of functionality to be moved
into different packages, if that proves to be necessary.
llvm-svn: 354662
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Each process plug-in can create its own custom commands. I figured it would be nice to be able to dump things from the minidump file from the lldb command line, so I added the start of the some custom commands.
Currently you can dump:
minidump stream directory
all linux specifc streams, most of which are strings
each linux stream individually if desired, or all with --linux
The idea is we can expand the command set to dump more things, search for data in the core file, and much more. This patch gets us started.
Differential Revision: https://reviews.llvm.org/D55727
llvm-svn: 349429
Breakpad creates minidump files that sometimes have:
- linux maps textual content
- no MemoryInfoList
Right now unless the file has a MemoryInfoList we get no region information.
This patch:
- reads and caches the memory region info one time and sorts it for easy subsequent access
- get the region info from the best source in this order:
- linux maps info (if available)
- MemoryInfoList (if available)
- MemoryList or Memory64List
- returns memory region info for the gaps between regions (before the first and after the last)
Differential Revision: https://reviews.llvm.org/D55522
llvm-svn: 349182
This commit added new test inputs, but it did not add them to the cmake
files. This caused the test to fail at runtime.
While in there, I also sorted the list of minidump test inputs.
llvm-svn: 349154
The MinidumpParser::GetFilteredModuleList() code was attempting to iterate through the entire module list and if it found more than one entry for a given module name, it wanted to pick the MinidumpModule with the lowest address. A bug existed where it wasn't doing that due to "exists" variable being inverted. "exists" was set to true if it was inserted, not if it existed. Furthermore, the order of the modules would be modified by sorting all modules from low address to high address (using MinidumpModule::base_of_image). This fix also maintains the original order which means your executable is at index 0 as intended instead of some random shared library.
Tests were added to ensure this functionality doesn't regress.
Differential Revision: https://reviews.llvm.org/D55614
llvm-svn: 349062
This patch removes the comments grouping header includes. They were
added after running IWYU over the LLDB codebase. However they add little
value, are often outdates and burdensome to maintain.
llvm-svn: 346626
This moves construction of data buffers into the FileSystem class. Like
some of the previous refactorings we don't translate the path yet
because the functionality hasn't been landed in LLVM yet.
Differential revision: https://reviews.llvm.org/D54272
llvm-svn: 346598
This patch removes the logic for resolving paths out of FileSpec and
updates call sites to rely on the FileSystem class instead.
Differential revision: https://reviews.llvm.org/D53915
llvm-svn: 345890
Corrupted minidumps was leading to unpredictable behavior.
This change adds explicit consistency checks for the minidump early on. The
checks are not comprehensive but they should catch obvious structural violations:
streams with type == 0
duplicate streams (same type)
overlapping streams
truncated minidumps
Another early check is to make sure we actually support the minidump architecture
instead of crashing at a random place deep inside LLDB.
Differential Revision: https://reviews.llvm.org/D49202
llvm-svn: 336918
Summary:
During the previous attempt to generalize the UUID class, it was
suggested that we represent invalid UUIDs as length zero (previously, we
used an all-zero UUID for that). This meant that some valid build-ids
could not be represented (it's possible however unlikely that a checksum of
some file would be zero) and complicated adding support for variable
length build-ids (should a 16-byte empty UUID compare equal to a 20-byte
empty UUID?).
This patch resolves these issues by introducing a canonical
representation for an invalid UUID. The slight complication here is that
some clients (MachO) actually use the all-zero notation to mean "no UUID
has been set". To keep this use case working (while making it very
explicit about which construction semantices are wanted), replaced the
UUID constructors and the SetBytes functions with named factory methods.
- "fromData" creates a UUID from the given data, and it treats all bytes
equally.
- "fromOptionalData" first checks the data contents - if all bytes are
zero, it treats this as an invalid/empty UUID.
Reviewers: clayborg, sas, lemo, davide, espindola
Subscribers: emaste, lldb-commits, arichardson
Differential Revision: https://reviews.llvm.org/D48479
llvm-svn: 335612
Summary:
The llvm version of the enum has the same enumerators, with stlightly
different names, so this is mostly just a search&replace exercise. One
concrete benefit of this is that we can remove the function for
converting between the two enums.
To avoid typing llvm::sys::path::Style::windows everywhere I import the
enum into the FileSpec class, so it can be referenced as
FileSpec::Style::windows.
Reviewers: zturner, clayborg
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D46753
llvm-svn: 332247
In case we are building with xml enabled, the GetMemoryRegionInfo
function will send extra packets to query te extended memory map, which
the tests were not expecting.
Add an expectation for this to the test. Right now, it's just a basic
one which pretends we don't support the extension, however, it would be
also interesting the add a test which verifies the extension-enabled
case.
I also noticed that the test does a pretty lousy job of validating the
returned memory region info, so I add a couple of extra assertions to
improve that.
llvm-svn: 331374
Summary:
We've had a mismatch in the checksum computation between the sender and
receiver. The sender computed the payload checksum using the wire
encoding of the packet, while the receiver did this after expanding
un-escaping and expanding run-length-encoded sequences. This resulted in
communication breakdown if packets using these feature were sent in the
ack mode.
Normally, this did not cause any issues since the only packet we send in
the ack-mode is the QStartNoAckMode packet, but I ran into this when
debugging the lldb-server tests which (for better or worse) don't use
this mode.
According to the gdb-remote documentation "The two-digit checksum is computed as
the modulo 256 sum of all characters between the leading ‘$’ and the
trailing ‘#’", it seems that our sender is doing the right thing here.
Therefore, I fix the receiver the match the sender behavior and add a
test.
With this bug fixed, we can see that lldb-server is sending a stop-reply
after receiving the "k" in the same way as debugserver does (but we
weren't detecting this because at that point the connection was dead
already). I fix that expectation as well.
Reviewers: clayborg, jasonmolenda
Subscribers: mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D44922
llvm-svn: 328693
Summary:
Gdb servers like openocd may send many $O reply packets for the client to output during a qRcmd command sequence. Currently, lldb interprets the first O packet as an unexpected response. Besides generating no output, this causes lldb to get out of sync with future commands because it continues reading O packets from the first command as response to subsequent commands.
This patch handles any O packets during an qRcmd, treating the first non-O packet as the true response.
Preliminary discussion at http://lists.llvm.org/pipermail/lldb-dev/2018-January/013078.html
Reviewers: clayborg
Reviewed By: clayborg
Subscribers: labath, lldb-commits
Differential Revision: https://reviews.llvm.org/D41745
Patch by Owen Shaw <llvm@owenpshaw.net>
llvm-svn: 322190
The recent UUID cleanups exposed a bug in the parsing code for the
jModulesInfo response, which was passing wrong value for the second
argument to UUID::SetFromStringRef (it passed the length of the string,
whereas the correct value should be the number of decoded bytes we
expect to receive).
This was not picked up by tests, because they test with 16-byte uuids,
for which the function happens to do the right thing even if the length
does not match (if the length does not match, the function does not
update m_num_uuid_bytes member, but that member is already 16 to begin
with).
I fix that and add a test with 20-byte uuid to catch if this regresses.
I have also added more safeguards into the parsing code to fail if we
cannot parse the entire uuid field we recieve. While testing the latter
part, I noticed that the "negative" jModulesInfo tests were succeeding
because we were sending malformed json (and not because the json
contents was invalid), so I make those tests a bit more robuts as well.
llvm-svn: 320985
The rationale here is that ArchSpec is used throughout the codebase,
including in places which should not depend on the rest of the code in
the Core module.
This commit touches many files, but most of it is just renaming of
#include lines. In a couple of cases, I removed the #include ArchSpec
line altogether, as the file was not using it. In one or two places,
this necessitated adding other #includes like lldb-private-defines.h.
llvm-svn: 318048
Summary:
At present, several gtests in the lldb open source codebase are using
#include statements rooted at $(SOURCE_ROOT)/${LLDB_PROJECT_ROOT}.
This patch cleans up this directory/include structure for both CMake and
Xcode build systems.
rdar://problem/33835795
Reviewers: zturner, jingham, beanz
Reviewed By: beanz
Subscribers: emaste, lldb-commits, mgorny
Differential Revision: https://reviews.llvm.org/D36598
llvm-svn: 314849
Summary:
The classes have no dependencies, and they are used both by lldb and
lldb-server, so it makes sense for them to live in the lowest layers.
Reviewers: zturner, jingham
Subscribers: emaste, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D34746
llvm-svn: 306682
Summary:
Fetching an input file required about five lines of code, and this was
repeated in multiple unit tests, with slight variations. Add a helper
function for doing that into the lldbUtilityMocks module (which I rename
to lldbUtilityHelpers to commemorate the fact it includes more than
mocks)
Reviewers: zturner, eugene
Subscribers: emaste, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D34683
llvm-svn: 306668
Summary:
This patch implements support for Intel(R) Processor Trace
in lldb server. The changes have support for
starting/stopping and reading the trace data. The code
is only available on Linux versions where the perf
attributes for aux buffers are available.
The patch also consists of Unit tests for testing the
core buffer reading function.
Reviewers: lldb-commits, labath, clayborg, zturner, tberghammer
Reviewed By: labath, clayborg
Subscribers: mgorny
Differential Revision: https://reviews.llvm.org/D33674
llvm-svn: 306516
Summary:
It had a dependency on StringConvert and file reading code, which is not
in Utility. I've replaced that code by equivalent llvm operations.
I've added a unit test to demonstrate that parsing a file still works.
Reviewers: zturner, jingham
Subscribers: kubamracek, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D34625
llvm-svn: 306394
Instead of every test creating a client-server combo, do that in the
SetUp method of the test fixture. This also means that we can rely on
gtest to not run the test if the SetUp method fails and delete the
if(HasFailure) calls.
llvm-svn: 306013
Summary:
The changes consist of new packets for trace manipulation and
trace collection. The new packets are also documented. The packets
are capable of providing custom trace specific parameters to start
tracing and also retrieve such configuration from the server.
Reviewers: clayborg, lldb-commits, tberghammer, labath, zturner
Reviewed By: clayborg, labath
Subscribers: krytarowski, lldb-commits
Differential Revision: https://reviews.llvm.org/D32585
llvm-svn: 303972
Antonio Afonso