Summary:
Many of our tests need to initialize certain subsystems/plugins of LLDB such as
`FileSystem` or `HostInfo` by calling their static `Initialize` functions before the
test starts and then calling `::Terminate` after the test is done (in reverse order).
This adds a lot of error-prone boilerplate code to our testing code.
This patch adds a RAII called SubsystemRAII that ensures that we always call
::Initialize and then call ::Terminate after the test is done (and that the Terminate
calls are always in the reverse order of the ::Initialize calls). It also gets rid of
all of the boilerplate that we had for these calls.
Per-fixture initialization is still not very nice with this approach as it would
require some kind of static unique_ptr that gets manually assigned/reseted
from the gtest SetUpTestCase/TearDownTestCase functions. Because of that
I changed all per-fixture setup to now do per-test setup which can be done
by just having the SubsystemRAII as a member of the test fixture. This change doesn't
influence our normal test runtime as LIT anyway runs each test case separately
(and the Initialize/Terminate calls are anyway not very expensive). It will however
make running all tests in a single executable slightly slower.
Reviewers: labath, JDevlieghere, martong, espindola, shafik
Reviewed By: labath
Subscribers: mgorny, rnkovacs, emaste, MaskRay, abidh, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71630
The goal of this refactor is to enable ProcessMinidump to take into
account the loaded modules and their sections when computing the
permissions of various ranges of memory, as discussed in D66638.
This patch moves some of the responsibility for computing the ranges
from MinidumpParser into ProcessMinidump. MinidumpParser still does the
parsing, but ProcessMinidump becomes responsible for answering the
actual queries about memory ranges. This will enable it (in a follow-up
patch) to augment the information obtained from the parser with data
obtained from actual object files.
The changes in the actual code are fairly straight-forward and just
involve moving code around. MinidumpParser::GetMemoryRegions is renamed
to BuildMemoryRegions to emphasize that it does no caching. The only new
thing is the additional bool flag returned from this function. This
indicates whether the returned regions describe all memory mapped into
the target process. Data obtained from /proc/maps and the MemoryInfoList
stream is considered to be exhaustive. Data obtained from Memory(64)List
is not. This will be used to determine whether we need to augment the
data or not.
This reshuffle means that it is no longer possible/easy to test some of
this code via unit tests, as constructing a ProcessMinidump instance is
hard. Instead, I update the unit tests to only test the parsing of the
actual data, and test the answering of queries through a lit test using
the "memory region" command. The patch also includes some tweaks to the
MemoryRegion class to make the unit tests easier to write.
Reviewers: amccarth, clayborg
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D69035
Summary: The types defined for it in LLDB are now redundant with core types.
Reviewers: labath, clayborg
Reviewed By: clayborg
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D68658
llvm-svn: 375243
EXPECT_EQ contains an if-else statement. It also contains some magic to
suppress the dangling else warnings, but it seems that some new
compilers can see through that...
llvm-svn: 374341
Stack unwinding was sometimes failing when trying to unwind stacks in 32 bit ARM. I discovered this was because the EH frame register numbers were not set. This patch fixes this issue and adds a unit test to verify this doesn't regress.
Differential Revision: https://reviews.llvm.org/D68088
llvm-svn: 374246
This patch replaces the hand-rolled JSON emission in StructuredData with
LLVM's JSON library.
Differential revision: https://reviews.llvm.org/D68248
llvm-svn: 373359
To support dumping the reproducer's GDB remote packets, we need the
(de)serialization logic to live in Utility rather than the GDB remote
plugin. This patch renames StreamGDBRemote to GDBRemote and moves the
relevant packet code there.
Its uses in the GDBRemoteCommunicationHistory and the
GDBRemoteCommunicationReplayServer are updated as well.
Differential revision: https://reviews.llvm.org/D67523
llvm-svn: 371907
previously they used a minidump-specific function for this purpose, but
this is no longer needed now that whole of yaml2obj is available as a
library.
llvm-svn: 369379
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368933
Summary:
This is the third patch to improve module loading in a series that started here (where I explain the motivation and solution): D62499
Add functions to read the r_debug location to know where the linked list of loaded libraries are so I can generate the `xfer:libraries-svr4` packet.
I'm also using this function to implement `GetSharedLibraryInfoAddress` that was "not implemented" for linux.
Most of this code was inspired by the current ds2 implementation here: https://github.com/facebook/ds2/blob/master/Sources/Target/POSIX/ELFProcess.cpp.
Reviewers: clayborg, xiaobai, labath
Reviewed By: clayborg, labath
Subscribers: emaste, krytarowski, mgorny, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D62501
llvm-svn: 363458
Summary:
This is the first of a few patches I have to improve the performance of dynamic module loading on Android.
In this first diff I'll describe the context of my main motivation and will then link to it in the other diffs to avoid repeating myself.
## Motivation
I have a few scenarios where opening a specific feature on an Android app takes around 40s when lldb is attached to it. The reason for that is because 40 modules are dynamicly loaded at that point in time and each one of them is taking ~1s.
## The problem
To learn about new modules we have a breakpoint on a linker function that is called twice whenever a module is loaded. One time just before it's loaded (so lldb can check which modules are loaded) and another right after it's loaded (so lldb can check again which ones are loaded and calculate the diference).
It's figuring out which modules are loaded that is taking quite some time. This is currently done by traversing the linked list of loaded shared libraries that the linker maintains in memory. Each item in the linked list requires its own `x` packet sent to the gdb server (this is android so the network also plays a part). In my scenario there are 400+ loaded libraries and even though we read 0x800 worth of bytes at a time we still make ~180 requests that end up taking 150-200ms.
We also do this twice, once before the module is loaded (state = eAdd) and another right after (state = eConsistent) which easly adds up to ~400ms per module.
## A solution
**Implement `xfer:libraries-svr4` in lldb-server:**
I noticed in the code that loads the new modules that it had support for the `xfer:libraries-svr4` packet (added ~4 years ago to support the ds2 debug server) but we didn't support it in lldb-server. This single packet returns an xml list of all the loaded modules by the process. The advantage is that there's no more need to make 180 requests to read the linked list. Additionally this new requests takes around 10ms.
**More efficient usage of the `xfer:libraries-svr4` packet in lldb:**
When `xfer:libraries-svr4` is available the Process class has a `LoadModules` function that requests this packet and then loads or unloads modules based on the current list of loaded modules by the process.
This is the function that is used by the DYLDRendezvous class to get the list of loaded modules before and after the module is loaded. However, this is really not needed since the LoadModules function already loaded or unloaded the modules accordingly. I changed this strategy to call LoadModules only once (after the process has loaded the module).
**Bugs**
I found a few issues in lldb while implementing this and have submitted independent patches for them.
I tried to devide this into multiple logical patches to make it easier to review and discuss.
## Tests
I wanted to put these set of diffs up before having all the tests up and running to start having them reviewed from a techical point of view. I'm also having some trouble making the tests running on linux so I need more time to make that happen.
# This diff
The `xfer` packages follow the same protocol, they are requested with `xfer:<object>:<read|write>:<annex>:<offset,length>` and a return that starts with `l` or `m` depending if the offset and length covers the entire data or not. Before implementing the `xfer:libraries-svr4` I refactored the `xfer:auxv` to generically handle xfer packets so we can easly add new ones.
The overall structure of the function ends up being:
* Parse the packet into its components: object, offset etc.
* Depending on the object do its own logic to generate the data.
* Return the data based on its size, the requested offset and length.
Reviewers: clayborg, xiaobai, labath
Reviewed By: labath
Subscribers: mgorny, krytarowski, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D62499
llvm-svn: 362982
Summary:
According to [C128] "Virtual functions should specify exactly one
of `virtual`, `override`, or `final`", I've added override where a
virtual function is overriden but the explicit `override` keyword
was missing. Whenever both `virtual` and `override` were specified,
I removed `virtual`. As C.128 puts it:
> [...] writing more than one of these three is both redundant and
> a potential source of errors.
I anticipate a discussion about whether or not to add `override` to
destructors but I went for it because of an example in [ISOCPP1000].
Let me repeat the comment for you here:
Consider this code:
```
struct Base {
virtual ~Base(){}
};
struct SubClass : Base {
~SubClass() {
std::cout << "It works!\n";
}
};
int main() {
std::unique_ptr<Base> ptr = std::make_unique<SubClass>();
}
```
If for some odd reason somebody removes the `virtual` keyword from the
`Base` struct, the code will no longer print `It works!`. So adding
`override` to destructors actively protects us from accidentally
breaking our code at runtime.
[C128]: https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#c128-virtual-functions-should-specify-exactly-one-of-virtual-override-or-final
[ISOCPP1000]: https://github.com/isocpp/CppCoreGuidelines/issues/1000#issuecomment-476951555
Reviewers: teemperor, JDevlieghere, davide, shafik
Reviewed By: teemperor
Subscribers: kwk, arphaman, kadircet, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D61440
llvm-svn: 359868
The tests reading the untouched module list are now not using any lldb
code (as module list loading lives in llvm now), so they can be removed.
The "filtering" of the module list remains (and probably will remain) an
lldb concept, so I keep those tests, but replace the checked-in binaries
with their yaml equivalents.
The binaries which are no longer referenced by any tests have been
removed.
llvm-svn: 358850
In this patch, I just remove the structure definitions for the
ModuleList stream and the associated parsing code. The rest of the code
is converted to work with the definitions in llvm. NFC.
llvm-svn: 358070
I also update the tests for SystemInfo parsing to use the yaml2minidump
capabilities in llvm instead of relying on checked-in binaries.
llvm-svn: 357896
This patch removes the lower layers of the minidump parsing code from
the MinidumpParser class, and replaces it with the minidump parser in
llvm.
Not all functionality is already avaiable in the llvm class, but it is
enough for us to be able to stop enumerating streams manually, and rely
on the minidump directory parsing code from the llvm class.
This also removes some checked-in binaries which were used to test error
handling in the parser, as the error handling is now done (and tested)
in llvm. Instead I just add one test that ensures we correctly propagate
the errors reported by the llvm parser. The input for this test can be
written in yaml instead of a checked-in binary.
llvm-svn: 357748
remove the Initialize function, move the things that can fail into the
static factory function. The factory function now returns
Expected<Parser> instead of Optional<Parser> so that it can give a
reason why creation failed.
llvm-svn: 354668
The tests were doing two somewhat independent things:
- checking that the registers can be retrieved from the minidump file
- checking that they can be converted into a form suitable for
consumption by lldb
The first thing requires a minidump file (but it's independent of other
lldb structures), while the second one does not require a minidump file
(but it needs lldb register info structures).
Splitting this into two tests gives an opportunity to write more
detailed tests, and allows the two pieces of functionality to be moved
into different packages, if that proves to be necessary.
llvm-svn: 354662
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Each process plug-in can create its own custom commands. I figured it would be nice to be able to dump things from the minidump file from the lldb command line, so I added the start of the some custom commands.
Currently you can dump:
minidump stream directory
all linux specifc streams, most of which are strings
each linux stream individually if desired, or all with --linux
The idea is we can expand the command set to dump more things, search for data in the core file, and much more. This patch gets us started.
Differential Revision: https://reviews.llvm.org/D55727
llvm-svn: 349429
Breakpad creates minidump files that sometimes have:
- linux maps textual content
- no MemoryInfoList
Right now unless the file has a MemoryInfoList we get no region information.
This patch:
- reads and caches the memory region info one time and sorts it for easy subsequent access
- get the region info from the best source in this order:
- linux maps info (if available)
- MemoryInfoList (if available)
- MemoryList or Memory64List
- returns memory region info for the gaps between regions (before the first and after the last)
Differential Revision: https://reviews.llvm.org/D55522
llvm-svn: 349182
This commit added new test inputs, but it did not add them to the cmake
files. This caused the test to fail at runtime.
While in there, I also sorted the list of minidump test inputs.
llvm-svn: 349154
The MinidumpParser::GetFilteredModuleList() code was attempting to iterate through the entire module list and if it found more than one entry for a given module name, it wanted to pick the MinidumpModule with the lowest address. A bug existed where it wasn't doing that due to "exists" variable being inverted. "exists" was set to true if it was inserted, not if it existed. Furthermore, the order of the modules would be modified by sorting all modules from low address to high address (using MinidumpModule::base_of_image). This fix also maintains the original order which means your executable is at index 0 as intended instead of some random shared library.
Tests were added to ensure this functionality doesn't regress.
Differential Revision: https://reviews.llvm.org/D55614
llvm-svn: 349062
This patch removes the comments grouping header includes. They were
added after running IWYU over the LLDB codebase. However they add little
value, are often outdates and burdensome to maintain.
llvm-svn: 346626
This moves construction of data buffers into the FileSystem class. Like
some of the previous refactorings we don't translate the path yet
because the functionality hasn't been landed in LLVM yet.
Differential revision: https://reviews.llvm.org/D54272
llvm-svn: 346598
This patch removes the logic for resolving paths out of FileSpec and
updates call sites to rely on the FileSystem class instead.
Differential revision: https://reviews.llvm.org/D53915
llvm-svn: 345890
Corrupted minidumps was leading to unpredictable behavior.
This change adds explicit consistency checks for the minidump early on. The
checks are not comprehensive but they should catch obvious structural violations:
streams with type == 0
duplicate streams (same type)
overlapping streams
truncated minidumps
Another early check is to make sure we actually support the minidump architecture
instead of crashing at a random place deep inside LLDB.
Differential Revision: https://reviews.llvm.org/D49202
llvm-svn: 336918
Summary:
During the previous attempt to generalize the UUID class, it was
suggested that we represent invalid UUIDs as length zero (previously, we
used an all-zero UUID for that). This meant that some valid build-ids
could not be represented (it's possible however unlikely that a checksum of
some file would be zero) and complicated adding support for variable
length build-ids (should a 16-byte empty UUID compare equal to a 20-byte
empty UUID?).
This patch resolves these issues by introducing a canonical
representation for an invalid UUID. The slight complication here is that
some clients (MachO) actually use the all-zero notation to mean "no UUID
has been set". To keep this use case working (while making it very
explicit about which construction semantices are wanted), replaced the
UUID constructors and the SetBytes functions with named factory methods.
- "fromData" creates a UUID from the given data, and it treats all bytes
equally.
- "fromOptionalData" first checks the data contents - if all bytes are
zero, it treats this as an invalid/empty UUID.
Reviewers: clayborg, sas, lemo, davide, espindola
Subscribers: emaste, lldb-commits, arichardson
Differential Revision: https://reviews.llvm.org/D48479
llvm-svn: 335612
Summary:
The llvm version of the enum has the same enumerators, with stlightly
different names, so this is mostly just a search&replace exercise. One
concrete benefit of this is that we can remove the function for
converting between the two enums.
To avoid typing llvm::sys::path::Style::windows everywhere I import the
enum into the FileSpec class, so it can be referenced as
FileSpec::Style::windows.
Reviewers: zturner, clayborg
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D46753
llvm-svn: 332247
In case we are building with xml enabled, the GetMemoryRegionInfo
function will send extra packets to query te extended memory map, which
the tests were not expecting.
Add an expectation for this to the test. Right now, it's just a basic
one which pretends we don't support the extension, however, it would be
also interesting the add a test which verifies the extension-enabled
case.
I also noticed that the test does a pretty lousy job of validating the
returned memory region info, so I add a couple of extra assertions to
improve that.
llvm-svn: 331374
Summary:
We've had a mismatch in the checksum computation between the sender and
receiver. The sender computed the payload checksum using the wire
encoding of the packet, while the receiver did this after expanding
un-escaping and expanding run-length-encoded sequences. This resulted in
communication breakdown if packets using these feature were sent in the
ack mode.
Normally, this did not cause any issues since the only packet we send in
the ack-mode is the QStartNoAckMode packet, but I ran into this when
debugging the lldb-server tests which (for better or worse) don't use
this mode.
According to the gdb-remote documentation "The two-digit checksum is computed as
the modulo 256 sum of all characters between the leading ‘$’ and the
trailing ‘#’", it seems that our sender is doing the right thing here.
Therefore, I fix the receiver the match the sender behavior and add a
test.
With this bug fixed, we can see that lldb-server is sending a stop-reply
after receiving the "k" in the same way as debugserver does (but we
weren't detecting this because at that point the connection was dead
already). I fix that expectation as well.
Reviewers: clayborg, jasonmolenda
Subscribers: mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D44922
llvm-svn: 328693
Summary:
Gdb servers like openocd may send many $O reply packets for the client to output during a qRcmd command sequence. Currently, lldb interprets the first O packet as an unexpected response. Besides generating no output, this causes lldb to get out of sync with future commands because it continues reading O packets from the first command as response to subsequent commands.
This patch handles any O packets during an qRcmd, treating the first non-O packet as the true response.
Preliminary discussion at http://lists.llvm.org/pipermail/lldb-dev/2018-January/013078.html
Reviewers: clayborg
Reviewed By: clayborg
Subscribers: labath, lldb-commits
Differential Revision: https://reviews.llvm.org/D41745
Patch by Owen Shaw <llvm@owenpshaw.net>
llvm-svn: 322190
The recent UUID cleanups exposed a bug in the parsing code for the
jModulesInfo response, which was passing wrong value for the second
argument to UUID::SetFromStringRef (it passed the length of the string,
whereas the correct value should be the number of decoded bytes we
expect to receive).
This was not picked up by tests, because they test with 16-byte uuids,
for which the function happens to do the right thing even if the length
does not match (if the length does not match, the function does not
update m_num_uuid_bytes member, but that member is already 16 to begin
with).
I fix that and add a test with 20-byte uuid to catch if this regresses.
I have also added more safeguards into the parsing code to fail if we
cannot parse the entire uuid field we recieve. While testing the latter
part, I noticed that the "negative" jModulesInfo tests were succeeding
because we were sending malformed json (and not because the json
contents was invalid), so I make those tests a bit more robuts as well.
llvm-svn: 320985
Raphael Isemann