As usual with that header cleanup series, some implicit dependencies now need to
be explicit:
llvm/DebugInfo/DWARF/DWARFContext.h no longer includes:
- "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
- "llvm/DebugInfo/DWARF/DWARFCompileUnit.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAranges.h"
- "llvm/DebugInfo/DWARF/DWARFDebugFrame.h"
- "llvm/DebugInfo/DWARF/DWARFDebugLoc.h"
- "llvm/DebugInfo/DWARF/DWARFDebugMacro.h"
- "llvm/DebugInfo/DWARF/DWARFGdbIndex.h"
- "llvm/DebugInfo/DWARF/DWARFSection.h"
- "llvm/DebugInfo/DWARF/DWARFTypeUnit.h"
- "llvm/DebugInfo/DWARF/DWARFUnitIndex.h"
Plus llvm/Support/Errc.h not included by a bunch of llvm/DebugInfo/DWARF/DWARF*.h files
Preprocessed lines to build llvm on my setup:
after: 1065629059
before: 1066621848
Which is a great diff!
Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119723
lldb reports (and lldbutil.continue_to_breakpoint returns) a stop reason
even for suspended threads. Fix the test to expect that.
This was making the test flaky, as most of the time, the two threads
stop simultaneously, and the synchronization code is not executed.
The symbolicator assumes that the first image in the image list is the
main image. That isn't always the case. For JSON crashlogs we can use
the procName to move the main image to the front of the list.
rdar://83907760
One of the tests in this test setup was copied from a more complex test, and I didn't know
if the setup or the subsequent parts of the test were the ones that fail on Linux. Looks
like it was the latter, so let's mark this succeeding.
Some dyld cross library stubs can have line information but no function. Make sure you
check that you have a valid Function object before asking it questions.
Differential Revision: https://reviews.llvm.org/D119297
This way if you have a long stack, you can issue "thread backtrace --count 10"
and then subsequent <Return>-s will page you through the stack.
This took a little more effort than just adding the repeat command, since
the GetRepeatCommand API was returning a "const char *". That meant the command
had to keep the repeat string alive, which is inconvenient. The original
API returned either a nullptr, or a const char *, so I changed the private API to
return an llvm::Optional<std::string>. Most of the patch is propagating that change.
Also, there was a little thinko in fetching the repeat command. We don't
fetch repeat commands for commands that aren't being added to history, which
is in general reasonable. And we don't add repeat commands to the history -
also reasonable. But we do want the repeat command to be able to generate
the NEXT repeat command. So I adjusted the logic in HandleCommand to work
that way.
Differential Revision: https://reviews.llvm.org/D119046
Instead trying to pro-actively determine if the first line in a
crashlog contains meta data, change the heuristic to do the following:
1. To trying to parse the whole file. If that fails, then:
2. Strip the first line and try parsing the remainder of the file. If
that fails, then:
3. Fall back to the textual crashlog parser.
rdar://88580543
Differential revision: https://reviews.llvm.org/D119755
ObjectFileMachO, for a couple of special binaries at the initial
launch, needs to find segment load addresses before the Target's
SectionLoadList has been initialized. The calculation to find
the first segment, which is at the same address as the mach header,
was not correct if the binary was in the Darwin shared cache.
Update the logic to handle that case.
Differential Revision: https://reviews.llvm.org/D119602
rdar://88802629
The IR interpreter supports const operands to the `GetElementPtr` IR
instruction, so it should be able to evaluate expression without JIT.
Follow up to https://reviews.llvm.org/D113498
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D119734
The current dectorator (@skipIfLinux) will skip the test if the lldb
platform is the linux platform, but the issue is with the OS that lldb
is running on, not the OS that lldb is debugging. Update the decorator
to skip the test if the host is Linux.
Thank you Ted Woodward for pointing this out.
Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.
The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.
For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.
The recommit fixes the LLDB build failure.
Differential Revision: https://reviews.llvm.org/D117348
Replace forms of `assertTrue(err.Success())` with `assertSuccess(err)` (added in D82759).
* `assertSuccess` prints out the error's message
* `assertSuccess` expresses explicit higher level semantics, both to the reader and for test failure output
* `assertSuccess` seems not to be well known, using it where possible will help spread knowledge
* `assertSuccess` statements are more succinct
Differential Revision: https://reviews.llvm.org/D119616
This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list.
So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files.
Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress.
For a module with a dSYM file, we can see the "symbolFilePath" is included:
```
"modules": [
{
"debugInfoByteSize": 1070,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 4873280600,
"path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out",
"symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out",
"symbolTableIndexTime": 7.9999999999999996e-06,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 7.8999999999999996e-05,
"symbolTableSavedToCache": false,
"triple": "arm64-apple-macosx12.0.0",
"uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3"
},
```
And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats:
```
"modules": [
{
"debugInfoByteSize": 0,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 4603526968,
"path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out",
"symbolFileModuleIdentifiers": [
4604429832
],
"symbolTableIndexTime": 7.9999999999999996e-06,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.000112,
"symbolTableSavedToCache": false,
"triple": "arm64-apple-macosx12.0.0",
"uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569"
},
```
And the .o file for 4604429832 looks like:
```
{
"debugInfoByteSize": 1028,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 6.0999999999999999e-05,
"identifier": 4604429832,
"path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o",
"symbolTableIndexTime": 0,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0,
"symbolTableSavedToCache": false,
"triple": "arm64-apple-macosx"
}
```
Differential Revision: https://reviews.llvm.org/D119400
This reverts commit 0df522969a.
Additional checks are added to fix the detection of the last memory region
in GetMemoryRegions or repeating the "memory region" command when the
target has non-address bits.
Normally you keep reading from address 0, looking up each region's end
address until you get LLDB_INVALID_ADDR as the region end address.
(0xffffffffffffffff)
This is what the remote will return once you go beyond the last mapped region:
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
[0x0001000000000000-0xffffffffffffffff) ---
Problem is that when we "fix" the lookup address, we remove some bits
from it. On an AArch64 system we have 48 bit virtual addresses, so when
we fix the end address of the [stack] region the result is 0.
So we loop back to the start.
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
[0x0000000000000000-0x0000000000400000) ---
To fix this I added an additional check for the last range.
If the end address of the region is different once you apply
FixDataAddress, we are at the last region.
Since the end of the last region will be the last valid mappable
address, plus 1. That 1 will be removed by the ABI plugin.
The only side effect is that on systems with non-address bits, you
won't get that last catch all unmapped region from the max virtual
address up to 0xf...f.
[0x0000fffff8000000-0x0000fffffffdf000) ---
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
<ends here>
Though in some way this is more correct because that region is not
just unmapped, it's not mappable at all.
No extra testing is needed because this is already covered by
TestMemoryRegion.py, I simply forgot to run it on system that had
both top byte ignore and pointer authentication.
This change has been tested on a qemu VM with top byte ignore,
memory tagging and pointer authentication enabled.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D115508
This patch fixes the register parser for arm64 crashlogs.
Compared to x86_64 crashlogs, the arm64 crashlogs nests the general
purpose registers into a separate dictionary within `thread_state`
dictionary. It uses the dictionary key as the the register number.
Differential Revision: https://reviews.llvm.org/D119168
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This splits the scripted process tests to be able to run in parallel
since some of test functions can take a very long time to run.
This also disables debug info testing.
Differential Revision: https://reviews.llvm.org/D118513
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch fixes a timeout issue on the ScriptedProcess test that was
happening on intel platforms. The timeout was due to a misreporting of
the StopInfo in the ScriptedThread that caused the ScriptedProcess to
never stop.
To solve this, this patch changes the way a ScriptedThread reports its
stop reason by making it more architecture specific. In order to do so,
this patch also refactors the ScriptedProcess & ScriptedThread
initializer methods to provide an easy access to the target architecture.
Differential Revision: https://reviews.llvm.org/D118484
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
After 9611282c, TestGdbRemoteThreadsInStopReply is not non-deterministic
-- instead it deterministically fails due to extra threads created by
std::thread thread pool.
Adjust the tests to account for that.
Operands to `getelementptr` can be constants or constant expressions. Check
that all operands can be constant-resolved and resolve them during the
evaluation. If some operands can't be resolved as constants -- the expression
evaluation will fallback to JIT.
Fixes: https://bugs.llvm.org/show_bug.cgi?id=52449
Reviewed By: #lldb, shafik
Differential Revision: https://reviews.llvm.org/D113498
The tests enabled in 9e699595 are not passing reliably -- sometimes they
report seeing fewer threads than expected.
Based on my (limited) research, this is not a lldb bug, but simply a
consequence of the operating system reporting their presence
asynchronously -- they're reported when they are scheduled to run (or
something similar), and not at the time of the CreateThread call.
To fix this, I add some additional synchronization to the test inferior,
which makes sure that the created thread is alive before continuing to
do other things.
This patch updates toolchain-msvc.test to cater for Arm64 windows platform.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D117676
D119167 changed the meaning of that test by removing the use of the
interrupt packet. I did not notice this because the interrupting
happened in a shared utility function.
This patch restores the original meaning of the test, but (almost)
avoids sleeps by using process stdout to synchronize. Sadly, this means
the test stays disabled on windows, as it does not implement output
forwarding.
A couple of additional tests pass with that patch. One new test fails
(because it's not testing a slightly different thing). I'll update it
later to restore the original meaning (I don't want to revert as the net
effect is still very positive), but for now this gets the bot green.
Instead of using sleeps, have the inferior notify us (via a trap opcode) that
the requested number of threads have been created.
This allows us to get rid of some fairly dodgy test utility code --
wait_for_thread_count seemed like it was waiting for the threads to
appear, but it never actually let the inferior run, so it only succeeded
if the threads were already started when the function was called. Since
the function was called after a fairly small delay (1s, usually), this
is probably the reason why the tests were failing on some bots.
Differential Revision: https://reviews.llvm.org/D119167
Major user-facing changes:
Many headers in llvm/DebugInfo/CodeView no longer include
llvm/Support/BinaryStreamReader.h or llvm/Support/BinaryStreamWriter.h,
those headers may need to be included manually.
Several headers in llvm/DebugInfo/CodeView no longer include
llvm/DebugInfo/CodeView/EnumTables.h or llvm/DebugInfo/CodeView/CodeView.h,
those headers may need to be included manually.
Some statistics:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/DebugInfo/CodeView/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
after: 2794466
before: 2832765
Discourse thread on the topic: https://discourse.llvm.org/t/include-what-you-use-include-cleanup/
Differential Revision: https://reviews.llvm.org/D119092
Reviewing some recent fixes to the platform packet implementations
in lldb, I saw the docs were out of sync in a few spots. Updated them.
Differential Revision: https://reviews.llvm.org/D118842
When LLVM_ENABLE_ZLIB is ON gdb-remote should link against ZLIB::ZLIB.
This fixes
```
/mnt/b/yoe/master/build/tmp/hosttools/ld: lib/liblldbPluginProcessGDBRemote.a(GDBRemoteCommunication.cpp.o): in function `lldb_private::process_gdb_remote::GDBRemoteCommunication::DecompressPacket() [clone .localalias]':
GDBRemoteCommunication.cpp:(.text._ZN12lldb_private18process_gdb_remote22GDBRemoteCommunication16DecompressPacketEv+0x59a): undefined reference to `inflateInit2_'
/mnt/b/yoe/master/build/tmp/hosttools/ld: GDBRemoteCommunication.cpp:(.text._ZN12lldb_private18process_gdb_remote22GDBRemoteCommunication16DecompressPacketEv+0x5af): undefined reference to `inflate'
```
Reviewed By: JDevlieghere, MaskRay
Differential Revision: https://reviews.llvm.org/D119186
Update `__init__.py` generation to implement `__lldb_init_module`, which calls
`__lldb_init_module` on submodules that define it.
This allows the use case where a user runs `command script import lldb.macosx`.
With this change, the `__lldb_init_module` function in `crashlog.py` and
`heap.py` will be run, which is where command registration is occurring.
Differential Revision: https://reviews.llvm.org/D119179
Previously, importing `crashlog` resulted in a message being printed. The
message was about other commands (those in heap.py), not `crashlog`. The
changes in D117237 made it so that the heap.py messages were printed only when
importing `lldb.macosx.heap`, not when importing `lldb.macosx.crashlog`. Some
users may see no output and think `crashlog` wasn't successfully loaded. This
ensures users see that `crashlog` is loaded.
rdar://88283132
Differential Revision: https://reviews.llvm.org/D119155
As Pavel pointed out, on Apple Silicon "b main" stops at a point after
the variable has already been initialized. This patch updates the test
case to avoids that. I've also split the test into separate files so its
easier to reproduce the individual scenarios without having to build any
shared state.
This fixes TestGdbRemoteSingleStep.py and TestGdbRemote_vCont.py. This
patch updates the test to account for the possibility that the constants
are already materialized. This appears to behave differently between
embedded arm64 devices and Apple Silicon.
After aed965d55d we no longer demangle and store the full name. The
test was updated accordingly but the comment still specified that we
should be able to find the symbol by its full demangled name.
Jonas Devlieghere