(The upgrade of the ppc64le bot and D121257 have fixed compiler-rt failures. Tested by nemanjai.)
Default the option introduced in D113372 to ON to match all(?) major Linux
distros. This matches GCC and improves consistency with Android and linux-musl
which always default to PIE.
Note: CLANG_DEFAULT_PIE_ON_LINUX may be removed in the future.
Differential Revision: https://reviews.llvm.org/D120305
This matches how another similar warning is silenced in
Host/posix/ProcessLauncherPosixFork.cpp.
Differential Revision: https://reviews.llvm.org/D123205
This silences warnings like this:
lldb/source/Core/DebuggerEvents.cpp: In member function ‘llvm::StringRef lldb_private::DiagnosticEventData::GetPrefix() const’:
lldb/source/Core/DebuggerEvents.cpp:55:1: warning: control reaches end of non-void function [-Wreturn-type]
55 | }
Differential Revision: https://reviews.llvm.org/D123203
If testing for a warning option like -Wno-<foo> with GCC, GCC won't
print any diagnostic at all, leading to the options being accepted
incorrectly. However later, if compiling a file that actually prints
another warning, GCC will also print warnings about these -Wno-<foo>
options being unrecognized.
This avoids warning spam like this, for every lldb source file that
produces build warnings with GCC:
At global scope:
cc1plus: warning: unrecognized command line option ‘-Wno-vla-extension’
cc1plus: warning: unrecognized command line option ‘-Wno-deprecated-register’
This matches how such warning options are detected and added in
llvm/cmake/modules/HandleLLVMOptions.cmake, e.g. like this:
check_cxx_compiler_flag("-Wclass-memaccess" CXX_SUPPORTS_CLASS_MEMACCESS_FLAG)
append_if(CXX_SUPPORTS_CLASS_MEMACCESS_FLAG "-Wno-class-memaccess" CMAKE_CXX_FLAGS)
Differential Revision: https://reviews.llvm.org/D123202
In order to support quick arbitrary access to instructions in the trace, we need
each instruction to have an id. It could be an index or any other value that the
trace plugin defines.
This will be useful for reverse debugging or for creating callstacks, as each
frame will need an instruction id associated with them.
I've updated the `thread trace dump instructions` command accordingly. It now
prints the instruction id instead of relative offset. I've also added a new --id
argument that allows starting the dump from an arbitrary position.
Differential Revision: https://reviews.llvm.org/D122254
After enabling the LLDB index cache in production we discovered that some distributed build systems play with the modification times of any .o files that were downloaded from the build cache. This was causing the LLDB index cache to read the wrong cache file for files that didn't have a UUID as all of the modfication times were set to the same value by the build system. When new .o files were downloaded, the only unique identifier was the mod time which were all the same, and we would load an older cache for the updated .o file. So disabling caching of files that have no UUIDs for now until we can create a more solid solution.
Differential Revision: https://reviews.llvm.org/D120948
Currently, all data buffers are assumed to be writable. This is a
problem on macOS where it's not allowed to load unsigned binaries in
memory as writable. To be more precise, MAP_RESILIENT_CODESIGN and
MAP_RESILIENT_MEDIA need to be set for mapped (unsigned) binaries on our
platform.
Binaries are mapped through FileSystem::CreateDataBuffer which returns a
DataBufferLLVM. The latter is backed by a llvm::WritableMemoryBuffer
because every DataBuffer in LLDB is considered to be writable. In order
to use a read-only llvm::MemoryBuffer I had to split our abstraction
around it.
This patch distinguishes between a DataBuffer (read-only) and
WritableDataBuffer (read-write) and updates LLDB to use the appropriate
one.
rdar://74890607
Differential revision: https://reviews.llvm.org/D122856
In https://reviews.llvm.org/D118972 I increased this buffer to be
big enough to import 261,144 classes but this is a lot more than
we currently have, an allocating a too-large buffer can add memory
pressure even if it's only for a short time. Reduce the size of
this memory buffer to big enough to import 163,840 classes. I'll
probably move to a scheme where we read the objc classes in chunks,
with a smaller buffer and multiple inferior function calls.
rdar://91275493
This patch implements stderr/stdout forwarding on windows.
This was previously not implemented in D99974.
I added separate callbacks so the output can be sent to the different channels VSCode provides (OutputType::Stdout, OutputType::Stderr, OutputType::Console).
This patch also passes a log callback handler to SBDebugger::Create to be able to see logging output when it is enabled.
Since the output is now redirect on early startup I removed the calls to SetOutputFileHandle/SetErrorFileHandle, which set them to /dev/null.
I send the output of stderr/stdout/lldb log to OutputType::Console
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D123025
Improve the documentation for the platform functions that take a process
host architecture as input.
Differential revision: https://reviews.llvm.org/D122767
Fix undefined behavior in AppleObjCRuntimeV2 where we were left shifting
a signed value. This also removes redundant casts of unobfuscated to
uint64_t which it already is.
rdar://91242879
Differential revision: https://reviews.llvm.org/D123098
When opening core files (and also in some other situations) we could end
up with two vdso modules. This could happen because the vdso module is
very special, and over the years, we have accumulated various ways to
load it.
In D10800, we added one mechanism for loading it, which took the form of
a generic load-from-memory capability. Unfortunately loading an elf file
from memory is not possible (because the loader never loads the entire
file), and our attempts to do so were causing crashes. So, in D34352, we
partially reverted D10800 and implemented a custom mechanism specific to
the vdso.
Unfortunately, enough of D10800 remained such that, under the right
circumstances, it could end up loading a second (non-functional) copy of
the vdso module. This happened when the process plugin did not support
the extended MemoryRegionInfo query (added in D22219, to workaround a
different bug), which meant that the loader plugin was not able to
recognise that the linux-vdso.so.1 module (this is how the loader calls
it) is in fact the same as the [vdso] module (the name used in
/proc/$PID/maps) we loaded before. This typically happened in a core
file, as they don't store this kind of information.
This patch fixes the issue by completing the revert of D10800 -- the
memory loading code is removed completely. It also reduces the scope of
the hackaround introduced in D22219 -- it isn't completely sound and is
only relevant for fairly old (but still supported) versions of android.
I added the memory loading logic to the wasm dynamic loader, which has
since appeared and is relying on this feature (it even has a test). As
far as I can tell loading wasm modules from memory is possible and
reliable. MachO memory loading is not affected by this patch, as it uses
a completely different code path.
Since the scenarios/patches I described came without test cases, I have
created two new gdb-client tests cases for them. They're not
particularly readable, but right now, this is the best way we can
simulate the behavior (bugs) of a particular dynamic linker.
Differential Revision: https://reviews.llvm.org/D122660
This patch handles the situation where the main thread exits (through
the SYS_exit syscall). In this case, the process as a whole continues
running until all of the other threads exit, or one of them issues an
exit_group syscall.
The patch consists of two changes:
- a moderate redesign of the handling of thread exit (WIFEXITED) events.
Previously, we were removing (forgetting) a thread once we received
the WIFEXITED (or WIFSIGNALED) event. This was problematic for the
main thread, since the main thread WIFEXITED event (which is better thought
of as a process-wide event) gets reported only after the entire process
exits. This resulted in deadlocks, where we were waiting for the
process to stop (because we still considered the main thread "live").
This patch changes the logic such that the main thread is removed as
soon as its PTRACE_EVENT_EXIT (the pre-exit) event is received. At
this point we can consider the thread gone (for most purposes). As a
corrolary, I needed to add special logic to catch process-wide exit
events in the cases where we don't have the main thread around.
- The second part of the patch is the removal of the assumptions that
the main thread is always available. This generally meant replacing
the uses of GetThreadByID(process_id) with GetCurrentThread() in
various process-wide operations (such as memory reads).
Differential Revision: https://reviews.llvm.org/D122716
About half of our host platform code was implemented in the Platform
class, while the rest was it RemoteAwarePlatform. Most of the time, this
did not matter, as nearly all our platforms are also
RemoteAwarePlatforms. It makes a difference for PlatformQemu, which
descends directly from the base class (as it is local-only).
This patch moves all host code paths into the base class, and marks
PlatformQemu as a "host" platform so it can make use of them (it sounds
slightly strange, but that is consistent with what the apple simulator
platforms are doing). Not all of the host implementations make sense for
this platform, but it can always override those that don't.
I add some basic tests using the platform file apis to exercise this
functionality.
Differential Revision: https://reviews.llvm.org/D122898
Since the threads/frame view is taking only a small part on the right side
of the screen, only a part of the function name of each frame is visible.
It seems rather wasteful to spell out 'frame' there when it's obvious
that it is a frame, it's better to use the space for more of the function
name.
Differential Revision: https://reviews.llvm.org/D122998
It's rather annoying if it's there after every startup,
and that 'Help (F6)' at the top should be enough to help people
who don't know.
Differential Revision: https://reviews.llvm.org/D122997
All uses of JSONGenerator in debugserver would create a JSON text
dump of the object collection, then copy that string into a
binary-escaped string, then send it up to the lldb side or
make a compressed version and send that.
This adds a DumpBinaryEscaped method to JSONGenerator which
does the gdb remote serial protocol binary escaping directly,
and removes the need to pass over the string and have an
additional copy in memory.
Differential Revision: https://reviews.llvm.org/D122882
rdar://91117456
The ordering is not needed, and DenseMap is faster. I can measure
time spent in the SaveToCache() calls reduced to ~40% during LLDB
startup (and the total startup cost reduced to ~70%).
Differential Revision: https://reviews.llvm.org/D122980
The current design allows that the object file contents could be mapped
by one object file plugin and then used by another. Presumably the idea
here was to avoid mapping the same file twice.
This becomes an issue when one object file plugin wants to map the file
differently from the others. For example, ObjectFileELF needs to map its
memory as writable while others likeObjectFileMachO needs it to be
mapped read-only.
This patch prevents plugins from changing the buffer by passing them is
by value rather than by reference.
Differential revision: https://reviews.llvm.org/D122944
Environments are optional and a missing environment is distinct from
the default "unknown" environment enumerator. The test is negative,
because the function uses the host triple and is unpredictable.
rdar://91007207
https://reviews.llvm.org/D122946
Differential Revision: https://reviews.llvm.org/D122946
This updates the disassembler to enable every optional extension.
Previously we had added things that we added "support" for in lldb.
(where support means significant work like new registers, fault types, etc.)
Something like TME (transactional memory) wasn't added because
there are no new lldb features for it. However we should still be
disassembling the instructions.
So I went through the AArch64 extensions and added all the missing
ones. The new test won't prevent us missing a new extension but it
does at least document our current settings.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D121999
It is the PC line, selected or not, that gets the blue-background
highlight. Without this, a keyword like 'bool' got black background
if the line wasn't selected.
And the blue-background highlight is handled by OutputColoredStringTruncated(),
so no point in setting it explicitly in the calling code.
A problem that I introduced in the decoder is that I was considering TSC decoding
errors as actual instruction errors, which mean that the trace has a gap. This is
wrong because a TSC decoding error doesn't mean that there's a gap in the trace.
Instead, now I'm just counting how many of these errors happened and I'm using
the `dump info` command to check for this number.
Besides that, I refactored the decoder a little bit to make it simpler, more
readable, and to handle TSCs in a cleaner way.
Differential Revision: https://reviews.llvm.org/D122867
Update the Linux and NetBSD Host libraries for 2165c36be4 which
removed DataBufferLLVM::GetChars. These files are compiled conditionally
based on the host platform.
I found this function somewhat hard to read and removed a few entirely
redundant checks and converted it to early exits.
Differential Revision: https://reviews.llvm.org/D122912
This creates inline functions decls in the TUs where the funcitons are inlined and local variable decls inside those functions.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D121967
Storing timestamps (TSCs) in a more efficient map at the decoded thread level to speed up TSC lookup, as well as reduce the amount of memory used by each decoded instruction. Also introduced TSC range which keeps the current timestamp valid for all subsequent instructions until the next timestamp is emitted.
Differential Revision: https://reviews.llvm.org/D122603
Many callers of SendPacket() in RNBRemote.cpp have a local std::string
object, call c_str() on it to pass a c-string, which is then copied into
a std::string temporary object.
Also free JSONGenerator objects once we've formatted them into
ostringstream and don't need the objects any longer, to reduce max
memory use in debugserver.
Differential Revision: https://reviews.llvm.org/D122848
rdar://91117263
Protecting against accidental overwriting of commands is good, but
having to pass a flag to overwrite the command when developing your
commands is pretty annoying. This adds a setting to defeat the protection
so you can do this once at the start of your session and not have to
worry about it again.
Differential Revision: https://reviews.llvm.org/D122680
Applied modernize-use-equals-default clang-tidy check over LLDB.
This check is already present in the lldb/.clang-tidy config.
Differential Revision: https://reviews.llvm.org/D121844
NSIndexPathSyntheticFrontEnd::Impl::Clear() currently calls Clear() on both
unions members regardless of which one is active. I modified it to only call
Clear() on the active member.
Differential Revision: https://reviews.llvm.org/D122753
Currently, when creating a target for a fat binary, we error out if more
than one platforms can support the different architectures in the
binary. There are situations where it makes sense for multiple platforms
to support the same architectures: for example the host and
remote-macosx platform on Darwin.
The only way to currently disambiguate between them is to specify the
architecture. This patch changes that to take into account the selected
and host platform. The new algorithm works a follows:
1. Pick the selected platform if it matches any of the architectures.
2. Pick the host platform if it matches any of the architectures.
3. If there's one platform that works for all architectures, pick that.
If none of the above apply then we either have no platform supporting
the architectures in the fat binary or multiple platforms with no good
way to disambiguate between them.
I've added a bunch of unit tests to codify this new behavior.
rdar://90360204
Differential revision: https://reviews.llvm.org/D122684
This recommits dddf4ce03, which was reverted because of a couple of test
failures on macos. The reason behind the failures was that the patch
inadvertenly changed the value returned by the host platform from
"macosx" to "darwin". The new version fixes that.
Original commit message was:
The decision which categories are relevant for a particular test run
happen very early in the test setup process. They use the SBPlatform
object to determine which categories should be skipped. The platform
object created for this purpose transcends individual test runs.
This setup is not compatible with the direction discussed in
<https://discourse.llvm.org/t/multiple-platforms-with-the-same-name/59594>
-- when platform objects are tied to a specific (SB)Debugger, they need
to be created alongside it, which currently happens in the test setUp
method.
This patch is the first step in that direction -- it rewrites the
category skipping logic to avoid depending on a global SBPlatform
object. Fortunately, the skipping logic is fairly simple (and I believe
it outght to stay that way) and mainly consists of comparing the
platform name against some hardcoded lists. This patch bases this
comparison on the platform name instead of the os part of the triple (as
reported by the platform).
Differential Revision: https://reviews.llvm.org/D121605
lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp:472:16: warning: enumeration value 'HLSL' not handled in switch [-Wswitch]
switch (IK.getLanguage()) {
^
Some signal handlers were set up within an !_MSC_VER condition,
i.e. omitted in MSVC builds but included in mingw builds. Previously
sigtstp_handler was defined in all builds, but since
4bcadd6686 / D120320 it's only
defined non platforms other than Windows.
Change the condition to !_WIN32 for consistency between the MSVC
and mingw builds, fixing the build for mingw.
Differential Revision: https://reviews.llvm.org/D122486
Now the decoded thread has Append methods that provide more flexibility
in terms of the underlying data structure that represents the
instructions. In this case, we are able to represent the sporadic errors
as map and thus reduce the size of each instruction.
Differential Revision: https://reviews.llvm.org/D122293
With the shared cache getting split into multiple files, the current
way we created ObjectFileMachO objects for shared cache dylib images
will break.
This patch conditionally adopts new SPIs which will do the right
thing in the new world of multi-file caches.
With Scripted Processes, in order to create scripted threads, the blueprint
provides a dictionary that have each thread index as the key with the respective
thread instance as the pair value.
In Python, this is fine because a dictionary key can be of any type including
integer types:
```
>>> {1: "one", 2: "two", 10: "ten"}
{1: 'one', 2: 'two', 10: 'ten'}
```
However, when the python dictionary gets bridged to C++ we convert it to a
`StructuredData::Dictionary` that uses a `std::map<ConstString, ObjectSP>`
for storage.
Because `std::map` is an ordered container and ours uses the `ConstString`
type for keys, the thread indices gets converted to strings which makes the
dictionary sorted alphabetically, instead of numerically.
If the ScriptedProcess has 10 threads or more, it causes thread “10”
(and higher) to be after thread “1”, but before thread “2”.
In order to solve this, this sorts the thread info dictionary keys
numerically, before iterating over them to create ScriptedThreads.
rdar://90327854
Differential Revision: https://reviews.llvm.org/D122429
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch changes `StructuredData::Dictionary::GetKeys` return type
from an `StructuredData::ObjectSP` to a `StructuredData::ArraySP`.
The function already stored the keys in an array but implicitely upcasted
it to an `ObjectSP`, which required the user to convert it again to a
Array object to access each element.
Since we know the keys should be held by an iterable container, it makes
more sense to return the allocated ArraySP as-is.
Differential Revision: https://reviews.llvm.org/D122426
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Previously, the ScriptedThread used the thread index as the thread id.
This patch parses the crashlog json to extract the actual thread "id" value,
and passes this information to the Crashlog ScriptedProcess blueprint,
to create a higher fidelity ScriptedThreaad.
It also updates the blueprint to show the thread name and thread queue.
Finally, this patch updates the interactive crashlog test to reflect
these changes.
rdar://90327854
Differential Revision: https://reviews.llvm.org/D122422
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Fixes "Cannot specify link libraries for target "lldb-target-fuzzer"
which is not built by this project." Normally that's taken care of by
add_llvm_fuzzer but we need target_link_libraries for liblldb and our
utility library.
This patch adds a generic fuzzer that interprets inputs as object files
and uses them to create a target in lldb. It is very similar to the
llvm-dwarfdump fuzzer which found a bunch of issues in libObject.
Differential revision: https://reviews.llvm.org/D122461
When iterating over all Platforms looking for the best one, on a Mac the
Simulator platforms (iOS, tvOS, watchOS) will first find their SDK
directory by calling xcrun, then decide if they should activate or not.
When that SDK is absent, the call to xcrun to find it can be very slow.
This patch delays that directory search until we know we're activating
this platform, so non-simulator environments don't pay a perf cost ever
time they go through the list of platforms.
Differential Revision: https://reviews.llvm.org/D122373
rdar://87960090
Update the response schema of the TraceGetState packet and add
Intel PT specific response structure that contains the TSC conversion,
if it exists. The IntelPTCollector loads the TSC conversion and caches
it to prevent unnecessary calls to perf_event_open. Move the TSC conversion
calculation from Perf.h to TraceIntelPTGDBRemotePackets.h to remove
dependency on Linux specific headers.
Differential Revision: https://reviews.llvm.org/D122246
This patch adds a test for b0dc2fae60. That commit fixed a bug where
we could increment the indirect symbol offset every time we parsed the
symbol table.
In ProcessMachCore::DoLoadCore when we have a standalone
binary and a 'main bin spec' LC_NOTE detailing the UUID and
load address, ProcessMachCore will do a (potentially slow)
lookup to try to find the binary and/or dSYM. For kernel and
userland corefile using 'main bin spec', we would follow the
normal schemes of locating them. DynamicLoaderDarwinKernel would
use the same (possibly expensive) calls to find the correct
binary. dyld by default would use the in-core-file binary image,
and so if the corefile didn't include the entire address space,
the LINKEDIT for dyld could be missing. This means we can't find
the dyld4::dyld_all_image_infos struct, which tells us where the
other binaries are loaded in memory.
Treat userland 'bin main spec' like we do standalone firmewares;
try the expensive checks to find the best dyld we can, before
falling back to using a memory module out of the corefile.
Also add a little TODO for myself in this load_standalone_binary
function that we should handle the case of a binary in the shared
cache correctly, creating a memory module in the corefile and
using the segment load addresses from that to set our segment
load addresses for the final binary.
rdar://89717101
The current code increment the indirect symbol offset with the LINKEDIT
slide every time ObjectFileMachO::ParseSymtab is called.
This resulted in a crash when calling add-dsym which causes us to
potentially re-parse the original binary's symbol table. There's a
separate question about whether we should re-parse the symbol table at
all which was fixed by D114288. Regardless, copying the load command is
cheap enough that this is still the right thing to do.
rdar://72337717
Differential revision: https://reviews.llvm.org/D122349
The assertion checks that the command output doesn't contain any null
bytes. I'm not sure if the intention was to make sure the string wasn't
shorter than the reported length or if this was a way to catch us
accidentally writing an (unformatted) null byte.
The consensus is that we don't want to have embedded nulls in the
command output, but that this isn't the right place to enforce that.
Differential revision: https://reviews.llvm.org/D122025
This patch introduces 2 new lldb utility functions:
- lldbutil.start_listening_from: This can be called in the test setup to
create a listener and set it up for a specific event mask and add it
to the user-provided broadcaster's list.
- lldbutil.fetch_next_event: This will use fetch a single event from the
provided istener and return it if it matches the provided broadcaster.
The motivation behind this is to easily test new kinds of events
(i.e. Swift type-system progress events). However, this patch also
updates `TestProgressReporting.py` and `TestDiagnosticReporting.py`
to make use of these new helper functions.
Differential Revision: https://reviews.llvm.org/D122193
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Avoid "TERM environment variable not set" by either propagating the TERM
environment variable or defaulting to vt100. All of our CI is already
doing this explicitly through the --env dotest arg, but it's easy to
forget when setting up a new job. I don't see any downside in making it
the default.
By default these timeouts are extremely small (0.1s). This means that
100ms after sending an EOF, pexpect will start sending the process
increasingly aggressive signals, but the small timeouts mean that (on a
loaded machine) the kernel may not have enough time to process the
signal even if the overall effect of the signal is to kill the
application.
It turns out we were already relying on this signals (instead of regular
EOF quits) in our tests. In my experiments it was sufficient to block
SIGINT and SIGHUP to cause some test to become flaky. This was most
likely the reason of a couple of flakes on the lldb-x86_64-debian bot,
and is probably the reason why the pexpect tests are flaky on several
other (e.g. asan) bots.
This patch increses the timeout to 6 seconds (60-fold increase), which
is hopefully sufficient to avoid flakes even in the most extreme
situations.
There's a bug caused when a process is relaunched: the target, which
doesn't change, keeps the Trace object from the previous process, which
is already defunct, and causes segmentation faults when it's attempted
to be used.
A fix is to clean up the Trace object when the target is disposing of
the previous process during relaunches.
A way to reproduce this:
```
lldb a.out
b main
r
process trace start
c
r
process trace start
```
Differential Revision: https://reviews.llvm.org/D122176
I incorrectly returned an ArrayRef when the underlying object didn't own
the data. Instead, returning a vector<uint8_t> is what we should do.
This fixes an issue when trying to access an intel-pt trace buffer
larger than 16 MB.
repro
```
go to a breakpoint
thread trace start -s 16777216
n
thread trace dump instructions # this doesn't fail anymore
```
Differential Revision: https://reviews.llvm.org/D122192
Fix the log and progress report message for in-memory binaries. If
there's no object file, use the name from the Module. With this patch we
correctly show the library name when attaching to a remote process
without an expanded shared cache.
Differential revision: https://reviews.llvm.org/D122177
Fangrui Song