The per-PSB packet decoding logic was wrong because it was assuming that pt_insn_get_sync_offset was being udpated after every PSB. Silly me, that is not true. It returns the offset of the PSB packet after invoking pt_insn_sync_forward regardless of how many PSBs are visited later. Instead, I'm now following the approach described in https://github.com/intel/libipt/blob/master/doc/howto_libipt.md#parallel-decode for parallel decoding, which is basically what we need.
A nasty error that happened because of this is that when we had two PSBs (A and B), the following was happening
1. PSB A was processed all the way up to the end of the trace, which includes PSB B.
2. PSB B was then processed until the end of the trace.
The instructions emitted by step 2. were also emitted as part of step 1. so our trace had duplicated chunks. This problem becomes worse when you many PSBs.
As part of making sure this diff is correct, I added some other features that are very useful.
- Added a "synchronization point" event to the TraceCursor, so we can inspect when PSBs are emitted.
- Removed the single-thread decoder. Now the per-cpu decoder and single-thread decoder use the same code paths.
- Use the query decoder to fetch PSBs and timestamps. It turns out that the pt_insn_sync_forward of the instruction decoder can move past several PSBs (this means that we could skip some TSCs). On the other hand, the pt_query_sync_forward method doesn't skip PSBs, so we can get more accurate sync events and timing information.
- Turned LibiptDecoder into PSBBlockDecoder, which decodes single PSB blocks. It is the fundamental processing unit for decoding.
- Added many comments, asserts and improved error handling for clarity.
- Improved DecodeSystemWideTraceForThread so that a TSC is emitted always before a cpu change event. This was a bug that was annoying me before.
- SplitTraceInContinuousExecutions and FindLowestTSCInTrace are now using the query decoder, which can identify precisely each PSB along with their TSCs.
- Added an "only-events" option to the trace dumper to inspect only events.
I did extensive testing and I think we should have an in-house testing CI. The LLVM buildbots are not capable of supporting testing post-mortem traces of hundreds of megabytes. I'll leave that for later, but at least for now the current tests were able to catch most of the issues I encountered when doing this task.
A sample output of a program that I was single stepping is the following. You can see that only one PSB is emitted even though stepping happened!
```
thread #1: tid = 3578223
0: (event) trace synchronization point [offset = 0x0xef0]
a.out`main + 20 at main.cpp:29:20
1: 0x0000000000402479 leaq -0x1210(%rbp), %rax
2: (event) software disabled tracing
3: 0x0000000000402480 movq %rax, %rdi
4: (event) software disabled tracing
5: (event) software disabled tracing
6: 0x0000000000402483 callq 0x403bd4 ; std::vector<int, std::allocator<int>>::vector at stl_vector.h:391:7
7: (event) software disabled tracing
a.out`std::vector<int, std::allocator<int>>::vector() at stl_vector.h:391:7
8: 0x0000000000403bd4 pushq %rbp
9: (event) software disabled tracing
10: 0x0000000000403bd5 movq %rsp, %rbp
11: (event) software disabled tracing
```
This is another trace of a long program with a few PSBs.
```
(lldb) thread trace dump instructions -E -f thread #1: tid = 3603082
0: (event) trace synchronization point [offset = 0x0x80]
47417: (event) software disabled tracing
129231: (event) trace synchronization point [offset = 0x0x800]
146747: (event) software disabled tracing
246076: (event) software disabled tracing
259068: (event) trace synchronization point [offset = 0x0xf78]
259276: (event) software disabled tracing
259278: (event) software disabled tracing
no more data
```
Differential Revision: https://reviews.llvm.org/D131630
When targeting macOS Ventura, ld64 will use authenticated fixups for
x86_64 as well as arm64 (where that has always been the case). This
results in test failures when using an Xcode 14 toolchain on an Intel
mac running macOS Ventura:
Failed Tests (3):
lldb-api :: commands/target/basic/TestTargetCommand.py
lldb-api :: lang/c/global_variables/TestGlobalVariables.py
lldb-api :: lang/cpp/char8_t/TestCxxChar8_t.py
Rather than trying to come up with a sophisticated decorator based off
the deployment target, I marked them all as skipped with a comment
explaining why.
Differential revision: https://reviews.llvm.org/D131741
This patch should just a crash caused by a null pointer dereferencing
when dumping a type. It makes sure that the pointer is valid.
rdar://97455134
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch improve exception reporting when loading a crash report in a
scripted process. Now, we parse the `exception` dictionary from the
crash report use it the create a higher fidelity `MachException` stop info.
This patch also updates the test to reflect that change.
rdar://97096486
Differential Revision: https://reviews.llvm.org/D131086
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch removes the system library names and mangled symbol from
the expected output for the interactive crashlog tests.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch parses CrashLog exception data from the raw
text format and adapts it to the new JSON format.
This is necessary for feature parity between the 2 formats.
Differential Revision: https://reviews.llvm.org/D131719
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
A missing call to `format` resulted in curly braces getting printed in
the reason a test was modified by a decorator. For example it would
print "{} unconditionally" instead of "skipping unconditionally" for
tests that were marked as such.
A spiritual follow up to D131032. I noticed some regex could be simplified.
This does some of the following:
1. Removes unused capture groups
2. Uses non-capturing `(?:...)` groups where grouping is needed but capturing isn't
3. Removes trailing `.*`
4. Uses `\d` over `[0-9]`
5. Uses raw strings
6. Uses `{N,}` to indicate N-or-more
Also improves the call site of a `re.findall`.
Differential Revision: https://reviews.llvm.org/D131305
Pavel Labath taught me that clang-format sorts headers automatically
using llvm's rules, and it's better not to have spaces between
So in this diff I'm removing those spaces and formatting them as well.
I used `clang-format -i` to format these files.
Fixed an inconsistency between D130985 and D130342
This should be a follow-up of D130985
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D131667
https://reviews.llvm.org/D131658 found a bug in
ReadPseudoRegisterValue which would mean we read out
of bounds if the s register number was high enough.
This adds a memory check to vpush-1-thumb, which
should have been doing that anyway. Then copies that
test and uses the last 4 s registers instead.
Without the mentioned fix we see random values in
the final memory, with the fix it passes.
Reviewed By: fixathon
Differential Revision: https://reviews.llvm.org/D131663
Functionally broken code for reading and writing registers, likely due to typos,
and could cause out-of-bounds memory access.
Differential Revision: https://reviews.llvm.org/D131658
Added:
- Take RISC-V `ebreak` instruction as breakpoint trap code, so our breakpoint works as expected now.
Further work:
- RISC-V does not support hardware single stepping yet. A software implementation may come in future PR.
- Add support for RVC extension (the trap code, etc.).
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D131566
This patch is based on the minimal extract of D128250.
What is implemented:
- Use the same register layout as Linux kernel and mock read/write for `x0` register (the always zero register).
- Refactor some duplicate code, and delete unused register definitions.
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D130342
This fixes the stand-alone build configuration where LLVM_MAIN_SRC_DIR
does not exist.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D124314
This patch introduces a new option to the crashlog command to get the
the script version.
Since `crashlog.py` is not actually versioned, this returns lldb's
version instead.
rdar://98392669
Differential Revision: https://reviews.llvm.org/D131542
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Static code inspection guided fixes for the following issues:
- dead code
- buffer not null-terminated
- null-dereference
- out-of-bounds access
Differential Revision: https://reviews.llvm.org/D131554
Patch adds support for fadd, fsub, fdiv, fmul and fcmp to IR interpreter.
~~~
OS Laboratory. Huawei RRI. Saint-Petersburg
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D126359
We don't want the DEFINE_ macros or the array registers
being clang formatted.
RegisterInfos_arm64.h was missing the off annotation and
RegisterInfos_arm64_sve.h needed the off moving to before
the macro definitions.
1438639a2f removed a test
that was using undefined behaviour setting a non-typed enum
to a value outside its known range.
That test also checked if we formatted the value properly
when it could contain >1 valid enum value.
I don't think there's anything special about how we format
typed vs non-typed enums so I'm adding a test for ScopedEnum
that will expect to see 2 enum values plus extra.
Reviewed By: labath, Michael137, shafik
Differential Revision: https://reviews.llvm.org/D131472
This patch changes the CrashLogParser class to be both the base class
and a Factory for the JSONCrashLogParser & TextCrashLogParser.
That should help remove some code duplication and ensure both class
have a parse method.
Differential Revision: https://reviews.llvm.org/D131085
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Sometimes, it can happen that a crash report has null images in its list
of used binaries. This manifests like such:
```
0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
```
When fetching debug symbols to symbolicate the crashlog stackframe,
having null images causes `dsymForUUID` to hang for few seconds.
This patch addresses that by skipping null images from being load by the
scripted process.
rdar://97419487
Differential Revision: https://reviews.llvm.org/D131038
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new option for the interactive crashlog mode,
that will prevent it from dumping the `process status` & `thread backtrace`
output to the debugger console.
This is necessary when lldb in running from an IDE, to prevent flooding
the console with information that should be already present in the UI.
rdar://96813296
Differential Revision: https://reviews.llvm.org/D131036
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In can happen when creating stackshot crash report that that key is missing.
Moreover, we try to parse that key but don't use it, or need it, since we
fetch images and symbolicate the stackframes using the binaries UUIDs.
This is why this patch removes everything that is related to the
`process_path`/`procPath` parsing.
rdar://95054188
Differential Revision: https://reviews.llvm.org/D131033
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch updates the regular expression matching stackframes in
crashlog to allow addresses that are 7 characters long and more (vs. 8
characters previously).
It changes the `0x[0-9a-fA-F]{7}[0-9a-fA-F]+` by `0x[0-9a-fA-F]{7,}`.
rdar://97684839
Differential Revision: https://reviews.llvm.org/D131032
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch allows the crashlog script to surface its errors to lldb by
using the provided SBCommandReturnObject argument.
rdar://95048193
Differential Revision: https://reviews.llvm.org/D129614
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new flag for the interactive crashlog mode, that
allow the user to specify, which target to use to create the scripted
process.
This can be very useful when lldb already have few targets created:
Instead of taking the first one (zeroth index), we will use that flag to
create a new target. If the user didn't provide a target path, we will rely
on the symbolicator to create a targer.If that fails and there are already
some targets loaded in lldb, we use the first one.
rdar://94682869
Differential Revision: https://reviews.llvm.org/D129611
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch should fix the interactive crashlog test by checking in the
binary as a yaml to regeneate the binary with the addresses and offsets
when running the test.
rdar://93655633
Differential Revision: https://reviews.llvm.org/D129603
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
bot. This happens because they are building a long result into
a Python string, and the asan checker is making that very slow.
The last two tests here are both slow, but the 'test_command' is the
really slow one. I'm going to start disabling just that one and see
if that gets the ASAN bots clean.
This removes some error-prone repetition in
FormatManager::GetPossibleMatches, where the same three boolean flags
are passed in a row multiple times as arguments to recursive calls to
GetPossibleMatches.
Instead of:
```
// same flags, but with did_strip_typedef set to true.
GetPossibleMatches(..., did_strip_ptr, did_strip_ref, true);
```
we can now say
```
GetPossibleMatches(..., current_flags.WithStrippedTypedef());
```
which hopefully makes the intent clearer, and more readable in case we
add another flag.
Reviewed by: DavidSpickett, labath
Differential Revision: https://reviews.llvm.org/D131459
- Reduce indentation
- Extract caching of the DbgShellCommand and the dsymForUUID executable
(or equivalent)
- Check the DBGShellCommands before falling back to
/usr/local/bin/dsymForUUID
- Don't check ~rc/bin/dsymForUUID
- Improve error reporting
- Don't cache the value of LLDB_APPLE_DSYMFORUUID_EXECUTABLE
Differential revision: https://reviews.llvm.org/D131303
Setting an enum without a fixed underlying type to a value which is outside the
value range is undefined behavior.
The initializer needs to be a constant expression and therefore this was always
ill-formed we just were not diagnosing it before.
See D130058 and D131307 for more details.
Differential Revision: https://reviews.llvm.org/D131460
Currently a FileSpecList::FindFileIndex(...) will only match the specified FileSpec if:
- it has filename and directory and both match exactly
- if has a filename only and any filename in the list matches
Because of this, we modify our breakpoint resolving so it can handle relative paths by doing some extra code that removes the directory from the FileSpec when searching if the path is relative.
This patch is intended to fix breakpoints so they work as users expect them to by adding the following features:
- allow matches to relative paths in the file list to match as long as the relative path is at the end of the specified path at valid directory delimiters
- allow matches to paths to match if the specified path is relative and shorter than the file paths in the list
This allows us to remove the extra logic from BreakpointResolverFileLine.cpp that added support for setting breakpoints with relative paths.
This means we can still set breakpoints with relative paths when the debug info contains full paths. We add the ability to set breakpoints with full paths when the debug info contains relative paths.
Debug info contains "./a/b/c/main.cpp", the following will set breakpoints successfully:
- /build/a/b/c/main.cpp
- a/b/c/main.cpp
- b/c/main.cpp
- c/main.cpp
- main.cpp
- ./c/main.cpp
- ./a/b/c/main.cpp
- ./b/c/main.cpp
- ./main.cpp
This also ensures that we won't match partial directory names, if a relative path is in the list or is used for the match, things must match at the directory level.
The breakpoint resolving code will now use the new FileSpecList::FindCompatibleIndex(...) function to allow this fuzzy matching to work for breakpoints.
Differential Revision: https://reviews.llvm.org/D130401
@clayborg found a potential race condition when setting a static
variable. The fix seems simply to use call_once.
All relevant tests pass.
Differential Revision: https://reviews.llvm.org/D131081
The header from 62e0681afb does something with
LLVM_FALLTHROUGH. Now that llvm-project has switched to C++17 and
LLVM_FALLTHROUGH uses have been migrated to [[fallthrough]], the header is
unneeded.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D131422
Walter Erquinigo