A few fixes while trying to figure out why tests are being skipped for arsenm:
- We check `$compiler -v`, but `-v` is `--verbose`, not `--version`. Use the long flag name.
- We check all lines matching `version ...`, but we should exit early for the first version string we see (which should be the main one). I'm not sure if this is the issue, but perhaps this is causing some users to skip some tests if another "version ..." is showing up later.
- Having `\.` in a python string is triggering pylint warnings, because it should be escaped as a regex string, e.g. `r'\.' However, `.` in a character class does not need to be escaped, as it matches only a literal `.` in that context.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D88051
Register the `faulthandler` module so we can see what lldb tests are doing when they misbehave (e.g. run under a test runner that sets a timeout). This will print a stack trace for the following signals:
- `SIGSEGV`, `SIGFPE`, `SIGABRT`, `SIGBUS`, and `SIGILL` (via `faulthandler.enable()`)
- `SIGTERM` (via `faulthandler.register(SIGTERM)`) [This is what our test runners sends when it times out].
The only signal we currently handle is `SIGINT` (via `unittest2.signals.installHandler()`) so there should be no overlap added by this patch.
Because this import is not available until python3, and the `register()` method is not available on Windows, this is enabled defensively.
This should have absolutely no effect when tests are passing (or even normally failing), but can be observed by running this while ninja is running:
```
kill -s SIGTERM $(ps aux | grep dotest.py | head -1 | awk '{print $2}')
```
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D87637
targetHasSVE helper function was added to test for availability of SVE support
by connected platform. We now intend to use this function in other testcases
and I am moving it to a generic location in lldbtest.py to allow usage by
other upcoming testcases.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86872
This updates the errors reported by expect()
to something like:
```
Ran command:
"help"
Got output:
Debugger commands:
<...>
Expecting start string: "Debugger commands:" (was found)
Expecting end string: "foo" (was not found)
```
(see added tests for more examples)
This shows the user exactly what was run,
what checks passed and which failed. Along with
whether that check was supposed to pass.
(including what regex patterns matched)
These lines are also output to the test
trace file, whether the test passes or not.
Note that expect() will still fail at the first failed
check, in line with previous behaviour.
Also I have flipped the wording of the assert
message functions (.*_MSG) to describe failures
not successes. This makes more sense as they are
only shown on assert failures.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86792
TestCompletion is randomly failing on some bots. The error message however states
that the computed completions actually do contain the expected pid we're
looking for, so there shouldn't be any test failure.
The reason for that turns out to be that complete_from_to is actually used
for testing two different features. It can be used for testing what the
common prefix for the list of completions is and *also* for checking all the
possible completions that are returned for a command. Which one of the two
things should be checked can't be defined by a parameter to the function, but
is instead guessed by the test method instead based on the results that were
returned. If there is a common prefix in all completions, then that prefix
is searched and otherwise all completions are searched.
For TestCompletion's pid test this behaviour leads to the strange test failures.
If all the pid's that our test LLDB can see have a common prefix (e.g., it
can only see pids [123, 122, 10004, 10000] -> common prefix '1'), then
complete_from_to check that the common prefix contains our pid, which is
always fails ('1' doesn't contain '123' or any other valid pid). If there
isn't a common prefix (e.g., pids are [123, 122, 10004, 777]) then
complete_from_to will check the list of completions instead which works correctly.
This patch is fixing this by adding a simple check method that doesn't
have this behaviour and is simply searching the returned list of completions.
This should get the bots green while I'm working on a proper fix that fixes
complete_from_to.
This patch removes the rather confusing LLDB_LIB_DIR and LLDB_IMPLIB_DIR
environment variables. They are confusing because LLDB_LIB_DIR would
point to the bin subdirectory in the build root while LLDB_IMPLIB_DIR
would point to the lib subdirectory. The reason far this was
LLDB.framework, which gets build under bin.
This patch replaces their uses with configuration.lldb_framework_path
and configuration.lldb_libs_dir respectively.
Differential revision: https://reviews.llvm.org/D86817
Annotating `PExpectTest` with `@skipIfWindows` instead of marking it as an empty class will make the test runner recognize it as a test class, which should allow me to reland adb5c23f8c.
I don't have a windows machine to verify this works, but I did some tests using `@skipIfLinux` and they all worked as expected. In case the `pexpect` import is not at all available on windows, I moved it to within the method where it's used.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D86745
EXP_MSG generates a message to show on assert
failure. Currently it looks like:
AssertionError: False is not True : '<cmd>'
returns expected result, got '<actual output>'
Which seems to say that the test failed but
also got the expected result.
It should say:
AssertionError: False is not True : '<cmd>'
returned unexpected result, got '<actual output>'
Reviewed By: teemperor, #lldb
Differential Revision: https://reviews.llvm.org/D86603
The logic behind --rerun-all-issues was removed when we switched to LIT
as the test driver. This patch just removes the dotest option and
corresponding entry in configuration.py.
Instead of a new method for each variable any subclass might want to
set, have a method getExtraMakeArgs that each subclass can use to return
whatever extra Make arguments it wants.
As per Pavel's suggestion in D85539.
Rather than have different modules for different platforms, use
inheritance so we can have a Builer base class and optional child
classes that override platform specific methods.
Differential revision: https://reviews.llvm.org/D86174
TypeSystemClang::CreateTypedef was creating a typedef in the right
DeclContext, but it was not actually adding it as a child of the
context. The resulting inconsistent state meant that we would be unable
to reference the typedef from an expression directly, but we could use
them if they end up being pulled in by some previous subexpression
(because the ASTImporter will set up the correct links in the expression
ast).
This patch adds the typedef to the decl context it is created in.
Differential Revision: https://reviews.llvm.org/D86140
Rename the existing expectedFailure to expectedFailureIfFn to better
describe its purpose and provide an overload for
unittest2.expectedFailure in decorators.py.
Right now the only places in the SB API where lldb:: ModuleSP instances are
destroyed are in SBDebugger::MemoryPressureDetected (where it's just attempted
but not guaranteed) and in SBDebugger::DeleteTarget (which will be removed in
D83933). Tests that directly create an lldb::ModuleSP and never create a target
therefore currently leak lldb::Module instances. This triggers the sanity checks
in lldbtest that make sure that the global module list is empty after a test.
This patch adds SBModule::GarbageCollectAllocatedModules as an explicit way to
clean orphaned lldb::ModuleSP instances. Also we now start calling this method
at the end of each test run and move the sanity check behind that call to make
this work. This way even tests that don't create targets can pass the sanity
check.
This fixes TestUnicodeSymbols.py when D83865 is applied (which makes that the
sanity checks actually fail the test).
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D83876
Right now if the test suite encounters a cleanup error it just prints "CLEANUP
ERROR:" but not any additional information.
This patch just prints the exception that caused the cleanup error. This should
make debugging the failing tests for D83865 easier (and seems in general nice to
have).
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D83874
In sanitized builds the last packet this function finds for the
TestMacCatalyst and TestPlatformSimulator tests is for the asan runtime.
```
< 69> send packet: $jGetLoadedDynamicLibrariesInfos:{"solib_addresses":[4296048640]}]#3a <
715> read packet: ${"images":[{"load_address":4296048640,"mod_date":0,"pathname":
"/Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/host-compiler/lib/clang/12.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib",
"uuid":"8E38A2CD-753F-3E0F-8EB0-F4BD5788A5CA",
"min_version_os_name":"macosx","min_version_os_sdk":"10.9",
"mach_header":{"magic":4277009103,"cputype":16777223,"cpusubtype":3,"filetype":6,
"flags":43090053}],"segments":[{"name":"__TEXT","vmaddr":0,"vmsize":565248,"fileoff":0,
"filesize":565248,"maxprot":5}],{"name":"__DATA","vmaddr":565248,"vmsize":13152256,"fileoff":565248,
"filesize":20480,"maxprot":3}],{"name":"__LINKEDIT","vmaddr":13717504,"vmsize":438272,"fileoff":585728,
"filesize":435008,"maxprot":1}]]}]]}]#00
```
This just fetches the last package which has fetch_all_solibs and we know
it will contain the image of our test executable to get the tests running again.
This is relanding D81001. The patch originally failed as on newer editline
versions it seems CC_REFRESH will move the cursor to the start of the line via
\r and then back to the original position. On older editline versions like
the one used by default on macOS, CC_REFRESH doesn't move the cursor at all.
As the patch changed the way we handle tab completion (previously we did
REDISPLAY but now we're doing CC_REFRESH), this caused a few completion tests
to receive this unexpected cursor movement in the output stream.
This patch updates those tests to also accept output that contains the specific
cursor movement commands (\r and then \x1b[XC). lldbpexpect.py received an
utility method for generating the cursor movement escape sequence.
Original summary:
I implemented autosuggestion if there is one possible suggestion.
I set the keybinds for every character. When a character is typed, Editline::TypedCharacter is called.
Then, autosuggestion part is displayed in gray, and you can actually input by typing C-k.
Editline::Autosuggest is a function for finding completion, and it is like Editline::TabCommand now, but I will add more features to it.
Testing does not work well in my environment, so I can't confirm that it goes well, sorry. I am dealing with it now.
Reviewed By: teemperor, JDevlieghere, #lldb
Differential Revision: https://reviews.llvm.org/D81001
Some of the test methods were already skipped because of an unexpected
packet. The test started failing after it was expanded. Skip the whole
test with reproducers so we don't have to add the decorator for every
method.
expect_expr currently can't verify the children of the result SBValue.
This patch adds the ability to check them. The idea is to have a CheckValue
class where one can specify what attributes of a SBValue should be checked.
Beside the properties we already check for (summary, type, etc.) this also
has a list of children which is again just a list of CheckValue object (which
can also have children of their own).
The main motivation is to make checking the children no longer based
on error-prone substring checks that allow tests to pass just because
for example the error message contains the expected substrings by accident.
I also expect that we can just have a variant of `expect_expr` for LLDB's
expression paths (aka 'frame var') feature.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D83792
We've been seeing this failure on green dragon when the system is
under high load. Unfortunately this is outside of LLDB's control.
Differential Revision: https://reviews.llvm.org/D85542
This patch stores the --apple-sdk argument in the dotest configuration.
When it's set, use it instead of the triple to determine the current
platform.
Differential revision: https://reviews.llvm.org/D85537
In order to be able to run the debugserver tests against the Rosetta
debugserver, detect the Rosetta run configuration and return the
system Rosetta debugserver.
Currently, the skipIfRosetta decorator will skip tests with the message
"not on macOS" on all platforms that are not `darwin` or `macosx`.
Instead, it should only check the platform and architecture when running
on these platforms.
This triggers for example when running the test suite on device.
Differential revision: https://reviews.llvm.org/D85388
This patch modifies the skipIfRemote decorator so it can apply to a
whole class, which allows us to skip all PExpect tests as a whole.
Differential revision: https://reviews.llvm.org/D85365
`getCompilerVersion` assumes that `clang --version` prints out a string like `version [0-9\.]+`.
If clang is built from trunk, the version line might look like `clang version trunk (123abc)`.
Since there isn't any way of knowing by the commit id alone whether one commit is newer or older than another git commit (or clang version), assume that clang with a version id like this is very close to trunk. For example, any tests with `@skipIf(compiler="clang", compiler_version=['<', '8'])` should be run.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D85248
LLDB tests assume that tests are in the test tree (the `LLDB_TEST_SRC` env variable, configured by `dotest.py`).
If this assertion doesn't hold, tests fail in strange ways. An early place this goes wrong is in `compute_mydir` which does a simple length-based substring to get the relative path. Later, we use that path to chdir to. If the test file and test tree don't agree in realpath-ness (and therefore length), this will be a cryptic error of chdir-ing to a directory that does not exist.
The actual discrepency is that the places we look for `use_lldb_suite.py` don't use a realpath, but `dotest.py` does (see initialization of `configuration.testdirs`).
It doesn't particularly matter whether we use realpath or abspath to canonicalize things, but many places end up with implicit dependencies on the canonicalized pwd being a realpath, so make them realpath consistently. Also, in the `compute_mydir` method mentioned, raise an error if the path types don't agree.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85258
Add an option that allows the user to decide to not make the inferior is
responsible for its own TCC permissions. If you don't make the inferior
responsible, it inherits the permissions of its parent. The motivation
is the scenario of running the LLDB test suite from an external hard
drive. If the inferior is responsible, every test needs to be granted
access to the external volume. When the permissions are inherited,
approval needs to be granted only once.
Differential revision: https://reviews.llvm.org/D85237
Between the time it was created and it was pushed upstream,
99451b4453 has moved the existing
gui gui tests to lldb/test, so move this one too.
And update it to contain TestGuiBasic.py changes since the time
when it was based on that test.
Differential Revision: https://reviews.llvm.org/D85106
This patch is similar in spirit to https://reviews.llvm.org/D84480,
but does the maccatalyst/macosx disambiguation. I also took the
opportunity to factor out the gdb-remote packet log scanning used by
several testcases into lldbutil functions.
rdar://problem/66059257
Differential Revision: https://reviews.llvm.org/D84576
This is a followup to 817b3a6fe3a4452eb61a2503c8beaa7267ca0351: in `builder_base` we should use abspath, not realpath, because the name is significant.
This is used by test cases that use `@skipIf(compiler="clang", compiler_version=['<', <version>])`
In these two cases, use of `os.path.realpath` is problematic:
- The name of the compiler is significant [1] . For testing purposes, we might
provide a compiler called "clang" which is actually a symlink to some build
script (which does some flag processing before invoking the real clang). The
destination the symlink may not be called "clang", but we still want it to be
treated as such.
- When using a build system that puts build artifacts in an arbitrary build
location, and later creates a symlink for it (e.g. creates a
"<lldb root>/lldbsuite/test/dotest.py" symlinks that points to
"/build/artifact/<hash>/dotest.py"), looking at the realpath will not match
the "test" convention required here.
[1] See `Makefile.rules` in the lldb tree, e.g. we use different flags if the compiler is named "clang"
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85175
Jordan Rupprecht