Summary:
dotest.py currently requires a filterspec to be of the
form `TestCase.test_method`. This patch makes it more
flexible, so you can pass `TestModule.TestCase.test_method`
or `TestModule.TestCase` or `TestCase.test_method` or just
`test_method`.
This makes it more convenient to just copy a test name
out of the terminal after running a bunch of tests and use
it as a filterspec.
Reviewers: JDevlieghere, jasonmolenda, labath
Reviewed By: JDevlieghere
Subscribers: jingham, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D68545
llvm-svn: 373997
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
It used to be possible to enable logging through environment variables
read by dotest. This approach is deprecated, as stated in the dotest
help output. Instead --channel should be used.
Differential revision: https://reviews.llvm.org/D66920
llvm-svn: 370387
This removes the curses result formatter which appears to be broken.
Passing --curses to dotest.py screws up my terminal and doesn't run any
tests. It even crashes Python on occasion.
Differential revision: https://reviews.llvm.org/D66917
llvm-svn: 370386
This removes support for reading the LLDB_TEST_ARGUMENTS environment
variable and instead requires all arguments to be specified as part of
the invocation. This ensures that dotest.py invocations are easily
repeatable.
Differential revision: https://reviews.llvm.org/D66912
llvm-svn: 370278
This argument was used by dosep.py to pass information around from the
workers. With dosep.py gone, I'm fairly sure we don't need this any
longer.
llvm-svn: 370266
I was looking at the session directory logic for unrelated reasons and
noticed that the logic spread out across dotest. This simplifies things
a bit by moving the logic together.
llvm-svn: 370259
Now that all supported build systems create a valid dotest.py
invocation, we no longer need to guess the location of the lldb binary
and Python directory.
Differential revision: https://reviews.llvm.org/D66896
llvm-svn: 370234
Now that all supported build systems create a valid dotest.py
invocation, we no longer need to guess the directory where any of the
llvm tools live. Additionally, the current logic is incomplete: it
doesn't try to find any other tools than FileCheck, such as dsymutil for
example.
If no FileCheck is provided, we should print a warning and skip the
tests that need it, but that's not part of this patch.
Differential revision: https://reviews.llvm.org/D66893
llvm-svn: 370232
This patch removes the -q (quiet) flag and changing the default
behavior. Currently the flag serves two purposes that are somewhat
contradictory, as illustrated by the difference between the argument
name (quiet) and the configuration flag (parsable). On the one hand it
reduces output, but on the other hand it prints more output, like the
result of individual tests. My proposal is to guard the extra output
behind the verbose flag and always print the individual test results.
Differential revision: https://reviews.llvm.org/D66837
llvm-svn: 370226
The results port was used by dosep.py to deal with test results coming
form different processes. With dosep.py gone, I don't think we need this
any longer.
Differential revision: https://reviews.llvm.org/D66811
llvm-svn: 370090
Today I discovered the skipLongRunningTest decorator and to my surprise
all the tests were passing without the decorator. They don't seem to be
that expensive either, they take a few seconds but we have tests that
take much longer than that. As such I propose to remove the decorator
and enable them by default.
Differential revision: https://reviews.llvm.org/D66774
llvm-svn: 369995
With this patch dotest.py will print the full invocation whenever it
fails to parse its arguments. The dotest invocation is usually build up
with different inputs, potentially coming from CMake, lldb-dotest, lit
or passed directly. This can make debugging hard, especially on CI,
where there might be another layer of indirection. This aims to make
that a bit easier.
llvm-svn: 369922
We cannot override the DWARF version in the CFLAGS_EXTRA because they
are used by tests that explicitly build without debug info. Instead, we
pass them through the regular CFLAGS.
llvm-svn: 369477
On the matrix bot on GreenDragon [1] we want to run the test suite
against different DWARF versions. The idea here is not to replace
targeted tests for certain DWARF features, but rather to provide an easy
way to support this configuration.
[1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/
Differential revision: https://reviews.llvm.org/D66370
llvm-svn: 369272
Summary: Thanks to Hui Huang and reviewers for all the help with this patch!
Reviewers: labath, jfb, clayborg
Reviewed By: labath
Subscribers: Hui, clayborg, dexonsmith, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D61687
llvm-svn: 368776
Now that the Xcode project is removed, I want to focus on dotest as a
test framework, and remove its driver capabilities for which we already
rely on llvm's lit. Removing multiprocessing is the first step in that
direction.
Differential revision: https://reviews.llvm.org/D65311
llvm-svn: 367331
Summary: lldb-mi has been removed, but there are still a bunch of references in the code base. This patch removes all of them.
Reviewers: JDevlieghere, jfb
Reviewed By: JDevlieghere
Subscribers: dexonsmith, ki.stfu, mgorny, abidh, jfb, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D64992
llvm-svn: 366590
This patch adds a dotest flag for setting environment variables for the
inferior. This is different from the current --env flag, which sets
variables in the debugger's environment. This allows us to set things
like LD_LIBRARY_PATH for testing.
Differential revision: https://reviews.llvm.org/D63790
llvm-svn: 364443
Skip watchpoint tests if security.models.extensions.user_set_dbregs
is disabled. This indicates that unprivileged processes are not allowed
to write to debug registers which is a prerequisite for using hardware
watchpoints.
Differential Revision: https://reviews.llvm.org/D63380
llvm-svn: 363536
Summary:
In python 2, strings and bytes are the same, but they're not in
python 3, hence the return of read() needs an explicit conversion.
While I'm around, rename the return of Popen() from `pipe` to
`process`, as that's what Popen returns.
Reviewers: JDevlieghere, friss, zturner, aprantl, serge-sans-paille
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D56517
llvm-svn: 350788
This allows bots which haven't updated to pass in --filecheck to
dotest.py to run more tests. FileCheck-dependent tests will continue to
fail.
Differential Revision: https://reviews.llvm.org/D53175
llvm-svn: 344401
Add a "filecheck" method to the LLDB test base. This allows test authors
to pattern match command output using FileCheck, making it possible to
write stricter tests than what `self.expect` allows.
For context (motivation, examples of stricter checking, etc), see the
lldb-dev thread: "Using FileCheck in lldb inline tests".
Differential Revision: https://reviews.llvm.org/D50751
llvm-svn: 342508
The single-process test runner is invoked in a number of different
scenarios, including when multiple test dirs are specified or (afaict)
when lit is used to drive the test suite.
Unfortunately the --test-subdir option did not work with the single
process test runner, breaking an important use case (using lit to run
swift-lldb Linux tests):
Failure URL: https://ci.swift.org/job/swift-PR-Linux/6841
We won't be able to run lldb tests within swift PR testing without
filtering down the set of tests.
This change makes --test-subdir work with the single-process runner.
llvm-svn: 339929
This patch adds a new lldb-vscode tool that speaks the Microsoft Visual Studio Code debug adaptor protocol. It has full unit tests that test all packets.
This tool can be easily packaged up into a native extension and used with Visual Studio Code, and it can also be used by Nuclide
Differential Revision: https://reviews.llvm.org/D50365
llvm-svn: 339911
On systems where it's not supported.
As far as I understand Linux is the only systems which now ships
with libstdcxx (maybe NetBSD?, but I'm not entirely sure of the
state of lldb on the platform).
We could make this more fine grained looking for the header as
we do for libcxx. This is a little tricky as there's no such
thing as /usr/include/c++/v1, but libstdcxx encodes the version
number in the path (i.e. /usr/include/c++/5.4). I guess we might
match a regex, but it seems fragile to me.
Differential Revision: https://reviews.llvm.org/D49110
llvm-svn: 336724
Summary:
1) In TestLinuxCore rather than skipping the tests on Windows, mark them as expected failures and add a bug reference
2) In dotest.py replace the undefined property in the exceptions with the actual property causing the exception
Reviewers: asmith, labath, zturner
Reviewed By: labath, zturner
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D46590
llvm-svn: 331886
Summary:
In the magic test duplicator, we were making the decision whether to
create a test variant based on the compiler and the target platform.
This meant that the set of known tests was different for each test
configuration.
This patch makes the set of generated test variants static and handles
the skipping via runtime checks instead. This is more consistent with
how we do other test-skipping decision (e.g. for libc++ tests), and
makes it easier to expose the full set of tests to lit, which now does
not need to know anything about what things can potentially cause tests
to appear or disappear.
Reviewers: JDevlieghere, aprantl
Subscribers: eraman, lldb-commits
Differential Revision: https://reviews.llvm.org/D45949
llvm-svn: 330708
Summary:
With the upstream implementation of dsymutil containing almost all
functionality from the one shipped with Xcode, we want to use the
in-tree version for running the test suite.
This will also allow us to re-enable TestUnicodeSymbols which was
failing because of the discrepancy in how Unicode symbols were hashed in
lldb and older versions of dsymutil.
Reviewers: aprantl, davide, jingham, labath
Subscribers: mgorny, llvm-commits, lldb-commits
Differential Revision: https://reviews.llvm.org/D45518
llvm-svn: 329889
The test "test_fp_special_purpose_register_read" in TestRegisters.py
fails on Darwin machines configured to use an out-of-tree debugserver.
The error message is: 'register read ftag' returns expected result, got
'ftag = 0x80'. This indicates that the debugserver in use is too old.
This commit introduces a decorator which can be used to skip tests which
rely on having a just-built debugserver. This resolves the issue:
$ ./bin/llvm-dotest -p TestRegisters.py -v
1 out of 617 test suites processed - TestRegisters.py
Test Methods: 7
Success: 6
Skip: 1
...
llvm-svn: 327052
LLDB queries Spotlight to locate .dSYM bundles based on the UUID
embedded in a binary, and because the UUID is a hash of filename and
.text section, there *will* be conflicts inside $BUILD_DIR.
This should fix the broken green dragon bots.
llvm-svn: 324019
This patch is the result of a discussion on lldb-dev, see
http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for
background.
For each test (should be eventually: each test configuration) a
separate build directory is created and we execute
make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test
In order to make this work all LLDB tests need to be updated to find
the executable in the test build directory, since CWD still points at
the test's source directory, which is a requirement for unittest2.
Although we have done extensive testing, I'm expecting that this first
attempt will break a few bots. Please DO NOT HESITATE TO REVERT this
patch in order to get the bots green again. We will likely have to
iterate on this some more.
Differential Revision: https://reviews.llvm.org/D42281
llvm-svn: 323803
Now the function returns `False`, and the caller can take the
appropriate action (in this case, for --executable, reporting
that the file doesn't exist).
llvm-svn: 323566
Summary:
This used to be important when all tests were run in a single process,
but that has no longer been the case for a while. Furthermore, this hook fails
to build on new mac versions for several people, and it's not clear
whether fixing it is worth the effort.
Reviewers: jingham, clayborg, davide
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D41871
llvm-svn: 322167
The commit breaks the case where you specify just a filename to the
compiler. Previously, it would look up the compiler in your path, now it
complains that the compiler is not found. One of the lldb buildbots is
depending on this. It seems like a nice feature to have, as it means
less typing and being able to avoid hard-coding the system compiler path
in the bot config.
This reverts commit r316393.
llvm-svn: 316451
Lawrence D'Anna