Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden.
Reviewers: zturner, tfiala
Subscribers: labath, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D25714
llvm-svn: 284893
Summary:
The test suite calls realpath on the lldb executable then append "-mi" to it to find the path of the lldb-mi executable. This does not work when using CMake builds on *nix platforms. On *nix platforms when a version number is set on executables CMake generates the binary as ${name}-${version} with a symlink named ${name} pointing to it.
This results in the lldb executable being named lldb-4.0.0, and since lldb-4.0.0-mi doesn't ever match the lldb-mi executable these tests are always disabled.
This patch looks for lldb-mi in the same directory as lldb.
Reviewers: zturner, tfiala
Subscribers: ki.stfu, enlight, lldb-commits
Differential Revision: https://reviews.llvm.org/D25486
llvm-svn: 284041
Summary:
This patch is necessary because individual test cases are not required
to have unique names. Therefore, test cases must now
be specified explicitly in the form <TestCase>.<TestMethod>.
Because it works by regex matching, passing just <TestCase> will
still disable an entire file.
This also allows for multiple exclusion files to be specified.
Reviewers: zturner, labath, jingham, tfiala
Subscribers: lldb-commits, sas
Differential Revision: https://reviews.llvm.org/D24988
llvm-svn: 283238
Summary:
The current implementation of the test suite allows the user to run
a certain subset of tests using '-p', but does not allow the inverse,
where a user wants to run all but some number of known failing tests.
Implement this functionality.
Reviewers: labath, zturner, tfiala
Subscribers: jingham, sas, lldb-commits
Differential Revision: https://reviews.llvm.org/D24629
llvm-svn: 282298
*** to conform to clang-format’s LLVM style. This kind of mass change has
*** two obvious implications:
Firstly, merging this particular commit into a downstream fork may be a huge
effort. Alternatively, it may be worth merging all changes up to this commit,
performing the same reformatting operation locally, and then discarding the
merge for this particular commit. The commands used to accomplish this
reformatting were as follows (with current working directory as the root of
the repository):
find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} +
find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ;
The version of clang-format used was 3.9.0, and autopep8 was 1.2.4.
Secondly, “blame” style tools will generally point to this commit instead of
a meaningful prior commit. There are alternatives available that will attempt
to look through this change and find the appropriate prior commit. YMMV.
llvm-svn: 280751
The variables referenced in the print message are not defined. Simply state
that the requisite script is not found. Correct grammar to indicate that the
tests are rather likely to fail rather than unlikely to fail.
llvm-svn: 269628
The parameter here is a list, not a string. Ensure that the we splat the list
into arguments prior to invoke os.path.join. This would previously fail with a
`startswith` is not a member of `list`.
llvm-svn: 269627
Summary:
This change enhances the LLDB test infrastructure to convert
load-time exceptions in a given Python test module into errors.
Before this change, specifying a non-existent test decorator,
or otherwise having some load-time error in a python test module,
would not get flagged as an error.
With this change, typos and other load-time errors in a python
test file get converted to errors and reported by the
test runner.
This change also includes test infrastructure tests that include
covering the new work here. I'm going to wait until we have
these infrastructure tests runnable on the main platforms before
I try to work that into all the normal testing workflows.
The test infrastructure tests can be run by using the standard python module testing practice of doing the following:
cd packages/Python/lldbsuite/test_event
python -m unittest discover -s test/src -p 'Test*.py'
Those tests run the dotest inferior with a known broken test and verify that the errors are caught. These tests did not pass until I modified dotest.py to capture them properly.
@zturner, if you have the chance, if you could try those steps above (the python -m unittest ... line) on Windows, that would be great if we can address any python2/3/Windows bits there. I don't think there's anything fancy, but I didn't want to hook it into test flow until I know it works there.
I'll be slowly adding more tests that cover some of the other breakage I've occasionally seen that didn't get collected as part of the summarization. This is the biggest one I'm aware of.
Reviewers: zturner, labath
Subscribers: zturner, lldb-commits
Differential Revision: http://reviews.llvm.org/D20193
llvm-svn: 269489
Summary:
lldb-server tests are currently being skipped on the
check-lldb target. This is because we get the path of
lldb-server by modifying the path to the lldb executable.
However, by this point, we've changed directories, and a
relative path to the build/bin directory will no longer point
to the location of lldb-server.
Storing an absolute path solves this issue.
Reviewers: vharron, zturner, tfiala, labath
Subscribers: labath, lldb-commits, sas
Differential Revision: http://reviews.llvm.org/D19082
llvm-svn: 267463
This change moves all the test event handling and its related
ResultsFormatter classes out of the packages/Python/lldbsuite/test dir
into a packages/Python/lldbsuite/test_event package. Formatters are
moved into a sub-package under that.
I am limiting the scope of this change to just the motion and a few
minor issues caught by a static Python checker (e.g. removing unused
import statements).
This is a pre-step for adding package-level tests to the test event
system. I also intend to simplify test event results formatter selection
after I make sure this doesn't break anybody.
See:
http://reviews.llvm.org/D19288
Reviewed by:
Pavel Labath
llvm-svn: 266885
Win32 API calls that are Unicode aware require wide character
strings, but LLDB uses UTF8 everywhere. This patch does conversions
wherever necessary when passing strings into and out of Win32 API
calls.
Patch by Cameron
Differential Revision: http://reviews.llvm.org/D17107
Reviewed By: zturner, amccarth
llvm-svn: 264074
Nobody seems to know what purpose these files serve, yet they were accumulating by the thousands in the test traces directory. I'm proposing we delete them.
Creating these files accounted for about 2.5% of the time to run ninja check-lldb on my machine, which isn't a lot, but it's something.
llvm-svn: 263122
This is meant to reduce the typing that one needs to do to get from the test subdirectory to actual test cases. Now one can just do
$ ./dotest.py ./testcases/<yaddayaddayadda>
llvm-svn: 255741
Use of --rerun-all-issues will enable any test method failure, not just
test methods marked with the flakey decorator, to rerun.
Currently this does not change the flakey logic's immediate rerun
attempt. I want to make sure this doesn't cause any significant issues
before changing that part.
The rerun reporting is only known to work properly with the
default (new) BasicResultsFormatter reporting. Once we work out
any issues, I'll go back and make sure the curses output handles
it properly as well.
llvm-svn: 255543
Also adds full path info for exceptional exits and timeouts when
no test method is currently running.
Adds --rerun-all-issues command line arg. If specified, all
test issues are eligible for rerun. If not specified, only tests
marked flakey are eligible for rerun.
The actual rerunning will occur in an upcoming change. This
change just handles tha accounting of what should be rerun.
llvm-svn: 255438
The new test summary formatter does not honor the "expected timeout" markings, which makes our
buildbots all red. I'm switching it off by default until we figure out a way to make this work.
llvm-svn: 255335
This was an option to display a graphical progress bar. Nobody
is using this, and it doesn't work correctly anyway with the new
result formatter.
llvm-svn: 255153
This removes the failfast command line option as part of an effort
to simplify dotest and remove unused command line options. You can
still Ctrl+C any time you want to exit early.
llvm-svn: 255150
The results formatter system is now fed timeouts and exceptional process
exits (i.e. inferior dotest.py process that exited by signal on POSIX
systems).
If a timeout or exceptional exit happens while a test method is running
on the worker queue, the timeout or exceptional exit is charged and
reported against that test method. Otherwise, if no test method was
running at the time of the timeout or exceptional exit, only the test
filename will be reported as the TIMEOUT or ERROR.
Implements:
https://llvm.org/bugs/show_bug.cgi?id=24830https://llvm.org/bugs/show_bug.cgi?id=25703
In support of:
https://llvm.org/bugs/show_bug.cgi?id=25450
llvm-svn: 255097
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today. git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.
llvm-svn: 255060
This removes the option to exclude a single directory. This is
part of an effort to remove unused options and cleanup the interface
to the test suite.
llvm-svn: 255048
This moves all the global variables into a separate module called
`configuration`. This has a number of advantages:
1. Configuration data is centrally maintained so it's easy to get
a high level overview of what configuration data the test suite
makes use of.
2. The method of sharing configuration data among different parts
of the test suite becomes standardized. Previously we would
put some things into the `lldb` module, some things into the
`lldbtest_config` module, and some things would not get shared.
Now everything is shared through one module and is available to
the entire test suite.
3. It opens the door to moving some of the initialization code into
the `configuration` module, simplifying the implementation of
`dotest.py`.
There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily. This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable. We can address these in the future.
llvm-svn: 254982
This cleans up dotest.py and is a pre-step for getting
the test inferior runner to send post-inferior run events
to the events collector, as this code needs to be accessed
from within dosep.py.
llvm-svn: 254979
Chris Bieneman