Summary:
Log files produced by dotest have names derived from the test name, and this produces errors in
case we have multiple tests with the same name. Additionally, it's good if the test name explains
what the test is testing (which it clearly doesn't do well if there are multiple tests with
identical names). This commit makes the presence of such tests a hard error.
Reviewers: tberghammer, zturner
Subscribers: iancottrell, lldb-commits
Differential Revision: http://reviews.llvm.org/D13588
llvm-svn: 249828
We want to skip these tests when the target platform is darwin, not
when the host because they have to be enabled in case of darwin ->
android
llvm-svn: 249825
dwo symbol files are generated when code compiled with the "-gsplit-dwarf"
command option (https://gcc.gnu.org/wiki/DebugFission). This CL modifies
the test system to run tests with inferiors compile with the "-gsplit-dwarf"
Differential revision: http://reviews.llvm.org/D13300
llvm-svn: 249530
* --results-formatter-options renamed to --results-formatter-option,
with short version of -O
* Multiple --results-formatter-option=OPTION can be specified. The
comma-separating mechanism has been removed.
* XunitFormatter options modified: -n and -r are now short forms of
--ignore-skip-name and --ignore-skip-reason. Those long option
names were tweaked lightly. They also can be specified multiple
times on the command line. The comma-separating, multiple-pattern-
per-option mechanism has been removed.
One can now specify:
dotest.py --results-file stdout -O-ndsym -O-nlldb-mi
for example, to ignore reporting skips for dsym-related or lldb-mi-related
tests in the xUnit report.
llvm-svn: 248384
The parallel test runner now sends the terminate event to the formatter
(if there is one) after the parallel test runs but before dumping anything
to stdout/stderr at the end of the run. This allows the existing
stdout/stderr summary reporting to co-exist nicely with a formatter like
the test_results.Curses that otherwise clobbers the screen.
llvm-svn: 248228
The test events had worker indexes coming across as strings. I
want them to be ints. worker_index now comes across as an int in
the dicationary.
The optional type can be specified with:
--event-add-entries key=val[:type][,key2=val2[:type2]...]
The type piece may be 'int' at this time. That is all. Otherwise
it will be a string.
llvm-svn: 248066
This patch fixes the following case:
```
$ ./dotest.py --executable=~/p/llvm/build_ninja/bin/lldb tools/lldb-mi/
'~/p/llvm/build_ninja/bin/lldb' is not a path to a valid executable
Traceback (most recent call last):
File "./dotest.py", line 1306, in <module>
setupSysPath()
File "./dotest.py", line 1004, in setupSysPath
if not lldbtest_config.lldbExec:
AttributeError: 'module' object has no attribute 'lldbExec'
```
And with this fix:
```
$ ./dotest.py --executable=~/p/llvm/build_ninja/bin/lldb tools/lldb-mi/
'~/p/llvm/build_ninja/bin/lldb' is not a path to a valid executable
The 'lldb' executable cannot be located. Some of the tests may not be run as a result.
```
llvm-svn: 247256
See https://llvm.org/bugs/show_bug.cgi?id=24708 for details.
Using '-f FILTER' or unnamed arguments (directories) to dotest.py
will now force no-multiprocessing mode. This stops a bombardment
of test output spam when following the instructions we provide
in a test session trace for rerunning the test.
llvm-svn: 247101
Summary: This fixes the issue of multiple test suites opening the same file for writing.
Reviewers: clayborg, zturner
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D10284
llvm-svn: 239188
dotest will select clang-3.5 by default and fall back on clang/gcc
dotest will look for the lldb executable in the typical cmake
output locations:
{lldb}/../../../build/bin (next to llvm directory)
{lldb}/../../../build/host/bin (next to llvm directory)
{lldb}/../build/bin (next to lldb directory)
{lldb}/../build/host/bin (next to lldb directory)
llvm-svn: 237632
The lldb executable was referenced through the code by 7 different
(effectively) global variables.
global lldbExecutablePath
global lldbExecutable
os.environ['LLDB_EXEC']
os.environ['LLDB_TEST']
dotest.lldbExec
dotest.lldbHere
lldbtest.lldbExec
This change uses one global variable lldbtest_config.lldbExec to
replace them all.
Differential Revision: http://reviews.llvm.org/D9817
llvm-svn: 237600
lock imports fcntl, which doesn't exist on Windows. If we need
to use this class on Windows, we will need to implement something
based on the CreateMutex API.
llvm-svn: 237292
Summary:
"import crashinfo" was probably failing because multiple dotest invocations
are all trying to compile crashinfo.so at the same time.
I've put a mutex around the compile step to prevent this.
Reviewers: clayborg, chying
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D9732
llvm-svn: 237230
Test Plan: run tests with/without python debugger
Tested on OSX & Linux with PyCharm
Reviewers: chying, clayborg
Differential Revision: http://reviews.llvm.org/D9593
llvm-svn: 236957
Summary:
New dotest options that allow arbitrary log channels and
categories to be enabled. Also enables logging for locally run
debug servers.
Log messages are separated into separate files per test case.
(this makes it possible to log in dosep runs)
These new log files are stored side-by-side with trace files in the
session directory.
These files are deleted by default if the test run is successful.
If --log-success is specified, even successful logs are retained.
--log-success is useful for creating reference log files.
Test Plan:
add '--channel "lldb all" --channel "gdb-remote packets" --log-success'
to your dotest options
Tested on OSX and Linux
Differential Revision: http://reviews.llvm.org/D9594
llvm-svn: 236956
The decorators to skip if or unless certain platforms relied on calling
skipTest, which only worked if they were decorating individual test cases.
However for decorating entire classes it needs different behavior. This behavior
is already encapsulated in unittest2.skipIf and unittest2.skipUnless so this
modifies the platform decorators to use these unittest skip decorators.
Test Plan:
./dotest.py -t -p TestObjCMethods2.py
See skipped tests on linux, and see tests run on macosx.
Differential Revision: http://reviews.llvm.org/D8903
llvm-svn: 234514
We try to run lldb -P and jump through a bunch of other hoops
to figure out where the Python module is so that we can add its
location to sys.path. The error message printed when this
doesn't work was left over from the time when Mac was the only
platform, so it presented a Mac-specific diagnostic on all
platforms related to the LLDB Framework. This patch prints
a more useful diagnostic when not on Mac that gives a number
of other diagnostic messages to check.
llvm-svn: 234049
Most expected OS failures or skipped tests are about the target platform on
which the test binary is being run, not the host platform launching the tests.
This changes expectedFailureOS and skipIfPlatform to check against the remote
platform when running remote tests.
Test Plan:
Run ./do_sep.py test suite against a remote target on a different platform and
verify that tests which should be excluded on the remote platform are excluded.
Differential Revision: http://reviews.llvm.org/D8611
llvm-svn: 233311
Summary:
This patch allows not specify search path in each lldb-mi test. It makes tests easier.
This fix was requested by vharron.
All test pass on OS X.
Reviewers: vharron, clayborg
Subscribers: lldb-commits, vharron
Differential Revision: http://reviews.llvm.org/D8207
llvm-svn: 232019
Setting environment variables are required when testing on a remote
target requiring different flags then the ones specified on the host OS.
Differential revision: http://reviews.llvm.org/D7854
llvm-svn: 230485
If stdout is not a terminal Python executes
rl_variable_bind ("enable-meta-key", "off");
This produces a warning with FreeBSD's libedit because the
enable-meta-key variable is unknown. Not an issue on Apple because
cpython commit f0ab6f9f0603 added a #ifndef __APPLE__ around the call.
See http://bugs.python.org/issue19884 for more information.
For now we just discard the warning output to get the tests working
again on FreeBSD.
llvm-svn: 226037
Summary: This will ease llgs development a bit by not requiring an lldb/lldb.py build to launch the tests. Also, we can now use LLDB_DEBUGSERVER_PATH to point to a debug server to use to run the tests. I used that to point to a ds2 build and run llgs tests against ds2.
Reviewers: clayborg, tfiala, vharron
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D6554
llvm-svn: 225549
Change the default of prefer-dynamic-value to eDynamicDontRunTarget (i.e. enable dynamic values, but do not run code to do so)
Of course, disable this for the test suite, since testing no-dynamic-values is actually valuable
Fixes rdar://17363061
llvm-svn: 225486
When running the test suite on Windows, we can't have Windows popping
up dialogs when LLDB crashes in native code because it will hang
the test suite. This patch silences those dialogs by checking an
environment variable at startup and configuring Windows based on
its value.
This patch also adds an environment variable to force inferiors to
never spawn in their own console window. This is useful to prevent
new window spawm when running the test suite.
Reviewed by: Scott Graham
Differential Revision: http://reviews.llvm.org/D6628
llvm-svn: 224137
Summary: If lldb is not built, dotest.py throws an exception because we are using an unset variable.
Reviewers: clayborg
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D6516
llvm-svn: 223446
Enrico Granata