Assign the test method name to self.testMethodName. This can be useful for the
test directory (see test/types for a good example) which houses a bunch of executables
compiled from different source files. The default build action is to create a.out as
the binary executable, which can confuse the module cacheing mechanism and result in
the debugger getting a stale image as the target to be debugged, and chaos ensues.
o AbstractBase.py, TestThreadAPI.py:
Use self.testMethodName to our advantage.
o TestLoadUnload.py:
Add expected failure marker to test case test_modules_search_paths().
llvm-svn: 133768
employ different executable names when creating the target for lldb to debug.
Comparing the log files for the same test case with success/failure results show
that the failed one was setting the breakpoint on some different address than the
succeeded one, leading us to suspect that the file system maybe think it knows better
and caches the wrong executable file (they were all named 'a.out' before) which lldb
subsequently reads from.
Now './dotest.py -v types' passes without specifying the '-w' option.
rdar://problem/9644488
rdar://problem/9649573
llvm-svn: 133649
with no delay between successive test cases. This one happend to capture a test failure where
the inferior shoud stop at a breakpoint but did not (radar will be filed to capture the log files
as well as the test session files):
[12:40:37] johnny:/Volumes/data/lldb/svn/trunk/test $ DEBUG_LLDB_LOG=/tmp/lldb.log DEBUG_GDB_REMOTE_LOG=/tmp/gdb-remote.log ./dotest.py -v -p TestFailures.py
LLDB-63
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 133529
Node Kind: directory
Schedule: normal
Last Changed Author: jmolenda
Last Changed Rev: 133500
Last Changed Date: 2011-06-20 19:57:15 -0700 (Mon, 20 Jun 2011)
Session logs for test failures/errors/unexpected successes will go into directory '2011-06-21-12_40_42'
Command invoked: python ./dotest.py -v -p TestFailures.py
----------------------------------------------------------------------
Collected 10 tests
1: test_char_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that char-type variables are displayed correctly. ... ok
2: test_char_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that char-type variables are displayed correctly. ... ok
3: test_int_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that int-type variables are displayed correctly. ... ok
4: test_int_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that int-type variables are displayed correctly. ... ok
5: test_long_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that 'long long'-type variables are displayed correctly. ... FAIL
6: test_long_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that 'long long'-type variables are displayed correctly. ... ok
7: test_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly. ... TestFailures.DebugIntegerTypesFailures.test_long_type_with_dsym
FAIL
8: test_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly. ... FAIL
9: test_short_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that short-type variables are displayed correctly. ... ok
10: test_short_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that short-type variables are displayed correctly. ... ok
======================================================================
FAIL: test_long_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that 'long long'-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 102, in test_long_long_type_with_dsym
self.long_long_type()
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 129, in long_long_type
self.generic_type_tester(set(['long long']))
File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
"stop reason = breakpoint"])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint
======================================================================
FAIL: test_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 87, in test_long_type_with_dsym
self.long_type()
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 125, in long_type
self.generic_type_tester(set(['long']))
File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
"stop reason = breakpoint"])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint
======================================================================
FAIL: test_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 94, in test_long_type_with_dwarf
self.long_type()
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 125, in long_type
self.generic_type_tester(set(['long']))
File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
"stop reason = breakpoint"])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint
----------------------------------------------------------------------
Ran 10 tests in 5.617s
FAILED (failures=3)
Session logs for test failures/errors/unexpected successes can be found in directory '2011-06-21-12_40_42'
[12:40:47] johnny:/Volumes/data/lldb/svn/trunk/test $
llvm-svn: 133537
if not already specified by the test driver (via ./dotest -w). Remove the AbstractBase.setUp()
method definition when/if we find out the cause of the failures if no delays are inserted
between these test cases.
llvm-svn: 133495
We still have the the issue where running:
./dotest.py -v types
we have test failures where the inferior either runs to exited with status 0 or
the inferior stops but not because of breakpoint (for example):
runCmd: process status
output: Process 90060 stopped
* thread #1: tid = 0x2d03, 0x000000010000e2ca, stop reason = EXC_BAD_ACCESS (code=2, address=0x10000e2ca)
frame #0: 0x000000010000e2ca
There are two cases where the inferior stops for the breakpoint (good), but the expression parser
prints out the wrong information. The two failures are:
Failure-TestFloatTypesExpr.FloatTypesExprTestCase.test_double_type_with_dsym.log
Failure-TestFloatTypesExpr.FloatTypesExprTestCase.test_double_type_with_dwarf.log
I'll file a radar on the two expression parser misbehave, while continue investigating why the
inferior stops for the wrong reason or does not stop at all.
For now, you'll need to do:
./dotest.py -v -w types
llvm-svn: 133488
rdar://problem/9173060 lldb hangs while running unique-types
disappears if running with clang version >= 3. Modify the TestUniqueTypes.py
to detect if we are running with clang version < 3 and, if true, skip the test.
Update the lldbtest.system() function to return a tuple of (stdoutdata, stderrdata)
since we need the stderr data from "clang -v" command. Modify existing clients of
lldbtest.system() to now use, for example:
# First, capture the golden output emitted by the oracle, i.e., the
# series of printf statements.
- go = system("./a.out", sender=self)
+ go = system("./a.out", sender=self)[0]
# This golden list contains a list of (variable, value) pairs extracted
# from the golden output.
gl = []
And add two utility functions to lldbutil.py.
llvm-svn: 128162
Don't overwhelm the expression mechanism. Previously, there was no delay
between successive invocations of 'expression' command. The test suite
run has experienced 'Resume timed out' or similar error messages. Now we
currently add a sleep of 1.0 second before issuing the next 'expression'
command.
llvm-svn: 120626
Also modify the:
self.runCmd("expr %s" % var)
to be:
self.runCmd("expression %s" % var)
since 'expr' is an alias of expression. And fix the assert message to be either
'frame variable -t %s' or 'expression %s'.
llvm-svn: 118085
The failures that remain are:
# rdar://problem/8620735 test/types: frame variable -t a_class_ref.m_b fails
# The reference type related failures that remain are:
notnow = set(['a_class_ref.m_b',
'a_struct_ref.b',
'a_union_nonzero_ref.u.a'])
llvm-svn: 118021
pass in a 'sender' arg to the buildDefault(), buildDsym(), buildDwarf(), and
cleanup() functions. The sender arg will be the test instance itself (i.e.,
an instance of TestBase). This is so that the relevant command execution can be
recorded in the TestBase.session object if sender is available.
The lldbtest.system() command has been modified to pop the 'sender' arg out of
the keyword arguments dictionary and use it as the test instance to facilitate
seesion recordings. An example is in test/types/AbstractBase.py:
def generic_type_tester(self, atoms, quotedDisplay=False):
"""Test that variables with basic types are displayed correctly."""
# First, capture the golden output emitted by the oracle, i.e., the
# series of printf statements.
go = system("./a.out", sender=self)
There are cases when sender is None. This is the case when the @classmethod is
involved in the use of these APIs. When this happens, there is no recording
into a session object, but printing on the sys.stderr is still honored if the
trace flag is ON.
An example is in test/settings/TestSettings.py:
@classmethod
def classCleanup(cls):
system(["/bin/sh", "-c", "rm -f output.txt"])
system(["/bin/sh", "-c", "rm -f stdout.txt"])
llvm-svn: 116648
files to a different top level directory than those specified on the command line.
When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all. This allows for an easier
postmortem analysis of test failures.
Example:
./dotest.py -v -t -r /tmp/lldbtest types
will create a /tmp/lldbtest directory which houses the types directory and its supported
files.
Files modified:
o dotest.py, lldbtest.py:
Add logic to process '-r dir' option to support relocating the tests to a different
top level directory instead of exected in place.
o darwin.py, test/types/Makefile:
The 'make clean' should only clean the minimum .o and .d files.
llvm-svn: 116255
# rdar://problem/8493023
# test/types failures for Test*TypesExpr.py: element offset computed wrong and sign error?
Two failures remain for test_short* test cases.
llvm-svn: 115229
command on the various basic types, similar to TestIntegerTypes.py and
TestFloatTypes.py, which exercise 'frame variable' on the various basic types.
Right now, they don't employ the self.expect() facility to compare against the
golden output. They just invoke the self.runCmd() method to run the 'expr'
command. Decorated the two classes with @unittest2.skip decorators for the time
being.
llvm-svn: 114987
method. Renamed it to be AbstractBase.py, which contains the GenericTester class that
both IntegerTypesTestCase and FloatTypesTestCase inherit from.
llvm-svn: 114926
type tester method into its own abstarct base class 'AbstractBase'. And
added TestIntegerTypes.py and TestFloatTypes.py.
Added an option "-p reg-exp-pattern" to the test driver to specify a regular
expression pattern to match against eligible filenames as our test cases.
An example:
/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -p "TestFloat.*" types
----------------------------------------------------------------------
Collected 4 tests
test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase)
Test that double-type variables are displayed correctly. ... ok
test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase)
Test that double-type variables are displayed correctly. ... ok
test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase)
Test that float-type variables are displayed correctly. ... ok
test_float_types_with_dsym (TestFloatTypes.FloatTypesTestCase)
Test that float-type variables are displayed correctly. ... ok
----------------------------------------------------------------------
Ran 4 tests in 5.592s
OK
llvm-svn: 114923
Added a special "clean" target to the types/Makefile to clean up all the *.o/.d
files.
The generic_type_tester() method is modified to take a set of atoms, instead of
type string as a required parameter, for example:
o unsigned int => set(['unsigned', 'int'])
o unsigned long long => set(['unsigned', 'long long'])
o long long => set(['long long'])
llvm-svn: 114871
Extended generic_type_tester() method to take an additional keyword argument
quoteDisplay (default to False) to facilitate comparison with frame variable
display output of character types.
llvm-svn: 114769
Johnny Chen