directory used to dump the session info for test failures/errors.
Example:
/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -s jason -v array_types
Session info for test errors or failures will go into directory jason
----------------------------------------------------------------------
Collected 4 tests
test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dsym_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dwarf_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... FAIL
======================================================================
FAIL: test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 27, in test_with_dwarf_and_run_command
self.array_types()
File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 62, in array_types
'stop reason = breakpoint'])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 594, in expect
self.runCmd(str, trace = (True if trace else False), check = not error)
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 564, in runCmd
msg if msg else CMD_MSG(cmd, True))
AssertionError: False is not True : Command 'thread list' returns successfully
----------------------------------------------------------------------
Ran 4 tests in 3.086s
FAILED (failures=1)
/Volumes/data/lldb/svn/trunk/test $ ls jason
TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
/Volumes/data/lldb/svn/trunk/test $ head -10 jason/TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
Session info generated @ Thu Oct 21 09:54:15 2010
os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=NO']]
stdout: rm -rf "a.out" "a.out.dSYM" main.o main.d
cc -arch x86_64 -gdwarf-2 -O0 -c -o main.o main.c
cc -arch x86_64 -gdwarf-2 -O0 main.o -o "a.out"
stderr: None
retcode: 0
/Volumes/data/lldb/svn/trunk/test $
llvm-svn: 117028
parent call frame information. And comment out the check for stop reason for the
time being. The stop reason disappeared from the "thread backtrace" output for
breakpoint stop with condition.
llvm-svn: 116939
to be run during tearDown() to effect the restore action instead of executing it inline
during the test method, because the test may already fail and bailout before the inline
restore action.
Fix test_set_output_path() and pass_run_args_and_env_vars() to use this mechanism.
llvm-svn: 116881
now goes into a timestamp-specific directory instead of the previous .session-*
files.
[17:24:34] johnny:/Volumes/data/lldb/svn/trunk $ ls -l test/2010-10-18-16:56:12.935342
total 48
-rw-r--r-- 1 johnny admin 1695 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dsym_and_run_command.log
-rw-r--r-- 1 johnny admin 1652 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r-- 1 johnny admin 2967 Oct 18 16:56 TestBreakpointCommand.BreakpointCommandTestCase.test_with_dsym.log
-rw-r--r-- 1 johnny admin 1648 Oct 18 16:56 TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r-- 1 johnny admin 1665 Oct 18 16:56 TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase.test_with_dsym_and_python_api.log
-rw-r--r-- 1 johnny admin 3873 Oct 18 16:58 TestFloatTypesExpr.FloatTypesTestCase.test_float_types_with_dsym.log
[17:24:37] johnny:/Volumes/data/lldb/svn/trunk $
Also, the dumping happens when a test errored in additioned to when it failed.
llvm-svn: 116778
pass in a 'sender' arg to the buildDefault(), buildDsym(), buildDwarf(), and
cleanup() functions. The sender arg will be the test instance itself (i.e.,
an instance of TestBase). This is so that the relevant command execution can be
recorded in the TestBase.session object if sender is available.
The lldbtest.system() command has been modified to pop the 'sender' arg out of
the keyword arguments dictionary and use it as the test instance to facilitate
seesion recordings. An example is in test/types/AbstractBase.py:
def generic_type_tester(self, atoms, quotedDisplay=False):
"""Test that variables with basic types are displayed correctly."""
# First, capture the golden output emitted by the oracle, i.e., the
# series of printf statements.
go = system("./a.out", sender=self)
There are cases when sender is None. This is the case when the @classmethod is
involved in the use of these APIs. When this happens, there is no recording
into a session object, but printing on the sys.stderr is still honored if the
trace flag is ON.
An example is in test/settings/TestSettings.py:
@classmethod
def classCleanup(cls):
system(["/bin/sh", "-c", "rm -f output.txt"])
system(["/bin/sh", "-c", "rm -f stdout.txt"])
llvm-svn: 116648
breakpoint modify -c 'val == 3' 1
after:
breakpoint set -n c
which sets a breakpoint on function 'c'. The breakpoint should only stop if
expression 'val == 3' evaluates to true.
llvm-svn: 116607
session info after a test case failure, allowing more direct inspection of
debugger session which leads to the test failure.
For a simple usage scenario:
[18:06:26] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log
...
[18:14:43] johnny:/Volumes/data/lldb/svn/trunk/test $ ls -l .session-*
-rw-r--r-- 1 johnny admin 1359 Oct 14 18:06 .session-TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command
-rw-r--r-- 1 johnny admin 2054 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dsym_and_expr_parser
-rw-r--r-- 1 johnny admin 2055 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser
-rw-r--r-- 1 johnny admin 1351 Oct 14 17:57 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command
[18:14:51] johnny:/Volumes/data/lldb/svn/trunk/test $
The test case which failed will have its recorded session info dumped to a
.session-* file in the current working directory. For test suite using
relocated directory, expect to find the .session-* files there.
In this checkin, I also add @skip decorator to the two test methods in
test/foundation/TestObjCMethods.py as it looks like the test suite is
deadlocking when running the tests. More investigations are needed.
llvm-svn: 116552
and 'process handle'. The test suite would like to control the asynch/sync
execution of the interpreter during the middle of the test method, so the
CommandInterpreter::SetSynchronous(bool value) is modified to allow the mode to
be changed more than once.
In practice, it would be advisable to control the process and to set the
async/sync mode from a single thread, too.
llvm-svn: 116467
within a file. This is to be used within the test case to avoid hardcoded line number.
array_types/TestArrayTypes.py is modified first to use this pattern. Other test modules
to follow.
rdar://problem/8537816
Testsuite: don't set breakpoints by exact file & line number
llvm-svn: 116270
files to a different top level directory than those specified on the command line.
When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all. This allows for an easier
postmortem analysis of test failures.
Example:
./dotest.py -v -t -r /tmp/lldbtest types
will create a /tmp/lldbtest directory which houses the types directory and its supported
files.
Files modified:
o dotest.py, lldbtest.py:
Add logic to process '-r dir' option to support relocating the tests to a different
top level directory instead of exected in place.
o darwin.py, test/types/Makefile:
The 'make clean' should only clean the minimum .o and .d files.
llvm-svn: 116255
Update the expected match string.
o lldbtest.py:
Indicate when a command fails, even if there is nothing in the error stream.
o TestHelp.py:
Add a regression test case for 'help image dump symtab'.
o CommandObjectHelp.cpp:
Some of the logic branches with successful help command results were not tagged
with a Success Status. They are fixed now. This is important for Python
interaction.
llvm-svn: 116062
Temporarily commenting out the deprecated LaunchProcess() method.
SWIG is not able to handle the overloaded functions.
o dotest.py/lldbtest.py:
Add an '-w' option to insert some wait time between consecutive test cases.
o TestClassTypes.py:
Make the breakpoint_creation_by_filespec_python() test method more robust and
more descriptive by printing out a more insightful assert message.
o lldb.swig: Coaches swig to treat StateType as an int type, instead of a C++ class.
llvm-svn: 115899
To not skip long running tests, pass '-l' to the test driver (dotest.py).
An example:
@unittest2.skipIf(TestBase.skipLongRunningTest(), "Skip this long running test")
def test_foundation_disasm(self):
...
o Added a long running disassemble test to the foundation directory, which iterates
the code symbols from Foundation.framework and kicks off a disassemble command for
for the named function symbol. Found a crasher: rdar://problem/8504895.
o Plus added/updated some comments for the TestBase class.
llvm-svn: 115368
testing various combinations of displaying variales of basic types.
The generic_type_tester() works by first capturing the golden output produced
by the printf stmts of ./a.out, creating a list of (var, value) pairs, and then
running the a.out to a stop point, and comparing the 'frame variable var' output
against the list of (var, value) pairs.
Modified int_type() and added long_type() to use generic_type_tester().
Also modified TestBase.expect() such that substring matching also return ok if
the substring starts at the 0-th position.
llvm-svn: 114708
actually test-and-compare anything yet. The lldbtest.TestBase has an added
method setTearDownCleanup(dictionary=None) to facilitate running the cleanup
right after each data type test is run. The test case can pass a dictionary
object when registering the test case cleanup.
There is currently only int_type test in the repository.
llvm-svn: 114600
themselves. Right now, it tests a breakpoint both before and after it has been
resolved.
Updated lldbtest.TestBase.expect() with an additional keyword argument 'exe' (
default to True), which if set to False, will treat the mandatory first argument
as just the string to be matched/or not-matched against the golden input.
llvm-svn: 114501
of 'breakpoint command add/list/remove' commands to set breakpoint callbacks,
list them, and then remove one.
Modified the lldbtest.TestBase.expect() method to add two additional keyword
arguments:
o matching (default to True), which, if set to False, reverses the semantics of
'expect' to 'expect not'
o patterns (default to None), which specifies a list of regexp patterns to match
against the output from running the command
TestBreakpointCommand.py uses the matching=False and the patterns=[...] expect()
API.
llvm-svn: 114480
to evaluate expressions. Marked with @expectedFailure decorators for the time
being.
Enhanced the lldbtest.TestBase.expect() API to allow an additional keyword arg
named "error". If the client passes "error=True", it signifies that an error
while running the command is expected. The golden input is then compared
against the return object's error output.
llvm-svn: 114228
This will remove the confusion experienced when previous test runs left some
files (both intermediate or by-product as a result of the test).
lldbtest.TestBase defines a classmethod tearDownClass(cls) which invokes the
platform-specific cleanup() function as defined by the plugin; after that, it
invokes a subclass-specific function classCleanup(cls) if defined; and, finally,
it restores the old working directory.
An example of classCleanup(cls) is in settings/TestSettings.py:
@classmethod
def classCleanup(cls):
system(["/bin/sh", "-c", "rm output.txt"])
where it deletes the by-product "output.txt" as a result of running a.out.
llvm-svn: 114058
as it now passes. Added some extra tests to breakpoint_creation_by_filespec_python().
More clarification for the "os command" output and error as defined in
lldbtest.system() function.
Cleaned up the option processing of the test driver (dotest.py) and fixed the comment
about enabling gdb-remote logging. Example:
$ GDB_REMOTE_LOG=/tmp/log.txt ./dotest.py -v -t enum_types
llvm-svn: 113868
the binaries.
If the build* function is passed the compiler argument, for example, 'llvm-gcc',
it is passed as a make variable to the make command. Otherwise, we check the
LLDB_CC environment variable; if it is defined, it is passed as a make variable
to the make command.
If neither the compiler keyword argument nor the LLDB_CC environment variable is
specified, no CC make variable is passed to the make command. The Makefile gets
to define the default CC being used.
--------------------------------------------------------------------------------
Example usage follows:
o Via the keyword argument:
class ArrayTypesTestCase(TestBase):
mydir = "array_types"
@unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
def test_with_dsym_and_run_command(self):
"""Test 'frame variable var_name' on some variables with array types."""
self.buildDsym(compiler='llvm-gcc')
self.array_types()
...
o Via LLDB_CC environment variable:
$ LLDB_CC=llvm-gcc ./dotest.py -v -t array_types
----------------------------------------------------------------------
Collected 4 tests
test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ...
os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=YES CC=llvm-gcc']]
output: rm -rf "a.out" "a.out.dSYM" main.o main.d
llvm-gcc -arch x86_64 -gdwarf-2 -O0 -arch x86_64 -gdwarf-2 -O0 -c -o main.o main.c
llvm-gcc -arch x86_64 -gdwarf-2 -O0 main.o -o "a.out"
/usr/bin/dsymutil -o "a.out.dSYM" "a.out"
...
llvm-svn: 113781
execution context only when the process is still alive. When running the test
suite, the debugger is launching and killing processes constantly.
This might be the cause of the test hang as reported in rdar://problem/8377854,
where the debugger was looping infinitely trying to update a supposedly stale
thread list.
llvm-svn: 113022
method where they belong. Also fixed a logic error in maintaining the command
interface flag (runStarted) indicating whether the lldb "run"/"process launch"
command has been issued. It was erroneously cleared.
Modified the test cases to take advantage of the refactoring.
llvm-svn: 112863
Johnny Chen