Commit Graph

269 Commits

Author SHA1 Message Date
Johnny Chen 025d1b8f95 For SBTarget.Launch()/LaunchProcess(), there's no need to pass an empty string
as the args and the envs to the launched process.

o lldbtest.py:

  Forgot to check in some assertion messages changes for lldbtest.py.

o dotest.py:

  Also add "api" category to the default lldb log option list.

llvm-svn: 121220
2010-12-08 01:25:21 +00:00
Johnny Chen 729c339a1c Add the '-F' option to the test driver, which makes the running of test suite failfast.
As soon as an error or a failure is encountered, it stops the test suite.

llvm-svn: 120837
2010-12-03 19:59:35 +00:00
Johnny Chen eb3411dc41 Log just the thread name, not process/thread id for easier human reading.
llvm-svn: 120751
2010-12-02 23:31:02 +00:00
Johnny Chen 7528af2f86 When logging, enable process and thread ID recordings. Also add 'state' to the lldb
log-categories, so it now becomes "event process expr state".

llvm-svn: 120745
2010-12-02 22:47:52 +00:00
Johnny Chen 20d0b2198b For lldb loggings, prepend log lines with a timestamp. And change the default log options:
o lldb => event process expr
o gdb-remote => packets process

llvm-svn: 120725
2010-12-02 18:35:13 +00:00
Johnny Chen 75739045d3 Add a '-b blacklistFile' option to the test driver to take a file specifying the
test classes or test cases to be excludued from the test suite.

Check in an example blacklist file: blacklist.py:

"""
'blacklist' is a Python dictionary, it stores the mapping of a string describing
either a testclass or a testcase, i.e, testclass.testmethod, to the reason (a
string) it is blacklisted.

Following is an example which states that test class IntegerTypesExprTestCase
should be skipped because 'This test class crashed' and the test case
FoundationTestCase.test_data_type_and_expr_with_dsym should be skipped because
it is 'Temporarily disabled'.

blacklist = {'IntegerTypesExprTestCase': 'This test class crashed',
             'FoundationTestCase.test_data_type_and_expr_with_dsym': 'Temporarily disabled'
             }
"""

blacklist = {}

An example of invoking the test driver and specifying a blacklist file:

./dotest.py -b blacklist.py -v types

This runs the tests under 'types' directory but excludes the tests specified in
balcklist.py.

llvm-svn: 120620
2010-12-01 22:47:54 +00:00
Johnny Chen bbda8ed39a Add comment.
llvm-svn: 120295
2010-11-29 17:52:43 +00:00
Johnny Chen 46389e9ddf When looping through the test suite more than once (repeat count > 1), we should
ignore and not enforce the signleton pattern for the LLDBTestResult class.

llvm-svn: 120294
2010-11-29 17:50:10 +00:00
Johnny Chen ecb072d840 Add an option '-# count' to run the specified test suite for a said number of times.
This is not to be used during normal test suite run, but to be used to stress test
specific test sequences repeatedly.

Example:

./dotest.py -# 3 -v breakpoint_conditions

will repeat the test suite 3 times for tests under the breakpoint_conditions directory.

llvm-svn: 119399
2010-11-16 22:42:58 +00:00
Johnny Chen 5f98696f51 Add help message for the scenario of using lldb.py module distributed by Xcode4
to invoke the test driver.

llvm-svn: 118847
2010-11-11 22:14:56 +00:00
Johnny Chen cc78b0b67f Add more comments on LLDBTestResult class.
llvm-svn: 118655
2010-11-09 23:56:14 +00:00
Johnny Chen bb4f8e6f7b Rephrase the output message about session information; make it both "session logs".
llvm-svn: 118650
2010-11-09 23:42:00 +00:00
Johnny Chen 27f391be61 Better wording of the help text for the '-f testclass.testmethod' option usage.
llvm-svn: 118438
2010-11-08 20:17:04 +00:00
Johnny Chen c06f422d73 Let's also emit the informational message for where to find the session logs for
failed/errored tests at the start of the test run, in case the test run crashes
for any reason.  That way, it is easy to locate the session logs for accumulated
failures/errors.

llvm-svn: 118427
2010-11-08 19:03:37 +00:00
Johnny Chen fa491601b5 Change the test driver so that simply specifying '-f testclass.testmethod' means that
we want to run just the instance of testclass-testmethod combination and nothing else.

Specifying '-g' now will admit the whole .py test module if it does not contain a matching
testclass-testmethod combination at all.

This option arrangement adheres to the RISC principle of making the common cases fast. :-)

rdar://problem/8584914 Can the default for dotest.py's "-g" flag be switched?

llvm-svn: 118392
2010-11-08 01:21:03 +00:00
Johnny Chen 9a6706bca4 Add comment about iterating over possible architecture and compiler combinations.
llvm-svn: 118279
2010-11-05 17:30:53 +00:00
Johnny Chen 110bd0f8cf Now that the test suite has very few expected failures remaining (2), it is a good
idea to also dump the session information for expected failures in addition to
failures and errors (tests with unexpected exceptions).

Modify the LLDBTestResult class which derives from unittest2.TextTestResult to
intercept the addExpectedFailure() method in order to mark the test instance as
needing its session information dumped.

llvm-svn: 118185
2010-11-03 18:17:03 +00:00
Johnny Chen 721f298a60 Add some comment about the default session dir name which is a timestamp.
llvm-svn: 117730
2010-10-29 22:26:38 +00:00
Johnny Chen 8ff5d56c20 The informational message for where to find the session logs for failed/errored
tests should be near the test summary message, instead of way up in the beginning.

llvm-svn: 117726
2010-10-29 22:20:36 +00:00
Johnny Chen bdc5c27894 Fix the test driver as part of the fix for http://llvm.org/bugs/show_bug.cgi?id=8493
File name "2010-10-19-14:10:49.059609" is non-portable.

Do not use ':' in the pathname as it is not permitted on windows platforms.

llvm-svn: 117556
2010-10-28 16:32:13 +00:00
Johnny Chen 8c251f4209 Add some more usage text.
llvm-svn: 117129
2010-10-22 19:00:18 +00:00
Johnny Chen c14e08f437 Rewording of output message.
llvm-svn: 117030
2010-10-21 17:00:35 +00:00
Johnny Chen 096011eebf Add an option '-s session-dir-name' to overwrite the default timestamp-named
directory used to dump the session info for test failures/errors.

Example:

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -s jason -v array_types

Session info for test errors or failures will go into directory jason
----------------------------------------------------------------------
Collected 4 tests

test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dsym_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dwarf_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... FAIL

======================================================================
FAIL: test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 27, in test_with_dwarf_and_run_command
    self.array_types()
  File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 62, in array_types
    'stop reason = breakpoint'])
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 594, in expect
    self.runCmd(str, trace = (True if trace else False), check = not error)
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 564, in runCmd
    msg if msg else CMD_MSG(cmd, True))
AssertionError: False is not True : Command 'thread list' returns successfully

----------------------------------------------------------------------
Ran 4 tests in 3.086s

FAILED (failures=1)
/Volumes/data/lldb/svn/trunk/test $ ls jason
TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
/Volumes/data/lldb/svn/trunk/test $ head -10 jason/TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log 
Session info generated @ Thu Oct 21 09:54:15 2010

os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=NO']]
stdout: rm -rf "a.out" "a.out.dSYM"  main.o main.d
cc -arch x86_64 -gdwarf-2 -O0   -c -o main.o main.c
cc -arch x86_64 -gdwarf-2 -O0  main.o -o "a.out"

stderr: None
retcode: 0

/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 117028
2010-10-21 16:55:35 +00:00
Johnny Chen 5858df1315 Get rid of the microsecond field in the timestamp directory name used to dump the
session information files to.  This makes the directory name less intimidating.
Currently, the directory only gets created if there are failures/errors while
running the test suite.

llvm-svn: 116982
2010-10-21 01:03:38 +00:00
Johnny Chen 12672250c1 Add an example of option combination for running a single test method to the help text.
llvm-svn: 116980
2010-10-21 00:47:52 +00:00
Johnny Chen 04d2c5cb0e Modify the test driver and lldbtest.TestBase so that the dumping of session info
now goes into a timestamp-specific directory instead of the previous .session-*
files.

[17:24:34] johnny:/Volumes/data/lldb/svn/trunk $ ls -l test/2010-10-18-16:56:12.935342
total 48
-rw-r--r--  1 johnny  admin  1695 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dsym_and_run_command.log
-rw-r--r--  1 johnny  admin  1652 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r--  1 johnny  admin  2967 Oct 18 16:56 TestBreakpointCommand.BreakpointCommandTestCase.test_with_dsym.log
-rw-r--r--  1 johnny  admin  1648 Oct 18 16:56 TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r--  1 johnny  admin  1665 Oct 18 16:56 TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase.test_with_dsym_and_python_api.log
-rw-r--r--  1 johnny  admin  3873 Oct 18 16:58 TestFloatTypesExpr.FloatTypesTestCase.test_float_types_with_dsym.log
[17:24:37] johnny:/Volumes/data/lldb/svn/trunk $ 

Also, the dumping happens when a test errored in additioned to when it failed.

llvm-svn: 116778
2010-10-19 00:25:01 +00:00
Johnny Chen 56f7939cdc Simply use the TestBase.markFailure() callback method to set the __failed__ flag.
llvm-svn: 116559
2010-10-15 02:28:13 +00:00
Johnny Chen 150c3cc825 This is an initial version of test driver enhanceent to be able to dump the
session info after a test case failure, allowing more direct inspection of
debugger session which leads to the test failure.

For a simple usage scenario:

[18:06:26] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log 

...

[18:14:43] johnny:/Volumes/data/lldb/svn/trunk/test $ ls -l .session-*
-rw-r--r--  1 johnny  admin  1359 Oct 14 18:06 .session-TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command
-rw-r--r--  1 johnny  admin  2054 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dsym_and_expr_parser
-rw-r--r--  1 johnny  admin  2055 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser
-rw-r--r--  1 johnny  admin  1351 Oct 14 17:57 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command
[18:14:51] johnny:/Volumes/data/lldb/svn/trunk/test $ 

The test case which failed will have its recorded session info dumped to a
.session-* file in the current working directory.  For test suite using
relocated directory, expect to find the .session-* files there.

In this checkin, I also add @skip decorator to the two test methods in
test/foundation/TestObjCMethods.py as it looks like the test suite is
deadlocking when running the tests.  More investigations are needed.

llvm-svn: 116552
2010-10-15 01:18:29 +00:00
Johnny Chen cd8b2bbed2 Change the call within lldb.py to 'SBDebugger.Initialize()' from 'lldb.SBDebugger.Initialize()'.
Inside the lldb module, there's no need (and as a matter of fact, incorrect) to specify the 'lldb'
module name.

Comment out the call to lldb.SBDebugger.Initialize() within the test driver itself, since it is
already done when we import the lldb.py module.

llvm-svn: 116485
2010-10-14 16:36:49 +00:00
Johnny Chen 370799c272 Fix an obvious cut-and-paste error.
llvm-svn: 116343
2010-10-12 21:50:36 +00:00
Johnny Chen 10093aa534 Added the capability for the test driver to split the sys.stderr/sys.stdout into
different configuration-based files using the config file.  For example:

    sys.stderr = open("/tmp/lldbtest-stderr", "w")
    sys.stdout = open("/tmp/lldbtest-stdout", "w")
    compilers = ["gcc", "llvm-gcc"]
    archs = ["x86_64", "i386"]
    split_stderr = True # This will split the stderr into configuration-specific file
    split_stdout = True # This will split the stdout into configuration-specific file

will produce:

/tmp/lldbtest-stderr
/tmp/lldbtest-stderr.arch=i386-compiler=gcc
/tmp/lldbtest-stderr.arch=i386-compiler=llvm-gcc
/tmp/lldbtest-stderr.arch=x86_64-compiler=gcc
/tmp/lldbtest-stderr.arch=x86_64-compiler=llvm-gcc
/tmp/lldbtest-stdout
/tmp/lldbtest-stdout.arch=i386-compiler=gcc
/tmp/lldbtest-stdout.arch=i386-compiler=llvm-gcc
/tmp/lldbtest-stdout.arch=x86_64-compiler=gcc
/tmp/lldbtest-stdout.arch=x86_64-compiler=llvm-gcc

as a result of splitting stderr and stdout.  In addition, each configuration can have
its individual top level relocated directory to house the test files as well as the
intermediate files by using '-r dir' to relocate the tests into a new relocated directory
instead of running the tests in place.

llvm-svn: 116341
2010-10-12 21:35:54 +00:00
Johnny Chen ae19e46ca5 Update comments.
llvm-svn: 116309
2010-10-12 15:53:22 +00:00
Johnny Chen 707b3c9c73 Added the capability for the test driver to relocate the tests and the intermediate
files to a different top level directory than those specified on the command line.

When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all.  This allows for an easier
postmortem analysis of test failures.

Example:

./dotest.py -v -t -r /tmp/lldbtest types

will create a /tmp/lldbtest directory which houses the types directory and its supported
files.

Files modified:

o dotest.py, lldbtest.py:

  Add logic to process '-r dir' option to support relocating the tests to a different
  top level directory instead of exected in place.

o darwin.py, test/types/Makefile:

  The 'make clean' should only clean the minimum .o and .d files.

llvm-svn: 116255
2010-10-11 22:25:46 +00:00
Johnny Chen afef640a9a Help text modification.
llvm-svn: 116206
2010-10-11 16:19:48 +00:00
Johnny Chen fbfba899a6 Fix logic error of the option processing loop.
llvm-svn: 115916
2010-10-07 15:41:55 +00:00
Johnny Chen 0ed37c9615 o SBtarget.cpp/.h:
Temporarily commenting out the deprecated LaunchProcess() method.
  SWIG is not able to handle the overloaded functions.

o dotest.py/lldbtest.py:

  Add an '-w' option to insert some wait time between consecutive test cases.

o TestClassTypes.py:

  Make the breakpoint_creation_by_filespec_python() test method more robust and
  more descriptive by printing out a more insightful assert message.

o lldb.swig: Coaches swig to treat StateType as an int type, instead of a C++ class.

llvm-svn: 115899
2010-10-07 02:04:14 +00:00
Johnny Chen 56e6cbdaf0 Enhance the test driver with a '-f filterspec' option to specify the
testclass.testmethod to be run and with a '-g' option which instructs the test
driver to only admit the module which satisfy the filterspec condition to the
test suite.

Example:

# This only runs the test case under the array_types directory which has class
# name of 'ArrayTypesTestCase' and the test method name of 'test_with_dwarf_and_run_command'.

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -f 'ArrayTypesTestCase.test_with_dwarf_and_run_command' -g array_types
----------------------------------------------------------------------
Collected 1 test

test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.353s

OK

# And this runs the test cases under the array_types and the hello_world directories.
# If the module discovered has the 'ArrayTypesTestCase.test_with_dwarf_and_run_command'
# attribute, only the test case specified by the filterspec for the module will be run.
# If the module does not have the said attribute, e.g., the module under hello_world,
# the whole module is still admitted to the test suite.

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -f 'ArrayTypesTestCase.test_with_dwarf_and_run_command' array_types hello_world
----------------------------------------------------------------------
Collected 3 tests

test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dsym_and_run_command (TestHelloWorld.HelloWorldTestCase)
Create target, breakpoint, launch a process, and then kill it. ... ok
test_with_dwarf_and_process_launch_api (TestHelloWorld.HelloWorldTestCase)
Create target, breakpoint, launch a process, and then kill it. ... ok

----------------------------------------------------------------------
Ran 3 tests in 4.964s

OK

llvm-svn: 115832
2010-10-06 20:40:56 +00:00
Johnny Chen da88434b39 o Added a new feature to the test framework to skip long running tests conditionally.
To not skip long running tests, pass '-l' to the test driver (dotest.py).

  An example:

    @unittest2.skipIf(TestBase.skipLongRunningTest(), "Skip this long running test")
    def test_foundation_disasm(self):
        ...

o Added a long running disassemble test to the foundation directory, which iterates
  the code symbols from Foundation.framework and kicks off a disassemble command for
  for the named function symbol.  Found a crasher: rdar://problem/8504895.

o Plus added/updated some comments for the TestBase class.

llvm-svn: 115368
2010-10-01 22:59:49 +00:00
Johnny Chen 92609b52ab Stop using LLDB_CC/LLDB_ARCH in the plugins for test configurations. Use make
friendly CC and ARCH, instead.

llvm-svn: 115169
2010-09-30 17:11:58 +00:00
Johnny Chen 6316021bae Added "float" and "double" to types/TestBasicTypes.py. Abstracted the generic
type tester method into its own abstarct base class 'AbstractBase'.  And
added TestIntegerTypes.py and TestFloatTypes.py.

Added an option "-p reg-exp-pattern" to the test driver to specify a regular
expression pattern to match against eligible filenames as our test cases.

An example: 

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -p "TestFloat.*" types
----------------------------------------------------------------------
Collected 4 tests

test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase)
Test that double-type variables are displayed correctly. ... ok
test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase)
Test that double-type variables are displayed correctly. ... ok
test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase)
Test that float-type variables are displayed correctly. ... ok
test_float_types_with_dsym (TestFloatTypes.FloatTypesTestCase)
Test that float-type variables are displayed correctly. ... ok

----------------------------------------------------------------------
Ran 4 tests in 5.592s

OK

llvm-svn: 114923
2010-09-27 23:29:54 +00:00
Johnny Chen 958da040e4 Changed the order of two assignment stmts.
llvm-svn: 114381
2010-09-21 00:16:09 +00:00
Johnny Chen 209cdbef64 Added the capability to source the configFile specified via the "-c" option in
order to customize the running of the test suite.  For the time being, the
supported customizations are:

o redirecting stdout and/or stderr
o specifying a list of compilers to build the test programs
o specifying a list of architectures to build the test programs for

Also checked into the examples/test directory some example files which
demonstrate the usage for the above customizations.

$ ./dotest.py -v -c ~/.lldbtest-config persistent_variables
$ cat ~/.lldbtest-config
sys.stderr = open("/tmp/lldbtest-stderr", "w")
sys.stdout = open("/tmp/lldbtest-stdout", "w")
compilers = ["gcc", "llvm-gcc"]
archs = ["x86_64", "i386"]
$ cat /tmp/lldbtest-stderr
----------------------------------------------------------------------
Collected 1 test


Configuration: arch=x86_64 compiler=gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.397s

OK

Configuration: arch=x86_64 compiler=llvm-gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.282s

OK

Configuration: arch=i386 compiler=gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.297s

OK

Configuration: arch=i386 compiler=llvm-gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.269s

OK
$ cat /tmp/lldbtest-stdout
$ 

llvm-svn: 114380
2010-09-21 00:09:27 +00:00
Johnny Chen daebb6d805 More modularizations.
llvm-svn: 114333
2010-09-20 18:07:50 +00:00
Johnny Chen a4b9aa0538 Added comments.
llvm-svn: 114332
2010-09-20 17:25:45 +00:00
Johnny Chen 49b415de1f Added a hook for the test driver to take an optional config file to customize
the running of the test suite.  Right now, it doesn't do anything with the
config file.

Pass "-c myConfigFile" to specify the config file.

llvm-svn: 114245
2010-09-18 00:16:47 +00:00
Johnny Chen df80a4cf25 Cleaned up the option processing logic and added a '-i' option to ignore the
build tree relative search path in order to locate the lldb.py module.  When
'-i' is present, the test driver relies on the PYTHONPATH environment variable
to locate the lldb.py module.

llvm-svn: 114094
2010-09-16 17:11:30 +00:00
Johnny Chen e9a1ef700b Added some comments.
llvm-svn: 114081
2010-09-16 15:44:23 +00:00
Johnny Chen 73982b94e3 Also added BuildAndIntegration to the directories to search for an existing
lldb.py module.  The priorities to search for are Debug, Release, then
BuildAndIntegration.  You can always override this with a valid PYTHONPATH
environment variable before running the test driver.

For example:

$ PYTHONPATH=/Find/My/LLDB/Module/Here ./dotest.py -v .

Python runtime will try to locate the lldb.py module from
/Find/My/LLDB/Module/Here first before trying the Debug, Release, and then
BuildAndIntegration directories.

llvm-svn: 113991
2010-09-15 18:11:19 +00:00
Johnny Chen 1aad5c61f8 Removed the expectedFailure decorator from test_with_dwarf_and_run_command() test case
as it now passes.  Added some extra tests to breakpoint_creation_by_filespec_python().

More clarification for the "os command" output and error as defined in
lldbtest.system() function.

Cleaned up the option processing of the test driver (dotest.py) and fixed the comment
about enabling gdb-remote logging.  Example:

$ GDB_REMOTE_LOG=/tmp/log.txt ./dotest.py -v -t enum_types

llvm-svn: 113868
2010-09-14 22:01:40 +00:00
Johnny Chen 2634032c66 Added '-d' option to the test driver to spit out the process id and do a delay
of 10 seconds in order for the debugger to attach.

llvm-svn: 113407
2010-09-08 20:56:16 +00:00
Johnny Chen 8d55a34a34 Changed the buildDsym()/buildDwarf() TestBase methods to use a plugin framework
to delegate the building of binaries to a sys.platform-sepcific plugin.

Modified the dotest.py test driver to add the "plugins" directory to the
PYTHONPATH as well.

darwin.py is the Mac OS X plugin module.

llvm-svn: 112606
2010-08-31 17:42:54 +00:00
Johnny Chen d0190a6182 Changed the keyword argument for runCmd()/expect() from 'verbose' to 'trace',
which, defaults to False, and if set to True, will trace lldb command execution
and result.

Added "-t" command option to the test driver dotest.py which sets the
LLDB_COMMAND_TRACE environment variable to "YES" and as a result always turns on
command tracing regardless of the 'trace' keyword argument to runCmd()/expect().

llvm-svn: 111811
2010-08-23 17:10:44 +00:00
Johnny Chen 83072afbcd Write out the informational message about the total number of test cases to be
run to stderr, instead of stdout.  The same as what the unittest framework uses.

llvm-svn: 111319
2010-08-17 23:00:13 +00:00
Johnny Chen 8a05de4d29 Add the ability to specify logging options for lldb and gdb-remote through two
additional env variables.

llvm-svn: 111295
2010-08-17 21:36:09 +00:00
Johnny Chen 46c2614b99 Added logging for process.gdb-remote by defining an environment variable named
GDB_REMOTE_LOG which specifies the log file pathname.

llvm-svn: 111198
2010-08-16 22:37:45 +00:00
Johnny Chen dd63f5dbe1 Added a workaround for test suite hang while terminating by checking env variable
LLDB_TESTSUITE_FORCE_FINISH and, if defined, kill the test suite.

llvm-svn: 111056
2010-08-13 22:58:44 +00:00
Johnny Chen edcbecfb1f Call lldb.SBDebugger.Terminate() at the end of test suite run through the atexit
handler similar to what's happening for the individual test case run.

llvm-svn: 110719
2010-08-10 20:23:55 +00:00
Johnny Chen f4804c696d Install the SIGINT (control-c) handler before invoking the text test runner.
llvm-svn: 110595
2010-08-09 20:40:52 +00:00
Johnny Chen f5957e0603 Putting out messages about the number of test cases to be run before running the
whole test suite.

llvm-svn: 110487
2010-08-07 00:16:07 +00:00
Johnny Chen 73258830f3 o Added unittest2 which has added the new features in unittest for Python 2.7
backported to Python 2.3+.  Some of the features desired include better
  verbose reporting in unittest2.TextTestRunner and decorator support for
  skipping tests and expected failures.

  http://pypi.python.org/pypi/unittest2

o Modified the existing .py tests to use unittest2 and decorated
  TestSTL.test_step_into_stl(), which is known to always fail currently, with
  @unittest2.expectedFailure.

llvm-svn: 110397
2010-08-05 23:42:46 +00:00
Johnny Chen bf6ffa3b89 Abstracted the lldb-specific unittest.TestCase.setUp()/tearDown() in a separate
module lldbtest.py and refactored the existing test cases to derive from the
abstract base class lldbtest.TestBase.

MOdified the test driver (dotest.py and dotest.pl) to set up additional
PYTHONPATH component for locating the lldbtest module, which sits in the same
directory.

llvm-svn: 107563
2010-07-03 03:41:59 +00:00
Johnny Chen c653d4c574 Fixed logic error; checking return status and raising exception should only be
done if the environment variable LLDB_LOG is defined.

llvm-svn: 107508
2010-07-02 20:35:23 +00:00
Johnny Chen 119b53ea6d Turn on logging for debugging purposes if ${LLDB_LOG} environment variable is
is defined.  Use ${LLDB_LOG} to specify the log file.

Create a singleton SBDebugger in the lldb namespace, that gets used when running
the entire test suite.

llvm-svn: 107445
2010-07-01 22:52:57 +00:00
Johnny Chen 94a6899e23 Add some delay before calling SBDebugger.Terminate().
llvm-svn: 107221
2010-06-29 23:17:15 +00:00
Johnny Chen cb9b857959 Added TestArrayTypes.py for test/array_types directory.
Also modified dotest.py so that it sets the LLDB_TEST environment variable
so that individual test cases can locate their supporting files correctly.

llvm-svn: 107220
2010-06-29 23:10:39 +00:00
Johnny Chen 4657be6e63 For the time being, let's bracket the test runner within the
lldb.SBDebugger.Initialize()/Terminate() pair.

Change TestHelp.py to use synchronous debugging, which is easier to work with.

llvm-svn: 107198
2010-06-29 19:44:16 +00:00
Johnny Chen c43d789a38 No need to add the test script containing directory to sys.path more than once.
llvm-svn: 106929
2010-06-26 00:19:32 +00:00
Johnny Chen 6d436d63ff Removed debug print statements.
llvm-svn: 106913
2010-06-25 23:34:17 +00:00
Johnny Chen a765c28db3 A simple testing framework for lldb using python's unit testing framework.
Tests for lldb are written as python scripts which take advantage of the script
bridging provided by LLDB.framework to interact with lldb core.

A specific naming pattern is followed by the .py script to be recognized as
a module which implements a test scenario, namely, Test*.py.

To specify the directories where "Test*.py" python test scripts are located,
you need to pass in a list of directory names.  By default, the current
working directory is searched if nothing is specified on the command line.

An example:

[14:10:20] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v example
test_choice (TestSequenceFunctions.TestSequenceFunctions) ... ok
test_sample (TestSequenceFunctions.TestSequenceFunctions) ... ok
test_shuffle (TestSequenceFunctions.TestSequenceFunctions) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

llvm-svn: 106890
2010-06-25 21:14:08 +00:00