Commit Graph

274 Commits

Author SHA1 Message Date
Enrico Granata e5cc5cd5ac Making it possible to specify a substring for a category name as long as the match is unique
llvm-svn: 181281
2013-05-07 00:36:28 +00:00
Enrico Granata af5bbe8f74 New test suite option (-T)
When -T is specified, the test suite will call svn info and dump the output on screen (this used to be the default behavior)
When -T is not specified, this step won't be performed (the new default)

llvm-svn: 179342
2013-04-11 23:48:00 +00:00
Enrico Granata ea6a58e2ae When specifying a relative path for the --framework option to dotest.py, Python would end up being confused and unable to locate the embedded_interpreter module, causing every testcase that uses the Script Interpreter (e.g. functionalities/data-formatter/data-formatter-stl/libstdcpp) to fail without even trying
This checkin fixes that problem by absolutizing the path before pushing it to the sys.path

llvm-svn: 179341
2013-04-11 23:40:59 +00:00
Daniel Malea 10320dd06f Minor LLDB test improvements
- fix TestCPPBool (was failing with some versions of GCC due to num_breakpoint_locations)
- speed up tests by skipping git/svn revision parsing by the test harness when running in lit-compatible (parsable) mode

This should resolve the failures reported by http://lab.llvm.org:8011/builders/lldb-x86_64-linux

llvm-svn: 176972
2013-03-13 20:50:05 +00:00
Daniel Malea 4185a92b9d Fix issue that causes linux buildbots to ignore errors
- change string "ERROR" to "FAIL" to match clang lit test results

Also, make LLDB tests work on machines that do not have svn and/or git installed

llvm-svn: 176633
2013-03-07 18:39:26 +00:00
Enrico Granata d7ac9816eb <rdar://problem/13289828>
Categories were conceptually meant to be placeable on test methods as well as test classes and test directories
However, that was broken. This checkin fixes that.
The incantation required to put categories on individual test case methods is not exactly elegant, unfortunately:

def test_case(self):
    """Test me."""
    self.do_it()

def _test_case_get_categories(self):
    return ["demo"]

test_case.getCategories = _test_case_get_categories
del _test_case_get_categories

llvm-svn: 176158
2013-02-27 02:37:12 +00:00
Jim Ingham d882998eff Fix the .categories, it had "dataformatter" not "dataformatters".
Remove the getCategory from TestDataFormatterObjC.py, since it was superceded by the .categories file, 
and didn't work anyway (getCategories currently has to be a method on the test class, not on the test.)
Add a "basic_process" category, and start to find some tests for simple process running sniff tests.

llvm-svn: 176061
2013-02-25 23:51:06 +00:00
Enrico Granata e6cedc1296 <rdar://problem/12362092>
The decorators @expectedFailure (plain and special-case like i386, clang, ...) are modified to optionally take a bugnumber argument
If such an argument is specified, the failure report (or unexpected success report) will include the information passed in as part of the message
This is mostly useful for associating failures to issue IDs in issue management systems (e.g. the LLVM bugzilla)

llvm-svn: 175942
2013-02-23 01:05:23 +00:00
Matt Kopec 93ee88179b Fix TestPublicAPIHeaders test on Linux.
-Change the build_dir variable name to lib_dir
 -Set lib_dir to the correct location on Linux
 -Set LD_EXTRAS to the actual lldb library

llvm-svn: 175664
2013-02-20 20:54:10 +00:00
Daniel Malea fce7f1f434 Fix a bug that causes dotest.py to exit with a passing exit-code, even if some tests fail
- occurs when multiple compilers/architectures are tested (via -C or -A flags)

llvm-svn: 175656
2013-02-20 20:12:11 +00:00
Jim Ingham 64ce7f4303 Make the progress bar opt in (using the -P option.)
llvm-svn: 175557
2013-02-19 20:39:27 +00:00
Filipe Cabecinhas 103e16e4cc Include a small argparse compatibility layer for Python < 2.7
llvm-svn: 175341
2013-02-16 09:05:23 +00:00
Daniel Malea cbaef26678 Improve test harness for the buildbots
- Add a "parsable" mode to dotest.py that outputs test results in exactly the same format as clang's lit tests
- Improve dosep script to output list of failing tests (output should look like clang test failure summaries)
- Cleanup lldb/test/Makefile to remove needless parameters and environment variables
- Switch makefile tests to use parsable-mode output; should make the buildbot results parsable
- Switch makefile tests to use dosep to log catch crashing tests (instead of halting the test suite)

llvm-svn: 175309
2013-02-15 21:31:37 +00:00
Enrico Granata 8628bc59e0 Daniel Malea caught an issue where calling dotest.py with an invalid directory would cause the progressbar init code to raise an exception
This commit fixes it

llvm-svn: 175229
2013-02-15 00:32:05 +00:00
Enrico Granata eba9e4a3aa The new progress bar mode was losing us information compared to the old dots mode in that we would have no way of knowing about test failures (short of peeking into the test result directory.. and you're not supposed to peek!)
Added a new line of information that reports the count of tests that pass, fail or have other things happen to them.

Again no flag to have the dots back. If you care, let us know!

llvm-svn: 174784
2013-02-09 00:37:07 +00:00
Enrico Granata a94ee7da81 <rdar://problem/13176279>
The LLDB test suite now shows a progress bar instead of dots when not in verbose mode
If you crave the dots, make your Terminal window smaller than 10 columns :-)
(or ask for a flag to have the dots come back on demand)

llvm-svn: 174777
2013-02-08 23:39:18 +00:00
Greg Clayton 0dd5e7b598 Unset the environment variables as soon as possible when running the test suite. Also don't store the unset list into a global when they aren't needed in a global variable.
llvm-svn: 174750
2013-02-08 21:52:32 +00:00
Daniel Malea 0e1cf09c89 Fix bug in test harness to allow running dotest.py without "--executable" flag on Linux
- resolves errors in cases that run the lldb CLI utility

llvm-svn: 174522
2013-02-06 16:55:07 +00:00
Daniel Malea 53430eb877 Fix lldb -P on Linux
- now prints the correct PYTHONPATH
- update dotest.py to use lldb -P result correctly
- resolves TestPublicAPIHeaders test failure (on Linux)

llvm-svn: 171558
2013-01-04 23:35:13 +00:00
Jim Ingham e2231ac783 Added an SBAPI to get the PythonPath (if the Host knows how to do that). And a -P option to the Driver
to print it out.  Changed dotest.py to use that to find the PythonPath it should use given the lldb binary
it was told to run.

llvm-svn: 170932
2012-12-21 22:22:26 +00:00
Enrico Granata 73f601fbd9 Fixing the -f option so that one specify multiple filters, e.g.
./dotest.py  -C clang --arch x86_64 --arch i386  -v -t -f ObjCDataFormatterTestCase.test_appkit_with_dsym_and_run_command -f ObjCDataFormatterTestCase.test_appkit_with_dwarf_and_run_command -f TestObjCBuiltinTypes.test_with_dsym_and_python_api -f TestObjCBuiltinTypes.test_with_dwarf_and_python_api -f ObjCDataFormatterTestCase.test_appkit_with_dsym_and_run_command -f ObjCDataFormatterTestCase.test_appkit_with_dwarf_and_run_command -f TestObjCBuiltinTypes.test_with_dsym_and_python_api -f -TestObjCBuiltinTypes.test_with_dwarf_and_python_api

The previous implementation would only remember the last filter passed, and consequently break redo.py

llvm-svn: 170163
2012-12-14 00:07:09 +00:00
Daniel Malea a750049171 Skip dsym tests on Linux
llvm-svn: 168370
2012-11-20 16:07:33 +00:00
Sean Callanan 0c23368a34 Fixes to make the test suite work better when
explicitly pointed at an LLDB framework and
executable.

llvm-svn: 167250
2012-11-01 21:23:21 +00:00
Sean Callanan ee1450aece Added --framework and --executable arguments
to dotest.py so that the testsuite can run against
any LLDB.

<rdar://problem/12512268>

llvm-svn: 166635
2012-10-24 22:45:39 +00:00
Enrico Granata 6c61cbbc49 Better error message for invalid argument to --category - silenced the printout of the raw args when not in verbose mode
llvm-svn: 166514
2012-10-23 22:52:49 +00:00
Enrico Granata bc08ab450a <rdar://problem/12500212> Test case for the new plugin feature
llvm-svn: 166453
2012-10-23 00:09:02 +00:00
Sean Callanan 4ca7389925 Added support for the DebugClang configuration to the
testsuite.

llvm-svn: 164723
2012-09-26 21:16:15 +00:00
Enrico Granata 165f8af8c5 Initial commit of a new testsuite feature: test categories.
This feature allows us to group test cases into logical groups (categories), and to only run a subset of test cases based on these categories.

Each test-case can have a new method getCategories(self): which returns a list of strings that are the categories to which the test case belongs.
If a test-case does not provide its own categories, we will look for categories in the class that contains the test case.
If that fails too, the default implementation looks for a .category file, which contains a comma separated list of strings.
The test suite will recurse look for .categories up until the top level directory (which we guarantee will have an empty .category file).

The driver dotest.py has a new --category <foo> option, which can be repeated, and specifies which categories of tests you want to run.
(example: ./dotest.py --category objc --category expression)

All tests that do not belong to any specified category will be skipped. Other filtering options still exist and should not interfere with category filtering.
A few tests have been categorized. Feel free to categorize others, and to suggest new categories that we could want to use.

All categories need to be validly defined in dotest.py, or the test suite will refuse to run when you use them as arguments to --category.

In the end, failures will be reported on a per-category basis, as well as in the usual format.

This is the very first stage of this feature. Feel free to chime in with ideas for improvements!

llvm-svn: 164403
2012-09-21 19:10:53 +00:00
Greg Clayton 5ec9645f83 Patch from Filipe Cabecinhas that uses argparse in dotest.py instead of a hand coded option. I made a few modifications:
Changed the '-A' option to also have a long option of '--arch'. This is now specified multiple times to get multiple architectures.

Old: -A i386^x86_64
New: -A i386 -A x86_64
     --arch i386 --arch x86_64
     
Changed the '-C' option to also have a long option of '--compiler'. This is now specified multiple times to get multiple compiler.

Old: -C clang^gcc
New: -C clang -C gcc
     --compiler clang --compiler gcc
llvm-svn: 163141
2012-09-04 15:42:49 +00:00
Johnny Chen e3fe26c5dd Fix wrong directory pathname.
llvm-svn: 162372
2012-08-22 17:53:02 +00:00
Johnny Chen a60c5ad819 Catch timestamps for the beginning and end of the test suite run.
llvm-svn: 162040
2012-08-16 19:15:21 +00:00
Filipe Cabecinhas f2bac60f74 Indentation fix for dotest.py.
llvm-svn: 161493
2012-08-08 15:23:24 +00:00
Filipe Cabecinhas d6cab1f6af Make dotest.py runnable from a directory other than lldb/test
llvm-svn: 161492
2012-08-08 15:05:04 +00:00
Johnny Chen 058bb9908f Fix a bug where if I just run:
./dotest.py

No progress bar appears.

llvm-svn: 157226
2012-05-22 00:22:18 +00:00
Greg Clayton 6d98f56c58 Maked LLDB into a package so we can import things without poluting the global namespace.
Enrico will follow this up with fixing the data formatter test cases that are failing.

llvm-svn: 155514
2012-04-25 00:58:03 +00:00
Johnny Chen 8f198c96fd Add a '-R' option, which is similar to '-r', except that the relocated directory, if exists, will be removed entirely
before running the test suite.  A usage example looks like this:

test $ ./dotest.py -A x86_64 -R /tmp/x86_64 &
test $ ./dotest.py -A i386 -R /tmp/i386 &

where we would want to run the x86_64 and i386 archs concurrently but relocate the test suite to different directory
hierarchies in order not to stump on each other's intermediate files.

llvm-svn: 155491
2012-04-24 21:44:10 +00:00
Johnny Chen c9cb71a0b3 LLDB test suite should also output the config info string along with the stack trace.
rdar://problem/11283401

Example:

Collected 1 test

1: test_with_dwarf (TestCallStdStringFunction.ExprCommandCallFunctionTestCase)
   Test calling std::String member function. ... FAIL

======================================================================
FAIL: test_with_dwarf (TestCallStdStringFunction.ExprCommandCallFunctionTestCase)
   Test calling std::String member function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/ToT/test/lldbtest.py", line 427, in wrapper
    return func(self, *args, **kwargs)
  File "/Volumes/data/lldb/svn/ToT/test/expression_command/call-function/TestCallStdStringFunction.py", line 34, in test_with_dwarf
    self.call_function()
  File "/Volumes/data/lldb/svn/ToT/test/expression_command/call-function/TestCallStdStringFunction.py", line 48, in call_function
    substrs = ['Hello world'])
  File "/Volumes/data/lldb/svn/ToT/test/lldbtest.py", line 1235, in expect
    msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : 'Hello world' returns expected result
Config=i386-clang
----------------------------------------------------------------------
Ran 1 test in 1.148s

FAILED (failures=1)

llvm-svn: 155157
2012-04-19 21:33:55 +00:00
Johnny Chen 39b8c3567a Simplify the progress bar display when neither "-v" nor "-t" is passed to the test driver.
llvm-svn: 155152
2012-04-19 20:09:44 +00:00
Johnny Chen 44d2497138 Add the capability of supplying the pre/post-flight functions to the test suite such that
the pre-flight code gets executed during setUp() after the debugger instance is available
and the post-flight code gets executed during tearDown() after the debugger instance has
done killing the inferior and deleting all the target programs.

Example:

[11:32:48] johnny:/Volumes/data/lldb/svn/ToT/test $ ./dotest.py -A x86_64 -v -c ../examples/test/.lldb-pre-post-flight  functionalities/watchpoint/hello_watchpoint
config: {'pre_flight': <function pre_flight at 0x1098541b8>, 'post_flight': <function post_flight at 0x109854230>}
LLDB build dir: /Volumes/data/lldb/svn/ToT/build/Debug
LLDB-139
Path: /Volumes/data/lldb/svn/ToT
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 154753
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 154730
Last Changed Date: 2012-04-13 18:42:46 -0700 (Fri, 13 Apr 2012)


lldb.pre_flight: def pre_flight(test):
    __import__("lldb")
    __import__("lldbtest")
    print "\nRunning pre-flight function:"
    print "for test case:", test

lldb.post_flight: def post_flight(test):
    __import__("lldb")
    __import__("lldbtest")
    print "\nRunning post-flight function:"
    print "for test case:", test


Session logs for test failures/errors/unexpected successes will go into directory '2012-04-16-11_34_08'
Command invoked: python ./dotest.py -A x86_64 -v -c ../examples/test/.lldb-pre-post-flight functionalities/watchpoint/hello_watchpoint
compilers=['clang']

Configuration: arch=x86_64 compiler=clang
----------------------------------------------------------------------
Collected 2 tests

1: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... 
Running pre-flight function:
for test case: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)

Running post-flight function:
for test case: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
ok
2: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... 
Running pre-flight function:
for test case: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)

Running post-flight function:
for test case: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
ok

----------------------------------------------------------------------
Ran 2 tests in 1.584s

OK

llvm-svn: 154847
2012-04-16 18:55:15 +00:00
Johnny Chen b127f00b51 Make the default architectures to run as both 64 and 32-bit.
rdar://problem/11031264

llvm-svn: 154573
2012-04-12 00:55:57 +00:00
Johnny Chen f1548d4f74 Add a new option to the test driver, -N dsym or -N dwarf, in order to exclude tests decorated with
either @dsym_test or @dwarf_test to be executed during the testsuite run.  There are still lots of
Test*.py files which have not been decorated with the new decorator.

An example:

# From TestMyFirstWatchpoint.py ->
class HelloWatchpointTestCase(TestBase):

    mydir = os.path.join("functionalities", "watchpoint", "hello_watchpoint")

    @dsym_test
    def test_hello_watchpoint_with_dsym_using_watchpoint_set(self):
        """Test a simple sequence of watchpoint creation and watchpoint hit."""
        self.buildDsym(dictionary=self.d)
        self.setTearDownCleanup(dictionary=self.d)
        self.hello_watchpoint()

    @dwarf_test
    def test_hello_watchpoint_with_dwarf_using_watchpoint_set(self):
        """Test a simple sequence of watchpoint creation and watchpoint hit."""
        self.buildDwarf(dictionary=self.d)
        self.setTearDownCleanup(dictionary=self.d)
        self.hello_watchpoint()


# Invocation ->
[17:50:14] johnny:/Volumes/data/lldb/svn/ToT/test $ ./dotest.py -N dsym -v -p TestMyFirstWatchpoint.py
LLDB build dir: /Volumes/data/lldb/svn/ToT/build/Debug
LLDB-137
Path: /Volumes/data/lldb/svn/ToT
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 154133
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 154109
Last Changed Date: 2012-04-05 10:43:02 -0700 (Thu, 05 Apr 2012)



Session logs for test failures/errors/unexpected successes will go into directory '2012-04-05-17_50_49'
Command invoked: python ./dotest.py -N dsym -v -p TestMyFirstWatchpoint.py
compilers=['clang']

Configuration: arch=x86_64 compiler=clang
----------------------------------------------------------------------
Collected 2 tests

1: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... skipped 'dsym tests'
2: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... ok

----------------------------------------------------------------------
Ran 2 tests in 1.138s

OK (skipped=1)
Session logs for test failures/errors/unexpected successes can be found in directory '2012-04-05-17_50_49'
[17:50:50] johnny:/Volumes/data/lldb/svn/ToT/test $ 

llvm-svn: 154154
2012-04-06 00:56:05 +00:00
Johnny Chen e344486e63 Add a '-E' option to the test driver for the purpose of specifying some extra CFLAGS
to pass to the toolchain in order to build the inferior programs to be run/debugged
duirng the test suite.  The architecture might dictate some special CFLAGS which are
more easily specified in a central place (like the command line) instead of inside
make rules.

For Example,

./dotest.py -v -r /shared/phone -A armv7 -E "-isysroot your_sdk_root" functionalities/watchpoint/hello_watchpoint

will relocate the particular test directory ('functionalities/watchpoint/hello_watchpoint' in this case) to a
new directory named '/shared/phone'.  The particular incarnation of the architecture-compiler combination of the
test support files are therefore to be found under:

/shared/phone.arch=armv7-compiler=clang/functionalities/watchpoint/hello_watchpoint

The building of the inferior programs under testing is now working.

The actual launching/debugging of the inferior programs are not yet working,
neither is the setting of a watchpoint on the phone.

llvm-svn: 153070
2012-03-20 00:33:51 +00:00
Sean Callanan ddcda28fee Disabled the i386 tests pending fixes in the JIT.
llvm-svn: 152568
2012-03-12 18:54:10 +00:00
Johnny Chen 934c05d284 Add the capability on OS X to utilize 'xcrun' to locate the compilers used for building the inferior programs
to be debugged while running the test suite.  By default, compilers is set to ['clang'] and can be overridden
using the "-C compilerA^compilerB" option.

llvm-svn: 152367
2012-03-09 02:11:37 +00:00
Johnny Chen 5a9a9883be Change the test driver so that, by default, it takes into consideration of both 'x86_64' and 'i386' architectures
when building the inferior programs.

Example:

/Volumes/data/lldb/svn/ToT/test $ ./dotest.py -v functionalities/watchpoint
LLDB build dir: /Volumes/data/lldb/svn/ToT/build/Debug
LLDB-123
Path: /Volumes/data/lldb/svn/ToT
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 152244
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 152244
Last Changed Date: 2012-03-07 13:03:09 -0800 (Wed, 07 Mar 2012)



Session logs for test failures/errors/unexpected successes will go into directory '2012-03-08-16_43_51'
Command invoked: python ./dotest.py -v functionalities/watchpoint

Configuration: arch=x86_64
----------------------------------------------------------------------
Collected 21 tests

 1: test_hello_watchlocation_with_dsym (TestWatchLocation.HelloWatchLocationTestCase)
    Test watching a location with '-x size' option. ... ok
 2: test_hello_watchlocation_with_dwarf (TestWatchLocation.HelloWatchLocationTestCase)
    Test watching a location with '-x size' option. ... ok
 3: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
    Test a simple sequence of watchpoint creation and watchpoint hit. ... ok
 4: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
    Test a simple sequence of watchpoint creation and watchpoint hit. ... ok
 5: test_watchpoint_multiple_threads_with_dsym (TestWatchpointMultipleThreads.WatchpointForMultipleThreadsTestCase)
    Test that lldb watchpoint works for multiple threads. ... ok
 6: test_watchpoint_multiple_threads_with_dwarf (TestWatchpointMultipleThreads.WatchpointForMultipleThreadsTestCase)
    Test that lldb watchpoint works for multiple threads. ... ok
 7: test_rw_disable_after_first_stop__with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint but disable it after the first stop. ... ok
 8: test_rw_disable_after_first_stop_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint but disable it after the first stop. ... ok
 9: test_rw_disable_then_enable_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint, disable initially, then enable it. ... ok
10: test_rw_disable_then_enable_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint, disable initially, then enable it. ... ok
11: test_rw_watchpoint_delete_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test delete watchpoint and expect not to stop for watchpoint. ... ok
12: test_rw_watchpoint_delete_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test delete watchpoint and expect not to stop for watchpoint. ... ok
13: test_rw_watchpoint_set_ignore_count_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test watchpoint ignore count and expect to not to stop at all. ... ok
14: test_rw_watchpoint_set_ignore_count_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test watchpoint ignore count and expect to not to stop at all. ... ok
15: test_rw_watchpoint_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint and expect to stop two times. ... ok
16: test_rw_watchpoint_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint and expect to stop two times. ... ok
17: test_watchpoint_cond_with_dsym (TestWatchpointConditionCmd.WatchpointConditionCmdTestCase)
    Test watchpoint condition. ... ok
18: test_watchpoint_cond_with_dwarf (TestWatchpointConditionCmd.WatchpointConditionCmdTestCase)
    Test watchpoint condition. ... ok
19: test_watchlocation_with_dsym_using_watchpoint_set (TestWatchLocationWithWatchSet.WatchLocationUsingWatchpointSetTestCase)
    Test watching a location with 'watchpoint set expression -w write -x size' option. ... ok
20: test_watchlocation_with_dwarf_using_watchpoint_set (TestWatchLocationWithWatchSet.WatchLocationUsingWatchpointSetTestCase)
    Test watching a location with 'watchpoint set expression -w write -x size' option. ... ok
21: test_error_cases_with_watchpoint_set (TestWatchpointSetErrorCases.WatchpointSetErrorTestCase)
    Test error cases with the 'watchpoint set' command. ... ok

----------------------------------------------------------------------
Ran 21 tests in 74.590s

OK

Configuration: arch=i386
----------------------------------------------------------------------
Collected 21 tests

 1: test_hello_watchlocation_with_dsym (TestWatchLocation.HelloWatchLocationTestCase)
    Test watching a location with '-x size' option. ... ok
 2: test_hello_watchlocation_with_dwarf (TestWatchLocation.HelloWatchLocationTestCase)
    Test watching a location with '-x size' option. ... ok
 3: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
    Test a simple sequence of watchpoint creation and watchpoint hit. ... ok
 4: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
    Test a simple sequence of watchpoint creation and watchpoint hit. ... ok
 5: test_watchpoint_multiple_threads_with_dsym (TestWatchpointMultipleThreads.WatchpointForMultipleThreadsTestCase)
    Test that lldb watchpoint works for multiple threads. ... ok
 6: test_watchpoint_multiple_threads_with_dwarf (TestWatchpointMultipleThreads.WatchpointForMultipleThreadsTestCase)
    Test that lldb watchpoint works for multiple threads. ... ok
 7: test_rw_disable_after_first_stop__with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint but disable it after the first stop. ... ok
 8: test_rw_disable_after_first_stop_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint but disable it after the first stop. ... ok
 9: test_rw_disable_then_enable_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint, disable initially, then enable it. ... ok
10: test_rw_disable_then_enable_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint, disable initially, then enable it. ... ok
11: test_rw_watchpoint_delete_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test delete watchpoint and expect not to stop for watchpoint. ... ok
12: test_rw_watchpoint_delete_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test delete watchpoint and expect not to stop for watchpoint. ... ok
13: test_rw_watchpoint_set_ignore_count_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test watchpoint ignore count and expect to not to stop at all. ... ok
14: test_rw_watchpoint_set_ignore_count_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test watchpoint ignore count and expect to not to stop at all. ... ok
15: test_rw_watchpoint_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint and expect to stop two times. ... ok
16: test_rw_watchpoint_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
    Test read_write watchpoint and expect to stop two times. ... ok
17: test_watchpoint_cond_with_dsym (TestWatchpointConditionCmd.WatchpointConditionCmdTestCase)
    Test watchpoint condition. ... ok
18: test_watchpoint_cond_with_dwarf (TestWatchpointConditionCmd.WatchpointConditionCmdTestCase)
    Test watchpoint condition. ... ok
19: test_watchlocation_with_dsym_using_watchpoint_set (TestWatchLocationWithWatchSet.WatchLocationUsingWatchpointSetTestCase)
    Test watching a location with 'watchpoint set expression -w write -x size' option. ... ok
20: test_watchlocation_with_dwarf_using_watchpoint_set (TestWatchLocationWithWatchSet.WatchLocationUsingWatchpointSetTestCase)
    Test watching a location with 'watchpoint set expression -w write -x size' option. ... ok
21: test_error_cases_with_watchpoint_set (TestWatchpointSetErrorCases.WatchpointSetErrorTestCase)
    Test error cases with the 'watchpoint set' command. ... ok

----------------------------------------------------------------------
Ran 21 tests in 67.059s

OK

llvm-svn: 152357
2012-03-09 00:44:59 +00:00
Johnny Chen 5d0d607b6b Add a period.
llvm-svn: 149305
2012-01-31 00:48:02 +00:00
Johnny Chen 8f8a2a7774 Add a '-u ENV_VAR_NAME' option to the test driver, whose purpose is to unset the said
environment variable before starting the test runner which executes the test cases and
may spawn child processes.  An example:

    ./dotest.py -u MY_ENV1 -u MY_ENV2 -v -p TestWatchLocationWithWatchSet.py

llvm-svn: 149304
2012-01-31 00:38:03 +00:00
Johnny Chen bc774c1156 Add a '-X excluded-dir' option to the test driver to exclude a directory from consideration during test discovery. For example:
./dotest.py -X types -v

from the test dir will ignore test cases under test/types.

llvm-svn: 148379
2012-01-18 05:15:00 +00:00
Johnny Chen 2e275c9e6a Add usage example of specifying the full path to the compiler used while building the debuggees for running the test suite.
llvm-svn: 148273
2012-01-17 01:26:06 +00:00
Johnny Chen 316651ab21 Add a simple progress bar when neither '-v' nor '-t' is specified.
llvm-svn: 144940
2011-11-18 00:19:29 +00:00
Johnny Chen 0fddfb2ceb Add an option '-S' to skip the build and cleanup while running the test.
Use this option with care as you would need to build the inferior(s) by hand
and build the executable(s) with the correct name(s).  This option can be used
with '-# n' to stress test certain test cases for n number of times.

An example:

[11:55:11] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ ls
Makefile		TestValueAPI.pyc	linked_list
TestValueAPI.py		change_values		main.c
[11:55:14] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ make EXE=test_with_dsym
clang -gdwarf-2 -O0  -arch x86_64   -c -o main.o main.c
clang -gdwarf-2 -O0  -arch x86_64   main.o -o "test_with_dsym"
/usr/bin/dsymutil  -o "test_with_dsym.dSYM" "test_with_dsym"
[11:55:20] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ cd ../..
[11:55:24] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -# 10 -S -f ValueAPITestCase.test_with_dsym
LLDB build dir: /Volumes/data/lldb/svn/trunk/build/Debug
LLDB-89
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 144914
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 144911
Last Changed Date: 2011-11-17 09:22:31 -0800 (Thu, 17 Nov 2011)



Session logs for test failures/errors/unexpected successes will go into directory '2011-11-17-11_55_29'
Command invoked: python ./dotest.py -v -# 10 -S -f ValueAPITestCase.test_with_dsym
----------------------------------------------------------------------
Collected 1 test

1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.163s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.200s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.198s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.199s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.239s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.215s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.105s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.098s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.195s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.197s

OK
[11:55:34] johnny:/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 144919
2011-11-17 19:57:27 +00:00
Daniel Dunbar 16d88ff045 tests: Improve Makefile/dotest to properly set LLDB_HERE variable used in some
tests.

llvm-svn: 143394
2011-10-31 23:27:06 +00:00
Johnny Chen 6c75b61f08 Fix a typo.
llvm-svn: 143207
2011-10-28 17:56:02 +00:00
Johnny Chen d9809f542b Print out the version of the locally built 'lldb' binary, not the one found in your PATH env variable.
llvm-svn: 143170
2011-10-28 00:59:00 +00:00
Johnny Chen fc9e79fb95 Benchmark the turnaround time starting a debugger and run to the breakpoint with lldb vs. gdb.
An example (with /Developer/usr/bin/lldb vs. /usr/bin/gdb):

[13:05:04] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v +b -n -p TestCompileRunToBreakpointTurnaround.py
1: test_run_lldb_then_gdb (TestCompileRunToBreakpointTurnaround.CompileRunToBreakpointBench)
   Benchmark turnaround time with lldb vs. gdb. ... 
lldb turnaround benchmark: Avg: 4.574600 (Laps: 3, Total Elapsed Time: 13.723799)
gdb turnaround benchmark: Avg: 7.966713 (Laps: 3, Total Elapsed Time: 23.900139)
lldb_avg/gdb_avg: 0.574214
ok

----------------------------------------------------------------------
Ran 1 test in 55.462s

OK

llvm-svn: 142949
2011-10-25 20:08:03 +00:00
Johnny Chen b8da426285 Add bench.py as a driver script to run some benchmarks on lldb.
Add benchmarks for expression evaluations (TestExpressionCmd.py) and disassembly (TestDoAttachThenDisassembly.py).

An example:
[17:45:55] johnny:/Volumes/data/lldb/svn/trunk/test $ ./bench.py 2>&1 | grep -P '^lldb.*benchmark:'
lldb startup delay (create fresh target) benchmark: Avg: 0.104274 (Laps: 30, Total Elapsed Time: 3.128214)
lldb startup delay (set first breakpoint) benchmark: Avg: 0.102216 (Laps: 30, Total Elapsed Time: 3.066470)
lldb frame variable benchmark: Avg: 1.649162 (Laps: 20, Total Elapsed Time: 32.983245)
lldb stepping benchmark: Avg: 0.104409 (Laps: 50, Total Elapsed Time: 5.220461)
lldb expr cmd benchmark: Avg: 0.206774 (Laps: 25, Total Elapsed Time: 5.169350)
lldb disassembly benchmark: Avg: 0.089086 (Laps: 10, Total Elapsed Time: 0.890859)

llvm-svn: 142708
2011-10-22 00:57:05 +00:00
Johnny Chen 4d4363ba56 Add a '-n' option to turn off printings of build dir, lldb version, svn info, and other headers
which happen before the listingings of test cases.

llvm-svn: 142668
2011-10-21 18:33:27 +00:00
Johnny Chen ff7fc9cfa4 Breakpoint specification can have the form '-n main', so it's not a good idea to
check that the option arg in '-x opt_arg' does not start with a '-' char.

llvm-svn: 142625
2011-10-20 22:16:24 +00:00
Johnny Chen 38f9daa303 Parameterize the iteration count used when running benchmarks, instead of hard-coded inside the test case.
Add a '-y count' option to the test driver for this purpose.  An example:

 $  ./dotest.py -v -y 25 +b -p TestDisassembly.py

...

----------------------------------------------------------------------
Collected 2 tests

1: test_run_gdb_then_lldb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
gdb benchmark: Avg: 0.226305 (Laps: 25, Total Elapsed Time: 5.657614)
lldb benchmark: Avg: 0.113864 (Laps: 25, Total Elapsed Time: 2.846606)
lldb_avg/gdb_avg: 0.503146
ok
2: test_run_lldb_then_gdb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
lldb benchmark: Avg: 0.113008 (Laps: 25, Total Elapsed Time: 2.825201)
gdb benchmark: Avg: 0.225240 (Laps: 25, Total Elapsed Time: 5.631001)
lldb_avg/gdb_avg: 0.501723
ok

----------------------------------------------------------------------
Ran 2 tests in 41.346s

OK

llvm-svn: 142598
2011-10-20 18:43:28 +00:00
Johnny Chen 999f7ab9bd Add '-k' option to the test driver to be able to specify a runhook, which is an lldb command
for the debugger to execute for certain kind of tests (for example, a benchmark).
A list of runhooks can be used to steer the debugger into the desired state before more
actions can be performed.

llvm-svn: 141626
2011-10-11 01:30:27 +00:00
Johnny Chen a10e6c1052 Add '-e' and '-x' options to the test driver to be able to specify an executable (full path)
and the breakpoint specification for the benchmark purpose.  This is used by TestSteppingSpeed.py
to benchmark the lldb stepping speed.  Without '-e' and 'x' specified, the test defaults to
run the built lldb against itself and stopped on Driver::MainLoop, then stepping for 50 times.

rdar://problem/7511193

llvm-svn: 141584
2011-10-10 22:03:44 +00:00
Johnny Chen 9846019114 Minor change for output message (less is better).
llvm-svn: 139924
2011-09-16 18:09:45 +00:00
Johnny Chen 9c0a94ad64 To be more paranoid, check cmd_output before searching in it.
llvm-svn: 139920
2011-09-16 18:03:19 +00:00
Johnny Chen 095ff6fdf6 Rephrase the checkDsymForUUIDIsNotOn() a bit so that in the normal case where automatic
lookup and cacheing of dSYMs is not enabled, the 'defaults read com.apple.DebugSymbols'
output is not shown.

llvm-svn: 139914
2011-09-16 17:50:44 +00:00
Johnny Chen 3cc78e608e Add a check for the test driver to make sure that, on Mac OS X, the automatic lookup and caching
of dSYMs is not turned on before running the test suite.

llvm-svn: 139896
2011-09-16 01:04:26 +00:00
Johnny Chen d890bfc962 Add a new attribute self.lldbHere, representing the fullpath to the 'lldb' executable
built locally from the source tree.  This is distinguished from self.lldbExec, which
can be used by test/benchmarks to measure the performances against other debuggers.

You can use environment variable LLDB_EXEC to specify self.lldbExec to the dotest.py
test driver, otherwise it is going to be populated with self.lldbHere.

Modify the regular tests under test dir, i.e., not test/benchmarks, to use self.lldbHere.
Also modify the benchmarks tests to use self.lldbHere when it needs an 'lldb' executable
with debug info to do the performance measurements.

llvm-svn: 138608
2011-08-26 00:00:01 +00:00
Johnny Chen c5cc625c79 Test driver should also report skipped tests because there were cases when tests were skipped
due to incorrect skip-logic.

llvm-svn: 137678
2011-08-15 23:09:08 +00:00
Johnny Chen 60726fab78 Fix a test driver bug where multiple filter specs (through the -f TestClassName.test_method_name
command line option) belonging to the same Test*.py file are ignored, except for the first one.

llvm-svn: 137540
2011-08-12 23:55:07 +00:00
Johnny Chen 70cb84a0f6 When running dotest.py under pdb (the python debugger), define DOTEST_PDB=YES and
DOTEST_SCRIPT_DIR=/path/leading/to/your/dotest/script/directory.

llvm-svn: 137492
2011-08-12 18:54:11 +00:00
Johnny Chen 19bfde34ed Add a test to check the integrity of the lldb public api directory containing SB*.h headers.
There should be nothing unwanted there and a simpe main.cpp (generated from main.cpp.template)
which includes SB*.h should compile and link with the LLDB framework.

llvm-svn: 136894
2011-08-04 18:17:16 +00:00
Johnny Chen 5ccbccfce0 Add a @benchmarks_test decorator for test method we want to categorize as benchmarks test.
The test driver now takes an option "+b" which enables to run just the benchmarks tests.
By default, tests decorated with the @benchmarks_test decorator do not get run.

Add an example benchmarks test directory which contains nothing for the time being,
just to demonstrate the @benchmarks_test concept.

For example,

$ ./dotest.py -v benchmarks

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with gdb. ... skipped 'benchmarks tests'
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with lldb. ... skipped 'benchmarks tests'

----------------------------------------------------------------------
Ran 2 tests in 0.047s

OK (skipped=2)
$ ./dotest.py -v +b benchmarks

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with gdb. ... running test_with_gdb
benchmarks result for test_with_gdb
ok
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with lldb. ... running test_with_lldb
benchmarks result for test_with_lldb
ok

----------------------------------------------------------------------
Ran 2 tests in 0.270s

OK

Also mark some Python API tests which are missing the @python_api_test decorator.

llvm-svn: 136553
2011-07-30 01:39:58 +00:00
Johnny Chen 4a57d122fe Add a redo.py script which takes a session directory name as arg and digs into the directory
to find out the tests which failed/errored and need re-running.  The dotest.py test driver
script is modified to allow specifying multiple -f testclass.testmethod in the command line
to accommodate the redo functionality.

An example,

 $ ./redo.py -n 2011-07-29-11_50_14
adding filterspec: TargetAPITestCase.test_find_global_variables_with_dwarf
adding filterspec: DisasmAPITestCase.test_with_dsym
Running ./dotest.py -v  -f TargetAPITestCase.test_find_global_variables_with_dwarf -f DisasmAPITestCase.test_with_dsym

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_dsym (TestDisasmAPI.DisasmAPITestCase)
   Exercise getting SBAddress objects, disassembly, and SBAddress APIs. ... ok
2: test_find_global_variables_with_dwarf (TestTargetAPI.TargetAPITestCase)
   Exercise SBTarget.FindGlobalVariables() API. ... ok

----------------------------------------------------------------------
Ran 2 tests in 15.328s

OK

llvm-svn: 136533
2011-07-29 22:54:56 +00:00
Johnny Chen cbdfe81171 Compute svn_info for git-svn bridge, too.
llvm-svn: 133836
2011-06-24 22:52:05 +00:00
Johnny Chen d0abdab8d1 Fix the log channel for gdb-remote. It should be "gdb-remote" now, instead of "process.gdb-remote".
llvm-svn: 133534
2011-06-21 19:25:45 +00:00
Peter Collingbourne 6f2b85221d Set LLDB_SESSION_DIRNAME to absolute path
This is to ensure that session files are written to the correct
directory if the -r option is given.

llvm-svn: 133483
2011-06-20 23:55:53 +00:00
Peter Collingbourne d6824de00e Modify TestHelp to search for LLDB-Info.plist in the correct place
This is required to run the test suite outside of the source tree
or on a case sensitive file system.

llvm-svn: 133460
2011-06-20 19:06:45 +00:00
Peter Collingbourne 28b5eb14c7 Have dotest.py search for test cases in its own directory by default,
rather than the current directory.

This is required to run the test suite outside of the source tree.

llvm-svn: 133458
2011-06-20 19:06:29 +00:00
Peter Collingbourne d736854419 Fix typo: curret -> current
llvm-svn: 132969
2011-06-14 03:55:45 +00:00
Johnny Chen 81a9df5b22 Let's also stick the svn info for the base directory into a file under
the session directory, which also includes the command line used to invoke
the test suite.

llvm-svn: 131501
2011-05-17 22:58:50 +00:00
Johnny Chen 44932b6805 For a test with unexpected success status, we also dump its session info into a unique file.
llvm-svn: 131011
2011-05-06 20:30:22 +00:00
Johnny Chen 0a8a54c117 Docstring.
llvm-svn: 130254
2011-04-26 23:10:51 +00:00
Johnny Chen 4fdcebda1d Docstring changees.
llvm-svn: 130251
2011-04-26 22:53:38 +00:00
Johnny Chen 489b52572c Simplify the parsing of ARCH and Compiler specs.
llvm-svn: 130240
2011-04-26 20:45:00 +00:00
Johnny Chen 71aaf75e3d Use self.TraceOn() API.
llvm-svn: 129936
2011-04-21 20:48:32 +00:00
Jim Ingham 5d3e822926 Have "-h" print just the command arguments, and "-h -v" print the examples as well.
llvm-svn: 129470
2011-04-13 21:11:41 +00:00
Greg Clayton 8b82f087a0 Moved the execution context that was in the Debugger into
the CommandInterpreter where it was always being used.

Make sure that Modules can track their object file offsets correctly to
allow opening of sub object files (like the "__commpage" on darwin).

Modified the Platforms to be able to launch processes. The first part of this
move is the platform soon will become the entity that launches your program
and when it does, it uses a new ProcessLaunchInfo class which encapsulates
all process launching settings. This simplifies the internal APIs needed for
launching. I want to slowly phase out process launching from the process
classes, so for now we can still launch just as we used to, but eventually
the platform is the object that should do the launching.

Modified the Host::LaunchProcess in the MacOSX Host.mm to correctly be able
to launch processes with all of the new eLaunchFlag settings. Modified any
code that was manually launching processes to use the Host::LaunchProcess
functions.

Fixed an issue where lldb_private::Args had implicitly defined copy 
constructors that could do the wrong thing. This has now been fixed by adding
an appropriate copy constructor and assignment operator.

Make sure we don't add empty ModuleSP entries to a module list.

Fixed the commpage module creation on MacOSX, but we still need to train
the MacOSX dynamic loader to not get rid of it when it doesn't have an entry
in the all image infos.

Abstracted many more calls from in ProcessGDBRemote down into the 
GDBRemoteCommunicationClient subclass to make the classes cleaner and more
efficient.

Fixed the default iOS ARM register context to be correct and also added support
for targets that don't support the qThreadStopInfo packet by selecting the
current thread (only if needed) and then sending a stop reply packet.

Debugserver can now start up with a --unix-socket (-u for short) and can 
then bind to port zero and send the port it bound to to a listening process
on the other end. This allows the GDB remote platform to spawn new GDB server
instances (debugserver) to allow platform debugging.

llvm-svn: 129351
2011-04-12 05:54:46 +00:00
Johnny Chen dbfc256eec As a simple measure, output the lldb version and the svn info for
which the testsuite is run against.

llvm-svn: 127782
2011-03-17 00:38:22 +00:00
Johnny Chen b7cfba4cb1 Add pexpect-2.4 (a pure Python module for controlling and automating other programs) to the test directory.
http://pypi.python.org/pypi/pexpect/

llvm-svn: 127484
2011-03-11 20:13:06 +00:00
Johnny Chen cd29b050b7 Add a mechanism to discover the full path of the 'lldb' program into the test driver.
This provides a way to potentially provide conversational interactions with 'lldb' in the test suite.

llvm-svn: 127479
2011-03-11 19:47:23 +00:00
Johnny Chen e0ec9ea5ce Add the ability for the test suite to specify a list of compilers and a list of architectures
on the command line.  For example, use '-A x86_64^i386' to launch the inferior use both x86_64
and i386.

This is an example of building the debuggee using both clang and gcc compiers:

[17:30:46] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -C clang^gcc -v -f SourceManagerTestCase.test_modify_source_file_while_debugging

Session logs for test failures/errors will go into directory '2011-03-03-17_31_39'
Command invoked: python ./dotest.py -C clang^gcc -v -f SourceManagerTestCase.test_modify_source_file_while_debugging

Configuration:  compiler=clang
----------------------------------------------------------------------
Collected 1 test

1: test_modify_source_file_while_debugging (TestSourceManager.SourceManagerTestCase)
   Modify a source file while debugging the executable. ... Command 'run' failed!

original content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

new content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello lldb.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after writing new content: 1299202305.0
content restored to: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after restore: 1299202307.0
ok

----------------------------------------------------------------------
Ran 1 test in 8.259s

OK

Configuration:  compiler=gcc
----------------------------------------------------------------------
Collected 1 test

1: test_modify_source_file_while_debugging (TestSourceManager.SourceManagerTestCase)
   Modify a source file while debugging the executable. ... original content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

new content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello lldb.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after writing new content: 1299202307.0
content restored to: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after restore: 1299202309.0
ok

----------------------------------------------------------------------
Ran 1 test in 2.301s

OK
[17:31:49] johnny:/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 126979
2011-03-04 01:35:22 +00:00
Greg Clayton 64195a2c8b Abtracted all mach-o and ELF out of ArchSpec. This patch is a modified form
of Stephen Wilson's idea (thanks for the input Stephen!). What I ended up
doing was:
- Got rid of ArchSpec::CPU (which was a generic CPU enumeration that mimics
  the contents of llvm::Triple::ArchType). We now rely upon the llvm::Triple 
  to give us the machine type from llvm::Triple::ArchType.
- There is a new ArchSpec::Core definition which further qualifies the CPU
  core we are dealing with into a single enumeration. If you need support for
  a new Core and want to debug it in LLDB, it must be added to this list. In
  the future we can allow for dynamic core registration, but for now it is
  hard coded.
- The ArchSpec can now be initialized with a llvm::Triple or with a C string
  that represents the triple (it can just be an arch still like "i386").
- The ArchSpec can still initialize itself with a architecture type -- mach-o
  with cpu type and subtype, or ELF with e_machine + e_flags -- and this will
  then get translated into the internal llvm::Triple::ArchSpec + ArchSpec::Core.
  The mach-o cpu type and subtype can be accessed using the getter functions:
  
  uint32_t
  ArchSpec::GetMachOCPUType () const;

  uint32_t
  ArchSpec::GetMachOCPUSubType () const;
  
  But these functions are just converting out internal llvm::Triple::ArchSpec 
  + ArchSpec::Core back into mach-o. Same goes for ELF.

All code has been updated to deal with the changes.

This should abstract us until later when the llvm::TargetSpec stuff gets
finalized and we can then adopt it.

llvm-svn: 126278
2011-02-23 00:35:02 +00:00
Greg Clayton 1971900fdc Don't enable thread safe logging as it currently deadlocks logging.
llvm-svn: 126260
2011-02-22 23:08:31 +00:00
Greg Clayton 76b436e21d Make logs threadsafe (add the -t option) when logging API stuff.
llvm-svn: 126257
2011-02-22 22:59:30 +00:00
Johnny Chen ffb3b51ba8 Refactored the test driver to abstract out the Python sys.path specification for
different build configurations.

llvm-svn: 125584
2011-02-15 18:50:19 +00:00
Greg Clayton c2448f6175 Added support for Xcode 4 build directories.
llvm-svn: 125522
2011-02-14 21:17:06 +00:00
Johnny Chen 74d53758c5 Added comment.
llvm-svn: 124517
2011-01-29 01:21:04 +00:00
Johnny Chen 6573818e38 Add a "-D" option to the test driver which dumps the Python sys.path variable
to be used for the test run.  Could be useful for debugging the setup of the
test environment.

llvm-svn: 124516
2011-01-29 01:16:52 +00:00
Johnny Chen 4f7370f7ae Add "-A arch_spec" and "-C compiler_spec" options to the test driver. Example:
$ ./dotest.py -A i386 -C clang -v -w -t -p TestObjCMethods.py
$ ./dotest.py -A x86_64 -C gcc -v -w types

llvm-svn: 124303
2011-01-26 19:07:42 +00:00
Johnny Chen f249090d7b Print out the command line used to invoke the test suite so I don't get confused
since different options can affect the run time.

llvm-svn: 123843
2011-01-19 19:31:46 +00:00
Johnny Chen 30775d0107 Added special logic to faciliate profiling the test suite run with the cProfile.py
module.

On my MBP running SnowLeopard:

$ DOTEST_PROFILE=YES DOTEST_SCRIPT_DIR=/Volumes/data/lldb/svn/trunk/test /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/cProfile.py -o my.profile ./dotest.py -v -w 2> ~/Developer/Log/lldbtest.log

After that, I used the pstats.py module to browse the statistics recorded in the my.profile file. 

llvm-svn: 123807
2011-01-19 02:10:40 +00:00
Johnny Chen 64e3868cf0 Properly indent the short description of the test case to make it align with the
previously added ordinal number of the currently running test case.

llvm-svn: 122922
2011-01-05 22:50:11 +00:00
Johnny Chen 77c81d3ee7 Enhance the test framework to be able to emit a counter value in verbose mode
describing the ordinal number of the currently running test case.

llvm-svn: 122901
2011-01-05 20:24:11 +00:00
Johnny Chen 240f8c7dd1 Do not specify both '-a' and '+a' at the same time.
llvm-svn: 121501
2010-12-10 19:02:23 +00:00
Johnny Chen f3e22ac3c7 Add a '+a' command line option to the test driver to run only the Python API tests.
Add an attribute __python_api_test__ (set to True) to the @python_api_test decorated
test method to distinguish them from the lldb command line tests.

llvm-svn: 121500
2010-12-10 18:52:10 +00:00
Johnny Chen 7c7d936e48 Add an infrastructure to mark the Python APIs only test using a decorator.
Example:

    @python_api_test
    def test_evaluate_expression_python(self):
        """Test SBFrame.EvaluateExpression() API for evaluating an expression."""
    ...

The opposite of Python APIs only test is an lldb command line test, which sends
commands to the lldb command interpreter.  Add a '-a' option to the test driver
to skip Python API only tests.

Modify TestExprs.py to mark a test as @python_api_test and remove an @expectedFailure
decorator as the bug has been fixed.

llvm-svn: 121442
2010-12-10 00:51:23 +00:00
Johnny Chen 025d1b8f95 For SBTarget.Launch()/LaunchProcess(), there's no need to pass an empty string
as the args and the envs to the launched process.

o lldbtest.py:

  Forgot to check in some assertion messages changes for lldbtest.py.

o dotest.py:

  Also add "api" category to the default lldb log option list.

llvm-svn: 121220
2010-12-08 01:25:21 +00:00
Johnny Chen 729c339a1c Add the '-F' option to the test driver, which makes the running of test suite failfast.
As soon as an error or a failure is encountered, it stops the test suite.

llvm-svn: 120837
2010-12-03 19:59:35 +00:00
Johnny Chen eb3411dc41 Log just the thread name, not process/thread id for easier human reading.
llvm-svn: 120751
2010-12-02 23:31:02 +00:00
Johnny Chen 7528af2f86 When logging, enable process and thread ID recordings. Also add 'state' to the lldb
log-categories, so it now becomes "event process expr state".

llvm-svn: 120745
2010-12-02 22:47:52 +00:00
Johnny Chen 20d0b2198b For lldb loggings, prepend log lines with a timestamp. And change the default log options:
o lldb => event process expr
o gdb-remote => packets process

llvm-svn: 120725
2010-12-02 18:35:13 +00:00
Johnny Chen 75739045d3 Add a '-b blacklistFile' option to the test driver to take a file specifying the
test classes or test cases to be excludued from the test suite.

Check in an example blacklist file: blacklist.py:

"""
'blacklist' is a Python dictionary, it stores the mapping of a string describing
either a testclass or a testcase, i.e, testclass.testmethod, to the reason (a
string) it is blacklisted.

Following is an example which states that test class IntegerTypesExprTestCase
should be skipped because 'This test class crashed' and the test case
FoundationTestCase.test_data_type_and_expr_with_dsym should be skipped because
it is 'Temporarily disabled'.

blacklist = {'IntegerTypesExprTestCase': 'This test class crashed',
             'FoundationTestCase.test_data_type_and_expr_with_dsym': 'Temporarily disabled'
             }
"""

blacklist = {}

An example of invoking the test driver and specifying a blacklist file:

./dotest.py -b blacklist.py -v types

This runs the tests under 'types' directory but excludes the tests specified in
balcklist.py.

llvm-svn: 120620
2010-12-01 22:47:54 +00:00
Johnny Chen bbda8ed39a Add comment.
llvm-svn: 120295
2010-11-29 17:52:43 +00:00
Johnny Chen 46389e9ddf When looping through the test suite more than once (repeat count > 1), we should
ignore and not enforce the signleton pattern for the LLDBTestResult class.

llvm-svn: 120294
2010-11-29 17:50:10 +00:00
Johnny Chen ecb072d840 Add an option '-# count' to run the specified test suite for a said number of times.
This is not to be used during normal test suite run, but to be used to stress test
specific test sequences repeatedly.

Example:

./dotest.py -# 3 -v breakpoint_conditions

will repeat the test suite 3 times for tests under the breakpoint_conditions directory.

llvm-svn: 119399
2010-11-16 22:42:58 +00:00
Johnny Chen 5f98696f51 Add help message for the scenario of using lldb.py module distributed by Xcode4
to invoke the test driver.

llvm-svn: 118847
2010-11-11 22:14:56 +00:00
Johnny Chen cc78b0b67f Add more comments on LLDBTestResult class.
llvm-svn: 118655
2010-11-09 23:56:14 +00:00
Johnny Chen bb4f8e6f7b Rephrase the output message about session information; make it both "session logs".
llvm-svn: 118650
2010-11-09 23:42:00 +00:00
Johnny Chen 27f391be61 Better wording of the help text for the '-f testclass.testmethod' option usage.
llvm-svn: 118438
2010-11-08 20:17:04 +00:00
Johnny Chen c06f422d73 Let's also emit the informational message for where to find the session logs for
failed/errored tests at the start of the test run, in case the test run crashes
for any reason.  That way, it is easy to locate the session logs for accumulated
failures/errors.

llvm-svn: 118427
2010-11-08 19:03:37 +00:00
Johnny Chen fa491601b5 Change the test driver so that simply specifying '-f testclass.testmethod' means that
we want to run just the instance of testclass-testmethod combination and nothing else.

Specifying '-g' now will admit the whole .py test module if it does not contain a matching
testclass-testmethod combination at all.

This option arrangement adheres to the RISC principle of making the common cases fast. :-)

rdar://problem/8584914 Can the default for dotest.py's "-g" flag be switched?

llvm-svn: 118392
2010-11-08 01:21:03 +00:00
Johnny Chen 9a6706bca4 Add comment about iterating over possible architecture and compiler combinations.
llvm-svn: 118279
2010-11-05 17:30:53 +00:00
Johnny Chen 110bd0f8cf Now that the test suite has very few expected failures remaining (2), it is a good
idea to also dump the session information for expected failures in addition to
failures and errors (tests with unexpected exceptions).

Modify the LLDBTestResult class which derives from unittest2.TextTestResult to
intercept the addExpectedFailure() method in order to mark the test instance as
needing its session information dumped.

llvm-svn: 118185
2010-11-03 18:17:03 +00:00
Johnny Chen 721f298a60 Add some comment about the default session dir name which is a timestamp.
llvm-svn: 117730
2010-10-29 22:26:38 +00:00
Johnny Chen 8ff5d56c20 The informational message for where to find the session logs for failed/errored
tests should be near the test summary message, instead of way up in the beginning.

llvm-svn: 117726
2010-10-29 22:20:36 +00:00
Johnny Chen bdc5c27894 Fix the test driver as part of the fix for http://llvm.org/bugs/show_bug.cgi?id=8493
File name "2010-10-19-14:10:49.059609" is non-portable.

Do not use ':' in the pathname as it is not permitted on windows platforms.

llvm-svn: 117556
2010-10-28 16:32:13 +00:00
Johnny Chen 8c251f4209 Add some more usage text.
llvm-svn: 117129
2010-10-22 19:00:18 +00:00
Johnny Chen c14e08f437 Rewording of output message.
llvm-svn: 117030
2010-10-21 17:00:35 +00:00
Johnny Chen 096011eebf Add an option '-s session-dir-name' to overwrite the default timestamp-named
directory used to dump the session info for test failures/errors.

Example:

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -s jason -v array_types

Session info for test errors or failures will go into directory jason
----------------------------------------------------------------------
Collected 4 tests

test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dsym_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dwarf_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... FAIL

======================================================================
FAIL: test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 27, in test_with_dwarf_and_run_command
    self.array_types()
  File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 62, in array_types
    'stop reason = breakpoint'])
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 594, in expect
    self.runCmd(str, trace = (True if trace else False), check = not error)
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 564, in runCmd
    msg if msg else CMD_MSG(cmd, True))
AssertionError: False is not True : Command 'thread list' returns successfully

----------------------------------------------------------------------
Ran 4 tests in 3.086s

FAILED (failures=1)
/Volumes/data/lldb/svn/trunk/test $ ls jason
TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
/Volumes/data/lldb/svn/trunk/test $ head -10 jason/TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log 
Session info generated @ Thu Oct 21 09:54:15 2010

os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=NO']]
stdout: rm -rf "a.out" "a.out.dSYM"  main.o main.d
cc -arch x86_64 -gdwarf-2 -O0   -c -o main.o main.c
cc -arch x86_64 -gdwarf-2 -O0  main.o -o "a.out"

stderr: None
retcode: 0

/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 117028
2010-10-21 16:55:35 +00:00
Johnny Chen 5858df1315 Get rid of the microsecond field in the timestamp directory name used to dump the
session information files to.  This makes the directory name less intimidating.
Currently, the directory only gets created if there are failures/errors while
running the test suite.

llvm-svn: 116982
2010-10-21 01:03:38 +00:00
Johnny Chen 12672250c1 Add an example of option combination for running a single test method to the help text.
llvm-svn: 116980
2010-10-21 00:47:52 +00:00
Johnny Chen 04d2c5cb0e Modify the test driver and lldbtest.TestBase so that the dumping of session info
now goes into a timestamp-specific directory instead of the previous .session-*
files.

[17:24:34] johnny:/Volumes/data/lldb/svn/trunk $ ls -l test/2010-10-18-16:56:12.935342
total 48
-rw-r--r--  1 johnny  admin  1695 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dsym_and_run_command.log
-rw-r--r--  1 johnny  admin  1652 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r--  1 johnny  admin  2967 Oct 18 16:56 TestBreakpointCommand.BreakpointCommandTestCase.test_with_dsym.log
-rw-r--r--  1 johnny  admin  1648 Oct 18 16:56 TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r--  1 johnny  admin  1665 Oct 18 16:56 TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase.test_with_dsym_and_python_api.log
-rw-r--r--  1 johnny  admin  3873 Oct 18 16:58 TestFloatTypesExpr.FloatTypesTestCase.test_float_types_with_dsym.log
[17:24:37] johnny:/Volumes/data/lldb/svn/trunk $ 

Also, the dumping happens when a test errored in additioned to when it failed.

llvm-svn: 116778
2010-10-19 00:25:01 +00:00
Johnny Chen 56f7939cdc Simply use the TestBase.markFailure() callback method to set the __failed__ flag.
llvm-svn: 116559
2010-10-15 02:28:13 +00:00
Johnny Chen 150c3cc825 This is an initial version of test driver enhanceent to be able to dump the
session info after a test case failure, allowing more direct inspection of
debugger session which leads to the test failure.

For a simple usage scenario:

[18:06:26] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log 

...

[18:14:43] johnny:/Volumes/data/lldb/svn/trunk/test $ ls -l .session-*
-rw-r--r--  1 johnny  admin  1359 Oct 14 18:06 .session-TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command
-rw-r--r--  1 johnny  admin  2054 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dsym_and_expr_parser
-rw-r--r--  1 johnny  admin  2055 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser
-rw-r--r--  1 johnny  admin  1351 Oct 14 17:57 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command
[18:14:51] johnny:/Volumes/data/lldb/svn/trunk/test $ 

The test case which failed will have its recorded session info dumped to a
.session-* file in the current working directory.  For test suite using
relocated directory, expect to find the .session-* files there.

In this checkin, I also add @skip decorator to the two test methods in
test/foundation/TestObjCMethods.py as it looks like the test suite is
deadlocking when running the tests.  More investigations are needed.

llvm-svn: 116552
2010-10-15 01:18:29 +00:00
Johnny Chen cd8b2bbed2 Change the call within lldb.py to 'SBDebugger.Initialize()' from 'lldb.SBDebugger.Initialize()'.
Inside the lldb module, there's no need (and as a matter of fact, incorrect) to specify the 'lldb'
module name.

Comment out the call to lldb.SBDebugger.Initialize() within the test driver itself, since it is
already done when we import the lldb.py module.

llvm-svn: 116485
2010-10-14 16:36:49 +00:00
Johnny Chen 370799c272 Fix an obvious cut-and-paste error.
llvm-svn: 116343
2010-10-12 21:50:36 +00:00
Johnny Chen 10093aa534 Added the capability for the test driver to split the sys.stderr/sys.stdout into
different configuration-based files using the config file.  For example:

    sys.stderr = open("/tmp/lldbtest-stderr", "w")
    sys.stdout = open("/tmp/lldbtest-stdout", "w")
    compilers = ["gcc", "llvm-gcc"]
    archs = ["x86_64", "i386"]
    split_stderr = True # This will split the stderr into configuration-specific file
    split_stdout = True # This will split the stdout into configuration-specific file

will produce:

/tmp/lldbtest-stderr
/tmp/lldbtest-stderr.arch=i386-compiler=gcc
/tmp/lldbtest-stderr.arch=i386-compiler=llvm-gcc
/tmp/lldbtest-stderr.arch=x86_64-compiler=gcc
/tmp/lldbtest-stderr.arch=x86_64-compiler=llvm-gcc
/tmp/lldbtest-stdout
/tmp/lldbtest-stdout.arch=i386-compiler=gcc
/tmp/lldbtest-stdout.arch=i386-compiler=llvm-gcc
/tmp/lldbtest-stdout.arch=x86_64-compiler=gcc
/tmp/lldbtest-stdout.arch=x86_64-compiler=llvm-gcc

as a result of splitting stderr and stdout.  In addition, each configuration can have
its individual top level relocated directory to house the test files as well as the
intermediate files by using '-r dir' to relocate the tests into a new relocated directory
instead of running the tests in place.

llvm-svn: 116341
2010-10-12 21:35:54 +00:00
Johnny Chen ae19e46ca5 Update comments.
llvm-svn: 116309
2010-10-12 15:53:22 +00:00
Johnny Chen 707b3c9c73 Added the capability for the test driver to relocate the tests and the intermediate
files to a different top level directory than those specified on the command line.

When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all.  This allows for an easier
postmortem analysis of test failures.

Example:

./dotest.py -v -t -r /tmp/lldbtest types

will create a /tmp/lldbtest directory which houses the types directory and its supported
files.

Files modified:

o dotest.py, lldbtest.py:

  Add logic to process '-r dir' option to support relocating the tests to a different
  top level directory instead of exected in place.

o darwin.py, test/types/Makefile:

  The 'make clean' should only clean the minimum .o and .d files.

llvm-svn: 116255
2010-10-11 22:25:46 +00:00
Johnny Chen afef640a9a Help text modification.
llvm-svn: 116206
2010-10-11 16:19:48 +00:00
Johnny Chen fbfba899a6 Fix logic error of the option processing loop.
llvm-svn: 115916
2010-10-07 15:41:55 +00:00
Johnny Chen 0ed37c9615 o SBtarget.cpp/.h:
Temporarily commenting out the deprecated LaunchProcess() method.
  SWIG is not able to handle the overloaded functions.

o dotest.py/lldbtest.py:

  Add an '-w' option to insert some wait time between consecutive test cases.

o TestClassTypes.py:

  Make the breakpoint_creation_by_filespec_python() test method more robust and
  more descriptive by printing out a more insightful assert message.

o lldb.swig: Coaches swig to treat StateType as an int type, instead of a C++ class.

llvm-svn: 115899
2010-10-07 02:04:14 +00:00
Johnny Chen 56e6cbdaf0 Enhance the test driver with a '-f filterspec' option to specify the
testclass.testmethod to be run and with a '-g' option which instructs the test
driver to only admit the module which satisfy the filterspec condition to the
test suite.

Example:

# This only runs the test case under the array_types directory which has class
# name of 'ArrayTypesTestCase' and the test method name of 'test_with_dwarf_and_run_command'.

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -f 'ArrayTypesTestCase.test_with_dwarf_and_run_command' -g array_types
----------------------------------------------------------------------
Collected 1 test

test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.353s

OK

# And this runs the test cases under the array_types and the hello_world directories.
# If the module discovered has the 'ArrayTypesTestCase.test_with_dwarf_and_run_command'
# attribute, only the test case specified by the filterspec for the module will be run.
# If the module does not have the said attribute, e.g., the module under hello_world,
# the whole module is still admitted to the test suite.

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -f 'ArrayTypesTestCase.test_with_dwarf_and_run_command' array_types hello_world
----------------------------------------------------------------------
Collected 3 tests

test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dsym_and_run_command (TestHelloWorld.HelloWorldTestCase)
Create target, breakpoint, launch a process, and then kill it. ... ok
test_with_dwarf_and_process_launch_api (TestHelloWorld.HelloWorldTestCase)
Create target, breakpoint, launch a process, and then kill it. ... ok

----------------------------------------------------------------------
Ran 3 tests in 4.964s

OK

llvm-svn: 115832
2010-10-06 20:40:56 +00:00
Johnny Chen da88434b39 o Added a new feature to the test framework to skip long running tests conditionally.
To not skip long running tests, pass '-l' to the test driver (dotest.py).

  An example:

    @unittest2.skipIf(TestBase.skipLongRunningTest(), "Skip this long running test")
    def test_foundation_disasm(self):
        ...

o Added a long running disassemble test to the foundation directory, which iterates
  the code symbols from Foundation.framework and kicks off a disassemble command for
  for the named function symbol.  Found a crasher: rdar://problem/8504895.

o Plus added/updated some comments for the TestBase class.

llvm-svn: 115368
2010-10-01 22:59:49 +00:00
Johnny Chen 92609b52ab Stop using LLDB_CC/LLDB_ARCH in the plugins for test configurations. Use make
friendly CC and ARCH, instead.

llvm-svn: 115169
2010-09-30 17:11:58 +00:00
Johnny Chen 6316021bae Added "float" and "double" to types/TestBasicTypes.py. Abstracted the generic
type tester method into its own abstarct base class 'AbstractBase'.  And
added TestIntegerTypes.py and TestFloatTypes.py.

Added an option "-p reg-exp-pattern" to the test driver to specify a regular
expression pattern to match against eligible filenames as our test cases.

An example: 

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -p "TestFloat.*" types
----------------------------------------------------------------------
Collected 4 tests

test_double_type_with_dsym (TestFloatTypes.FloatTypesTestCase)
Test that double-type variables are displayed correctly. ... ok
test_double_type_with_dwarf (TestFloatTypes.FloatTypesTestCase)
Test that double-type variables are displayed correctly. ... ok
test_float_type_with_dwarf (TestFloatTypes.FloatTypesTestCase)
Test that float-type variables are displayed correctly. ... ok
test_float_types_with_dsym (TestFloatTypes.FloatTypesTestCase)
Test that float-type variables are displayed correctly. ... ok

----------------------------------------------------------------------
Ran 4 tests in 5.592s

OK

llvm-svn: 114923
2010-09-27 23:29:54 +00:00
Johnny Chen 958da040e4 Changed the order of two assignment stmts.
llvm-svn: 114381
2010-09-21 00:16:09 +00:00
Johnny Chen 209cdbef64 Added the capability to source the configFile specified via the "-c" option in
order to customize the running of the test suite.  For the time being, the
supported customizations are:

o redirecting stdout and/or stderr
o specifying a list of compilers to build the test programs
o specifying a list of architectures to build the test programs for

Also checked into the examples/test directory some example files which
demonstrate the usage for the above customizations.

$ ./dotest.py -v -c ~/.lldbtest-config persistent_variables
$ cat ~/.lldbtest-config
sys.stderr = open("/tmp/lldbtest-stderr", "w")
sys.stdout = open("/tmp/lldbtest-stdout", "w")
compilers = ["gcc", "llvm-gcc"]
archs = ["x86_64", "i386"]
$ cat /tmp/lldbtest-stderr
----------------------------------------------------------------------
Collected 1 test


Configuration: arch=x86_64 compiler=gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.397s

OK

Configuration: arch=x86_64 compiler=llvm-gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.282s

OK

Configuration: arch=i386 compiler=gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.297s

OK

Configuration: arch=i386 compiler=llvm-gcc
test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
Test that lldb persistent variables works correctly. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.269s

OK
$ cat /tmp/lldbtest-stdout
$ 

llvm-svn: 114380
2010-09-21 00:09:27 +00:00
Johnny Chen daebb6d805 More modularizations.
llvm-svn: 114333
2010-09-20 18:07:50 +00:00
Johnny Chen a4b9aa0538 Added comments.
llvm-svn: 114332
2010-09-20 17:25:45 +00:00
Johnny Chen 49b415de1f Added a hook for the test driver to take an optional config file to customize
the running of the test suite.  Right now, it doesn't do anything with the
config file.

Pass "-c myConfigFile" to specify the config file.

llvm-svn: 114245
2010-09-18 00:16:47 +00:00