llvm-project/lldb
Todd Fiala 7c5f7caa20 test infra: catch bad decorators and import-time errors
Summary:
This change enhances the LLDB test infrastructure to convert
load-time exceptions in a given Python test module into errors.
Before this change, specifying a non-existent test decorator,
or otherwise having some load-time error in a python test module,
would not get flagged as an error.

With this change, typos and other load-time errors in a python
test file get converted to errors and reported by the
test runner.

This change also includes test infrastructure tests that include
covering the new work here.  I'm going to wait until we have
these infrastructure tests runnable on the main platforms before
I try to work that into all the normal testing workflows.

The test infrastructure tests can be run by using the standard python module testing practice of doing the following:

cd packages/Python/lldbsuite/test_event
python -m unittest discover -s test/src -p 'Test*.py'

Those tests run the dotest inferior with a known broken test and verify that the errors are caught.  These tests did not pass until I modified dotest.py to capture them properly.

@zturner, if you have the chance, if you could try those steps above (the python -m unittest ... line) on Windows, that would be great if we can address any python2/3/Windows bits there.  I don't think there's anything fancy, but I didn't want to hook it into test flow until I know it works there.

I'll be slowly adding more tests that cover some of the other breakage I've occasionally seen that didn't get collected as part of the summarization.  This is the biggest one I'm aware of.

Reviewers: zturner, labath

Subscribers: zturner, lldb-commits

Differential Revision: http://reviews.llvm.org/D20193

llvm-svn: 269489
2016-05-13 21:36:26 +00:00
..
cmake Add CMake bits necessary for standalone build 2016-05-12 19:49:17 +00:00
docs test commit 2016-04-25 19:02:05 +00:00
examples Another little example use of scripted thread plans. 2016-05-03 00:14:52 +00:00
include/lldb Fix some long standing issues that caused tests to be flaky. 2016-05-12 22:58:52 +00:00
lit Replace hardcoded comment at 'lit.site.cfg.in' 2016-04-16 07:14:05 +00:00
lldb.xcodeproj Import block pointers from DWARF as Clang block pointers, not as structs. 2016-05-02 21:15:31 +00:00
lldb.xcworkspace Working on getting the OSX build green 2015-05-12 02:20:27 +00:00
packages/Python/lldbsuite test infra: catch bad decorators and import-time errors 2016-05-13 21:36:26 +00:00
resources Bump the lldb version # in the xcode project files from 2016-03-15 04:36:11 +00:00
scripts Add CMake bits necessary for standalone build 2016-05-12 19:49:17 +00:00
source Clean up test results on Windows. 2016-05-13 18:26:30 +00:00
test Fix ResourceWarning about unclosed file in use_lldb_suite_root.py. 2016-01-15 22:22:35 +00:00
third_party/Python/module Put progress.py back, apparently this can't be deleted. 2015-12-09 21:32:28 +00:00
tools Don't crash when a process' task port goes bad. 2016-05-12 22:36:47 +00:00
unittests Fix SymbolFilePDBTests.cpp 2016-05-10 13:46:22 +00:00
utils Python 3: Modernize exception raising syntax. 2015-11-03 21:01:45 +00:00
www Update the website with lots of new info about building / testing. 2016-02-24 22:19:23 +00:00
.arcconfig
.clang-format Update .clang-format file to support break after return type. 2015-12-28 22:09:29 +00:00
.gitignore modify Xcode build to use cmake/ninja for internal llvm/clang 2016-01-28 07:36:44 +00:00
CMakeLists.txt Fix NetBSD build with CMake 3.5.2 2016-05-01 10:23:24 +00:00
CODE_OWNERS.txt Added myself to the CODE_OWNERS.txt list for a few subsystems. 2015-11-09 01:24:36 +00:00
INSTALL.txt Revert the patch to Test Commit Access 2015-07-06 11:26:51 +00:00
LICENSE.TXT
use_lldb_suite_root.py Preparation for turning lldbsuite into a Python package. 2015-10-27 22:33:47 +00:00