SUMMARY
Flakey tests get two chances to pass
Also, switched a bunch of tests to use new decorator.
TEST PLAN
Add one of these decorators to a test
Edit a test to pass on the first invocation, confirm test appears as pass
Edit a test to pass on the first invocation, pass on the second, confirm test appears as xfail
Edit a test to fail on two consecutive runs, confirm test appears in results as fail/error
Differential Revision: http://reviews.llvm.org/D10721
llvm-svn: 240789
Summary: Updated `append_to_remote_wd` to work for both remote and local.
Reviewers: clayborg, ovyalov
Reviewed By: ovyalov
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10288
llvm-svn: 239203
Summary:
Before:
AssertionError: False is not True : Process is launched successfully
After:
AssertionError: False is not True : Command 'run a.out' failed.
>>> error: invalid target, create a target using the 'target create' command
>>> Process could not be launched successfully
Reviewers: clayborg
Reviewed By: clayborg
Subscribers: lldb-commits, vharron
Differential Revision: http://reviews.llvm.org/D9948
llvm-svn: 238363
Summary:
Since we don't yet have remote windows debugging, it should be safe to assume
that the remote target uses unix path separators.
Reviewers: ovyalov, zturner, clayborg, vharron
Reviewed By: vharron
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D9633
llvm-svn: 237006
Summary:
If a struct type S has a member T that has a member that is a function that
returns a typedef of S* the respective field would be duplicated, which caused
an assert down the line in RecordLayoutBuilder. This patch adds a check that
removes the possibility of trying to resolve the same type twice within the
same callstack.
This commit also adds unit tests for these failures.
Fixes https://llvm.org/bugs/show_bug.cgi?id=20486.
Patch by Cristian Hancila.
Test Plan: Run unit tests.
Reviewers: clayborg spyffe
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D8561
llvm-svn: 234441
Adds @skipIfPlatform and @skipUnlessPlatform decorators which will skip if /
unless the target platform is in the provided platform list.
Test Plan:
ninja check-lldb shows no regressions.
When running cross platform, tests which cannot run on the target platform are
skipped.
Differential Revision: http://reviews.llvm.org/D8665
llvm-svn: 233547
Tests derive from TestBase, which derives from Base. In the
test setUp() methods, we always call TestBase.setUp() first and
then call implementation-specific setup. Tear down needs to do
the reverse.
This was causing over 20 failures on Windows, and was the culprit
behind about 80% of the files not being cleaned up after test run.
TestBase.tearDown() is responsible for deleting all targets created
during the test run and without this step, on Windows files will
be locked and cannot be deleted. But TestBase.tearDown() was
calling Base.tearDown() before its own cleanup (i.e. deleting the
targets) and in some cases one of the teardown hooks would be to
call make clean. So make clean would be run before the targets
had been deleted, and fail to remove the files, and subsequently
result in a failed test as well.
llvm-svn: 233284
redirecting output to a path that will work well on host or target.
copying file from output location to location on local host that
test will read from
llvm-svn: 228217
Fixes include:
- dont set or change LDFLAGS, but set LD_EXTRAS instead
- fix compilation errors for iOS based builds with objective C code
- fix test cases to create classes instead of relying on classes from AppKit
- rename things where it makes sense
llvm-svn: 221496
This has led to many test suite failures because of copy and paste where new test cases were based off of other test cases and the "mydir" variable wasn't updated.
Now you can call your superclasses "compute_mydir()" function with "__file__" as the sole argument and the relative path will be computed for you.
llvm-svn: 196985
Cleaned up the option parsing code to always pass around the short options as integers. Previously we cast this down to "char" and lost some information. I recently added an assert that would detect duplicate short character options which was firing during the test suite.
This fix does the following:
- make sure all short options are treated as "int"
- make sure that short options can be non-printable values when a short option is not required or when an option group is mixed into many commands and a short option is not desired
- fix the help printing to "do the right thing" in all cases. Previously if there were duplicate short character options, it would just not emit help for the duplicates
- fix option parsing when there are duplicates to parse options correctly. Previously the option parsing, when done for an OptionGroup, would just start parsing options incorrectly by omitting table entries and it would end up setting the wrong option value
llvm-svn: 169189
Make breakpoint setting by file and line much more efficient by only looking for inlined breakpoint locations if we are setting a breakpoint in anything but a source implementation file. Implementing this complex for a many reasons. Turns out that parsing compile units lazily had some issues with respect to how we need to do things with DWARF in .o files. So the fixes in the checkin for this makes these changes:
- Add a new setting called "target.inline-breakpoint-strategy" which can be set to "never", "always", or "headers". "never" will never try and set any inlined breakpoints (fastest). "always" always looks for inlined breakpoint locations (slowest, but most accurate). "headers", which is the default setting, will only look for inlined breakpoint locations if the breakpoint is set in what are consudered to be header files, which is realy defined as "not in an implementation source file".
- modify the breakpoint setting by file and line to check the current "target.inline-breakpoint-strategy" setting and act accordingly
- Modify compile units to be able to get their language and other info lazily. This allows us to create compile units from the debug map and not have to fill all of the details in, and then lazily discover this information as we go on debuggging. This is needed to avoid parsing all .o files when setting breakpoints in implementation only files (no inlines). Otherwise we would need to parse the .o file, the object file (mach-o in our case) and the symbol file (DWARF in the object file) just to see what the compile unit was.
- modify the "SymbolFileDWARFDebugMap" to subclass lldb_private::Module so that the virtual "GetObjectFile()" and "GetSymbolVendor()" functions can be intercepted when the .o file contenst are later lazilly needed. Prior to this fix, when we first instantiated the "SymbolFileDWARFDebugMap" class, we would also make modules, object files and symbol files for every .o file in the debug map because we needed to fix up the sections in the .o files with information that is in the executable debug map. Now we lazily do this in the DebugMapModule::GetObjectFile()
Cleaned up header includes a bit as well.
llvm-svn: 162860
For the types directory, we were running lldbtest.system() to execute the compiled program
on the test host to collect golden output in order to compare with the output of various
lldb debugger commands as performed later. This won't work for the remote platform
scenario where the architecture of the target and host platforms are different.
Modify the AbstractBase class to use lldb to launch the inferior while specifying the
output file, from which the golden output is collected and grokked. How to bootstrap and
to connect to the remote platform is still being worked at.
llvm-svn: 154699
Assign the test method name to self.testMethodName. This can be useful for the
test directory (see test/types for a good example) which houses a bunch of executables
compiled from different source files. The default build action is to create a.out as
the binary executable, which can confuse the module cacheing mechanism and result in
the debugger getting a stale image as the target to be debugged, and chaos ensues.
o AbstractBase.py, TestThreadAPI.py:
Use self.testMethodName to our advantage.
o TestLoadUnload.py:
Add expected failure marker to test case test_modules_search_paths().
llvm-svn: 133768
employ different executable names when creating the target for lldb to debug.
Comparing the log files for the same test case with success/failure results show
that the failed one was setting the breakpoint on some different address than the
succeeded one, leading us to suspect that the file system maybe think it knows better
and caches the wrong executable file (they were all named 'a.out' before) which lldb
subsequently reads from.
Now './dotest.py -v types' passes without specifying the '-w' option.
rdar://problem/9644488
rdar://problem/9649573
llvm-svn: 133649
with no delay between successive test cases. This one happend to capture a test failure where
the inferior shoud stop at a breakpoint but did not (radar will be filed to capture the log files
as well as the test session files):
[12:40:37] johnny:/Volumes/data/lldb/svn/trunk/test $ DEBUG_LLDB_LOG=/tmp/lldb.log DEBUG_GDB_REMOTE_LOG=/tmp/gdb-remote.log ./dotest.py -v -p TestFailures.py
LLDB-63
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 133529
Node Kind: directory
Schedule: normal
Last Changed Author: jmolenda
Last Changed Rev: 133500
Last Changed Date: 2011-06-20 19:57:15 -0700 (Mon, 20 Jun 2011)
Session logs for test failures/errors/unexpected successes will go into directory '2011-06-21-12_40_42'
Command invoked: python ./dotest.py -v -p TestFailures.py
----------------------------------------------------------------------
Collected 10 tests
1: test_char_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that char-type variables are displayed correctly. ... ok
2: test_char_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that char-type variables are displayed correctly. ... ok
3: test_int_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that int-type variables are displayed correctly. ... ok
4: test_int_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that int-type variables are displayed correctly. ... ok
5: test_long_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that 'long long'-type variables are displayed correctly. ... FAIL
6: test_long_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that 'long long'-type variables are displayed correctly. ... ok
7: test_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly. ... TestFailures.DebugIntegerTypesFailures.test_long_type_with_dsym
FAIL
8: test_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly. ... FAIL
9: test_short_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that short-type variables are displayed correctly. ... ok
10: test_short_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that short-type variables are displayed correctly. ... ok
======================================================================
FAIL: test_long_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that 'long long'-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 102, in test_long_long_type_with_dsym
self.long_long_type()
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 129, in long_long_type
self.generic_type_tester(set(['long long']))
File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
"stop reason = breakpoint"])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint
======================================================================
FAIL: test_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 87, in test_long_type_with_dsym
self.long_type()
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 125, in long_type
self.generic_type_tester(set(['long']))
File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
"stop reason = breakpoint"])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint
======================================================================
FAIL: test_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
Test that long-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 94, in test_long_type_with_dwarf
self.long_type()
File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 125, in long_type
self.generic_type_tester(set(['long']))
File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
"stop reason = breakpoint"])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint
----------------------------------------------------------------------
Ran 10 tests in 5.617s
FAILED (failures=3)
Session logs for test failures/errors/unexpected successes can be found in directory '2011-06-21-12_40_42'
[12:40:47] johnny:/Volumes/data/lldb/svn/trunk/test $
llvm-svn: 133537
if not already specified by the test driver (via ./dotest -w). Remove the AbstractBase.setUp()
method definition when/if we find out the cause of the failures if no delays are inserted
between these test cases.
llvm-svn: 133495
We still have the the issue where running:
./dotest.py -v types
we have test failures where the inferior either runs to exited with status 0 or
the inferior stops but not because of breakpoint (for example):
runCmd: process status
output: Process 90060 stopped
* thread #1: tid = 0x2d03, 0x000000010000e2ca, stop reason = EXC_BAD_ACCESS (code=2, address=0x10000e2ca)
frame #0: 0x000000010000e2ca
There are two cases where the inferior stops for the breakpoint (good), but the expression parser
prints out the wrong information. The two failures are:
Failure-TestFloatTypesExpr.FloatTypesExprTestCase.test_double_type_with_dsym.log
Failure-TestFloatTypesExpr.FloatTypesExprTestCase.test_double_type_with_dwarf.log
I'll file a radar on the two expression parser misbehave, while continue investigating why the
inferior stops for the wrong reason or does not stop at all.
For now, you'll need to do:
./dotest.py -v -w types
llvm-svn: 133488
rdar://problem/9173060 lldb hangs while running unique-types
disappears if running with clang version >= 3. Modify the TestUniqueTypes.py
to detect if we are running with clang version < 3 and, if true, skip the test.
Update the lldbtest.system() function to return a tuple of (stdoutdata, stderrdata)
since we need the stderr data from "clang -v" command. Modify existing clients of
lldbtest.system() to now use, for example:
# First, capture the golden output emitted by the oracle, i.e., the
# series of printf statements.
- go = system("./a.out", sender=self)
+ go = system("./a.out", sender=self)[0]
# This golden list contains a list of (variable, value) pairs extracted
# from the golden output.
gl = []
And add two utility functions to lldbutil.py.
llvm-svn: 128162
Don't overwhelm the expression mechanism. Previously, there was no delay
between successive invocations of 'expression' command. The test suite
run has experienced 'Resume timed out' or similar error messages. Now we
currently add a sleep of 1.0 second before issuing the next 'expression'
command.
llvm-svn: 120626
Also modify the:
self.runCmd("expr %s" % var)
to be:
self.runCmd("expression %s" % var)
since 'expr' is an alias of expression. And fix the assert message to be either
'frame variable -t %s' or 'expression %s'.
llvm-svn: 118085
The failures that remain are:
# rdar://problem/8620735 test/types: frame variable -t a_class_ref.m_b fails
# The reference type related failures that remain are:
notnow = set(['a_class_ref.m_b',
'a_struct_ref.b',
'a_union_nonzero_ref.u.a'])
llvm-svn: 118021
Tamas Berghammer