Commit Graph

226 Commits

Author SHA1 Message Date
Chaoren Lin f62b7fc317 Fix typo. platfrom -> platform
llvm-svn: 236543
2015-05-05 20:34:36 +00:00
Robert Flack 068898cb83 Fix @skipUnlessPlatform and @skipIfPlatform decorators to work on classes.
The decorators to skip if or unless certain platforms relied on calling
skipTest, which only worked if they were decorating individual test cases.
However for decorating entire classes it needs different behavior. This behavior
is already encapsulated in unittest2.skipIf and unittest2.skipUnless so this
modifies the platform decorators to use these unittest skip decorators.

Test Plan:
./dotest.py -t -p TestObjCMethods2.py
See skipped tests on linux, and see tests run on macosx.

Differential Revision: http://reviews.llvm.org/D8903

llvm-svn: 234514
2015-04-09 18:07:58 +00:00
Chaoren Lin d702203c5e Use insert instead of append when adding to sys.path.
Similar problem as in the last part of r200486.

llvm-svn: 234071
2015-04-04 01:11:31 +00:00
Zachary Turner 43bac098b0 [dotest.py] Present a better error when lldb module can't be found.
We try to run lldb -P and jump through a bunch of other hoops
to figure out where the Python module is so that we can add its
location to sys.path.  The error message printed when this
doesn't work was left over from the time when Mac was the only
platform, so it presented a Mac-specific diagnostic on all
platforms related to the LLDB Framework.  This patch prints
a more useful diagnostic when not on Mac that gives a number
of other diagnostic messages to check.

llvm-svn: 234049
2015-04-03 20:56:15 +00:00
Tamas Berghammer de7867938b Fix TestGdbRemote* on remote linux
Differential revision: http://reviews.llvm.org/D8454

llvm-svn: 233531
2015-03-30 10:52:32 +00:00
Adrian McCarthy b016b3cfea Fix lld command on Windows for many tests.
llvm-svn: 233416
2015-03-27 20:47:35 +00:00
Robert Flack efa49c2665 Change expectedFailureOS and skipIfPlatform checks to be based on target platform when running test suite remotely.
Most expected OS failures or skipped tests are about the target platform on
which the test binary is being run, not the host platform launching the tests.
This changes expectedFailureOS and skipIfPlatform to check against the remote
platform when running remote tests.

Test Plan:
Run ./do_sep.py test suite against a remote target on a different platform and
verify that tests which should be excluded on the remote platform are excluded.

Differential Revision: http://reviews.llvm.org/D8611

llvm-svn: 233311
2015-03-26 19:34:26 +00:00
Ilia K ba6d7b6004 Add lldb-mi/lldb-server test folders to PATH before in dotest.py
Summary:
This patch allows not specify search path in each lldb-mi test. It makes tests easier.

This fix was requested by vharron.

All test pass on OS X.

Reviewers: vharron, clayborg

Subscribers: lldb-commits, vharron

Differential Revision: http://reviews.llvm.org/D8207

llvm-svn: 232019
2015-03-12 07:32:32 +00:00
Robert Flack 8cc4cf178c Rename test/tools/lldb-gdbserver to test/tools/lldb-server
As requested in http://reviews.llvm.org/D7545 this change moves test/tools/lldb-gdbserver to test/tools/lldb-server ot match the name of the target being tested.

Differential Revision: http://reviews.llvm.org/D8061

llvm-svn: 231479
2015-03-06 14:36:33 +00:00
Tamas Berghammer 6723ecb552 Add option to set environment variables from dotest.py
Setting environment variables are required when testing on a remote
target requiring different flags then the ones specified on the host OS.

Differential revision: http://reviews.llvm.org/D7854

llvm-svn: 230485
2015-02-25 12:22:40 +00:00
Ed Maste 55f74e829b test: Add workaround for FreeBSD libedit vs readline issue
If stdout is not a terminal Python executes
    rl_variable_bind ("enable-meta-key", "off");

This produces a warning with FreeBSD's libedit because the
enable-meta-key variable is unknown.  Not an issue on Apple because
cpython commit f0ab6f9f0603 added a #ifndef __APPLE__ around the call.
See http://bugs.python.org/issue19884 for more information.

For now we just discard the warning output to get the tests working
again on FreeBSD.

llvm-svn: 226037
2015-01-14 20:32:33 +00:00
Stephane Sezer 24a14c93e6 Modify dotest.py to be able to run without an lldb build.
Summary: This will ease llgs development a bit by not requiring an lldb/lldb.py build to launch the tests. Also, we can now use LLDB_DEBUGSERVER_PATH to point to a debug server to use to run the tests. I used that to point to a ds2 build and run llgs tests against ds2.

Reviewers: clayborg, tfiala, vharron

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D6554

llvm-svn: 225549
2015-01-09 21:54:27 +00:00
Enrico Granata 9aa7e8e9ff Dynamic values have been around (and stable and reliable) for long enough that we can turn them on by default
Change the default of prefer-dynamic-value to eDynamicDontRunTarget (i.e. enable dynamic values, but do not run code to do so)

Of course, disable this for the test suite, since testing no-dynamic-values is actually valuable

Fixes rdar://17363061

llvm-svn: 225486
2015-01-09 00:47:24 +00:00
Hafiz Abid Qadeer 547cb78774 Fixed 2 typos in help.
llvm-svn: 224331
2014-12-16 10:20:35 +00:00
Zachary Turner 555a7a6ad2 Add a method to disable the Windows crash / assert dialogs.
When running the test suite on Windows, we can't have Windows popping
up dialogs when LLDB crashes in native code because it will hang
the test suite.  This patch silences those dialogs by checking an
environment variable at startup and configuring Windows based on
its value.

This patch also adds an environment variable to force inferiors to
never spawn in their own console window.  This is useful to prevent
new window spawm when running the test suite.

Reviewed by: Scott Graham

Differential Revision: http://reviews.llvm.org/D6628

llvm-svn: 224137
2014-12-12 18:10:52 +00:00
Stephane Sezer 7a1d22a2af Fix a crash in dotest.py when the lldb executable is not built.
Summary: If lldb is not built, dotest.py throws an exception because we are using an unset variable.

Reviewers: clayborg

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D6516

llvm-svn: 223446
2014-12-05 05:02:41 +00:00
Hafiz Abid Qadeer 1cbac4e94f Add initial lldb-mi tests.
Test 'test_lldbmi_interrupt' is only enabled for Darwin as
it seems to cause a timeout error on Linux.
Patch from dawn@burble.org.

llvm-svn: 222750
2014-11-25 10:41:57 +00:00
Greg Clayton ce0b8e0fc9 Fix some issue with running the test suite remotely on iOS.
- Added a new "--apple-sdk" flag that can be specified on Darwin only so the correct cross compilers can be auto-selected without having to specify the "--compiler" flag.
- Set SDKROOT if needed

llvm-svn: 222153
2014-11-17 18:32:17 +00:00
Greg Clayton 2dd4365038 Make sure stderr is filtered out in case xcodebuild or xcrun print errors when getting the SDK path and use xcrun to find the SDK path.
llvm-svn: 221492
2014-11-06 22:55:09 +00:00
Todd Fiala d7e27eac19 Fix dotest.py test runner exit code to return non-zero on failure/error.
This addresses this bug:
http://www.llvm.org/bugs/show_bug.cgi?id=21267

Which has been broken since svn r215256 on Aug 8 2014.

DO NOT REVERT THIS COMMIT EVEN IF IT CREATES TEST FAILURES.

The test failures are a result of accumulation of hidden failures
that were masked by the bug this change fixes.  Most of our test
runners as part of build testing rely on dotest.py returning non-zero to indicate
some kind of errant test run.  Thus, we have been flying blind
since Aug 8 2014.

llvm-svn: 219689
2014-10-14 16:17:02 +00:00
Zachary Turner e96838e304 Fix some python shutdown / ordering issues.
Differential Revision: http://reviews.llvm.org/D4826

Reviewed by: Enrico Granata

llvm-svn: 215256
2014-08-08 23:20:25 +00:00
Zachary Turner 7c1bc2b8ae Make CMake choose the target architecture according to the build.
Previously, CMake was invoking the test runner and not specifying
what architecture to use when building test executables.  The
Makefiles for the test executables then had logic to choose x64
by default.  This doesn't work on Windows because the test compiler
would then try to link against the 64-bit MSVCRT and not find them
since only the 32-bit MSVCRT was in the path.

This patch addresses this by figuring out, at CMake time, whether
or not you are building LLDB with a 64 or 32-bit toolchain.  Then,
it explicitly passes this value to the test runner, causing the
test runner to build tests whose architecture matches that of LLDB
itself.  This can still be overridden by setting the CMake variable
LLDB_TEST_EXECUTABLE_ARCH=(x64|x86)

llvm-svn: 214443
2014-07-31 21:07:41 +00:00
Zachary Turner c9bf0c70b5 Make lldb -P work on Windows.
lldb -P, which outputs its python path, works by using Host-layer
facilities to get information about the loaded python module.  This
Host functionality was unimplemented on Windows, so this patch
implements it.  Additionally, it removes a pexpect dependency from
the test runner and uses an equivalent invocation of subprocess.

Reviewed by: Todd Fiala

Differential Revision: http://reviews.llvm.org/D4548

llvm-svn: 213410
2014-07-18 20:36:08 +00:00
Zachary Turner 045fde58d1 Fixes a number of issue related to test portability on Windows.
99% of this CL is simply moving calls to "import pexpect" to a more
narrow scope - i.e. the function that actually runs a particular
test.  This way the test suite can run on Windows, which doesn't have
pexpect, and the individual tests that use pexpect can be disabled on
a platform-specific basis.

Additionally, this CL fixes a few other cases of non-portability.
Notably, using "ps" to get the command line, and os.uname() to
determine the architecture don't work on Windows.  Finally, this
also adds a stubbed out builder_win32 module.

The full test suite runs correctly on Windows after this CL, although
there is still some work remaining on the C++ side to fix one-shot
script commands from LLDB (e.g. script print "foo"), which currently
deadlock.

Reviewed by: Todd Fiala

Differential Revision: http://reviews.llvm.org/D4573

llvm-svn: 213343
2014-07-18 01:02:02 +00:00
Jason Molenda 277335f9b2 Add a new 'stresstest' category, set the api/multiple-debuggers test case as a stresstest.
llvm-svn: 212673
2014-07-10 02:21:16 +00:00
Todd Fiala 9189e034f1 Remove hack from dotest.py to fix extra output on -P command, and fix MacOSX multi-threaded test run collisions on crashlog.dylib.
On MacOSX, we need to adjust the way we clean up the crashlog dylib in deleteCrashInfoDylib().
Right now it is only geared to run one test at a time.  For now I'm just skipping the delete.
I'll work with Apple on a fix that handles this.  It seems to only cause one dylib total to
hang around that might otherwise have been deleted.  Fixes MacOSX multiple tests running
at the same time.  (I didn't hit this on Yosemite, might be an issue that only shows up
on Mavericks?)

llvm-svn: 212548
2014-07-08 16:29:54 +00:00
Todd Fiala 3f0a360f18 Enable multi-process testing for MacOSX.
This change modifies the way the multi-threaded test runner works.
It uses the Python multiprocessing library rather than the threading
library.  Investigation showed that all MacOSX threads were waiting on
the global python lock when using the threading approach.  Not sure
why that differed from the Linux/FreeBSD implementations.

The new approach uses the multiprocessing library's Pool class.  It's
mildly cleaner than the other version, runs multithreaded on MacOSX,
and seems to have caused no performance regression on Linux.  The
worker thread logic is simpler with the Pool managing the worker
processes.

This also includes a minor change to the test runner's python
lldb dir logic using the -P option.  It now looks at the last line
of output rather than the first line.  This covers part of the issue
of extra options validation logic getting spit out.  The test runner
will now pick up the right python library directory.  It does not
fix all the issues, though, as a ton of tests (50+ on Linux) are
failing due to unexpected output when running lldb.

llvm-svn: 212513
2014-07-08 06:42:37 +00:00
Bruce Mitchener d93c4a3339 Fix typos.
llvm-svn: 212132
2014-07-01 21:22:11 +00:00
Todd Fiala a41d48cac8 Added two new test types: @debugserver_test and @llgs_test.
TestLldbGdbServer now supports both lldb-gdbserver (llgs) and
debugserver tests.  Similar to the dsym/dwarf tests, they allow
running the same underlying gdb remote protocol tests against
lldb-gdbserver and debugserver.  This will help make sure the
protocol-level tests for lldb-gdbserver faithfully represent
what debugserver does on OS X.

Switched back gdb remote protocol test logging to warning
and above (accidentally submitted it at debug level in a
recent commit).

llvm-svn: 207395
2014-04-28 04:49:40 +00:00
Ed Maste 1087f2980d Add errno import missed in r205246
llvm-svn: 205306
2014-04-01 12:49:21 +00:00
Ed Maste ce5ac7d776 Workaround test trace output dir creation race condition
Since dosep.ty started invoking multiple tests in parallel, the FreeBSD
buildbot occasionally has a failure due to os.mkdir returning EEXIST.
Silently ignore that exception, but reraise any other.

llvm-svn: 205246
2014-03-31 20:36:38 +00:00
Todd Fiala 3452df6723 Fixed b18655: cleaned up script interpreter file reference handling.
This change addresses shutdown crashes in the python lldb module when
the script interpreter was hanging on to saved file references after
leaving a session.  It also gets rid of extra references to the
stdin/stdout/stderr python file objects that are created when entering
the session.

This change also moves the bundled pyexpect 2.4 library to the front
of the python library path so that a python distribution default
pyexpect (2.3 in Ubuntu 12.04) is not picked up first.

llvm-svn: 200486
2014-01-30 20:19:22 +00:00
Ed Maste e8b2ef56e1 Handle LLDB_LIB_DIR on FreeBSD as on Linux
Diagnosed and tested by Mike Ma.
http://llvm.org/pr18191

llvm-svn: 199549
2014-01-18 14:22:28 +00:00
Enrico Granata 739050236e Having binary files in a repository is not a good thing
With this checkin, we use the installed clang compiler to build crashinfo.so from crashinfo.c upon every test suite execution
We also try to cleanup after ourselves, which of course will only work if the test suite does not actually crash

llvm-svn: 197106
2013-12-12 01:42:17 +00:00
Enrico Granata 2c3f140551 <rdar://problem/15640353>
Add an hook for the test suite into the OSX-only CrashReporter "App-specific info"

This allows the test suite to set the crash info to the name and file location of every test as the test gets executed
If the test suite crashes, the crash log will then report which test is the culprit, even when not using verbose mode

This only works on OSX, and defaults to not doing anything on other platforms, but OS/platform-specific invocations
can be devised by each individual platform

llvm-svn: 197095
2013-12-12 00:02:05 +00:00
Ed Maste a2b2465e45 test: Fix punctuation in exceptions for log env vars
The Exception text was missing a closing parenthesis, and for consistency
with other Exceptions it should not end with a period.

llvm-svn: 195659
2013-11-25 16:57:16 +00:00
Greg Clayton fb90931b60 Improved platform support.
Improved the detection of a valid GDB server where we actually can connect to a socket, but then it doesn't read or write anything (which happens with some USB mux software).

Host::MakeDirectory() now can make as many intermediate directories as needed.

The testsuite now has very initial support for remote test suite running. When running on a remote platform, the setUp function for the test will make a new directory and select it as the working directory on the remote host. 

Added a common function that can be used to create the short option string for getopt_long calls.

llvm-svn: 195541
2013-11-23 01:58:15 +00:00
Enrico Granata dc4db5a6eb <rdar://problem/15144376>
This commit reimplements the TypeImpl class (the class that backs SBType) in terms of a static,dynamic type pair

This is useful for those cases when the dynamic type of an ObjC variable can only be obtained in terms of an "hollow" type with no ivars
In that case, we could either go with the static type (+iVar information) or with the dynamic type (+inheritance chain)

With the new TypeImpl implementation, we try to combine these two sources of information in order to extract as much information as possible
This should improve the functionality of tools that are using the SBType API to do extensive dynamic type inspection

llvm-svn: 193564
2013-10-29 00:28:35 +00:00
Enrico Granata 52b4b6cddc This is the last piece of work for "formats in categories": we now cache formats as well as summaries and synthetics
llvm-svn: 192928
2013-10-17 22:27:19 +00:00
Matt Kopec 326bfc5a9e Enabling 32-bit tests for clang on Linux by default (ie. for the buildbot.).
llvm-svn: 191714
2013-09-30 23:33:43 +00:00
Daniel Malea 1efb418c9d Improve stability of Linux ProcessMonitor by not using fds for synchronization:
- ProcessMonitor::[Do|Serve]Operation no longer depend on file descriptors!
- removed unused member functions CloseFD and EnableIPC
- add semaphores to signal when an Operation is ready to be processed/complete.

This commit fixes a bug that was identified under stress-testing (i.e. build
LLVM while running tests) that led to LLDB becoming unresponsive because the
read/write operations on file descriptors in ProcessMonitor were not checked.

Other test runner improvement/convenience:
- pickup environment variables LLDB_LINUX_LOG and LLDB_LINUX_LOG_OPTIONS to
  enable (Linux) logging when running the test suite. Example usage:

        $ LLDB_LINUX_LOG="mylog.txt" LLDB_LINUX_LOG_OPTIONS="process thread" python dotest.py

llvm-svn: 190820
2013-09-16 23:12:18 +00:00
Daniel Malea e0f8f574c7 merge lldb-platform-work branch (and assorted fixes) into trunk
Summary:
    This merge brings in the improved 'platform' command that knows how to
    interface with remote machines; that is, query OS/kernel information, push
    and pull files, run shell commands, etc... and implementation for the new
    communication packets that back that interface, at least on Darwin based
    operating systems via the POSIXPlatform class. Linux support is coming soon.

    Verified the test suite runs cleanly on Linux (x86_64), build OK on Mac OS
    X Mountain Lion.

    Additional improvements (not in the source SVN branch 'lldb-platform-work'):
    - cmake build scripts for lldb-platform
    - cleanup test suite
    - documentation stub for qPlatform_RunCommand
    - use log class instead of printf() directly
    - reverted work-in-progress-looking changes from test/types/TestAbstract.py that work towards running the test suite remotely.
    - add new logging category 'platform'

    Reviewers: Matt Kopec, Greg Clayton

    Review: http://llvm-reviews.chandlerc.com/D1493

llvm-svn: 189295
2013-08-26 23:57:52 +00:00
Daniel Malea 9115f07465 Allow building test suite with non-default libc++
- add new "--libcxx" parameter to dotest.py to specify path to custom libc++

llvm-svn: 187802
2013-08-06 15:02:32 +00:00
Stefanus Du Toit 6809d79cc6 Allow test categories to be skipped.
This adds a new parameter, --skip-category, that can be used to list
categories that should be skipped. For example, to run all tests except for
Objective-C ones, one can now write:

  ./dotest.py --skip-category objc [...]

llvm-svn: 187590
2013-08-01 17:59:20 +00:00
Daniel Malea bbd174b6f0 Fix problematic override _exc_info_to_string
- pass through to base-class implementation when raised exception is not from an LLDBTest
- should make the test suite errors a little easier to root-cause

llvm-svn: 187450
2013-07-30 21:28:32 +00:00
Daniel Malea 7a433802cf Fix python 2.6 compatibility issue introduced by r184615
- argparse_compat library does not support reading environment variables
- should unblock Linux GCC buildbot from running tests again

llvm-svn: 185567
2013-07-03 18:50:03 +00:00
Enrico Granata 7934a878a0 More useful printout of arguments after the recent changes to support LLDB_TEST_ARGUMENTS
llvm-svn: 185246
2013-06-29 01:53:55 +00:00
Ed Maste 3d3295daab Skip dsym tests also on FreeBSD
llvm-svn: 184874
2013-06-25 19:59:56 +00:00
Enrico Granata 091dd7be12 Test suite support for setting arguments through the environment
Set your env variable LLDB_TEST_ARGUMENTS to one or more options to be passed to the lldb test suite and those will be picked automatically

No more fighting about whether the progress bar is good or bad :-)

llvm-svn: 184615
2013-06-22 00:15:25 +00:00
Enrico Granata 6d2cfb80ba Huge speedup for testsuite categories
This ensures that we won't try to do cleanups of test cases that we are skipping
e.g. this brings down the time required to run the cmdline category on my machine from ~70s to ~30s

llvm-svn: 184363
2013-06-19 21:48:09 +00:00