Most expected OS failures or skipped tests are about the target platform on
which the test binary is being run, not the host platform launching the tests.
This changes expectedFailureOS and skipIfPlatform to check against the remote
platform when running remote tests.
Test Plan:
Run ./do_sep.py test suite against a remote target on a different platform and
verify that tests which should be excluded on the remote platform are excluded.
Differential Revision: http://reviews.llvm.org/D8611
llvm-svn: 233311
Summary:
This patch allows not specify search path in each lldb-mi test. It makes tests easier.
This fix was requested by vharron.
All test pass on OS X.
Reviewers: vharron, clayborg
Subscribers: lldb-commits, vharron
Differential Revision: http://reviews.llvm.org/D8207
llvm-svn: 232019
Setting environment variables are required when testing on a remote
target requiring different flags then the ones specified on the host OS.
Differential revision: http://reviews.llvm.org/D7854
llvm-svn: 230485
If stdout is not a terminal Python executes
rl_variable_bind ("enable-meta-key", "off");
This produces a warning with FreeBSD's libedit because the
enable-meta-key variable is unknown. Not an issue on Apple because
cpython commit f0ab6f9f0603 added a #ifndef __APPLE__ around the call.
See http://bugs.python.org/issue19884 for more information.
For now we just discard the warning output to get the tests working
again on FreeBSD.
llvm-svn: 226037
Summary: This will ease llgs development a bit by not requiring an lldb/lldb.py build to launch the tests. Also, we can now use LLDB_DEBUGSERVER_PATH to point to a debug server to use to run the tests. I used that to point to a ds2 build and run llgs tests against ds2.
Reviewers: clayborg, tfiala, vharron
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D6554
llvm-svn: 225549
Change the default of prefer-dynamic-value to eDynamicDontRunTarget (i.e. enable dynamic values, but do not run code to do so)
Of course, disable this for the test suite, since testing no-dynamic-values is actually valuable
Fixes rdar://17363061
llvm-svn: 225486
When running the test suite on Windows, we can't have Windows popping
up dialogs when LLDB crashes in native code because it will hang
the test suite. This patch silences those dialogs by checking an
environment variable at startup and configuring Windows based on
its value.
This patch also adds an environment variable to force inferiors to
never spawn in their own console window. This is useful to prevent
new window spawm when running the test suite.
Reviewed by: Scott Graham
Differential Revision: http://reviews.llvm.org/D6628
llvm-svn: 224137
Summary: If lldb is not built, dotest.py throws an exception because we are using an unset variable.
Reviewers: clayborg
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D6516
llvm-svn: 223446
- Added a new "--apple-sdk" flag that can be specified on Darwin only so the correct cross compilers can be auto-selected without having to specify the "--compiler" flag.
- Set SDKROOT if needed
llvm-svn: 222153
This addresses this bug:
http://www.llvm.org/bugs/show_bug.cgi?id=21267
Which has been broken since svn r215256 on Aug 8 2014.
DO NOT REVERT THIS COMMIT EVEN IF IT CREATES TEST FAILURES.
The test failures are a result of accumulation of hidden failures
that were masked by the bug this change fixes. Most of our test
runners as part of build testing rely on dotest.py returning non-zero to indicate
some kind of errant test run. Thus, we have been flying blind
since Aug 8 2014.
llvm-svn: 219689
Previously, CMake was invoking the test runner and not specifying
what architecture to use when building test executables. The
Makefiles for the test executables then had logic to choose x64
by default. This doesn't work on Windows because the test compiler
would then try to link against the 64-bit MSVCRT and not find them
since only the 32-bit MSVCRT was in the path.
This patch addresses this by figuring out, at CMake time, whether
or not you are building LLDB with a 64 or 32-bit toolchain. Then,
it explicitly passes this value to the test runner, causing the
test runner to build tests whose architecture matches that of LLDB
itself. This can still be overridden by setting the CMake variable
LLDB_TEST_EXECUTABLE_ARCH=(x64|x86)
llvm-svn: 214443
lldb -P, which outputs its python path, works by using Host-layer
facilities to get information about the loaded python module. This
Host functionality was unimplemented on Windows, so this patch
implements it. Additionally, it removes a pexpect dependency from
the test runner and uses an equivalent invocation of subprocess.
Reviewed by: Todd Fiala
Differential Revision: http://reviews.llvm.org/D4548
llvm-svn: 213410
99% of this CL is simply moving calls to "import pexpect" to a more
narrow scope - i.e. the function that actually runs a particular
test. This way the test suite can run on Windows, which doesn't have
pexpect, and the individual tests that use pexpect can be disabled on
a platform-specific basis.
Additionally, this CL fixes a few other cases of non-portability.
Notably, using "ps" to get the command line, and os.uname() to
determine the architecture don't work on Windows. Finally, this
also adds a stubbed out builder_win32 module.
The full test suite runs correctly on Windows after this CL, although
there is still some work remaining on the C++ side to fix one-shot
script commands from LLDB (e.g. script print "foo"), which currently
deadlock.
Reviewed by: Todd Fiala
Differential Revision: http://reviews.llvm.org/D4573
llvm-svn: 213343
On MacOSX, we need to adjust the way we clean up the crashlog dylib in deleteCrashInfoDylib().
Right now it is only geared to run one test at a time. For now I'm just skipping the delete.
I'll work with Apple on a fix that handles this. It seems to only cause one dylib total to
hang around that might otherwise have been deleted. Fixes MacOSX multiple tests running
at the same time. (I didn't hit this on Yosemite, might be an issue that only shows up
on Mavericks?)
llvm-svn: 212548
This change modifies the way the multi-threaded test runner works.
It uses the Python multiprocessing library rather than the threading
library. Investigation showed that all MacOSX threads were waiting on
the global python lock when using the threading approach. Not sure
why that differed from the Linux/FreeBSD implementations.
The new approach uses the multiprocessing library's Pool class. It's
mildly cleaner than the other version, runs multithreaded on MacOSX,
and seems to have caused no performance regression on Linux. The
worker thread logic is simpler with the Pool managing the worker
processes.
This also includes a minor change to the test runner's python
lldb dir logic using the -P option. It now looks at the last line
of output rather than the first line. This covers part of the issue
of extra options validation logic getting spit out. The test runner
will now pick up the right python library directory. It does not
fix all the issues, though, as a ton of tests (50+ on Linux) are
failing due to unexpected output when running lldb.
llvm-svn: 212513
TestLldbGdbServer now supports both lldb-gdbserver (llgs) and
debugserver tests. Similar to the dsym/dwarf tests, they allow
running the same underlying gdb remote protocol tests against
lldb-gdbserver and debugserver. This will help make sure the
protocol-level tests for lldb-gdbserver faithfully represent
what debugserver does on OS X.
Switched back gdb remote protocol test logging to warning
and above (accidentally submitted it at debug level in a
recent commit).
llvm-svn: 207395
Since dosep.ty started invoking multiple tests in parallel, the FreeBSD
buildbot occasionally has a failure due to os.mkdir returning EEXIST.
Silently ignore that exception, but reraise any other.
llvm-svn: 205246
This change addresses shutdown crashes in the python lldb module when
the script interpreter was hanging on to saved file references after
leaving a session. It also gets rid of extra references to the
stdin/stdout/stderr python file objects that are created when entering
the session.
This change also moves the bundled pyexpect 2.4 library to the front
of the python library path so that a python distribution default
pyexpect (2.3 in Ubuntu 12.04) is not picked up first.
llvm-svn: 200486
With this checkin, we use the installed clang compiler to build crashinfo.so from crashinfo.c upon every test suite execution
We also try to cleanup after ourselves, which of course will only work if the test suite does not actually crash
llvm-svn: 197106
Add an hook for the test suite into the OSX-only CrashReporter "App-specific info"
This allows the test suite to set the crash info to the name and file location of every test as the test gets executed
If the test suite crashes, the crash log will then report which test is the culprit, even when not using verbose mode
This only works on OSX, and defaults to not doing anything on other platforms, but OS/platform-specific invocations
can be devised by each individual platform
llvm-svn: 197095
Improved the detection of a valid GDB server where we actually can connect to a socket, but then it doesn't read or write anything (which happens with some USB mux software).
Host::MakeDirectory() now can make as many intermediate directories as needed.
The testsuite now has very initial support for remote test suite running. When running on a remote platform, the setUp function for the test will make a new directory and select it as the working directory on the remote host.
Added a common function that can be used to create the short option string for getopt_long calls.
llvm-svn: 195541
This commit reimplements the TypeImpl class (the class that backs SBType) in terms of a static,dynamic type pair
This is useful for those cases when the dynamic type of an ObjC variable can only be obtained in terms of an "hollow" type with no ivars
In that case, we could either go with the static type (+iVar information) or with the dynamic type (+inheritance chain)
With the new TypeImpl implementation, we try to combine these two sources of information in order to extract as much information as possible
This should improve the functionality of tools that are using the SBType API to do extensive dynamic type inspection
llvm-svn: 193564
- ProcessMonitor::[Do|Serve]Operation no longer depend on file descriptors!
- removed unused member functions CloseFD and EnableIPC
- add semaphores to signal when an Operation is ready to be processed/complete.
This commit fixes a bug that was identified under stress-testing (i.e. build
LLVM while running tests) that led to LLDB becoming unresponsive because the
read/write operations on file descriptors in ProcessMonitor were not checked.
Other test runner improvement/convenience:
- pickup environment variables LLDB_LINUX_LOG and LLDB_LINUX_LOG_OPTIONS to
enable (Linux) logging when running the test suite. Example usage:
$ LLDB_LINUX_LOG="mylog.txt" LLDB_LINUX_LOG_OPTIONS="process thread" python dotest.py
llvm-svn: 190820
Summary:
This merge brings in the improved 'platform' command that knows how to
interface with remote machines; that is, query OS/kernel information, push
and pull files, run shell commands, etc... and implementation for the new
communication packets that back that interface, at least on Darwin based
operating systems via the POSIXPlatform class. Linux support is coming soon.
Verified the test suite runs cleanly on Linux (x86_64), build OK on Mac OS
X Mountain Lion.
Additional improvements (not in the source SVN branch 'lldb-platform-work'):
- cmake build scripts for lldb-platform
- cleanup test suite
- documentation stub for qPlatform_RunCommand
- use log class instead of printf() directly
- reverted work-in-progress-looking changes from test/types/TestAbstract.py that work towards running the test suite remotely.
- add new logging category 'platform'
Reviewers: Matt Kopec, Greg Clayton
Review: http://llvm-reviews.chandlerc.com/D1493
llvm-svn: 189295
This adds a new parameter, --skip-category, that can be used to list
categories that should be skipped. For example, to run all tests except for
Objective-C ones, one can now write:
./dotest.py --skip-category objc [...]
llvm-svn: 187590
- pass through to base-class implementation when raised exception is not from an LLDBTest
- should make the test suite errors a little easier to root-cause
llvm-svn: 187450
Set your env variable LLDB_TEST_ARGUMENTS to one or more options to be passed to the lldb test suite and those will be picked automatically
No more fighting about whether the progress bar is good or bad :-)
llvm-svn: 184615
This ensures that we won't try to do cleanups of test cases that we are skipping
e.g. this brings down the time required to run the cmdline category on my machine from ~70s to ~30s
llvm-svn: 184363
When -T is specified, the test suite will call svn info and dump the output on screen (this used to be the default behavior)
When -T is not specified, this step won't be performed (the new default)
llvm-svn: 179342
Tamas Berghammer