This patch is the result of a discussion on lldb-dev, see
http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for
background.
For each test (should be eventually: each test configuration) a
separate build directory is created and we execute
make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test
In order to make this work all LLDB tests need to be updated to find
the executable in the test build directory, since CWD still points at
the test's source directory, which is a requirement for unittest2.
Although we have done extensive testing, I'm expecting that this first
attempt will break a few bots. Please DO NOT HESITATE TO REVERT this
patch in order to get the bots green again. We will likely have to
iterate on this some more.
Differential Revision: https://reviews.llvm.org/D42281
llvm-svn: 323803
Summary:
Adds new utilities that make it easier to write test cases for lldb acting as a client over a gdb-remote connection.
- A GDBRemoteTestBase class that starts a mock GDB server and provides an easy way to check client packets
- A MockGDBServer that, via MockGDBServerResponder, can be made to issue server responses that test client behavior.
- Utility functions for handling common data encoding/decoding
- Utility functions for creating dummy targets from YAML files
----
Split from the review at https://reviews.llvm.org/D42145, which was a new feature that necessitated the new testing capabilities.
Reviewers: clayborg, labath
Reviewed By: clayborg, labath
Subscribers: hintonda, davide, jingham, krytarowski, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D42195
Patch by Owen Shaw <llvm@owenpshaw.net>
llvm-svn: 323636
On Darwin, if a test machine isn't set up for code-signing (see
docs/code-signing.txt), running check-lldb should use the system
debugserver instead of the unsigned one built in-tree. This makes it
possible to run lldb's test suite without having code-signing set up,
which is really convenient.
Differential Revision: https://reviews.llvm.org/D42215
llvm-svn: 322803
Summary:
This is required when using the in-tree clang for building tests,
because -fuse-ld=lld is used by default.
Subscribers: mgorny
Differential Revision: https://reviews.llvm.org/D39689
llvm-svn: 317501
Summary:
Using the in-tree clang should be the default test configuration as that
is the one compiler that we can be sure everyone has (better
reproducibility of test results). Also, it should hopefully reduce the
impact of pr35040.
This also reduces the number of settings which control the compiler
used. LLDB_TEST_C_COMPILER is used for C files and
LLDB_TEST_CXX_COMPILER for C++ files. Both of the settings default to
the in-tree clang.
Reviewers: zturner
Subscribers: mgorny, davide, lldb-commits
Differential Revision: https://reviews.llvm.org/D39215
llvm-svn: 316728
CMake target "check-lldb" runs the lldb dotest.py suite, but doesn't
collect the results in a usable format. In adding the arguments
necessary to collect these results, I found some minor bugs in CMake
that prevented dotest overrides from being used. This patch fixes them.
<rdar://problem/33389717> cmake build needs to run tests AND collect results
llvm-svn: 308393
This patch adds support to the test suite for overriding the path to debugserver, and uses the override to point to the build tree's debugserver on Darwin.
llvm-svn: 297776
Summary:
The test runner has code to autodetect this, but it's not very smart --
in particular, it fails in the case where we build the test executables
with the just-built clang. Since cmake already has the knowledge about
the right toolchain, we can just have it pass the appropriate flags to
the test runner.
This also removes the "temporary" cache-scrubbing hack added a couple
months ago.
Reviewers: zturner, beanz
Subscribers: mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D30453
llvm-svn: 296593
CMake's framework target generation was unable to generate POST_BUILD steps (see: https://gitlab.kitware.com/cmake/cmake/issues/16363).
It turns out working around this is really not reasonable. The more reasonable solution to me is just to not support LLDB.framework unless you are on CMake 3.7 or newer.
Since CMake 3.7.1 is released that's how I'm going to handle this.
llvm-svn: 289841
This ensures that the Resources and clang headers are properly symlinked in LLDB's framework. This should fix the modules-related tests when building on Darwin with CMake if you are building a framework.
I have another fix coming which gets them working on Darwin if you're building liblldb instead of a framework.
llvm-svn: 285651
Summary:
dotest.py has a framework option that is not respected. This patch makes the framework path properly configurable via the --framework option.
This patch also adds a function to the lldbtest.Base class named "hasDarwinFramework" which allows us to not rely on the host platform to determine if a framework is present. If running on Darwin, and not building a framework, this will follow the *nix code paths which are appropriate for Darwin.
Reviewers: tfiala
Subscribers: lldb-commits, mgorny
Differential Revision: https://reviews.llvm.org/D25886
llvm-svn: 285541
Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden.
Reviewers: zturner, tfiala
Subscribers: labath, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D25714
llvm-svn: 284893
Summary:
CMake has no builtin mechanism for cache invalidation. As a general convention you want to not expand user-specified variables in other cached variables because they will not get updated when the user changes their specified value.
This patch moves the "-C" option for dotest.py into the LLDB_TEST_COMMON_ARGS and out of the CMake cache. In order to prevent issues with out-of-date cache files on builders I've added code to scrub "-C ${LLDB_TEST_COMPILER}" out of the CMake caches, by Force writing the variable. This code can be removed in a few days once the change has trickled through CI systems.
Reviewers: tfiala, labath, zturner
Subscribers: lldb-commits, mgorny
Differential Revision: https://reviews.llvm.org/D25751
llvm-svn: 284551
Summary:
This patch adds the following fixes to the check-lldb targets:
* Adds missing dependencies on lldb tools so they get built before tests execute
* Adds Ninja USES_TERMINAL to the target so that the output streams to stdout as it executes
* Uses a generator expression to find the lldb executable, this is more robust than constructing the path manually
Reviewers: tfiala, zturner
Subscribers: mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D25490
llvm-svn: 284046
There is flakiness somewhere in the core infrastructure on Windows,
so to get the buildbot reliably green we need to mark all tests
as flaky.
llvm-svn: 270460
When running the test suite on Windows, we can't have Windows popping
up dialogs when LLDB crashes in native code because it will hang
the test suite. This patch silences those dialogs by checking an
environment variable at startup and configuring Windows based on
its value.
This patch also adds an environment variable to force inferiors to
never spawn in their own console window. This is useful to prevent
new window spawm when running the test suite.
Reviewed by: Scott Graham
Differential Revision: http://reviews.llvm.org/D6628
llvm-svn: 224137
Previously, CMake was invoking the test runner and not specifying
what architecture to use when building test executables. The
Makefiles for the test executables then had logic to choose x64
by default. This doesn't work on Windows because the test compiler
would then try to link against the 64-bit MSVCRT and not find them
since only the 32-bit MSVCRT was in the path.
This patch addresses this by figuring out, at CMake time, whether
or not you are building LLDB with a 64 or 32-bit toolchain. Then,
it explicitly passes this value to the test runner, causing the
test runner to build tests whose architecture matches that of LLDB
itself. This can still be overridden by setting the CMake variable
LLDB_TEST_EXECUTABLE_ARCH=(x64|x86)
llvm-svn: 214443
- should fix automatic tests set up on http://llvm-jenkins.debian.net
- needed in order to make Debian package builds depend on passing test suite
llvm-svn: 181736
- resolved circular dependency issue by making liblldb depend directly on LLDBWrapPython.cpp
- removed use of '..' for relative directories -- ninja doesn't like this
- fixed build-order problem
llvm-svn: 176517
Adrian Prantl