Commit Graph

18 Commits

Author SHA1 Message Date
Stefan Granitz 79f41967dc [CMake] In standalone builds, LLVM_BINARY_DIR should point to LLVM's binary directory
Summary: In standalone builds `LLVM_BINARY_DIR` was equal to `LLDB_BINARY_DIR` so far. This is counterintuitive and invalidated the values of `LLDB_DEFAULT_TEST_DSYMUTIL/FILECHECK/COMPILER` etc.

Reviewers: zturner, labath, clayborg, JDevlieghere, stella.stamenova, serge-sans-paille

Reviewed By: labath

Subscribers: mgorny, friss, lldb-commits, #lldb

Differential Revision: https://reviews.llvm.org/D56443

llvm-svn: 350738
2019-01-09 16:25:37 +00:00
Adrian Prantl 938d52538a Remove ineffective (misspelled) sanitizer option
llvm-svn: 349864
2018-12-21 01:09:14 +00:00
Jonas Devlieghere dcc8fc9e98 [dotest] Consider unexpected passes as failures.
Unexpected successes should be considered failures because they can hide
regressions when not addressed. When a test is fixed and not re-enabled,
it can easily regress without us noticing.

I couldn't find a good way to make this change other than changing it in
the unittest2 framework. I know this is less than optimal but since we
have the dependency checked in and the change is pretty fundamental to
the framework I think it's not unreasonable.

Differential revision: https://reviews.llvm.org/D55835

llvm-svn: 349818
2018-12-20 20:44:23 +00:00
Jonas Devlieghere cba1b00c42 [lit] Detect unexpected passes in lldbtest.
This patch will have lit report unexpected passes when dotest reports at
least one XPASS and no failures.

llvm-svn: 349401
2018-12-17 21:40:37 +00:00
Pavel Labath d9e8c773de Fix a typo in mac SIP workaround
presumably the or subexpression was meant to be evaluated first. The way
it is now means that we apply the workaround for any python in `/usr`,
which matches pretty much every unix system.

llvm-svn: 341167
2018-08-31 06:01:02 +00:00
Adrian Prantl dfa7755c52 Make the DYLD_INSERT_LIBRARIES workaround for SIP more robut for the various configurations that bots are running
llvm-svn: 340792
2018-08-27 23:06:38 +00:00
Adrian Prantl 7e6ce43ef0 Add a mkdir -p to builddir into lldbtest.py
Based on how it is executed, it may not have been yet created.

llvm-svn: 340791
2018-08-27 23:06:37 +00:00
Adrian Prantl 6c7f5884d5 Fix check for dictionary entry
llvm-svn: 340557
2018-08-23 17:51:14 +00:00
Adrian Prantl 748310b193 lldbtest.py: Work around macOS SIP when testing ASANified builds.
llvm-svn: 340548
2018-08-23 17:19:08 +00:00
Adrian Prantl 0e45df4c0e Reflow comments
llvm-svn: 340219
2018-08-20 22:00:32 +00:00
Adrian Prantl c14fd34803 Set path to sanitizer runtime when running tests through LIT on macOS.
rdar://problem/42984739

Differential Revision: https://reviews.llvm.org/D50997

llvm-svn: 340218
2018-08-20 22:00:31 +00:00
Jonas Devlieghere 12543bd023 [lit] Don't require semicolon separator
This patch removes the requirement for a semicolon as a separator when
passing arguments to lit. It relies on the shlex module that is part of
Python to do simple lexical analysis, similar to what happens in a Unix
shell.

llvm-svn: 336290
2018-07-04 17:14:52 +00:00
Michal Gorny 6dde836844 [lit] Do not run Python tests w/ LLDB_DISABLE_PYTHON
Skip all Python-based tests as unsupported when LLDB_DISABLE_PYTHON is
enabled.  Otherwise, those tests simply fail being unable to import lldb
module.

Differential Revision: https://reviews.llvm.org/D47812

llvm-svn: 334080
2018-06-06 09:44:14 +00:00
Jonas Devlieghere 36ab8956f4 [lit] Add support for passing arguments to dotest.py via lit.
The lldb test suite is highly configurable. While part of this
configuration takes place at configure/build-time, a common scenario
involves running the test suite several times with different
configuration. For example, we might want to test the current lldb
against inferiors built with different compilers.

This configuration was already possible for lldb-dotest, but was lacking
for the lit counterpart. It is now possible to pass arguments to pass
  arguments like this:

  ./bin/llvm-lit ../llvm/tools/lldb/lit/Suite/ -Ddotest-args="-foo;-bar"

llvm-svn: 333432
2018-05-29 16:49:07 +00:00
Jonas Devlieghere 52721286b2 [CMake] Unify and relayer testing
This patch restructures part of LLDB's testing configuration:

1. I moved the test dependencies up the chain so every dotest dependency
   becomes a lit dependency as well. It wouldn't make sense for dotest to
   have other dependencies when it's being run by lit. Lit on the other
   hand can still specify extra dependencies.

2. I replaced as much generator expressions with variables as possible.
   This is consistent with the rest of LLVM and doesn't break generators
   that support multiple targets (MSVC, Xcode). This wasn't a problem
   before, but now we need to expand the dotest arguments in the lit
   configuration and there's only one test suite even with multiple
   targets.

3. I moved lldb-dotest into it's own directory under utils since there's
   no need anymore for it to located under `test/`.

Differential revision: https://reviews.llvm.org/D46334

llvm-svn: 331463
2018-05-03 16:54:10 +00:00
Aaron Smith ba48c9bde1 [lit, lldbsuite] Update the lldbsuite to correctly run tests on windows and windows server
Summary:
The new script to run the lldbtests as part of lit invokes each test by calling dotest.py, however, we cannot rely on the system to always correctly interpret the script as python causing the tests to be unresolved on windows (at least). To fix this, we need to make sure that the first parameter in the command line is the python executable itself.

In Makefile.rules, there are a number of windows specific definitions that rely on the HOST_OS being set as Windows_NT but the logic detecting the OS currently does not detect server versions of windows correctly. This change updates the logic to detect windows server as well.

Reviewers: asmith, labath, JDevlieghere, zturner

Reviewed By: JDevlieghere, zturner

Subscribers: zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D46020

llvm-svn: 330740
2018-04-24 17:08:05 +00:00
Jonas Devlieghere 49f4aee0e3 [lit] Generate a single lit cfg file for tests that require dotest.py
The current way that the lit configuration is generated for the LLDB
tests that run using dotest causes cmake to fail when using a generator
which supports multiple configurations (such as Visual Studio). The
failure is because file GENERATE will create a file *per possible
configuration* resulting in the same lit configuration file being
overwritten multiple times.

To fix the issue, we need to create a single lit file that is agnostic
of the configurations and can be used for any configuration.

Patch by: Stella Stamenova

Differential revision: https://reviews.llvm.org/D45918

llvm-svn: 330518
2018-04-21 18:23:04 +00:00
Jonas Devlieghere 1a928f3fc7 [LIT] Have lit run the lldb test suite
This is the first in what will hopefully become a series of patches to
replace the driver logic in dotest.py with LIT. The motivation for this
change is that there's no point in maintaining two driver
implementations. Since all of the LLVM projects are using lit, this is
the obvious choice.

Obviously the goal is maintain full compatibility with the functionality
offered by dotest. As such we won't be removing anything until that
point has been reached.

This patch is the initial attempt (referred to as v1) to run the lldb
test suite with lit. To do so we introduced a custom LLDB test format
that invokes dotest.py with a single test file.

Differential revision: https://reviews.llvm.org/D45333

llvm-svn: 330275
2018-04-18 17:08:49 +00:00