Commit Graph

31 Commits

Author SHA1 Message Date
Todd Fiala 9187f27e32 Add test event marking a test as explicitly eligible for rerun if it is marked flakey.
This will be used in a future change to support rerunning flakey tests
that hit a test result isue in a low-load, single worker test runner phase.

This is implemented as an additive-style event rather than being
evaluated and added to the start_test event because the decorator code
only runs after the start_test event is created and sent.  i.e.
LLDBTestResult.startTest() runs before the test method decorators run.

llvm-svn: 255351
2015-12-11 18:06:47 +00:00
Kamil Rytarowski f5d34b7b9d Add NetBSD support in the buildDriver and buildLibrary routines
Summary: NetBSD is like FreeBSD and Linux in these routines.

Reviewers: clay.chang, tfiala, emaste, joerg

Subscribers: lldb-commits, emaste

Differential Revision: http://reviews.llvm.org/D15374

llvm-svn: 255308
2015-12-10 22:56:56 +00:00
Zachary Turner b08ab72427 Remove -w option from dotest.py.
llvm-svn: 255275
2015-12-10 18:50:32 +00:00
Zachary Turner 76972031fa Remove -k command line option from dotest.py.
This is part of an effort to remove unused command line options.

llvm-svn: 255143
2015-12-09 19:45:51 +00:00
Pavel Labath 7ead0b937c XFAIL TestReturnValue for remote Windows->Android tests
this also adds the ability to match the host platform to the expectedFailureAll decorator.

llvm-svn: 255105
2015-12-09 10:54:18 +00:00
Zachary Turner d865c6b707 Remove the -c option from dotest.py.
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today.  git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.

llvm-svn: 255060
2015-12-08 22:15:48 +00:00
Zachary Turner 2155d5d301 Remove the -b option from dotest.py
This removes the blacklist option as part of an effort to remove
unused / unmaintained command line options from the test suite.

llvm-svn: 255040
2015-12-08 18:43:16 +00:00
Zachary Turner aad25fb9a5 Remove +b option from dotest.py
llvm-svn: 255037
2015-12-08 18:36:05 +00:00
Tamas Berghammer ccd6cffba3 Modify "platform connect" to connect to processes as well
The standard remote debugging workflow with gdb is to start the
application on the remote host under gdbserver (e.g.: gdbserver :5039
a.out) and then connect to it with gdb.

The same workflow is supported by debugserver/lldb-gdbserver with a very
similar syntax but to access all features of lldb we need to be
connected also to an lldb-platform instance running on the target.

Before this change this had to be done manually with starting a separate
lldb-platform on the target machine and then connecting to it with lldb
before connecting to the process.

This change modifies the behavior of "platform connect" with
automatically connecting to the process instance if it was started by
the remote platform. With this command replacing gdbserver in a gdb
based worflow is usually as simple as replacing the command to execute
gdbserver with executing lldb-platform.

Differential revision: http://reviews.llvm.org/D14952

llvm-svn: 255016
2015-12-08 14:08:19 +00:00
Zachary Turner 606e3a5221 Get rid of global variables in dotest.py
This moves all the global variables into a separate module called
`configuration`.  This has a number of advantages:

1. Configuration data is centrally maintained so it's easy to get
   a high level overview of what configuration data the test suite
   makes use of.
2. The method of sharing configuration data among different parts
   of the test suite becomes standardized.  Previously we would
   put some things into the `lldb` module, some things into the
   `lldbtest_config` module, and some things would not get shared.
   Now everything is shared through one module and is available to
   the entire test suite.
3. It opens the door to moving some of the initialization code into
   the `configuration` module, simplifying the implementation of
   `dotest.py`.

There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily.  This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable.  We can address these in the future.

llvm-svn: 254982
2015-12-08 01:15:30 +00:00
Kamil Rytarowski 49f9fb8d26 Add initial NetBSD support in lldbsuite/test/lldbtest.py
Summary:
Add new functions:

  - expectedFailureNetBSD()
  - expectedFlakeyNetBSD()
  - skipIfNetBSD()

Add new NetBSD entry in:

  - getPlatform()
  - getHostPlatform()

Assume that libc++ is installed and use the GNU toolchain

Reviewers: joerg, emaste, tfiala, clayborg

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15262

llvm-svn: 254948
2015-12-07 21:25:57 +00:00
Kamil Rytarowski 0b655da7e4 Define new builder_netbsd
Summary: This is used in tests.

Reviewers: emaste, tfiala, clayborg

Subscribers: zturner, lldb-commits, joerg

Differential Revision: http://reviews.llvm.org/D15065

llvm-svn: 254853
2015-12-05 18:46:56 +00:00
Siva Chandra 7dcad3178b Revert "Make skipIf support the not_in function (second attempt)."
Summary: This reverts commit 70dca28976ee8137acce2cc203dd394f4d761276.

Reviewers: amccarth, zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14881

llvm-svn: 253704
2015-11-20 20:30:36 +00:00
Adrian McCarthy 4d5d1dd6d5 Make skipIf support the not_in function (second attempt).
llvm-svn: 253683
2015-11-20 18:14:24 +00:00
Siva Chandra d2e90129d9 Revert "Make skipIf support the not_in function."
Summary:
This reverts commit 251965377bdfb6227eea42c12a792c059e4e8a4b
as a test marked "skipIf(compiler='gcc')" runs when testing with GCC.

Reviewers: amccarth

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14852

llvm-svn: 253631
2015-11-20 01:54:24 +00:00
Adrian McCarthy 257da8ea50 Make skipIf support the not_in function.
llvm-svn: 253623
2015-11-20 00:11:38 +00:00
Enrico Granata 55d99f0e7c Cleanup work required to get the ASAN tests to run in the new test suite infrastructure; the tests are now xfailed on Darwin pending investigation
llvm-svn: 253604
2015-11-19 21:45:07 +00:00
Ying Chen ca922bb9b9 Support unix-abstract-connect scheme as platform url in lldb testsuite
Reviewers: ovyalov

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14765

llvm-svn: 253488
2015-11-18 19:03:20 +00:00
Zachary Turner 48ef8d4c37 Fix some issues with swig & string conversion.
This patch fixes two issues:

1) Popen needs to be used with universal_newlines=True by default.
   This elicits automatic decoding from bytes -> string in Py3,
   and has no negative effects in other Py versions.
2) The swig typemaps for converting between string and (char*, int)
   did not work correctly when the length of the string was 0,
   indicating an error.  In this case we would try to construct a
   string from uninitialized data.
3) Ironically, the bug mentioned in #2 led to a test passing on
   Windows that was actually broken, because the test was written
   such that the assertion was never even getting checked, so it
   passed by default.  So we additionally fix this test to also
   fail if the method errors.  By fixing this test it's now broken
   on Windows, so we also xfail it.

llvm-svn: 253487
2015-11-18 18:40:16 +00:00
Zachary Turner ba1057022c Python 3 - Skip a certain test for a particular (swig,python) combo.
Current versions of SWIG have a bug with Python 3 that causes
Python to assert when iterating over a generator.  This patch
skips the test for the right combination of Python version and
SWIG version.  I'm attempting to upstream a patch to SWIG to
fix this in a subsequent as-of-yet unreleased version, but
I don't know how long that will take.

llvm-svn: 253273
2015-11-16 23:58:20 +00:00
Ying Chen 0c35282c65 Re-Apply "Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll ..." with fix
Summary:
- Re-Commit r253106
- Initialize self.debug_info in Base::setUp()
- Fix argument order when calling check_list_or_lambda for compiler

Reviewers: jingham

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14673

llvm-svn: 253272
2015-11-16 23:41:02 +00:00
Zachary Turner abdb839228 Add the ability to xfail or skip based on swig / python version.
llvm-svn: 253263
2015-11-16 22:40:30 +00:00
Pavel Labath 311cc7d133 Revert "Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll to reverse"
This reverts commit r253106.

llvm-svn: 253197
2015-11-16 11:11:10 +00:00
Jim Ingham 6d048942c5 Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll to reverse
the sense of the test.

llvm-svn: 253106
2015-11-14 00:20:33 +00:00
Zachary Turner 8d13fab183 Python 3 - Don't use unbuffered I/O in text mode.
This is unsupported in Python 3.  This could also have been fixed
by using "wb" instead of "w", but it doesn't seem like writing the
session log absolutely *needs* to be unbuffered.

llvm-svn: 252381
2015-11-07 01:08:15 +00:00
Zachary Turner 5cb8e67b17 Don't use module internal implementation details in our decorators.
We tried implementing something akin to a conditionalExpectedFailure
decorator for unittest2.  We did this by making use of some
implementation details of the unittest2 module.  In an effort to make
this work with unittest, this patch removes the reliance on the
implementation details.  I have a hard time wrapping my head around
how this all works with the deeply nested decorators, but the spirit
of the patch here is to do do the following: If the condition function
is true, use the original unittest2.expectedFailure decorator.  Otherwise
don't use any decorator, just call the test function.

Differential Revision: http://reviews.llvm.org/D14406
Reviewed By: tberghammer, labath

llvm-svn: 252326
2015-11-06 18:14:42 +00:00
Zachary Turner c1b7cd72db Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.

When absolute imports are enabled, the import system changes in
a couple of ways:

1) The `import foo` syntax will *only* search sys.path.  If `foo`
   isn't in sys.path, it won't be found.  Period.  Without absolute
   imports, the import system will also search the same directory
   that the importing file resides in, so that you can easily
   import from the same folder.

2) From inside a package, you can use a dot syntax to refer to higher
   levels of the current package.  For example, if you are in the
   package lldbsuite.test.utility, then ..foo refers to
   lldbsuite.test.foo.  You can use this notation with the
   `from X import Y` syntax to write intra-package references.  For
   example, using the previous locationa s a starting point, writing
   `from ..support import seven` would import lldbsuite.support.seven

Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.

See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.

Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala

llvm-svn: 252191
2015-11-05 19:22:28 +00:00
Enrico Granata 5f92a130ff Teach LLDB how to directly launch processes on the iOS simulator
This allows for command-line debugging of iOS simulator binaries (as long as UI is not required, or a full UI simulator has previously been otherwise launched), as well as execution of the LLDB test suite on the iOS simulator

This is known to compile on OSX 10.11 GM - feedback from people on other platforms and/or older versions of OSX as to the buildability of this code is greatly appreciated

llvm-svn: 252112
2015-11-05 00:46:25 +00:00
Zachary Turner 19474e1801 Remove `use_lldb_suite` from the package, and don't import it anymore.
This module was originally intended to be imported by top-level
scripts to be able to find the LLDB packages and third party
libraries.  Packages themselves shouldn't need to import it,
because by the time it gets into the package, the top-level
script should have already done this.  Indeed, it was just
adding the same values to sys.path multiple times, so this
patch is essentially no functional change.

To make sure it doesn't get re-introduced, we also delete the
`use_lldb_suite` module from `lldbsuite/test`, although the
original copy still remains in `lldb/test`

llvm-svn: 251963
2015-11-03 19:20:39 +00:00
Zachary Turner 95c453a221 Tighten up sys.path, and use absolute imports everywhere.
For convenience, we had added the folder that dotest.py was in
to sys.path, so that we could easily write things like
`import lldbutil` from anywhere and any test.  This introduces
a subtle problem when using Python's package system, because when
unittest2 imports a particular test suite, the test suite is detached
from the package.  Thus, writing "import lldbutil" from dotest imports
it as part of the package, and writing the same line from a test
does a fresh import since the importing module was not part of
the same package.

The real way to fix this is to use absolute imports everywhere.  Instead
of writing "import lldbutil", we need to write "import
lldbsuite.test.util".  This patch fixes up that and all other similar
cases, and additionally removes the script directory from sys.path
to ensure that this can't happen again.

llvm-svn: 251886
2015-11-03 02:06:18 +00:00
Zachary Turner c432c8f856 Move lldb/test to lldb/packages/Python/lldbsuite/test.
This is the conclusion of an effort to get LLDB's Python code
structured into a bona-fide Python package.  This has a number
of benefits, but most notably the ability to more easily share
Python code between different but related pieces of LLDB's Python
infrastructure (for example, `scripts` can now share code with
`test`).

llvm-svn: 251532
2015-10-28 17:43:26 +00:00