forked from OSchip/llvm-project
223 lines
7.9 KiB
ReStructuredText
223 lines
7.9 KiB
ReStructuredText
Testing
|
|
=======
|
|
|
|
The LLDB test suite consists of three different kinds of test:
|
|
|
|
* Python scripts located under ``lldb/packages/Python/lldbsuite``.
|
|
These are written using python's unittest2 testing framework.
|
|
|
|
* Unit tests, located under ``lldb/unittests``. These are written in C++,
|
|
using googletest.
|
|
|
|
* LIT tests, located under ``lldb/lit``. These use the LLVM Integrated Tester.
|
|
|
|
Many of the tests are accompanied by a C (C++, ObjC, etc.) source file. Each test
|
|
first compiles the source file and then uses LLDB to debug the resulting executable.
|
|
|
|
The tests verify both the LLDB command line interface and the scripting API.
|
|
|
|
.. contents::
|
|
:local:
|
|
|
|
.. note::
|
|
|
|
On Windows any invocations of python should be replaced with python_d, the
|
|
debug interpreter, when running the test suite against a debug version of
|
|
LLDB.
|
|
|
|
.. note::
|
|
|
|
On NetBSD you must export ``LD_LIBRARY_PATH=$PWD/lib`` in your environment.
|
|
This is due to lack of the ``$ORIGIN`` linker feature.
|
|
|
|
Running the Full Test Suite
|
|
---------------------------
|
|
|
|
The easiest way to run the LLDB test suite is to use the ``check-lldb`` build
|
|
target.
|
|
|
|
By default, the ``check-lldb`` target builds the test programs with the same
|
|
compiler that was used to build LLDB. To build the tests with a different
|
|
compiler, you can set the ``LLDB_TEST_C_COMPILER`` or the
|
|
``LLDB_TEST_CXX_COMPILER`` CMake variables.
|
|
|
|
It is possible to customize the architecture of the test binaries and compiler
|
|
used by appending ``-A`` and ``-C`` options respectively to the CMake variable
|
|
``LLDB_TEST_USER_ARGS``. For example, to test LLDB against 32-bit binaries
|
|
built with a custom version of clang, do:
|
|
|
|
::
|
|
|
|
> cmake -DLLDB_TEST_USER_ARGS="-A i386 -C /path/to/custom/clang" -G Ninja
|
|
> ninja check-lldb
|
|
|
|
Note that multiple ``-A`` and ``-C`` flags can be specified to
|
|
``LLDB_TEST_USER_ARGS``.
|
|
|
|
|
|
Running a Specific Test or Set of Tests: Python
|
|
-----------------------------------------------
|
|
|
|
In addition to running all the LLDB test suites with the ``check-lldb`` CMake
|
|
target above, it is possible to run individual LLDB tests. If you have a CMake
|
|
build you can use the ``lldb-dotest`` binary, which is a wrapper around
|
|
``dotest.py`` that passes all the arguments configured by CMake. Alternatively,
|
|
you can use ``dotest.py`` directly, if you want to run a test one-off with a
|
|
different configuration.
|
|
|
|
|
|
For example, to run the test cases defined in TestInferiorCrashing.py, run:
|
|
|
|
::
|
|
|
|
> lldb-dotest -p TestInferiorCrashing.py
|
|
|
|
::
|
|
|
|
> cd $lldb/test
|
|
> python dotest.py --executable <path-to-lldb> -p TestInferiorCrashing.py ../packages/Python/lldbsuite/test
|
|
|
|
If the test is not specified by name (e.g. if you leave the ``-p`` argument
|
|
off), all tests in that directory will be executed:
|
|
|
|
|
|
::
|
|
|
|
> lldb-dotest functionalities/data-formatter
|
|
|
|
::
|
|
|
|
> python dotest.py --executable <path-to-lldb> functionalities/data-formatter
|
|
|
|
Many more options that are available. To see a list of all of them, run:
|
|
|
|
::
|
|
|
|
> python dotest.py -h
|
|
|
|
|
|
Running a Specific Test or Set of Tests: Unit Tests
|
|
---------------------------------------------------
|
|
|
|
The unit tests are simple executables, located in the build directory under ``tools/lldb/unittests``.
|
|
|
|
To run them, just run the test binary, for example, to run all the Host tests:
|
|
|
|
::
|
|
|
|
> ./tools/lldb/unittests/Host/HostTests
|
|
|
|
|
|
To run a specific test, pass a filter, for example:
|
|
|
|
::
|
|
|
|
> ./tools/lldb/unittests/Host/HostTests --gtest_filter=SocketTest.DomainListenConnectAccept
|
|
|
|
|
|
Running a Specific Test or Set of Tests: LIT
|
|
--------------------------------------------
|
|
|
|
LIT automatically scans a directory for tests. To run a subset of the LIT tests, pass it a
|
|
subdirectory, for example:
|
|
|
|
::
|
|
|
|
> ./bin/llvm-lit -sv tools/lldb/lit/Commands/CommandScriptImmediateOutput
|
|
|
|
|
|
LIT can also filter based on test name.
|
|
|
|
::
|
|
|
|
> ./bin/llvm-lit -sv tools/lldb/lit --filter CommandScriptImmediateOutput
|
|
|
|
|
|
It is also possible to forward arguments to dotest.py by passing ``--param`` to
|
|
lit and setting a value for ``dotest-args``.
|
|
|
|
::
|
|
|
|
> ./bin/llvm-lit -sv tools/lldb/lit --param dotest-args='-C gcc'
|
|
|
|
|
|
Running the Test Suite Remotely
|
|
-------------------------------
|
|
|
|
Running the test-suite remotely is similar to the process of running a local
|
|
test suite, but there are two things to have in mind:
|
|
|
|
1. You must have the lldb-server running on the remote system, ready to accept
|
|
multiple connections. For more information on how to setup remote debugging
|
|
see the Remote debugging page.
|
|
2. You must tell the test-suite how to connect to the remote system. This is
|
|
achieved using the ``--platform-name``, ``--platform-url`` and
|
|
``--platform-working-dir`` parameters to ``dotest.py``. These parameters
|
|
correspond to the platform select and platform connect LLDB commands. You
|
|
will usually also need to specify the compiler and architecture for the
|
|
remote system.
|
|
|
|
Currently, running the remote test suite is supported only with ``dotest.py`` (or
|
|
dosep.py with a single thread), but we expect this issue to be addressed in the
|
|
near future.
|
|
|
|
Debugging Test Failures
|
|
-----------------------
|
|
|
|
On non-Windows platforms, you can use the ``-d`` option to ``dotest.py`` which
|
|
will cause the script to wait for a while until a debugger is attached.
|
|
|
|
Debugging Test Failures on Windows
|
|
----------------------------------
|
|
|
|
On Windows, it is strongly recommended to use Python Tools for Visual Studio
|
|
for debugging test failures. It can seamlessly step between native and managed
|
|
code, which is very helpful when you need to step through the test itself, and
|
|
then into the LLDB code that backs the operations the test is performing.
|
|
|
|
A quick guide to getting started with PTVS is as follows:
|
|
|
|
#. Install PTVS
|
|
#. Create a Visual Studio Project for the Python code.
|
|
#. Go to File -> New -> Project -> Python -> From Existing Python Code.
|
|
#. Choose llvm/tools/lldb as the directory containing the Python code.
|
|
#. When asked where to save the .pyproj file, choose the folder ``llvm/tools/lldb/pyproj``. This is a special folder that is ignored by the ``.gitignore`` file, since it is not checked in.
|
|
#. Set test/dotest.py as the startup file
|
|
#. Make sure there is a Python Environment installed for your distribution. For example, if you installed Python to ``C:\Python35``, PTVS needs to know that this is the interpreter you want to use for running the test suite.
|
|
#. Go to Tools -> Options -> Python Tools -> Environment Options
|
|
#. Click Add Environment, and enter Python 3.5 Debug for the name. Fill out the values correctly.
|
|
#. Configure the project to use this debug interpreter.
|
|
#. Right click the Project node in Solution Explorer.
|
|
#. In the General tab, Make sure Python 3.5 Debug is the selected Interpreter.
|
|
#. In Debug/Search Paths, enter the path to your ninja/lib/site-packages directory.
|
|
#. In Debug/Environment Variables, enter ``VCINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\``.
|
|
#. If you want to enabled mixed mode debugging, check Enable native code debugging (this slows down debugging, so enable it only on an as-needed basis.)
|
|
#. Set the command line for the test suite to run.
|
|
#. Right click the project in solution explorer and choose the Debug tab.
|
|
#. Enter the arguments to dotest.py.
|
|
#. Example command options:
|
|
|
|
::
|
|
|
|
--arch=i686
|
|
# Path to debug lldb.exe
|
|
--executable D:/src/llvmbuild/ninja/bin/lldb.exe
|
|
# Directory to store log files
|
|
-s D:/src/llvmbuild/ninja/lldb-test-traces
|
|
-u CXXFLAGS -u CFLAGS
|
|
# If a test crashes, show JIT debugging dialog.
|
|
--enable-crash-dialog
|
|
# Path to release clang.exe
|
|
-C d:\src\llvmbuild\ninja_release\bin\clang.exe
|
|
# Path to the particular test you want to debug.
|
|
-p TestPaths.py
|
|
# Root of test tree
|
|
D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test
|
|
|
|
::
|
|
|
|
--arch=i686 --executable D:/src/llvmbuild/ninja/bin/lldb.exe -s D:/src/llvmbuild/ninja/lldb-test-traces -u CXXFLAGS -u CFLAGS --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe -p TestPaths.py D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --no-multiprocess
|
|
|
|
|
|
|