llvm-project/compiler-rt/test/CMakeLists.txt

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

118 lines
4.1 KiB
CMake
Raw Normal View History

# Needed for lit support in standalone builds.
include(AddLLVM)
[Compiler-rt] Distinguish between testing just built runtime libraries and the libraries shipped with the compiler. The path to the runtime libraries used by the compiler under test is normally identical to the path where just built libraries are created. However, this is not necessarily the case when doing standalone builds. This is because the external compiler used by tests may choose to get its runtime libraries from somewhere else. When doing standalone builds there are two types of testing we could be doing: * Test the just built runtime libraries. * Test the runtime libraries shipped with the compile under test. Both types of testing are valid but it confusingly turns out compiler-rt actually did a mixture of these types of testing. * The `test/builtins/Unit/` test suite always tested the just built runtime libraries. * All other testsuites implicitly use whatever runtime library the compiler decides to link. There is no way for us to infer which type of testing the developer wants so this patch introduces a new `COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` CMake option which explicitly declares which runtime libraries should be tested. If it is `ON` then the just built libraries should be tested, otherwise the libraries in the external compiler should be tested. When testing starts the lit test suite queries the compiler used for testing to see where it will get its runtime libraries from. If these paths are identical no action is taken (the common case). If the paths are not identical then we check the value of `COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` (progated into the config as `test_standalone_build_libs`) and check if the test suite supports testing in the requested configuration. * If we want to test just built libs and the test suite supports it (currently only `test/builtins/Unit`) then testing proceeds without any changes. * If we want to test the just built libs and the test suite doesn't support it we emit a fatal error to prevent the developer from testing the wrong runtime libraries. * If we are testing the compiler's built libs then we adjust `config.compiler_rt_libdir` to point at the compiler's runtime directory. This makes the `test/builtins/Unit` tests use the compiler's builtin library. No other changes are required because all other testsuites implicitly use the compiler's built libs. To make the above work the `test_suite_supports_overriding_runtime_lib_path` test suite config option has been introduced so we can identify what each test suite supports. Note all of these checks **have to be performed** when lit runs. We cannot run the checks at CMake generation time because multi-configuration build systems prevent us from knowing what the paths will be. We could perhaps support `COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` being `ON` for most test suites (when the runtime library paths differs) in the future by specifiying a custom compiler resource directory path. Doing so is out of scope for this patch. rdar://77182297 Differential Revision: https://reviews.llvm.org/D101681
2021-05-01 07:24:33 +08:00
option(COMPILER_RT_TEST_STANDALONE_BUILD_LIBS
"When set to ON and testing in a standalone build, test the runtime \
libraries built by this standalone build rather than the runtime libraries \
shipped with the compiler (used for testing). When set to OFF and testing \
in a standalone build, test the runtime libraries shipped with the compiler \
(used for testing). This option has no effect if the compiler and this \
build are configured to use the same runtime library path."
ON)
pythonize_bool(COMPILER_RT_TEST_STANDALONE_BUILD_LIBS)
pythonize_bool(LLVM_ENABLE_EXPENSIVE_CHECKS)
configure_compiler_rt_lit_site_cfg(
${CMAKE_CURRENT_SOURCE_DIR}/lit.common.configured.in
${CMAKE_CURRENT_BINARY_DIR}/lit.common.configured)
# BlocksRuntime (and most of builtins) testsuites are not yet ported to lit.
# add_subdirectory(BlocksRuntime)
[CMake] Add the way to run tests in standalone build. 1) Depend on llvm-config (configured in LLVM_CONFIG_PATH) to get necessary LLVM source/binary directories. 2) Add basic support for running lit tests (check-xsan commands). For now this "support" is far from what we want: * unit tests are not built currently. * lit tests use Clang/compiler-rt from LLVM build directory, not the host compiler or just-built compiler-rt libraries. We should make a choice on the way we intend ti run compiler-rt lit testsuite: a) use either Clang from LLVM build tree, or the host compiler. b) use either just-built runtimes, or the runtimes shipped with the host compiler. Using just-built runtimes is tricky - we have to know where to put them, so that Clang/GCC driver would pick them up (and not overwrite the existing runtimes). Using a host compiler instead of Clang from LLVM build tree will give us a chance to run lit tests under GCC (which already has support for several sanitizers). That is, I tend to make the following choice: if we're in a standalone compiler-rt build, use host compiler with its set of runtime libraries to run lit tests. This will effectively decouple "make compiler-rt" and "make check-compiler-rt" in a standalone build - the latter wouldn't invoke the former. Note that if we decide to fix LLVM/Clang/compiler-rt build system so that it would configure/build compiler-rt with just-built Clang (as we do in Makefile-based build), this will not be a problem - we can add a dependency to ensure that clang/compiler-rt are rebuilt before running compiler-rt tests. llvm-svn: 201656
2014-02-19 18:04:29 +08:00
set(SANITIZER_COMMON_LIT_TEST_DEPS)
if(COMPILER_RT_BUILD_PROFILE AND COMPILER_RT_HAS_PROFILE)
list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS profile)
endif()
# When ANDROID, we build tests with the host compiler (i.e. CMAKE_C_COMPILER),
# and run tests with tools from the host toolchain.
[CMake] Add the way to run tests in standalone build. 1) Depend on llvm-config (configured in LLVM_CONFIG_PATH) to get necessary LLVM source/binary directories. 2) Add basic support for running lit tests (check-xsan commands). For now this "support" is far from what we want: * unit tests are not built currently. * lit tests use Clang/compiler-rt from LLVM build directory, not the host compiler or just-built compiler-rt libraries. We should make a choice on the way we intend ti run compiler-rt lit testsuite: a) use either Clang from LLVM build tree, or the host compiler. b) use either just-built runtimes, or the runtimes shipped with the host compiler. Using just-built runtimes is tricky - we have to know where to put them, so that Clang/GCC driver would pick them up (and not overwrite the existing runtimes). Using a host compiler instead of Clang from LLVM build tree will give us a chance to run lit tests under GCC (which already has support for several sanitizers). That is, I tend to make the following choice: if we're in a standalone compiler-rt build, use host compiler with its set of runtime libraries to run lit tests. This will effectively decouple "make compiler-rt" and "make check-compiler-rt" in a standalone build - the latter wouldn't invoke the former. Note that if we decide to fix LLVM/Clang/compiler-rt build system so that it would configure/build compiler-rt with just-built Clang (as we do in Makefile-based build), this will not be a problem - we can add a dependency to ensure that clang/compiler-rt are rebuilt before running compiler-rt tests. llvm-svn: 201656
2014-02-19 18:04:29 +08:00
if(NOT ANDROID)
if(NOT COMPILER_RT_STANDALONE_BUILD AND NOT LLVM_RUNTIMES_BUILD)
[CMake] Add the way to run tests in standalone build. 1) Depend on llvm-config (configured in LLVM_CONFIG_PATH) to get necessary LLVM source/binary directories. 2) Add basic support for running lit tests (check-xsan commands). For now this "support" is far from what we want: * unit tests are not built currently. * lit tests use Clang/compiler-rt from LLVM build directory, not the host compiler or just-built compiler-rt libraries. We should make a choice on the way we intend ti run compiler-rt lit testsuite: a) use either Clang from LLVM build tree, or the host compiler. b) use either just-built runtimes, or the runtimes shipped with the host compiler. Using just-built runtimes is tricky - we have to know where to put them, so that Clang/GCC driver would pick them up (and not overwrite the existing runtimes). Using a host compiler instead of Clang from LLVM build tree will give us a chance to run lit tests under GCC (which already has support for several sanitizers). That is, I tend to make the following choice: if we're in a standalone compiler-rt build, use host compiler with its set of runtime libraries to run lit tests. This will effectively decouple "make compiler-rt" and "make check-compiler-rt" in a standalone build - the latter wouldn't invoke the former. Note that if we decide to fix LLVM/Clang/compiler-rt build system so that it would configure/build compiler-rt with just-built Clang (as we do in Makefile-based build), this will not be a problem - we can add a dependency to ensure that clang/compiler-rt are rebuilt before running compiler-rt tests. llvm-svn: 201656
2014-02-19 18:04:29 +08:00
# Use LLVM utils and Clang from the same build tree.
list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS
clang clang-resource-headers FileCheck count not llvm-config llvm-nm llvm-objdump
llvm-readelf llvm-readobj llvm-size llvm-symbolizer compiler-rt-headers sancov
llvm-jitlink)
if (WIN32)
list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS KillTheDoctor)
endif()
[CMake] Add the way to run tests in standalone build. 1) Depend on llvm-config (configured in LLVM_CONFIG_PATH) to get necessary LLVM source/binary directories. 2) Add basic support for running lit tests (check-xsan commands). For now this "support" is far from what we want: * unit tests are not built currently. * lit tests use Clang/compiler-rt from LLVM build directory, not the host compiler or just-built compiler-rt libraries. We should make a choice on the way we intend ti run compiler-rt lit testsuite: a) use either Clang from LLVM build tree, or the host compiler. b) use either just-built runtimes, or the runtimes shipped with the host compiler. Using just-built runtimes is tricky - we have to know where to put them, so that Clang/GCC driver would pick them up (and not overwrite the existing runtimes). Using a host compiler instead of Clang from LLVM build tree will give us a chance to run lit tests under GCC (which already has support for several sanitizers). That is, I tend to make the following choice: if we're in a standalone compiler-rt build, use host compiler with its set of runtime libraries to run lit tests. This will effectively decouple "make compiler-rt" and "make check-compiler-rt" in a standalone build - the latter wouldn't invoke the former. Note that if we decide to fix LLVM/Clang/compiler-rt build system so that it would configure/build compiler-rt with just-built Clang (as we do in Makefile-based build), this will not be a problem - we can add a dependency to ensure that clang/compiler-rt are rebuilt before running compiler-rt tests. llvm-svn: 201656
2014-02-19 18:04:29 +08:00
endif()
# Tests use C++ standard library headers.
if (TARGET cxx-headers OR HAVE_LIBCXX)
list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS cxx-headers)
endif()
endif()
function(compiler_rt_test_runtime runtime)
string(TOUPPER ${runtime} runtime_uppercase)
if(COMPILER_RT_HAS_${runtime_uppercase})
add_subdirectory(${runtime})
foreach(directory ${ARGN})
add_subdirectory(${directory})
endforeach()
endif()
endfunction()
# Run sanitizer tests only if we're sure that clang would produce
# working binaries.
if(COMPILER_RT_CAN_EXECUTE_TESTS)
if(COMPILER_RT_BUILD_BUILTINS)
add_subdirectory(builtins)
endif()
if(COMPILER_RT_BUILD_SANITIZERS)
compiler_rt_test_runtime(interception)
compiler_rt_test_runtime(lsan)
# CFI tests require diagnostic mode, which is implemented in UBSan.
compiler_rt_test_runtime(ubsan cfi)
compiler_rt_test_runtime(sanitizer_common)
# OpenBSD not supporting asan, cannot run the tests
if(COMPILER_RT_BUILD_LIBFUZZER AND NOT "${CMAKE_SYSTEM_NAME}" STREQUAL "OpenBSD" AND NOT ANDROID)
compiler_rt_test_runtime(fuzzer)
endif()
foreach(sanitizer ${COMPILER_RT_SANITIZERS_TO_BUILD})
# cfi testing is gated on ubsan
if(NOT ${sanitizer} STREQUAL cfi)
compiler_rt_test_runtime(${sanitizer})
endif()
endforeach()
endif()
if(COMPILER_RT_BUILD_PROFILE AND COMPILER_RT_HAS_PROFILE)
compiler_rt_test_runtime(profile)
endif()
if(COMPILER_RT_BUILD_MEMPROF)
compiler_rt_test_runtime(memprof)
endif()
if(COMPILER_RT_BUILD_XRAY)
compiler_rt_test_runtime(xray)
endif()
if(COMPILER_RT_BUILD_ORC)
compiler_rt_Test_runtime(orc)
endif()
if(COMPILER_RT_BUILD_CRT AND COMPILER_RT_HAS_CRT)
add_subdirectory(crt)
endif()
# ShadowCallStack does not yet provide a runtime with compiler-rt, the tests
# include their own minimal runtime
add_subdirectory(shadowcallstack)
endif()
if(COMPILER_RT_STANDALONE_BUILD)
# Now that we've traversed all the directories and know all the lit testsuites,
# introduce a rule to run to run all of them.
get_property(LLVM_LIT_TESTSUITES GLOBAL PROPERTY LLVM_LIT_TESTSUITES)
get_property(LLVM_LIT_DEPENDS GLOBAL PROPERTY LLVM_LIT_DEPENDS)
add_lit_target(check-compiler-rt
"Running all regression tests"
${LLVM_LIT_TESTSUITES}
DEPENDS ${LLVM_LIT_DEPENDS})
if(NOT TARGET check-all)
add_custom_target(check-all)
endif()
add_custom_target(compiler-rt-test-depends DEPENDS ${LLVM_LIT_DEPENDS})
add_dependencies(check-all check-compiler-rt)
endif()