Go to file
David Zarzycki 1d297f9064 [lit] Sort test start times based on prior test timing data
Lit as it exists today has three hacks that allow users to run tests earlier:

1) An entire test suite can set the `is_early` boolean.
2) A very recently introduced "early_tests" feature.
3) The `--incremental` flag forces failing tests to run first.

All of these approaches have problems.

1) The `is_early` feature was until very recently undocumented. Nevertheless it still lacks testing and is a imprecise way of optimizing test starting times.
2) The `early_tests` feature requires manual updates and doesn't scale.
3) `--incremental` is undocumented, untested, and it requires modifying the *source* file system by "touching" the file. This "touch" based approach is arguably a hack because it confuses editors (because it looks like the test was modified behind the back of the editor) and "touching" the test source file doesn't work if the test suite is read only from the perspective of `lit` (via advanced filesystem/build tricks).

This patch attempts to simplify and address all of the above problems.

This patch formalizes, documents, tests, and defaults lit to recording the execution time of tests and then reordering all tests during the next execution. By reordering the tests, high core count machines run faster, sometimes significantly so.

This patch also always runs failing tests first, which is a positive user experience win for those that didn't know about the hidden `--incremental` flag.

Finally, if users want, they can _optionally_ commit the test timing data (or a subset thereof) back to the repository to accelerate bots and first-time runs of the test suite.

Reviewed By: jhenderson, yln

Differential Revision: https://reviews.llvm.org/D98179
2021-03-16 05:23:04 -04:00
.github Removing the main to master sync GitHub workflow. 2021-01-28 12:18:25 -08:00
clang [RISCV] Don't emit #undef BUILTIN from RISCVVEmitter.cpp 2021-03-16 14:57:45 +08:00
clang-tools-extra [clangd] Optionally add reflection for clangd-index-server 2021-03-15 21:07:25 +01:00
compiler-rt [AArch64][ASAN] Disable fgets_fputs.cpp test. 2021-03-16 07:00:19 +01:00
debuginfo-tests [dexter] Check path != None before calling os.path.exists 2021-03-15 11:40:05 +00:00
flang [flang] Create intrinsics modules directory (contd.) 2021-03-15 15:38:05 -07:00
libc [libc][Obvious] Fix except flags reading overflow detected by asan. 2021-03-13 16:30:33 -08:00
libclc libclc: Add clspv target to libclc 2021-03-04 00:19:10 -05:00
libcxx [libcxx] [test] Fix the temp_directory_path test for windows 2021-03-15 19:24:56 +02:00
libcxxabi [runtimes] Use add_lit_testsuite to register lit testsuites 2021-03-05 10:37:21 -08:00
libunwind [libunwind] Install the DLL when doing "ninja install" 2021-03-07 10:36:22 +02:00
lld [test] Add ability to get error messages from CMake for errc substitution 2021-03-15 20:56:08 +01:00
lldb Fix 34885bffdf 2021-03-15 16:36:32 -07:00
llvm [lit] Sort test start times based on prior test timing data 2021-03-16 05:23:04 -04:00
mlir [lit] Sort test start times based on prior test timing data 2021-03-16 05:23:04 -04:00
openmp [OpenMP][FIX] Repair accidental replacement of _shfl_sync with _shfl 2021-03-15 22:46:00 -05:00
parallel-libs Reapply "Try enabling -Wsuggest-override again, using add_compile_options instead of add_compile_definitions for disabling it in unittests/ directories." 2020-07-22 17:50:19 -07:00
polly [Polly] Fix deprecation warning. NFC. 2021-03-15 14:31:16 -05:00
pstl Rename top-level LICENSE.txt files to LICENSE.TXT 2021-03-10 21:26:24 -08:00
runtimes [CMake] Rename RUNTIMES_BUILD to LLVM_RUNTIMES_BUILD 2021-03-03 10:58:51 -08:00
utils/arcanist Fix arc lint's clang-format rule: only format the file we were asked to format. 2020-10-11 14:24:23 -07:00
.arcconfig Add modern arc config for default "onto" branch 2021-02-22 11:58:13 -08:00
.arclint PR46997: don't run clang-format on clang's testcases. 2020-08-04 17:53:25 -07:00
.clang-format
.clang-tidy - Update .clang-tidy to ignore parameters of main like functions for naming violations in clang and llvm directory 2020-01-31 16:49:45 +00:00
.git-blame-ignore-revs NFC: Add whitespace-changing revisions to .git-blame-ignore-revs 2020-09-21 20:17:24 -04:00
.gitignore [NFC] Add CMakeUserPresets.json filename to .gitignore 2021-01-22 12:45:29 +01:00
CONTRIBUTING.md
README.md [doc] Use cmake's -S option to simplify the build instructions 2021-02-16 14:47:06 -06:00

README.md

The LLVM Compiler Infrastructure

This directory and its sub-directories contain source code for LLVM, a toolkit for the construction of highly optimized compilers, optimizers, and run-time environments.

The README briefly describes how to get started with building LLVM. For more information on how to contribute to the LLVM project, please take a look at the Contributing to LLVM guide.

Getting Started with the LLVM System

Taken from https://llvm.org/docs/GettingStarted.html.

Overview

Welcome to the LLVM project!

The LLVM project has multiple components. The core of the project is itself called "LLVM". This contains all of the tools, libraries, and header files needed to process intermediate representations and converts it into object files. Tools include an assembler, disassembler, bitcode analyzer, and bitcode optimizer. It also contains basic regression tests.

C-like languages use the Clang front end. This component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode -- and from there into object files, using LLVM.

Other components include: the libc++ C++ standard library, the LLD linker, and more.

Getting the Source Code and Building LLVM

The LLVM Getting Started documentation may be out of date. The Clang Getting Started page might have more accurate information.

This is an example work-flow and configuration to get and build the LLVM source:

  1. Checkout LLVM (including related sub-projects like Clang):

    • git clone https://github.com/llvm/llvm-project.git

    • Or, on windows, git clone --config core.autocrlf=false https://github.com/llvm/llvm-project.git

  2. Configure and build LLVM and Clang:

    • cd llvm-project

    • cmake -S llvm -B build -G <generator> [options]

      Some common build system generators are:

      • Ninja --- for generating Ninja build files. Most llvm developers use Ninja.
      • Unix Makefiles --- for generating make-compatible parallel makefiles.
      • Visual Studio --- for generating Visual Studio projects and solutions.
      • Xcode --- for generating Xcode projects.

      Some Common options:

      • -DLLVM_ENABLE_PROJECTS='...' --- semicolon-separated list of the LLVM sub-projects you'd like to additionally build. Can include any of: clang, clang-tools-extra, libcxx, libcxxabi, libunwind, lldb, compiler-rt, lld, polly, or debuginfo-tests.

        For example, to build LLVM, Clang, libcxx, and libcxxabi, use -DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi".

      • -DCMAKE_INSTALL_PREFIX=directory --- Specify for directory the full path name of where you want the LLVM tools and libraries to be installed (default /usr/local).

      • -DCMAKE_BUILD_TYPE=type --- Valid options for type are Debug, Release, RelWithDebInfo, and MinSizeRel. Default is Debug.

      • -DLLVM_ENABLE_ASSERTIONS=On --- Compile with assertion checks enabled (default is Yes for Debug builds, No for all other build types).

    • cmake --build build [-- [options] <target>] or your build system specified above directly.

      • The default target (i.e. ninja or make) will build all of LLVM.

      • The check-all target (i.e. ninja check-all) will run the regression tests to ensure everything is in working order.

      • CMake will generate targets for each tool and library, and most LLVM sub-projects generate their own check-<project> target.

      • Running a serial build will be slow. To improve speed, try running a parallel build. That's done by default in Ninja; for make, use the option -j NNN, where NNN is the number of parallel jobs, e.g. the number of CPUs you have.

    • For more information see CMake

Consult the Getting Started with LLVM page for detailed information on configuring and compiling LLVM. You can visit Directory Layout to learn about the layout of the source code tree.