[lit] Add "early_tests" config option

With enough cores, the slowest tests can significantly change the total testing time if they happen to run late. With this change, a test suite can improve performance (for high-end systems) by listing just a few of the slowest tests up front.

Reviewed By: jdenny, jhenderson

Differential Revision: https://reviews.llvm.org/D96594

llvm/docs/CommandGuide/lit.rst

@@ -397,6 +397,13 @@ executed, two important global variables are predefined:
  **root** The root configuration.  This is the top-most :program:`lit` configuration in
  the project.
 
+ **is_early** Whether the test suite as a whole should be given a head start
+ before other test suites run.
+
+ **early_tests** An explicit set of '/' separated test paths that should be
+ given a head start before other tests run. For example, the top five or so
+ slowest tests. See also: `--time-tests`
+
  **pipefail** Normally a test using a shell pipe fails if any of the commands
  on the pipe fail. If this is not desired, setting this variable to false
  makes the test fail only if the last command in the pipe fails.

llvm/utils/lit/lit/Test.py

@@ -404,4 +404,6 @@ class Test:
         This can be used for test suites with long running tests to maximize
         parallelism or where it is desirable to surface their failures early.
         """
+        if '/'.join(self.path_in_suite) in self.suite.config.early_tests:
+            return True
         return self.suite.config.is_early

llvm/utils/lit/lit/TestingConfig.py

@@ -127,6 +127,8 @@ class TestingConfig(object):
         self.limit_to_features = set(limit_to_features)
         # Whether the suite should be tested early in a given run.
         self.is_early = bool(is_early)
+        # List of tests to run early.
+        self.early_tests = {}
         self.parallelism_group = parallelism_group
         self._recursiveExpansionLimit = None
 

llvm/utils/lit/tests/Inputs/early-tests/aaa.txt

@@ -0,0 +1 @@
+# RUN: true

llvm/utils/lit/tests/Inputs/early-tests/bbb.txt

@@ -0,0 +1 @@
+# RUN: true

llvm/utils/lit/tests/Inputs/early-tests/lit.cfg

@@ -0,0 +1,7 @@
+import lit.formats
+config.name = 'early-tests'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+config.early_tests = { "subdir/ccc.txt" }

llvm/utils/lit/tests/Inputs/early-tests/subdir/ccc.txt

@@ -0,0 +1 @@
+# RUN: true

llvm/utils/lit/tests/early-tests.py

@@ -0,0 +1,9 @@
+## Check that we can run tests early.
+
+# RUN: %{lit} -j1 %{inputs}/early-tests | FileCheck %s
+
+# CHECK:     -- Testing: 3 tests, 1 workers --
+# CHECK-NEXT: PASS: early-tests :: subdir/ccc.txt
+# CHECK-NEXT: PASS: early-tests :: aaa.txt
+# CHECK-NEXT: PASS: early-tests :: bbb.txt
+# CHECK:     Passed: 3