maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/utils/lit/tests/max-failures.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

25 lines
808 B
Python
Raw Normal View History

[lit] Temporarily disable failing tests on Windows
2020-04-11 11:03:44 +08:00
# UNSUPPORTED: system-windows
[lit] Add a --max-failures option. - This is primarily useful as a "fail fast" mode for lit, where it will stop running tests after the first failure. - Patch by Max Moiseev. llvm-svn: 282452
2016-09-27 07:38:23 +08:00
# Check the behavior of --max-failures option.
#
[llvm] [lit] Support forcing lexical test order Add a new --order option to choose between available test orders: the default "smart" order, predictable "lexical" order or "random" order. Default to using lexical order and one job in the lit test suite. Differential Revision: https://reviews.llvm.org/D107695
2021-08-07 22:40:09 +08:00
# RUN: not %{lit} %{inputs}/max-failures > %t.out 2>&1
# RUN: not %{lit} --max-failures=1 %{inputs}/max-failures >> %t.out 2>&1
# RUN: not %{lit} --max-failures=2 %{inputs}/max-failures >> %t.out 2>&1
# RUN: not %{lit} --max-failures=0 %{inputs}/max-failures 2>> %t.out
[lit] Add a --max-failures option. - This is primarily useful as a "fail fast" mode for lit, where it will stop running tests after the first failure. - Patch by Max Moiseev. llvm-svn: 282452
2016-09-27 07:38:23 +08:00
# RUN: FileCheck < %t.out %s
#
[lit] Add SKIPPED test result category Track and print the number of skipped tests. Skipped tests are tests that should have been executed but weren't due to: * user interrupt [Ctrl+C] * --max-time (overall lit timeout) * --max-failures This is part of a larger effort to ensure that all discovered tests are properly accounted for. Add test for overall lit timeout feature (`--max-time` option) to observe skipped tests. Extend test for `--max-failures` option. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D77819
2019-03-08 12:48:24 +08:00
# CHECK-NOT: reached maximum number of test failures
[lit] Improve naming of test result categories Improve consistency when printing test results: Previously we were using different labels for group names (the header for the list of, e.g., failing tests) and summary count lines. For example, "Failing Tests"/"Unexpected Failures". This commit changes lit to label things consistently. Improve wording of labels: When talking about individual test results, the first word in "Unexpected Failures", "Expected Passes", and "Individual Timeouts" is superfluous. Some labels contain the word "Tests" and some don't. Let's simplify the names. Before: ``` Failing Tests (1): ... Expected Passes : 3 Unexpected Failures: 1 ``` After: ``` Failed Tests (1): ... Passed: 3 Failed: 1 ``` Reviewed By: ldionne Differential Revision: https://reviews.llvm.org/D77708
2020-04-08 13:48:39 +08:00
# CHECK-NOT: Skipped
# CHECK: Failed: 3
[lit] Add SKIPPED test result category Track and print the number of skipped tests. Skipped tests are tests that should have been executed but weren't due to: * user interrupt [Ctrl+C] * --max-time (overall lit timeout) * --max-failures This is part of a larger effort to ensure that all discovered tests are properly accounted for. Add test for overall lit timeout feature (`--max-time` option) to observe skipped tests. Extend test for `--max-failures` option. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D77819
2019-03-08 12:48:24 +08:00
# CHECK: reached maximum number of test failures, skipping remaining tests
[lit] Improve naming of test result categories Improve consistency when printing test results: Previously we were using different labels for group names (the header for the list of, e.g., failing tests) and summary count lines. For example, "Failing Tests"/"Unexpected Failures". This commit changes lit to label things consistently. Improve wording of labels: When talking about individual test results, the first word in "Unexpected Failures", "Expected Passes", and "Individual Timeouts" is superfluous. Some labels contain the word "Tests" and some don't. Let's simplify the names. Before: ``` Failing Tests (1): ... Expected Passes : 3 Unexpected Failures: 1 ``` After: ``` Failed Tests (1): ... Passed: 3 Failed: 1 ``` Reviewed By: ldionne Differential Revision: https://reviews.llvm.org/D77708
2020-04-08 13:48:39 +08:00
# CHECK: Skipped: 2
# CHECK: Failed : 1
[lit] Add SKIPPED test result category Track and print the number of skipped tests. Skipped tests are tests that should have been executed but weren't due to: * user interrupt [Ctrl+C] * --max-time (overall lit timeout) * --max-failures This is part of a larger effort to ensure that all discovered tests are properly accounted for. Add test for overall lit timeout feature (`--max-time` option) to observe skipped tests. Extend test for `--max-failures` option. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D77819
2019-03-08 12:48:24 +08:00
# CHECK: reached maximum number of test failures, skipping remaining tests
[lit] Improve naming of test result categories Improve consistency when printing test results: Previously we were using different labels for group names (the header for the list of, e.g., failing tests) and summary count lines. For example, "Failing Tests"/"Unexpected Failures". This commit changes lit to label things consistently. Improve wording of labels: When talking about individual test results, the first word in "Unexpected Failures", "Expected Passes", and "Individual Timeouts" is superfluous. Some labels contain the word "Tests" and some don't. Let's simplify the names. Before: ``` Failing Tests (1): ... Expected Passes : 3 Unexpected Failures: 1 ``` After: ``` Failed Tests (1): ... Passed: 3 Failed: 1 ``` Reviewed By: ldionne Differential Revision: https://reviews.llvm.org/D77708
2020-04-08 13:48:39 +08:00
# CHECK: Skipped: 1
# CHECK: Failed : 2
[lit] Add SKIPPED test result category Track and print the number of skipped tests. Skipped tests are tests that should have been executed but weren't due to: * user interrupt [Ctrl+C] * --max-time (overall lit timeout) * --max-failures This is part of a larger effort to ensure that all discovered tests are properly accounted for. Add test for overall lit timeout feature (`--max-time` option) to observe skipped tests. Extend test for `--max-failures` option. Reviewed By: jdenny Differential Revision: https://reviews.llvm.org/D77819
2019-03-08 12:48:24 +08:00
[lit] Leverage argparse features to remove some code Reviewed By: rnk, serge-sans-paille Differential Revision: https://reviews.llvm.org/D68589 llvm-svn: 374405
2019-10-11 02:03:37 +08:00
# CHECK: error: argument --max-failures: requires positive integer, but found '0'