Graceful lit shutdown on user keyboard interrupt [Ctrl+C] was a
longstanding goal of mine. After a few refactorings this revision
finally enables it. We use the following strategy to deal with
KeyboardInterrupt:
https://noswap.com/blog/python-multiprocessing-keyboardinterrupt
Printing of a helpful summary for interrupted runs (just as the one for
completed runs) will be tackled in future revisions.
Reviewed By: serge-sans-paille, rnk
Differential Revision: https://reviews.llvm.org/D77365
On shutdown, the result complete handler is not racing with the main
thread anymore because we are now always waiting for process pool
termination via
```
finally:
pool.join()
```
Diagnostics from modules do not have a `main-file` listed. Tweak
`clang-parse-diagnostics-file` to patch this up. Previously, the call
to `os.path.basename` would crash.
Radar-Id: rdar://problem/59000292
The problem on Windows was that the \b in "..\bin" was interpreted
as an escape sequence. Use r"" strings to prevent that.
This reverts commit ab11b9eefa,
with raw strings in the lit.site.cfg.py.in files.
Differential Revision: https://reviews.llvm.org/D77184
Currently, all generated lit.site.cfg files contain absolute paths.
This makes it impossible to build on one machine, and then transfer the
build output to another machine for test execution. Being able to do
this is useful for several use cases:
1. When running tests on an ARM machine, it would be possible to build
on a fast x86 machine and then copy build artifacts over after building.
2. It allows running several test suites (clang, llvm, lld) on 3
different machines, reducing test time from sum(each test suite time) to
max(each test suite time).
This patch makes it possible to pass a list of variables that should be
relative in the generated lit.site.cfg.py file to
configure_lit_site_cfg(). The lit.site.cfg.py.in file needs to call
`path()` on these variables, so that the paths are converted to absolute
form at lit start time.
The testers would have to have an LLVM checkout at the same revision,
and the build dir would have to be at the same relative path as on the
builder.
This does not yet cover how to figure out which files to copy from the
builder machine to the tester machines. (One idea is to look at the
`--graphviz=test.dot` output and copy all inputs of the `check-llvm`
target.)
Differential Revision: https://reviews.llvm.org/D77184
Having the sync script work for this file seems better
than matching the order of headers in the cmake file.
Also, not having to manually sort the list is nice, even
if gn's automated sorting doesn't quite match the artisanal
order in the cmake file.
Summary:
In the patch: https://reviews.llvm.org/D42654
De-duplicate utils/update_{llc_,}test_checks.py, Some common part has
been move to common.py. The SCRUB_LOOP_COMMENT_RE has been moved to
common.py, but forgetting to remove from asm.py.
This patch is to remove the redundant SCRUB_LOOP_COMMENT_RE in asm.py
and use common.SCRUB_LOOP_COMMENT_RE.
While D68850 allowed functions to be deleted I accidentally saved some
version of the function to be used once a suitable prefix was found.
This turned out to be problematic when the occasionally deleted function
is also occasionally modified. The test case is adjusted to resemble the
case in which the problem was found.
Reviewed By: lebedev.ri
Differential Revision: https://reviews.llvm.org/D76586
Summary:
This change makes the labels on the x-axis of a comparison chart look
like: "0%", "(0%, 10%)", "[10%, 20%)", and so on.
Previously, each label was the same (a concatenation of all the possible
coverage buckets).
Reviewers: djtodoro
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D77155
--no-threads is a name copied from gold.
gold has --no-thread, --thread-count and several other --thread-count-*.
There are needs to customize the number of threads (running several lld
processes concurrently or customizing the number of LTO threads).
Having a single --threads=N is a straightforward replacement of gold's
--no-threads + --thread-count.
--no-threads is used rarely. So just delete --no-threads instead of
keeping it for compatibility for a while.
If --threads= is specified (ELF,wasm; COFF /threads: is similar),
--thinlto-jobs= defaults to --threads=,
otherwise all available hardware threads are used.
There is currently no way to override a --threads={1,2,...}. It is still
a debate whether we should use --threads=all.
Reviewed By: rnk, aganea
Differential Revision: https://reviews.llvm.org/D76885
The shtest-inject test relied on being executed in "single process" mode
and started to fail with a `PicklingError` after it was removed:
```
Can't pickle <class 'lit.TestingConfig.CustomFormat'>: attribute
lookup lit.TestingConfig.CustomFormat failed
```
This happened because the test config has to be serialized to the worker
process, but apparently the `CustomFormat` class defined inline is not
serializable.
This change allows passing the tested functionality (preamble_commands)
directly to `lit.formats.ShTest` so we can use it directly in the test.
Remove the "serial run" abstraction which bypasses Python's
`multiprocessing.Pool` and instead directly runs tests without spawning
worker processes. This abstraction has not offered the benefits I hoped
it would and therefore does not carry its weight.
In previous commits [1,2] I changed worker.py to only send back the test
result from the worker process instead of the whole test object. This
was a mistake. lit.Test contains fields (e.g., xfials, requires,
unsupported) that are only populated when we actually execute the test,
but are queried when we report the results in the parent process. This
commit essentially reverts the following changes:
[1] a3d2f9b53a
[2] 17bb660fb8
- only complain about duplicate 'sources' lines if they contain
more than 1 elements each (before, 0 and 1 element lists were
counted too). In practice, this only has an effect for
clang/lib/Headers/BUILD.gn
- make the '# NOSORT' diag actually work. This too only affects
clang/lib/Headers/BUILD.gn.
In aggregate, changes to clang/lib/Headers/BUILD.gn still can't
be auto-merged, but for a slighly better reason.
A previous attempt to cleanup module imports broke installing via
pip/setup.py [1]. This should be fixed now.
[1] cf252240e8
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D76940
This minor refactoring allows reducing the amount of processing that
is duplicated when we re-run a flaky test. It also has the nice
side effect that libc++'s current test format supports flaky .sh.cpp
tests, because those are built on top of _runShTest, not executeShTest.
This allows defining substitutions in terms of other substitutions. For
example, a %build substitution could be defined in terms of a %cxx
substitution as '%cxx %s -o %t.exe' and the script would be properly
expanded.
Differential Revision: https://reviews.llvm.org/D76178
Summary:
Some compressed instructions match against negative values; store
immediates as a signed value such that these patterns will now match
the intended instructions.
Reviewers: asb, lenary, PaoloS
Reviewed By: asb
Subscribers: rbar, johnrusso, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, Jim, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, evandro, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D76767
Follow-up of D72172 and D72180
This patch passes `uint64_t Address` to print methods of PC-relative
operands so that subsequent target specific patches can change
`*InstPrinter::print{Operand,PCRelImm,...}` to customize the output.
Add MCInstPrinter::PrintBranchImmAsAddress which is set to true by
llvm-objdump.
```
// Current llvm-objdump -d output
aarch64: 20000: bl #0
ppc: 20000: bl .+4
x86: 20000: callq 0
// Ideal output
aarch64: 20000: bl 0x20000
ppc: 20000: bl 0x20004
x86: 20000: callq 0x20005
// GNU objdump -d. The lack of 0x is not ideal because the result cannot be re-assembled
aarch64: 20000: bl 20000
ppc: 20000: bl 0x20004
x86: 20000: callq 20005
```
In `lib/Target/X86/X86GenAsmWriter1.inc` (generated by `llvm-tblgen -gen-asm-writer`):
```
case 12:
// CALL64pcrel32, CALLpcrel16, CALLpcrel32, EH_SjLj_Setup, JCXZ, JECXZ, J...
- printPCRelImm(MI, 0, O);
+ printPCRelImm(MI, Address, 0, O);
return;
```
Some targets have 2 `printOperand` overloads, one without `Address` and
one with `Address`. They should annotate derived `Operand` properly with
`let OperandType = "OPERAND_PCREL"`.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D76574
This allows creating custom test formats on top of `executeShTest` that
inject commands at the beginning of the file being parsed, without
requiring these commands to physically appear in the test file itself.
For example, one could define a test format that prints out additional
debug information at the beginning of each test. More realistically,
this has been used to define custom test formats like one that supports
compilation failure tests (e.g. with the extension `compile.fail.cpp`)
by injecting a command that calls the compiler on the file itself and
expects it to fail.
Without this change, the only alternative is to create a temporary file
with the same content as the original test, then prepend the desired
`// RUN:` lines to that file, and call `executeShTest` on that file
instead. This is both slow and cumbersome to do.
Differential Revision: https://reviews.llvm.org/D76290
Remove the local versions of the IR_FUNCTION_RE matcher (they weren't doing anything different), and ensure all the function name matchers accept '.' and '-'.
We don't need to use '\.' inside python regex sets either, or '\-' as long as thats at the end of the set.
For MTE error reporting we will need to expose interfaces for crash handlers
to use to interpret scudo headers and metadata. The intent is that these
interfaces will live in scudo/interface.h.
Move the existing interface.h into an include/scudo directory and make it
independent of the internal headers, so that we will be able to add the
interfaces there.
Differential Revision: https://reviews.llvm.org/D76648
This reverts commit e9f22fd429.
When building with -DLLVM_USE_SANITIZER="Thread", check-llvm has 70
failing tests with this revision, and 29 without this revision.
This commit adds a new keyword in lit called ALLOW_RETRIES. This keyword
takes a single integer as an argument, and it allows the test to fail that
number of times before it first succeeds.
This work attempts to make the existing test_retry_attempts more flexible
by allowing by-test customization, as well as eliminate libc++'s FLAKY_TEST
custom logic.
Differential Revision: https://reviews.llvm.org/D76288
This patch generates TableGen descriptions for the specified register
banks which contain a list of register sizes corresponding to the
available HwModes. The appropriate size is used during codegen according
to the current HwMode. As this HwMode was not available on generation,
it is set upon construction of the RegisterBankInfo class. Targets
simply need to provide the HwMode argument to the
<target>GenRegisterBankInfo constructor.
The RISC-V RegisterBankInfo constructor has been updated accordingly
(plus an unused argument removed).
Differential Revision: https://reviews.llvm.org/D76007
This patch rewrites the RegisterBankEmitter class to derive
RegisterClassHierarchy from CodeGenTarget::getRegBank() rather than
constructing our own copy. All are now accessed through a const
reference.
Differential Revision: https://reviews.llvm.org/D76006
For context, the proposed RISC-V bit manipulation extension has a subset
of instructions which require one of two SubtargetFeatures to be
enabled, 'zbb' or 'zbp', and there is no defined feature which both of
these can imply to use as a constraint either (see comments in D65649).
AssemblerPredicates allow multiple SubtargetFeatures to be declared in
the "AssemblerCondString" field, separated by commas, and this means
that the two features must both be enabled. There is no equivalent to
say that _either_ feature X or feature Y must be enabled, short of
creating a dummy SubtargetFeature for this purpose and having features X
and Y imply the new feature.
To solve the case where X or Y is needed without adding a new feature,
and to better match a typical TableGen style, this replaces the existing
"AssemblerCondString" with a dag "AssemblerCondDag" which represents the
same information. Two operators are defined for use with
AssemblerCondDag, "all_of", which matches the current behaviour, and
"any_of", which adds the new proposed ORing features functionality.
This was originally proposed in the RFC at
http://lists.llvm.org/pipermail/llvm-dev/2020-February/139138.html
Changes to all current backends are mechanical to support the replaced
functionality, and are NFCI.
At this stage, it is illegal to combine features with ands and ors in a
single AssemblerCondDag. I suspect this case is sufficiently rare that
adding more complex changes to support it are unnecessary.
Differential Revision: https://reviews.llvm.org/D74338
Summary: This patch renames the "llvm-gsym" tool directory to "llvm-gsymutil". Dependencies are also reduced to the bare minimum for llvm-gsymutil.
Reviewers: aprantl, thakis
Subscribers: mgorny, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D75291
Lots of headers pass around MemoryBuffer objects, but very few open
them. Let those that do include FileSystem.h.
Saves ~250 includes of Chrono.h & FileSystem.h:
$ diff -u thedeps-before.txt thedeps-after.txt | grep '^[-+] ' | sort | uniq -c | sort -nr
254 - ../llvm/include/llvm/Support/FileSystem.h
253 - ../llvm/include/llvm/Support/Chrono.h
237 - ../llvm/include/llvm/Support/NativeFormatting.h
237 - ../llvm/include/llvm/Support/FormatProviders.h
192 - ../llvm/include/llvm/ADT/StringSwitch.h
190 - ../llvm/include/llvm/Support/FormatVariadicDetails.h
...
This requires duplicating the file_t typedef, which is unfortunate. I
sunk the choice of mapping mode down into the cpp file using variable
template specializations instead of class members in headers.
Now that the Windows installer no longer does anything besides
self-extract, maybe it would make sense to distribute the toolchain as a
plain zip file in addition to the current installer.
Differential revision: https://reviews.llvm.org/D74896
Summary:
The type used to represent functional units in MC is
'unsigned', which is 32 bits wide. This is currently
not a problem in any upstream target as no one seems
to have hit the limit on this yet, but in our
downstream one, we need to define more than 32
functional units.
Increasing the size does not seem to cause a huge
size increase in the binary (an llc debug build went
from 1366497672 to 1366523984, a difference of 26k),
so perhaps it would be acceptable to have this patch
applied upstream as well.
Subscribers: hiraditya, jsji, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D71210
isPrefix was added to support the patches to align branches.
it relies on a switch over instruction names.
This moves those opcodes to a new format so the information is
tablegen and we can just check for a specific value in some bits
in TSFlags instead.
I've left the other function in place for now so that the
existing patches in phabricator will still work. I'll work with
the owner to get them migrated.
The Blocks runtime provide a header named Block.h.
It is generally preferable to avoid name collision with system headers
(reducing reliance on -isystem order, more friendly when navigating files in
an editor, etc).
Reviewed By: gribozavr2
Differential Revision: https://reviews.llvm.org/D74934
I was drafting a patch that would increase broadcast load usage,
but our shuffle scrubbing makes it impossible to see if the memory
operand offset was getting created correctly. I'm proposing to make
that an option (defaulted to 'off' for now to reduce regression
test churn).
The updated files provide examples of tests where we can now verify
that the pointer offset for a loaded memory operand is correct. We
still have stack and constant scrubbing that can obscure the operand
even if we don't scrub the entire instruction.
Differential Revision: https://reviews.llvm.org/D74775
This was checking for default operands in the current DAG instruction,
rather than the correct result operand list. I'm not entirly sure how
this managed to work before, but was failing for me when multiple
default operands were overridden.
\<foo\> is more correct, but since we use shell=True on Windows,
the < and > get interpreted as redirection operators.
Rather than adding cmd escaping, just use \bfoo\b, which is Good
Enough Often Enough.
Summary:
Previously TableGen would crash trying to print the undefined value as
an integer.
Change-Id: I3900071ceaa07c26acafb33bc49966d7d7a02828
Reviewers: nhaehnle
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D74210
Essentially, fold OrderedBasicBlock into BasicBlock, and make it
auto-invalidate the instruction ordering when new instructions are
added. Notably, we don't need to invalidate it when removing
instructions, which is helpful when a pass mostly delete dead
instructions rather than transforming them.
The downside is that Instruction grows from 56 bytes to 64 bytes. The
resulting LLVM code is substantially simpler and automatically handles
invalidation, which makes me think that this is the right speed and size
tradeoff.
The important change is in SymbolTableTraitsImpl.h, where the numbering
is invalidated. Everything else should be straightforward.
We probably want to implement a fancier re-numbering scheme so that
local updates don't invalidate the ordering, but I plan for that to be
future work, maybe for someone else.
Reviewed By: lattner, vsk, fhahn, dexonsmith
Differential Revision: https://reviews.llvm.org/D51664
Summary:
In the DAG pattern backend, `SimplifyTree` simplifies a pattern by
removing bitconverts between two identical types. But that function is
also run on the fragments list in instances of `PatFrags`, in which
the types haven't been specified yet. So the input and output of the
bitconvert always evaluate to the empty set of types, which makes them
compare equal. So the test always passes, and bitconverts are
unconditionally removed from the PatFrag RHS.
Fixed by spotting the empty type set and using it to inhibit the
optimization.
Reviewers: nhaehnle, hfinkel
Reviewed By: nhaehnle
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D74627
Summary:
Many directives are unavailable, and support for others may be limited.
This first draft has preliminary support for:
- conditional directives (including errors),
- data allocation (unsigned types up to 8 bytes, and ALIGN),
- equates/variables (numeric and text),
- and procedure directives (without parameters),
as well as COMMENT, ECHO, INCLUDE, INCLUDELIB, PUBLIC, and EXTERN. Text variables (aka text macros) are expanded in-place wherever the identifier occurs.
We deliberately ignore all ml.exe processor directives.
Prominent features not yet supported:
- structs
- macros (both procedures and functions)
- procedures (with specified parameters)
- substitution & expansion operators
Conditional directives are complicated by the fact that "ifdef rax" is a valid way to check if a file is being assembled for a 64-bit x86 processor; we add support for "ifdef <register>" in general, which requires adding a tryParseRegister method to all MCTargetAsmParsers. (Some targets require backtracking in the non-register case.)
Reviewers: rnk, thakis
Reviewed By: thakis
Subscribers: kerbowa, merge_guards_bot, wuzish, arsenm, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, mgorny, sbc100, jgravelle-google, hiraditya, aheejin, kbarton, fedor.sergeev, asb, rbar, johnrusso, simoncook, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, jsji, Jim, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72680
This is like -fdata-sections, and it's not part of /O2 by default for some reason.
In the cmake build, reduces the size of clang.exe from 70,358,016 bytes to 69,982,720 bytes.
clang-format.exe goes from 3,703,296 bytes to 3,331,072 bytes.
Differential Revision: https://reviews.llvm.org/D74573
Before, the script used `git log -SFoo.cpp` to find a commit where
the number of occurrences of "Foo.cpp" changed -- but since
a patch with
+ LLVMFoo.cpp
- Foo.cpp
contains the same number of instances of "Foo.cpp", the script
incorrectly skipped this type of rename.
As fix, look for '\bFoo\.cpp\b' instead and pass --pickaxe-regex
so that we can grep for word boundaries.
To test, check out 7531a5039f (which renamed in llvm/lib/IR
RemarkStreamer.cpp to LLVMRemarkStreamer.cpp) and look at the output of
the script. Before this change, it correctly assigned the addition
of LLVMRemarkStreamer.cpp to 7531a5039f but incorrectly assigned
the removal of RemarkStreamer.cpp to b8a847c. With this, it
correctly assigns both to 7531a5039f.
Basically change the layout to please `go build` and remove references to
`llvm-go`.
Update llvm/test/Bindings/Go/ to use the system go compiler
Differential Revision: https://reviews.llvm.org/D74540
Tablegen's DAGISelMatcher emits integers in a VBR format,
so if an integer is below 128 it can fit into a single
byte, otherwise high bit is set, next byte is used etc.
MatcherTable is essentially an unsigned char table. When
SelectionDAGISel parses the table it does a reverse translation.
In a situation when numeric value of an integer to emit is
unknown it can be emitted not as OPC_EmitInteger but as
OPC_EmitStringInteger using a symbolic name of the value.
In this situation the value should not exceed 127.
One of the situations when OPC_EmitStringInteger is used is
if we need to emit a subreg into a matcher table. However,
number of subregs can exceed 127. Currently last defined subreg
for AMDGPU is 192. That results in a silent bug in the ISel
with matcher reading from an invalid offset.
Fixed this bug to emit actual VBR encoded value for a subregs
which value exceeds 127.
Differential Revision: https://reviews.llvm.org/D74368
Update test scripts were limited because they performed a single action
on the entire file and if that action was controlled by arguments, like
the one introduced in D68819, there was no record of it.
This patch introduces the capability of changing the arguments passed to
the script "on-the-fly" while processing a test file. In addition, an
"on/off" switch was added so that processing can be disabled for parts
of the file where the content is simply copied. The last extension is a
record of the invocation arguments in the auto generated NOTE. These
arguments are also picked up in a subsequent invocation, allowing
updates with special options enabled without user interaction.
To change the arguments the string `UTC_ARGS:` has to be present in a
line, followed by "additional command line arguments". That is
everything that follows `UTC_ARGS:` will be added to a growing list
of "command line arguments" which is reparsed after every update.
Reviewed By: arichardson
Differential Revision: https://reviews.llvm.org/D69701
Summary:
Simplifies the C++11-style "-> decltype(...)" return-type deduction.
Note that you have to be careful about whether the function return type
is `auto` or `decltype(auto)`. The difference is that bare `auto`
strips const and reference, just like lambda return type deduction. In
some cases that's what we want (or more likely, we know that the return
type is a value type), but whenever we're wrapping a templated function
which might return a reference, we need to be sure that the return type
is decltype(auto).
No functional change.
Subscribers: dexonsmith, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D74383
LLVM GN Syncbot