Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.
The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.
For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.
Differential Revision: https://reviews.llvm.org/D117348
Makes lld-link work in a non-MSVC shell by autodetecting MSVC toolchain. Also
adds support for /winsysroot and a few other switches.
All this is done by refactoring to share code with clang-cl's existing support
for the same.
Differential Revision: https://reviews.llvm.org/D118070
The module flag to indicate use of hostcall is insufficient to catch
all cases where hostcall might be in use by a kernel. This is now
replaced by a function attribute that gets propagated to top-level
kernel functions via their respective call-graph.
If the attribute "amdgpu-no-hostcall-ptr" is absent on a kernel, the
default behaviour is to emit kernel metadata indicating that the
kernel uses the hostcall buffer pointer passed as an implicit
argument.
The attribute may be placed explicitly by the user, or inferred by the
AMDGPU attributor by examining the call-graph. The attribute is
inferred only if the function is not being sanitized, and the
implictarg_ptr does not result in a load of any byte in the hostcall
pointer argument.
Reviewed By: jdoerfert, arsenm, kpyzhov
Differential Revision: https://reviews.llvm.org/D119216
OpenCL C 3.0 __opencl_c_subgroups feature is slightly different
then other equivalent features and extensions (fp64 and 3d image writes):
OpenCL C 3.0 device can support the extension but not the feature.
cl_khr_subgroups requires subgroup independent forward progress.
This patch adjusts the check which is used when translating language
builtins to check either the extension or feature is supported.
Reviewed By: Anastasia
Differential Revision: https://reviews.llvm.org/D118999
OpenCL C 3.0 introduces optionality to some builtins, in particularly
to those which are conditionally supported with pipe, device enqueue
and generic address space features.
The idea is to conditionally support such builtins depending on the language options
being set for a certain feature. This allows users to define functions with names
of those optional builtins in OpenCL (as such names are not reserved).
Reviewed By: Anastasia
Differential Revision: https://reviews.llvm.org/D118605
This will be necessary later when we add support for evaluating logic
expressions such as && and ||.
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.
Reviewed-by: xazax.hun
Differential Revision: https://reviews.llvm.org/D119447
The pointer is always dereferenced, so assert the cast is correct (which it should be as we just created that ScalableVectorType) instead of returning nullptr
Add the atomic overloads for the `global` and `local` address spaces,
which are new in OpenCL 3.0. Ensure the preexisting `generic`
overloads are guarded by the generic address space feature macro.
Ensure a subset of the atomic builtins are guarded by the
`__opencl_c_atomic_order_seq_cst` and `__opencl_c_atomic_scope_device`
feature macros, and enable those macros for SPIR/SPIR-V targets in
`opencl-c-base.h`.
Also guard the `cl_ext_float_atomics` builtins with the atomic order
and scope feature macros.
Differential Revision: https://reviews.llvm.org/D119420
`CallDescriptions` for builtin functions relaxes the match rules
somewhat, so that the `CallDescription` will match for calls that have
some prefix or suffix. This was achieved by doing a `StringRef::contains()`.
However, this is somewhat problematic for builtins that are substrings
of each other.
Consider the following:
`CallDescription{ builtin, "memcpy"}` will match for
`__builtin_wmemcpy()` calls, which is unfortunate.
This patch addresses/works around the issue by checking if the
characters around the function's name are not part of the 'name'
semantically. In other words, to accept a match for `"memcpy"` the call
should not have alphanumeric (`[a-zA-Z]`) characters around the 'match'.
So, `CallDescription{ builtin, "memcpy"}` will not match on:
- `__builtin_wmemcpy: there is a `w` alphanumeric character before the match.
- `__builtin_memcpyFOoBar_inline`: there is a `F` character after the match.
- `__builtin_memcpyX_inline`: there is an `X` character after the match.
But it will still match for:
- `memcpy`: exact match
- `__builtin_memcpy`: there is an _ before the match
- `__builtin_memcpy_inline`: there is an _ after the match
- `memcpy_inline_builtinFooBar`: there is an _ after the match
Reviewed By: NoQ
Differential Revision: https://reviews.llvm.org/D118388
In Clang we can attach TBAA metadata based on the load/store intrinsics
based on the operation's element type.
This also contains changes to InstCombine where the AArch64-specific
intrinsics are transformed into generic LLVM load/store operations,
to ensure that all metadata is transferred to the new instruction.
There will be some further work after this patch to also emit TBAA
metadata for SVE's gather/scatter- and struct load/store intrinsics.
Reviewed By: paulwalker-arm
Differential Revision: https://reviews.llvm.org/D119319
An impilt used of C++ module without prebuild path may cause crash.
For example:
```
// ./dir1/C.cppm
export module C;
// ./dir2/B.cppm
export module B;
import C;
// ./A.cpp
import B;
import C;
```
When we compile A.cpp without the prebuild path of C.pcm, the compiler
will crash.
```
clang++ -std=c++20 --precompile -c ./dir1/C.cppm -o ./dir1/C.pcm
clang++ -std=c++20 --precompile -fprebuilt-module-path=./dir2 -c
./dir2/B.cppm -o ./dir2/B.pcm
clang++ -std=c++20 -fprebuilt-module-path=./dir2 A.cpp
```
The prebuilt path of module C is cached when import module B, and in the
function HeaderSearch::getCachedModuleFileName, the compiler try to get
the filename by modulemap without check if modulemap exists, and there
is no modulemap in C++ module.
Reviewed By: ChuanqiXu
Differential review: https://reviews.llvm.org/D119426
The superclass method handles a bunch of useful things. For example
it applies flags such as `-fnew-alignment` which doesn't work without
this patch.
Differential Revision: https://reviews.llvm.org/D118573
Due to the way type units work, this would lead to a declaration in a
type unit of a local type in a CU - which is ambiguous. Rather than
trying to resolve that relative to the CU that references the type unit,
let's just not try to simplify these names.
Longer term this should be fixed by not putting the template
instantiation in a type unit to begin with - since it references an
internal linkage type, it can't legitimately be duplicated/in more than
one translation unit, so skip the type unit overhead. (but the right fix
for that is to move type unit management into a DICompositeType flag
(dropping the "identifier" field is not a perfect solution since it
breaks LLVM IR linking decl/def merging during IR linking))
Lambda names aren't entirely canonical (as demonstrated by the
cross-project-test added here) at the moment (we should fix that for a
bunch of reasons) - even if the template referencing them is
non-simplified, other names referencing /that/ template can't be
simplified either because type units might cause a different template to
be picked up that would conflict with the expected name.
(other than for roundtripping precision, it'd be OK to simplify types
that reference types that reference lambdas - but best be consistent
between the roundtrip/verify mode and the actual simplified template
names mode)
The introduction and some examples are on this page:
https://devblogs.microsoft.com/cppblog/announcing-jmc-stepping-in-visual-studio/
The `/JMC` flag enables these instrumentations:
- Insert at the beginning of every function immediately after the prologue with
a call to `void __fastcall __CheckForDebuggerJustMyCode(unsigned char *JMC_flag)`.
The argument for `__CheckForDebuggerJustMyCode` is the address of a boolean
global variable (the global variable is initialized to 1) with the name
convention `__<hash>_<filename>`. All such global variables are placed in
the `.msvcjmc` section.
- The `<hash>` part of `__<hash>_<filename>` has a one-to-one mapping
with a directory path. MSVC uses some unknown hashing function. Here I
used DJB.
- Add a dummy/empty COMDAT function `__JustMyCode_Default`.
- Add `/alternatename:__CheckForDebuggerJustMyCode=__JustMyCode_Default` link
option via ".drectve" section. This is to prevent failure in
case `__CheckForDebuggerJustMyCode` is not provided during linking.
Implementation:
All the instrumentations are implemented in an IR codegen pass. The pass is placed immediately before CodeGenPrepare pass. This is to not interfere with mid-end optimizations and make the instrumentation target-independent (I'm still working on an ELF port in a separate patch).
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D118428
In preparing for module mangling changes I noticed some issues with
the way we check for std::basic_string instantiations and friends.
*) there's a single routine for std::basic_{i,o,io}stream but it is
templatized on the length of the name. Really? just use a
StringRef, rather than clone the entire routine just for
'basic_iostream'.
*) We have a helper routine to check for char type, and call it from
several places. But given all the instantiations are of the form
TPL<char, Other<char> ...> we could just check the first arg is char
and the later templated args are instantiating that same type. A
simpler type comparison.
*) Because basic_string has a third allocator parameter, it is open
coded, which I found a little confusing. But otherwise it's exactly
the same pattern as the iostream ones. Just tell that checker about
whether there's an expected allocator argument.[*]
*) We may as well return in each block of mangleStandardSubstitution
once we determine it is not one of the entities of interest -- it
certainly cannot be one of the other kinds of entities.
FWIW this shaves about 500 bytes off the executable.
[*] I suppose we could also have this routine a tri-value, with one to
indicat 'it is this name, but it's not the one you're looking for', to
avoid later calls trying different names?
Reviewd By: ChuanqiXu
Differential Revision: https://reviews.llvm.org/D119333
Even after D86621 <https://reviews.llvm.org/D86621>, `clang -m32` on
Solaris/sparcv9 doesn't inline atomics with 8-byte operands, unlike `gcc`.
This leads to many link failures in the testsuite (undefined references to
`__atomic_load_8` and `__sync_val_compare_and_swap_8`. Until a proper
codegen fix can be implemented, this patch works around the first of those
by linking with `-latomic`.
Tested on `sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D118021
Reduce the amount of repetition in the declarations by leveraging more
TableGen constructs. This is in preparation for adding the OpenCL 3.0
atomics feature optionality.
This introduces the new __ARM_FEATURE_MOPS ACLE feature test macro,
which signals the availability of the new Armv8.8-A/Armv9.3-A
instructions for standardising memcpy, memset and memmove operations.
This patch supersedes the one from https://reviews.llvm.org/D116160.
Differential Revision: https://reviews.llvm.org/D118199
Neither LLDB nor GDB seem to work with DWARF 5 debug information on
Windows right now.
This applies the same change as in
9c62728610 (Default to DWARFv4 on Windows)
to the MinGW driver too.
Differential Revision: https://reviews.llvm.org/D119326
ASTReader
This is a cleanup to reduce the lines of code to handle default template
argument in ASTReader.
Reviewed By: urnathan
Differential Revision: https://reviews.llvm.org/D118437
Ensure CLANG_PLUGIN_SUPPORT is compatible with llvm_add_library.
Fixes an issue noted in D111100.
Differential Revision: https://reviews.llvm.org/D119199
Fixes https://github.com/llvm/llvm-project/issues/53576.
There was an inconsistency in formatting of delete expressions.
Before:
```
delete (void*)a;
delete[](void*) a;
```
After this patch:
```
delete (void*)a;
delete[] (void*)a;
```
Reviewed By: HazardyKnusperkeks, owenpan
Differential Revision: https://reviews.llvm.org/D119117
An error in the tablegen description affects the declarations
provided by `-fdeclare-opencl-builtins` for `atomic_fetch_add` and
`atomic_fetch_sub`.
The atomic argument should be an atomic_half, not an atomic_float.
LRGraph is the key component of the clang pseudo parser, it is a
deterministic handle-finding finite-state machine, which is used to
generated the LR parsing table.
Separate from https://reviews.llvm.org/D118196.
Differential Revision: https://reviews.llvm.org/D119172
This will allow moving the IncludeCleaner library essentials to Clang
and decoupling them from the majority of clangd.
The patch itself just moves the code, it doesn't change existing
functionality.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D119130
Alex Lorenz