This change eliminate the stack frame for the fast path and improves runtime performance.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106505
signal(2) and sigaction(2) have defined behaviors for invalid signal number
(EINVAL) and some programs rely on it.
The added test case also reveals that MSAN is too strict in this regard.
Test case passed on x86_64 Linux and AArch64 Linux.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106468
The current (default) line length is 80 columns.
That's based on old hardware and historical conventions.
There are no existent reasons to keep line length that small,
especially provided that our coding style uses quite lengthy
identifiers. The Linux kernel recently switched to 100,
let's start with 100 as well.
This change intentionally does not re-format code.
Re-formatting is intended to happen incrementally,
or on dir-by-dir basis separately.
Reviewed By: vitalybuka, melver, MaskRay
Differential Revision: https://reviews.llvm.org/D106436
This reverts commit 6b2a96285b.
The ccache builders are still failing. Looks like they need to be updated to
get the llvm-zorg config change in 490633945677656ba75d42ff1ca9d4a400b7b243.
I'll re-apply this as soon as the builders are updated.
This reapplies commit a7733e9556 ("Re-apply
[ORC][ORC-RT] Add initial native-TLV support to MachOPlatform."), and
d4abdefc99 ("[ORC-RT] Rename macho_tlv.x86-64.s
to macho_tlv.x86-64.S (uppercase suffix)").
These patches were reverted in 48aa82cacb while I
investigated bot failures (e.g.
https://lab.llvm.org/buildbot/#/builders/109/builds/18981). The fix was to
disable building of the ORC runtime on buliders using ccache (which is the same
fix used for other compiler-rt projects containing assembly code). This fix was
commited to llvm-zorg in 490633945677656ba75d42ff1ca9d4a400b7b243.
This patch changes the linkage type of a compiler-rt func
(__llvm_write_binary_ids) to fix the sanitizer-windows bot
build issue introduced in change f984ac271.
The issue is as the following:
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingInternal.h(201):
error C2496: '__llvm_write_binary_ids': 'selectany' can only be applied
to data items with external linkage
Since d564cfb53c moved
__hwasan_tag_mismatch4 this test has been reporting
a frame 0 of __hwasan_tag_mismatch_v2.
This failure can be seen on our bots:
https://lab.llvm.org/buildbot/#/builders/185/builds/170
Before the change:
#0 0xaaaaba100e40 in main <...>/register-dump-read.c:21:10
After the change:
#0 0xaaaab8494bec in __hwasan_tag_mismatch_v2 <...>/hwasan/hwasan_tag_mismatch_aarch64.S:147
#1 0xaaaab84b4df8 in main <..>/register-dump-read.c:14:10
Update the test to check for a main frame as either frame
0 or frame 1.
Patch "sanitizer_common: modernize SpinMutex" added default
ctor to StaticSpinMutex. But it broke some gcc bots with:
scudo_tsd_exclusive.cpp:25:22: error: non-local variable
‘__scudo::TSD’ declared ‘__thread’ needs dynamic initialization
https://lab.llvm.org/buildbot/#/builders/105/builds/12649
Unfortunatly none of empty ctor {}, no ctor, default constexpr ctor
work for different reasons. So remove StaticSpinMutex ctor
entirely and move deleted copy ctor back to SpinMutex.
Differential Revision: https://reviews.llvm.org/D106424
Some minor improvements:
1. Make StaticSpinMutex non-copyable.
2. Add LIKELY to Lock.
3. Move LockSlow into the .cpp file (now that we have it).
4. The only non-trivial change: use proc_yield(1) instread of proc_yield(10)
with the proportional increase in the number of spin iterations.
Latency of the PAUSE instruction has raised from ~1 cycle to ~100 cycles
in the recent Intel CPUs. So proc_yield(10) is too aggressive backoff.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106350
This reverts commit d4abdefc99 ("[ORC-RT] Rename
macho_tlv.x86-64.s to macho_tlv.x86-64.S (uppercase suffix)", and
a7733e9556 ("Re-apply "[ORC][ORC-RT] Add initial
native-TLV support to MachOPlatform."), while I investigate failures on
ccache builders (e.g. https://lab.llvm.org/buildbot/#/builders/109/builds/18981)
Reapplies fe1fa43f16, which was reverted in
6d8c63946c, with fixes:
1. Remove .subsections_via_symbols directive from macho_tlv.x86-64.s (it's
not needed here anyway).
2. Return error from pthread_key_create to the MachOPlatform to silence unused
variable warning.
Adds code to LLVM (MachOPlatform) and the ORC runtime to support native MachO
thread local variables. Adding new TLVs to a JITDylib at runtime is supported.
On the LLVM side MachOPlatform is updated to:
1. Identify thread local variables in the LinkGraph and lower them to GOT
accesses to data in the __thread_data or __thread_bss sections.
2. Merge and report the address range of __thread_data and thread_bss sections
to the runtime.
On the ORC runtime a MachOTLVManager class introduced which records the address
range of thread data/bss sections, and creates thread-local instances from the
initial data on demand. An orc-runtime specific tlv_get_addr implementation is
included which saves all register state then calls the MachOTLVManager to get
the address of the requested variable for the current thread.
We currently have 3 different mutexes:
- RWMutex
- BlockingMutex
- __tsan::Mutex
RWMutex and __tsan::Mutex are roughly the same,
except that tsan version supports deadlock detection.
BlockingMutex degrades better under heavy contention
from lots of threads (blocks in OS), but much slower
for light contention and has non-portable performance
and has larger static size and is not reader-writer.
Add a new mutex that combines all advantages of these
mutexes: it's reader-writer, has fast non-contended path,
supports blocking to gracefully degrade under higher contention,
has portable size/performance.
For now it's named Mutex2 for incremental submission. The plan is to:
- land this change
- then move deadlock detection logic from tsan
- then rename it to Mutex and remove tsan Mutex
- then typedef RWMutex/BlockingMutex to this mutex
SpinMutex stays as separate type because it has faster fast path:
1 atomic RMW per lock/unlock as compared to 2 for this mutex.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106231
I don't think the stat subsystem was ever used since tsan
development in 2012. But it adds lots of code and this
effectively dead code needs to be updated if the runtime
code changes, which adds maintanance cost for no benefit.
Normal profiler usually gives enough info and that info
is more trustworthy.
Remove the stats subsystem.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106276
Add pragma line so that errors messages point to the actual
source files rather than to the concatenated gotsan.cpp.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106275
We need the compiler generated variable to override the weak symbol of
the same name inside the profile runtime, but using LinkOnceODRLinkage
results in weak symbol being emitted in which case the symbol selected
by the linker is going to depend on the order of inputs which can be
fragile.
This change replaces the use of weak definition inside the runtime with
a weak alias. We place the compiler generated symbol inside a COMDAT
group so dead definition can be garbage collected by the linker.
We also disable the use of runtime counter relocation on Darwin since
Mach-O doesn't support weak external references, but Darwin already uses
a different continous mode that relies on overmapping so runtime counter
relocation isn't needed there.
Differential Revision: https://reviews.llvm.org/D105176
For some reason we have 2 switches on arch and add
half of arch flags in one place and half in another.
Merge these 2 switches.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D106274
We obtain the current PC is all interceptors and collectively
common interceptor code contributes to overall slowdown
(in particular cheaper str/mem* functions).
The current way to obtain the current PC involves:
4493e1: e8 3a f3 fe ff callq 438720 <_ZN11__sanitizer10StackTrace12GetCurrentPcEv>
4493e9: 48 89 c6 mov %rax,%rsi
and the called function is:
uptr StackTrace::GetCurrentPc() {
438720: 48 8b 04 24 mov (%rsp),%rax
438724: c3 retq
The new way uses address of a local label and involves just:
44a888: 48 8d 35 fa ff ff ff lea -0x6(%rip),%rsi
I am not switching all uses of StackTrace::GetCurrentPc to GET_CURRENT_PC
because it may lead some differences in produced reports and break tests.
The difference comes from the fact that currently we have PC pointing
to the CALL instruction, but the new way does not yield any code on its own
so the PC points to a random instruction in the function and symbolizing
that instruction can produce additional inlined frames (if the random
instruction happen to relate to some inlined function).
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106046
This member is now only used when storage is heap-allocated so it does not
need to be const. Dropping 'const' eliminates cast warnings on many builders.
These have been failing on our bots for a while due to
incomplete backtraces. (you don't get the names of the
functions that did the access, just the reporter frames)
See:
https://lab.llvm.org/buildbot/#/builders/170/builds/180
Adds support for MachO static initializers/deinitializers and eh-frame
registration via the ORC runtime.
This commit introduces cooperative support code into the ORC runtime and ORC
LLVM libraries (especially the MachOPlatform class) to support macho runtime
features for JIT'd code. This commit introduces support for static
initializers, static destructors (via cxa_atexit interposition), and eh-frame
registration. Near-future commits will add support for MachO native
thread-local variables, and language runtime registration (e.g. for Objective-C
and Swift).
The llvm-jitlink tool is updated to use the ORC runtime where available, and
regression tests for the new MachOPlatform support are added to compiler-rt.
Notable changes on the ORC runtime side:
1. The new macho_platform.h / macho_platform.cpp files contain the bulk of the
runtime-side support. This includes eh-frame registration; jit versions of
dlopen, dlsym, and dlclose; a cxa_atexit interpose to record static destructors,
and an '__orc_rt_macho_run_program' function that defines running a JIT'd MachO
program in terms of the jit- dlopen/dlsym/dlclose functions.
2. Replaces JITTargetAddress (and casting operations) with ExecutorAddress
(copied from LLVM) to improve type-safety of address management.
3. Adds serialization support for ExecutorAddress and unordered_map types to
the runtime-side Simple Packed Serialization code.
4. Adds orc-runtime regression tests to ensure that static initializers and
cxa-atexit interposes work as expected.
Notable changes on the LLVM side:
1. The MachOPlatform class is updated to:
1.1. Load the ORC runtime into the ExecutionSession.
1.2. Set up standard aliases for macho-specific runtime functions. E.g.
___cxa_atexit -> ___orc_rt_macho_cxa_atexit.
1.3. Install the MachOPlatformPlugin to scrape LinkGraphs for information
needed to support MachO features (e.g. eh-frames, mod-inits), and
communicate this information to the runtime.
1.4. Provide entry-points that the runtime can call to request initializers,
perform symbol lookup, and request deinitialiers (the latter is
implemented as an empty placeholder as macho object deinits are rarely
used).
1.5. Create a MachO header object for each JITDylib (defining the __mh_header
and __dso_handle symbols).
2. The llvm-jitlink tool (and llvm-jitlink-executor) are updated to use the
runtime when available.
3. A `lookupInitSymbolsAsync` method is added to the Platform base class. This
can be used to issue an async lookup for initializer symbols. The existing
`lookupInitSymbols` method is retained (the GenericIRPlatform code is still
using it), but is deprecated and will be removed soon.
4. JIT-dispatch support code is added to ExecutorProcessControl.
The JIT-dispatch system allows handlers in the JIT process to be associated with
'tag' symbols in the executor, and allows the executor to make remote procedure
calls back to the JIT process (via __orc_rt_jit_dispatch) using those tags.
The primary use case is ORC runtime code that needs to call bakc to handlers in
orc::Platform subclasses. E.g. __orc_rt_macho_jit_dlopen calling back to
MachOPlatform::rt_getInitializers using __orc_rt_macho_get_initializers_tag.
(The system is generic however, and could be used by non-runtime code).
The new ExecutorProcessControl::JITDispatchInfo struct provides the address
(in the executor) of the jit-dispatch function and a jit-dispatch context
object, and implementations of the dispatch function are added to
SelfExecutorProcessControl and OrcRPCExecutorProcessControl.
5. OrcRPCTPCServer is updated to support JIT-dispatch calls over ORC-RPC.
6. Serialization support for StringMap is added to the LLVM-side Simple Packed
Serialization code.
7. A JITLink::allocateBuffer operation is introduced to allocate writable memory
attached to the graph. This is used by the MachO header synthesis code, and will
be generically useful for other clients who want to create new graph content
from scratch.
ptrauth stores info in the address of functions, so it's not the right address we should check if poisoned
rdar://75246928
Differential Revision: https://reviews.llvm.org/D106199
Vitaly Buka