Commit Graph

30 Commits

Author SHA1 Message Date
Amir Ayupov 883bf0e83d [BOLT][NFC] Fix braces usage in the rest of the codebase
Summary:
Refactor remaining bolt sources to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html).

(cherry picked from FBD33345885)
2021-12-28 18:43:53 -08:00
Maksim Panchenko 2f09f445b2 [BOLT][NFC] Fix file-description comments
Summary: Fix comments at the start of source files.

(cherry picked from FBD33274597)
2021-12-21 10:21:41 -08:00
Marius Wachtler 3b00a3a2f8 [PR] instr: change assert to allow FD 0 return by __open()
Summary: In some cases __open() is returning 0 for me.  The open
syscall will return a negative number (-1) on error so 0 should be
valid.

This happens when one compiles a BOLT instrumented executable of
pyston (python implementation) and afterwards pip installs a package
which needs to be compiled and sets CFLAGS=-pipe.  I could not reduce
it down to a small testcase but I guess it one can trigger when
manually closing std{in, out, err}.  Everything seems to work
normally when disabling the assert for 0 in getBinaryPath() - I
decided to also modify the second case in readDescriptions() even
though I did not run into that one yet.

(cherry picked from FBD32409548)
2021-11-05 16:47:30 +01:00
Marius Wachtler 46bc197d72 [PR] bolt_rt: getBinaryPath() increase max file path
Summary:
Increase the hard limit from 256 to 4096.
This fixes the 'Assertion failed: failed to open binary path' error I'm seeing.

(cherry picked from FBD31911946)
2021-10-25 20:42:29 +02:00
Vladislav Khmelevsky cb8d701b7b [PR] Disable instrumentation and hugify build for aarch64
Summary:
This patch temporarily disables instrumentation and higufy build not for
x86 platforms to be able to build llvm-bolt tool on aarch64.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD31738306)
2021-10-16 17:35:29 +03:00
Vladislav Khmelevsky dcdd37fdc2 [PR] Instrumentation: Sync file on dump
Summary:
Sync the file with storage device on data dump to stabilize
instrumentation testing

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD31738021)
2021-10-15 20:46:09 +03:00
Vasily Leonenko 9aa134dc2d [PR] Instrumentation: use TryLock for SimpleHashTable getter
Summary:
This commit introduces TryLock usage for SimpleHashTable getter to
avoid deadlock and relax syscalls usage which causes significant
overhead in runtime.
The old behavior left under -conservative-instrumentation option passed
to instrumentation library.
Also, this commit includes a corresponding test case: instrumentation of
executable which performs indirect calls from common code and signal
handler.

Note: in case if TryLock was failed to acquire the lock - this indirect
call will not be accounted in the resulting profile.

Vasily Leonenko,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30821949)
2021-08-08 04:50:06 +08:00
Vladislav Khmelevsky af58da4ef3 [PR] Instrumentation: Avoid generating GOT table in instrumentation library
Summary:
To avoid RELATIVE relocations avoid using of GOT table
by using hidden visibility for all symbols in library.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092712)
2021-07-22 00:04:28 +03:00
Vladislav Khmelevsky 553f28e921 [PR] Instrumentation: Fix start and fini trampoline pointers
Summary:
The trampolines are no loger pointers to the functions.  For
propper name resolving by bolt use extern "C" for all external symbols
in instr.cpp

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092698)
2021-07-31 00:29:23 +03:00
Vasily Leonenko 519cbbaa9a [PR] Instrumentation: Introduce instrumentation-binpath argument
Summary:
This commit introduces -instrumentation-binpath argument used
to point instuqmented binary in runtime in case if /proc/self/map_files
path is not accessible due to access restriction issues.

Vasily Leonenko
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092681)
2021-07-30 18:07:53 +03:00
Vladislav Khmelevsky 361f3b5576 [PR] Instrumentation: Fix runtime handlers for PIE files
Summary:
This commit fixes runtime instrumentation handlers for PIE
binaries case.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092522)
2021-06-23 18:24:09 +00:00
Elvina Yakubova 2ffd6e2b43 [PR] Instrumentation: Add support for opening libs based on links /proc/self/map_files
Summary:
This commit adds support for opening libs based on links
/proc/self/map_files.  For this we're getting current virtual address
and searching the lib in the directory with such address range. After
that, we're getting full path to the binary by using readlink
function. Direct read from link in /proc/self/map_files entries is not
possible because of lack of permissions.

Elvina Yakubova,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092422)
2021-01-19 02:08:55 +08:00
Vasily Leonenko ad79d51778 [PR] Instrumentation: Generate and use _start and _fini trampolines
Summary:
This commit implements new method for _start & _fini functions hooking
which allows to use relative jumps for future PIE & .so library support.
Instead of using absolute address of _start & _fini functions known on
linking stage - we'll use dynamically created trampoline functions and
use corresponding symbols in instrumentation runtime library.

As we would like to use instrumentation for dynamically loaded binaries
(with PIE & .so), thus we need to compile instrumentation library with
"-fPIC" flag to support relative address resolution for functions and
data.

For shared libraries we need to handle initialization of instrumentation
library case by using DT_INIT section entry point.

Also this commit adds detection if the binary is executable or shared
library based on existence of PT_INTERP header. In case of shared
library we save information about real library init function address
for further usage for instrumentation library init trampoline function
creation and also update DT_INIT to point instrumentation library init
function.

Functions called from init/fini functions should be called with forced
stack alignment to avoid issues with instructions which relies on it.
E.g. optimized string operations.

Vasily Leonenko,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30092316)
2021-06-19 04:08:35 +08:00
Amir Ayupov c7306cc219 Rebase: [BOLT][NFC] Expand auto types
Summary:
Expanded auto types across BOLT semi-automatically with the aid
of clangd LSP

(cherry picked from FBD33289309)
2021-04-08 00:19:26 -07:00
Vladislav Khmelevsky 76d346ca14 [BOLT][PR] Instrumentation: Introduce -no-counters-clear and -wait-forks options
Summary:
This PR introduces 2 new instrumentation options:
1. instrumentation-no-counters-clear: Discussed at https://github.com/facebookincubator/BOLT/issues/121
2. instrumentation-wait-forks: Since the instrumentation counters are mapped as MAP_SHARED it will be nice to add ability to wait until all forks of the parent process will die using tracking of process group.
The last patch is just emitBinary code refactor.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Pull Request resolved: https://github.com/facebookincubator/BOLT/pull/125
GitHub Author: Vladislav Khmelevskyi <Vladislav.Khmelevskyi@huawei.com>

(cherry picked from FBD26919011)
2021-03-09 16:18:11 -08:00
Rafael Auler da752c9c5c Fix license for a few remaining files
Summary: As titled.

(cherry picked from FBD28112137)
2021-03-17 15:04:19 -07:00
Alexander Shaposhnikov a0dd5b05dc [BOLT] Add support for dumping profile on MacOS
Summary: Add support for dumping profile on MacOS.

(cherry picked from FBD25751363)
2021-01-28 12:44:14 -08:00
Alexander Shaposhnikov 3b876cc3e7 [BOLT] Add support for dumping counters on MacOS
Summary: Add support for dumping counters on MacOS

(cherry picked from FBD25750516)
2021-01-28 12:32:03 -08:00
Alexander Shaposhnikov faaefff618 [BOLT] Fix operator new signature
Summary:
Use size_t for the first parameter of operator new.
https://en.cppreference.com/w/cpp/memory/new/operator_new

(cherry picked from FBD25750921)
2021-01-20 12:56:41 -08:00
Alexander Shaposhnikov e067f2adf4 Inject instrumentation's global dtor on MachO
Summary:
This diff is a preparation for dumping the profile generated by BOLT's instrumenation on MachO.

1/  Function "bolt_instr_fini" is placed into the predefined section "__fini"

2/ In the instrumentation pass we create a symbol "bolt_instr_fini" and
replace the last global destructor with it.

This is a temporary solution, in the future we need to register bolt_instr_fini in addition to the existing destructors without dropping the last one.

(cherry picked from FBD25071864)
2020-11-19 18:18:28 -08:00
Alexander Shaposhnikov 1cf23e5ee8 Link the instrumentation runtime on OSX
Summary: Link the instrumentation runtime on OSX.

(cherry picked from FBD24390019)
2020-11-17 13:57:29 -08:00
Alexander Shaposhnikov bbd9d610fe Add first bits to cross-compile the runtime for OSX
Summary: Add first bits to cross-compile the runtime for OSX.

(cherry picked from FBD24330977)
2020-10-15 03:51:56 -07:00
Rafael Auler c6799a689d [BOLT] Fix stack alignment for runtime lib
Summary:
Right now, the SAVE_ALL sequence executed upon entry of both
of our runtime libs (hugify and instrumentation) will cause the stack to
not be aligned at a 16B boundary because it saves 15 8-byte regs. Change
the code sequence to adjust for that. The compiler may generate code
that assumes the stack is aligned by using movaps instructions, which
will crash.

(cherry picked from FBD22744307)
2020-07-27 16:52:51 -07:00
Xun Li 9bd7161529 Adding automatic huge page support
Summary:
This patch enables automated hugify for Bolt.
When running Bolt against a binary with -hugify specified, Bolt will inject a call to a runtime library function at the entry of the binary. The runtime library calls madvise to map the hot code region into a 2M huge page. We support both new kernel with THP support and old kernels. For kernels with THP support we simply make a madvise call, while for old kernels, we first copy the code out, remap the memory with huge page, and then copy the code back.
With this change, we no longer need to manually call into hugify_self and precompile it with --hot-text. Instead, we could simply combine --hugify option with existing optimizations, and at runtime it will automatically move hot code into 2M pages.

Some details around the changes made:
1. Add an command line option to support --hugify. --hugify will automatically turn on --hot-text to get the proper hot code symbols. However, running with both --hugify and --hot-text is not allowed, since --hot-text is used on binaries that has precompiled call to hugify_self, which contradicts with the purpose of --hugify.
2. Moved the common utility functions out of instr.cpp to common.h, which will also be used by hugify.cpp. Added a few new system calls definitions.
3. Added a new class that inherits RuntimeLibrary, and implemented the necessary emit and link logic for hugify.
4. Added a simple test for hugify.

(cherry picked from FBD21384529)
2020-05-02 11:14:38 -07:00
Alexander Shaposhnikov 8c7f524afb [BOLT] Fix build of the runtime on OSX
Summary: Fix the compilation error on OSX

(cherry picked from FBD19269806)
2020-01-02 16:20:13 -08:00
Rafael Auler 16a497c627 [BOLT] Support full instrumentation
Summary:
Add full instrumentation support (branches, direct and
indirect calls). Add output statistics to show how many hot bytes
were split from cold ones in functions. Add -cold-threshold option
to allow splitting warm code (non-zero count). Add option in
bolt-diff to report missing functions in profile 2.

In instrumentation, fini hooks are fixed to run proper finalization
code after program finishes. Hooks for startup are added to setup
the runtime structures that needs initilization, such as indirect call
hash tables.

Add support for automatically dumping profile data every N seconds by
forking a watcher process during runtime.

(cherry picked from FBD17644396)
2019-12-13 17:27:03 -08:00
Rafael Auler ba31344fa9 [BOLT] Fix build for Mac
Summary:
Change our CMake config for the standalone runtime instrumentation
library to check for the elf.h header before using it, so the build
doesn't break on systems lacking it. Also fix a SmallPtrSet usage where
its elements are not really pointers, but uint64_t, breaking the build
in Apple's Clang.

(cherry picked from FBD17505759)
2019-09-20 11:29:35 -07:00
Rafael Auler cc4b2fb614 [BOLT] Efficient edge profiling in instrumented mode
Summary:
Change our edge profiling technique when using instrumentation
to do not instrument every edge. Instead, build the spanning tree
for the CFG and omit instrumentation for edges in the spanning tree.
Infer the edge count for these edges when writing the profile during
run time. The inference works with a bottom-up traversal of the spanning
tree and establishes the value of the edge connecting to the parent based
on a simple flow equation involving output and input edges, where the
only unknown variable is the parent edge.

This requires some engineering in the runtime lib to support dynamic
allocation for building these graphs at runtime.

(cherry picked from FBD17062773)
2019-08-07 16:09:50 -07:00
Rafael Auler 821480d27f [BOLT] Encode instrumentation tables in file
Summary:
Avoid directly allocating string and description tables in
binary's static data region, since they are not needed during runtime
except when writing the profile at exit. Change the runtime library to
open the tables on disk and read only when necessary.

(cherry picked from FBD16626030)
2019-08-02 11:20:13 -07:00
Rafael Auler 62aa74f836 [BOLT] Support instrumentation via runtime library
Summary:
To allow the development of future instrumentation work, this
patch adds support in BOLT for linking arbitrary libraries into the
binary processed by BOLT. We use orc relocation handling mechanism for
that. With this support, this patch also moves code programatically
generated in X86 assembly language by X86MCPlusBuilder to C code written
in a new library called bolt_rt. Change CMake to support this library as
an external project in the same way as clang does with compiler_rt. This
library is installed in the lib/ folder relative to BOLT root
installation and by default instrumentation will look for the library
at that location to finish processing the binary with instrumentation.

(cherry picked from FBD16572013)
2019-07-24 14:03:43 -07:00