llvm-project/llvm/tools/llvm-profgen
wlei 856a6a5041 [CSSPGO][llvm-profgen] Trim and merge context beforehand to reduce memory usage
Currently we use a centralized string map(StringMap<FunctionSamples> ProfileMap) to store the profile while populating the sample, which might cause the memory usage bottleneck. I saw in an extreme case, there are thousands of samples whose context stack depth is >= 100. The memory consumption can be greater than 100GB.

As here the context is used for inlining, we can assume we won't have so many of inlinees keeping inlined at the same root function, so this change tried to cap the context stack and merge the samples for peak memory reduction and this is done after recursion compression.

The default value is -1 meaning no depth limit, in the future we can tune to a smaller one.

Reviewed By: hoy, wenlei

Differential Revision: https://reviews.llvm.org/D107800
2021-08-11 16:02:35 -07:00
..
CMakeLists.txt [CSSPGO] Migrate and refactor the decoder of Pseudo Probe 2021-08-04 09:21:34 -07:00
CSPreInliner.cpp [CSSPGO] Fix an invalid hash table reference issue in the CS preinliner. 2021-06-18 11:54:23 -07:00
CSPreInliner.h [CSSPGO][llvm-profgen] Context-sensitive global pre-inliner 2021-03-29 09:46:14 -07:00
CallContext.h [CSSPGO][llvm-profgen] Instruction symbolization 2020-11-20 14:26:27 -08:00
ErrorHandling.h [llvm] Replace LLVM_ATTRIBUTE_NORETURN with C++11 [[noreturn]] 2021-07-28 09:31:14 -07:00
PerfReader.cpp [CSSPGO][llvm-profgen] Trim and merge context beforehand to reduce memory usage 2021-08-11 16:02:35 -07:00
PerfReader.h [CSSPGO] Migrate and refactor the decoder of Pseudo Probe 2021-08-04 09:21:34 -07:00
ProfileGenerator.cpp [CSSPGO][llvm-profgen] Trim and merge context beforehand to reduce memory usage 2021-08-11 16:02:35 -07:00
ProfileGenerator.h [CSSPGO][llvm-profgen] Trim and merge context beforehand to reduce memory usage 2021-08-11 16:02:35 -07:00
ProfiledBinary.cpp [CSSPGO][llvm-profgen] Trim and merge context beforehand to reduce memory usage 2021-08-11 16:02:35 -07:00
ProfiledBinary.h [CSSPGO] Migrate and refactor the decoder of Pseudo Probe 2021-08-04 09:21:34 -07:00
llvm-profgen.cpp [llvm-profgen] Refactor PerfReader to allow different types of perf scripts 2021-08-02 17:18:47 -07:00