llvm-project/lld/COFF
Reid Kleckner 6529d7c5a4 [PDB] Defer relocating .debug$S until commit time and parallelize it
This is a pretty classic optimization. Instead of processing symbol
records and copying them to temporary storage, do a first pass to
measure how large the module symbol stream will be, and then copy the
data into place in the PDB file. This requires defering relocation until
much later, which accounts for most of the complexity in this patch.

This patch avoids copying the contents of all live .debug$S sections
into heap memory, which is worth about 20% of private memory usage when
making PDBs. However, this is not an unmitigated performance win,
because it can be faster to read dense, temporary, heap data than it is
to iterate symbol records in object file backed memory a second time.

Results on release chrome.dll:
peak mem: 5164.89MB -> 4072.19MB (-1,092.7MB, -21.2%)
wall-j1:  0m30.844s -> 0m32.094s (slightly slower)
wall-j3:  0m20.968s -> 0m20.312s (slightly faster)
wall-j8:  0m19.062s -> 0m17.672s (meaningfully faster)

I gathered similar numbers for a debug, component build of content.dll
in Chrome, and the performance impact of this change was in the noise.
The memory usage reduction was visible and similar.

Because of the new parallelism in the PDB commit phase, more cores makes
the new approach faster. I'm assuming that most C++ developer machines
these days are at least quad core, so I think this is a win.

Differential Revision: https://reviews.llvm.org/D94267
2021-01-12 17:46:29 -08:00
..
CMakeLists.txt [COFF] Port CallGraphSort to COFF from ELF 2020-07-30 15:21:44 -07:00
CallGraphSort.cpp [COFF] Port CallGraphSort to COFF from ELF 2020-07-30 15:21:44 -07:00
CallGraphSort.h [COFF] Port CallGraphSort to COFF from ELF 2020-07-30 15:21:44 -07:00
Chunks.cpp [PDB] Defer relocating .debug$S until commit time and parallelize it 2021-01-12 17:46:29 -08:00
Chunks.h [PDB] Defer relocating .debug$S until commit time and parallelize it 2021-01-12 17:46:29 -08:00
Config.h [lld][COFF] Add command line options for LTO with new pass manager 2020-11-05 14:41:35 -05:00
DLL.cpp [CFGuard] Add address-taken IAT tables and delay-load support 2020-11-17 18:24:45 -08:00
DLL.h [COFF] Share the tail in delayimport symbol thunks 2019-07-11 21:19:11 +00:00
DebugTypes.cpp [LLD][COFF] Simplify function. NFC. 2021-01-07 22:37:11 -05:00
DebugTypes.h [LLD][COFF] Simplify function. NFC. 2021-01-07 22:37:11 -05:00
Driver.cpp [LLD] Rename lld port driver entry function to a consistent name 2020-12-18 12:18:37 +05:30
Driver.h [LLD] Rename lld port driver entry function to a consistent name 2020-12-18 12:18:37 +05:30
DriverUtils.cpp Avoid a possible one-byte OOB read off of .drectve sections 2020-12-09 13:32:28 -08:00
ICF.cpp [CFGuard] Add address-taken IAT tables and delay-load support 2020-11-17 18:24:45 -08:00
ICF.h [Coding style change][lld] Rename variables for non-ELF ports 2019-07-11 05:40:30 +00:00
InputFiles.cpp [CFGuard] Add address-taken IAT tables and delay-load support 2020-11-17 18:24:45 -08:00
InputFiles.h [gn build] (manually) port b534beabee 2020-11-25 20:19:46 -05:00
LLDMapFile.cpp [Support] Move LLD's parallel algorithm wrappers to support 2020-05-05 15:21:05 -07:00
LLDMapFile.h [lld-link] Support /map option, matching link.exe 's /map output format 2020-03-24 09:48:00 -04:00
LTO.cpp [lld][COFF] Add command line options for LTO with new pass manager 2020-11-05 14:41:35 -05:00
LTO.h [COFF] Assign unique identifiers to ObjFiles from LTO 2020-04-17 17:15:12 -07:00
MapFile.cpp [Support] Move LLD's parallel algorithm wrappers to support 2020-05-05 15:21:05 -07:00
MapFile.h [Coding style change][lld] Rename variables for non-ELF ports 2019-07-11 05:40:30 +00:00
MarkLive.cpp [COFF] Don't treat DWARF sections as GC roots 2020-03-27 12:37:43 -07:00
MarkLive.h [Coding style change][lld] Rename variables for non-ELF ports 2019-07-11 05:40:30 +00:00
MinGW.cpp [LLD] [COFF] Allow wrapping dllimported functions 2020-11-24 10:15:20 +02:00
MinGW.h Reapply [LLD] [COFF] Implement a GNU/ELF like -wrap option 2020-10-15 22:14:02 +03:00
Options.td [lld/mac] Add --version flag 2020-12-22 22:06:39 -05:00
PDB.cpp [PDB] Defer relocating .debug$S until commit time and parallelize it 2021-01-12 17:46:29 -08:00
PDB.h Re-land "[PDB] Merge types in parallel when using ghashing" 2020-09-30 15:44:38 -07:00
README.md
SymbolTable.cpp Reapply [LLD] [COFF] Implement a GNU/ELF like -wrap option 2020-10-15 22:14:02 +03:00
SymbolTable.h [COFF] Assign unique identifiers to ObjFiles from LTO 2020-04-17 17:15:12 -07:00
Symbols.cpp [COFF] Migrate COFFObjectFile to Expected<T> 2020-05-08 14:01:39 -07:00
Symbols.h [CFGuard] Add address-taken IAT tables and delay-load support 2020-11-17 18:24:45 -08:00
TypeMerger.h [PDB] Use one func id DenseMap instead of per-source maps, NFC 2020-10-01 12:22:27 -07:00
Writer.cpp [CFGuard] Add address-taken IAT tables and delay-load support 2020-11-17 18:24:45 -08:00
Writer.h [LLD][COFF] Cover usage of LLD-as-a-library in tests 2020-09-24 15:07:50 -04:00

README.md

See docs/NewLLD.rst