maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

Core: Make the resolver faster. 

In the resolver, we maintain a list of undefined symbols, and when we
visit an archive file, we check that file if undefined symbols can be
resolved using files in the archive. The archive file class provides
find() function to lookup a symbol.

Previously, we call find() for each undefined symbols. Archive files
may be visited multiple times if they are in a --start-group and
--end-group. If we visit a file M times and if we have N undefined
symbols, find() is called M*N times. I found that that is one of the
most significant bottlenecks in LLD when linking a large executable.

find() is not a very cheap operation because it looks up a hash table
for a given string. And a string, or a symbol name, can be pretty long
if you are dealing with C++ symbols.

We can eliminate the bottleneck.

Calling find() with the same symbol multiple times is a waste. If a
result of looking up a symbol is "not found", it stays "not found"
forever because the symbol simply doesn't exist in the archive.
Thus, we should call find() only for newly-added undefined symbols.
This optimization makes O(M*N) O(N).

In this patch, all undefined symbols are added to a vector. For each
archive/shared library file, we maintain a start position P. All
symbols [0, P) are already searched. [P, end of the vector) are not
searched yet. For each file, we scan the vector only once.

This patch changes the order in which undefined symbols are looked for.
Previously, we iterated over the result of _symbolTable.undefines().
Now we iterate over the new vector. This is a benign change but caused
differences in output if remaining undefines exist. This is why some
tests are updated.

The performance improvement of this patch seems sometimes significant.
Previously, linking chrome.dll on my workstation (Xeon 2.4GHz 8 cores)
took about 70 seconds. Now it takes (only?) 30 seconds!

http://reviews.llvm.org/D8091

llvm-svn: 231434
This commit is contained in:
Rui Ueyama 2015-03-06 00:28:41 +00:00
parent c99a38796c
commit 13003d7774
11 changed files with 120 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
lld/include/lld/Core/Resolver.h
View File

@ -73,7 +73,7 @@ private:
bool checkUndefines();
void removeCoalescedAwayAtoms();
void checkDylibSymbolCollisions();
void forEachUndefines(bool searchForOverrides, UndefCallback callback);
void forEachUndefines(File &file, bool searchForOverrides, UndefCallback callback);
void markLive(const Atom *atom);
void addAtoms(const std::vector<const DefinedAtom *>&);
@ -102,6 +102,16 @@ private:
// Preloading
std::map<StringRef, ArchiveLibraryFile *> _archiveMap;
std::unordered_set<ArchiveLibraryFile *> _archiveSeen;
// List of undefined symbols.
std::vector<StringRef> _undefines;
// Start position in _undefines for each archive/shared library file.
// Symbols from index 0 to the start position are already searched before.
// Searching them again would never succeed. When we look for undefined
// symbols from an archive/shared library file, start from its start
// position to save time.
std::map<File *, size_t> _undefineIndex;
};
} // namespace lld

54
lld/lib/Core/Resolver.cpp
View File

@ -46,33 +46,35 @@ bool Resolver::handleFile(File &file) {
return undefAdded;
}
void Resolver::forEachUndefines(bool searchForOverrides,
void Resolver::forEachUndefines(File &file, bool searchForOverrides,
UndefCallback callback) {
// Handle normal archives
unsigned undefineGenCount = 0;
size_t i = _undefineIndex[&file];
do {
undefineGenCount = _symbolTable.size();
for (const UndefinedAtom *undefAtom : _symbolTable.undefines()) {
StringRef undefName = undefAtom->name();
// load for previous undefine may also have loaded this undefine
if (!_symbolTable.isDefined(undefName))
callback(undefName, false);
}
// search libraries for overrides of common symbols
if (searchForOverrides) {
for (StringRef tentDefName : _symbolTable.tentativeDefinitions()) {
// Load for previous tentative may also have loaded
// something that overrode this tentative, so always check.
const Atom *curAtom = _symbolTable.findByName(tentDefName);
assert(curAtom != nullptr);
if (const DefinedAtom *curDefAtom = dyn_cast<DefinedAtom>(curAtom)) {
if (curDefAtom->merge() == DefinedAtom::mergeAsTentative)
callback(tentDefName, true);
}
for (; i < _undefines.size(); ++i) {
StringRef undefName = _undefines[i];
if (undefName.empty())
continue;
if (_symbolTable.isDefined(undefName) ||
_symbolTable.isCoalescedAway(_symbolTable.findByName(undefName))) {
// The symbol was resolved by some other file. Cache the result.
_undefines[i] = "";
continue;
}
callback(undefName, false);
}
} while (undefineGenCount != _symbolTable.size());
if (!searchForOverrides)
continue;
for (StringRef tentDefName : _symbolTable.tentativeDefinitions()) {
// Load for previous tentative may also have loaded
// something that overrode this tentative, so always check.
const Atom *curAtom = _symbolTable.findByName(tentDefName);
assert(curAtom != nullptr);
if (const DefinedAtom *curDefAtom = dyn_cast<DefinedAtom>(curAtom))
if (curDefAtom->merge() == DefinedAtom::mergeAsTentative)
callback(tentDefName, true);
}
} while (i < _undefines.size());
_undefineIndex[&file] = i;
}
bool Resolver::handleArchiveFile(File &file) {
@ -80,7 +82,7 @@ bool Resolver::handleArchiveFile(File &file) {
bool searchForOverrides =
_ctx.searchArchivesToOverrideTentativeDefinitions();
bool undefAdded = false;
forEachUndefines(searchForOverrides,
forEachUndefines(file, searchForOverrides,
[&](StringRef undefName, bool dataSymbolOnly) {
if (File *member = archiveFile->find(undefName, dataSymbolOnly)) {
member->setOrdinal(_ctx.getNextOrdinalAndIncrement());
@ -98,7 +100,7 @@ void Resolver::handleSharedLibrary(File &file) {
handleFile(*sharedLibrary);
bool searchForOverrides =
_ctx.searchSharedLibrariesToOverrideTentativeDefinitions();
forEachUndefines(searchForOverrides,
forEachUndefines(file, searchForOverrides,
[&](StringRef undefName, bool dataSymbolOnly) {
if (const SharedLibraryAtom *atom =
sharedLibrary->exports(undefName, dataSymbolOnly))
@ -117,6 +119,8 @@ bool Resolver::doUndefinedAtom(const UndefinedAtom &atom) {
// tell symbol table
bool newUndefAdded = _symbolTable.add(atom);
if (newUndefAdded)
_undefines.push_back(atom.name());
// If the undefined symbol has an alternative name, try to resolve the
// symbol with the name to give it a second chance. This feature is used

20
lld/test/elf/Mips/rel-dynamic-01-micro.test
View File

@ -68,16 +68,7 @@
# PLT-SYM-NEXT: Section: .bss (0xD)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T3@ (4)
# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
# PLT-SYM-NEXT: Other: 0
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T1@ (7)
# PLT-SYM-NEXT: Name: T1@ (4)
# PLT-SYM-NEXT: Value: 0x4001D5
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
@ -85,6 +76,15 @@
# PLT-SYM-NEXT: Other: 8
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T3@ (7)
# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
# PLT-SYM-NEXT: Other: 0
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: ]
# so.o

22
lld/test/elf/Mips/rel-dynamic-01.test
View File

@ -69,16 +69,7 @@
# PLT-SYM-NEXT: Section: .bss (0xD)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T3@ (4)
# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
# PLT-SYM-NEXT: Other: 0
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T1@ (7)
# PLT-SYM-NEXT: Name: T1@ (4)
# PLT-SYM-NEXT: Value: 0x400220
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
@ -87,7 +78,16 @@
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T2@ (10)
# PLT-SYM-NEXT: Name: T3@ (10)
# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
# PLT-SYM-NEXT: Other: 0
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
# PLT-SYM-NEXT: Name: T2@ (7)
# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)

22
lld/test/elf/Mips/rel-dynamic-04-micro.test
View File

@ -88,16 +88,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T3@ (1)
# PLT-NEXT: Value: 0x4010E5
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T2@ (7)
# PLT-NEXT: Name: T2@ (4)
# PLT-NEXT: Value: 0x4010D9
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
@ -106,7 +97,16 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T1@ (4)
# PLT-NEXT: Name: T3@ (7)
# PLT-NEXT: Value: 0x4010E5
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T1@ (1)
# PLT-NEXT: Value: 0x0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)

22
lld/test/elf/Mips/rel-dynamic-04.test
View File

@ -86,16 +86,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T3@ (1)
# PLT-NEXT: Value: 0x4010F0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T2@ (7)
# PLT-NEXT: Name: T2@ (4)
# PLT-NEXT: Value: 0x4010E0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
@ -104,7 +95,16 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T1@ (4)
# PLT-NEXT: Name: T3@ (7)
# PLT-NEXT: Value: 0x4010F0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T1@ (1)
# PLT-NEXT: Value: 0x0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)

22
lld/test/elf/Mips/rel-dynamic-05-micro.test
View File

@ -70,16 +70,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T3@ (1)
# PLT-NEXT: Value: 0x4001E1
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T1@ (4)
# PLT-NEXT: Name: T1@ (1)
# PLT-NEXT: Value: 0x4001C9
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
@ -88,7 +79,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T2@ (7)
# PLT-NEXT: Name: T2@ (4)
# PLT-NEXT: Value: 0x4001D5
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
@ -96,6 +87,15 @@
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T3@ (7)
# PLT-NEXT: Value: 0x4001E1
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: ]
# so.o

22
lld/test/elf/Mips/rel-dynamic-05.test
View File

@ -68,16 +68,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T3@ (1)
# PLT-NEXT: Value: 0x4001F0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T1@ (4)
# PLT-NEXT: Name: T1@ (1)
# PLT-NEXT: Value: 0x4001D0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
@ -86,7 +77,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T2@ (7)
# PLT-NEXT: Name: T2@ (4)
# PLT-NEXT: Value: 0x4001E0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
@ -94,6 +85,15 @@
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
# PLT-NEXT: Name: T3@ (7)
# PLT-NEXT: Value: 0x4001F0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
# PLT-NEXT: Other: 8
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: ]
# so.o

14
lld/test/elf/dynamic.test
View File

@ -46,19 +46,19 @@ CHECK-NEXT: }
CHECK: DynamicSymbols [
CHECK: Symbol {
CHECK: Name: i
CHECK-NEXT: Value: 0
CHECK-NEXT: Size:
CHECK-NEXT: Binding: Global
CHECK-NEXT: Type: Object
CHECK: }
CHECK: Symbol {
CHECK: Name: foo
CHECK-NEXT: Value: 0
CHECK-NEXT: Size:
CHECK-NEXT: Binding: Global
CHECK-NEXT: Type: Function
CHECK: }
CHECK: Symbol {
CHECK: Name: i
CHECK-NEXT: Value: 0
CHECK-NEXT: Size:
CHECK-NEXT: Binding: Global
CHECK-NEXT: Type: Object
CHECK: }
CHECK: DynamicSection [ (15 entries)
CHECK: Tag Type Name/Value

8
lld/test/mach-o/use-simple-dylib.yaml
View File

@ -119,13 +119,13 @@ install-name: libspecial.dylib
# CHECK: - name: _myStatic
# CHECK: - name: _myVariablePreviouslyKnownAsPrivateExtern
# CHECK: shared-library-atoms:
# CHECK: - name: _myHidden
# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myGlobal
# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myHiddenWeak
# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myGlobalWeak
# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myHidden
# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myHiddenWeak
# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myResolver
# CHECK: load-name: libspecial.dylib

8
lld/test/pecoff/hello64.test
View File

@ -11,12 +11,12 @@ CHECK: 6004: 48 c7 c1 00 00 00 00 movq $0, %rcx
CHECK: 600b: 48 8d 15 f4 af ff ff leaq -20492(%rip), %rdx
CHECK: 6012: 4c 8d 05 e7 af ff ff leaq -20505(%rip), %r8
CHECK: 6019: 41 b9 00 00 00 00 movl $0, %r9d
CHECK: 601f: e8 0a 00 00 00 callq 10
CHECK: 601f: e8 12 00 00 00 callq 18
CHECK: 6024: b9 00 00 00 00 movl $0, %ecx
CHECK: 6029: e8 08 00 00 00 callq 8
CHECK: 602e: ff 25 d4 cf ff ff jmpq *-12332(%rip)
CHECK: 6029: e8 00 00 00 00 callq 0
CHECK: 602e: ff 25 cc cf ff ff jmpq *-12340(%rip)
CHECK: 6034: cc int3
CHECK: 6035: cc int3
CHECK: 6036: ff 25 c4 cf ff ff jmpq *-12348(%rip)
CHECK: 6036: ff 25 cc cf ff ff jmpq *-12340(%rip)
CHECK: 603c: cc int3
CHECK: 603d: cc int3