Commit Graph

628 Commits

Author SHA1 Message Date
Fangrui Song cfca5095df [ELF] Error on relocations to STT_SECTION symbols if the sections were discarded
This is implemented by creating Undefined (instead of Defined) for such
local STT_SECTION symbols. It allows us to catch errors when there are
relocations to such discarded sections (e.g. in PR41693, ld.bfd and gold
error but we don't). Updated comdat-discarded-error.s checks we emit
friendly error message.

For relocatable-eh-frame.s, ld.lld -r a.o a.o will now error
"STT_SECTION symbol should be defined" because the section .eh_frame
refers to is now an Undefined instead of a Defined.
So I have to change `error()` to `warn()` to retain the output.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D61583

llvm-svn: 361792
2019-05-28 06:34:52 +00:00
Rui Ueyama 92069605bf Merge ELFFileBase::{initSymtab,parseHeader} as ELFFileBase:init. NFC.
This patch simplifies ELFFile instance initialization by merging
two similar functions into a single function and call it from the
ctor.

llvm-svn: 361789
2019-05-28 05:17:21 +00:00
Rui Ueyama 76737f4d19 Remove elf::createSharedFile and move its code to SharedFile's ctor. NFC.
llvm-svn: 361747
2019-05-27 07:26:13 +00:00
Rui Ueyama f5d9d23905 Simplify InputFile::fetch().
We don't have to return a value from the function. Instead, we can
directly call parseFile from the functions.

llvm-svn: 361478
2019-05-23 10:15:12 +00:00
Rui Ueyama 821a1ac050 Remove LazyObjFile::AddedToLink.
Instead we can just clear a MemoryBuffer so that we cannot get the
same buffer more than once.

llvm-svn: 361477
2019-05-23 10:08:56 +00:00
Rui Ueyama 7f7d2b2e62 Move code for symbol resolution from SymbolTable.cpp to Symbols.cpp.
My recent commits separated symbol resolution from the symbol table,
so the functions to resolve symbols are now in a somewhat wrong file.
This patch moves it to Symbols.cpp.

The functions are now member functions of the symbol.

This is code move change. I modified function names so that they are
appropriate as member functions, though. No functionality change
intended.

Differential Revision: https://reviews.llvm.org/D62290

llvm-svn: 361474
2019-05-23 09:58:08 +00:00
Rui Ueyama 4254840313 Speed up --start-lib and --end-lib.
--{start,end}-lib give files grouped by the options the archive file
semantics. That is, each object file between them acts as if it were
in an archive file whose sole member is the file.

Therefore, files between --{start,end}-lib are linked to the final
output only if they are needed to resolve some undefined symbols.

Previously, the feature was implemented this way:

 1. We read a symbol table and insert defined symbols to the symbol
    table as lazy symbols.

 2. If an undefind symbol is resolved to a lazy symbol, that lazy
    symbol instantiate ObjFile class for that symbol, which re-insert
    all defined symbols to the symbol table.

So, if an ObjFile is instantiated, defined symbols are inserted to the
symbol table twice. Since inserting long symbol names is not cheap,
there's a room to optimize here.

This patch optimzies it. Now, LazyObjFile remembers symbol handles and
passed them over to a new ObjFile instance, so that the ObjFile
doesn't insert the same strings.

Here is a quick benchmark to link clang. "Original" is the original
lld with unmodified command line options. For "Case 1" and "Case 2", I
extracted all files from archive files and replace .a's in a command
line with .o's wrapped with --{start,end}-lib. I used the original lld
for Case 1" and use this patch for Case 2.

  Original: 5.892
    Case 1: 6.001 (+1.8%)
    Case 2: 5.701 (-3.2%)

So, interestingly, --{start,end}-lib are now faster than the regular
linking scheme with archive files. That's perhaps not too surprising,
though, because for regular archive files, we look up the symbol table
with the same string twice.

Differential Revision: https://reviews.llvm.org/D62188

llvm-svn: 361473
2019-05-23 09:53:30 +00:00
Fangrui Song b72b091389 [ELF] Improve error message for relocations to symbols defined in discarded sections
Rather than report "undefined symbol: ", give more informative message
about the object file that defines the discarded section.

In particular, PR41133, if the section is a discarded COMDAT, print the
section group signature and the object file with the prevailing
definition. This is useful to track down some ODR issues.

We need to
* add `uint32_t DiscardedSecIdx` to Undefined for this feature.
* make ComdatGroups public and change its type to DenseMap<CachedHashStringRef, const InputFile *>

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D59649

llvm-svn: 361359
2019-05-22 09:06:42 +00:00
Rui Ueyama 33e74d9f62 Simplify the logic to instantiate Symbols. Should be NFC.
llvm-svn: 361350
2019-05-22 04:56:25 +00:00
Ben Dunbobbin 1d16515fb4 [ELF] Implement Dependent Libraries Feature
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.

Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.

The design goals were to provide:

- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
  environments (MSVC in particular).

Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.

In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:

1. There are no competing defined symbols in a given set of libraries, or
   if they exist, the program owner doesn't care which is linked to their
   program.
2. There may be circular dependencies between libraries.

The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:

.section ".deplibs","MS",@llvm_dependent_libraries,1
         .asciz "foo"

For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.

LLD processes the dependent library specifiers in the following way:

1. Dependent libraries which are found from the specifiers in .deplibs sections
   of relocatable object files are added when the linker decides to include that
   file (which could itself be in a library) in the link. Dependent libraries
   behave as if they were appended to the command line after all other options. As
   a consequence the set of dependent libraries are searched last to resolve
   symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
   to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
   strings in a .deplibs section by; first, handling the string as if it was
   specified on the command line; second, by looking for the string in each of the
   library search paths in turn; third, by looking for a lib<string>.a or
   lib<string>.so (depending on the current mode of the linker) in each of the
   library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
   dependent libraries.

Rationale for the above points:

1. Adding the dependent libraries last makes the process simple to understand
   from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
   failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
   will affect all of the dependent libraries. There is a potential problem of
   surprise for developers, who might not realize that these options would apply
   to these "invisible" input files; however, despite the potential for surprise,
   this is easy for developers to reason about and gives developers the control
   that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
   find input files. The different search methods are tried by the linker in most
   obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
   ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
   is not necessary: if finer control is required developers can fall back to using
   the command line directly.

RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.

Differential Revision: https://reviews.llvm.org/D60274

llvm-svn: 360984
2019-05-17 03:44:15 +00:00
Rui Ueyama bbf154cf9c Move symbol resolution code out of SymbolTable class.
This is the last patch of the series of patches to make it possible to
resolve symbols without asking SymbolTable to do so.

The main point of this patch is the introduction of
`elf::resolveSymbol(Symbol *Old, Symbol *New)`. That function resolves
or merges given symbols by examining symbol types and call
replaceSymbol (which memcpy's New to Old) if necessary.

With the new function, we have now separated symbol resolution from
symbol lookup. If you already have a Symbol pointer, you can directly
resolve the symbol without asking SymbolTable to do that.

Now that the nice abstraction become available, I can start working on
performance improvement of the linker. As a starter, I'm thinking of
making --{start,end}-lib faster.

--{start,end}-lib is currently unnecessarily slow because it looks up
the symbol table twice for each symbol.

 - The first hash table lookup/insertion occurs when we instantiate a
   LazyObject file to insert LazyObject symbols.

 - The second hash table lookup/insertion occurs when we create an
   ObjFile from LazyObject file. That overwrites LazyObject symbols
   with Defined symbols.

I think it is not too hard to see how we can now eliminate the second
hash table lookup. We can keep LazyObject symbols in Step 1, and then
call elf::resolveSymbol() to do Step 2.

Differential Revision: https://reviews.llvm.org/D61898

llvm-svn: 360975
2019-05-17 01:55:20 +00:00
Igor Kudrin 4669cf2750 [LTO] Improve readability of module IDs
Module IDs can appear in diagnostic messages.
This patch adds some auxiliary symbols to improve their readability.

Differential Revision: https://reviews.llvm.org/D61857

llvm-svn: 360858
2019-05-16 05:23:25 +00:00
Rui Ueyama 54ee6df247 Pemove SymbolTable::addBitcode as it is redundant.
Differential Revision: https://reviews.llvm.org/D61897

llvm-svn: 360846
2019-05-16 03:54:50 +00:00
Rui Ueyama 943cd00580 De-template parseFile() and SymbolTable's add-family functions.
Differential Revision: https://reviews.llvm.org/D61896

llvm-svn: 360844
2019-05-16 03:45:13 +00:00
Rui Ueyama 5c073a94f9 Introduce CommonSymbol.
Previously, we handled common symbols as a kind of Defined symbol,
but what we were doing for common symbols is pretty different from
regular defined symbols.

Common symbol and defined symbol are probably as different as shared
symbol and defined symbols are different.

This patch introduces CommonSymbol to represent common symbols.
After symbols are resolved, they are converted to Defined symbols
residing in a .bss section.

Differential Revision: https://reviews.llvm.org/D61895

llvm-svn: 360841
2019-05-16 03:29:03 +00:00
Rui Ueyama 7d4761928e Simplify SymbolTable::add{Defined,Undefined,...} functions.
SymbolTable's add-family functions have lots of parameters because
when they have to create a new symbol, they forward given arguments
to Symbol's constructors. Therefore, the functions take at least as
many arguments as their corresponding constructors.

This patch simplifies the add-family functions. Now, the functions
take a symbol instead of arguments to construct a symbol. If there's
no existing symbol, a given symbol is memcpy'ed to the symbol table.
Otherwise, the functions attempt to merge the existing and a given
new symbol.

I also eliminated `CanOmitFromDynSym` parameter, so that the functions
take really one argument.

Symbol classes are trivially constructible, so looks like constructing
them to pass to add-family functions is as cheap as passing a lot of
arguments to the functions. A quick benchmark showed that this patch
seems performance-neutral.

This is a preparation for
http://lists.llvm.org/pipermail/llvm-dev/2019-April/131902.html

Differential Revision: https://reviews.llvm.org/D61855

llvm-svn: 360838
2019-05-16 02:14:00 +00:00
Rui Ueyama 2dd5283d2a Move SymbolTable::addFile to InputFiles.cpp.
The symbol table used to be a container of vectors of input files,
but that's no longer the case because the vectors are moved out of
SymbolTable and are now global variables.

Therefore, addFile doesn't have to belong to any class. This patch
moves the function out of the class.

This patch is a preparation for my RFC [1].

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-April/131902.html

Differential Revision: https://reviews.llvm.org/D61854

llvm-svn: 360666
2019-05-14 12:03:13 +00:00
Rui Ueyama 0c01607bbf Rename a variable and add a comment.
llvm-svn: 358049
2019-04-10 06:32:05 +00:00
Rui Ueyama f432fa6eee De-template SymbolTable::addShared.
Because of r357925, this member function doesn't have to be a
template of ELFT.

llvm-svn: 357982
2019-04-09 08:52:00 +00:00
Peter Collingbourne d3e207057f ELF: Move verneed tracking data structures out of VersionNeedSection.
For partitions I intend to use the same set of version indexes in
each partition for simplicity. Since each partition will need its own
VersionNeedSection this will require moving the verneed tracking out of
VersionNeedSection. The way I've done this is to move most of the tracking
into SharedFile. What will eventually become the per-partition tracking
still lives in VersionNeedSection.

As a bonus the code gets a little simpler and more consistent with how we
handle verdef.

Differential Revision: https://reviews.llvm.org/D60307

llvm-svn: 357926
2019-04-08 17:48:05 +00:00
Peter Collingbourne cc1618e668 ELF: De-template SharedFile. NFCI.
Differential Revision: https://reviews.llvm.org/D60305

llvm-svn: 357925
2019-04-08 17:35:55 +00:00
Peter Collingbourne 883ab235ee ELF: De-template ELFFileBase. NFCI.
Differential Revision: https://reviews.llvm.org/D60304

llvm-svn: 357806
2019-04-05 20:16:26 +00:00
Peter Collingbourne ad4376e8af ELF: Simplify. NFCI.
Differential Revision: https://reviews.llvm.org/D60299

llvm-svn: 357739
2019-04-05 01:31:40 +00:00
Peter Collingbourne 8238604259 ELF: Move SymtabSHNDX and getSectionIndex() to ObjFile. NFCI.
Differential Revision: https://reviews.llvm.org/D60244

llvm-svn: 357670
2019-04-04 03:13:51 +00:00
Eli Friedman 3751ae4a94 [ELF] Print a better error for an archive containing a non-ELF file.
Hopefully gives a more readable error message for the most obvious
mistake.

Differential Revision: https://reviews.llvm.org/D59170

llvm-svn: 355888
2019-03-12 01:24:39 +00:00
Rui Ueyama 033c4d2126 Include an archive file name in an error message for a corrupted file.
Differential Revision: https://reviews.llvm.org/D59212

llvm-svn: 355886
2019-03-12 00:24:34 +00:00
Alexey Lapshin 77fc1f6049 [DebugInfo] add SectionedAddress to DebugInfo interfaces.
That patch is the fix for https://bugs.llvm.org/show_bug.cgi?id=40703
   "wrong line number info for obj file compiled with -ffunction-sections"
   bug. The problem happened with only .o files. If object file contains
   several .text sections then line number information showed incorrectly.
   The reason for this is that DwarfLineTable could not detect section which
   corresponds to specified address(because address is the local to the
   section). And as the result it could not select proper sequence in the
   line table. The fix is to pass SectionIndex with the address. So that it
   would be possible to differentiate addresses from various sections. With
   this fix llvm-objdump shows correct line numbers for disassembled code.

   Differential review: https://reviews.llvm.org/D58194

llvm-svn: 354972
2019-02-27 13:17:36 +00:00
Konstantin Zhuravlyov 87498153aa LLD/AMDGPU: Preserve ABI version during linking ELF for AMDGPU
Differential Revision: https://reviews.llvm.org/D58026

llvm-svn: 354086
2019-02-14 23:59:44 +00:00
Fangrui Song b4744d306c [ELF] Support --{,no-}allow-shlib-undefined
Summary:
In ld.bfd/gold, --no-allow-shlib-undefined is the default when linking
an executable. This patch implements a check to error on undefined
symbols in a shared object, if all of its DT_NEEDED entries are seen.

Our approach resembles the one used in gold, achieves a good balance to
be useful but not too smart (ld.bfd traces all DSOs and emulates the
behavior of a dynamic linker to catch more cases).

The error is issued based on the symbol table, different from undefined
reference errors issued for relocations. It is most effective when there
are DSOs that were not linked with -z defs (e.g. when static sanitizers
runtime is used).

gold has a comment that some system libraries on GNU/Linux may have
spurious undefined references and thus system libraries should be
excluded (https://sourceware.org/bugzilla/show_bug.cgi?id=6811). The
story may have changed now but we make --allow-shlib-undefined the
default for now. Its interaction with -shared can be discussed in the
future.

Reviewers: ruiu, grimar, pcc, espindola

Reviewed By: ruiu

Subscribers: joerg, emaste, arichardson, llvm-commits

Differential Revision: https://reviews.llvm.org/D57385

llvm-svn: 352826
2019-02-01 02:25:05 +00:00
Nick Desaulniers 6cff0cb35a lld: elf: discard more specific .gnu.linkonce section
Summary:
lld discards .gnu.linonce.* sections work around a bug in glibc.
https://sourceware.org/bugzilla/show_bug.cgi?id=20543

Unfortunately, the Linux kernel uses a section named
.gnu.linkonce.this_module to store infomation about kernel modules. The
kernel reads data from this section when loading kernel modules, and
errors if it fails to find this section. The current behavior of lld
discards this section when kernel modules are linked, so kernel modules
linked with lld are unloadable by the linux kernel.

The Linux kernel should use a comdat section instead of .gnu.linkonce.
The minimum version of binutils supported by the kernel supports comdat
sections. The kernel is also not relying on the old linkonce behavior;
it seems to have chosen a name that contains a deprecated GNU feature.

Changing the section name now in the kernel would require all kernel
modules to be recompiled to make use of the new section name. Instead,
rather than discarding .gnu.linkonce.*, let's discard the more specific
section name to continue working around the glibc issue while supporting
linking Linux kernel modules.

Link: https://github.com/ClangBuiltLinux/linux/issues/329

Reviewers: pcc, espindola

Reviewed By: pcc

Subscribers: nathanchance, emaste, arichardson, void, srhines

Differential Revision: https://reviews.llvm.org/D57294

llvm-svn: 352302
2019-01-27 02:54:23 +00:00
Rui Ueyama e14e46b3f1 Simplify. NFC.
llvm-svn: 352242
2019-01-25 21:25:25 +00:00
Rui Ueyama ec33fd6dd5 Untabify.
llvm-svn: 352070
2019-01-24 18:17:17 +00:00
Serge Guelton 1fa239f500 Partial support of SHT_GROUP without flag
This does *not* implement full SHT_GROUP semantic, yet it is a simple step forward:
Sections within a group are still considered valid, but they do not behave as
specified by the standard in case of garbage collection.

Differential Revision: https://reviews.llvm.org/D56437

llvm-svn: 352068
2019-01-24 17:56:08 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
George Rimar 73af3d4060 [LLD][ELF] - Support MSP430.
Patch by Michael Skvortsov!

This change adds a basic support for linking static MSP430 ELF code.
Implemented relocation types are intended to correspond to the BFD.

Differential revision: https://reviews.llvm.org/D56535

llvm-svn: 350819
2019-01-10 13:43:06 +00:00
Rui Ueyama 2d7d128117 Use unique_ptr to manage a TarWriter instance. NFC.
llvm-svn: 349581
2018-12-18 23:50:37 +00:00
George Rimar 3608decaa5 [ELF] - Do not crash when -r output uses linker script with `/DISCARD/`
This is https://bugs.llvm.org/show_bug.cgi?id=39493.

We crashed previously because did not handle /DISCARD/ properly
when -r was used. I think it is uncommon to use scripts with -r, though I see
nothing wrong to handle the /DISCARD/ so that we will not crash at least.

Differential revision: https://reviews.llvm.org/D53864

llvm-svn: 345819
2018-11-01 09:20:06 +00:00
Sam Clegg ea65647254 Use llvm::arrayRefFromStringRef
Differential Revision: https://reviews.llvm.org/D53432

llvm-svn: 344888
2018-10-22 08:35:39 +00:00
Rui Ueyama 2600e0946b Rename SymbolTable::addRegular -> SymbolTable::addDefined.
We have addAbsolute, addBitcode, addCommon, etc. addRegular looked a
bit inconsistent.

llvm-svn: 344294
2018-10-11 20:43:01 +00:00
Rui Ueyama e28c146423 Avoid unnecessary buffer allocation and memcpy for compressed sections.
Previously, we uncompress all compressed sections before doing anything.
That works, and that is conceptually simple, but that could results in
a waste of CPU time and memory if uncompressed sections are then
discarded or just copied to the output buffer.

In particular, if .debug_gnu_pub{names,types} are compressed and if no
-gdb-index option is given, we wasted CPU and memory because we
uncompress them into newly allocated bufers and then memcpy the buffers
to the output buffer. That temporary buffer was redundant.

This patch changes how to uncompress sections. Now, compressed sections
are uncompressed lazily. To do that, `Data` member of `InputSectionBase`
is now hidden from outside, and `data()` accessor automatically expands
an compressed buffer if necessary.

If no one calls `data()`, then `writeTo()` directly uncompresses
compressed data into the output buffer. That eliminates the redundant
memory allocation and redundant memcpy.

This patch significantly reduces memory consumption (20 GiB max RSS to
15 Gib) for an executable whose .debug_gnu_pub{names,types} are in total
5 GiB in an uncompressed form.

Differential Revision: https://reviews.llvm.org/D52917

llvm-svn: 343979
2018-10-08 16:58:59 +00:00
Shoaib Meenai 6883d882f1 [ELF] Fix crash on invalid undefined local symbols
r320770 made LLD handle invalid DSOs where local symbols were found in
the global part of the symbol table. Unfortunately, it didn't handle the
case where those local symbols were also undefined, and r326242 exposed
an assertion failure in that case. Just warn on that case instead of
crashing, by moving the local binding check before the undefined symbol
addition.

The input file for the test is crafted by hand, since I don't know of
any tool that would produce such a broken DSO. I also don't understand
what it even means for a symbol to be undefined but have STB_LOCAL
binding - I don't think that combination makes any sense - but we have
found broken DSOs of this nature that we were linking against. I've
included detailed instructions on how to produce the DSO in the test.

Differential Revision: https://reviews.llvm.org/D52815

llvm-svn: 343745
2018-10-03 23:53:11 +00:00
Michael J. Spencer 8222f902ca [ELF] Read the call graph profile from object files.
This uses the call graph profile embedded in the object files to construct the call graph.

This is read from a SHT_LLVM_CALL_GRAPH_PROFILE (0x6fff4c02) section as (uint32_t, uint32_t, uint64_t) tuples as (from symbol index, to symbol index, weight).

Differential Revision: https://reviews.llvm.org/D45850

llvm-svn: 343552
2018-10-02 00:17:15 +00:00
Rui Ueyama 4e247522ac Reset input section pointers to null on each linker invocation.
Previously, if you invoke lld's `main` more than once in the same process,
the second invocation could fail or produce a wrong result due to a stale
pointer values of the previous run.

Differential Revision: https://reviews.llvm.org/D52506

llvm-svn: 343009
2018-09-25 19:26:58 +00:00
Rui Ueyama ae34348354 Fix an error message. It must start with a lowercase letter.
llvm-svn: 342992
2018-09-25 17:12:50 +00:00
Sean Fertile 69e09116ba [PPC64] Handle ppc64le triple in getBitcodeMachineKind.
Enables lto and thinlto with bitcode targeting ppc64le.

Differential Revision: https://reviews.llvm.org/D52265

llvm-svn: 342604
2018-09-20 00:26:49 +00:00
Fangrui Song 1fe3e8b26f [ELF] Check if LinkSec is nullptr when initializing SHF_LINK_ORDER sections
Summary: This protects lld from a null pointer dereference when a faulty input file has such invalid sh_link fields.

Reviewers: ruiu, espindola

Reviewed By: ruiu

Subscribers: emaste, arichardson, llvm-commits

Differential Revision: https://reviews.llvm.org/D51743

llvm-svn: 341611
2018-09-07 00:18:07 +00:00
Matt Arsenault 8d6c7a63a1 Handle identifying AMDGPU bitcode files
llvm-svn: 340738
2018-08-27 12:40:00 +00:00
Fangrui Song 887ec75173 [ELF] -thinlto-object-suffix-replace=: don't error if the path does not end with old suffix
Summary:
For -thinlto-object-suffix-replace=old\;new, in
tools/gold/gold-plugin.cpp, the thinlto object filename is Path minus
optional old suffix.

    static std::string getThinLTOObjectFileName(StringRef Path, StringRef OldSuffix,
                                                StringRef NewSuffix) {
      if (OldSuffix.empty() && NewSuffix.empty())
        return Path;
      StringRef NewPath = Path;
      NewPath.consume_back(OldSuffix);
      std::string NewNewPath = NewPath;
      NewNewPath += NewSuffix;
      return NewNewPath;
    }

Currently lld will error that the path does not end with old suffix.

This patch makes lld accept such paths but only add new suffix if Path
ends with old suffix. This fixes a link error where bitcode members in
an archive are regular LTO objects without old suffix.

Acording to tejohnson, this will "enable supporting mix and match of
minimized ThinLTO bitcode files with normal ThinLTO bitcode files in a
single link (where we want to apply the suffix replacement to the
minimized files, and just ignore it for the normal ThinLTO files)."

Reviewers: ruiu, pcc, tejohnson, espindola

Reviewed By: tejohnson

Subscribers: emaste, inglorion, arichardson, eraman, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D51055

llvm-svn: 340364
2018-08-21 23:28:12 +00:00
George Rimar e2684662ee [LLD][ELF] - Check the architecture of lazy objects earlier.
Our code in LazyObjFile::parse() has an ELFT switch and
adds a lazy object by its ELFT kind.
Though it might be possible to add a file using a different
architecture and make LLD to silently accept it (if the file
is empty or contains only week symbols). That itself, not a
huge issue perhaps (because the error would be reported later
if the file is fetched), but still does not look clean and correct.

It is possible to report an error earlier and clean up the
code. That is what the patch does.

Ideally, we might want to reuse isCompatible from SymbolTable.cpp,
but it is static and accepts a file as an argument, what is not
convenient. Since such a situation should be rare, I think it
should be OK to go with the way chosen in this patch.

Differential revision: https://reviews.llvm.org/D50899

llvm-svn: 340257
2018-08-21 08:13:06 +00:00
George Rimar bf6132c139 [LLD][ELF] - Remove dead code. NFC.
These lines were unused.

llvm-svn: 340011
2018-08-17 11:19:55 +00:00
George Rimar 2835606889 [LLD][ELF] - Handle SHT_GROUP more carefully. NFCI.
This patch solves 2 problems:
1) It adds a test to check the line below:
https://github.com/llvm-mirror/lld/blob/master/ELF/InputFiles.cpp#L334
Test case contains SHT_GROUP section with a broken (0xFF) flag.

2) The patch fixes the case when we silently accepted such broken groups
in the case when there were no other objects with the same group signature.

llvm-svn: 339765
2018-08-15 12:20:38 +00:00
George Rimar 830ff2d0b1 [LLD][ELF] - Remove redundant code. NFC.
Code was never executed with our test cases,
though it is valid and I think we can always run it.
This improves code coverage.

llvm-svn: 338708
2018-08-02 12:20:36 +00:00
Reid Kleckner c07f9bb83a Update for DWARF API change
llvm-svn: 338642
2018-08-01 21:57:15 +00:00
Peter Smith 70997f9a4e [ELF][ARM] Implement support for Tag_ABI_VFP_args
The Tag_ABI_VFP_args build attribute controls the procedure call standard
used for floating point parameters on ARM. The values are:
0 - Base AAPCS (FP Parameters passed in Core (Integer) registers
1 - VFP AAPCS (FP Parameters passed in FP registers)
2 - Toolchain specific (Neither Base or VFP)
3 - Compatible with all (No use of floating point parameters)

If the Tag_ABI_VFP_args build attribute is missing it has an implicit value
of 0.
    
We use the attribute in two ways:
- Detect a clash in calling convention between Base, VFP and Toolchain.
we follow ld.bfd's lead and do not error if there is a clash between an
implicit Base AAPCS caused by a missing attribute. Many projects
including the hard-float (VFP AAPCS) version of glibc contain assembler
files that do not use floating point but do not have Tag_ABI_VFP_args.
- Set the EF_ARM_ABI_FLOAT_SOFT or EF_ARM_ABI_FLOAT_HARD ELF header flag
for Base or VFP AAPCS respectively. This flag is used by some ELF
loaders.
    
References:
- Addenda to, and Errata in, the ABI for the ARM Architecture for
Tag_ABI_VFP_args
- Elf for the ARM Architecture for ELF header flags
    
Fixes PR36009
    
Differential Revision: https://reviews.llvm.org/D49993

llvm-svn: 338377
2018-07-31 13:41:59 +00:00
Peter Collingbourne a327a4c34e ELF: Implement --icf=safe using address-significance tables.
Differential Revision: https://reviews.llvm.org/D48146

llvm-svn: 337429
2018-07-18 22:49:31 +00:00
Sterling Augustine 4fd84c18df Implement framework for linking split-stack object files, and x86_64 support.
llvm-svn: 337332
2018-07-17 23:16:02 +00:00
George Rimar 484aabc818 [ELF] - Eliminate ObjFile<ELFT>::getLineInfo. NFC.
Flow is the same, but a bit shorter after this change.

llvm-svn: 337183
2018-07-16 15:29:35 +00:00
Rui Ueyama c020064655 Remove a dead variable.
llvm-svn: 334341
2018-06-09 00:54:18 +00:00
Han Shen 08d1640535 Correct aligment computation for shared object symbols.
The original computation for shared object symbol alignment is wrong when
st_value equals 0. It is very unusual for dso symbols to have st_value equal 0.
But when it happens, it causes obscure run time bugs.

Differential Revision: https://reviews.llvm.org/D47602

llvm-svn: 334135
2018-06-06 21:43:34 +00:00
George Rimar 64091d5626 [ELF] - Also use DW_AT_linkage_name when gathering information about variables for error messages.
Currently, when LLD do a lookup for variables location, it uses DW_AT_name attribute.
That is not always enough.

Imagine code:

namespace A {
  int bar = 0;
}

namespace Z {
  int bar = 1;
}

int hoho;
In this case there are 3 variables and their debug attributes are following:

A::bar has: DW_AT_name [DW_FORM_string] ("bar") DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x00000006] = "_ZN1A3barE")
Z::bar has: DW_AT_name [DW_FORM_string] ("bar") DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x0000003f] = "_ZN1Z3barE")
hoho has: DW_AT_name [DW_FORM_strp] ( .debug_str[0x0000004a] = "hoho") and has NO DW_AT_linkage_name attribute. Because it would be
the same as DW_AT_name and DWARF producers avoids emiting excessive data.

Hence LLD should also use DW_AT_linkage_name when it is available.
(currently, LLD fails to report location correctly because thinks that A::bar and Z::bar are the same things)

Differential revision: https://reviews.llvm.org/D47373

llvm-svn: 333880
2018-06-04 10:28:45 +00:00
Sam Clegg 3ad27e92bc Code cleanup in preparation for adding LTO for wasm. NFC.
- Move some common code into Common/rrorHandler.cpp and
  Common/Strings.h.
- Don't use `fatal` when incompatible bitcode files are
  encountered.
- Rename NameRef variable to just Name

See D47162

Differential Revision: https://reviews.llvm.org/D47206

llvm-svn: 333021
2018-05-22 20:20:25 +00:00
James Henderson 1a7aaf3cd5 [ELF] Update due to API change in .debug_line parsing
See r332845.

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D46832

llvm-svn: 332846
2018-05-21 15:31:23 +00:00
Rui Ueyama 52d783962f Fix typo in error message.
llvm-svn: 332658
2018-05-17 20:25:35 +00:00
Rui Ueyama f06d494f46 Improve error message for -thinlto-object-suffix-replace and simplify code.
llvm-svn: 332643
2018-05-17 18:27:12 +00:00
Rumeet Dhindsa d2eb089a0e Add support for ThinLTO plugin option thinlto-object-suffix-replace
Differential Revision: https://reviews.llvm.org/D46608

llvm-svn: 332527
2018-05-16 21:04:08 +00:00
George Rimar 8f2b2f4e04 [ELF] - Simplify. NFC.
llvm-svn: 332242
2018-05-14 13:21:09 +00:00
James Henderson d621037788 [ELF] Rework debug line parsing to use llvm::Error and callbacks (LLD-side)
Reviewed by: ruiu, grimar, espindola

Differential Revision: https://reviews.llvm.org/D44562

Summary:
r331971 changes the debug line parser interface to report LLVM errors in an
interface that different executables can use, rather than always being printed
directly as warnings to stderr. This change allows LLD to make use of the new
interface and call its own warning methods to report problems.

llvm-svn: 331972
2018-05-10 10:52:21 +00:00
Rumeet Dhindsa d366e36bbf Added support for ThinLTO plugin options : thinlto-index-only and thinlto-prefix-replace
Differential Revision: https://reviews.llvm.org/D46034

llvm-svn: 331405
2018-05-02 21:40:07 +00:00
Rui Ueyama 50bf643cfb Do not set RequiresNullTerminator. NFC.
When reading object files, we don't need '\0' at end of each file.

llvm-svn: 331045
2018-04-27 15:32:04 +00:00
Rui Ueyama 1d92aa7380 Add --warn-backrefs to maintain compatibility with other linkers
I'm proposing a new command line flag, --warn-backrefs in this patch.
The flag and the feature proposed below don't exist in GNU linkers
nor the current lld.

--warn-backrefs is an option to detect reverse or cyclic dependencies
between static archives, and it can be used to keep your program
compatible with GNU linkers after you switch to lld. I'll explain the
feature and why you may find it useful below.

lld's symbol resolution semantics is more relaxed than traditional
Unix linkers. Therefore,

  ld.lld foo.a bar.o

succeeds even if bar.o contains an undefined symbol that have to be
resolved by some object file in foo.a. Traditional Unix linkers
don't allow this kind of backward reference, as they visit each
file only once from left to right in the command line while
resolving all undefined symbol at the moment of visiting.

In the above case, since there's no undefined symbol when a linker
visits foo.a, no files are pulled out from foo.a, and because the
linker forgets about foo.a after visiting, it can't resolve
undefined symbols that could have been resolved otherwise.

That lld accepts more relaxed form means (besides it makes more
sense) that you can accidentally write a command line or a build
file that works only with lld, even if you have a plan to
distribute it to wider users who may be using GNU linkers.  With
--check-library-dependency, you can detect a library order that
doesn't work with other Unix linkers.

The option is also useful to detect cyclic dependencies between
static archives. Again, lld accepts

  ld.lld foo.a bar.a

even if foo.a and bar.a depend on each other. With --warn-backrefs
it is handled as an error.

Here is how the option works. We assign a group ID to each file. A
file with a smaller group ID can pull out object files from an
archive file with an equal or greater group ID. Otherwise, it is a
reverse dependency and an error.

A file outside --{start,end}-group gets a fresh ID when
instantiated. All files within the same --{start,end}-group get the
same group ID. E.g.

  ld.lld A B --start-group C D --end-group E

A and B form group 0, C, D and their member object files form group
1, and E forms group 2. I think that you can see how this group
assignment rule simulates the traditional linker's semantics.

Differential Revision: https://reviews.llvm.org/D45195

llvm-svn: 329636
2018-04-09 23:05:48 +00:00
Rafael Espindola b4d3dfefb7 Avoid some temporary allocations.
Some system libraries have a lot of versioned symbols. When linking
scylla this brings the number of malloc calls from 49154 to 37944.

llvm-svn: 329453
2018-04-06 20:53:06 +00:00
Rui Ueyama 24a47201fd Merge LazyArchive::fetch() and ArchiveFile::getMember(). NFC.
They are to pull out an object file for a symbol, but for a historical
reason the code is written in two separate functions. This patch
merges them.

llvm-svn: 329039
2018-04-03 02:06:57 +00:00
Rui Ueyama 5a67a6ec4a Re-implement --just-symbols as a regular object file.
I tried a few different designs to find a way to implement it without
too much hassle and settled down with this. Unlike before, object files
given as arguments for --just-symbols are handled as object files, with
an exception that their section tables are handled as if they were all
null.

Differential Revision: https://reviews.llvm.org/D42025

llvm-svn: 328852
2018-03-30 01:15:36 +00:00
Rui Ueyama efe9af9f07 Rename NonLocal -> Global.
NonLocal is technically more accurate, but we already use the term
"Global" to specify the non-local part of the symbol table, and
Local <-> Global is easier to digest.

llvm-svn: 328740
2018-03-28 22:55:40 +00:00
Rafael Espindola c8f774b393 Strip @VER suffices from the LTO output.
This fixes pr36623.

The problem is that we have to parse versions out of names before LTO
so that LTO can use that information.

When we get the LTO produced .o files, we replace the previous symbols
with the LTO produced ones, but they still have @ in their names.

We could just trim the name directly, but calling parseSymbolVersion
to do it is simpler.

llvm-svn: 328738
2018-03-28 22:45:39 +00:00
Rafael Espindola 35aad41c1b Force SHF_MERGE optimizations with -r.
Some tools (dwarfdump for example) get confused by the current -O0 -r
output since it has multiple copies of .debug_str.

We cannot just merge sections with the same name as they can have
different sh_entsize.

We could have duplicated logic for merging sections based on name and
sh_entsize, but it seems better to just use the existing logic by
enabling optimizations.

llvm-svn: 328640
2018-03-27 17:09:23 +00:00
Rui Ueyama bc0d3b4e0f Remove extraneous local variable. NFC.
llvm-svn: 328605
2018-03-27 02:53:08 +00:00
Rui Ueyama b3e2f74517 Update comments.
llvm-svn: 328604
2018-03-27 02:52:58 +00:00
Rui Ueyama b391288af7 Refactor SharedFile::parseRest. NFC.
SharedFile::parseRest function grew organically and got a bit hard to
understand. This patch refactor it. This patch also adds comments.

Differential Revision: https://reviews.llvm.org/D44860

llvm-svn: 328579
2018-03-26 19:57:38 +00:00
Rui Ueyama d37c33aff2 Do not add a dummy entry to SharedFile::Verdefs. NFC.
Previously, we used 0 as an alias for VER_NDX_GLOBAL and had a dummy
entry in SharedFile::Verdefs so that the access to the array is within
its boundary. But that's not straightforwad. We can just stop doing both.

llvm-svn: 328401
2018-03-24 00:25:24 +00:00
Rui Ueyama bd63adcb26 Remove "FIXME" from a comment.
A bug in BFD linker is not our FIXME item.

llvm-svn: 328381
2018-03-23 22:48:17 +00:00
Rafael Espindola 85e77b26be Fix the MSVC build.
llvm-svn: 328285
2018-03-23 00:42:47 +00:00
Rafael Espindola 3c3ebcc5f4 Fix PR36793.
With this patch lld will iterate over compile units to find the line
tables instead of assuming there is only one at offset 0.

llvm-svn: 328284
2018-03-23 00:35:27 +00:00
Rafael Espindola c08e5afb7c Replace a std::pair with a struct.
This is more readable and should reduce the noise in a followup patch.

llvm-svn: 328164
2018-03-21 22:32:17 +00:00
Rui Ueyama 8e53917a26 Add a comment about ELF spec and the symbol table's sh_info.
llvm-svn: 327645
2018-03-15 17:10:50 +00:00
George Rimar 59ebd4adf0 [ELF] - Formatted comment, fixed mistype. NFC.
llvm-svn: 327280
2018-03-12 15:18:35 +00:00
George Rimar 1136ec64e8 [ELF] - Fix crash relative to SHF_LINK_ORDER sections.
Our code assumes all input sections in an output SHF_LINK_ORDER
section has SHF_LINK_ORDER flag. We do not check that and that can cause a crash.

That happens because we call 
std::stable_sort(Sections.begin(), Sections.end(), compareByFilePosition);, 
where compareByFilePosition predicate does not expect to see
null when calls getLinkOrderDep. 

The same might happen when sections refer to non-regular sections. 
Test cases demonstrate the issues, patch fixes them.

Differential revision: https://reviews.llvm.org/D44193

llvm-svn: 327006
2018-03-08 15:06:58 +00:00
Rui Ueyama e66d7a798f Return early. NFC.
We don't need to handle an object file having more than one symbol table,
so as soon as we find the first one, we can process it and then return
from the function.

llvm-svn: 326977
2018-03-08 01:22:30 +00:00
Rui Ueyama c13d858b6d Simplify LazyobjFile and readElfSymbols.
addElfSymbols and readJustSymbolsFile still has duplicate code, but
I didn't come up with a good idea to eliminate them. Since this patch
is an improvement, I'm sending this for review.

Differential Revision: https://reviews.llvm.org/D44187

llvm-svn: 326972
2018-03-08 01:05:58 +00:00
James Henderson cb4c19f315 [ELF] Prevent crash when reporting errors if debug line cannot be parsed
LLD uses the debug info and debug line sections to determine the location of
e.g. references to undefined symbols, when producing error messages. In the
event that debug info was present, but debug line parsing failed for some
reason, then a nullptr would end up being dereferenced by the location-lookup
code.

Differential Revision: https://reviews.llvm.org/D44205

Reviewers: grimar
llvm-svn: 326899
2018-03-07 15:22:58 +00:00
Rui Ueyama 09fcd4c34c Implement --just-symbols.
Differential Revision: https://reviews.llvm.org/D39348

llvm-svn: 326835
2018-03-06 21:25:37 +00:00
Rafael Espindola 3f4c673d38 Put undefined symbols from shared libraries in the symbol table.
With the recent fixes these symbols have more in common than not with
regular undefined symbols.

llvm-svn: 326242
2018-02-27 20:31:22 +00:00
Igor Kudrin 3345c9ac18 [ELF] Create and export symbols provided by a linker script if they referenced by DSOs.
It should be possible to resolve undefined symbols in dynamic libraries
using symbols defined in a linker script.

Differential Revision: https://reviews.llvm.org/D43011

llvm-svn: 326176
2018-02-27 07:18:07 +00:00
Peter Collingbourne 09e04af42f ELF: Stop collecting a list of symbols in ArchiveFile.
There seems to be no reason to collect this list of symbols.

Also fix a bug where --exclude-libs would apply to all symbols that
appear in an archive's symbol table, even if the relevant archive
member was not added to the link.

Differential Revision: https://reviews.llvm.org/D43369

llvm-svn: 325380
2018-02-16 20:23:54 +00:00
Simon Atanasyan 85815a3149 [ELF][MIPS] Ignore incorrect version definition index for _gp_disp symbol
MIPS BFD linker puts _gp_disp symbol into DSO files and assigns zero
version definition index to it. This value means 'unversioned local
symbol' while _gp_disp is a section global symbol. We have to handle
this bug in the LLD because BFD linker is used for building MIPS
toolchain libraries.

Differential revision: https://reviews.llvm.org/D42486

llvm-svn: 324467
2018-02-07 10:02:49 +00:00
George Rimar f9dc10cd89 [ELF] - Report valid binary filename when reporting error.
We did not report valid filename for duplicate symbol error when
symbol came from binary input file.
Patch fixes it.

Differential revision: https://reviews.llvm.org/D42635

llvm-svn: 324217
2018-02-05 09:47:24 +00:00
Rui Ueyama f46d3d1be9 Strip .note.gnu.build-id sections if --build-id is given.
Differential Revision: https://reviews.llvm.org/D42823

llvm-svn: 324146
2018-02-02 21:56:24 +00:00
Paul Robinson bf750c80e9 [DWARFv5] Re-enable dumping a line table with no CU.
r323476 added support for DW_FORM_line_strp, and incorrectly made that
depend on having a DWARFUnit available.  We shouldn't be tracking
.debug_line_str in DWARFUnit after all.  After this patch, I can do an
NFC follow up and undo a bunch of the "plumbing" part of r323476.

Differential Revision: https://reviews.llvm.org/D42609

llvm-svn: 323691
2018-01-29 20:57:43 +00:00
Rafael Espindola 9a84f6b954 Detemplate reportDuplicate.
We normally avoid "switch (Config->EKind)", but in this case I think
it is worth it.

It is only executed when there is an error and it allows detemplating
a lot of code.

llvm-svn: 321404
2017-12-23 17:21:39 +00:00
Rafael Espindola ce3b52c186 Pass an InputFile to the InputSection constructor.
This simplifies toRegularSection and reduces the noise in a followup
patch.

llvm-svn: 321240
2017-12-21 02:11:51 +00:00