NativeReferenceIvarsV1 cannot handle more than 65535 relocation targets
because its field to point to the target table is of type uint16_t. Because
of that limitation, the LLD couldn't link a file containing more than 65535
relocations. 65535 is not a big number - the LLD couldn't even link itself
with V1.
This patch solves the issue by adding NativeReferenceIvarsV2 support. The
new structure has more bits for the target table, so it can handle a large
number of relocatinos.
V2 structure is larger than V1. In order to prevent file bloating, V2 format
is used only when the resulting file cannot be represented in V1 format. The
writer and the reader support both V1 and V2 formats.
Differential Revision: http://llvm-reviews.chandlerc.com/D2217
llvm-svn: 195270
The fallback atom was used only when it's searching for a symbol in a library;
if an undefined symbol was not found in a library, the LLD looked for its
fallback symbol in the library.
Although it worked in most cases, because symbols with fallbacks usually occur
only in OLDNAMES.LIB (a standard library), that behavior was incompatible with
link.exe. This patch fixes the issue so that the semantics is the same as
MSVC's link.exe
The new (and correct, I believe) behavior is this:
- If there's no definition for an undefined atom, replace the undefined atom
with its fallback and then proceed (e.g. look in the next file or stop
linking as usual.)
Weak External symbols are underspecified in the Microsoft PE/COFF spec. However,
as long as I observed the behavior of link.exe, this seems to be what we want
for compatibility.
Differential Revision: http://llvm-reviews.chandlerc.com/D2162
llvm-svn: 195269
The maximum number of references the file with NativeReferenceIvarsV1 can
contain is 65534. If a file larger than that is converted to Native format,
the conversion will fail without any error message. This caused a subtle bug
that the LLD would produce a broken executable only when input files contain
too many references.
This issue exists since the RoundTripNativeTest is introduced in r193585. Since
then, it seems that nobody have linked any program having more than 65534
relocations with the LLD. Otherwise we would have found it earlier.
llvm-svn: 194987
This patch does not change the meaning of the program, but if something's wrong
in the linker or the compiler and the control reaches to the gap of imported
function table, it will stop immediately because of the presence of INT3. If
NOP, it'd fall through to the next call instruction, which is usually a
completely foreign function call.
llvm-svn: 194860
We can add multiple undefined atoms having the same name to the symbol table.
If such atoms are added, the symbol table compares their canBeNull attributes,
and select one having a stronger constraint. If their canBeNulls are the same,
the choice is arbitrary. Currently it choose the existing one.
This patch changes the preference, so that the symbol table choose the new one
if the new atom has a greater canBeNull or a fallback atom. This shouldn't
change the behavior except the case described below.
A new undefined atom may have a new fallback atom attribute. By choosing the new
atom, we can update the fallback atom during Core Linking. PE/COFF actually need
that. For example, _lseek is an alias for __lseek on Windows. One of an object
file in OLDNAMES.LIB has an undefined atom for _lseek with the fallback to
__lseek. When the linker tries to resolve _read, it supposed to read the file
from OLDNAMES.LIB and use the new fallback from the file. Currently LLD cannot
handle such case because duplicate undefined atoms with the same attributes are
ignored.
Differential Revision: http://llvm-reviews.chandlerc.com/D2161
llvm-svn: 194777
YAML files tend to be very large compared to binary formats because of ASCII
format inefficiency. And the YAML reader consumes an excessively large amount
of memory when parsing a large file. It's very slow too.
For example, I observed that 6MB executable became 120MB YAML file, and the
YAML reader consumed more than 1.5GB memory to load it. The YAML reader even
caused OOM error on 32 bit, causing the entire process to fail.
This patch sets the limit on the YAML file size the linker will try to load in
the RoundTripYAML test as a safeguard.
llvm-svn: 194666
The result of sizeof(SymbolTable<ELFT>::SymbolEntry) in DynamicSymbolTable
<ELFT>::write() was different from the same expression in RelocationTable
<ELFT>::write(), although the same template parameters were passed. They were
40 and 32, respectively. As a result, the same vector was treated as a
vector of 40 byte values in some places and a vector of 32 values in other
places. That caused an weird issue, resulting in collapse of the rela.dyn
section.
I suspect that this is a padding size calculation bug in MSVC 2012, but I
may be wrong. Reordering the fields to eliminate padding seems to fix the
issue.
llvm-svn: 194349
This patch adds support for converting normalized mach-o to and from binary
mach-o. It also changes WriterMachO (which previously directly wrote a
mach-o binary given a set of Atoms) to instead do it in two steps. The first
step uses normalizedFromAtoms() to convert Atoms to normalized mach-o, and the
second step uses writeBinary() which to generate the mach-o binary file.
llvm-svn: 194167
These fields are for /align option. Section alignment can be set per-section
basis with /section option too. In order to avoid name conflicts, rename the
existing identifiers to become more specific. No functionality change.
llvm-svn: 194160
/section command line option is to set/reset attributes of the Characteristics
field in the section header. You can set non-default values with this option.
You can make .data section executable with this, for example.
This patch implements the parser of the command line option. The code to use
the parsed values will be committed in a separate patch.
llvm-svn: 194133
I'm not sure if it is really an alias for /nodefaultlib, but I can say that
they are at least similar. Making it an alias would be better than ignoring it.
llvm-svn: 194131
/defaultlib options can be specified implicitly via the .drectve section, and
it's pretty common that multiple object files add the same library, such as
user32.lib, to the input. We shouldn't add the same library multiple times.
llvm-svn: 194129
We wrapped the linker internal file with a virtual archive file, so that the
linker internal file was linked only when it's actually used. This was to avoid
__ImageBase being included to the resulting executable. __ImageBase used to
occupy four bytes when emitted to executable.
And then it turned out that the implementation of __ImageBase was wrong -- it
shouldn't have been a regular atom but an absolute atom. Absolute atoms point
to some memory location, but they don't occupy disk space themselves. So it
wouldn't increase executable size (except the symbol table.) That means that
it's OK to link the linker internal file unconditionally.
So this patch does that, removing the wrapper archive file. Doing this
simplifies the code.
llvm-svn: 194127
msvcrt.lib contains "/disallowlib" command line option in its .drectve section.
I couldn't spot any documentation for the option. Ignore it for now so that we
can link the library without error.
llvm-svn: 194114
This patch should fix the test when it runs on Windows, by allowing drive
letter separator (colon) in the path. Now all LLD ELF tests passed on MSVC
2012 32-bit. Hooray!
llvm-svn: 193978
n_desc field in MachO string table was not initialized. On Unix,
test/darwin/hello-world.objtxt did not fail because I think an nlist object
is always allocated to a fresh heap initialized with zeros. On Windows,
uninitialized fields are filled with 0xCC when compiled with /GZ. Because
of that the test was failing on Windows.
llvm-svn: 193909
On Windows, neither "(" nor ")" are shell special characters, so -\( is passed
as-is to LLD. Because of that this test was failing on Windows.
llvm-svn: 193905
This patch adds "-target x86_64" to the command line. Without this option,
a 32 bit object file would be created on 32 bit machine, resulting in test
failure.
llvm-svn: 193904
Bugs that would be caught by this assertion would also be caught by
RoundTripYAMLPass test. We've enabled the pass for PECOFF, so we can remove
this.
llvm-svn: 193886
The data directory in the PE/COFF header consisted of list of data directory
atoms. This patch changes it -- now there's only one data directory entry that
contains former data directories. That's easier to handle in the writer as well
as to write to/read from YAML/Native files. The main purpose of this refactoring
is to enable RoundTrip tests for PE/COFF.
There's no functionality change.
llvm-svn: 193854
This reverts commit r193479.
The atoms are already added to the file, so re-adding them caused the YAML
writer to write the same atoms twice. That made the YAML reader to fail with
"duplicate atom name" error.
This is not the only error we've got for RoundTripYAMLPass for PECOFF, so we
cannot enable the test yet. More fixes will come.
Differential Revision: http://llvm-reviews.chandlerc.com/D2069
llvm-svn: 193762
Enable this for the following flavors
a) core
b) gnu
c) darwin
Its disabled for the flavor PECOFF. Convenient markers are added with FIXME
comments in the Driver that would be removed and code removed from each flavor.
llvm-svn: 193585
__ImageBase is an absolute symbol whose address is the same as the image base
address. What we did before this patch was to create __ImageBase symbol as a
symbol whose *contents* (not location) is the image base address, which is
clearly wrong.
llvm-svn: 193565
intended for debugging and diagnostic output), just inspect the spelling
to check for specific prefixes in drectve section flags.
In addition to being significantly cheaper and not relying on
a debugging interface, this also avoids creating a temporary string and
binding it to StringRef variable. We then went on to access it after the
memory had been deallocated.
This bug too was caught by ASan. I love ASan so much. =]
llvm-svn: 193487
More important than any performance concerns, the code was dropping the
temporary string on the floor after assigning it to a StringRef, and
then used the StringRef later. Caught by running the LLD tests under
ASan.
llvm-svn: 193486
On discussing this with Nick, it looks like the StubAtoms
that contain a lazyImmediate reference kind should be null
and the location needs to be fixed up later with some value
that is an offset into the __LINKEDIT segment.
The drawback is that it allows yaml files with references
that expect a target to be considered without one.
This results in bad yaml files that would need to be handled
in the YAML Reader.
Inorder to fix this, the Stub Atoms use a dummy target such
as itself.
llvm-svn: 193476
/merge:<from>=<to> option makes the linker to combine "from" section to "to"
section. This patch is to parse the option. The actual feature will be
implemented in a subsequent patch.
llvm-svn: 193454
We really need a test for the manifest file output, but because it depends
on external commands (CVTRES.EXE and RC.EXE), it's not very easy to write it.
llvm-svn: 193445
The internal byte array of the SmallString filled by createTemporaryFile() is
not guaranteed to be NUL-terminated. We need to call c_str() to handle it
safely.
llvm-svn: 193442
Disable tests to be run with REQUIRES: disable. Note disable is not added to the
config by the test runner Mkaefiles, so essentially disables the test.
Code changes would be required to fix these tests :-
test/darwin/hello-world.objtxt
test/elf/check.test
test/elf/phdr.test
test/elf/ppc.test
test/elf/undef-from-main-dso.test
test/elf/X86_64/note-sections-ro_plus_rw.test
test/pecoff/alignment.test
test/pecoff/base-reloc.test
test/pecoff/bss-section.test
test/pecoff/drectve.test
test/pecoff/dynamic.test
test/pecoff/dynamicbase.test
test/pecoff/entry.test
test/pecoff/hello.test
test/pecoff/imagebase.test
test/pecoff/importlib.test
test/pecoff/lib.test
test/pecoff/multi.test
test/pecoff/reloc.test
test/pecoff/weak-external.test
llvm-svn: 193300
Instead of making the linker to create a manifest XML file in the same
directory as the resulting binary, you can embed the XML as a part of
resource into the executable.
In order to do that, the linker first creates a resource script file containing
the XML file, compile it into a binary resource file with RC.EXE, and then
convert it to a COFF file with CVTRES.EXE.
llvm-svn: 193298
This patch won't change LLD's behavior because it's a temporary file and
LLD does not use the file extension to determine file type. But using the
correct file extension is a good thing.
llvm-svn: 193211
/manifestfile:<path> specifies an alternative manifest file output path.
Default is "<output-path>.manifest" where <output-path> is the executable's
path.
llvm-svn: 193195
The manifest file is an XML file that conveys some information to the loader,
such as whether the executable needs to run as Administrator or not. This patch
is to parse command line option for manifest file.
Actual XML file generation will be done in a separate patch.
llvm-svn: 193141
This patch fixes a bug in r190608. The results of a comparison function
passed to std::sort must be transitive, which is, if a < b and b < c, and if
a != b, a < c must be also true. CompareAtoms::compare did not actually
guarantee the transitivity. As a result the sort results were sometimes just
wrong.
Consider there are three atoms, X, Y, and Z, whose file ordinals are 1, 2, 3,
respectively. Z has a property "layout-after X". In this case, all the
following conditionals become true:
X < Y because X's ordinal is less than Y's
Y < Z because Y's ordinal is less than Z's
Z < X because of the layout-after relationship
This is not of course transitive. The reason why this happened is because
we used follow-on relationships for comparison if two atoms falls in the same
follow-on chain, but we used each atom's properties if they did not. This patch
fixes the issue by using follow-on root atoms for comparison to get consistent
results.
Differential Revision: http://llvm-reviews.chandlerc.com/D1980
llvm-svn: 193029
We should dead-strip atoms only if they are created for COMDAT symbols. If we
remove non-COMDAT atoms from a binary, it will no longer be guaranteed that
the binary will work correctly.
In COFF, you can manipulate the order of section contents in the resulting
binary by section name. For example, if you have four sections
.data$unique_prefix_{a,b,c,d}, it's guaranteed that the contents of A, B, C,
and D will be consecutive in the resulting .data section in that order.
Thus, you can access B's and C's contents by incrementing a pointer pointing
to A until it reached to D. That's why we cannot dead-strip B or C even if
no one is directly referencing to them.
Some object files in the standard library actually use that technique.
llvm-svn: 193017
INT 3 (machine code 0xCC) will raise an interrupt when executed. That is better
for filling the gap than NOP because we want to stop the execution immediately
when the control reached to non-code address.
llvm-svn: 192945
Instead of showing multiple lines of debug messages, show only one message
by CompareAtoms::operator(). Here is an example.
Before:
Sorting _main .text
Sorting by sectionPos(2,2)
Sorting by override
Sorting _main .text
Sorting by sectionPos(2,2)
Sorting by override
After:
Layout: '_main' > '.text' (override (1, 0))
Layout: '_main' > '.text' (override (1, 0))
Differential Revision: http://llvm-reviews.chandlerc.com/D1964
llvm-svn: 192941
Dead-strip root symbols can be undefined atoms, but should not really be
nonexistent, because dead-strip root symbols should be added to initial
undefined atoms at startup. Whenever you look up its name in the symbol
table, some type of atom will always exist.
llvm-svn: 192831
There are aliases for --start-group/--end-group options represented
by -( and -) respectively in the command line.
This change adds and improves the test for the alias options to be
tested.
Looks like users use this option widely than explicitly using
--start-group/--end-group.
llvm-svn: 192470
allowRemainingUndefines() is already checked in Resolver::resolve(), so we
don't need to check it again after returning from it. It's actually not only
superfluous but buggy because a failure of resolve() does not always mean that
there is a remaining undefines.
llvm-svn: 192423
-- so that command line options to specify new input files, such as
/defaultlib:foo, is handled properly. Such options were ignored before
this patch.
llvm-svn: 192342
A file with .objtxt extension is parsed in readFile(), but because we did not
propagate that information to the calling side, calling side would try to parse
it again. This patch will fix the issue by adding an extra parameter to
readFile().
llvm-svn: 192311
This change removes code in various places which was setting the File Ordinals.
This is because the file ordinals are assigned by the way files are resolved.
There was no other way than making the getNextFileAndOrdinal be set const and
change the _nextOrdinal to mutable.
There are so many places in code, that you would need to cleanup to make
LinkingContext non-const!
llvm-svn: 192280
This associates resolveState to FileNodes. The control node derive
their resolution state from the inputElements that are contained in
it.
This makes --start-group/--end-group to work with ELF linking.
llvm-svn: 192269
Output to llvm::err() is not guaranteed to be thread-safe, so it needs
to be guarded with a lock.
Differential Revision: http://llvm-reviews.chandlerc.com/D1862
llvm-svn: 192250
Summary:
The original code with enum "_" is intended to emulate scoped enums.
Now we have real scoped enums, so use it.
Reviewers: Bigcheese
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D1852
llvm-svn: 192148
This is the first step in how I plan to get mach-o object files support into
lld. We need to be able to test the mach-o Reader and Write on systems without
a mach-o tools. Therefore, we want to support a textual way (YAML) to represent
mach-o files.
MachONormalizedFile.h defines an in-memory abstraction of the content of mach-o
files. The in-memory data structures are always native endianess and always
use 64-bit sizes. That internal data structure can then be converted to or
from three different formats: 1) yaml (text) encoded mach-o, 2) binary mach-o
files, 3) lld Atoms.
This patch defines the internal model and uses YAML I/O to implement the
conversion to and from the model to yaml. The next patch will implement
the conversion from normalized to binary mach-o.
This patch includes unit tests to validate the yaml conversion APIs.
llvm-svn: 192147
Changes :-
a) Functionality in InputGraph to insert Input elements at any position
b) Functionality in the Resolver to use nextFile
c) Move the functionality of assigning file ordinals to InputGraph
d) Changes all inputs to MemoryBuffers
e) Remove LinkerInput, InputFiles, ReaderArchive
llvm-svn: 192081
Found this with asan. Code assumes that find doesn't return end, thus if
both atoms didn't have followon roots it would still compare their positions.
llvm-svn: 191865
Rui Ueyama