Commit Graph

80 Commits

Author SHA1 Message Date
Peter Wu f0ad811b54 [Object] Deduplicate long archive member names
Summary:
Import libraries as created by llvm-dlltool always use the same archive
member name for every object file (namely, the DLL library name). Ensure
that long names are not repeatedly stored in the string table.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D55860

llvm-svn: 349637
2018-12-19 16:15:05 +00:00
James Y Knight c0b28d55a7 llvm-ar: Darwin archive format fixes.
* Support writing the DARWIN64 symbol table format.

* In darwin archives, emit a symbol table whenever requested, even
  when there are no members, as the apple linker will abort if given
  an archive without a symbol table.

Added tests for same, and also simplified and moved the GNU 64-bit
symbol table test into archive-symtab.test.

llvm-svn: 344183
2018-10-10 21:07:02 +00:00
James Y Knight 0d1bb79a04 Give same-named members unique timestamps on Darwin in llvm-ar.
This change ensures that the (membername,timestamp) tuple uniquely
identifies an entry in an archive for format=darwin, in deterministic
mode (which is the default).

That, then, enables lldb and dsymutil to locate the appropriate object
within the archive.

Differential Revision: https://reviews.llvm.org/D47659

llvm-svn: 343805
2018-10-04 18:49:21 +00:00
Alexander Shaposhnikov 72b27a6a39 [object] Improve the performance of getSymbols used by ArchiveWriter
In this diff we adjust the code of getSymbols to avoid creating LLVMContext when it's not necessary.
Without this patch when the function getSymbols was called on a MachO object with a __bitcode section
it was parsing the embedded bitcode and then ignoring the result.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D51759

llvm-svn: 341998
2018-09-11 22:00:47 +00:00
Martin Storsjo a6ffc9c8df [COFF] Adjust how we flag weak externals
This fixes PR36096.

Originally based on a patch by Martell Malone.

Differential Revision: https://reviews.llvm.org/D44357

llvm-svn: 337613
2018-07-20 20:48:29 +00:00
Peter Collingbourne e3f652973e Support: Simplify endian stream interface. NFCI.
Provide some free functions to reduce verbosity of endian-writing
a single value, and replace the endianness template parameter with
a field.

Part of PR37466.

Differential Revision: https://reviews.llvm.org/D47032

llvm-svn: 332757
2018-05-18 19:46:24 +00:00
Nico Weber 712e8d29c4 s/LLVM_ON_WIN32/_WIN32/, llvm
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too.  Just use the
default macro instead of a reinvented one.

See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.

This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.

llvm-svn: 331127
2018-04-29 00:45:03 +00:00
Peter Collingbourne d579c31d68 [llvm-ar] Support multiple dashed options
This allows syntax like:
$ llvm-ar -c -r -u file.a file.o

This is in addition to the other formats that are already supported:
$ llvm-ar cru file.a file.o
$ llvm-ar -cru file.a file.o

Patch by Tom Anderson!

Differential Revision: https://reviews.llvm.org/D44452

llvm-svn: 328716
2018-03-28 17:21:14 +00:00
Jake Ehrlich 1a468481c0 Add flag to ArchiveWriter to test GNU64 format more efficiently
Even with the sparse file optimizations the SYM64 test can still be painfully
slow. This unnecessarily slows down devs. It's critical that we test that the
switch to the SYM64 format occurs at 4GB but there isn't any better of a way to
fake the size of the file than sparse files. This change introduces a flag that
allows the cutoff to be arbitrarily set to whatever power of two is desired.
The flag is hidden as it really isn't meant to be used outside this one test.
This is unfortunate but appears necessary, at least until the average hard
drive is much faster.

The changes to the test require some explanation. Prior to this change we knew
that the SYM64 format was being used because the file was simply too large to
have validly handled this case if the SYM64 format were not used. To ensure
that the SYM64 format is still being used I am grepping the file for "SYM64".
Without changing the filename however this would be pointless because "SYM64"
would occur in the file either way. So the filename of the test is also changed
in order to avoid this issue.

Differential Revision: https://reviews.llvm.org/D40632

llvm-svn: 319507
2017-12-01 00:54:28 +00:00
Rafael Espindola c02eacf4c4 Use TempFile in llvm-ar. NFC.
llvm-svn: 318127
2017-11-14 01:21:15 +00:00
Jake Ehrlich c3a89eefd6 Reland "Add support for writing 64-bit symbol tables for archives when offsets become too large for 32-bit"
Tests were failing because some bots were running out of address
space and memory. Additionally the test was very slow. These issues
were solved by changing the test to take advantage of sparse filse and
restricting the test to run only on 64-bit systems.

This should fix https://bugs.llvm.org//show_bug.cgi?id=34189

This change makes it so that if writing a K_GNU style archive, you need
to output a > 32-bit offset it should output in K_GNU64 style instead.

Differential Revision: https://reviews.llvm.org/D36812

llvm-svn: 317352
2017-11-03 19:15:06 +00:00
Jake Ehrlich f22728e636 Revert "Add support for writing 64-bit symbol tables for archives when offsets become too large for 32-bit"
This reverts commit r316805.

llvm-svn: 316813
2017-10-27 23:39:31 +00:00
Jake Ehrlich 9d5a7c3b8c Add support for writing 64-bit symbol tables for archives when offsets become too large for 32-bit
This should fix https://bugs.llvm.org//show_bug.cgi?id=34189

This change makes it so that if writing a K_GNU style archive, you need
to output a > 32-bit offset it should output in K_GNU64 style instead.

Differential Revision: https://reviews.llvm.org/D36812

llvm-svn: 316805
2017-10-27 22:26:37 +00:00
Rafael Espindola 476a7f9293 Pre-compute the tail of the archive
An archive looks like

<header>
<symbol table>
<tail>

The symbol table refers to offsets in the tail. A complication is that
we would like to support symbol tables that use 64 bit offsets if it
turns out that any of the offsets is too big.

This patch changes the archive writer to first compute the tail. We
cannot just compute one big StringRef since that would require reading
every member upfront, but we can represent it as a series of
StringRefs.

Having done that it is much easier to compute the symbol table and all
offsets are computed before it is written. With this if there is an
accounting problem it will show up with a regular symbol table, not
just when a 64 bit one is needed.

llvm-svn: 314844
2017-10-03 20:59:43 +00:00
Reid Kleckner 3fc649cb76 [Support] Rename tool_output_file to ToolOutputFile, NFC
This class isn't similar to anything from the STL, so it shouldn't use
the STL naming conventions.

llvm-svn: 314050
2017-09-23 01:03:17 +00:00
Rafael Espindola d901deee6e Move code to a helper function. NFC.
Part of a patch by Jake Ehrlich!

llvm-svn: 314012
2017-09-22 18:40:14 +00:00
Rafael Espindola 0bd982b79f llvm-ar: align the first archive member consistently.
Before we were aligning the member after the symbol table to 4 but
other members to 8.

llvm-svn: 314010
2017-09-22 18:36:00 +00:00
Rafael Espindola d5d77372d4 llvm-ar: Don't add an unnecessary alignment in gnu mode.
This is mostly for getting stricter testing in preparation for future
changes.

llvm-svn: 314000
2017-09-22 18:16:13 +00:00
Rafael Espindola 25cbdf25a6 Convert the archive writer to use Error.
This found one place in lld that was not checking the error.

llvm-svn: 313937
2017-09-21 23:13:36 +00:00
Rafael Espindola fa08397f20 Use raw_ostream in functions that don't need to seek. NFC.
llvm-svn: 313935
2017-09-21 23:06:23 +00:00
Rafael Espindola 540a8c7fad Simplify the logic for truncating UID and GID. NFC.
llvm-svn: 313933
2017-09-21 23:00:55 +00:00
Rafael Espindola 4c9e14f6c4 Use ArrayRef. NFC.
llvm-svn: 313895
2017-09-21 17:51:07 +00:00
Jake Ehrlich 1b30d63aeb Rename K_MIPS64 to K_GNU64
This patch renames K_MIPS64 to K_GNU64 as part of a change to add
support for writing archives with 64-bit indexes in the symbol table.

llvm-svn: 313787
2017-09-20 18:23:01 +00:00
Rui Ueyama 01d0265106 Simplify writeArchive return type.
writeArchive returned a pair, but the first element of the pair is always
its first argument on failure, so it doesn't make sense to return it from
the function. This patch change the return type so that it does't return it.

Differential Revision: https://reviews.llvm.org/D37313

llvm-svn: 312177
2017-08-30 22:11:03 +00:00
Martell Malone 1079ef8dfe llvm: add llvm-dlltool support to the archiver
A PE COFF spec compliant import library generator.
Intended to be used with mingw-w64.

Supports:
PE COFF spec (section 8, Import Library Format)
PE COFF spec (Aux Format 3: Weak Externals)

Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D29892

This reapplies rL308329, which was reverted in rL308374

llvm-svn: 308379
2017-07-18 21:26:38 +00:00
Rui Ueyama 6db83a3af3 Revert r308329: llvm: add llvm-dlltool support to the archiver
This reverts commit r308329 because it broke buildbots.

llvm-svn: 308374
2017-07-18 21:07:13 +00:00
Martell Malone afe8549269 llvm: add llvm-dlltool support to the archiver
A PE COFF spec compliant import library generator.
Intended to be used with mingw-w64.

Supports:
PE COFF spec (section 8, Import Library Format)
PE COFF spec (Aux Format 3: Weak Externals)

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D29892

llvm-svn: 308329
2017-07-18 17:39:11 +00:00
Reid Kleckner 2f3f503d13 [llvm-ar] Make llvm-lib behave more like the MSVC archiver
Summary:
Use the filepath used to open the archive member as the archive member
name instead of the file basename. This path might be absolute or
relative.  This is important because the archive member name will show
up in the PDB, and we want our PDBs to look as much like MSVC's as
possible.

This also helps avoid an issue in our PDB module descriptor writing
code, which assumes that all module names are unique. Relative paths
still aren't guaranteed to be unique, but they're much better than
basenames, which definitely aren't unique.

Reviewers: ruiu, zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33575

llvm-svn: 305223
2017-06-12 19:45:35 +00:00
Zachary Turner 264b5d9e88 Move Object format code to lib/BinaryFormat.
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.

Differential Revision: https://reviews.llvm.org/D33843

llvm-svn: 304864
2017-06-07 03:48:56 +00:00
Peter Collingbourne 00f200f385 ArchiveWriter: Remove unused variables. NFC.
llvm-svn: 297987
2017-03-16 20:35:07 +00:00
Rafael Espindola 8f2dd7c042 Archives require a symbol table on Solaris, even if empty.
On Solaris ld (and some other tools that use the underlying utility
libraries, such as elfdump) chokes on an archive library that has no
symbol table. The Solaris tools always create one, even if it's empty.

That bug has been fixed in the latest development line, and can
probably be backported to a supported release, but it would be nice if
LLVM's archiver could emit the empty symbol table, too.

Patch by Danek Duvall!

llvm-svn: 297773
2017-03-14 19:57:13 +00:00
Rafael Espindola f133ccbd8d Move llvm_unreachable out of switch.
This should make gcc happy and still produce a clang warning if we add
another value to the enum.

llvm-svn: 295865
2017-02-22 19:42:14 +00:00
Rui Ueyama e67e162654 Fix -Wcovered-switch-default.
llvm-svn: 295799
2017-02-22 01:01:45 +00:00
Rafael Espindola 23a76be5ad Don't modify archive members unless really needed.
For whatever reason ld64 requires that member headers (not the member
themselves) should be aligned. The only way to do that is to edit the
previous member so that it ends at an aligned boundary.

Since modifying data put in an archive is an undesirable property,
llvm-ar should only do it when it is absolutely necessary.

llvm-svn: 295765
2017-02-21 20:40:54 +00:00
Saleem Abdulrasool 111cd669e9 Object: pad out BSD archive members to 8-bytes
ld64 requires its archive members to be 8-byte aligned for 64-bit
content and 4-byte aligned for 32-bit content.  Opt for the larger
alignment requirement.  This ensures that ld64 can consume archives
generated by llvm-ar.

Thanks to Kevin Enderby for the hint about the ld64/cctools behaviours!

Resolves PR28361!

llvm-svn: 294615
2017-02-09 19:29:35 +00:00
Saleem Abdulrasool d3faeaf8a2 Object: add a comment explaining a divergence
Add a note about the reason for the divergence from the specification
for ld64.  Addresses post-commit review comments from Davide.  NFC.

llvm-svn: 294594
2017-02-09 15:47:58 +00:00
Saleem Abdulrasool b4a162be21 Object: pad BSD ar string table to 4-bytes
cctools would pad the string table to a sizeof(int32_t) (explicitly
printed out by cctools rather than 4).  This adjusts the string table to
make it more compatible with cctools, but is insufficient to make ld64
happy.

llvm-svn: 294557
2017-02-09 04:26:21 +00:00
David Callahan 5cb34077e8 Only computeRelativePath() on new members
Summary:
When using thin archives, and processing the same archive multiple times, we were mangling existing entries.  The root cause is that we were calling computeRelativePath() more than once.   Here, we only call it when adding new members to an archive.

Note that D27218 changes the way thin archives are printed, and will break the new unit test included here.  Depending on which one lands first, the other will need to be slightly modified.

Reviewers: rafael, davide

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D27217

llvm-svn: 288280
2016-11-30 22:32:58 +00:00
Peter Collingbourne bc9a574657 Object: replace backslashes with slashes in embedded relative thin archive paths on Windows.
This makes these thin archives portable between *nix and Windows.

Differential Revision: https://reviews.llvm.org/D26696

llvm-svn: 287038
2016-11-15 21:36:35 +00:00
Pavel Labath bff47b51b6 [Object] Replace TimeValue with std::chrono
Summary:
Most of the changes are very straight-forward. The only choice I had to make was
to use second-precision time points in the Archive classes. I did this because
the archive files use that precision in the on-disk representation anyway.

Reviewers: rafael, zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25773

llvm-svn: 284974
2016-10-24 13:38:27 +00:00
Vedant Kumar 4031d9f80e Reapply "More fixes to get good error messages for bad archives."
This reverts commit the revert commit r277627. The build errors
mentioned in r277627 were likely caused by an unclean build directory.
Sorry for the noise.

llvm-svn: 277630
2016-08-03 19:02:50 +00:00
Vedant Kumar bfb6072d84 Revert "More fixes to get good error messages for bad archives."
This reverts commit r277540. It breaks the build with:

../lib/Object/Archive.cpp:264:41: error: return type of out-of-line definition of 'llvm::object::ArchiveMemberHeader::getUID' differs from that in the declaration
Expected<unsigned> ArchiveMemberHeader::getUID() const {
~~~~~~~~~~~~~~~~~~                      ^
include/llvm/Object/Archive.h:53:12: note: previous declaration is here
  unsigned getUID() const;
  ~~~~~~~~ ^

llvm-svn: 277627
2016-08-03 18:44:32 +00:00
Kevin Enderby 395cc09444 More fixes to get good error messages for bad archives.
Fixed the last incorrect uses of llvm_unreachable() in the code
which were actually just cases of errors in the input Archives.

llvm-svn: 277540
2016-08-02 22:58:55 +00:00
Kevin Enderby f4586039f6 The next step along the way to getting good error messages for bad archives.
As mentioned in commit log for r276686 this next step is adding a new
method in the ArchiveMemberHeader class to get the full name that
does proper error checking, and can be use for error messages.

To do this the name of ArchiveMemberHeader::getName() is changed to
ArchiveMemberHeader::getRawName() to be consistent with
Archive::Child::getRawName().  Then the “new” method is the addition
of a new implementation of ArchiveMemberHeader::getName() which gets
the full name and provides proper error checking.  Which is mostly a rewrite
of what was Archive::Child::getName() and cleaning up incorrect uses of
llvm_unreachable() in the code which were actually just cases of errors
in the input Archives.

Then Archive::Child::getName() is changed to return Expected<> and use
the new implementation of ArchiveMemberHeader::getName() .

Also needed to change Archive::getMemoryBufferRef() with these
changes to return Expected<> as well to propagate Errors up.
As well as changing Archive::isThinMember() to return Expected<> .

llvm-svn: 277177
2016-07-29 17:44:13 +00:00
Peter Collingbourne 8ec68fad33 Object: Replace NewArchiveIterator with a simpler NewArchiveMember class. NFCI.
The NewArchiveIterator class has a problem: it requires too much context. Any
memory buffers added to the archive must be stored within an Archive::Member,
which must have an associated Archive. This makes it harder than necessary
to create new archive members (or new archives entirely) from scratch using
memory buffers.

This patch replaces NewArchiveIterator with a NewArchiveMember class that
stores just the memory buffer and the information that goes into the archive
member header.

Differential Revision: http://reviews.llvm.org/D21721

llvm-svn: 274183
2016-06-29 22:27:42 +00:00
Rafael Espindola 484983f453 Fix bug where temporary file would be left behind every time an archive was updated.
When updating an existing archive, llvm-ar opens the old archive into a
`MemoryBuffer`, does its thing, and writes the results to a temporary
file. That file is then renamed to the original archive filename, thus
replacing it with the updated contents. However, on Windows at least,
what would happen is that the `MemoryBuffer` for the old archive would
actually be an mmap'ed view of the file, so when it came time to do the
rename via Win32's `ReplaceFile`, it would succeed but would be unable
to fully replace the file since there would still be a handle open on
it; instead, the old version got renamed to a random temporary name and
left behind.

Patch by Cameron!

llvm-svn: 268916
2016-05-09 13:31:11 +00:00
Davide Italiano 8a2593331c [lib/Object] Make this assertion more useful.
llvm-svn: 268367
2016-05-03 07:30:56 +00:00
Rafael Espindola 21507a4a5a Don't try to create thin bsd archives.
Not such variant has been specified yet.

llvm-svn: 268305
2016-05-02 21:06:57 +00:00
Nico Weber 77729b82e7 Replace a loop with a for-each loop. No behavior change.
llvm-svn: 266903
2016-04-20 19:09:26 +00:00
Kevin Enderby 3fcdf6ae2a Thread Expected<...> up from createMachOObjectFile() to allow llvm-objdump to produce a real error message
Produce the first specific error message for a malformed Mach-O file describing
the problem instead of the generic message for object_error::parse_failed of
"Invalid data was encountered while parsing the file”.  Many more good error
messages will follow after this first one.

This is built on Lang Hames’ great work of adding the ’Error' class for
structured error handling and threading Error through MachOObjectFile
construction.  And making createMachOObjectFile return Expected<...> .

So to to get the error to the llvm-obdump tool, I changed the stack of
these methods to also return Expected<...> :

  object::ObjectFile::createObjectFile()
  object::SymbolicFile::createSymbolicFile()
  object::createBinary()

Then finally in ParseInputMachO() in MachODump.cpp the error can
be reported and the specific error message can be printed in llvm-objdump
and can be seen in the existing test case for the existing malformed binary
but with the updated error message.

Converting these interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. To contain the changes for now use of
errorToErrorCode() and errorOrToExpected() are used where the callers
are yet to be converted.

Also there some were bugs in the existing code that did not deal with the
old ErrorOr<> return values.  So now with Expected<> since they must be
checked and the error handled, I added a TODO and a comment:
“// TODO: Actually report errors helpfully” and a call something like
consumeError(ObjOrErr.takeError()) so the buggy code will not crash
since needed to deal with the Error.

Note there is one fix also needed to lld/COFF/InputFiles.cpp that goes along
with this that I will commit right after this.  So expect lld not to built
after this commit and before the next one.

llvm-svn: 265606
2016-04-06 22:14:09 +00:00