Commit Graph

508 Commits

Author SHA1 Message Date
Alexey Lapshin abc7f6800d [Dsymutil][Debuginfo][NFC] Refactor dsymutil to separate DWARF optimizing part 2.
That patch is extracted from the D70709. It moves CompileUnit, DeclContext
into llvm/DebugInfo/DWARF. It also adds new file DWARFOptimizer with
AddressesMap class. AddressesMap generalizes functionality
from RelocationManager.

Differential Revision: https://reviews.llvm.org/D71271
2019-12-19 15:41:48 +03:00
Alexey Lapshin 8c714c9302 Fix building shared libraries broken by 8e48e8e3e32. 2019-12-06 16:48:41 +03:00
Alexey Lapshin 9e8c799e2b [Dsymutil][NFC] Move NonRelocatableStringpool into common CodeGen folder.
That refactoring moves NonRelocatableStringpool into common CodeGen folder.
So that NonRelocatableStringpool could be used not only inside dsymutil.

Differential Revision: https://reviews.llvm.org/D71068
2019-12-06 10:02:27 +03:00
Jonas Devlieghere 9b15873c92 [dsymutil] Remove recursion from lookForChildDIEsToKeep (2/2) (NFC)
The functions lookForDIEsToKeep and keepDIEAndDependencies are mutually
recursive and can cause a stackoverflow for large projects. While this
has always been the case, it became a bigger issue when we parallelized
dsymutil, because threads get only a fraction of the stack space.

This patch removes the final recursive call from lookForDIEsToKeep. The
call was used to look at the current DIE's parent chain and mark
everything as kept.

This was tested by running dsymutil on clang built in debug (both with
and without modules) and comparing the MD5 hash of the generated dSYM
companion file.

Differential revision: https://reviews.llvm.org/D70994
2019-12-04 10:20:53 -08:00
Jonas Devlieghere 95a8e8a255 [dsymutil] Remove recursion from lookForChildDIEsToKeep (1/2) (NFC)
The functions lookForDIEsToKeep and keepDIEAndDependencies are mutually
recursive and can cause a stackoverflow for large projects. While this
has always been the case, it became a bigger issue when we parallelized
dsymutil, because threads get only a fraction of the stack space.

In an attempt to tackle this issue, we removed part of the recursion in
r338536 by introducing a worklist. Processing of child DIEs was no
longer recursive. However, we still received bug reports where we'd run
out of stack space.

This patch removes another recursive call from lookForDIEsToKeep. The
call was used to look at DIEs that reference the current DIE. To make
this possible, we inlined keepDIEAndDependencies and added this work to
the existing worklist. Because the function is not tail recursive, we
needed to add two more types of worklist entries to perform the
subsequent work.

This was tested by running dsymutil on clang built in debug (both with
and without modules) and comparing the MD5 hash of the generated dSYM
companion file.

Differential revision: https://reviews.llvm.org/D70990
2019-12-04 10:20:52 -08:00
Jonas Devlieghere ed1756cfbe [dsymutil] Support --out (NFC)
Seems like this got lost during the libOption conversion.
2019-12-03 17:08:32 -08:00
Francis Visoiu Mistrih 1ee18f275e [dsymutil] Add support for linking remarks
This adds support to dsymutil for linking remark files and placing them
in the final .dSYM bundle.

The result will be placed in:

* a.out.dSYM/Contents/Resources/Remarks/a.out
or
* a.out.dSYM/Contents/Resources/Remarks/a.out-<arch> for universal binaries

When multi-threaded, this runs a third thread which loops over all the
object files and parses remarks as it finds __remarks sections.

Testing this involves running dsymutil on pre-built binaries and object
files, then running llvm-bcanalyzer on the final result to check for
remarks.

Differential Revision: https://reviews.llvm.org/D69142
2019-11-01 15:38:49 -07:00
Jonas Devlieghere 96bbf478b6 [dsymutil] Add DW_TAG_common_block to dieNeedsChildrenToBeMeaningful
Ensure we walk the children of common blocks when deciding what DIEs to
keep. Otherwise we might incorrectly discard them leading to missing
variables in the linked debug info.

This also sorts the list of DW_TAGs alphabetically.
2019-11-01 10:49:07 -07:00
Mirko Brkusanin 4b63ca1379 [Mips] Use appropriate private label prefix based on Mips ABI
MipsMCAsmInfo was using '$' prefix for Mips32 and '.L' for Mips64
regardless of -target-abi option. By passing MCTargetOptions to MCAsmInfo
we can find out Mips ABI and pick appropriate prefix.

Tags: #llvm, #clang, #lldb

Differential Revision: https://reviews.llvm.org/D66795
2019-10-23 12:24:35 +02:00
Jonas Devlieghere 8cdc842c51 [dsymutil] Print warning/error for unknown/missing arguments.
After changing dsymutil to use libOption, we lost error reporting for
missing required arguments (input files). Additionally, we stopped
complaining about unknown arguments. This patch fixes both and adds a
test.

llvm-svn: 375044
2019-10-16 21:48:41 +00:00
Jonas Devlieghere 20c692a445 [dsymutil] Support and relocate base address selection entries for debug_loc
Since r374600 clang emits base address selection entries. Currently
dsymutil does not support these entries and incorrectly interprets them
as location list entries.

This patch adds support for base address selection entries in dsymutil
and makes sure they are relocated correctly.

Thanks to Dave for coming up with the test case!

Differential revision: https://reviews.llvm.org/D69005

llvm-svn: 374957
2019-10-15 23:43:37 +00:00
Jonas Devlieghere 644abcf09d [dsymutil] Move CommonSymbols.clear() in resetParserState.
This seems like a more natural place to clear the vector, especially
since this method is clearing other data structures as well.

llvm-svn: 374378
2019-10-10 16:37:38 +00:00
Jonas Devlieghere e7affcdbd2 Re-land "[dsymutil] Fix handling of common symbols in multiple object files."
The original patch got reverted because it hit a long-standing legacy
issue on Windows that prevents files from being named `com`. Thanks
Kristina & Jeremy for pointing this out.

llvm-svn: 374178
2019-10-09 16:19:13 +00:00
Jeremy Morse e9c8f6fea6 Revert r374139, "[dsymutil] Fix handling of common symbols in multiple object files."
The added test files ("com", "com1.o", "com2.o") are reserved names on
Windows, and makes 'git checkout' fail with a filesystem error.

llvm-svn: 374144
2019-10-09 08:27:48 +00:00
Jonas Devlieghere 4ac388f7ca [dsymutil] Fix handling of common symbols in multiple object files.
For common symbols the linker emits only a single symbol entry in the
debug map. This caused dsymutil to not relocate common symbols when
linking DWARF coming form object files that did not have this entry.
This patch fixes that by keeping track of common symbols in the object
files and synthesizing a debug map entry for them using the address from
the main binary.

Differential revision: https://reviews.llvm.org/D68680

llvm-svn: 374139
2019-10-09 04:16:18 +00:00
Jonas Devlieghere a3f794e9b4 [dsymutil] Improve verbose output (NFC)
The verbose output for finding relocations assumed that we'd always dump
the DIE after (which starts with a newline) and therefore didn't include
one itself. However, this isn't always true, leading to garbled output.

This patch adds a newline to the verbose output and adds a line that
says that the DIE is being kept (which isn't obvious otherwise). It also
adds a 0x prefix to the relocations.

llvm-svn: 374123
2019-10-08 22:03:13 +00:00
Jonas Devlieghere 016f6f6a58 [dsymutil] Fix stack-use-after-scope
The lambda is taking the stack-allocated Verify boolean by reference and
it would go out of scope on the next iteration. Moving it out of the
loop should fix the issue.

Fixes https://bugs.llvm.org/show_bug.cgi?id=43549

llvm-svn: 373683
2019-10-04 00:39:48 +00:00
Jonas Devlieghere 085baf8d4e [dsymutil] Don't overload LinkOptions.
This should fix the build bots:

  error: declaration of ‘llvm::dsymutil::LinkOptions
  DsymutilOptions::LinkOptions’ [-fpermissive]

llvm-svn: 373640
2019-10-03 18:02:09 +00:00
Michael Liao 9c1503f669 Fix build failure with GCC on identifier reusing.
- GCC is different from clang and other compilers on that.
  https://godbolt.org/z/CeQE1V

llvm-svn: 373633
2019-10-03 17:47:46 +00:00
Jonas Devlieghere 2733bde663 [dsymutil] Improve consistency by removing redundant namespaces (NFC)
The dsymutil implementation file has a using-directive for the llvm
namespace. This patch just removes redundant namespace qualifiers.

llvm-svn: 373623
2019-10-03 16:34:44 +00:00
Jonas Devlieghere eddc1a4e95 [dsymutil] Tablegenify option parsing
This patch reimplements command line option parsing in dsymutil with
Tablegen and libOption. The main motivation for this change is to
prevent clashes with other cl::opt options defined in llvm. Although
it's a bit more heavyweight, it has some nice advantages such as no
global static initializers and better separation between the code and
the option definitions.

I also used this opportunity to improve how dsymutil deals with
incompatible options. Instead of having checks spread across the code,
everything is now grouped together in verifyOptions. The fact that the
options are no longer global means that we need to pass them around a
bit more, but I think it's worth the trade-off.

Differential revision: https://reviews.llvm.org/D68361

llvm-svn: 373622
2019-10-03 16:34:41 +00:00
Guillaume Chatelet 18f805a7ea [Alignment][NFC] Remove unneeded llvm:: scoping on Align types
llvm-svn: 373081
2019-09-27 12:54:21 +00:00
Guillaume Chatelet 3620263532 [Alignment] Introduce llvm::Align to MCSection
Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, JDevlieghere

Subscribers: arsenm, sdardis, jvesely, nhaehnle, sbc100, hiraditya, aheejin, jrtc27, atanasyan, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67486

llvm-svn: 371831
2019-09-13 09:29:59 +00:00
Guillaume Chatelet af11cc7eb5 [Alignment] Move OffsetToAlignment to Alignment.h
Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, JDevlieghere, alexshap, rupprecht, jhenderson

Subscribers: sdardis, nemanjai, hiraditya, kbarton, jakehehrlich, jrtc27, MaskRay, atanasyan, jsji, seiya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D67499

llvm-svn: 371742
2019-09-12 15:20:36 +00:00
Jonas Devlieghere 0eaee545ee [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013
2019-08-15 15:54:37 +00:00
George Rimar bcc00e1afb Recommit r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"
Changes: no changes. A fix for the clang code will be landed right on top.

Original commit message:

SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368826
2019-08-14 11:10:11 +00:00
George Rimar 468919e182 Revert r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"
It broke clang BB: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/16455

llvm-svn: 368813
2019-08-14 08:56:55 +00:00
George Rimar a0c6a35714 [llvm/Object] - Convert SectionRef::getName() to return Expected<>
SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368812
2019-08-14 08:46:54 +00:00
David Blaikie 90146cd8b9 DebugInfo/DWARF: Normalize DWARFObject members on the DWARF spec section names
Some of these names were abbreviated, some were not, some pluralised,
some not. Made the API difficult to use - since it's an exact 1:1
mapping to the DWARF sections - use those names (changing underscore
separation for camel casing).

llvm-svn: 368189
2019-08-07 17:18:11 +00:00
Igor Kudrin f26a70a5e7 Switch LLVM to use 64-bit offsets (2/5)
This updates all libraries and tools in LLVM Core to use 64-bit offsets
which directly or indirectly come to DataExtractor.

Differential Revision: https://reviews.llvm.org/D65638

llvm-svn: 368014
2019-08-06 10:49:40 +00:00
Fangrui Song d9b948b6eb Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC
F_{None,Text,Append} are kept for compatibility since r334221.

llvm-svn: 367800
2019-08-05 05:43:48 +00:00
Jonas Devlieghere f93d162e33 [dsymutil] Fix heap-use-after-free related to the LinkOptions.
In r367348, I changed dsymutil to pass the LinkOptions by value isntead
of by const reference. However, the options were still captured by
reference in the LinkLambda. This patch fixes that by passing them in by
value.

llvm-svn: 367635
2019-08-01 23:37:33 +00:00
Jonas Devlieghere 3a56174d85 [dsymutil] Pass LinkOptions by value instead of const ref.
When looping over the difference architectures in a fat binary, we
modify the link options before dispatching the link step to a different
thread. Passing the options by cont reference is not thread safe, as we
might modify its fields before the whole sturct is copied over.

Given that the link options are already stored in the DwarfLinker, we
can easily fix this by passing a copy of the link options instead of a
reference, which would just get copied later on.

llvm-svn: 367348
2019-07-30 19:34:26 +00:00
Hsiangkai Wang 18ccfadd46 [DebugInfo] Generate fixups as emitting DWARF .debug_frame/.eh_frame.
It is necessary to generate fixups in .debug_frame or .eh_frame as
relaxation is enabled due to the address delta may be changed after
relaxation.

There is an opcode with 6-bits data in debug frame encoding. So, we
also need 6-bits fixup types.

Differential Revision: https://reviews.llvm.org/D58335

llvm-svn: 366524
2019-07-19 02:03:34 +00:00
Hsiangkai Wang 657277e0f1 Revert "[DebugInfo] Generate fixups as emitting DWARF .debug_frame/.eh_frame."
This reverts commit 17e3cbf5fe656483d9016d0ba9e1d0cd8629379e.

llvm-svn: 366444
2019-07-18 15:06:50 +00:00
Hsiangkai Wang e43ce1a958 [DebugInfo] Generate fixups as emitting DWARF .debug_frame/.eh_frame.
It is necessary to generate fixups in .debug_frame or .eh_frame as
relaxation is enabled due to the address delta may be changed after
relaxation.

There is an opcode with 6-bits data in debug frame encoding. So, we
also need 6-bits fixup types.

Differential Revision: https://reviews.llvm.org/D58335

llvm-svn: 366442
2019-07-18 14:47:34 +00:00
Fangrui Song 78ee2fbf98 Cleanup: llvm::bsearch -> llvm::partition_point after r364719
llvm-svn: 364720
2019-06-30 11:19:56 +00:00
Fangrui Song dc8de6037c Simplify std::lower_bound with llvm::{bsearch,lower_bound}. NFC
llvm-svn: 364006
2019-06-21 05:40:31 +00:00
Jonas Devlieghere 60e52cab86 [dsymutil] Remove stale comment (NFC)
The comment was no longer relevant after r362621.

llvm-svn: 363008
2019-06-10 23:30:20 +00:00
Jonas Devlieghere bb8de35384 [dsymutil] Use the number of threads specified.
Before this patch we used either a single thread, or the number of
hardware threads available, effectively ignoring the number of threads
specified on the command line.

llvm-svn: 362815
2019-06-07 17:35:19 +00:00
Jonas Devlieghere 3027a2999c [dsymutil] Support more than 4 architectures
When running dsymutil on a fat binary, we use temporary files in a small
vector of size four. When processing more than 4 architectures, this
resulted in a user-after-move, because the temporary files got moved to
the heap. Instead of storing an optional temp file, we now use a unique
pointer, so the location of the actual temp file doesn't change.

We could test this by checking in 5 binaries for 5 different
architectures, but this seems wasteful, especially since the number of
elements in the small vector is arbitrary.

llvm-svn: 362621
2019-06-05 17:14:32 +00:00
Jonas Devlieghere 0da8160df3 [dwarfdump] Add flag to limit the number of parents DIEs
This adds `-parent-recurse-depth` which limits the number of parent DIEs
being dumped.

Differential revision: https://reviews.llvm.org/D62359

llvm-svn: 361671
2019-05-24 21:11:28 +00:00
Daniel Sanders 3e4acaabb9 Break false dependencies on target libraries
Summary:
For the most part this consists of replacing ${LLVM_TARGETS_TO_BUILD} with
some combination of AllTargets* so that they depend on specific components
of a target backend rather than all of it. The overall effect of this is
that, for example, tools like opt no longer falsely depend on the
disassembler, while tools like llvm-ar no longer depend on the code
generator.

There's a couple quirks to point out here:
* AllTargetsCodeGens is a bit more prevalent than expected. Tools like dsymutil
  seem to need it which I was surprised by.
* llvm-xray linked to all the backends but doesn't seem to need any of them.
  It builds and passes the tests so that seems to be correct.
* I left gold out as it's not built when binutils is not available so I'm
  unable to test it

Reviewers: bogner, JDevlieghere

Reviewed By: bogner

Subscribers: mehdi_amini, mgorny, steven_wu, dexonsmith, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62331

llvm-svn: 361567
2019-05-23 23:02:56 +00:00
Jonas Devlieghere b43dfaa6c0 [Dsymutil] Remove redundant argument (NFC)
The dwarf streamer already holds a copy of the link options, so there's
no need to pass them as an argument.

llvm-svn: 361276
2019-05-21 17:31:51 +00:00
Fangrui Song e183340c29 Recommit [Object] Change object::SectionRef::getContents() to return Expected<StringRef>
r360876 didn't fix 2 call sites in clang.

Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360892
2019-05-16 13:24:04 +00:00
Hans Wennborg 4da9ff9fcf Revert r360876 "[Object] Change object::SectionRef::getContents() to return Expected<StringRef>"
It broke the Clang build, see llvm-commits thread.

> Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.
>
> Follow-up of D61781.

llvm-svn: 360878
2019-05-16 12:08:34 +00:00
Fangrui Song a076ec54be [Object] Change object::SectionRef::getContents() to return Expected<StringRef>
Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360876
2019-05-16 11:33:48 +00:00
Adrian Prantl 2351d6102f [dsymutil] Put Swift interface files into a per-arch subdirectory.
This was meant to be part of the original commit r358921, but somehow
got lost.

<rdar://problem/49751748>

llvm-svn: 359010
2019-04-23 16:42:35 +00:00
Adrian Prantl 53bd7ce42e [dsymutil] Fix use-after-free when sys::path::append grows the buffer.
<rdar://problem/50117620>

llvm-svn: 359003
2019-04-23 15:44:22 +00:00
Adrian Prantl c7bde29cfe Revert "[dsymutil] Fix use-after-free when sys::path::append grows the buffer."
llvm-svn: 359002
2019-04-23 15:44:19 +00:00
Adrian Prantl 03e906d9d5 [dsymutil] Fix use-after-free when sys::path::append grows the buffer.
<rdar://problem/50117620>

llvm-svn: 359001
2019-04-23 15:39:13 +00:00
Adrian Prantl 05aad1567b Fully qualify llvm::Optional, some compilers complain otherwise.
llvm-svn: 358933
2019-04-22 22:51:34 +00:00
Adrian Prantl e495ec23fe Try to work around compile errors with older versions of GCC.
llvm-svn: 358927
2019-04-22 22:40:37 +00:00
Adrian Prantl 0d809aa218 [dsymutil] Collect parseable Swift interfaces in the .dSYM bundle.
When a Swift module built with debug info imports a library without
debug info from a textual interface, the textual interface is
necessary to reconstruct types defined in the library's interface. By
recording the Swift interface files in DWARF dsymutil can collect them
and LLDB can find them.

This patch teaches dsymutil to look for DW_TAG_imported_modules and
records all references to parseable Swift ingterfrace files and copies
them to

  a.out.dSYM/Contents/Resources/<Arch>/<ModuleName>.swiftinterface

<rdar://problem/49751748>

llvm-svn: 358921
2019-04-22 21:33:22 +00:00
Fangrui Song 51873d3150 [dsymutil] DwarfLinker: delete unused parameter
llvm-svn: 358762
2019-04-19 15:45:25 +00:00
Vlad Tsyrklevich 53a9f1d367 Revert "[DWARFFormValue] Cleanup DWARFFormValue interface. (2/2) (NFC)"
This reverts commit r355233, it was causing UBSan failures.

llvm-svn: 355255
2019-03-02 01:10:00 +00:00
Jonas Devlieghere 2dc2baa8cc [DWARFFormValue] Cleanup DWARFFormValue interface. (2/2) (NFC)
Continues the work started in r354941. Changes (all but one) uses of the
extractValue to static createFromData.

llvm-svn: 355233
2019-03-01 22:14:24 +00:00
Adrian Prantl fa37a00044 dsymutil support for DW_OP_convert
Add support for cloning DWARF expressions that contain base type DIE
references in dsymutil.

<rdar://problem/48167812>

Differential Revision: https://reviews.llvm.org/D58534

llvm-svn: 355148
2019-02-28 22:12:32 +00:00
Jonas Devlieghere f9b0b9a504 [dsymutil] Use rfind for paths with parentheses
Dsymutil gets library member information is through the ambiguous
/path/to/archive.a(member.o). The current logic we use would get
confused by additional parentheses. Using rfind mitigates this issue.

llvm-svn: 355114
2019-02-28 18:46:04 +00:00
Alexey Lapshin 77fc1f6049 [DebugInfo] add SectionedAddress to DebugInfo interfaces.
That patch is the fix for https://bugs.llvm.org/show_bug.cgi?id=40703
   "wrong line number info for obj file compiled with -ffunction-sections"
   bug. The problem happened with only .o files. If object file contains
   several .text sections then line number information showed incorrectly.
   The reason for this is that DwarfLineTable could not detect section which
   corresponds to specified address(because address is the local to the
   section). And as the result it could not select proper sequence in the
   line table. The fix is to pass SectionIndex with the address. So that it
   would be possible to differentiate addresses from various sections. With
   this fix llvm-objdump shows correct line numbers for disassembled code.

   Differential review: https://reviews.llvm.org/D58194

llvm-svn: 354972
2019-02-27 13:17:36 +00:00
Jonas Devlieghere 9ea90acfeb [dsymutil] Improve readability of cloneAllCompileUnits (NFC)
Add some newlines and improve consistency between the two loops.

llvm-svn: 353904
2019-02-13 00:32:21 +00:00
Jonas Devlieghere 1bf1b9857f [dsymutil] Don't clone empty CUs
The DWARF standard says that an empty compile unit is not valid:

> Each such contribution consists of a compilation unit header (see
> Section 7.5.1.1 on page 200) followed by a single DW_TAG_compile_unit or
> DW_TAG_partial_unit debugging information entry, together with its
> children.

Therefore we shouldn't clone them in dsymutil.

Differential revision: https://reviews.llvm.org/D57979

llvm-svn: 353903
2019-02-13 00:32:06 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Jonas Devlieghere 40c63ed397 [dsymutil] Fix assertion triggered by empty address range.
An assertion was hit when running dsymutil on a gcc generated binary
that contained an empty address range. Address ranges are stored in an
interval map of half open intervals. Since the interval is empty and
therefore meaningless, we simply don't add it to the map.

llvm-svn: 350591
2019-01-08 01:08:09 +00:00
Jonas Devlieghere 91b43adb69 [dsymutil] Upstream unobfuscation logic.
The unobufscation support for BCSymbolMaps was the last piece of code
that hasn't been upstreamed yet. This patch contains a reworked version
of the existing code and relevant tests.

Differential revision: https://reviews.llvm.org/D56346

llvm-svn: 350580
2019-01-07 23:27:25 +00:00
Jonas Devlieghere e35f48f622 [dsymutil] Ensure we're comparing time stamps with the same precision.
After TimePoint's precision was increased in LLVM we started seeing
failures because the modification times didn't match. This adds a time
cast to ensure that we're comparing TimePoints with the same amount of
precision.

llvm-svn: 348283
2018-12-04 17:15:23 +00:00
Jonas Devlieghere d1c9751657 [dsymutil] Gather global and local symbol addresses in the main executable.
Usually local symbols will have their address described in the debug
map. Global symbols have to have their address looked up in the symbol
table of the main executable. By playing with 'ld -r' and export lists,
you can get a symbol described as global by the debug map while actually
being a local symbol as far as the link in concerned. By gathering the
address of local symbols, we fix this issue.

Also, we prefer a global symbol in case of a name collision to preserve
the previous behavior.

Note that using the 'ld -r' tricks, people can actually cause symbol
names collisions that dsymutil has no way to figure out. This fixes the
simple case where there is only one symbol of a given name.

rdar://problem/32826621

Differential revision: https://reviews.llvm.org/D54922

llvm-svn: 348021
2018-11-30 18:56:10 +00:00
Adrian Prantl a77eae96bf Make dsymutil more robust when parsing load commands.
rdar://problem/45883463

llvm-svn: 346815
2018-11-13 23:31:25 +00:00
Adrian Prantl 778fba3188 [dsymutil] Copy the LC_BUILD_VERSION load command into the companion binary.
LC_BUILD_VERSION contains platform information that is useful for LLDB
to match up dSYM bundles with binaries. This patch copies the load
command over into the dSYM.

rdar://problem/44145175
rdar://problem/45883463

Differential Revision: https://reviews.llvm.org/D54233

llvm-svn: 346412
2018-11-08 16:54:59 +00:00
Wolfgang Pieb f39a9bbe72 [DWARF] Revert r345546: Refactor range list extraction and dumping
This patch caused some internal tests to break which are being investigated.

llvm-svn: 345687
2018-10-31 01:12:58 +00:00
Wolfgang Pieb fb6cffca09 [DWARF][NFC] Refactor range list extraction and dumping
The purpose of this patch is twofold: 
- Fold pre-DWARF v5 functionality into v5 to eliminate the need for 2 different 
  versions of range list handling. We get rid of DWARFDebugRangelist{.cpp,.h}.
- Templatize the handling of range list tables so that location list handling
  can take advantage of it as well. Location list and range list tables have the 
  same basic layout.

A non-NFC version of this patch was previously submitted with r342218, but it caused
errors with some TSan tests. This patch has no functional changes. The difference to
the non-NFC patch is that there are no changes to rangelist dumping in this patch.

Differential Revision: https://reviews.llvm.org/D53545

llvm-svn: 345546
2018-10-29 22:16:47 +00:00
Jonas Devlieghere 18028f9d43 [dsymutil] Improve error reporting when we cannot create output file.
Before this patch we were returning an empty string in case we couldn't
create the output file. Now we return an expected string so we can
return and print the proper issue. We now return errors instead of bools
and defer printing to the call site.

llvm-svn: 344983
2018-10-23 00:32:22 +00:00
Fangrui Song 3507c6e884 Use the container form llvm::sort(C, ...)
There are a few leftovers in rL343163 which span two lines. This commit
changes these llvm::sort(C.begin(), C.end, ...) to llvm::sort(C, ...)

llvm-svn: 343426
2018-09-30 22:31:29 +00:00
Fangrui Song 0cac726a00 llvm::sort(C.begin(), C.end(), ...) -> llvm::sort(C, ...)
Summary: The convenience wrapper in STLExtras is available since rL342102.

Reviewers: dblaikie, javed.absar, JDevlieghere, andreadb

Subscribers: MatzeB, sanjoy, arsenm, dschuff, mehdi_amini, sdardis, nemanjai, jvesely, nhaehnle, sbc100, jgravelle-google, eraman, aheejin, kbarton, JDevlieghere, javed.absar, gbedwell, jrtc27, mgrang, atanasyan, steven_wu, george.burgess.iv, dexonsmith, kristina, jsji, llvm-commits

Differential Revision: https://reviews.llvm.org/D52573

llvm-svn: 343163
2018-09-27 02:13:45 +00:00
Alexander Kornienko e74e0f11d1 Revert "[DWARF] reposting r342048, which was reverted in r342056 due to buildbot errors. Adjusted 2 test cases for ARM and darwin and fixed a bug with the original change in dsymutil."
This reverts commit r342218. Due to a number of failures under TSAN. An isolated
test case is being worked on.

llvm-svn: 342399
2018-09-17 15:40:01 +00:00
Wolfgang Pieb 55dbac9f07 [DWARF] reposting r342048, which was reverted in r342056 due to buildbot
errors.
Adjusted 2 test cases for ARM and darwin and fixed a bug with the original
change in dsymutil.

llvm-svn: 342218
2018-09-14 09:14:10 +00:00
Wolfgang Pieb 233bc73047 Reverting r342048, which caused UBSan failures in dsymutil.
llvm-svn: 342056
2018-09-12 14:40:04 +00:00
Wolfgang Pieb 3a8781cf6c [DWARF] Refactoring range list dumping to fold DWARF v4 functionality into v5 handling
Eliminating some duplication of rangelist dumping code at the expense of
some version-dependent code in dump and extract routines.

Reviewer: dblaikie, JDevlieghere, vleschuk

Differential revision: https://reviews.llvm.org/D51081

llvm-svn: 342048
2018-09-12 12:01:19 +00:00
Jonas Devlieghere 475ce5a26c [dsymutil] Prevent non-determinism due to threading.
Before this patch, analyzeContext called getCanonicalDIEOffset(), for
which the result depends on the timings of the setCanonicalDIEOffset()
calls in the cloneLambda. This can lead to slightly different output
between runs due to threading.

To prevent this from happening, we now record the output debug info size
after importing the modules (before any concurrent processing takes
place). This value, named the ModulesEndOffset is used to compare the
canonical DIE offset against. If the value is greater than this offset,
the canonical DIE offset has been updated during cloning, and should
therefore not be considered for pruning.

Differential revision: https://reviews.llvm.org/D51443

llvm-svn: 341649
2018-09-07 10:29:22 +00:00
Jonas Devlieghere 5e322f5125 [dsymutil] Reduce peak memory usage for the single threaded execution.
Keeping the compile units in memory is expensive. For the single
threaded case we allocate them in the analyze part and deallocate them
again once we've finished cloning. This poses a problem in the single
threaded case where we did all the analysis first followed by all the
cloning. This meant we had all the link context in memory right after
analyzing finished.

This patch changes the way we order work in the single threaded case.
Instead of doing all the analysis and cloning in serial, we now
interleave the two so we can deallocate the memory as soon as a file is
processed. The result is binary identical and peak memory usage went
down from 13.43GB to 5.73GB for a debug build of trunk clang.

Differential revision: https://reviews.llvm.org/D51618

llvm-svn: 341568
2018-09-06 17:31:59 +00:00
Argyrios Kyrtzidis c30340b207 Add header guards to some headers that are missing them
Also adjust some of dsymutil's headers to put the header guards at the top,
otherwise the compiler will not recognize them as header guards.

llvm-svn: 341323
2018-09-03 16:22:05 +00:00
Adrian Prantl bdffea12d0 dsymutil: Avoid pruning non-type forward declarations inside DW_TAG_module
forward declarations.

Especially with template instantiations, there are legitimate reasons
why for declarations might be emitted into a DW_TAG_module skeleton /
forward-declaration sub-tree, that are not forward declarations in the
sense of that there is a more complete definition over in a .pcm file.

The example in the testcase is a constant DW_TAG_member of a
DW_TAG_class template instatiation.

rdar://problem/43623196

llvm-svn: 341123
2018-08-30 21:21:16 +00:00
Adrian Prantl 29729f1529 Reduce the memory footprint of dsymutil. (NFC)
This (partially) fixes a regression introduced by
https://reviews.llvm.org/D43945 / r327399, which parallelized
DwarfLinker. This patch avoids parsing and allocating the memory for
all input DIEs up front and instead only allocates them in the
concurrent loop in the AnalyzeLambda. At the end of the loop the
memory from the LinkContext is cleared again.

This reduces the peak memory needed to link the debug info of a
non-modular build of the Swift compiler by >3GB.

rdar://problem/43444464

Differential Revision: https://reviews.llvm.org/D51078

llvm-svn: 340650
2018-08-24 20:41:08 +00:00
Victor Leschuk cf1f714d3b [DWARF] Unify warning callbacks. NFC.
Both DWARFDebugLine and DWARFDebugAddr used the same callback mechanism
for handling recoverable errors. They both implemented similar warn() function
to be used as such callbacks.

In this revision we get rid of code duplication and move this warn() function
to DWARFContext as DWARFContext::dumpWarning().

Reviewers: lhames, jhenderson, aprantl, probinson, dblaikie, JDevlieghere

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D51033

llvm-svn: 340528
2018-08-23 12:43:33 +00:00
Pavel Labath 2f0881160c [DebugInfo] Reduce debug_str_offsets section size
Summary:
The accelerator tables use the debug_str section to store their strings.
However, they do not support the indirect method of access that is
available for the debug_info section (DW_FORM_strx et al.).

Currently our code is assuming that all strings can/will be referenced
indirectly, and puts all of them into the debug_str_offsets section.
This is generally true for regular (unsplit) dwarf, but in the DWO case,
most of the strings in the debug_str section will only be used from the
accelerator tables. Therefore the contents of the debug_str_offsets
section will be largely unused and bloating the main executable.

This patch rectifies this by teaching the DwarfStringPool to
differentiate between strings accessed directly and indirectly. When a
user inserts a string into the pool it has to declare whether that
string will be referenced directly or not. If at least one user requsts
indirect access, that string will be assigned an index ID and put into
debug_str_offsets table. Otherwise, the offset table is skipped.

This approach reduces the overall binary size (when compiled with
-gdwarf-5 -gsplit-dwarf) in my tests by about 2% (debug_str_offsets is
shrunk by 99%).

Reviewers: probinson, dblaikie, JDevlieghere

Subscribers: aprantl, mgrang, llvm-commits

Differential Revision: https://reviews.llvm.org/D49493

llvm-svn: 339122
2018-08-07 09:54:52 +00:00
Jonas Devlieghere 5361767210 [dsymutil] Convert recursion in lookForDIEsToKeep into worklist.
The functions `lookForDIEsToKeep` and `keepDIEAndDependencies` can have
some very deep recursion. This tackles part of this problem by removing
the recursion from `lookForDIEsToKeep` by turning it into a worklist.

The difficulty in doing so is the computation of incompleteness, which
depends on the incompleteness of its children. To compute this, we
insert "continuation markers" into the worklist. This informs the work
loop to (re)compute the incompleteness property of the DIE associated
with it (i.e. the parent of the previously processed DIE).

This patch should generate byte-identical output. Unfortunately it also
has some impact of performance, regressing by about 4% when processing
clang on my machine.

Differential revision: https://reviews.llvm.org/D48899

llvm-svn: 338536
2018-08-01 13:24:39 +00:00
Jonas Devlieghere ae1727e3dd [dsymutil] Simplify temporary file handling.
Dsymutil's update functionality was broken on Windows because we tried
to rename a file while we're holding open handles to that file. TempFile
provides a solution for this through its keep(Twine) method. This patch
changes dsymutil to make use of that functionality.

Differential revision: https://reviews.llvm.org/D49860

llvm-svn: 338216
2018-07-29 14:56:15 +00:00
Jonas Devlieghere 743d351120 [dsymutil] Add support for generating DWARF5 accelerator tables.
This patch add support for emitting DWARF5 accelerator tables
(.debug_names) from dsymutil. Just as with the Apple style accelerator
tables, it's possible to update existing dSYMs. This patch includes a
test that show how you can convert back and forth between the two types.

If no kind of table is specified, dsymutil will default to generating
Apple-style accelerator tables whenever it finds those in its input. The
same is true when there are no accelerator tables at all. Finally, in
the remaining case, where there's at least one DWARF v5 table and no
Apple ones, the output will contains a DWARF accelerator tables
(.debug_names).

Differential revision: https://reviews.llvm.org/D49137

llvm-svn: 337980
2018-07-25 23:01:38 +00:00
Jonas Devlieghere e13e6dbe40 [MC] Add interface to finish pending labels.
When manually finishing the object writer in dsymutil, it's possible
that there are pending labels that haven't been resolved. This results
in an assertion when the assembler tries to fixup a label that doesn't
have an address yet.

Differential revision: https://reviews.llvm.org/D49131

llvm-svn: 336688
2018-07-10 15:32:17 +00:00
Jonas Devlieghere 82dee6aca8 [dsymutil] Add support for outputting assembly
When implementing the DWARF accelerator tables in dsymutil I ran into an
assertion in the assembler. Debugging these kind of issues is a lot
easier when looking at the assembly instead of debugging the assembler
itself. Since it's only a matter of creating an AsmStreamer instead of a
MCObjectStreamer it made sense to turn this into a (hidden) dsymutil
feature.

Differential revision: https://reviews.llvm.org/D49079

llvm-svn: 336561
2018-07-09 16:58:48 +00:00
Jonas Devlieghere 7f19d0160b [dsymutil] Emit label at the begin of a CU
When emitting a CU, store the MCSymbol pointing to the beginning of the
CU. We'll need this information later when emitting the .debug_names
section (DWARF5 accelerator table).

llvm-svn: 336433
2018-07-06 12:49:54 +00:00
Jonas Devlieghere 8eeece38ef [dsymutil] Rename conflicting declaration
Using MemoryBuffer as member name clashed with the llvm::MemoryBuffer
class.

llvm-svn: 335995
2018-06-29 17:11:34 +00:00
Jonas Devlieghere a0857eaefe [dsymutil] Make the CachedBinaryHolder the default
Replaces all uses of the old binary holder with its cached variant.

Differential revision: https://reviews.llvm.org/D48770

llvm-svn: 335991
2018-06-29 16:51:52 +00:00
Jonas Devlieghere 3ff7915c33 [dsymutil] Introduce a new CachedBinaryHolder
The original binary holder has an optimization where it caches a static
library (archive) between consecutive calls to GetObjects. However, the
actual memory buffer wasn't cached between calls.

This made sense when dsymutil was processing objects one after each
other, but when processing them in parallel, several binaries have to be
in memory at the same time. For this reason, every link context
contained a binary holder.

Having one binary holder per context is problematic, because the same
static archive was cached for every object file. Luckily, when the file
is mmap'ed, this was only costing us virtual memory.

This patch introduces a new BinaryHolder variant that is fully cached,
for all the object files it load, as well as the static archives. This
way, we don't have to give up on this optimization of bypassing the
file system.

Differential revision: https://reviews.llvm.org/D48501

llvm-svn: 335990
2018-06-29 16:50:41 +00:00
Jonas Devlieghere 35bb57b202 [dsymutil] Use UnitListTy consistently (NFC)
Use the UnitListTy typedef consistently throughout the Dwarf linker and
pass it by const reference where possible.

llvm-svn: 335860
2018-06-28 15:01:42 +00:00
Jonas Devlieghere c0fb4b6b88 [dsymutil] Move abstractions into separate files (NFC)
This patch splits off some abstractions used by dsymutil's dwarf linker
and moves them into separate header and implementation files. This
almost halves the number of LOC in DwarfLinker.cpp and makes it a lot
easier to understand what functionality lives where.

Differential revision: https://reviews.llvm.org/D48647

llvm-svn: 335749
2018-06-27 16:13:40 +00:00
Jonas Devlieghere 0bad3f625e [dsymutil] Force mmap'ing of binaries
After the recent refactoring that introduced parallel handling of
different object, the binary holder became unique per object file. This
defeats its optimization of caching archives, leading to an archive
being opened for every binary it contains. This is obviously unfortunate
and will need to be refactored soon.

Luckily in practice, the impact of this is limited as most files are
mmap'ed instead of memcopy'd. There's a caveat however: when the memory
buffer requires a null terminator and it's a multiple of the page size,
we allocate instead of mmap'ing. If this happens for a static archive,
we end up with N copies of it in memory, where N is the number of
objects in the archive, leading to exuberant memory usage. This provided
a stopgap solution to ensure that all the files it loads are mmap in
memory by removing the requirement for a terminating null byte.

Differential revision: https://reviews.llvm.org/D48397

llvm-svn: 335293
2018-06-21 21:37:53 +00:00
Wolfgang Pieb 61d8c8d9b3 [DWARF] Improved error reporting for range lists.
Errors found processing the DW_AT_ranges attribute are propagated by lower level 
routines and reported by their callers.

Reviewer: JDevlieghere

Differential Revision: https://reviews.llvm.org/D48344

llvm-svn: 335188
2018-06-20 22:56:37 +00:00
Zachary Turner 08426e1f9f Refactor ExecuteAndWait to take StringRefs.
This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.

In the end, I think this makes for a more idiomatic and platform
agnostic API.  Not all platforms launch process with null terminated
c-string arrays for the environment pointer and argv, but the api
was designed that way because it allowed easy pass-through for
posix-based platforms.  There's a little additional overhead now
since on posix based platforms we'll be takign StringRefs which
were constructed from null terminated strings and then copying
them to null terminate them again, but from a readability and
usability standpoint of the API user, I think this API signature
is strictly better.

llvm-svn: 334518
2018-06-12 17:43:52 +00:00
Jonas Devlieghere 745918ff87 [ADT] Make escaping fn conform to coding guidelines
As noted by Adrian on llvm-commits, PrintHTMLEscaped and PrintEscaped in
StringExtras did not conform to the LLVM coding guidelines. This commit
rectifies that.

llvm-svn: 333669
2018-05-31 17:01:42 +00:00