Summary:
Fat object size has significantly increased after D65819 which changed bundler tool to add host object as a normal bundle to the fat output which almost doubled its size. That patch was fixing the following issues
1. Problems associated with the partial linking - global constructors were not called for partially linking objects which clearly resulted in incorrect behavior.
2. Eliminating "junk" target object sections from the linked binary on the host side.
The first problem is no longer relevant because we do not use partial linking for creating fat objects anymore. Target objects sections are now inserted into the resulting fat object with a help of llvm-objcopy tool.
The second issue, "junk" sections in the linked host binary, has been fixed in D73408 by adding "exclude" flag to the fat object's sections which contain target objects. This flag tells linker to drop section from the inputs when linking executable or shared library, therefore these sections will not be propagated in the linked binary.
Since both problems have been solved, we can revert D65819 changes to reduce fat object size and this patch essentially is doing that.
Reviewers: ABataev, alexshap, jdoerfert
Reviewed By: ABataev
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73642
Summary: This flag tells link editor to exclude section from linker inputs when linking executable or shared library.
Reviewers: ABataev, alexshap, jdoerfert
Reviewed By: ABataev
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73408
- Changed FileHandler read/write methods to return llvm::Error
- Using unified way of reporting errors
- Removed trailing '.' from the error messages
Differential Revision: https://reviews.llvm.org/D67031
- Changed FileHandler read/write methods to return llvm::Error
- Using unified way of reporting errors
- Removed trailing '.' from the error messages
Differential Revision: https://reviews.llvm.org/D67031
This reverts r371497 (git commit 3d7e9ab7b9)
Reorder `not` with `env` in these two tests so they pass:
Driver/rewrite-map-in-diagnostics.c
Index/crash-recovery-modules.m.
This will not be necessary after D66531 lands.
llvm-svn: 371552
I see in the history for some of these tests REQUIRES:shell was used as
a way to disable tests on Windows because they are flaky there. I tried
not to re-enable such tests, but it's possible that I missed some and
this will re-enable flaky tests on Windows. If so, we should disable
them with UNSUPPORTED:system-windows and add a comment that they are
flaky there. So far as I can tell, the lit internal shell is capable of
running all of these tests, and we shouldn't use REQUIRES:shell as a
proxy for Windows.
llvm-svn: 371478
Bundler currently requires host triple to be provided no matter if you are performing bundling or unbundling, but for unbundling operation such requirement is too restrictive. You may for example want to examine device part of the object for a particular offload target, but you have to extract host part as well even though you do not need it. Host triple isn't really needed for unbundling, so this patch removes that requirement.
Differential Revision: https://reviews.llvm.org/D66601
llvm-svn: 370143
clang-offload-bundler tool may hang under certain conditions when it extracts a subset of all available device bundles from the fat binary that is handled by the BinaryFileHandler. This patch fixes this problem.
Differential Revision: https://reviews.llvm.org/D66598
llvm-svn: 370115
clang-offload-bundler currently uses partial linking for creating fat object files, but such technique cannot be used on Windows due to the absence of partial linking support in the linker. This patch changes implementation to use llvm-objcopy for merging device and host objects instead of doing partial linking. This is one step forward towards enabling OpenMP offload on Windows.
Differential Revision: https://reviews.llvm.org/D66485
llvm-svn: 369955
Summary:
Previously, object files were bundled using partial linking. It resulted
in the following structure of the bundled objects:
```
<host_code>
clang-offload-bundle
__CLANG_OFFLOAD_BUNDLE__<target>
<target_code>
```
But when we tried to unbundle object files, it worked correctly only for
the target objects. The host object remains bundled. It produced a lot of
junk sections in the host object files and in some cases may caused
incorrect linking.
Patch improves bundling of the object files. After this patch the
bundled object looks like this:
```
<host_code>
clang-offload-bundle
__CLANG_OFFLOAD_BUNDLE__<target>
<target_code>
__CLANG_OFFLOAD_BUNDLE__<host>
<host_code>
```
With this structure we are able to unbundle the host object files too so
that after unbundling they are the same as were before.
The host section is bundled twice. The bundled section is used to
unbundle the original host section.
Reviewers: yaxunl, tra, jlebar, hfinkel, jdoerfert
Subscribers: caomhin, kkwli0, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65819
llvm-svn: 369019
Summary:
Make the test more portable and do not rely on the pre-bundled object
file.
Reviewers: Hahnfeld, hfinkel, jdoerfert
Subscribers: caomhin, kkwli0, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66296
llvm-svn: 369015
Summary:
By default, `parseCommandLineOptions()` will accept either a
`-` or `--` prefix for long options -- options with names longer than
a single character.
While this change does not affect behavior, it will be helpful with a
subsequent change that requires long options use the `--` prefix.
Reviewers: rnk, thopre
Reviewed By: thopre
Subscribers: thopre, cfe-commits, hiraditya, llvm-commits
Tags: #llvm, #clang
Differential Revision: https://reviews.llvm.org/D61269
llvm-svn: 359909
Summary:
This patch adds the capability to bundle object files in sections of the host binary using a designated naming convention for these sections. This patch uses the functionality of the object reader already in the LLVM library to read bundled files, and invokes clang with the incremental linking options to create bundle files.
Bundling files involves creating an IR file with the contents of the bundle assigned as initializers of globals binded to the designated sections. This way the bundling implementation is agnostic of the host object format.
The features added by this patch were requested in the RFC discussion in http://lists.llvm.org/pipermail/cfe-dev/2016-February/047547.html.
Reviewers: echristo, tra, jlebar, hfinkel, ABataev, Hahnfeld
Subscribers: mkuron, whchung, cfe-commits, andreybokhanko, Hahnfeld, arpith-jacob, carlo.bertolli, mehdi_amini, caomhin
Differential Revision: https://reviews.llvm.org/D21851
llvm-svn: 279634
Summary:
One of the goals of programming models that support offloading (e.g. OpenMP) is to enable users to offload with little effort, by annotating the code with a few pragmas. I'd also like to save users the trouble of changing their existent applications' build system. So having the compiler always return a single file instead of one for the host and each target even if the user is doing separate compilation is desirable.
This diff proposes a tool named clang-offload-bundler (happy to change the name if required) that is used to bundle files associated with the same user source file but different targets, or to unbundle a file into separate files associated with different targets.
This tool supports the driver support for OpenMP under review in http://reviews.llvm.org/D9888. The tool is used there to enable separate compilation, so that the very first action on input files that are not source files is a "unbundling action" and the very last non-linking action is a "bundling action".
The format of the bundled files is currently very simple: text formats are concatenated with comments that have a magic string and target identifying triple in between, and binary formats have a header that contains the triple and the offset and size of the code for host and each target.
The goal is to improve this tool in the future to deal with archive files so that each individual file in the archive is properly dealt with. We see that archives are very commonly used in current applications to combine separate compilation results. So I'm convinced users would enjoy this feature.
This tool can be used like this:
`clang-offload-bundler -targets=triple1,triple2 -type=ii -inputs=a.triple1.ii,a.triple2.ii -outputs=a.ii`
or
`clang-offload-bundler -targets=triple1,triple2 -type=ii -outputs=a.triple1.ii,a.triple2.ii -inputs=a.ii -unbundle`
I implemented the tool under clang/tools. Please let me know if something like this should live somewhere else.
This patch is prerequisite for http://reviews.llvm.org/D9888.
Reviewers: hfinkel, rsmith, echristo, chandlerc, tra, jlebar, ABataev, Hahnfeld
Subscribers: whchung, caomhin, andreybokhanko, arpith-jacob, carlo.bertolli, mehdi_amini, guansong, Hahnfeld, cfe-commits
Differential Revision: https://reviews.llvm.org/D13909
llvm-svn: 279632
Sergey Dmitriev