Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.
Solution:
This patch adds two new flags
- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.
Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.
So this is the behaviour per platform with my patch:
z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode
Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return
The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```
These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D99426
The contents of the string returned by getenv() is not guaranteed across calls to getenv(). The code to handle the CC_PRINT etc env vars calls getenv() and saves the results in just a char *. The string returned by getenv() needs to be copied and saved. Switching the type of the strings from char * to std::string will do this and manage the alloated memory.
Differential Revision: https://reviews.llvm.org/D98554
Summary:
Object of type `Compilation` now can keep a callback that is called
after each execution of `Command`. This must simplify adaptation of
clang in custom distributions and allow facilities like collection of
execution statistics.
Reviewers: rsmith, rjmccall, Eugene.Zelenko
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D78899
The argument after -Xarch_device will be added to the arguments for CUDA/HIP
device compilation and will be removed for host compilation.
The argument after -Xarch_host will be added to the arguments for CUDA/HIP
host compilation and will be removed for device compilation.
Differential Revision: https://reviews.llvm.org/D76520
This reverts commits f41ec709d9 and 5fedc2b410. On some buildbots, Clang :: Driver/crash-report.c is broken with:
```
Command Output (stderr):
--
/home/buildslave/ps4-buildslave1/clang-with-thin-lto-ubuntu/llvm-project/clang/test/Driver/crash-report.c:48:11: error: CHECK: expected string not found in input
// CHECK: Preprocessed source(s) and associated run script(s) are located at:
^
<stdin>:1:1: note: scanning from here
/home/buildslave/ps4-buildslave1/clang-with-thin-lto-ubuntu/llvm-project/clang/test/Driver/crash-report.c:50:1: error: unknown type name 'BAZ'
```
Example: http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/21321/steps/test-stage1-compiler/logs/stdio
Previously, when using '-MF file.d' on the command line, 'file.d' would not be deleted after a compiler crash.
The code path in Compilation::initCompilationForDiagnostics() that was modifying 'TranslatedArgs' had no effect, because 'TCArgs' was already created after the crash.
This was covered by clang/test/Driver/output-file-cleanup.c, the test was succeeding by fluke because Driver::generateCompilationDiagnostics() would fail to launch the subsequent clang -E (see D74070 for a fix for this). So the test was only covering Driver.cpp, C.CleanupFileMap().
After this patch, both cleanup and removal of -MF are exercised.
Differential Revision: https://reviews.llvm.org/D74076
The issue was reported by @xazax.hun here: https://reviews.llvm.org/D69825#1827826
"This patch (D69825) breaks scan-build-py which parses the output of "-###" to get -cc1 command. There might be other tools with the same problems. Could we either remove (in-process) from CC1Command::Print or add a line break?
Having the last line as a valid invocation is valuable and there might be tools relying on that."
Differential Revision: https://reviews.llvm.org/D72982
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
In rL327851 the createUniqueFile() and createTemporaryFile()
variants that do not return the file descriptors were changed to
create empty files, rather than only check if the paths are free.
This change was done in order to make the functions race-free.
That change led to clang-tidy (and possibly other tools) leaving
behind temporary assembly files, of the form placeholder-*, when
using a target that does not support the internal assembler.
The temporary files are created when building the Compilation
object in stripPositionalArgs(), as a part of creating the
compilation database for the arguments after the double-dash. The
files are created by Driver::GetNamedOutputPath().
Fix this issue by cleaning out temporary files at the deletion of
Compilation objects.
This fixes https://bugs.llvm.org/show_bug.cgi?id=37091.
Reviewers: klimek, sepavloff, arphaman, aaron.ballman, john.brawn, mehdi_amini, sammccall, bkramer, alexfh, JDevlieghere
Reviewed By: aaron.ballman, JDevlieghere
Subscribers: erichkeane, lebedev.ri, Ka-Ka, cfe-commits
Differential Revision: https://reviews.llvm.org/D45686
llvm-svn: 333637
There are quite differences in HIP action builder and action job creation,
which justifies to define a separate offload kind.
Differential Revision: https://reviews.llvm.org/D46471
llvm-svn: 331811
Summary:
This is basically reverting r261774 with a tweak for clang-cl. UNIX
standard states:
When c99 encounters a compilation error that causes an object file not
to be created, it shall write a diagnostic to standard error and
continue to compile other source code operands, but it shall not perform
the link phase and it shall return a non-zero exit status
The same goes for c89 or cc. And they are all alias or shims pointing to
clang on Darwin.
The original commit was intended for CUDA so the error message doesn't
get emit twice for both host and device. It seems that the clang driver
has been changed to model the CUDA dependency differently. Now the
driver behaves the same without this commit.
rdar://problem/32223263
Reviewers: thakis, dexonsmith, tra
Reviewed By: tra
Subscribers: jlebar, cfe-commits
Differential Revision: https://reviews.llvm.org/D39502
llvm-svn: 317860
The recent fix in D38258 was wrong: getAuxTriple() only returns
non-null values for the CUDA toolchain. That is why the now added
test for PPC and X86 failed.
Differential Revision: https://reviews.llvm.org/D38372
llvm-svn: 314902
ToolChain::TranslateArgs() returns nullptr if no changes are performed.
This would currently mean that OpenMPArgs are lost. Patch fixes this
by falling back to simply using OpenMPArgs in that case.
Differential Revision: https://reviews.llvm.org/D38259
llvm-svn: 314330
Parsing the argument after -Xopenmp-target allocates memory that needs
to be freed. Associate it with the final DerivedArgList after we know
which one will be used.
Differential Revision: https://reviews.llvm.org/D38257
llvm-svn: 314328
Summary:
OpenMP has the ability to offload target regions to devices which may have different architectures.
A new -fopenmp-target-arch flag is introduced to specify the device architecture.
In this patch I use the new flag to specify the compute capability of the underlying NVIDIA architecture for the OpenMP offloading CUDA tool chain.
Only a host-offloading test is provided since full device offloading capability will only be available when [[ https://reviews.llvm.org/D29654 | D29654 ]] lands.
Reviewers: hfinkel, Hahnfeld, carlo.bertolli, caomhin, ABataev
Reviewed By: hfinkel
Subscribers: guansong, cfe-commits
Tags: #openmp
Differential Revision: https://reviews.llvm.org/D34784
llvm-svn: 310263
This is a more principled version of r303756. That change was both very
brittle about the state of the Diags object going into the driver and
also broke tooling in funny ways.
In particular it prevented tools from capturing diagnostics properly and
made the compilation database logic fail to provide arguments to the
tool, falling back to scanning directories for JSON files.
llvm-svn: 306822
Summary:
This patch includes support for argument translation that is specific of a given offloading kind. Additionally, it implements the translation for OpenMP device kinds in the gcc tool chain.
With this patch, it is possible to compile a functional OpenMP application with offloading capabilities with no separate compilation.
Reviewers: echristo, tra, jlebar, rsmith, ABataev, hfinkel
Subscribers: whchung, mehdi_amini, cfe-commits, Hahnfeld, andreybokhanko, arpith-jacob, carlo.bertolli, caomhin
Differential Revision: https://reviews.llvm.org/D21848
llvm-svn: 285320
Summary:
Currently output of child process, however in my use case, it
needs to be captured and presented to the user.
Add Redirect method to Compilation and use existing infrastructure
for redirecting output of commands.
Reviewers: tstellarAMD
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D21224
llvm-svn: 273997
Summary:
This patch introduces the concept of offloading tool chain and offloading kind. Each tool chain may have associated an offloading kind that marks it as used in a given programming model that requires offloading.
It also adds the logic to iterate on the tool chains based on the kind. Currently, only CUDA is supported, but in general a programming model (an offloading kind) may have associated multiple tool chains that require supporting offloading.
This patch does not add tests - its goal is to keep the existing functionality.
This patch is the first of a series of three that attempts to make the current support of CUDA more generic and easier to extend to other programming models, namely OpenMP. It tries to capture the suggestions/improvements/concerns on the initial proposal in http://lists.llvm.org/pipermail/cfe-dev/2016-February/047547.html. It only tackles the more consensual part of the proposal, i.e.does not address the problem of intermediate files bundling yet.
Reviewers: ABataev, jlebar, echristo, hfinkel, tra
Subscribers: guansong, Hahnfeld, andreybokhanko, tcramer, mkuron, cfe-commits, arpith-jacob, carlo.bertolli, caomhin
Differential Revision: http://reviews.llvm.org/D18170
llvm-svn: 272571
Summary:
(Re-land of r260448, which was reverted in r260522 due to a test failure
in Driver/output-file-cleanup.c that only showed up in fresh builds.)
Previously we attempted to be smart; if one job failed, we'd run all
jobs that didn't depend on the failing job.
Problem is, this doesn't work well for e.g. CUDA compilation without
-save-temps. In this case, the device-side and host-side Assemble
actions (which actually are responsible for preprocess, compile,
backend, and assemble, since we're not saving temps) are necessarily
distinct. So our clever heuristic doesn't help us, and we repeat every
error message once for host and once for each device arch.
The main effect of this change, other than fixing CUDA, is that if you
pass multiple cc files to one instance of clang and you get a compile
error, we'll stop when the first cc1 job fails.
Reviewers: echristo
Subscribers: cfe-commits, jhen, echristo, tra, rafael
Differential Revision: http://reviews.llvm.org/D17217
llvm-svn: 261774
Also introduce inputs() that reutnrs an llvm::iterator_range.
Iterating over A->inputs() is much less mysterious than
iterating over *A. No intended behavior change.
llvm-svn: 261674
Previously we attempted to be smart; if one job failed, we'd run all
jobs that didn't depend on the failing job.
Problem is, this doesn't work well for e.g. CUDA compilation without
-save-temps. In this case, the device-side and host-side Assemble
actions (which actually are responsible for preprocess, compile,
backend, and assemble, since we're not saving temps) are necessarily
distinct. So our clever heuristic doesn't help us, and we repeat every
error message once for host and once for each device arch.
The main effect of this change, other than fixing CUDA, is that if you
pass multiple cc files to one instance of clang and you get a compile
error, we'll stop when the first cc1 job fails.
Reviewers: tra, echristo
Subscribers: jhen, cfe-commits
Differential Revision: http://reviews.llvm.org/D16514
llvm-svn: 260448
Summary:
This makes constructing Action graphs which are DAGs much simpler. It
also just simplifies in general the ownership semantics of Actions.
Depends on D15910.
Reviewers: echristo
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D15911
llvm-svn: 257407
Clang needs to know target triple for both sides of compilation so that
preprocessor macros and target builtins from both sides are available.
This change augments Compilation class to carry information about
toolchains used during different CUDA compilation passes and refactors
BuildActions to use it when it constructs CUDA jobs.
Removed DeviceTriple from CudaHostAction/CudaDeviceAction as it's no
longer needed.
Differential Revision: http://reviews.llvm.org/D13144
llvm-svn: 253385
We had a strange relationship here where we made a list of Jobs
inherit from a single Job, but there weren't actually any places where
this arbitrary nesting was used or needed.
Simplify all of this by removing Job entirely and updating all of the
users to either work with a JobList or a single Command.
llvm-svn: 241310
There's probably never a good reason to iterate over unique_ptrs. This
lets us use range-for and say Job.foo instead of (*it)->foo in a few
places.
llvm-svn: 218938
Patch by Rafael Auler!
This patch addresses PR15171 and teaches Clang how to call other tools
with response files, when the command line exceeds system limits. This
is a problem for Windows systems, whose maximum command-line length is
32kb.
I introduce the concept of "response file support" for each Tool object.
A given Tool may have full support for response files (e.g. MSVC's
link.exe) or only support file names inside response files, but no flags
(e.g. Apple's ld64, as commented in PR15171), or no support at all (the
default case). Therefore, if you implement a toolchain in the clang
driver and you want clang to be able to use response files in your
tools, you must override a method (getReponseFileSupport()) to tell so.
I designed it to support different kinds of tools and
internationalisation needs:
- VS response files ( UTF-16 )
- GNU tools ( uses system's current code page, windows' legacy intl.
support, with escaped backslashes. On unix, fallback to UTF-8 )
- Clang itself ( UTF-16 on windows, UTF-8 on unix )
- ld64 response files ( only a limited file list, UTF-8 on unix )
With this design, I was able to test input file names with spaces and
international characters for Windows. When the linker input is large
enough, it creates a response file with the correct encoding. On a Mac,
to test ld64, I temporarily changed Clang's behavior to always use
response files regardless of the command size limit (avoiding using huge
command line inputs). I tested clang with the LLVM test suite (compiling
benchmarks) and it did fine.
Test Plan: A LIT test that tests proper response files support. This is
tricky, since, for Unix systems, we need a 2MB response file, otherwise
Clang will simply use regular arguments instead of a response file. To
do this, my LIT test generate the file on the fly by cloning many -DTEST
parameters until we have a 2MB file. I found out that processing 2MB of
arguments is pretty slow, it takes 1 minute using my notebook in a debug
build, or 10s in a Release build. Therefore, I also added "REQUIRES:
long_tests", so it will only run when the user wants to run long tests.
In the full discussion in
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130408/171463.html,
Rafael Espindola discusses a proper way to test
llvm::sys::argumentsFitWithinSystemLimits(), and, there, Chandler
suggests to use 10 times the current system limit (20MB resp file), so
we guarantee that the system will always use response file, even if a
new linux comes up that can handle a few more bytes of arguments.
However, by testing with a 20MB resp file, the test takes long 8 minutes
just to perform a silly check to see if the driver will use a response
file. I found it to be unreasonable. Thus, I discarded this approach and
uses a 2MB response file, which should be enough.
Reviewers: asl, rafael, silvas
Reviewed By: silvas
Subscribers: silvas, rnk, thakis, cfe-commits
Differential Revision: http://reviews.llvm.org/D4897
llvm-svn: 217792
It's more flexible and arguably better layering to set flags to modify
compiling for diagnostics in the CC1 job themselves, rather than
tweaking the driver flags and letting them propagate.
There is one visible change this causes: crash report files will now
get preprocessed names (.i and friends).
llvm-svn: 211411
I think it makes sense that a Command knows how to execute itself.
There's no functionality change but i rewrote the code to avoid the manual
memory management of Argv.
My motivation for this is that I plan to subclass Command to build fall-back
functionality into clang-cl.
Differential Revision: http://llvm-reviews.chandlerc.com/D1654
llvm-svn: 190621
This moves the code to Job.cpp, which seems like a more natural fit,
and replaces the "is this a JobList? is this a Command?" logic with
a virtual function call.
It also removes the code duplication between PrintJob and
PrintDiagnosticJob and simplifies the code a little.
There's no functionality change here, except that the Executable is
now always printed within quotes, whereas it would previously not be
quoted in crash reports, which I think was a bug.
Differential Revision: http://llvm-reviews.chandlerc.com/D1653
llvm-svn: 190620
The way we decide which file to remove is fairly odd. I took a quick look at
maybe changing that, but it would be a more work than I want to put at this
right now, so I left pair of FIXMEs.
llvm-svn: 184766
Abhina Sreeskantharajan