Summary:
We've seen cases of bots failing with:
clang: error: unable to execute command: posix_spawn failed: Interrupted system call
Add a small retry loop to posix_spawn in case this happens. Don't retry too much in case there's some systemic problem going on, but retry a few times.
<rdar://problem/50181448>
Reviewers: Bigcheese, arphaman
Subscribers: jkorous, dexonsmith, kristina, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61096
llvm-svn: 359152
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.
In the end, I think this makes for a more idiomatic and platform
agnostic API. Not all platforms launch process with null terminated
c-string arrays for the environment pointer and argv, but the api
was designed that way because it allowed easy pass-through for
posix-based platforms. There's a little additional overhead now
since on posix based platforms we'll be takign StringRefs which
were constructed from null terminated strings and then copying
them to null terminate them again, but from a readability and
usability standpoint of the API user, I think this API signature
is strictly better.
llvm-svn: 334518
I took some liberties and quoted fewer characters than before,
based on an article from MSDN which says that only certain characters
cause an arg to require quoting. This seems to be incorrect, though,
and worse it seems to be a difference in Windows version. The bot
that fails is Windows 7, and I can't reproduce the failure on Win
10. But it's definitely related to quoting and special characters,
because both tests that fail have a * in the argument, which is one
of the special characters that would cause an argument to be quoted
before but not any longer after the new patch.
Since I don't have Win 7, all I can do is just guess that I need to
restore the old quoting rules. So this patch does that in hopes that
it fixes the problem on Windows 7.
llvm-svn: 334375
This reverts commit 65243b6d19143cb7a03f68df0169dcb63e8b4632.
Seems like it's not a flake. It might have something to do with
the '*' character being in a command line.
llvm-svn: 334356
There were a few linux compilation failures, but other than that
I think this was just a flake that caused the tests to fail. I'm
going to resubmit and see if the failures go away, if not I'll
revert again.
llvm-svn: 334355
Summary:
The function `llvm::sys::commandLineFitsWithinSystemLimits` appears to be overestimating the system limits. This issue was discovered while attempting to enable response files in the Swift compiler. When the compiler submits its frontend jobs, those jobs are subjected to the system limits on command line length. `commandLineFitsWithinSystemLimits` is used to determine if the job's arguments need to be wrapped in a response file. There are some cases where the argument size for the job passes `commandLineFitsWithinSystemLimits`, but actually exceeds the real system limit, and the job fails.
`clang` also uses this function to decide whether or not to wrap it's job arguments in response files. See: https://github.com/llvm-mirror/clang/blob/master/lib/Driver/Driver.cpp#L1341. Clang will also fail for response files who's size falls within a certain range. I wrote a script that should find a failure point for `clang++`. All that is needed to run it is Python 2.7, and a simple "hello world" program for `test.cc`. It should run on Linux and on macOS. The script is available here: https://gist.github.com/dabelknap/71bd083cd06b91c5b3cef6a7f4d3d427. When it hits a failure point, you should see a `clang: error: unable to execute command: posix_spawn failed: Argument list too long`.
The proposed solution is to mirror the behavior of `xargs` in `commandLinefitsWithinSystemLimits`. `xargs` defaults to 128k for the command line length size (See: https://fossies.org/dox/findutils-4.6.0/buildcmd_8c_source.html#l00551). It adjusts this depending on the value of `ARG_MAX`.
Reviewers: alexfh
Reviewed By: alexfh
Subscribers: llvm-commits
Tags: #clang
Patch by Austin Belknap!
Differential Revision: https://reviews.llvm.org/D47795
llvm-svn: 334295
NFC here, this just raises some platform specific ifdef hackery
out of a class and creates proper platform-independent typedefs
for the relevant things. This allows these typedefs to be
reused in other places without having to reinvent this preprocessor
logic.
llvm-svn: 334294
Summary:
Change the type of the Redirects parameter of llvm::sys::ExecuteAndWait,
ExecuteNoWait and other APIs that wrap them from `const StringRef **` to
`ArrayRef<Optional<StringRef>>`, which is safer and simplifies the use of these
APIs (no more local StringRef variables just to get a pointer to).
Corresponding clang changes will be posted as a separate patch.
Reviewers: bkramer
Reviewed By: bkramer
Subscribers: vsk, llvm-commits
Differential Revision: https://reviews.llvm.org/D37563
llvm-svn: 313155
Summary:
(re)definition of _RESTRICT_KYWD rightfully causes a warning message during the Solaris build.
This hack is not needed if build compiler is properly configured (.e.g /usr/bin/gcc) so just remove it.
Reviewers: ro, mgorny, krytarowski, joerg
Reviewed By: joerg
Subscribers: quenelle, llvm-commits
Patch by Fedor Sergeev (Oracle).
Differential Revision: https://reviews.llvm.org/D35054
llvm-svn: 307469
This is a workaround for large file writes. It has been witnessed that
write(2) failing with EINVAL (22) due to a large value (>2G). Thanks to
James Knight for the help with coming up with a sane test case.
llvm-svn: 305846
It doesn't seem relevant to set an address space limit - this isn't
important in any sense that I'm aware & it gets in the way of things
that use a lot of address space, like llvm-symbolizer.
This came up when I realized that bugpoint regression tests were much
slower with -gsplit-dwarf than plain -g. Turned out that bugpoint
subprocesses (opt, etc) were crashing and doing symbolization - but
bugpoint runs those subprocesses with a 400MB memory limit. So with
plain -g, mmaping the opt binary would exceed the memory limit, fail,
and thus be really fast - no symbolization occurred. Whereas with
-gsplit-dwarf, comically, having less to map in, it would succeed and
then spend lots of time symbolizing.
I've fixed at least the critical part of bugpoint's perf problem there
by adding an option to allow bugpoint to disable symbolization. Thus
improving the perfromance for -gsplit-dwarf and making the -g-esque
speed available without this quirk/accidental benefit.
llvm-svn: 305242
Some Include What You Use suggestions were used too.
Use anonymous namespaces in source files.
Differential revision: http://reviews.llvm.org/D18778
llvm-svn: 265454
Summary:
Hi Rafael,
Would you be able to review this patch, please?
(Clang part of the patch is D15832).
When clang runs an external tool, e.g. a linker, it may create a command line that exceeds the length limit.
Clang uses the llvm::sys::argumentsFitWithinSystemLimits function to check if command line length fits the OS
limitation. There are two problems in this function that may cause exceeding of the limit:
1. It ignores the length of the program path in its calculations. On the other hand, clang adds the program
path to the command line when it runs the program.
2. It assumes no space character is inserted after the last argument, which is not true for Windows. The flattenArgs function adds the trailing space for *each* argument. The result of this is that the terminating NULL character is not counted and may be placed beyond the length limit if the command line is exactly 32768 characters long. The WinAPI's CreateProcess does not find the NULL character and fails.
Reviewers: rafael, ygao, probinson
Subscribers: asl, llvm-commits
Differential Revision: http://reviews.llvm.org/D15831
llvm-svn: 256866
If we don't have sys/wait.h and we're on a unix system there's no way
that several of the llvm tools work at all. This includes clang.
Just remove the configure and cmake checks entirely - we'll get a
build error instead of building something broken now.
llvm-svn: 243957
The patch is generated using this command:
tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
llvm/lib/
Thanks to Eugene Kosov for the original patch!
llvm-svn: 240137
As noted on Errc.h:
// * std::errc is just marked with is_error_condition_enum. This means that
// common patters like AnErrorCode == errc::no_such_file_or_directory take
// 4 virtual calls instead of two comparisons.
And on some libstdc++ those virtual functions conclude that
------------------------
int main() {
std::error_code foo = std::make_error_code(std::errc::no_such_file_or_directory);
return foo == std::errc::no_such_file_or_directory;
}
-------------------------
should exit with 0.
llvm-svn: 239683
Older versions of the TargetConditionals header always defined TARGET_OS_IPHONE to something (0 or 1), so we need to test not only for the existence but also if it is 1.
This resolves PR22631.
llvm-svn: 229904
In both the Unix and Windows variants, std::getenv was called and the
result passed directly to a function accepting a StringRef. This isn't
OK because it might return a null pointer and that causes the StringRef
constructor to assert (and generally produces crash-prone code if
asserts are disabled). Fix this by independently testing the result as
non-null prior to splitting things.
This in turn uncovered another bug in the Unix variant where it would
infinitely recurse if PATH="", or after this fix if PATH isn't set.
There is no need to recurse at all. Slightly re-arrange the code to make
it clear that we can just fixup the Paths argument based on the
environment if we find anything.
I don't know of a particularly useful way to test these routines in
LLVM. I'll commit a test to Clang that ensures that its driver correctly
handles various settings of PATH. However, I have no idea how to
correctly write a Windows test for the PATHEXT change. Any Windows
developers who could provide such a test, please have at. =D
Many thanks to Nick Lewycky and others for helping debug this. =/ It was
quite nasty for us to track down.
llvm-svn: 223099
This patch adds to LLVMSupport the capability of writing files with
international characters encoded in the current system encoding. This
is relevant for Windows, where we can either use UTF16 or the current
code page (the legacy Windows international characters). On UNIX, the
file is always saved in UTF8.
This will be used in a patch for clang to thoroughly support response
files creation when calling other tools, addressing PR15171. On
Windows, to correctly support internationalization, we need the
ability to write response files both in UTF16 or the current code
page, depending on the tool we will call. GCC for mingw, for instance,
requires files to be encoded in the current code page. MSVC tools
requires files to be encoded in UTF16.
Patch by Rafael Auler!
llvm-svn: 217068
This patch fixes a subtle bug in the UNIX implementation of
llvm::sys::argumentsFitWithinSystemLimits() regarding the misuse of a static
variable. This bug causes our cached number that stores the system command line
maximum length to be halved after each call to the function. With a sufficient
number of calls to this function, it will eventually report any given command
line string to be over system limits.
Patch by Rafael Auler.
llvm-svn: 216415
where there is no timeout. In the case where there is a timeout though, the
code is still wrong since it doesn't check that the alarm really went off.
Without this patch, I cannot debug a program that forks itself using
sys::ExecuteAndWait with lldb.
llvm-svn: 211918
Patch by Vladimir Voskresensky. The erros were:
Path.inc:274:3: error: ‘Dl_info’ was not declared in this scope
...
and
usr/include/spawn.h:52:14: error: expected ‘,’ or ‘...’ before ‘argv’
llvm-svn: 192185
- New ProcessInfo class to encapsulate information about child processes.
- Generalized the Wait() to support non-blocking wait on child processes.
- ExecuteNoWait() now returns a ProcessInfo object with information about
the launched child. Users will be able to use this object to
perform non-blocking wait.
- ExecuteNoWait() now accepts an ExecutionFailed param that tells if execution
failed or not.
These changes will allow users to implement basic process parallel
tools.
Differential Revision: http://llvm-reviews.chandlerc.com/D1728
llvm-svn: 191763
Execute's Data parameter is now optional, so we won't allocate memory
for it on Windows and we'll close the process handle.
The Unix code should probably do something similar to avoid accumulation
of zombie children that haven't been waited on.
Tested on Linux and Windows.
llvm-svn: 183906
It was only used to implement ExecuteAndWait and ExecuteNoWait. Expose just
those two functions and make Execute and Wait implementations details.
llvm-svn: 183864
This change lets us bootstrap LLVM/Clang under ASan and MSan. It contains
fixes for 2 issues:
- X86JIT reads return address from stack, which MSan does not know is
initialized.
- bugpoint tests run binaries with RLIMIT_AS. This does not work with certain
Sanitizers.
We are no longer including config.h in Compiler.h with this change.
llvm-svn: 174306
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.
Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]
llvm-svn: 169131
JF Bastien