On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).
We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.
One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.
A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.
Differential Revision: https://reviews.llvm.org/D48051
llvm-svn: 334630
With the upcoming patch to add summary parsing support, IsAnalysis would
be true in contexts where we are not performing module summary analysis.
Rename to the more specific and approprate HaveGVs, which is essentially
what this flag is indicating.
llvm-svn: 334140
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.
In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.
Differential Revision: https://reviews.llvm.org/D43624
llvm-svn: 332240
Summary:
This change is necessary for D46464, which will pass -1 as the Task
ID for distributed backends, so that the save temps files don't end
up with "4294967295" in their path. For distributed back ends, when -1
is passed, don't append any Task ID.
An existing test (tools/clang/test/CodeGen/thinlto_backend.ll) will
fail without this change after D46464.
Reviewers: pcc
Subscribers: mehdi_amini, inglorion, llvm-commits
Differential Revision: https://reviews.llvm.org/D46488
llvm-svn: 331591
Summary:
Set setDiagnosticsHotnessRequested before the early exit check for a
diagnostic output file, so that pass remarks with hotness works when
emitting pass remarks to stderr (e.g. via -pass-remarks=.).
Also fix the llvm-lto2 diagnistic handler so that it only calls exit(1)
when the diagnistic is an error type. Otherwise the new test invocation
of llvm-lto2 with -pass-remarks causes it to fail. The new code is
consistent with the diagnostic handler elsewhere (e.g. on the
LLVMContext).
Reviewers: pcc, davide
Subscribers: fhahn, mehdi_amini, llvm-commits, inglorion
Differential Revision: https://reviews.llvm.org/D46387
llvm-svn: 331569
Summary:
Support was added to the regular LTO backend, but not thinBackend.
This patch adds that support.
Reviewers: pcc, davide
Subscribers: mehdi_amini, inglorion, llvm-commits
Differential Revision: https://reviews.llvm.org/D46376
llvm-svn: 331481
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
llvm-svn: 331184
This patch adds a StatsFile option to LTO/Config.h and updates both
LLVMGold and llvm-lto2 to set it.
Reviewers: MatzeB, tejohnson, espindola
Reviewed By: tejohnson
Differential Revision: https://reviews.llvm.org/D45531
llvm-svn: 330411
Summary: As discussed in https://reviews.llvm.org/D45606, it makes more sense to name the class as SmallVectorMemoryBuffer
Reviewers: bkramer, dblaikie
Reviewed By: dblaikie
Subscribers: mehdi_amini, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D45661
llvm-svn: 330107
Summary:
Since the class is used by both MCJIT and LTO, it makes more sense to move it to Support lib.
This is a follow up patch to r329929 and https://reviews.llvm.org/D45244
Reviewers: bkramer, dblaikie
Reviewed By: bkramer
Subscribers: mehdi_amini, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D45606
llvm-svn: 330093
Summary:
r327219 added wrappers to std::sort which randomly shuffle the container before sorting.
This will help in uncovering non-determinism caused due to undefined sorting
order of objects having the same key.
To make use of that infrastructure we need to invoke llvm::sort instead of std::sort.
Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort.
Refer D44363 for a list of all the required patches.
Reviewers: pcc, mehdi_amini, ruiu
Reviewed By: ruiu
Subscribers: ruiu, inglorion, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D45137
llvm-svn: 330053
Summary: This enables debug fission on implicit ThinLTO when linked with gold. It will put the .dwo files in a directory specified by user.
Reviewers: tejohnson, pcc, dblaikie
Reviewed By: pcc
Subscribers: JDevlieghere, mehdi_amini, inglorion
Differential Revision: https://reviews.llvm.org/D44792
llvm-svn: 329988
Make sure ThinLTO with caching doesn't use non-atomic writes to the cache file (to prevent data races and cache files corruption).
1. Place temp file to the same place where the caching directory is (instead of creating it the directory pointed to by TMP/TEMP variable). This will help to prevent using non-atomic rename and falling back to non-atomic "direct" write to the cache file.
2. if rename failed do not write to the cache file directly (direct write to the file is non-atomic and could cause data race conditions).
3. if cache file doesn't exist (e.g., because 'rename' failed or because some other reasons), bypass using the cache altogether.
Differential Revision: https://reviews.llvm.org/D45076
llvm-svn: 328904
This reverts commit 1f3bd185c53beb6aa68446974b7e80837abd6ef0 (r326107)
because it fails
ThinLTO/X86/diagnostic-handler-remarks-with-hotness.ll.
llvm-svn: 326975
This wires up -pass-remarks-hotness-threshold to LTO and ThinLTO.
Next is to change the clang driver to pass this
with -fdiagnostics-hotness-threshold.
Differential Revision: https://reviews.llvm.org/D41465
llvm-svn: 326107
Summary:
ThinLTO indexing may decide to skip all objects. If we don't write something to
the list build system may consider this as failure or linker can reuse a file
from the previews build.
Reviewers: pcc, tejohnson
Subscribers: mehdi_amini, inglorion, eraman, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D43415
llvm-svn: 325819
Summary:
With D43396, no clients use the Path parameter anymore.
Depends on D43396.
Reviewers: pcc
Subscribers: mehdi_amini, inglorion, llvm-commits
Differential Revision: https://reviews.llvm.org/D43400
llvm-svn: 325619
Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes.
Third attempt - moved function from lambda to static function due to build failures.
llvm-svn: 325506
Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes.
Second attempt, since last patch caused stage2 build to fail (now using function_ref rather than std::function).
Reverted due to buildbot failures
llvm-svn: 325454
Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes.
Second attempt, since last patch caused stage2 build to fail (now using function_ref rather than std::function).
llvm-svn: 325448
It caused assertion failure
Assertion failed: (!DD.IsLambda && !MergeDD.IsLambda && "faked up lambda definition?"), function MergeDefinitionData, file /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp, line 1675.
on the second stage build bots.
llvm-svn: 324932
Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes.
llvm-svn: 324854
As of r323633, this bit started controlling whether symbol definitions
appear in object files, and it also became sensitive to the prevailing
bit, so it needs to be included in the key.
Differential Revision: https://reviews.llvm.org/D43109
llvm-svn: 324711
Summary:
Removing the dropped symbols will prevent indirect call promotion in the
ThinLTO Backend from adding a new reference to a symbol, which can
result in linker unsats. This can happen when we compile with a sample
profile collected from one binary by used for another, which may have
profiled targets that aren't used in the new binary.
Note that until dropDeadSymbols handles variables and aliases (in
progress), we may not be able to remove the declaration and can still
have an issue.
Reviewers: grimar, davidxl
Subscribers: mehdi_amini, inglorion, llvm-commits, eraman
Differential Revision: https://reviews.llvm.org/D42816
llvm-svn: 324299
Summary:
This complements the fixes in r323633 and r324075 which drop the
definitions of dead functions and variables, respectively.
Fixes PR36208.
Reviewers: grimar, rafael
Subscribers: mehdi_amini, llvm-commits, inglorion
Differential Revision: https://reviews.llvm.org/D42856
llvm-svn: 324242
This fixes PR36187.
Patch teaches ThinLTO to drop non-prevailing variables,
just like we recently did for functions (in r323633).
Differential revision: https://reviews.llvm.org/D42798
llvm-svn: 324075
Summary: ThinLTO may skip object for other reasons, e.g. if there is no summary.
Reviewers: pcc, eugenis
Subscribers: mehdi_amini, inglorion, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D42514
llvm-svn: 323818
Combine expression patterns to form expressions with fewer, simple instructions.
This pass does not modify the CFG.
For example, this pass reduce width of expressions post-dominated by TruncInst
into smaller width when applicable.
It differs from instcombine pass in that it contains pattern optimization that
requires higher complexity than the O(1), thus, it should run fewer times than
instcombine pass.
Differential Revision: https://reviews.llvm.org/D38313
llvm-svn: 323321
LTO sets dso_local as an optimization, so don't clear it.
This avoid clearing it from undefined hidden symbols, which would then
fail the verifier.
llvm-svn: 322814
Peter Collingbourne