single quotes are not digit separators after a valid character literal prefix
The single quote character can act as a c++ digit separator.
However, the minimizer shouldn't treat it as such when it's actually following
a valid character literal prefix, like L, U, u, or u8.
Differential Revision: https://reviews.llvm.org/D64525
llvm-svn: 365700
Summary:
Currently HeaderSearch only looks at SearchDir's passed into it, but in
addition to those paths headers can be relative to including file's directory.
This patch makes sure that is taken into account.
Reviewers: gribozavr
Subscribers: jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63295
llvm-svn: 365005
r362459 introduced DependencyDirectivesSourceMinimizerTest.cpp, which
hits an MSVC bug:
developercommunity.visualstudio.com/content/problem/67300/stringifying-raw-string-literal.html
This only happens when the parameter to a macro is stringified in the
macro. This patch removes the string from the assert so that the
warning no longer happens.
llvm-svn: 363074
that might affect the dependency list for a compilation
This commit introduces a dependency directives source minimizer to clang
that minimizes header and source files to the minimum necessary preprocessor
directives for evaluating includes. It reduces the source down to #define, #include,
The source minimizer works by lexing the input with a custom fast lexer that recognizes
the preprocessor directives it cares about, and emitting those directives in the minimized source.
It ignores source code, comments, and normalizes whitespace. It gives up and fails if seems
any directives that it doesn't recognize as valid (e.g. #define 0).
In addition to the source minimizer this patch adds a
-print-dependency-directives-minimized-source CC1 option that allows you to invoke the minimizer
from clang directly.
Differential Revision: https://reviews.llvm.org/D55463
llvm-svn: 362459
Summary:
Include insertion in clangd was inserting absolute paths when the
include directory was an absolute path with a double dot. This patch makes sure
double dots are handled both with absolute and relative paths.
Reviewers: sammccall
Subscribers: ilya-biryukov, ioeric, jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60873
llvm-svn: 359078
It now comes with a follow-up fix for the clients of this API
in clangd and clang-tidy.
Differential Revision: https://reviews.llvm.org/D59977
llvm-svn: 359035
As the unit test demonstrates, subtracting 1 from the offset was unnecessary.
The only user of this function was the plist file emitter (in Static Analyzer
and ARCMigrator). It means that a lot of Static Analyzer's plist arrows
are in fact off by one character. The patch carefully preserves this
completely incorrect behavior and causes no functional change,
i.e. no plist format breakage.
Differential Revision: https://reviews.llvm.org/D59977
llvm-svn: 357823
Change MemoryBufferCache to InMemoryModuleCache, moving it from Basic to
Serialization. Another patch will start using it to manage module build
more explicitly, but this is split out because it's mostly mechanical.
Because of the move to Serialization we can no longer abuse the
Preprocessor to forward it to the ASTReader. Besides the rename and
file move, that means Preprocessor::Preprocessor has one fewer parameter
and ASTReader::ASTReader has one more.
llvm-svn: 355777
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
The test case had a parse error that was causing the condition string to be misreported. We now have better fallback code for error cases.
llvm-svn: 351470
This adjusts the source range passed in to the preprocessor callbacks to only include the condition range itself, rather than all of the conditionally skipped tokens.
llvm-svn: 350891
This patch moves the virtual file system form clang to llvm so it can be
used by more projects.
Concretely the patch:
- Moves VirtualFileSystem.{h|cpp} from clang/Basic to llvm/Support.
- Moves the corresponding unit test from clang to llvm.
- Moves the vfs namespace from clang::vfs to llvm::vfs.
- Formats the lines affected by this change, mostly this is the result of
the added llvm namespace.
RFC on the mailing list:
http://lists.llvm.org/pipermail/llvm-dev/2018-October/126657.html
Differential revision: https://reviews.llvm.org/D52783
llvm-svn: 344140
This commit relands r331904.
Adding a SrcMgr::CharacteristicKind parameter to the InclusionDirective
in PPCallbacks, and updating calls to that function. This will be useful
in https://reviews.llvm.org/D43778 to determine which includes are
system
headers.
Differential Revision: https://reviews.llvm.org/D46614
llvm-svn: 332021
Adding a SrcMgr::CharacteristicKind parameter to the InclusionDirective
in PPCallbacks, and updating calls to that function. This will be useful
in https://reviews.llvm.org/D43778 to determine which includes are system
headers.
Differential Revision: https://reviews.llvm.org/D46614
llvm-svn: 331904
When a '>>' token is split into two '>' tokens (in C++11 onwards), or (as an
extension) when we do the same for other tokens starting with a '>', we can't
just use a location pointing to the first '>' as the location of the split
token, because that would result in our miscomputing the length and spelling
for the token. As a consequence, for example, a refactoring replacing 'A<X>'
with something else would sometimes replace one character too many, and
similarly diagnostics highlighting a template-id source range would highlight
one character too many.
Fix this by creating an expansion range covering the first character of the
'>>' token, whose spelling is '>'. For this to work, we generalize the
expansion range of a macro FileID to be either a token range (the common case)
or a character range (used in this new case).
llvm-svn: 331155
Fix makes the loop in LexAngledStringLiteral more like the loops in
LexStringLiteral, LexCharConstant. When we skip a character after
backslash, we need to check if we reached the end of the file instead of
reading the next character unconditionally.
Discovered by OSS-Fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3832
rdar://problem/35572754
Reviewers: arphaman, kcc, rsmith, dexonsmith
Reviewed By: rsmith, dexonsmith
Subscribers: cfe-commits, rsmith, dexonsmith
Differential Revision: https://reviews.llvm.org/D41423
llvm-svn: 322390
Summary: This patch implements 4.3 of http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4220.pdf. If a raw string contains a newline character, replace each newline character with the \n escape code. Without this patch, included test case (macro_raw_string.cpp) results compilation failure.
Reviewers: rsmith, doug.gregor, jkorous-apple
Reviewed By: jkorous-apple
Subscribers: jkorous-apple, vsapsai, cfe-commits
Differential Revision: https://reviews.llvm.org/D39279
llvm-svn: 319904
We currently use target_link_libraries without an explicit scope
specifier (INTERFACE, PRIVATE or PUBLIC) when linking executables.
Dependencies added in this way apply to both the target and its
dependencies, i.e. they become part of the executable's link interface
and are transitive.
Transitive dependencies generally don't make sense for executables,
since you wouldn't normally be linking against an executable. This also
causes issues for generating install export files when using
LLVM_DISTRIBUTION_COMPONENTS. For example, clang has a lot of LLVM
library dependencies, which are currently added as interface
dependencies. If clang is in the distribution components but the LLVM
libraries it depends on aren't (which is a perfectly legitimate use case
if the LLVM libraries are being built static and there are therefore no
run-time dependencies on them), CMake will complain about the LLVM
libraries not being in export set when attempting to generate the
install export file for clang. This is reasonable behavior on CMake's
part, and the right thing is for LLVM's build system to explicitly use
PRIVATE dependencies for executables.
Unfortunately, CMake doesn't allow you to mix and match the keyword and
non-keyword target_link_libraries signatures for a single target; i.e.,
if a single call to target_link_libraries for a particular target uses
one of the INTERFACE, PRIVATE, or PUBLIC keywords, all other calls must
also be updated to use those keywords. This means we must do this change
in a single shot. I also fully expect to have missed some instances; I
tested by enabling all the projects in the monorepo (except dragonegg),
and configuring both with and without shared libraries, on both Darwin
and Linux, but I'm planning to rely on the buildbots for other
configurations (since it should be pretty easy to fix those).
Even after this change, we still have a lot of target_link_libraries
calls that don't specify a scope keyword, mostly for shared libraries.
I'm thinking about addressing those in a follow-up, but that's a
separate change IMO.
Differential Revision: https://reviews.llvm.org/D40823
llvm-svn: 319840
Summary:
Lexer::GetBeginningOfToken produced invalid location when
backtracking across escaped new lines.
This fixes PR26228
Reviewers: akyrtzi, alexfh, rsmith, doug.gregor
Reviewed By: alexfh
Subscribers: alexfh, cfe-commits
Patch by Paweł Żukowski!
Differential Revision: https://reviews.llvm.org/D30748
llvm-svn: 310576
- Extracted the reading of the tokens out into a separate function.
- Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).
This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.
I will also directly update some extra clang tooling that is broken by the change from Argument to Parameter.
Hopefully the bots will stay appeased.
Thanks!
llvm-svn: 308190
- Extracted the reading of the tokens out into a separate function.
- Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).
This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.
Thanks!
llvm-svn: 308157
A new LexerTest unittest introduced a memory leak. This patch
uses a unique_ptr with a custom deleter to ensure it is properly
deleted.
llvm-svn: 305491
correct getNumArguments
StringifiedArguments is allocated (resized) based on the size the
getNumArguments function. However, this function ACTUALLY currently
returns the amount of total UnexpArgTokens which is minimum the same as
the new implementation of getNumMacroArguments, since empty/omitted arguments
result in 1 UnexpArgToken, and included ones at minimum include 2
(1 for the arg itself, 1 for eof).
This patch renames the otherwise unused getNumArguments to be more clear
that it is the number of arguments that the Macro expects, and thus the maximum
number that can be stringified. This patch also replaces the explicit memset
(which results in value instantiation of the new tokens, PLUS clearing the
memory) with brace initialization.
Differential Revision: https://reviews.llvm.org/D32046
llvm-svn: 305425
as part of a compilation.
This is intended for two purposes:
1) Writing self-contained test cases for modules: we can now write a single
source file test that builds some number of module files on the side and
imports them.
2) Debugging / test case reduction. A single-source testcase is much more
amenable to reduction, compared to a VFS tarball or .pcm files.
llvm-svn: 305101
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298278
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
The lexer sets the end location of macro arguments incorrectly *if*,
while merging consecutive args to fit into a single SLocEntry, it finds
args which come from different macro files.
Fix the issue by using separate SLocEntries in this situation.
This fixes a code coverage crasher (rdar://problem/26181005). Because
the lexer reported end locations for certain macro args incorrectly, we
would generate bogus coverage mappings with negative line offsets.
Reviewed-by: akyrtzi
Differential Revision: http://reviews.llvm.org/D20401
llvm-svn: 270160
This is a mechanical move of CodeGenOptions from libFrontend to libBasic. This
fixes the layering violation introduced earlier by threading CodeGenOptions into
TargetInfo. It should also fix the modules based self-hosting builds. NFC.
llvm-svn: 265702
This threads CodeGenOptions into the TargetInfo hierarchy. This is motivated by
ARM which can change some target information based on the EABI selected
(-meabi). Similar options exist for other platforms (e.g. MIPS) and thus is
generally useful. NFC.
llvm-svn: 265640
Use it to calculate UserLabelPrefix, instead of specifying it (often
incorrectly).
Note that the *actual* user label prefix has always come from the
DataLayout, and is handled within LLVM. The main thing clang's
TargetInfo::UserLabelPrefix did was to set the #define value. Having
these be different from each-other is just silly.
Differential Revision: http://reviews.llvm.org/D17183
llvm-svn: 262737
Change getString() to return Optional<StringRef>, and change
lookupFilename() to return an empty string if either one of the prefix
and suffix can't be found.
This is a more robust follow-up to r261461, but it's still not entirely
satisfactory. Ideally we'd report that the header map is corrupt;
perhaps something for a follow-up.
llvm-svn: 261596
Switch to using `isPowerOf2_32()` to check whether the buckets are a
power of two, and as a side benefit reject loading a header map with no
buckets. This is a follow-up to r261448.
llvm-svn: 261585
If a header map file is corrupt, the strings in the string table may not
be null-terminated. The logic here previously relied on `MemoryBuffer`
always being null-terminated, but this isn't actually guaranteed by the
class AFAICT. Moreover, we're seeing a lot of crash traces at calls to
`strlen()` inside of `lookupFilename()`, so something is going wrong
there.
Instead, use `strnlen()` to get the length, and check for corruption.
Also remove code paths that could call `StringRef(nullptr)`. r261459
made these rather obvious (although they'd been there all along).
llvm-svn: 261461
Add a simple test for `HeaderMap::lookupFileName()`. I'm planning to
add better error checking in a moment, and I'll add more tests like this
then.
llvm-svn: 261455
Alex Lorenz