lexer, add the token buffer underneath the caching lexer where possible and
push the tokens directly into the caching lexer otherwise. We previously
put the lexer into a corrupted state where we could not guarantee to provide
the tokens in the right order and would sometimes assert.
llvm-svn: 218333
1. We were hitting the NextIsPrevious assertion because we were trying
to merge decl chains that were independent of each other because we had
no Sema object to allow them to find existing decls. This is fixed by
delaying loading the "preloaded" decls until Sema is available.
2. We were trying to get identifier info from an annotation token, which
asserts. The fix is to special-case the module annotations in the
preprocessed output printer.
Fixed in a single commit because when you hit 1 you almost invariably
hit 2 as well.
llvm-svn: 217550
A couple of these arguments were passed by void* as a rather extreme
example of pimpling. Adjusting this to a more classic form of the idiom
(involving forward declarations) makes this more legible and allows
explicit passing of ownership via std::unique_ptr.
llvm-svn: 216785
Only those callers who are dynamically passing ownership should need the
3 argument form. Those accepting the default ("do pass ownership")
should do so explicitly with a unique_ptr now.
llvm-svn: 216614
Currently the analyzer lazily models some functions using 'BodyFarm',
which constructs a fake function implementation that the analyzer
can simulate that approximates the semantics of the function when
it is called. BodyFarm does this by constructing the AST for
such definitions on-the-fly. One strength of BodyFarm
is that all symbols and types referenced by synthesized function
bodies are contextual adapted to the containing translation unit.
The downside is that these ASTs are hardcoded in Clang's own
source code.
A more scalable model is to allow these models to be defined as source
code in separate "model" files and have the analyzer use those
definitions lazily when a function body is needed. Among other things,
it will allow more customization of the analyzer for specific APIs
and platforms.
This patch provides the initial infrastructure for this feature.
It extends BodyFarm to use an abstract API 'CodeInjector' that can be
used to synthesize function bodies. That 'CodeInjector' is
implemented using a new 'ModelInjector' in libFrontend, which lazily
parses a model file and injects the ASTs into the current translation
unit.
Models are currently found by specifying a 'model-path' as an
analyzer option; if no path is specified the CodeInjector is not
used, thus defaulting to the current behavior in the analyzer.
Models currently contain a single function definition, and can
be found by finding the file <function name>.model. This is an
initial starting point for something more rich, but it bootstraps
this feature for future evolution.
This patch was contributed by Gábor Horváth as part of his
Google Summer of Code project.
Some notes:
- This introduces the notion of a "model file" into
FrontendAction and the Preprocessor. This nomenclature
is specific to the static analyzer, but possibly could be
generalized. Essentially these are sources pulled in
exogenously from the principal translation.
Preprocessor gets a 'InitializeForModelFile' and
'FinalizeForModelFile' which could possibly be hoisted out
of Preprocessor if Preprocessor exposed a new API to
change the PragmaHandlers and some other internal pieces. This
can be revisited.
FrontendAction gets a 'isModelParsingAction()' predicate function
used to allow a new FrontendAction to recycle the Preprocessor
and ASTContext. This name could probably be made something
more general (i.e., not tied to 'model files') at the expense
of losing the intent of why it exists. This can be revisited.
- This is a moderate sized patch; it has gone through some amount of
offline code review. Most of the changes to the non-analyzer
parts are fairly small, and would make little sense without
the analyzer changes.
- Most of the analyzer changes are plumbing, with the interesting
behavior being introduced by ModelInjector.cpp and
ModelConsumer.cpp.
- The new functionality introduced by this change is off-by-default.
It requires an analyzer config option to enable.
llvm-svn: 216550
Changes diagnostic options, language standard options, diagnostic identifiers, diagnostic wording to use c++14 instead of c++1y. It also modifies related test cases to use the updated diagnostic wording.
llvm-svn: 215982
We already verified the primary module map file (either the one that
defines the top-level module, or the one that allows inferring it if it
is an inferred framework module). Now we also verify any other module
map files that define submodules, such as when there is a
module.private.modulemap file.
llvm-svn: 215455
According to the gcc docs, -include uses the current working directory
for the lookup instead of the main source file.
This patch gets rid of NormalizeIncludePath (which relied on an
implementation detail of FileManager / FileEntry for the include path
logic to work), and instead hands the correct lookup information down to
LookupFile.
This will allow us to change the FileEntry's behavior regarding its Name
caching.
llvm-svn: 215433
(dropping const from the reference as MemoryBuffer is immutable already,
so const is just redundant - and while I'd personally put const
everywhere, that's not the LLVM Way (see llvm::Type for another example
of an immutable type where "const" is omitted for brevity))
Changing the pointer argument to a reference parameter makes call sites
identical between callers with unique_ptrs or raw pointers, minimizing
the churn in a pending unique_ptr migrations.
llvm-svn: 215391
And in the process, discover that FileManager::removeStatCache had a
double-delete when removing an element from the middle of the list (at
the beginning or the end of the list, there was no problem) and add a
unit test to exercise the code path (which successfully crashed when run
(with modifications to match the old API) without this patch applied)
llvm-svn: 215388
Clang :: Frontend/iframework.c
Clang :: Frontend/system-header-prefix.c
Clang :: Index/annotate-comments-objc.m
Clang :: Index/annotate-module.m
Clang :: Index/index-module.m
Clang :: Index/index-pch-with-module.m
Clang :: PCH/case-insensitive-include.c
Suprisingly the normalize_separators() was no-op when LLVM_ON_WIN32.
Its replacement native() does change path separators into \ as expected,
breaking these tests.
I had fixed the tests by #ifndef LLVM_ON_WIN32 on the native call,
to match the previous behaviour.
If this logic is not used on Windows host, it might be completely
deleted as there should not be windows path seperators on Linux hosts.
I can't test on Linux but if someone can run tests on Linux after
commenting out the line
llvm::sys::path::native(NormalizedPath);
and the tests pass, the whole if (LangOpts.MSVCCompat) could be deleted.
llvm-svn: 215290
class Module. It's almost always going to be the same as
getContainingModule() for top-level modules, so just add a map to cover
the remaining cases. This lets us do less bookkeeping to keep the
ModuleMap fields up to date.
llvm-svn: 215268
intent when we added remark support, but was never implemented in the general
case, because the first -R flags didn't need it. (-Rpass= had special handling
to accomodate its argument.)
-Rno-foo, -Reverything, and -Rno-everything can be used to turn off a remark,
or to turn on or off all remarks. Per discussion on cfe-commits, -Weverything
does not affect remarks, and -Reverything does not affect warnings or errors.
The only "real" -R flag we have right now is -Rmodule-build; that flag is
effectively renamed from -Wmodule-build to -Rmodule-build by this change.
-Wpass and -Wno-pass (and their friends) are also renamed to -Rpass and
-Rno-pass by this change; it's not completely clear whether we intended to have
a -Rpass (with no =pattern), but that is unchanged by this commit, other than
the flag name. The default pattern is effectively one which matches no passes.
In future, we may want to make the default pattern be .*, so that -Reverything
works for -Rpass properly.
llvm-svn: 215046
* Track override set across module load and save
* Track originating module to allow proper re-export of #undef
* Make override set properly transitive when it picks up a #undef
This fixes nearly all of the remaining macro issues with self-host.
llvm-svn: 213922
Remove pointless MICache: it only ever contained up to 1 object, and was only
non-empty when recovering from an error. There's no performance or memory win
from maintaining this cache.
llvm-svn: 213825
This flag specifies that we are building an implementation file of the
module <name>, preventing importing <name> as a module. This does not
consider this to be the 'current module' for the purposes of doing
modular checks like decluse or non-modular-include warnings, unlike
-fmodule-name.
This is needed as a stopgap until:
1) we can resolve relative includes to a VFS-mapped module (or can
safely import a header textually and as part of a module)
and ideally
2) we can safely do incremental rebuilding when implementation files
import submodules.
llvm-svn: 213767
thorough tests.
Original commit message:
[modules] Fix macro hiding bug exposed if:
* A submodule of module A is imported into module B
* Another submodule of module A that is not imported into B exports a macro
* Some submodule of module B also exports a definition of the macro, and
happens to be the first submodule of B that imports module A.
In this case, we would incorrectly determine that A's macro redefines B's
macro, and so we don't need to re-export B's macro at all.
This happens with the 'assert' macro in an LLVM self-host. =(
llvm-svn: 213416
Just because we can open a directory named "COcoa.framework" doesn't
mean we should provide a "COcoa" module on a case-insensitive filesystem.
llvm-svn: 212975
Something went wrong with r211426, it is an older version of this code
and should not have been committed. It was reverted with r211434.
Original commit message:
We didn't properly implement support for the sized integer suffixes.
Suffixes like i16 were essentially ignored instead of mapping them to
the appropriately sized integer type.
This fixes PR20008.
Differential Revision: http://reviews.llvm.org/D4132
llvm-svn: 211441
This reverts commit r211426.
This broke the arm bots. The crash can be reproduced on X86 by running.
./bin/clang -cc1 -fsyntax-only -verify -fms-extensions ~/llvm/clang/test/Lexer/ms-extensions.c -triple arm-linux
llvm-svn: 211434
We didn't properly implement support for the sized integer suffixes.
Suffixes like i16 were essentially ignored instead of mapping them to
the appropriately sized integer type.
This fixes PR20008.
Differential Revision: http://reviews.llvm.org/D4132
llvm-svn: 211426
The compilation pipeline doesn't actually need to know about the high-level
concept of diagnostic mappings, and hiding the final computed level presents
several simplifications and other potential benefits.
The only exceptions are opportunistic checks to see whether expensive code
paths can be avoided for diagnostics that are guaranteed to be ignored at a
certain SourceLocation.
This commit formalizes that invariant by introducing and using
DiagnosticsEngine::isIgnored() in place of individual level checks throughout
lex, parse and sema.
llvm-svn: 211005
This begins to address cognitive dissonance caused by treating the Note
diagnostic level as a severity in the diagnostic engine.
No change in functionality.
llvm-svn: 210758
Diagnostic mappings are used to calculate the final severity of diagnostic
instances.
Detangle the implementation to reflect the terminology used in documentation
and bindings.
No change in functionality.
llvm-svn: 210518
With recent changes, this is now a compatible language extension and can be
safely enabled with -ms-extensions instead of requiring the full
-ms-compatibility MSVC drop-in mode. As such we can now also emit an extension
warning under -Wmicrosoft to help users port their code.
llvm-svn: 209978
This failure mode shows up occasionally when users try to include C headers in
C++ projects or when porting from Windows. We might as well recover in the way
the user expected, thus avoiding confusing diagnostic messages at point of use.
llvm-svn: 209963
The checks below can hypothetically apply to converted operator name
identifiers.
In practice there are no builtin macros etc. with those names so there's no
behavioural change to test.
llvm-svn: 209962
Summary:
The limits on the number of fix-it hints and ranges attached to a
diagnostic are arbitrary and don't apply universally to all users of the
DiagnosticsEngine. The way the limits are enforced may lead to diagnostics
generating invalid sets of fixes. I suggest removing the limits, which will also
simplify the implementation.
Reviewers: rsmith
Reviewed By: rsmith
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D3879
llvm-svn: 209468
When using the VFS, we want the virtual header location when searching
for a framework module, since that will be the one in the correct
directory structure for the module.
I'll add a regression test once I finish reducing the larger one I have.
llvm-svn: 208901
MacroArgs are owned by TokenLexer, and when a TokenLexer is destroyed, it'll
call its MacroArgs's destroy() method. destroy() only appends the MacroArg to
Preprocessor's MacroArgCache list, and Preprocessor's destructor then calls
deallocate() on all MacroArgs in that list. This method then ends up freeing
the MacroArgs's memory.
In a code completion context, Parser::cutOffParsing() gets called when a code
completion token is hit, which changes the type of the current token to
tok::eof. eof tokens aren't always ConsumeToken()ed, so
Preprocessor::HandleEndOfFile() isn't always called, and that function is
responsible for popping the macro stack.
Due to this, Preprocessor::CurTokenLexer can be non-NULL when
~Preprocessor runs. It's a unique_ptr, so it ended up being destructed after
~Preprocessor completed, and its MacroArgs thus got added to the freelist after
the code freeing things on the freelist had already completed. The fix is to
explicitly call reset() before the freelist processing happens. (See the bug
for more notes.)
llvm-svn: 208438
But keep -Wnon-modular-include-in-[framework-]module
This warning is too noisy and doesn't really indicate a problem for most
people. Even though it would only really affect people using
-Weverything, that seems bad so remove it.
llvm-svn: 208345
Warn on non-modular includes in various contexts.
-Wnon-modular-include
-Wnon-modular-include-in-module
-Wnon-modular-include-in-framework-module
Where each group is a subgroup of those above it.
llvm-svn: 208004
whole code would be better with std::unique_ptr managing the lifetimes
of the handlers, but I wanted to make a targeted fix to the leaks first.
With this change, all of the Clang preprocessor tests are leak free with
LSan.
llvm-svn: 207872
The Preprocessor::Initialize() function already offers a clear interface to
achieve this, further reducing the confusing number of states a newly
constructed preprocessor can have.
llvm-svn: 207825
Otherwise including a header in your source file that is not included by
framework's umbrella header will silently add an empty submodule with that
name.
is automatically translated to
@import Foo.NotInModule;
which then would have succeeded because the inferred module map
contained an empty submodule called NotInModule.
llvm-svn: 207024
digits. Turns out we have completely separate lexing codepaths for floating
point numbers depending on whether or not they start with a zero. Who knew...
=)
llvm-svn: 206932
Unless they are in submodules that aren't available anyway, due to
requirements not being met. Also, mark children as unavailable when the
parent is.
llvm-svn: 206664
This paves the way to making OnDiskHashTable work with hashes that are
not 32 bits wide and to making OnDiskHashTable work very large hash
tables. The LLVM change to use these types is upcoming.
llvm-svn: 206640
At one point, -fexceptions was a synonym for -fcxx-exceptions. While
the driver options still enables cxx-exceptions by default, the cc1
flag is purely about exception tables and this doesn't account for
objective C exceptions. Because of this, checking for the
cxx_exceptions feature in objective C++ often gives the wrong answer.
The cxx_exceptions feature should be based on the -fcxx-exceptions cc1
flag, not -fexceptions. Furthermore, at some point the tests were
changed to use cc1 even though they were testing the driver behaviour.
We're better off testing both the driver and cc1 here.
llvm-svn: 206352
To differentiate between two modules with the same name, we will
consider the path the module map file that they are defined by* part of
the ‘key’ for looking up the precompiled module (pcm file).
Specifically, this patch renames the precompiled module (pcm) files from
cache-path/<module hash>/Foo.pcm
to
cache-path/<module hash>/Foo-<hash of module map path>.pcm
In addition, I’ve taught the ASTReader to re-resolve the names of
imported modules during module loading so that if the header search
context changes between when a module was originally built and when it
is loaded we can rebuild it if necessary. For example, if module A
imports module B
first time:
clang -I /path/to/A -I /path/to/B ...
second time:
clang -I /path/to/A -I /different/path/to/B ...
will now rebuild A as expected.
* in the case of inferred modules, we use the module map file that
allowed the inference, not the __inferred_module.map file, since the
inferred file path is the same for every inferred module.
llvm-svn: 206201
The -fms-extensions option affects a number of subtle front-end C/C++
behaviors, and it would be useful to be able to distinguish MS keywords
from regular identifiers in the ms-extensions mode even if the triple
does not define a Windows target. It should make life easier if anyone
needs to port their Windows codes to elsewhere.
Differential Revision: http://reviews.llvm.org/D3034
llvm-svn: 206069
Extend the SSE2 comment lexing to AVX2. Only 16byte align when not on AVX2.
This provides some 3% speedup when preprocessing gcc.c as a single file.
The patch is wrong, it always uses SSE2, and when I fix that there's no speedup
at all. I am not sure where the 3% came from previously.
--Thi lie, and those below, will be ignored--
M Lex/Lexer.cpp
llvm-svn: 205548
Clean up the __has_attribute implementation without modifying its behavior.
Replaces the tablegen-driven AttrSpellings.inc, which lived in the lexing layer with AttrHasAttributeImpl.inc, which lives in the basic layer. Updates the preprocessor to call through to this new functionality which can take additional information into account (such as scopes and syntaxes).
Expose the ability for parts of the compiler to ask whether an attribute is supported for a given spelling (including scope), syntax, triple and language options.
llvm-svn: 205181
Committed this by accident before it was done last time.
Original message:
Rather than rolling our own functions to read little endian data
from a buffer, we can use the support in llvm's Endian.h.
No functional change.
llvm-svn: 205062
Richard Smith