Commit Graph

281 Commits

Author SHA1 Message Date
Craig Topper 4b56692e30 [C++11] Use 'nullptr'.
llvm-svn: 210448
2014-06-09 02:04:02 +00:00
Craig Topper d2d442ca73 [C++11] Use 'nullptr'. Lex edition.
llvm-svn: 209083
2014-05-17 23:10:59 +00:00
Alp Toker 2d57cea256 Provide and use a safe Token::getRawIdentifier() accessor
llvm-svn: 209061
2014-05-17 04:53:25 +00:00
Alp Toker 6ac2cd01d2 Rename SourceManager::createFileIDForMemBuffer()
It makes more sense to just overload createFileID().

Gardening only.

llvm-svn: 209002
2014-05-16 17:23:01 +00:00
Nico Weber 5f5b94141c Don't leak MacroArgs when using code completion, PR19688.
MacroArgs are owned by TokenLexer, and when a TokenLexer is destroyed, it'll
call its MacroArgs's destroy() method.  destroy() only appends the MacroArg to
Preprocessor's MacroArgCache list, and Preprocessor's destructor then calls
deallocate() on all MacroArgs in that list.  This method then ends up freeing
the MacroArgs's memory.

In a code completion context, Parser::cutOffParsing() gets called when a code
completion token is hit, which changes the type of the current token to
tok::eof.  eof tokens aren't always ConsumeToken()ed, so
Preprocessor::HandleEndOfFile() isn't always called, and that function is
responsible for popping the macro stack.

Due to this, Preprocessor::CurTokenLexer can be non-NULL when
~Preprocessor runs.  It's a unique_ptr, so it ended up being destructed after
~Preprocessor completed, and its MacroArgs thus got added to the freelist after
the code freeing things on the freelist had already completed.  The fix is to
explicitly call reset() before the freelist processing happens.  (See the bug
for more notes.)

llvm-svn: 208438
2014-05-09 18:09:42 +00:00
Alp Toker 9663780e35 Reformat code following Preprocessor constructor updates
Landing this separately to make the previous commits easy to follow at home.

llvm-svn: 207826
2014-05-02 03:43:38 +00:00
Alp Toker 1ae02f68be Factor TargetInfo pointer/DelayInitialization bool pair out of Preprocessor ctor
The Preprocessor::Initialize() function already offers a clear interface to
achieve this, further reducing the confusing number of states a newly
constructed preprocessor can have.

llvm-svn: 207825
2014-05-02 03:43:30 +00:00
Alp Toker 23aa353291 Remove unused IncrProcessing parameter from Preprocessor ctor
Preprocessor::enableIncrementalProcessing() provides a consistent interface to
enable the feature.

llvm-svn: 207824
2014-05-02 03:43:21 +00:00
Benjamin Kramer 329c596e58 Preprocessor: Clarify the ownership of the IncludeMacroStack with unique_ptr.
llvm-svn: 204007
2014-03-15 16:40:40 +00:00
Argyrios Kyrtzidis e1974dcd92 [Preprocessor] Pass TranslationUnitKind to the preprocessor and if it is TU_Prefix
avoid warning for unused macros.

rdar://15034698

llvm-svn: 203213
2014-03-07 07:47:58 +00:00
Alp Toker 7d2fea2fd6 Use a more conventional logical op instead of bitwise and
llvm-svn: 202006
2014-02-24 04:35:58 +00:00
Richard Smith 8b7258bdb3 PR18855: Add support for UCNs and UTF-8 encoding within ud-suffixes.
llvm-svn: 201532
2014-02-17 21:52:30 +00:00
Reid Kleckner c0dca6ded7 MS ABI: Implement #pragma vtordisp() and clang-cl /vdN
These features are new in VS 2013 and are necessary in order to layout
std::ostream correctly.  Currently we have an ABI incompatibility when
self-hosting with the 2013 stdlib in our convertible_fwd_ostream wrapper
in gtest.

This change adds another implicit attribute, MSVtorDispAttr, because
implicit attributes are currently the best way to make sure the
information stays on class templates through instantiation.

Reviewers: majnemer

Differential Revision: http://llvm-reviews.chandlerc.com/D2746

llvm-svn: 201274
2014-02-12 23:50:26 +00:00
Richard Smith 67294e253e Track the currently-being-built submodule inside the preprocessor (rather than
just storing a flag indicating if there was one), and include it in the 'end of
module' annotation. No functionality change.

llvm-svn: 200573
2014-01-31 20:47:44 +00:00
Chandler Carruth 5553d0d4ca Sort all the #include lines with LLVM's utils/sort_includes.py which
encodes the canonical rules for LLVM's style. I noticed this had drifted
quite a bit when cleaning up LLVM, so wanted to clean up Clang as well.

llvm-svn: 198686
2014-01-07 11:51:46 +00:00
Richard Smith 34f30516aa Generate a marker token when entering or leaving a submodule when building a
module. Use the marker to diagnose cases where we try to transition between
submodules when not at the top level (most likely because a closing brace was
missing at the end of a header file, but is also possible if submodule headers
attempt to do something fundamentally non-modular, like our .def files).

llvm-svn: 195543
2013-11-23 04:06:09 +00:00
Douglas Gregor 594b8c934f Modules: Teach the preprocessor to recognize 'import' only after an '@'.
The preprocessor currently recognizes module declarations to load a
module based on seeing the 'import' keyword followed by an
identifier. This sequence is fairly unlikely in C (one would need a
type named 'import'), but is more common in Objective-C (where a
variable named 'import' can cause problems). Since import declarations
currently require a leading '@', recognize that in the preprocessor as
well. Fixes <rdar://problem/15084587>.

llvm-svn: 194225
2013-11-07 22:55:02 +00:00
Eli Friedman 0834a4b901 Make Preprocessor::Lex non-recursive.
Before this patch, Lex() would recurse whenever the current lexer changed (e.g.
upon entry into a macro). This patch turns the recursion into a loop: the
various lex routines now don't return a token when the current lexer changes,
and at the top level Preprocessor::Lex() now loops until it finds a token.
Normally, the recursion wouldn't end up being very deep, but the recursion depth
can explode in edge cases like a bunch of consecutive macros which expand to
nothing (like in the testcase test/Preprocessor/macro_expand_empty.c in this
patch).

<rdar://problem/14569770>

llvm-svn: 190980
2013-09-19 00:41:32 +00:00
Daniel Jasper 07e6c407bc Add option to disable module loading.
This patch was created by Lawrence Crowl and reviewed in:
http://llvm-reviews.chandlerc.com/D963

llvm-svn: 187738
2013-08-05 20:26:17 +00:00
Argyrios Kyrtzidis 37e48ff547 [Preprocessor] For the MacroExpands preprocessor callback, also pass the MacroArgs object that provides information about
the argument tokens for a function macro.

llvm-svn: 181065
2013-05-03 22:31:32 +00:00
Argyrios Kyrtzidis d48b91dbbe [PCH] Fix memory leak related to deserialized MacroInfo objects.
Deserialized MacroInfos were not destroyed and if their SmallVector did heap allocation,
it was leaked.

rdar://13768967

llvm-svn: 180771
2013-04-30 05:05:35 +00:00
Argyrios Kyrtzidis 6c811411cf [Preprocessor] Remove PPMutationListener.
It's not used anymore.

llvm-svn: 178106
2013-03-27 01:25:24 +00:00
Argyrios Kyrtzidis b6210dff97 [Preprocessor/Modules] Separate the macro directives kinds into their own MacroDirective's subclasses.
For each macro directive (define, undefine, visibility) have a separate object that gets chained
to the macro directive history. This has several benefits:

-No need to mutate a MacroDirective when there is a undefine/visibility directive. Stuff like
 PPMutationListener become unnecessary.
-No need to keep extra source locations for the undef/visibility locations for the define directive object
 (which is the majority of the directives)
-Much easier to hide/unhide a section in the macro directive history.
-Easier to track the effects of the directives across different submodules.

llvm-svn: 178037
2013-03-26 17:17:01 +00:00
Argyrios Kyrtzidis fead64be9b [preprocessor] Use MacroDirective in the preprocessor callbacks to make available the
full information about the macro (e.g if it was imported and where).

llvm-svn: 175978
2013-02-24 00:05:14 +00:00
Argyrios Kyrtzidis 09c9e811de [preprocessor] Split the MacroInfo class into two separate concepts, MacroInfo class
for the data specific to a macro definition (e.g. what the tokens are), and
MacroDirective class which encapsulates the changes to the "macro namespace"
(e.g. the location where the macro name became active, the location where it was undefined, etc.)

(A MacroDirective always points to a MacroInfo object.)

Usually a macro definition (MacroInfo) is where a macro name becomes active (MacroDirective) but
splitting the concepts allows us to better model the effect of modules to the macro namespace
(also as a bonus it allows better modeling of push_macro/pop_macro #pragmas).
Modules can have their own macro history, separate from the local (current translation unit)
macro history; MacroDirectives will be used to model the macro history (changes to macro namespace).

For example, if "@import A;" imports macro FOO, there will be a new local MacroDirective created
to indicate that "FOO" became active at the import location. Module "A" itself will contain another
MacroDirective in its macro history (at the point of the definition of FOO) and both MacroDirectives
will point to the same MacroInfo object.

Introducing the separation of macro concepts is the first part towards better modeling of module macros.

llvm-svn: 175585
2013-02-20 00:54:57 +00:00
Argyrios Kyrtzidis 22c22f5f9a Introduce SourceManager::PredefinesFileID, to allow each checking of whether
a source location came from the predefines buffer.

llvm-svn: 174190
2013-02-01 16:36:07 +00:00
Jordan Rose 324ec4221a Preprocessor: initialize member introduced in r173881.
Compilation always sets this explicitly, but creating a preprocessor
manually should still put the 'IsPreprocessedOutput' flag in a valid state.

llvm-svn: 174077
2013-01-31 19:26:01 +00:00
Dmitri Gribenko 9feeef40f5 Move UTF conversion routines from clang/lib/Basic to llvm/lib/Support
This is required to use them in TableGen.

llvm-svn: 173924
2013-01-30 12:06:08 +00:00
Jordan Rose 7f43dddae0 Handle universal character names and Unicode characters outside of literals.
This is a missing piece for C99 conformance.

This patch handles UCNs by adding a '\\' case to LexTokenInternal and
LexIdentifier -- if we see a backslash, we tentatively try to read in a UCN.
If the UCN is not syntactically well-formed, we fall back to the old
treatment: a backslash followed by an identifier beginning with 'u' (or 'U').

Because the spelling of an identifier with UCNs still has the UCN in it, we
need to convert that to UTF-8 in Preprocessor::LookUpIdentifierInfo.

Of course, valid code that does *not* use UCNs will see only a very minimal
performance hit (checks after each identifier for non-ASCII characters,
checks when converting raw_identifiers to identifiers that they do not
contain UCNs, and checks when getting the spelling of an identifier that it
does not contain a UCN).

This patch also adds basic support for actual UTF-8 in the source. This is
treated almost exactly the same as UCNs except that we consider stray
Unicode characters to be mistakes and offer a fixit to remove them.

llvm-svn: 173369
2013-01-24 20:50:46 +00:00
David Blaikie f157e90135 Remove unnecessary initialization i Added in r172668.
echristo already fixed this in r172649, but I'll leave the reformatting in
since I'm in the blame history for it now anyway.

llvm-svn: 172672
2013-01-16 23:18:16 +00:00
David Blaikie 687cd95941 Fix -Wreorder warning.
Rewrapping courtesy of clang-format.

llvm-svn: 172668
2013-01-16 23:13:36 +00:00
Eric Christopher 5e4696d2f5 Move initialization of ParsingIfOrElifDirective down next to the macro
initializations to fix Wreorder warning.

llvm-svn: 172649
2013-01-16 20:09:36 +00:00
Aaron Ballman 6ce0000dd5 No longer crashing with an assert when __has_include or __has_include_next is used outside of a preprocessor directive. This fixes PR14837.
llvm-svn: 172639
2013-01-16 19:32:21 +00:00
Dmitri Gribenko f857950d39 Remove useless 'llvm::' qualifier from names like StringRef and others that are
brought into 'clang' namespace by clang/Basic/LLVM.h

llvm-svn: 172323
2013-01-12 19:30:44 +00:00
Richard Smith 181879c293 Don't warn about disabled macro expansion if we see the name of a function-like macro which isn't immediately followed by '('. FreeBSD's stdio.h #defines foo(x) to (foo)(x), apparently.
llvm-svn: 169960
2012-12-12 02:46:14 +00:00
Douglas Gregor c50d4924eb Use @import rather than @__experimental_modules_import, since the
latter is rather a mess to type.

llvm-svn: 169919
2012-12-11 22:11:52 +00:00
Chandler Carruth 3a02247dc9 Sort all of Clang's files under 'lib', and fix up the broken headers
uncovered.

This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.

I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.

llvm-svn: 169237
2012-12-04 09:13:33 +00:00
Argyrios Kyrtzidis f3d587ea7c Refactor recording the preprocessor conditional directive regions out of
PreprocessingRecord and into its own class, PPConditionalDirectiveRecord.

Decoupling allows a client to use the functionality of PPConditionalDirectiveRecord
without needing a PreprocessingRecord.

llvm-svn: 169229
2012-12-04 07:27:05 +00:00
Andy Gibbs a8df57a962 Made the "expected string literal" diagnostic more expressive
llvm-svn: 168267
2012-11-17 19:16:52 +00:00
Andy Gibbs 58905d251b Refactored duplicate string literal lexing code within Preprocessor, into a
common LexStringLiteral function.  In doing so, some consistency problems have
been ironed out (e.g. where the first token in the string literal was lexed
with macro expansion, but subsequent ones were not) and also an erroneous
diagnostic has been corrected.

LexStringLiteral is complemented by a FinishLexStringLiteral function which
can be used in the situation where the first token of the string literal has
already been lexed.

llvm-svn: 168266
2012-11-17 19:15:38 +00:00
Douglas Gregor 1452ff155b Teach the preprocessor to hold onto the preprocessor options.
llvm-svn: 166599
2012-10-24 17:46:57 +00:00
Douglas Gregor cb28f9d7ad Rework the (de-)serialization of macros, as stored in
MacroInfo*. Instead of simply dumping an offset into the current file,
give each macro definition a proper ID with all of the standard
modules-remapping facilities. Additionally, when a macro is modified
in a subsequent AST file (e.g., #undef'ing a macro loaded from another
module or from a precompiled header), provide a macro update record
rather than rewriting the entire macro definition. This gives us
greater consistency with the way we handle declarations, and ties
together macro definitions much more cleanly.

Note that we're still not actually deserializing macro history (we
never were), but it's far easy to do properly now.

llvm-svn: 165560
2012-10-09 23:05:51 +00:00
Dmitri Gribenko 6743e04699 Move the 'find macro by spelling' infrastructure to the Preprocessor class and
use it to suggest appropriate macro for __attribute__((deprecated)) in
-Wdocumentation-deprecated-sync.

llvm-svn: 164892
2012-09-29 11:40:46 +00:00
Argyrios Kyrtzidis 19d78b743f For PPCallbacks::InclusionDirective() add a parameter for the module, whenever
an inclusion directive was automatically turned into a module import, and
PPCallbacks::moduleImport() for an explicit module import.

llvm-svn: 164874
2012-09-29 01:06:10 +00:00
Dmitri Gribenko b8e9e7507e StringRef'ize Preprocessor::CreateString().
llvm-svn: 164555
2012-09-24 21:07:17 +00:00
Alexander Kornienko 199cd94f50 Added some preprocessor memory usage stats to -cc1 -print-stats
llvm-svn: 161752
2012-08-13 10:46:42 +00:00
Douglas Gregor 3f568c100e Patch for handling C99 veriadic macros when using precompiled headers,
from Filipe Cabecinhas!

llvm-svn: 159446
2012-06-29 18:27:59 +00:00
Dmitri Gribenko aab8383a2b Structured comment parsing, first step.
* Retain comments in the AST
* Serialize/deserialize comments
* Find comments attached to a certain Decl
* Expose raw comment text and SourceRange via libclang

llvm-svn: 158771
2012-06-20 00:34:58 +00:00
Meador Inge cdc0057e3c Revert predefined decl tracking.
r158085 added some logic to track predefined declarations.  The main reason we
had predefined declarations in the input was because the __builtin_va_list
declarations were injected into the preprocessor input.  As of r158592 we 
explicitly build the __builtin_va_list declarations.  Therefore the predefined
decl tracking is no longer needed.

llvm-svn: 158732
2012-06-19 18:17:30 +00:00
Jordan Rose de1a29277e Disable _Pragma during HTML macro rewriting to keep from crashing.
The preprocessor's handling of diagnostic push/pops is stateful, so
encountering pragmas during a re-parse causes problems. HTMLRewrite
already filters out normal # directives including #pragma, so it's
clear it's not expected to be interpreting pragmas in this mode.

This fix adds a flag to Preprocessor to explicitly disable pragmas.
The "right" fix might be to separate pragma lexing from pragma
parsing so that we can throw away pragmas like we do preprocessor
directives, but right now it's important to get the fix in.

Note that this has nothing to do with the "hack" of re-using the
input preprocessor in HTMLRewrite. Even if we someday copy the
preprocessor instead of re-using it, the copy would (and should) include
the diagnostic level tables and have the same problems.

llvm-svn: 158214
2012-06-08 18:06:21 +00:00