Commit Graph

1736 Commits

Author SHA1 Message Date
Kostya Serebryany 6c2479bee4 Fix buffer overflow in Lexer
Summary:
Fix PR22407, where the Lexer overflows the buffer when parsing
 #include<\
(end of file after slash)

Test Plan:
Added a test that will trigger in asan build.
This case is also covered by the clang-fuzzer bot.

Reviewers: rnk

Reviewed By: rnk

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D9489

llvm-svn: 236466
2015-05-04 22:30:29 +00:00
Richard Smith 36bd40df72 Switch PPCallbacks to take the new MacroDefinition instead of MacroDirective*, in order to preserve full information on module macro expansion.
llvm-svn: 236404
2015-05-04 03:15:40 +00:00
Richard Smith 66a8186ed4 Rename MacroDefinition -> MacroDefinitionRecord, Preprocessor::MacroDefinition -> MacroDefinition.
clang::MacroDefinition now models the currently-defined value of a macro. The
previous MacroDefinition type, which represented a record of a macro definition
directive for a detailed preprocessing record, is now called MacroDefinitionRecord.

llvm-svn: 236400
2015-05-04 02:25:31 +00:00
Yaron Keren 203439f1f8 clang-format function definition header. NFC.
llvm-svn: 236390
2015-05-02 19:29:29 +00:00
Richard Smith 10434f307c [modules] Remove dead code from Module for tracking macro import locations.
llvm-svn: 236376
2015-05-02 02:08:26 +00:00
Richard Smith 32dbd69ce2 [modules] Don't bother creating a ModuleMacro representing a #undef that overrides nothing.
llvm-svn: 236374
2015-05-02 01:14:40 +00:00
Richard Smith 38477db8c4 [modules] If a module #includes a modular header that #undef's its macro, it
should not export the macro.

... at least, not unless we have local submodule visibility enabled.

llvm-svn: 236369
2015-05-02 00:45:56 +00:00
Richard Smith ee977933f7 [modules] Add -fmodules-local-submodule-visibility flag.
This flag specifies that the normal visibility rules should be used even for
local submodules (submodules of the currently-being-built module). Thus names
will only be visible if a header / module that declares them has actually been
included / imported, and not merely because a submodule that happened to be
built earlier declared those names. This also removes the need to modularize
bottom-up: textually-included headers will be included into every submodule
that includes them, since their include guards will not leak between modules.

So far, this only governs visibility of macros, not of declarations, so is not
ready for real use yet.

llvm-svn: 236350
2015-05-01 21:22:17 +00:00
Richard Smith a7e2cc684f [modules] Start moving the module visibility information off the Module itself.
It has no place there; it's not a property of the Module, and it makes
restoring the visibility set when we leave a submodule more difficult.

llvm-svn: 236300
2015-05-01 01:53:09 +00:00
Richard Smith d0014bf6f8 Make macro dumping robust against a nonexistent macro.
llvm-svn: 236285
2015-04-30 23:42:10 +00:00
Richard Smith 3ffa61d576 [modules] Add a mechanism to dump information about a macro.
Wire this up to "#pragma clang __debug macro <name>".

llvm-svn: 236280
2015-04-30 23:10:40 +00:00
Richard Smith 3981b17709 Remove dead code: a MacroDirective can't be imported or ambiguous any more.
llvm-svn: 236197
2015-04-30 02:16:23 +00:00
Richard Smith f5ec2acca9 Add an assert to get information on buildbot failure.
llvm-svn: 236181
2015-04-29 23:40:48 +00:00
Richard Smith 3d5925b0e4 Fix unused variable warning.
llvm-svn: 236178
2015-04-29 23:26:13 +00:00
Richard Smith 20e883e59b [modules] Stop trying to fake up a linear MacroDirective history.
Modules builds fundamentally have a non-linear macro history. In the interest
of better source fidelity, represent the macro definition information
faithfully: we have a linear macro directive history within each module, and at
any point we have a unique "latest" local macro directive and a collection of
visible imported directives. This also removes the attendent complexity of
attempting to create a correct MacroDirective history (which we got wrong
in the general case).

No functionality change intended.

llvm-svn: 236176
2015-04-29 23:20:19 +00:00
Richard Smith 1e1728536a Refactor to make MacroState ownership and lifetime clearer.
llvm-svn: 236032
2015-04-28 21:05:07 +00:00
Richard Smith 66f5ab26e3 Fix memory leak found by asan buildbot.
llvm-svn: 235957
2015-04-28 01:11:23 +00:00
Richard Smith 753e007091 [modules] Incrementally compute the list of overridden module macros based on
the active module macros at the point of definition, rather than reconstructing
it from the macro history. No functionality change intended.

llvm-svn: 235941
2015-04-27 23:21:38 +00:00
David Majnemer b9ac794f57 Remove unused variable to silence GCC warning
llvm-svn: 235693
2015-04-24 04:14:25 +00:00
Richard Smith 6b77f549cb [modules] Partial revert of r235669: don't create ModuleMacros for imported local macros.
The surrounding infrastructure isn't quite ready for this yet.

llvm-svn: 235677
2015-04-24 00:41:09 +00:00
Richard Smith 50474bf5d2 [modules] Refactor creation of ModuleMacros and create them when importing from local submodules.
llvm-svn: 235669
2015-04-23 23:29:05 +00:00
Richard Smith 2a553089c3 [modules] Properly attribute macros to modules if they're in a file textually included into a file in the module.
llvm-svn: 235661
2015-04-23 22:58:06 +00:00
Richard Smith 713369b057 [modules] Store a ModuleMacro* on an imported macro directive rather than duplicating the info within it.
llvm-svn: 235644
2015-04-23 20:40:50 +00:00
Richard Smith b8b2ed6529 [modules] Determine the set of macros exported by a submodule at the end of that submodule.
Previously we'd defer this determination until writing the AST, which doesn't
allow us to use this information when building other submodules of the same
module. This change also allows us to use a uniform mechanism for writing
module macro records, independent of whether they are local or imported.

llvm-svn: 235614
2015-04-23 18:18:26 +00:00
Richard Smith 447ed43cc1 [modules] Actually allocate the extra space we use for the tail-allocated array
in this class.

llvm-svn: 235570
2015-04-23 04:13:52 +00:00
Richard Smith 5dbef92932 [modules] Cope with partial module macro information, fix memory leak found by buildbot.
llvm-svn: 235464
2015-04-22 02:09:43 +00:00
Richard Smith e56c8bc30e [modules] Build a DAG of module macros for each identifier.
This graph will be used to determine the current set of active macros. This is
foundation work for getting macro visibility correct across submodules of the
current module. No functionality change for now.

llvm-svn: 235461
2015-04-22 00:26:11 +00:00
Will Wilson db2588ab82 [MSVC] Mimic MSVC whitespace collapse for incompatible token pasting
In public MS headers for XAudio, clang would fail to generate a valid UUID due to the UUID components being combined with the '-' UUID separators. Clang would attempting to recover but would preserve the leading whitespace from the tokens after each failed paste leading to spaces creeping into the UUID and causing an error in the __declspace(uuid()) parsing.

Reference: Microsoft DirectX SDK (June 2010)\Include\XAudio2.h(51)

Resolves http://llvm.org/pr23071

llvm-svn: 235186
2015-04-17 12:43:57 +00:00
Benjamin Kramer 040725723e MSan told me that we actually dump the entire scratch buffer into PCH files, initialize it.
Writing 4k of zeros is preferrable to 4k of random memory. Document that. While
there remove the initialization of the first byte of the buffer and start at
index zero. It was writing a literal '0' instead of a null byte at the
beginning anyways, which didn't matter since we never read it.

llvm-svn: 234202
2015-04-06 20:01:49 +00:00
Benjamin Kramer c7dd599e9d Prefer uninitialized memory for scratch space.
No functional change intended.

llvm-svn: 234184
2015-04-06 17:45:11 +00:00
Benjamin Kramer 0a126adc68 [lex] Provide a valid token when __has_include is found outside of a pp directive
ExpandBuiltinMacro would strip the identifier and downstream users crash
when they encounter an identifier token with nullptr identifier info.

Found by afl-fuzz.

llvm-svn: 233497
2015-03-29 19:05:27 +00:00
Benjamin Kramer 18ff02dd51 [lex] Don't create a garbage token if parsing of __has_include fails.
It will crash downstream somewhere. Found by afl-fuzz.

llvm-svn: 233493
2015-03-29 15:33:29 +00:00
Benjamin Kramer 8671028e95 [lex] Don't read past the end of the buffer
While dereferencing ThisTokEnd is fine and we know that it's not in
[a-zA-Z0-9_.], ThisTokEnd[1] is really past the end.

Found by asan and with a little help from clang-fuzz.

llvm-svn: 233491
2015-03-29 14:11:37 +00:00
Benjamin Kramer 7fd88386b0 [lex] Turn range checks into asserts.
We know that the last accessible char is not in [a-zA-Z0-9_.] so we can
happily scan on as long as it is. No functionality change.

llvm-svn: 233490
2015-03-29 14:11:22 +00:00
Richard Smith 8f4d3ff146 [modules] Restrict the module use-declaration to only appear in top-level
modules, and allow sub-modules of a module with a use-declaration to make use
of the nominated modules.

llvm-svn: 233323
2015-03-26 22:10:01 +00:00
David Blaikie 96cedb52b3 Make Oveflow tracking more legible (CR feedback from Richard Smith on r232999)
llvm-svn: 233006
2015-03-23 19:54:44 +00:00
David Blaikie 252f743858 Refactor: Simplify boolean expresssions in lib/Lex
Simplify boolean expressions using `true` and `false` with `clang-tidy`

Patch by Richard Thomson.

Differential Revision: http://reviews.llvm.org/D8531

llvm-svn: 232999
2015-03-23 19:39:19 +00:00
Yaron Keren 92e1b62d45 Remove many superfluous SmallString::str() calls.
Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.

There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while 
SmallString do not. We may wish to change this, but it may introduce
ambiguity.

llvm-svn: 232622
2015-03-18 10:17:07 +00:00
David Majnemer d8dee1f54b Lex: Don't call getIdentifierInfo on annotation tokens
These calls are usually guarded by checks for isAnnotation() but it
looks like we missed a spot.  This would cause the included test to
crash clang.

llvm-svn: 232616
2015-03-18 07:53:20 +00:00
Benjamin Kramer 15b9717873 Implement PreprocessingRecord's and LazyVector's iterators on top of iterator_adaptor_base
This basically creates a wrapper around an 'int' that poses as an iterator.
While that looks a bit counter-intuitive it works just fine because iterator
operations and basic integer arithmetic works in exactly the same way.

Remove the manual integer wrapping code and reduce the reliance on iterator
internals in the implementation. No functionality change intended.

llvm-svn: 232322
2015-03-15 15:27:19 +00:00
Daniel Jasper 0ab544f1a5 When building a module, all headers of submodules can be used.
This extends r232159.

llvm-svn: 232168
2015-03-13 14:29:39 +00:00
Daniel Jasper b6ba46ef30 Make a module "use" also count as use of all its submodules
llvm-svn: 232159
2015-03-13 11:26:16 +00:00
Yaron Keren 09fb7c6e7a Teach raw_ostream to accept SmallString.
Saves adding .str() call to any raw_ostream << SmallString usage
and a small step towards making .str() consistent in the ADTs by
removing one of the SmallString::str() use cases, discussion at

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20141013/240026.html

I'll update the Phabricator patch http://reviews.llvm.org/D6372
for review of the Twine SmallString support, it's more complex
than this one.

llvm-svn: 231763
2015-03-10 07:33:23 +00:00
Richard Smith 2708e52029 [modules] This check is run before we resolve the header, not after, so just
check that private headers are in a list matching the role. (We can't perform
the opposite checks for non-private headers because we infer those.)

llvm-svn: 231728
2015-03-10 00:19:04 +00:00
Richard Smith 00bc95ec9a [modules] Don't assert if the same header is named as both a public and a
private header within the same module.

llvm-svn: 231725
2015-03-09 23:46:50 +00:00
David Blaikie 7a3cbb2997 Simplify boolean expressions in clang with clang-tidy
Patch by Richard (legalize at xmission dot com).

Differential Revision: http://reviews.llvm.org/D8155

llvm-svn: 231619
2015-03-09 02:02:07 +00:00
Benjamin Kramer 3162f29fcd Properly initialize the parser_crash debug token.
Found by msan.

llvm-svn: 231605
2015-03-08 19:28:24 +00:00
Benjamin Kramer f04f98d543 Use delegating ctors to reduce code duplication. NFC.
llvm-svn: 231476
2015-03-06 14:15:57 +00:00
Argyrios Kyrtzidis 5c585253e5 [Modules] Fix crash in Preprocessor::getLastMacroWithSpelling().
Macro names that got undefined inside a module may not have their MacroInfo set.

llvm-svn: 231251
2015-03-04 16:03:07 +00:00
Bob Wilson b111ec94b3 Add clang support for Objective-C application extensions.
This adds the -fapplication-extension option, along with the
ios_app_extension and macosx_app_extension availability attributes.
Patch by Ted Kremenek

llvm-svn: 230989
2015-03-02 19:01:14 +00:00