right number of newlines between tokens when needed. This reduces the
delta of the gcc -E output from 12198 differences to 6764. It still needs
to emit filenames on #line directives, track filename switches, and track
entry/exit of include files.
llvm-svn: 38559
came from a macro expansion, this allows us to keep track of both where the
character data came from and where the logical position of the token is (at
the expansion site). This implements Preprocessor/indent_macro.c, and
reduces the number of cpp iostream -E diffs vs GCC from 2589 to 2297.
llvm-svn: 38557
Now, instead of keeping a pointer to the start of the token in memory, we keep the
start of the token as a SourceLocation node. This means that each LexerToken knows
the full include stack it was created with, and means that the LexerToken isn't
reliant on a "CurLexer" member to be around (lexer tokens would previously go out of
scope when their lexers were deallocated).
This simplifies several things, and forces good cleanup elsewhere. Now the
Preprocessor is the one that knows how to dump tokens/macros and is the one that
knows how to get the spelling of a token (it has all the context).
llvm-svn: 38551