maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/lib/AST/CommentParser.cpp

733 lines
22 KiB
C++
Raw Normal View History

Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
//===--- CommentParser.cpp - Doxygen comment parser -----------------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#include "clang/AST/CommentParser.h"
#include "clang/AST/CommentSema.h"
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
#include "clang/AST/CommentDiagnostic.h"
Comment parsing: extract TableGen'able pieces into new CommandTraits class. llvm-svn: 161548
2012-08-09 08:03:17 +08:00
#include "clang/AST/CommentCommandTraits.h"
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
#include "clang/Basic/SourceManager.h"
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
#include "llvm/Support/ErrorHandling.h"
namespace clang {
namespace comments {
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
/// Re-lexes a sequence of tok::text tokens.
class TextTokenRetokenizer {
llvm::BumpPtrAllocator &Allocator;
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
Parser &P;
Comment parsing: allow newlines between \param, direction specification (e.g., [in]), parameter name and description paragraph. llvm-svn: 160682
2012-07-25 02:23:31 +08:00
/// This flag is set when there are no more tokens we can fetch from lexer.
bool NoMoreInterestingTokens;
/// Token buffer: tokens we have processed and lookahead.
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
SmallVector<Token, 16> Toks;
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
Comment parsing: allow newlines between \param, direction specification (e.g., [in]), parameter name and description paragraph. llvm-svn: 160682
2012-07-25 02:23:31 +08:00
/// A position in \c Toks.
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
struct Position {
unsigned CurToken;
const char *BufferStart;
const char *BufferEnd;
const char *BufferPtr;
SourceLocation BufferStartLoc;
};
/// Current position in Toks.
Position Pos;
bool isEnd() const {
return Pos.CurToken >= Toks.size();
}
/// Sets up the buffer pointers to point to current token.
void setupBuffer() {
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
assert(!isEnd());
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
const Token &Tok = Toks[Pos.CurToken];
Pos.BufferStart = Tok.getText().begin();
Pos.BufferEnd = Tok.getText().end();
Pos.BufferPtr = Pos.BufferStart;
Pos.BufferStartLoc = Tok.getLocation();
}
SourceLocation getSourceLocation() const {
const unsigned CharNo = Pos.BufferPtr - Pos.BufferStart;
return Pos.BufferStartLoc.getLocWithOffset(CharNo);
}
char peek() const {
assert(!isEnd());
assert(Pos.BufferPtr != Pos.BufferEnd);
return *Pos.BufferPtr;
}
void consumeChar() {
assert(!isEnd());
assert(Pos.BufferPtr != Pos.BufferEnd);
Pos.BufferPtr++;
if (Pos.BufferPtr == Pos.BufferEnd) {
Pos.CurToken++;
Comment parsing: allow newlines between \param, direction specification (e.g., [in]), parameter name and description paragraph. llvm-svn: 160682
2012-07-25 02:23:31 +08:00
if (isEnd() && !addToken())
return;
assert(!isEnd());
setupBuffer();
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
}
}
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
/// Add a token.
/// Returns true on success, false if there are no interesting tokens to
/// fetch from lexer.
bool addToken() {
Comment parsing: allow newlines between \param, direction specification (e.g., [in]), parameter name and description paragraph. llvm-svn: 160682
2012-07-25 02:23:31 +08:00
if (NoMoreInterestingTokens)
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
return false;
Comment parsing: allow newlines between \param, direction specification (e.g., [in]), parameter name and description paragraph. llvm-svn: 160682
2012-07-25 02:23:31 +08:00
if (P.Tok.is(tok::newline)) {
// If we see a single newline token between text tokens, skip it.
Token Newline = P.Tok;
P.consumeToken();
if (P.Tok.isNot(tok::text)) {
P.putBack(Newline);
NoMoreInterestingTokens = true;
return false;
}
}
if (P.Tok.isNot(tok::text)) {
NoMoreInterestingTokens = true;
return false;
}
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
Toks.push_back(P.Tok);
P.consumeToken();
if (Toks.size() == 1)
setupBuffer();
return true;
}
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
static bool isWhitespace(char C) {
return C == ' ' || C == '\n' || C == '\r' ||
C == '\t' || C == '\f' || C == '\v';
}
void consumeWhitespace() {
while (!isEnd()) {
if (isWhitespace(peek()))
consumeChar();
else
break;
}
}
void formTokenWithChars(Token &Result,
SourceLocation Loc,
const char *TokBegin,
unsigned TokLength,
StringRef Text) {
Result.setLocation(Loc);
Result.setKind(tok::text);
Result.setLength(TokLength);
#ifndef NDEBUG
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
Result.TextPtr = "<UNSET>";
Result.IntVal = 7;
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
#endif
Result.setText(Text);
}
public:
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
TextTokenRetokenizer(llvm::BumpPtrAllocator &Allocator, Parser &P):
Comment parsing: allow newlines between \param, direction specification (e.g., [in]), parameter name and description paragraph. llvm-svn: 160682
2012-07-25 02:23:31 +08:00
Allocator(Allocator), P(P), NoMoreInterestingTokens(false) {
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
Pos.CurToken = 0;
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
addToken();
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
}
/// Extract a word -- sequence of non-whitespace characters.
bool lexWord(Token &Tok) {
if (isEnd())
return false;
Position SavedPos = Pos;
consumeWhitespace();
SmallString<32> WordText;
const char *WordBegin = Pos.BufferPtr;
SourceLocation Loc = getSourceLocation();
while (!isEnd()) {
const char C = peek();
if (!isWhitespace(C)) {
WordText.push_back(C);
consumeChar();
} else
break;
}
const unsigned Length = WordText.size();
if (Length == 0) {
Pos = SavedPos;
return false;
}
char *TextPtr = Allocator.Allocate<char>(Length + 1);
memcpy(TextPtr, WordText.c_str(), Length + 1);
StringRef Text = StringRef(TextPtr, Length);
formTokenWithChars(Tok, Loc, WordBegin,
Pos.BufferPtr - WordBegin, Text);
return true;
}
bool lexDelimitedSeq(Token &Tok, char OpenDelim, char CloseDelim) {
if (isEnd())
return false;
Position SavedPos = Pos;
consumeWhitespace();
SmallString<32> WordText;
const char *WordBegin = Pos.BufferPtr;
SourceLocation Loc = getSourceLocation();
bool Error = false;
if (!isEnd()) {
const char C = peek();
if (C == OpenDelim) {
WordText.push_back(C);
consumeChar();
} else
Error = true;
}
char C = '\0';
while (!Error && !isEnd()) {
C = peek();
WordText.push_back(C);
consumeChar();
if (C == CloseDelim)
break;
}
if (!Error && C != CloseDelim)
Error = true;
if (Error) {
Pos = SavedPos;
return false;
}
const unsigned Length = WordText.size();
char *TextPtr = Allocator.Allocate<char>(Length + 1);
memcpy(TextPtr, WordText.c_str(), Length + 1);
StringRef Text = StringRef(TextPtr, Length);
formTokenWithChars(Tok, Loc, WordBegin,
Pos.BufferPtr - WordBegin, Text);
return true;
}
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
/// Put back tokens that we didn't consume.
void putBackLeftoverTokens() {
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
if (isEnd())
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
return;
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
bool HavePartialTok = false;
Token PartialTok;
if (Pos.BufferPtr != Pos.BufferStart) {
formTokenWithChars(PartialTok, getSourceLocation(),
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
Pos.BufferPtr, Pos.BufferEnd - Pos.BufferPtr,
StringRef(Pos.BufferPtr,
Pos.BufferEnd - Pos.BufferPtr));
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
HavePartialTok = true;
Pos.CurToken++;
}
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
P.putBack(llvm::makeArrayRef(Toks.begin() + Pos.CurToken, Toks.end()));
Pos.CurToken = Toks.size();
if (HavePartialTok)
P.putBack(PartialTok);
Move TextTokenRetokenizer from CommentLexer.h to CommentParser.cpp since it is an implementation detail of the parser. llvm-svn: 160679
2012-07-25 01:43:18 +08:00
}
};
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
Parser::Parser(Lexer &L, Sema &S, llvm::BumpPtrAllocator &Allocator,
Comment parsing: extract TableGen'able pieces into new CommandTraits class. llvm-svn: 161548
2012-08-09 08:03:17 +08:00
const SourceManager &SourceMgr, DiagnosticsEngine &Diags,
const CommandTraits &Traits):
L(L), S(S), Allocator(Allocator), SourceMgr(SourceMgr), Diags(Diags),
Traits(Traits) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
consumeToken();
}
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
void Parser::parseParamCommandArgs(ParamCommandComment *PC,
TextTokenRetokenizer &Retokenizer) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
Token Arg;
// Check if argument looks like direction specification: [dir]
// e.g., [in], [out], [in,out]
if (Retokenizer.lexDelimitedSeq(Arg, '[', ']'))
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnParamCommandDirectionArg(PC,
Arg.getLocation(),
Arg.getEndLocation(),
Arg.getText());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
if (Retokenizer.lexWord(Arg))
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnParamCommandParamNameArg(PC,
Arg.getLocation(),
Arg.getEndLocation(),
Arg.getText());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
void Parser::parseTParamCommandArgs(TParamCommandComment *TPC,
TextTokenRetokenizer &Retokenizer) {
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
Token Arg;
if (Retokenizer.lexWord(Arg))
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnTParamCommandParamNameArg(TPC,
Arg.getLocation(),
Arg.getEndLocation(),
Arg.getText());
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
}
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
void Parser::parseBlockCommandArgs(BlockCommandComment *BC,
TextTokenRetokenizer &Retokenizer,
unsigned NumArgs) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
typedef BlockCommandComment::Argument Argument;
Stop using new[] on llvm::BumpPtrAllocator. llvm-svn: 159833
2012-07-07 00:41:59 +08:00
Argument *Args =
new (Allocator.Allocate<Argument>(NumArgs)) Argument[NumArgs];
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
unsigned ParsedArgs = 0;
Token Arg;
while (ParsedArgs < NumArgs && Retokenizer.lexWord(Arg)) {
Args[ParsedArgs] = Argument(SourceRange(Arg.getLocation(),
Arg.getEndLocation()),
Arg.getText());
ParsedArgs++;
}
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnBlockCommandArgs(BC, llvm::makeArrayRef(Args, ParsedArgs));
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
BlockCommandComment *Parser::parseBlockCommand() {
assert(Tok.is(tok::command));
ParamCommandComment *PC;
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
TParamCommandComment *TPC;
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
BlockCommandComment *BC;
bool IsParam = false;
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
bool IsTParam = false;
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
const CommandInfo *Info = Traits.getCommandInfo(Tok.getCommandID());
if (Info->IsParamCommand) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
IsParam = true;
PC = S.actOnParamCommandStart(Tok.getLocation(),
Tok.getEndLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
Tok.getCommandID());
} if (Info->IsTParamCommand) {
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
IsTParam = true;
TPC = S.actOnTParamCommandStart(Tok.getLocation(),
Tok.getEndLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
Tok.getCommandID());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
} else {
BC = S.actOnBlockCommandStart(Tok.getLocation(),
Tok.getEndLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
Tok.getCommandID());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
consumeToken();
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
if (Tok.is(tok::command) &&
Traits.getCommandInfo(Tok.getCommandID())->IsBlockCommand) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
// Block command ahead. We can't nest block commands, so pretend that this
// command has an empty argument.
Comment parser: don't crash on a completely empty \param followed by a block command llvm-svn: 160975
2012-07-31 02:05:28 +08:00
ParagraphComment *Paragraph = S.actOnParagraphComment(
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
ArrayRef<InlineContentComment *>());
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
if (IsParam) {
Comment parsing: fix crash on \tparam followed immediately by another block command, for example: \tparam\brief. llvm-svn: 161361
2012-08-07 07:48:44 +08:00
S.actOnParamCommandFinish(PC, Paragraph);
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
return PC;
Comment parsing: fix crash on \tparam followed immediately by another block command, for example: \tparam\brief. llvm-svn: 161361
2012-08-07 07:48:44 +08:00
} else if (IsTParam) {
S.actOnTParamCommandFinish(TPC, Paragraph);
return TPC;
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
} else {
S.actOnBlockCommandFinish(BC, Paragraph);
return BC;
}
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
if (IsParam || IsTParam || Info->NumArgs > 0) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
// In order to parse command arguments we need to retokenize a few
// following text tokens.
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
TextTokenRetokenizer Retokenizer(Allocator, *this);
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
if (IsParam)
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
parseParamCommandArgs(PC, Retokenizer);
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
else if (IsTParam)
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
parseTParamCommandArgs(TPC, Retokenizer);
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
else
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
parseBlockCommandArgs(BC, Retokenizer, Info->NumArgs);
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
Retokenizer.putBackLeftoverTokens();
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
BlockContentComment *Block = parseParagraphOrBlockCommand();
// Since we have checked for a block command, we should have parsed a
// paragraph.
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
ParagraphComment *Paragraph = cast<ParagraphComment>(Block);
if (IsParam) {
S.actOnParamCommandFinish(PC, Paragraph);
return PC;
} else if (IsTParam) {
S.actOnTParamCommandFinish(TPC, Paragraph);
return TPC;
} else {
S.actOnBlockCommandFinish(BC, Paragraph);
return BC;
}
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
InlineCommandComment *Parser::parseInlineCommand() {
assert(Tok.is(tok::command));
const Token CommandTok = Tok;
consumeToken();
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
TextTokenRetokenizer Retokenizer(Allocator, *this);
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
Token ArgTok;
bool ArgTokValid = Retokenizer.lexWord(ArgTok);
InlineCommandComment *IC;
if (ArgTokValid) {
IC = S.actOnInlineCommand(CommandTok.getLocation(),
CommandTok.getEndLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
CommandTok.getCommandID(),
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
ArgTok.getLocation(),
ArgTok.getEndLocation(),
ArgTok.getText());
} else {
IC = S.actOnInlineCommand(CommandTok.getLocation(),
CommandTok.getEndLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
CommandTok.getCommandID());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
Comment parsing: couple TextTokenRetokenizer and comment parser together to remove one of the two variable-length lookahead buffers. Now retokenizer will ask for more tokens when it needs them. llvm-svn: 160680
2012-07-25 01:52:18 +08:00
Retokenizer.putBackLeftoverTokens();
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
return IC;
}
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
HTMLStartTagComment *Parser::parseHTMLStartTag() {
assert(Tok.is(tok::html_start_tag));
HTMLStartTagComment *HST =
S.actOnHTMLStartTagStart(Tok.getLocation(),
Tok.getHTMLTagStartName());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
consumeToken();
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
SmallVector<HTMLStartTagComment::Attribute, 2> Attrs;
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
while (true) {
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
switch (Tok.getKind()) {
case tok::html_ident: {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
Token Ident = Tok;
consumeToken();
if (Tok.isNot(tok::html_equals)) {
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
Attrs.push_back(HTMLStartTagComment::Attribute(Ident.getLocation(),
Ident.getHTMLIdent()));
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
continue;
}
Token Equals = Tok;
consumeToken();
if (Tok.isNot(tok::html_quoted_string)) {
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
Diag(Tok.getLocation(),
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
diag::warn_doc_html_start_tag_expected_quoted_string)
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
<< SourceRange(Equals.getLocation());
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
Attrs.push_back(HTMLStartTagComment::Attribute(Ident.getLocation(),
Ident.getHTMLIdent()));
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
while (Tok.is(tok::html_equals) ||
Tok.is(tok::html_quoted_string))
consumeToken();
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
continue;
}
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
Attrs.push_back(HTMLStartTagComment::Attribute(
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
Ident.getLocation(),
Ident.getHTMLIdent(),
Equals.getLocation(),
SourceRange(Tok.getLocation(),
Tok.getEndLocation()),
Tok.getHTMLQuotedString()));
consumeToken();
continue;
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
}
case tok::html_greater:
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnHTMLStartTagFinish(HST,
S.copyArray(llvm::makeArrayRef(Attrs)),
Tok.getLocation(),
/* IsSelfClosing = */ false);
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
consumeToken();
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
return HST;
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
case tok::html_slash_greater:
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnHTMLStartTagFinish(HST,
S.copyArray(llvm::makeArrayRef(Attrs)),
Tok.getLocation(),
/* IsSelfClosing = */ true);
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
consumeToken();
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
return HST;
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
case tok::html_equals:
case tok::html_quoted_string:
Diag(Tok.getLocation(),
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
diag::warn_doc_html_start_tag_expected_ident_or_greater);
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
while (Tok.is(tok::html_equals) ||
Tok.is(tok::html_quoted_string))
consumeToken();
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
if (Tok.is(tok::html_ident) ||
Tok.is(tok::html_greater) ||
Tok.is(tok::html_slash_greater))
continue;
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnHTMLStartTagFinish(HST,
S.copyArray(llvm::makeArrayRef(Attrs)),
SourceLocation(),
/* IsSelfClosing = */ false);
return HST;
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
default:
// Not a token from an HTML start tag. Thus HTML tag prematurely ended.
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnHTMLStartTagFinish(HST,
S.copyArray(llvm::makeArrayRef(Attrs)),
SourceLocation(),
/* IsSelfClosing = */ false);
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
bool StartLineInvalid;
const unsigned StartLine = SourceMgr.getPresumedLineNumber(
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
HST->getLocation(),
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
&StartLineInvalid);
bool EndLineInvalid;
const unsigned EndLine = SourceMgr.getPresumedLineNumber(
Tok.getLocation(),
&EndLineInvalid);
if (StartLineInvalid || EndLineInvalid || StartLine == EndLine)
Diag(Tok.getLocation(),
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
diag::warn_doc_html_start_tag_expected_ident_or_greater)
<< HST->getSourceRange();
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
else {
Diag(Tok.getLocation(),
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
diag::warn_doc_html_start_tag_expected_ident_or_greater);
Diag(HST->getLocation(), diag::note_doc_html_tag_started_here)
<< HST->getSourceRange();
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
}
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
return HST;
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
}
}
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
HTMLEndTagComment *Parser::parseHTMLEndTag() {
assert(Tok.is(tok::html_end_tag));
Token TokEndTag = Tok;
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
consumeToken();
SourceLocation Loc;
if (Tok.is(tok::html_greater)) {
Loc = Tok.getLocation();
consumeToken();
}
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
return S.actOnHTMLEndTag(TokEndTag.getLocation(),
Loc,
TokEndTag.getHTMLTagEndName());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
BlockContentComment *Parser::parseParagraphOrBlockCommand() {
SmallVector<InlineContentComment *, 8> Content;
while (true) {
switch (Tok.getKind()) {
case tok::verbatim_block_begin:
case tok::verbatim_line_name:
case tok::eof:
assert(Content.size() != 0);
break; // Block content or EOF ahead, finish this parapgaph.
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
case tok::unknown_command:
Content.push_back(S.actOnUnknownCommand(Tok.getLocation(),
Tok.getEndLocation(),
Tok.getUnknownCommandName()));
consumeToken();
continue;
case tok::command: {
const CommandInfo *Info = Traits.getCommandInfo(Tok.getCommandID());
if (Info->IsBlockCommand) {
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
if (Content.size() == 0)
return parseBlockCommand();
break; // Block command ahead, finish this parapgaph.
}
Comment parsing: handle non-builtin commands correctly. After semantic analysis registers a command, it becomes a "known" command for the lexer, since it has an ID. Having this freedom of choice to register a command is a good thing since BriefParser does not need this. But the parser should still invoke the correct semantic analysis method (actOnUnknownCommand) in this case. llvm-svn: 163646
2012-09-12 03:22:03 +08:00
if (Info->IsUnknownCommand) {
Content.push_back(S.actOnUnknownCommand(Tok.getLocation(),
Tok.getEndLocation(),
Info->getID()));
consumeToken();
continue;
}
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
assert(Info->IsInlineCommand);
Content.push_back(parseInlineCommand());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
continue;
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
}
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
case tok::newline: {
consumeToken();
if (Tok.is(tok::newline) || Tok.is(tok::eof)) {
consumeToken();
break; // Two newlines -- end of paragraph.
}
if (Content.size() > 0)
Content.back()->addTrailingNewline();
continue;
}
// Don't deal with HTML tag soup now.
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
case tok::html_start_tag:
Content.push_back(parseHTMLStartTag());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
continue;
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
case tok::html_end_tag:
Content.push_back(parseHTMLEndTag());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
continue;
case tok::text:
Content.push_back(S.actOnText(Tok.getLocation(),
Tok.getEndLocation(),
Tok.getText()));
consumeToken();
continue;
case tok::verbatim_block_line:
case tok::verbatim_block_end:
case tok::verbatim_line_text:
case tok::html_ident:
case tok::html_equals:
case tok::html_quoted_string:
case tok::html_greater:
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
case tok::html_slash_greater:
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
llvm_unreachable("should not see this token");
}
break;
}
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
return S.actOnParagraphComment(S.copyArray(llvm::makeArrayRef(Content)));
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
VerbatimBlockComment *Parser::parseVerbatimBlock() {
assert(Tok.is(tok::verbatim_block_begin));
VerbatimBlockComment *VB =
S.actOnVerbatimBlockStart(Tok.getLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
Tok.getVerbatimBlockID());
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
consumeToken();
// Don't create an empty line if verbatim opening command is followed
// by a newline.
if (Tok.is(tok::newline))
consumeToken();
SmallVector<VerbatimBlockLineComment *, 8> Lines;
while (Tok.is(tok::verbatim_block_line) ||
Tok.is(tok::newline)) {
VerbatimBlockLineComment *Line;
if (Tok.is(tok::verbatim_block_line)) {
Line = S.actOnVerbatimBlockLine(Tok.getLocation(),
Tok.getVerbatimBlockText());
consumeToken();
if (Tok.is(tok::newline)) {
consumeToken();
}
} else {
// Empty line, just a tok::newline.
This fits into 80 columns without wrapping. llvm-svn: 160453
2012-07-19 05:27:38 +08:00
Line = S.actOnVerbatimBlockLine(Tok.getLocation(), "");
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
consumeToken();
}
Lines.push_back(Line);
}
Fix PR13411: Comment parsing: failed assertion on unterminated verbatim block. The assertion was wrong in case we have a verbatim block without a closing command. Also add tests for closing command name in a verbatim block, since now it can be empty in such cases. llvm-svn: 160568
2012-07-21 04:18:53 +08:00
if (Tok.is(tok::verbatim_block_end)) {
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
const CommandInfo *Info = Traits.getCommandInfo(Tok.getVerbatimBlockID());
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnVerbatimBlockFinish(VB, Tok.getLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
Info->Name,
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.copyArray(llvm::makeArrayRef(Lines)));
Fix PR13411: Comment parsing: failed assertion on unterminated verbatim block. The assertion was wrong in case we have a verbatim block without a closing command. Also add tests for closing command name in a verbatim block, since now it can be empty in such cases. llvm-svn: 160568
2012-07-21 04:18:53 +08:00
consumeToken();
} else {
// Unterminated \\verbatim block
Comment parser and sema: remove useless return values llvm-svn: 161331
2012-08-07 03:03:12 +08:00
S.actOnVerbatimBlockFinish(VB, SourceLocation(), "",
S.copyArray(llvm::makeArrayRef(Lines)));
Fix PR13411: Comment parsing: failed assertion on unterminated verbatim block. The assertion was wrong in case we have a verbatim block without a closing command. Also add tests for closing command name in a verbatim block, since now it can be empty in such cases. llvm-svn: 160568
2012-07-21 04:18:53 +08:00
}
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
return VB;
}
VerbatimLineComment *Parser::parseVerbatimLine() {
assert(Tok.is(tok::verbatim_line_name));
Token NameTok = Tok;
consumeToken();
SourceLocation TextBegin;
StringRef Text;
// Next token might not be a tok::verbatim_line_text if verbatim line
// starting command comes just before a newline or comment end.
if (Tok.is(tok::verbatim_line_text)) {
TextBegin = Tok.getLocation();
Text = Tok.getVerbatimLineText();
} else {
TextBegin = NameTok.getEndLocation();
Text = "";
}
VerbatimLineComment *VL = S.actOnVerbatimLine(NameTok.getLocation(),
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
NameTok.getVerbatimLineID(),
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
TextBegin,
Text);
consumeToken();
return VL;
}
BlockContentComment *Parser::parseBlockContent() {
switch (Tok.getKind()) {
case tok::text:
Comment AST: TableGen'ize all command lists in CommentCommandTraits.cpp. Now we have a list of all commands. This is a good thing in itself, but it also enables us to easily implement typo correction for command names. With this change we have objects that contain information about each command, so it makes sense to resolve command name just once during lexing (currently we store command names as strings and do a linear search every time some property value is needed). Thus comment token and AST nodes were changed to contain a command ID -- index into a tables of builtin and registered commands. Unknown commands are registered during parsing and thus are also uniformly assigned an ID. Using an ID instead of a StringRef is also a nice memory optimization since ID is a small integer that fits into a common bitfield in Comment class. This change implies that to get any information about a command (even a command name) we need a CommandTraits object to resolve the command ID to CommandInfo*. Currently a fresh temporary CommandTraits object is created whenever it is needed since it does not have any state. But with this change it has state -- new commands can be registered, so a CommandTraits object was added to ASTContext. Also, in libclang CXComment has to be expanded to include a CXTranslationUnit so that all functions working on comment AST nodes can get a CommandTraits object. This breaks binary compatibility of CXComment APIs. Now clang_FullComment_getAsXML(CXTranslationUnit TU, CXComment CXC) doesn't need TU parameter anymore, so it was removed. This is a source-incompatible change for this C API. llvm-svn: 163540
2012-09-11 04:32:42 +08:00
case tok::unknown_command:
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
case tok::command:
Comment parsing: repaint the bikesched: rename 'HTML open tags' to 'HTML start tags' and 'HTML close tags' to 'HTML end tags' according to HTML spec. llvm-svn: 160153
2012-07-13 08:44:24 +08:00
case tok::html_start_tag:
case tok::html_end_tag:
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
return parseParagraphOrBlockCommand();
case tok::verbatim_block_begin:
return parseVerbatimBlock();
case tok::verbatim_line_name:
return parseVerbatimLine();
case tok::eof:
case tok::newline:
case tok::verbatim_block_line:
case tok::verbatim_block_end:
case tok::verbatim_line_text:
case tok::html_ident:
case tok::html_equals:
case tok::html_quoted_string:
case tok::html_greater:
Enable comment parsing and semantic analysis to emit diagnostics. A few diagnostics implemented -- see testcases. I created a new TableGen file for comment diagnostics, DiagnosticCommentKinds.td, because comment diagnostics don't logically fit into AST diagnostics file. But I don't feel strongly about it. This also implements support for self-closing HTML tags in comment lexer and parser (for example, <br />). In order to issue precise diagnostics CommentSema needs to know the declaration the comment is attached to. There is no easy way to find a decl by comment, so we match comments and decls in lockstep: after parsing one declgroup we check if we have any new, not yet attached comments. If we do -- then we do the usual comment-finding process. It is interesting that this automatically handles trailing comments. We pick up not only comments that precede the declaration, but also comments that *follow* the declaration -- thanks to the lookahead in the lexer: after parsing the declgroup we've consumed the semicolon and looked ahead through comments. Added -Wdocumentation-html flag for semantic HTML errors to allow the user to disable only HTML warnings (but not HTML parse errors, which we emit as warnings in -Wdocumentation). llvm-svn: 160078
2012-07-12 05:38:39 +08:00
case tok::html_slash_greater:
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
llvm_unreachable("should not see this token");
}
Sprinkle llvm_unreachable around to placate GCC's -Wreturn-type. llvm-svn: 159860
2012-07-07 05:13:09 +08:00
llvm_unreachable("bogus token kind");
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
FullComment *Parser::parseFullComment() {
// Skip newlines at the beginning of the comment.
while (Tok.is(tok::newline))
consumeToken();
SmallVector<BlockContentComment *, 8> Blocks;
while (Tok.isNot(tok::eof)) {
Blocks.push_back(parseBlockContent());
// Skip extra newlines after paragraph end.
while (Tok.is(tok::newline))
consumeToken();
}
Comment parsing: add support for \tparam command on all levels. The only caveat is renumbering CXCommentKind enum for aesthetic reasons -- this breaks libclang binary compatibility, but should not be a problem since API is so new. This also fixes PR13372 as a side-effect. llvm-svn: 161087
2012-08-01 06:37:06 +08:00
return S.actOnFullComment(S.copyArray(llvm::makeArrayRef(Blocks)));
Implement AST classes for comments, a real parser for Doxygen comments and a very simple semantic analysis that just builds the AST; minor changes for lexer to pick up source locations I didn't think about before. Comments AST is modelled along the ideas of HTML AST: block and inline content. * Block content is a paragraph or a command that has a paragraph as an argument or verbatim command. * Inline content is placed within some block. Inline content includes plain text, inline commands and HTML as tag soup. llvm-svn: 159790
2012-07-06 08:28:32 +08:00
}
} // end namespace comments
} // end namespace clang