Summary:
Detect if the file is valid UTF-8, and if this is the case, count code
points instead of just using number of bytes in all (hopefully) places, where
number of columns is needed. In particular, use the new
FormatToken.CodePointCount instead of TokenLength where appropriate.
Changed BreakableToken implementations to respect utf-8 character boundaries
when in utf-8 mode.
Reviewers: klimek, djasper
Reviewed By: djasper
CC: cfe-commits, rsmith, gribozavr
Differential Revision: http://llvm-reviews.chandlerc.com/D918
llvm-svn: 183312
An oversight in this detection made clang-format unable to format
the following nicely:
void aaaaaaaaaaaaaaaaaaa<aaaaaaaaaaaaaaaaaaaaaaaaaaa,
bbbbbbbbbbbbbbbbbbbbbbbbbb>(
cccccccccccccccccccccccccccc);
llvm-svn: 183097
Before, clang-format would not find a solution for formatting:
if ((aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ||
bbbbbbbbbbbbbbbbbb) && // aaaaaaaaaaaaaaaa
cccccc) {
}
llvm-svn: 183096
If a "}" is found inside parenthesis, this is probably a case of
missing parenthesis. This enables continuing to format after stuff code
like:
class A {
void f(
};
..
llvm-svn: 183009
With this patch, the simplified rule is:
If the block is part of a declaration (class, namespace, function,
enum, ..), merge an empty block onto a single line. Otherwise
(specifically for the compound statements of if, for, while, ...),
keep the braces on two separate lines.
The reasons are:
- Mostly the formatting of empty blocks does not matter much.
- Empty compound statements are really rare and are usually just
inserted while still working on the code. If they are on two lines,
inserting code is easier. Also, overlooking the "{}" of an
"if (...) {}" can be really bad.
- Empty declarations are not uncommon, e.g. empty constructors. Putting
them on one line saves vertical space at no loss of readability.
llvm-svn: 183008
When trying to fall back to search from the end onwards, we
would still find leading whitespace if the leading whitespace
went on after the end of the line.
llvm-svn: 182886
Now that the TokenAnnotator does not require stack space anymore,
reconstructing the lines has become the limiting factor. This patch
fixes that problem, allowing large files with multiple megabytes of
single unwrapped lines to be formatted.
llvm-svn: 182861
Gets rid of AnnotatedToken, putting everything into FormatToken.
FormatTokens are created once, and only referenced by pointer. This
enables multiple future features, like having tokens shared between
multiple UnwrappedLines (while there's still work to do to fully enable
that).
llvm-svn: 182859
If an identifier is on its own line and it is all upper case, it is highly
likely that this is a macro that is meant to stand on a line by itself.
Before:
class A : public QObject {
Q_OBJECT A() {}
};
Ater:
class A : public QObject {
Q_OBJECT
A() {}
};
llvm-svn: 182855
With option enabled (e.g. in Google-style):
template <typename T>
void f() {}
With option disabled:
template <typename T> void f() {}
Enabling this for Google-style and Chromium-style, not sure which other
styles would prefer that.
llvm-svn: 182849
This made it necessary to remove an error detection which would let us
bail out of braced lists in certain situations of missing "}". However,
as we always entirely escape from the braced list on finding ";", this
should not be a big problem.
With this, we can no format braced lists with uniformat inits:
return { arg1, SomeType { parameter } };
llvm-svn: 182788
With this patch, we create all tokens in one go before parsing and pass
an ArrayRef<FormatToken*> to the UnwrappedLineParser. The
UnwrappedLineParser is switched to use pointer-to-token internally.
The UnwrappedLineParser still copies the tokens into the UnwrappedLines.
This will be fixed in an upcoming patch.
llvm-svn: 182768
To fully support this, we also need to expand tabs in the text before
the block comment. This patch breaks indentation when there was a
non-standard mixture of spaces and tabs used for indentation, but
fixes a regression in the simple case:
{
/*
* Comment.
*/
int i;
}
Is now formatted correctly, if there were tabs used for indentation
before.
llvm-svn: 182760
Block comment indentation of empty lines regressed, as we did not
have a test for it.
/* Comment with...
*
* empty line. */
is now formatted correctly again.
llvm-svn: 182757
Before:
int (*func)(void*);
void f() { int(*func)(void*); }
After (consistent space after "int"):
int (*func)(void*);
void f() { int (*func)(void*); }
llvm-svn: 182756
This gets turned into two ">" operators at the beginning in order to
simplify template parameter handling. Thus, we need a special case to
handle those two binary operators correctly.
With this patch, clang-format can now correctly handle cases like:
aaaaaa = aaaaaaa(aaaaaaa, // break
aaaaaa) >>
bbbbbb;
llvm-svn: 182754
Unify handling of whitespace when breaking protruding tokens with other
whitespace replacements.
As a side effect, the BreakableToken structure changed significantly:
- have a common base class for single-line breakable tokens, as they are
much more similar
- revamp handling of multi-line comments; we now calculate the
information about lines in multi-line comments similar to normal
tokens, and always issue replacements
As a result, we were able to get rid of special casing of trailing
whitespace deletion for comments in the whitespace manager and the
BreakableToken and fixed bugs related to tab handling and escaped
newlines.
llvm-svn: 182738
In general, we like to avoid line breaks like:
...
SomeParameter, OtherParameter).DoSomething(
...
as they tend to make code really hard to read (how would you even indent the
next line?). Previously we have implemented this in a hacky way, which has now
shown to lead to problems. This fixes a few weird looking formattings, such as:
Before:
aaaaa(
aaaaa(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa, aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)
.aaaaa(aaaaa),
aaaaaaaaaaaaaaaaaaaaa);
After:
aaaaa(aaaaa(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa,
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa).aaaaa(aaaaa),
aaaaaaaaaaaaaaaaaaaaa);
llvm-svn: 182731
Before:
@{ NSFontAttributeNameeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
: regularFont, };
Now:
@{ NSFontAttributeNameeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee :
regularFont, };
':'s in dictionary literals (and the corresponding {}s) are now marked as
TT_ObjCDictLiteral too, which makes further improvements to dict literal
layout possible.
llvm-svn: 182716
Previously we started sequences to align for single line comments when
the previous line had a trailing comment, but the sequence was broken
for other reasons.
Now we re-format:
// a
// b
f(); // c
to:
// a
// b
f(); // c
llvm-svn: 182608
Now correctly leaves:
f(); // comment
// comment
g(); // comment
... alone if the middle comment was aligned with g() before formatting.
llvm-svn: 182605
Previously we would align:
f(); // comment
// other comment
g();
Even if // other comment was at the start of the line. Now we do not
align trailing comments if they have been already aligned correctly
with the next line.
Thus,
f(); // comment
// other comment
g();
will not be changed, while:
f(); // comment
// other commment
g();
will lead to the two trailing comments being aligned.
llvm-svn: 182577
Replaces the use of WhitespaceStart + WhitspaceLength.
This made a bug in the formatter obvous where we would incorrectly
calculate the next column.
FIXME: There's a similar bug left regarding TokenLength. We should
probably also move to have a TokenRange instead.
llvm-svn: 182572
Before:
vector<int> x { 1, 2, 3 };
After:
vector<int> x{ 1, 2, 3 };
Also add a style option to remove the spaces inside braced lists,
so that the above becomes:
std::vector<int> v{1, 2, 3};
llvm-svn: 182570
Allows formatting of C++11 braced init list constructs, like:
vector<int> v { 1, 2, 3 };
f({ 1, 2 });
This involves some changes of how tokens are handled in the
UnwrappedLineFormatter. Note that we have a plan to evolve the
design of the token flow into one where we create all tokens
up-front and then annotate them in the various layers (as we
currently already have to create all tokens at once anyway, the
current abstraction does not help). Thus, this introduces
FIXMEs towards that goal.
llvm-svn: 182568
Daniel Jasper