Commit Graph

677 Commits

Author SHA1 Message Date
Sam McCall 091557d2a8 [clangd] BindWithForward -> Bind. NFC
llvm-svn: 325868
2018-02-23 07:54:17 +00:00
Sam McCall b1ac64927f [clangd] fix test use-after-free from r325774
llvm-svn: 325801
2018-02-22 15:33:33 +00:00
Simon Marchi 5178f924c4 [clangd] DidChangeConfiguration Notification
Summary:

Implementation of DidChangeConfiguration notification handling in
clangd.  This currently only supports changing one setting: the path of
the compilation database to be used for the current project.   In other
words, it is no longer necessary to restart clangd with a different
command line argument in order to change the compilation database.

Reviewers: malaperle, krasimir, bkramer, ilya-biryukov

Subscribers: jkorous-apple, ioeric, simark, klimek, ilya-biryukov, arphaman, rwols, cfe-commits

Differential Revision: https://reviews.llvm.org/D39571

Signed-off-by: Simon Marchi <simon.marchi@ericsson.com>
Signed-off-by: William Enright <william.enright@polymtl.ca>
llvm-svn: 325784
2018-02-22 14:00:39 +00:00
Sam McCall 568e17f1e6 [clangd] Allow embedders some control over when diagnostics are generated.
Summary:
Through the C++ API, we support for a given snapshot version:
 - Yes: make sure we generate diagnostics for exactly this version
 - Auto: generate eventually-consistent diagnostics for at least this version
 - No: don't generate diagnostics for this version
Eventually auto should be debounced for better UX.

Through LSP, we force diagnostics for initial load (bypassing future debouncing)
and all updates follow the "auto" policy.

This is complicated to implement under the CancellationFlag design, so
rewrote that part to just inspect the queue instead.

It turns out we never pass None to the diagnostics callback, so remove Optional
from the signature. The questionable behavior of not invoking the callback at
all if CppFile::rebuild fails is not changed.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43518

llvm-svn: 325774
2018-02-22 13:11:12 +00:00
Eric Liu 02ce01f1e8 [clangd] Not collect include headers for dynamic index for now.
Summary:
The new behaviors introduced by this patch:
o When include collection is enabled, we always set IncludeHeader field in Symbol
even if it's the same as FileURI in decl.
o Disable include collection in FileIndex which is currently only used to build
dynamic index. We should revisit when we actually want to use FileIndex to global
index.
o Code-completion only uses IncludeHeader to insert headers but not FileURI in
CanonicalDeclaration. This ensures that inserted headers are always canonicalized.
Note that include insertion can still be triggered for symbols that are already
included if they are merged from dynamic index and static index, but we would
only use includes that are already canonicalized (e.g. from static index).

Reason for change:
Collecting header includes in dynamic index enables inserting includes for headers
that are not indexed but opened in the editor. Comparing to inserting includes for
symbols in global/static index, this is nice-to-have but would probably require
non-trivial amount of work to get right. For example:
o Currently it's not easy to fully support CanonicalIncludes in dynamic index, given the way
we run dynamic index.
o It's also harder to reason about the correctness of include canonicalization for dynamic index
(i.e. symbols in the current file/TU) than static index where symbols are collected
offline and sanity check is possible before shipping to production.
o We have less control/flexibility over symbol info in the dynamic index
(e.g. URIs, path normalization), which could be used to help make decision when inserting includes.

As header collection (especially canonicalization) is relatively new, and enabling
it for dynamic index would immediately affect current users with only dynamic
index support, I propose we disable it for dynamic index for now to avoid
compromising other hot features like code completion and only support it for
static index where include insertion would likely to bring more value.

Reviewers: ilya-biryukov, sammccall, hokein

Subscribers: klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43550

llvm-svn: 325764
2018-02-22 10:14:05 +00:00
Marc-Andre Laperle 63a1098d73 [clangd] #include statements support for Open definition
Summary: ctrl-clicking on #include statements now opens the file being pointed by that statement.

Reviewers: malaperle, krasimir, bkramer, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: jkorous-apple, ioeric, mgrang, klimek, ilya-biryukov, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D38639

llvm-svn: 325662
2018-02-21 02:39:08 +00:00
Eric Liu 709bde886d [clangd] Fixes for #include insertion.
Summary:
o Avoid inserting a header include into the header itself.
o Avoid inserting non-header files (by not indexing symbols in main
files at all).
o Canonicalize include paths for symbols in dynamic index.

Reviewers: sammccall, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43462

llvm-svn: 325523
2018-02-19 18:48:44 +00:00
Ilya Biryukov 7fac6e92a7 [clangd] Do not reuse preamble if compile args changed
Reviewers: hokein, ioeric, sammccall, simark

Reviewed By: simark

Subscribers: klimek, jkorous-apple, cfe-commits, simark

Differential Revision: https://reviews.llvm.org/D43454

llvm-svn: 325522
2018-02-19 18:18:49 +00:00
Sam McCall ab8e393f62 [clangd] Invert return value of fuzzyFind() (fix MemIndex's return value)
Have had way too many bugs by converting between "isComplete" and
"isIncomplete". LSP is immovable, so use isIncomplete everywhere.

llvm-svn: 325493
2018-02-19 13:04:41 +00:00
Sam McCall c901c5db78 [clangd] Tracing: name worker threads, and enforce naming scheduled async tasks
Summary:
This has a bit of a blast radius, but I think there's enough value in "forcing"
us to give names to these async tasks for debugging. Guessing about what
multithreaded code is doing is so unfun...

The "file" param attached to the tasks may seem to be redundant with the thread
names, but note that thread names are truncated to 15 chars on linux!
We'll be lucky to get the whole basename...

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43388

llvm-svn: 325480
2018-02-19 09:56:28 +00:00
Heejin Ahn f4a29252fb [clangd] Fix link failures for Preprocessor::addCommentHandler
Summary:
D42640 adds calls to `Preprocessor::addCommentHandler` in
`unittests/clangd/SymbolCollectorTests.cpp` and
`clangd/global-symbol-builder/GlobalSymbolBuilderMain.cpp` but does not
link `clangLex` library. This causes undefined reference errors when
built with `-DBUILD_SHARED_LIBS=ON`.

Reviewers: ioeric

Subscribers: klimek, mgorny, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43437

llvm-svn: 325458
2018-02-18 10:50:16 +00:00
Marc-Andre Laperle 373e30a264 [clangd] Rename some protocol field to lower case
Summary:
Also fixes a GCC compilation error.

Signed-off-by: Marc-Andre Laperle <marc-andre.laperle@ericsson.com>

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43411

llvm-svn: 325409
2018-02-16 23:12:26 +00:00
Marc-Andre Laperle 3e618ed8f0 [clangd] Implement textDocument/hover
Summary: Implemention of textDocument/hover as described in LSP definition.

This patch adds a basic Hover implementation.  When hovering a variable,
function, method or namespace, clangd will return a text containing the
declaration's scope, as well as the declaration of the hovered entity.
For example, for a variable:

  Declared in class Foo::Bar

  int hello = 2

For macros, the macro definition is returned.

This patch doesn't include:

- markdown support (the client I use doesn't support it yet)
- range support (optional in the Hover response)
- comments associated to variables/functions/classes

They are kept as future work to keep this patch simpler.

I added tests in XRefsTests.cpp.  hover.test contains one simple
smoketest to make sure the feature works from a black box perspective.

Reviewers: malaperle, krasimir, bkramer, ilya-biryukov

Subscribers: sammccall, mgrang, klimek, rwols, ilya-biryukov, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D35894

Signed-off-by: Simon Marchi <simon.marchi@ericsson.com>
Signed-off-by: William Enright <william.enright@polymtl.ca>
llvm-svn: 325395
2018-02-16 21:38:15 +00:00
Eric Liu c5105f9e3c [clangd] collect symbol #include & insert #include in global code completion.
Summary:
o Collect suitable #include paths for index symbols. This also does smart mapping
for STL symbols and IWYU pragma (code borrowed from include-fixer).
o For global code completion, add a command for inserting new #include in each code
completion item.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, jkorous-apple, hintonda, cfe-commits

Differential Revision: https://reviews.llvm.org/D42640

llvm-svn: 325343
2018-02-16 14:15:55 +00:00
Sam McCall c156806844 [clangd] TestFS cleanup: getVirtualBlahBlah -> testPath/testRoot. Remove SmallString micro-opt for more ergonomic tests. NFC
llvm-svn: 325326
2018-02-16 09:41:43 +00:00
Ilya Biryukov 40bf17c1ea [clangd] Fixed compilation with MVSC.
llvm-svn: 325254
2018-02-15 15:41:49 +00:00
Ilya Biryukov 2c5e8e820b [clangd] Make functions of ClangdServer callback-based
Summary:
As a consequence, all LSP operations are now handled asynchronously,
i.e. they never block the main processing thread. However, if
-run-synchronously flag is specified, clangd still runs everything on
the main thread.

Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall, ioeric

Subscribers: klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43227

llvm-svn: 325233
2018-02-15 13:15:47 +00:00
Sam McCall 7929d006d7 [clangd] Fix tracing now that spans lifetimes can overlap on a thread.
Summary:
The chrome trace viewer requires events within a thread to strictly nest.
So we need to record the lifetime of the Span objects, not the contexts.

But we still want to show the relationship between spans where a context crosses
threads, so do this with flow events (i.e. arrows).

Before: https://photos.app.goo.gl/q4Dd9u9xtelaXk1v1
After: https://photos.app.goo.gl/5RNLmAMLZR3unvY83

(This could stand some further improvement, in particular I think we want a
container span whenever we schedule work on a thread. But that's another patch)

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43272

llvm-svn: 325220
2018-02-15 08:40:54 +00:00
Ilya Biryukov 12aa52fe22 [clangd] Fix data race in ClangdThreadingTest.StressTest
Prior to this patch, same instance of VFS was shared for concurrent
processing of the files in ClangdThreadingTest.StressTest.

It caused a data race as the same instance of InMemoryFileSystem was
mutated from multiple threads using setCurrentWorkingDirectory().

llvm-svn: 325132
2018-02-14 15:19:20 +00:00
Ilya Biryukov 7beea3ab73 [clangd] Explicitly initialize all primitive fields in Protocol.h
Summary:
Some of the existing structs had primimtive fields that were
not explicitly initialized on construction.
After this commit every struct consistently sets a defined value for
every field when default-initialized.

Reviewers: hokein, ioeric, sammccall

Reviewed By: sammccall

Subscribers: klimek, cfe-commits, jkorous-apple

Differential Revision: https://reviews.llvm.org/D43230

llvm-svn: 325113
2018-02-14 10:52:04 +00:00
Ilya Biryukov 18c0247852 [clangd] Fixed findDefinitions to always return absolute paths.
Relative paths could be returned in some cases, e.g. when relative
path is used in compilation arguments. This led to crash when trying
to convert the path to URI.

llvm-svn: 325029
2018-02-13 17:47:16 +00:00
Ilya Biryukov 42fed626bc [clangd] Remove the RealFS layer from test VFS. NFC.
It was required before because preambles could only be created on
disk. All tests use in-memory preambles now.

llvm-svn: 325021
2018-02-13 17:08:13 +00:00
Haojian Wu dc02a3d943 [clangd] SymbolLocation only covers symbol name.
Summary:
* Change the offset range to half-open, [start, end).
* Fix a few fixmes.

Reviewers: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43182

llvm-svn: 324992
2018-02-13 09:53:50 +00:00
Sam McCall 0bb24cd4fa [clangd] Stop exposing Futures from ClangdServer operations.
Summary:
LSP has asynchronous semantics, being able to block on an async operation
completing is unneccesary and leads to tighter coupling with the threading.

In practice only tests depend on this, so we add a general-purpose "block until
idle" function to the scheduler which will work for all operations.

To get this working, fix a latent condition-variable bug in ASTWorker, and make
AsyncTaskRunner const-correct.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43127

llvm-svn: 324990
2018-02-13 08:59:23 +00:00
Haojian Wu cee2059f9b [clang-move] Fix the incorrect expansion end location.
Summary:
Before the fix, if clang-move decides to move the following macro statement, it only moves the first line `DEFINE(A,`.

```
DEFINE(A,
       B);
```

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D43174

llvm-svn: 324886
2018-02-12 12:26:12 +00:00
Ilya Biryukov cd5eb00e8b [clangd] Remove codeComplete that returns std::future<>
Summary:
It was deprecated and callback version and is used everywhere.
Only changes to the testing code were needed.

Reviewers: hokein, ioeric, sammccall

Reviewed By: sammccall

Subscribers: mgorny, klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43068

llvm-svn: 324883
2018-02-12 11:37:28 +00:00
Haojian Wu d478634777 [clang-move] Don't dump macro symbols.
Reviewers: ioeric

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D43075

llvm-svn: 324742
2018-02-09 15:57:30 +00:00
Ilya Biryukov be58c9605d [clangd] Fix crash in tests in debug mode.
Caused by the lack of checking of an Expected value in the previous
commit.

llvm-svn: 324736
2018-02-09 15:11:07 +00:00
Sam McCall 6003951c66 [clangd] Collect definitions when indexing.
Within a TU:
 - as now, collect a declaration from the first occurrence of a symbol
   (taking clang's canonical declaration)
 - when we first see a definition occurrence, copy the symbol and add it
Across TUs/sources:
 - mergeSymbol in Merge.h is responsible for combining matching Symbols.
   This covers dynamic/static merges and cross-TU merges in the static index.
 - it prefers declarations from Symbols that have a definition.
 - GlobalSymbolBuilderMain is modified to use mergeSymbol as a reduce step.
Random cleanups (can be pulled out):
 - SymbolFromYAML -> SymbolsFromYAML, new singular SymbolFromYAML added
 - avoid uninit'd SymbolLocations. Add an idiomatic way to check "absent".
 - CanonicalDeclaration (as well as Definition) are mapped as optional in YAML.
 - added operator<< for Symbol & SymbolLocation, for debugging

Reviewers: ioeric, hokein

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42942

llvm-svn: 324735
2018-02-09 14:42:01 +00:00
Ilya Biryukov b6ad25cd4c [clangd] Fix crash when CompilerInvocation can't be created.
Summary:
This can happen if the CompileCommand provided by compilation database
is malformed.

Reviewers: hokein, ioeric, sammccall

Reviewed By: sammccall

Subscribers: klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43122

llvm-svn: 324732
2018-02-09 13:51:57 +00:00
Ilya Biryukov 7aca05102c [clangd] Update include guard in Annotations.h. NFC
llvm-svn: 324599
2018-02-08 12:46:34 +00:00
Ilya Biryukov 7e5ee26d1a Resubmit "[clangd] The new threading implementation"
Initially submitted as r324356 and reverted in r324386.

This change additionally contains a fix to crashes of the buildbots.
The source of the crash was undefined behaviour caused by
std::future<> whose std::promise<> was destroyed without calling
set_value().

llvm-svn: 324575
2018-02-08 07:37:35 +00:00
Eric Liu 7deda4f5af [clangd] Do not precent-encode numbers in URI.
Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, cfe-commits, sammccall

Differential Revision: https://reviews.llvm.org/D43009

llvm-svn: 324475
2018-02-07 12:12:06 +00:00
Ilya Biryukov 3693f5941a Revert "[clangd] The new threading implementation" (r324356)
And the follow-up changes r324361 and r324363.
These changes seem to break two buildbots:
  - http://lab.llvm.org:8011/builders/clang-atom-d525-fedora-rel/builds/14091
  - http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules-2/builds/16001

We will need to investigate what went wrong and resubmit the changes
afterwards.

llvm-svn: 324386
2018-02-06 19:22:40 +00:00
Ilya Biryukov b9cf83b16f [clangd] Fixed compilation on Windows buildbot.
llvm-svn: 324361
2018-02-06 16:32:36 +00:00
Eric Liu 7f24765912 [clangd] Use URIs in index symbols.
Reviewers: hokein, sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42915

llvm-svn: 324358
2018-02-06 16:10:35 +00:00
Ilya Biryukov cce8883094 [clangd] The new threading implementation
Summary:
In the new threading model clangd creates one thread per file to manage
the AST and one thread to process each of the incoming requests.
The number of actively running threads is bounded by the semaphore to
avoid overloading the system.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, jkorous-apple, ioeric, hintonda, cfe-commits

Differential Revision: https://reviews.llvm.org/D42573

llvm-svn: 324356
2018-02-06 15:53:42 +00:00
Haojian Wu 3b8e00c5e0 [clangd] Fix incorrect file path for symbols defined by the compile command-line option.
Summary:

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42913

llvm-svn: 324328
2018-02-06 09:50:35 +00:00
Sam McCall c40fdcdd7e [clangd] Fix ExternC test broken by r324081
llvm-svn: 324105
2018-02-02 17:01:36 +00:00
Eric Liu cf1773826f [clangd] Skip inline namespace when collecting scopes for index symbols.
Summary:
Some STL symbols are defined in inline namespaces. For example,
```
namespace std {
inline namespace __cxx11 {
 typedef ... string;
}
}
```
Currently, this will be `std::__cxx11::string`; however, `std::string` is desired.

Inline namespaces are treated as transparent scopes. This
reflects the way they're most commonly used for lookup. Ideally we'd
include them, but at query time it's hard to find all the inline
namespaces to query: the preamble doesn't have a dedicated list.

Reviewers: sammccall, hokein

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42796

llvm-svn: 324065
2018-02-02 10:31:42 +00:00
Haojian Wu a8d12bbc3f [clangd] remove the unused code NFC.
llvm-svn: 323960
2018-02-01 13:06:58 +00:00
Eric Liu cda2526d86 [clangd] Fix URI scheme conflict and an unused variable warning in tests. NFC
llvm-svn: 323957
2018-02-01 12:44:52 +00:00
Sam McCall d1a7a37c22 [clangd] Pass Context implicitly using TLS.
Summary:
Instead of passing Context explicitly around, we now have a thread-local
Context object `Context::current()` which is an implicit argument to
every function.
Most manipulation of this should use the WithContextValue helper, which
augments the current Context to add a single KV pair, and restores the
old context on destruction.

Advantages are:
- less boilerplate in functions that just propagate contexts
- reading most code doesn't require understanding context at all, and
  using context as values in fewer places still
- fewer options to pass the "wrong" context when it changes within a
  scope (e.g. when using Span)
- contexts pass through interfaces we can't modify, such as VFS
- propagating contexts across threads was slightly tricky (e.g.
  copy vs move, no move-init in lambdas), and is now encapsulated in
  the threadpool

Disadvantages are all the usual TLS stuff - hidden magic, and
potential for higher memory usage on threads that don't use the
context. (In practice, it's just one pointer)

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D42517

llvm-svn: 323872
2018-01-31 13:40:48 +00:00
Haojian Wu b018906051 [clangd] Better handling symbols defined in macros.
Summary:
For symbols defined inside macros:
 * use expansion location, if the symbol is formed via macro concatenation.
 * use spelling location, otherwise.

This will fix some symbols that have ill-format location (especial invalid filepath).

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42575

llvm-svn: 323867
2018-01-31 12:56:51 +00:00
Haojian Wu fb68ca1825 [clang-move] Clever on handling header file which includes itself.
Summary:
Previously, we assume only old.cc includes "old.h", which would
introduce incorrect fixes for the cases where old.h also includes `#include "old.h"`

Although it should not be occurred in real projects, clang-move should handle this.

Old.h:

```
class Foo {};
```

after moving to a new old.h:

```
class Foo {};
```

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D42639

llvm-svn: 323865
2018-01-31 12:12:29 +00:00
Ilya Biryukov 75f1dd9b98 [clangd] Refactored threading in ClangdServer
Summary:
We now provide an abstraction of Scheduler that abstracts threading
and resource management in ClangdServer.
No changes to behavior are intended with an exception of changed error
messages.
This patch is preliminary work to allow a revamped threading
implementation that will move the threading code out of CppFile.

Reviewers: sammccall, bkramer, jkorous-apple

Reviewed By: sammccall

Subscribers: hokein, mgorny, hintonda, ioeric, jkorous-apple, cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D42174

llvm-svn: 323851
2018-01-31 08:51:16 +00:00
Eric Liu b57da098e9 [clangd] Fix r323658 test failure on windows.
llvm-svn: 323703
2018-01-29 22:28:08 +00:00
Eric Liu 78ed91a76c [clangd] Use new URI with scheme support in place of the existing LSP URI
Summary:
o Replace the existing clangd::URI with a wrapper of FileURI which also
carries a resolved file path.
o s/FileURI/URI/
o Get rid of the URI hack in vscode extension.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42419

llvm-svn: 323660
2018-01-29 15:37:46 +00:00
Eric Liu 278e2d1a26 [clangd] Add a fallback directory for collected symbols with relative paths.
Reviewers: hokein, sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D42638

llvm-svn: 323658
2018-01-29 15:13:29 +00:00
Sam McCall 1b475a1ad0 [clangd] Modify the Span API so that Spans propagate with contexts.
Summary:
This is probably the right behavior for distributed tracers, and makes unpaired
begin-end events impossible without requiring Spans to be bound to a thread.

The API is conceptually clean but syntactically awkward. As discussed offline,
this is basically a naming problem and will go away if (when) we use TLS to
store the current context.

The apparently-unrelated change to onScopeExit are because its move semantics
broken if Func is POD-like since r322838. This is true of function pointers,
and the lambda I use here that captures two pointers only.
I've raised this issue on llvm-dev and will revert this part if we fix it in
some other way.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D42499

llvm-svn: 323511
2018-01-26 09:00:30 +00:00
Sam McCall 034e11aca5 [clangd] Add ClangdUnit diagnostics tests using annotated code.
Summary:
This adds checks that our diagnostics emit correct ranges in a bunch of cases,
as promised in D41118.

The diagnostics-preamble test is also converted and extended to be a little more
precise.

diagnostics.test stays around as the smoke test for this feature.

Reviewers: ilya-biryukov

Subscribers: klimek, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D41454

llvm-svn: 323448
2018-01-25 17:29:17 +00:00
Ilya Biryukov df84234364 [clangd] Provide a helper to report estimated memory usage per-file
Reviewers: sammccall, ioeric, hokein

Reviewed By: ioeric

Subscribers: klimek, cfe-commits, jkorous-apple

Differential Revision: https://reviews.llvm.org/D42480

llvm-svn: 323425
2018-01-25 14:32:21 +00:00
Haojian Wu 58d208d6e4 [clangd] Disable typo correction when doing code completion.
Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, jkorous-apple, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D42491

llvm-svn: 323410
2018-01-25 09:44:06 +00:00
Haojian Wu 48b4865683 [clangd] Limit completion results.
Summary:
* truncate symbols from static/dynamic index to the limited number
(which would save lots of cost in constructing the merged symbols).
* add an CLI option allowing to limit the number of returned completion results.
(default to 100)

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D42484

llvm-svn: 323408
2018-01-25 09:20:09 +00:00
Sam McCall d5ea3e32a6 [clangd] add test for r323347 CodeComplete behavior we rely on
llvm-svn: 323350
2018-01-24 17:53:32 +00:00
Haojian Wu 061c73eb28 [clangd] Use accessible scopes to query indexes for global code completion.
Summary:
 * For qualified completion (foo::a^)
       * unresolved qualifier - use global namespace ("::")
       * resolved qualifier - use all accessible namespaces inside the resolved qualifier.
 * For unqualified completion (vec^), use scopes that are accessible from the
      scope from which code completion occurs.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: jkorous-apple, ioeric, klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D42073

llvm-svn: 323189
2018-01-23 11:37:26 +00:00
Sam McCall d2a9592526 [clangd] Drop ~destructor completions - rarely helpful and work inconsistently
llvm-svn: 323149
2018-01-22 21:05:00 +00:00
Eric Liu 43cee17cbf [clangd] Try to fix build bot failue in r323101
llvm-svn: 323111
2018-01-22 13:03:25 +00:00
Eric Liu f5b8c82198 [clangd] Add support for different file URI schemas.
Summary: I will replace the existing URI struct in Protocol.h with the new URI and rename FileURI to URI in a followup patch.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: jkorous-apple, klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41946

llvm-svn: 323101
2018-01-22 11:48:20 +00:00
Sam McCall 8b2faeed02 [clangd] Change index scope convention from "outer::inner" to "outer::inner::"
Global scope is "" (was "")
Top-level namespace scope is "ns::" (was "ns")
Nested namespace scope is "ns::ns::" (was "ns::ns")

This composes more naturally:
- qname = scope + name
- full scope = resolved scope + unresolved scope (D42073 was the trigger)
It removes a wart from the old way: "foo::" has one more separator than "".

Another alternative that has these properties is "::ns", but that lacks
the property that both the scope and the name are substrings of the
qname as produced by clang.

This is re-landing r322996 which didn't build.

llvm-svn: 323000
2018-01-19 22:18:21 +00:00
Sam McCall 50584ea197 Revert "[clangd] Change index scope convention from "outer::inner" to "outer::inner::""
This reverts commit r322996.

llvm-svn: 322998
2018-01-19 22:09:34 +00:00
Sam McCall 1a8c55ecc6 [clangd] Change index scope convention from "outer::inner" to "outer::inner::"
Global scope is "" (was "")
Top-level namespace scope is "ns::" (was "ns")
Nested namespace scope is "ns::ns::" (was "ns::ns")

This composes more naturally:
 - qname = scope + name
 - full scope = resolved scope + unresolved scope (D42073 was the trigger)
It removes a wart from the old way: "foo::" has one more separator than "".

Another alternative that has these properties is "::ns", but that lacks
the property that both the scope and the name are substrings of the
qname as produced by clang.

llvm-svn: 322996
2018-01-19 21:58:58 +00:00
Sam McCall 545a20d90b [clangd] Merge index-provided completions with those from Sema.
Summary:
 - we match on USR, and do a field-by-field merge if both have results
 - scoring is post-merge, with both sets of information available
   (for now, sema priority is used if available, static score for index results)
 - limit is applied to the complete result set (previously index ignored limit)
 - CompletionItem is only produces for the returned results
 - If the user doesn't type a scope, we send the global scope for completion
   (we can improve this after D42073)

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D42181

llvm-svn: 322945
2018-01-19 14:34:02 +00:00
Haojian Wu 9873fdd4f5 [clangd] Collect enum constants in SymbolCollector
Summary:
* ignore nameless symbols
* include enum constant declarataion

Reviewers: ilya-biryukov, jkorous-apple

Reviewed By: ilya-biryukov

Subscribers: ioeric, cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D42074

llvm-svn: 322929
2018-01-19 09:35:55 +00:00
Sam McCall d8169a85cc [clangd] Remove unused IncludeGlobals completion option, always pass true to sema
llvm-svn: 322856
2018-01-18 15:31:30 +00:00
Sam McCall a15c2d683e [clangd] CodeCompleteTests cleanup: naming, ordering, helpers. NFC
llvm-svn: 322824
2018-01-18 09:27:56 +00:00
Sam McCall a5ce243fa3 [clangd] Use fuzzy match to select top N index results.
Summary:
This makes performance slower but more predictable (it always processes
every symbol). We need to find ways to make this fast, possibly by precomputing
short queries or capping the number of scored results. But our current approach
is too naive.

It also no longer returns results in a "good" order. In fact it's pathological:
the top N results are ranked from worst to best. Indexes aren't responsible for
ranking and MergedIndex can't do a good job, so I'm pleased that this will make
any hidden assumptions we have more noticeable :-)

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D42060

llvm-svn: 322821
2018-01-18 08:35:04 +00:00
Sam McCall 2c3849af22 [clangd] Avoid combinatorial explosion in CodeCompleteTests.
Summary:
This test dominates our unit test runtime, and the change speeds it up by 10x.
We lose coverage of some combinations of flags, but I'm not sure that's finding
many bugs.

3300 -> 300ms on my machine (3800 -> 800ms for the whole of CompletionTest).

Reviewers: ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D42063

llvm-svn: 322547
2018-01-16 12:21:24 +00:00
Sam McCall 0faecf0c33 [clangd] Merge results from static/dynamic index.
Summary:
We now hide the static/dynamic split from the code completion, behind a
new implementation of the SymbolIndex interface. This will reduce the
complexity of the sema/index merging that needs to be done by
CodeComplete, at a fairly small cost in flexibility.

Reviewers: hokein

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D42049

llvm-svn: 322480
2018-01-15 12:33:00 +00:00
Sam McCall 09b4be51a8 [clangd] Fix uninitialized-read found by asan
llvm-svn: 322443
2018-01-13 16:46:26 +00:00
Sam McCall 3d139c527b [clangd] Code completion uses Sema for NS-level things in the current file.
Summary:
To stay fast, it avoids deserializing anything outside the current file, by
disabling the LoadExternal code completion option added in r322377, when the
index is enabled.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41996

llvm-svn: 322387
2018-01-12 18:30:08 +00:00
Sam McCall 93483e7f39 [clangd] Include debugging tags for both static and dynamic index. NFC
llvm-svn: 322379
2018-01-12 17:09:49 +00:00
Sam McCall 84652cc63c [clangd] Incorporate fuzzy-match into result rankings.
Summary: The scoring function is fuzzy-match-quality * existing quality score.

Reviewers: ioeric

Subscribers: klimek, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40780

llvm-svn: 322377
2018-01-12 16:16:09 +00:00
Haojian Wu f4374f087f [clangd] Don't navigate to forward class declaration when go to definition.
Summary:
For some cases, GoToDefinition will navigate to the forward class
declaration, we should always navigate to the class definition.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41661

llvm-svn: 322370
2018-01-12 14:21:10 +00:00
Ilya Biryukov 9555839228 [clangd] Pass Context to onDiagnosticsReady callback
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41907

llvm-svn: 322199
2018-01-10 17:59:27 +00:00
Eric Liu 9af958fbf8 [clangd] Add more filters for collected symbols.
Summary:
o We only collect symbols in namespace or translation unit scopes.
o Add an option to only collect symbols in included headers.

Reviewers: hokein, ilya-biryukov

Reviewed By: hokein, ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41823

llvm-svn: 322193
2018-01-10 14:57:58 +00:00
Haojian Wu ba28e9ac5d [clangd] Add static index for the global code completion.
Summary:
Use the YAML-format symbols (generated by the global-symbol-builder tool) to
do the global code completion.
It is **experimental** only , but it allows us to experience global code
completion on a relatively small project.

Tested with LLVM project.

Reviewers: sammccall, ioeric

Reviewed By: sammccall, ioeric

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41668

llvm-svn: 322191
2018-01-10 14:44:34 +00:00
Ilya Biryukov f60bf347c9 [clangd] Remove duplicates from code completion
Summary:
This patch removes hidden items from code completion.
Items can be hidden, e.g., by other items in the child scopes.
This patch addresses a particular problem of a duplicate completion
item for the class in the following example:

    struct Adapter { void method(); };
    void Adapter::method() {
       Adapter^
    }

We should probably investigate if there are other duplicates in
completion and remove them, possibly adding assertions that it never
happens.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D41901

llvm-svn: 322185
2018-01-10 13:51:09 +00:00
Eric Liu 76f6b44443 [clangd] Add more symbol information for code completion.
Reviewers: hokein, sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41345

llvm-svn: 322097
2018-01-09 17:32:00 +00:00
Eric Liu 5732c0e6ea [clangd] Use ToolExecutor to write the global-symbol-builder tool.
Summary:
This enables more execution modes like standalone and Mapreduce-style execution.

See also D41729

Reviewers: hokein, sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41730

llvm-svn: 322084
2018-01-09 15:21:45 +00:00
Haojian Wu fe22b7474b [clangd] Catch more symbols in SymbolCollector.
Summary:
We currently only collect external-linkage symbols in the collector,
which results in missing some typical symbols (like no-linkage type alias symbols).

This patch relaxes the constraint a bit to allow collecting more symbols.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41759

llvm-svn: 322067
2018-01-09 10:44:09 +00:00
Ilya Biryukov 5a5e1caf25 [clangd] Properly set filterText for index-based completion items
It was previously set to an identifier that the user typed, leading to
surprising behavior in VSCode (probably in other editors too).

llvm-svn: 321554
2017-12-29 14:59:22 +00:00
Sam McCall 4b9bbb378b [clangd] Use Builder for symbol slabs, and use sorted-vector for storage
Summary:
This improves a few things:
 - the insert -> freeze -> read sequence is now enforced/communicated by the
   type system
 - SymbolSlab::const_iterator iterates over symbols, not over id-symbol pairs
 - we avoid permanently storing a second copy of the IDs, and the
   string map's hashtable

The slab size is now down to 21.8MB for the LLVM project.
Of this only 2.7MB is strings, the rest is #symbols * `sizeof(Symbol)`.
`sizeof(Symbol)` is currently 96, which seems too big - I think
SymbolInfo isn't efficiently packed. That's a topic for another patch!

Also added simple API to see the memory usage/#symbols of a slab, since
it seems likely we will continue to care about this.

Reviewers: ilya-biryukov

Subscribers: klimek, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D41506

llvm-svn: 321412
2017-12-23 19:38:03 +00:00
Paul Robinson 1db24196d1 Update test after r321312
llvm-svn: 321323
2017-12-21 23:19:25 +00:00
Sam McCall 6c0d0f5775 [clangd] Index symbols share storage within a slab.
Summary:
Symbols are not self-contained - it's only safe to hand them out if you
guarantee the lifetime of the underlying data.

Before this lands, I'm going to measure the before/after memory usage of the
LLVM index loaded into memory in a single slab.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41483

llvm-svn: 321272
2017-12-21 14:58:44 +00:00
Eric Liu 63696e14e3 [clangd] Pull CodeCompletionString handling logic into its own file and add unit test.
Reviewers: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41450

llvm-svn: 321193
2017-12-20 17:24:31 +00:00
Eric Liu 188adaf46b [clangd] Remove an unused lambda capture.
llvm-svn: 321192
2017-12-20 17:22:56 +00:00
Sam McCall 328cbdb9e4 [clangd] Switch xrefs and documenthighlight to annotated-code unit tests. NFC
Summary:
The goal here is again to make it easier to read and write the tests.

I've extracted `parseTextMarker` from CodeCompleteTests into an `Annotations`
class, adding features to it:
  - as well as points `^s` it allows ranges `[[...]]`
  - multiple points and ranges are supported
  - points and ranges may be named: `$name^` and `$name[[...]]`

These features are used for the xrefs tests. This also paves the way for
replacing the lit diagnostics.test with more readable unit tests, using named
ranges.

Alternative considered: `TestSelectionRange` in clang-refactor/TestSupport
Main problems were:
 - delimiting the end of ranges is awkward, requiring counting
 - comment syntax is long and at least as cryptic for most cases
 - no separate syntax for point vs range, which keeps xrefs tests concise
 - Still need to convert to Position everywhere
 - Still need helpers for common case of expecting exactly one point/range

(I'll probably promote the extra `PrintTo`s from some of the core Protocol types
into `operator<<` in `Protocol.h` itself in a separate, prior patch...)

Reviewers: ioeric

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41432

llvm-svn: 321184
2017-12-20 16:06:05 +00:00
Sam McCall fffa8229e3 [clangd] Add debug printers for basic protocol types. NFC
llvm-svn: 321161
2017-12-20 10:26:53 +00:00
Eric Liu 125cda7d3b [clangd] Igore cases in index fuzzy find.
llvm-svn: 321157
2017-12-20 09:29:54 +00:00
Eric Liu bfac8f782c [clangd] Build dynamic index and use it for code completion.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41289

llvm-svn: 321092
2017-12-19 18:00:37 +00:00
Sam McCall e3e1570695 [clangd] Fix warnings/compiler pickiness after r321083
llvm-svn: 321086
2017-12-19 17:05:00 +00:00
Eric Liu 6f648df1b9 [clangd] Index-based code completion.
Summary: Use symbol index to populate completion results for qualfified IDs e.g. "nx::A^".

Reviewers: ilya-biryukov, sammccall

Reviewed By: ilya-biryukov, sammccall

Subscribers: rwols, klimek, mgorny, cfe-commits, sammccall

Differential Revision: https://reviews.llvm.org/D41281

llvm-svn: 321083
2017-12-19 16:50:37 +00:00
Sam McCall b536a2a5ba [clangd] Expose offset <-> LSP position functions, and fix bugs
Summary:
- Moved these functions to SourceCode.h
- added unit tests
- fix off by one in positionToOffset: Offset - 1 in final calculation was wrong
- fixed formatOnType which had an equal and opposite off-by-one
- positionToOffset and offsetToPosition both consistently clamp to beginning/end
  of file when input is out of range
- gave variables more descriptive names
- removed windows line ending fixmes where there is nothing to fix
- elaborated on UTF-8 fixmes

This will conflict with Eric's D41281, but in a pretty easy-to-resolve way.

Reviewers: ioeric

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41351

llvm-svn: 321073
2017-12-19 12:23:48 +00:00
Eric Liu 4feda80a43 [clangd] Support filtering by fixing scopes in fuzzyFind.
Summary: When scopes are specified, only match symbols from scopes.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41367

llvm-svn: 321067
2017-12-19 11:37:40 +00:00
Sam McCall 800d4371f3 [clangd] Add unit tests for signature help. SigHelp/CodeComplete lit tests are smoke only.
llvm-svn: 321065
2017-12-19 10:29:27 +00:00
Eric Liu eea1633878 [clangd] Build in-memory index on symbols in files.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41276

llvm-svn: 320807
2017-12-15 12:25:02 +00:00
Ilya Biryukov ee27d2ebae [clangd] Implemented tracing using Context
Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall

Subscribers: klimek, luckygeck, cfe-commits

Differential Revision: https://reviews.llvm.org/D40488

llvm-svn: 320706
2017-12-14 15:04:59 +00:00
Eric Liu d293bf127a [clangd] Add a FileSymbols container that manages symbols from multiple files.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41232

llvm-svn: 320701
2017-12-14 14:50:58 +00:00
Eric Liu a2d4607a4b [clangd] Fix a potential use-after-move bug.
llvm-svn: 320695
2017-12-14 12:31:04 +00:00
Haojian Wu 56a5fca473 [clangd] Construct SymbolSlab from YAML format.
Summary: This will be used together with D40548 for the global index source (experimental).

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D41178

llvm-svn: 320694
2017-12-14 12:17:14 +00:00
Eric Liu 3732cadc73 [clangd] Symbol index interfaces and an in-memory index implementation.
Summary:
o Index interfaces to support using different index sources (e.g. AST index, global index) for code completion, cross-reference finding etc. This patch focuses on code completion.

The following changes in the original patch has been split out.
o Implement an AST-based index.
o Add an option to replace sema code completion for qualified-id with index-based completion.
o Implement an initial naive code completion index which matches symbols that have the query string as substring.

Reviewers: malaperle, sammccall

Reviewed By: sammccall

Subscribers: hokein, klimek, malaperle, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40548

llvm-svn: 320688
2017-12-14 11:25:49 +00:00
Haojian Wu 03e2bd76c8 [clangd] Fix the unitttest build error on buildbot.
llvm-svn: 320678
2017-12-14 09:20:21 +00:00
Ilya Biryukov 5a85b8e6dd [clangd] clang-format the source code. NFC
llvm-svn: 320577
2017-12-13 12:53:16 +00:00
Ilya Biryukov 940901e8b1 [clangd] Implemented logging using Context
Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40486

llvm-svn: 320576
2017-12-13 12:51:22 +00:00
Haojian Wu 4c1394d67d [clangd] Introduce a "Symbol" class.
Summary:
* The "Symbol" class represents a C++ symbol in the codebase, containing all the
  information of a C++ symbol needed by clangd. clangd will use it in clangd's
  AST/dynamic index and global/static index (code completion and code
  navigation).
* The SymbolCollector (another IndexAction) will be used to recollect the
  symbols when the source file is changed (for ASTIndex), or to generate
  all C++ symbols for the whole project.

In the long term (when index-while-building is ready), clangd should share a
same "Symbol" structure and IndexAction with index-while-building, but
for now we want to have some stuff working in clangd.

Reviewers: ioeric, sammccall, ilya-biryukov, malaperle

Reviewed By: sammccall

Subscribers: malaperle, klimek, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D40897

llvm-svn: 320486
2017-12-12 15:42:10 +00:00
Ilya Biryukov 9b5ffc22f6 [clangd] clang-format the code. NFC
llvm-svn: 320476
2017-12-12 12:56:46 +00:00
Ilya Biryukov 657159c273 [clangd] Introduced a Context that stores implicit data
Summary:
It will be used to pass around things like Logger and Tracer throughout
clangd classes.

Reviewers: sammccall, ioeric, hokein, bkramer

Reviewed By: sammccall

Subscribers: klimek, bkramer, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D40485

llvm-svn: 320468
2017-12-12 11:16:45 +00:00
Sam McCall 44fdcec26f [clangd] Convert lit code completion tests to unit-tests. NFC
Summary: This improves readability of tests and error messages.

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40952

llvm-svn: 320148
2017-12-08 15:00:59 +00:00
Eric Liu 5a6d57c75f [change-namespace] Fix crash when injected base-class name is used in friend declarations.
Reviewers: hokein

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41001

llvm-svn: 320139
2017-12-08 10:06:16 +00:00
Shoaib Meenai d806af3499 [CMake] Use PRIVATE in target_link_libraries for executables
We currently use target_link_libraries without an explicit scope
specifier (INTERFACE, PRIVATE or PUBLIC) when linking executables.
Dependencies added in this way apply to both the target and its
dependencies, i.e. they become part of the executable's link interface
and are transitive.

Transitive dependencies generally don't make sense for executables,
since you wouldn't normally be linking against an executable. This also
causes issues for generating install export files when using
LLVM_DISTRIBUTION_COMPONENTS. For example, clang has a lot of LLVM
library dependencies, which are currently added as interface
dependencies. If clang is in the distribution components but the LLVM
libraries it depends on aren't (which is a perfectly legitimate use case
if the LLVM libraries are being built static and there are therefore no
run-time dependencies on them), CMake will complain about the LLVM
libraries not being in export set when attempting to generate the
install export file for clang. This is reasonable behavior on CMake's
part, and the right thing is for LLVM's build system to explicitly use
PRIVATE dependencies for executables.

Unfortunately, CMake doesn't allow you to mix and match the keyword and
non-keyword target_link_libraries signatures for a single target; i.e.,
if a single call to target_link_libraries for a particular target uses
one of the INTERFACE, PRIVATE, or PUBLIC keywords, all other calls must
also be updated to use those keywords. This means we must do this change
in a single shot. I also fully expect to have missed some instances; I
tested by enabling all the projects in the monorepo (except dragonegg),
and configuring both with and without shared libraries, on both Darwin
and Linux, but I'm planning to rely on the buildbots for other
configurations (since it should be pretty easy to fix those).

Even after this change, we still have a lot of target_link_libraries
calls that don't specify a scope keyword, mostly for shared libraries.
I'm thinking about addressing those in a follow-up, but that's a
separate change IMO.

Differential Revision: https://reviews.llvm.org/D40823

llvm-svn: 319840
2017-12-05 21:49:56 +00:00
Sam McCall f6ae323ddf [clangd] Clean up code complete unit tests. NFC
llvm-svn: 319820
2017-12-05 20:11:29 +00:00
Ilya Biryukov d3b04e3517 [clangd] Set completion options per-request.
Summary:
Previously, completion options were set per ClangdServer instance.
It will allow to change completion preferences during the lifetime
of a single ClangdServer instance.

Also rewrote ClangdCompletionTest.CompletionOptions to reuse single
ClangdServer instance, the test now runs 2x faster on my machine.

Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall, ioeric

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40654

llvm-svn: 319753
2017-12-05 10:42:57 +00:00
Sam McCall 93cd99173f [clangd] Remove unused test param. NFC
llvm-svn: 319742
2017-12-05 07:34:35 +00:00
Sam McCall 9aad25f193 [clangd] Split code-completion tests out of ClangdTests. NFC.
Summary:
Common parts are mostly FS related, so pulled out TestFS.h for the common stuff.
Deliberately resisted cleaning up much here, so this is pretty mechanical.

Reviewers: hokein

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40784

llvm-svn: 319741
2017-12-05 07:20:26 +00:00
Sam McCall ecbeab0ea1 [clangd] GlobalCompilationDatabase interface changes
Summary:
- GlobalCompilationDatabase now returns a single command (that's all we use)
- fallback flags are now part of the GlobalCompilationDatabase.
  There's a default implementation that they can optionally customize.
- this allows us to avoid invoking the fallback logic on two separate codepaths
- race on extra flags fixed by locking the mutex
- made GCD const-correct (DBGCD does have mutating methods)

Reviewers: hokein

Subscribers: klimek, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40733

llvm-svn: 319647
2017-12-04 10:08:45 +00:00
Sam McCall 87496417ff [clangd] Fuzzy match scorer
Summary:
This will be used for rescoring code completion results based on partial
identifiers.
Short-term use:
  - we want to limit the number of code completion results returned to
  improve performance of global completion. The scorer will be used to
  rerank the results to return when the user has applied a filter.
Long-term use case:
  - ranking of completion results from in-memory index
  - merging of completion results from multiple sources (merging usually
  works best when done at the component-score level, rescoring the
  fuzzy-match quality avoids different backends needing to have
  comparable scores)

Reviewers: ilya-biryukov

Subscribers: cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D40060

llvm-svn: 319557
2017-12-01 17:08:02 +00:00
Sam McCall 9fbecd5b0d [clangd] Filter completion results by fuzzy-matching identifiers.
Summary:
This allows us to limit the number of results we return and still allow them
to be surfaced by refining a query (D39852).

The initial algorithm is very conservative - it accepts a completion if the
filter is any case-insensitive sub-sequence. It does not attempt to rank items
based on match quality.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39882

llvm-svn: 319552
2017-12-01 16:35:50 +00:00
Sam McCall ff8b874548 [clangd] New conventions for JSON-marshalling functions, centralize machinery
Summary:
 - JSON<->Obj interface is now ADL functions, so they play nicely with enums
 - recursive vector/map parsing and ObjectMapper moved to JSONExpr and tested
 - renamed (un)parse to (de)serialize, since text -> JSON is called parse
 - Protocol.cpp gets a bit shorter

Sorry for the giant patch, it's prety mechanical though

Reviewers: ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40596

llvm-svn: 319478
2017-11-30 21:32:29 +00:00
Sam McCall ec109029b1 [clangd] Switch from YAMLParser to JSONExpr
Summary:
 - Converted Protocol.h parse() functions to take JSON::Expr.
   These no longer detect and log unknown fields, as this is not that
   useful and no longer free.
   I haven't changed the error handling too much: fields that were
   treated as optional before are still optional, even when it's wrong.
   Exception: object properties with the wrong type are now ignored.
 - Made JSONRPCDispatcher parse using json::parse
 - The bug where 'method' must come before 'params' in the stream is
 fixed as a side-effect. (And the same bug in executeCommand).
 - Some parser crashers fixed as a side effect.
   e.g. https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3890
 - The debug stream now prettyprints the input messages with --pretty.
 - Request params are attached to traces when tracing is enabled.
 - Fixed some bugs in tests (errors tolerated by YAMLParser, and
 off-by-ones in Content-Length that our null-termination was masking)
 - Fixed a random double-escape bug in ClangdLSPServer (it was our last
 use of YAMLParser!)

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D40406

llvm-svn: 319159
2017-11-28 09:37:43 +00:00
Sam McCall 1bb272bc74 [clangd] Add missing (but documented!) JSONExpr typed accessors
Summary:
Noticed this when I tried to port the Protocol.h parsers.
And tests for the inspect API, which caught a small bug.

Reviewers: ioeric

Subscribers: ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40399

llvm-svn: 319157
2017-11-28 09:25:09 +00:00
Sam McCall 9cfd9c9a9b [clangd] Tracing improvements
Summary:
[clangd] Tracing improvements

Compose JSON using JSONExpr
Allow attaching metadata to spans (and avoid it if tracing is off)
Attach IDs and responses of JSON RPCs to their spans

The downside is that large responses make the trace viewer sluggish.
We should make our responses less huge :-) Or fix trace viewer.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D40132

llvm-svn: 318928
2017-11-23 17:12:04 +00:00
Alexander Kornienko 6f67bcbb93 [clang-tidy] rename_check.py misc-argument-comment bugprone-argument-comment
Summary: + manually convert the unit test to lit test.

Reviewers: hokein

Reviewed By: hokein

Subscribers: mgorny, xazax.hun, cfe-commits

Differential Revision: https://reviews.llvm.org/D40392

llvm-svn: 318926
2017-11-23 17:02:48 +00:00
Sam McCall adccab64f2 [clangd] Drop impossible completions (unavailable or inaccessible)
Summary: (There must be some reason why D38077 didn't just do this, but I don't get it!)

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39836

llvm-svn: 318925
2017-11-23 16:58:22 +00:00
Sam McCall cc713cf885 [clangd] avoid divide by literal zero to satisfy msvc
llvm-svn: 318798
2017-11-21 19:37:35 +00:00
Sam McCall b67ce00e4d [clangd] use u8 strings in tests to avoid problems on windows
llvm-svn: 318793
2017-11-21 19:10:22 +00:00
Sam McCall 40f191f724 [clangd] Add JSON tests with invalid unicode
llvm-svn: 318791
2017-11-21 18:40:43 +00:00
Sam McCall adbaebc242 [clangd] Add parsing and value inspection to JSONExpr.
Summary:
This will replace the places where we're using YAMLParser to parse JSON now:
  - the new marshalling code (T::parse()) should handle fewer cases and require
    fewer explicit casts
  - we'll early-reject invalid JSON that YAMLParser accepts
  - we'll be able to fix protocol-parsing bugs caused by the fact that YAML can
    only parse forward

I plan to do the conversion as soon as this lands, but I don't want it in one
patch as the protocol.cpp changes are conflict-prone.

Reviewers: ioeric

Subscribers: ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40182

llvm-svn: 318774
2017-11-21 16:00:53 +00:00
Ilya Biryukov e9eb7f0cb8 [clangd] Use in-memory preambles in clangd.
Reviewers: klimek, bkramer, sammccall

Reviewed By: sammccall

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39843

llvm-svn: 318412
2017-11-16 16:25:18 +00:00
Ilya Biryukov 9e11c4c957 [clangd] clang-format the source code. NFC.
llvm-svn: 318317
2017-11-15 18:04:56 +00:00
Sam McCall a40371bcb6 [clangd] Support returning a limited number of completion results.
Summary:
All results are scored, we only process CodeCompletionStrings for the winners.
We now return CompletionList rather than CompletionItem[] (both are valid).
sortText is now based on CodeCompletionResult::orderedName (mostly the same).

This is the first clangd-only completion option, so plumbing changed.
It requires a small clangd patch (exposing CodeCompletionResult::orderedName).

(This can't usefully be enabled yet: we don't support server-side filtering)

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39852

llvm-svn: 318287
2017-11-15 09:16:29 +00:00
Douglas Yung 76036fce25 Fix compile issue on MSVC.
llvm-svn: 317642
2017-11-08 00:29:42 +00:00
Sam McCall 784e88351b [clangd] MSVC - third time's the charm
llvm-svn: 317581
2017-11-07 14:59:21 +00:00
Sam McCall 99633ec527 [clangd] another try at fixing MSVC
llvm-svn: 317575
2017-11-07 14:14:58 +00:00
NAKAMURA Takumi e02398022d ClangdTests/JSONExprTests.cpp: Appease g++-4.8 to move raw string literal out of macro arg.
llvm-svn: 317538
2017-11-07 02:18:24 +00:00
Sam McCall dd0566bb2c Adds a json::Expr type to represent intermediate JSON expressions.
Summary:
This form can be created with a nice clang-format-friendly literal syntax,
and gets escaping right. It knows how to call unparse() on our Protocol types.
All the places where we pass around JSON internally now use this type.

Object properties are sorted (stored as std::map) and so serialization is
canonicalized, with optional prettyprinting (triggered by a -pretty flag).
This makes the lit tests much nicer to read and somewhat nicer to debug.
(Unfortunately the completion tests use CHECK-DAG, which only has
line-granularity, so pretty-printing is disabled there. In future we
could make completion ordering deterministic, or switch to unittests).

Compared to the current approach, it has some efficiencies like avoiding copies
of string literals used as object keys, but is probably slower overall.
I think the code/test quality benefits are worth it.

This patch doesn't attempt to do anything about JSON *parsing*.
It takes direction from the proposal in this doc[1], but is limited in scope
and visibility, for now.
I am of half a mind just to use Expr as the target of a parser, and maybe do a
little string deduplication, but not bother with clever memory allocation.
That would be simple, and fast enough for clangd...
[1] https://docs.google.com/document/d/1OEF9IauWwNuSigZzvvbjc1cVS1uGHRyGTXaoy3DjqM4/edit

+cc d0k so he can tell me not to use std::map.

Reviewers: ioeric, malaperle

Subscribers: bkramer, ilya-biryukov, mgorny, klimek

Differential Revision: https://reviews.llvm.org/D39435

llvm-svn: 317486
2017-11-06 15:40:30 +00:00
Sam McCall 32995b3056 Fix clangd test on platforms where get_thread_name does nothing.
llvm-svn: 317194
2017-11-02 09:48:55 +00:00
Sam McCall 8567cb3720 Performance tracing facility for clangd.
Summary:
This lets you visualize clangd's activity on different threads over time,
and understand critical paths of requests and object lifetimes.
The data produced can be visualized in Chrome (at chrome://tracing), or
in a standalone copy of catapult (http://github.com/catapult-project/catapult)

This patch consists of:
 - a command line flag "-trace" that causes clangd to emit JSON trace data
 - an API (in Trace.h) allowing clangd code to easily add events to the stream
 - several initial uses of this API to capture JSON-RPC requests, builds, logs

Example result: https://photos.app.goo.gl/12L9swaz5REGQ1rm1

Caveats:
 - JSON serialization is ad-hoc (isn't it everywhere?) so the API is
   limited to naming events rather than attaching arbitrary metadata.
   I'd like to fix this (I think we could use a JSON-object abstraction).
 - The recording is very naive: events are written immediately by
   locking a mutex. Contention on the mutex might disturb performance.
 - For now it just traces instants or spans on the current thread.
   There are other things that make sense to show (cross-thread flows,
   non-thread resources such as ASTs). But we have to start somewhere.

Reviewers: ioeric, ilya-biryukov

Subscribers: cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D39086

llvm-svn: 317193
2017-11-02 09:21:51 +00:00
Haojian Wu e010406e28 [clang-tidy ObjC] [3/3] New check objc-forbidden-subclassing
Summary:
This is part 3 of 3 of a series of changes to improve Objective-C
linting in clang-tidy.

This adds a new clang-tidy check `objc-forbidden-subclassing` which
ensures clients do not create subclasses of Objective-C classes which
are not designed to be subclassed.

(Note that for code under your control, you should use
__attribute__((objc_subclassing_restricted)) instead -- this
is intended for third-party APIs which cannot be modified.)

By default, the following classes (which are publicly documented
as not supporting subclassing) are forbidden from subclassing:

ABNewPersonViewController
ABPeoplePickerNavigationController
ABPersonViewController
ABUnknownPersonViewController
NSHashTable
NSMapTable
NSPointerArray
NSPointerFunctions
NSTimer
UIActionSheet
UIAlertView
UIImagePickerController
UITextInputMode
UIWebView

Clients can set a CheckOption
`objc-forbidden-subclassing.ClassNames` to a semicolon-separated
list of class names, which overrides this list.

Test Plan: `ninja check-clang-tools`

Patch by Ben Hamilton!

Reviewers: hokein, alexfh

Reviewed By: hokein

Subscribers: saidinwot, Wizard, srhines, mgorny, xazax.hun

Differential Revision: https://reviews.llvm.org/D39142

llvm-svn: 316744
2017-10-27 07:41:36 +00:00
Benjamin Kramer ee19f16633 [clangd] Report an error on findDefinitions/signatureHelp on an unopened file instead of crashing.
Found by clangd-fuzzer.

llvm-svn: 316659
2017-10-26 12:28:13 +00:00
Haojian Wu d2825825f8 [clang-tidy ObjC] [2/3] Support non-C++ files in ClangTidyTest
Summary:
This is part 2 of 3 of a series of changes to improve
Objective-C linting in clang-tidy.

Currently, `clang::tidy::test::runCheckOnCode()` assumes all files
are C++ and unconditionally adds `-std=c++11` to the list of
`clang-tidy` options.

This updates the logic to check the extension of the source file
and only add `-std=c++11` if the extension indicates C++ or
Objective-C++.

Depends On D39188

Test Plan:

  ninja ClangTidyTests && \
  ./tools/clang/tools/extra/unittests/clang-tidy/ClangTidyTests

Patch by Ben Hamilton!

Reviewers: hokein, alexfh

Reviewed By: hokein

Subscribers: Wizard

Differential Revision: https://reviews.llvm.org/D39189

llvm-svn: 316645
2017-10-26 08:37:25 +00:00
Haojian Wu abcd64ccbf [clang-tidy ObjC] [1/3] New module `objc` for Objective-C checks
Summary:
This is part 1 of 3 of a series of changes to improve Objective-C
linting in clang-tidy.

This introduces a new clang-tidy module, `objc`, specifically for
Objective-C / Objective-C++ checks.

The module is currently empty; D39142 adds the first check.

Test Plan: `ninja check-clang-tools`

Patch by Ben Hamilton!

Reviewers: hokein, alexfh

Reviewed By: hokein

Subscribers: Wizard, mgorny

Differential Revision: https://reviews.llvm.org/D39188

llvm-svn: 316643
2017-10-26 08:23:20 +00:00
Ilya Biryukov b080cb155b [clangd] Allow to pass code completion opts to ClangdServer.
Reviewers: bkramer, krasimir, sammccall

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D38731

llvm-svn: 316327
2017-10-23 14:46:48 +00:00
Eric Liu b583a7ead4 [change-namespace] do not change type locs in defaulted functions.
Reviewers: hokein

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D38893

llvm-svn: 315892
2017-10-16 08:20:10 +00:00
Ilya Biryukov dcd2169380 [clangd] Added async API to run code completion.
Summary:
ClangdServer now provides async code completion API.
It is still used synchronously by ClangdLSPServer, more work is needed
to allow processing other requests in parallel while completion (or
any other request) is running.

Reviewers: klimek, bkramer, krasimir

Reviewed By: klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D38583

llvm-svn: 314989
2017-10-05 17:04:13 +00:00
Marc-Andre Laperle 6571b3edd0 [clangd] LSP extension to switch between source/header file
Summary:
Small extension to LSP to allow clients to use clangd to switch between C header files and source files.
Final version will use the completed clangd indexer to use the index of symbols to be able to switch from header to source file when the file names don't match.

Reviewers: malaperle, krasimir, bkramer, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ilya-biryukov, cfe-commits, arphaman

Patch by: William Enright

Differential Revision: https://reviews.llvm.org/D36150

llvm-svn: 314377
2017-09-28 03:14:40 +00:00
Ilya Biryukov 47f22026b5 [clangd] Serialize onDiagnosticsReady callbacks for the same file.
Summary:
Calls to onDiagnosticsReady were done concurrently before. This sometimes
led to older versions of diagnostics being reported to the user after
the newer versions.

Reviewers: klimek, bkramer, krasimir

Reviewed By: klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D38032

llvm-svn: 313754
2017-09-20 12:58:55 +00:00
Ilya Biryukov e5128f7e91 [clangd] Introduced Logger interface.
Summary: This fixes a bunch of logging-related FIXMEs.

Reviewers: bkramer, krasimir, malaperle

Reviewed By: malaperle

Subscribers: malaperle, klimek, cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D37972

llvm-svn: 313730
2017-09-20 07:24:15 +00:00
Ilya Biryukov b33c15741b [clangd] Add support for snippet completions
Enhances CompletionItemsCollector in such a way that snippet
completions can be presented to the client. Enable snippet completion
items by specifying -enable-snippets while invoking the clangd
executable.

See: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#completion-request
See: https://github.com/Microsoft/vscode/blob/master/src/vs/editor/contrib/snippet/browser/snippet.md

Patch by Raoul Wols.

llvm-svn: 313029
2017-09-12 13:57:14 +00:00
Michal Gorny a81de445c1 [cmake] Support running extra clang tool tests without static analyzer
Support running the extra clang tool tests when the static analyzer
is disabled. Disable the relevant clang-tidy tests and one include-fixer
test that require it to work.

Previously, the tests were disabled entirely with
CLANG_ENABLE_STATIC_ANALYZER being false. Now, the tests are being
enabled and the relevant tests are excluded and marked unsupported
appropriately.

In order to disable clang-tidy tests, the whole test directory is added
to the exclude lists, to avoid having to explicitly add 'REQUIRES' line
to every single test. If the other solution is preferable, I can update
the patch.

The yamldb_plugin include-fixer test is also updated to be disabled
without static analyzer. It fails in that case because clang is not
outputting a replacement suggestion -- but I don't know the exact
reason why it does not do that.

Differential Revision: https://reviews.llvm.org/D37188

llvm-svn: 311983
2017-08-29 05:58:08 +00:00
Simon Pilgrim 0f311a4189 Add braces to silence gcc dangling-else warnings. NFCI.
llvm-svn: 310830
2017-08-14 11:05:56 +00:00
Ilya Biryukov db8b2d7b20 [clangd] Use multiple working threads in clangd.
Reviewers: bkramer, krasimir, klimek

Reviewed By: klimek

Subscribers: arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D36261

llvm-svn: 310821
2017-08-14 08:45:47 +00:00
Ilya Biryukov 91dbf5b6b4 [clangd] Check if CompileCommand has changed on forceReparse.
Reviewers: krasimir, bkramer, klimek

Reviewed By: klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D36398

llvm-svn: 310819
2017-08-14 08:37:32 +00:00
Alexander Kornienko a4d94689fc [clang-tidy] Add a test.
llvm-svn: 310586
2017-08-10 12:38:46 +00:00
Alexander Kornienko 9b8df6ae02 [clang-tidy] Ignore newlines in checks list
This is a follow up to https://reviews.llvm.org/D30567 where I overlooked that
LLVM YAML parser doesn't support multiline literal folding.

llvm-svn: 310491
2017-08-09 16:00:31 +00:00
NAKAMURA Takumi 55aebd689e ClangdTests: Try to unbreak the case CLANG_DEFAULT_CXX_STDLIB=libc++.
llvm-svn: 309936
2017-08-03 13:30:43 +00:00
Ilya Biryukov 02d5870de3 [clangd] Rewrote AST and Preamble management.
Summary: The new implementation allows code completion that never waits for AST.

Reviewers: bkramer, krasimir, klimek

Reviewed By: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D36133

llvm-svn: 309696
2017-08-01 15:51:38 +00:00
Ilya Biryukov ba3bd7ce3e [clangd] Specified --gcc-toolchain explicitly in VFS test.
In order to avoid platform-specific configuration quirks.
(Possible fix for https://bugs.llvm.org/show_bug.cgi?id=33842)

llvm-svn: 308721
2017-07-21 08:20:47 +00:00
NAKAMURA Takumi 8de96ca8bc ClangApplyReplacementsTests: Add clangBasic in libdeps.
r308015 introduced clangBasic to instantiate Diagnostics &c,

llvm-svn: 308085
2017-07-15 06:32:10 +00:00
Alexander Kornienko 8b4fcb7197 [Clang-Tidy] Preserve Message, FileOffset, FilePath in Clang-Tidy YAML output
Summary:
To get properly integration Clang-Tidy with CLion IDE, next things were implemented:
* Preserve `Message`, `FileOffset`, `FilePath` in the clang-tidy output.
* Export all diagnostics, not just the ones with fixes
* Test-cases

Reviewers: alexfh, ilya-biryukov

Subscribers: mgorny, JDevlieghere, xazax.hun, cfe-commits

Tags: #clang-tools-extra

Patch by Vladimir Plyashkun!

Differential Revision: https://reviews.llvm.org/D35349

llvm-svn: 308015
2017-07-14 10:37:46 +00:00
Ilya Biryukov 53d69e329b [clangd] Add -ffreestanding on VFS tests.
Summary:
They don't provide proper gcc installations and may fail on implicit
<stdc-predef.h> include.

Reviewers: klimek, krasimir, bkramer

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D34936

llvm-svn: 307025
2017-07-03 14:46:03 +00:00
Alex Lorenz 4abbd92bf4 [refactor] Move clang-rename into the clang repository
The core engine of clang-rename will be used for local and global renames in the
new refactoring engine, as mentioned in
http://lists.llvm.org/pipermail/cfe-dev/2017-June/054286.html.

The clang-rename tool is still supported but might get deprecated in the future.

Differential Revision: https://reviews.llvm.org/D34696

llvm-svn: 306840
2017-06-30 16:36:09 +00:00
Ilya Biryukov bb324b8844 [clangd] Run a test, searching for gcc install, only on Unix.
This should fix windows buildbots.

llvm-svn: 306656
2017-06-29 09:22:32 +00:00
Ilya Biryukov 82090a7d29 [clangd] Added a test, checking that gcc install is searched via VFS.
Reviewers: bkramer, krasimir, klimek

Reviewed By: klimek

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D34755

llvm-svn: 306652
2017-06-29 08:43:51 +00:00
Yan Wang 36206206cd [clang-tidy][Part1] Add a new module Android and three new checks.
Summary:
A common source of security bugs is code that opens a file descriptors without using the O_CLOEXEC flag.  (Without that flag, an opened sensitive file would remain open across a fork+exec to a lower-privileged SELinux domain, leaking that sensitive data.).

Add a new Android module and one checks in clang-tidy.
-- open(), openat(), and open64() should include O_CLOEXEC in their flags argument. [android-file-open-flag]

Links to part2 and part3:
https://reviews.llvm.org/D33745
https://reviews.llvm.org/D33747


Reviewers: chh, alexfh, aaron.ballman, hokein

Reviewed By: alexfh, hokein

Subscribers: jbcoe, joerg, malcolm.parsons, Eugene.Zelenko, srhines, mgorny, xazax.hun, cfe-commits, krytarowski

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D33304

llvm-svn: 306165
2017-06-23 21:37:29 +00:00
Ilya Biryukov af0c04b30b [clangd] Add a filename parameter to FileSystemProvider.
Reviewers: krasimir

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D34151

llvm-svn: 305376
2017-06-14 09:46:44 +00:00
Ilya Biryukov 103c951c52 [clangd] Store references instead of unique_ptrs in ClangdServer.
Summary:
ClangdServer owned objects passed to it in constructor for no good reason.
Lots of stuff was moved from the heap to the stack thanks to this change.

Reviewers: krasimir

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D34148

llvm-svn: 305298
2017-06-13 15:59:43 +00:00
Ilya Biryukov 0e27ce4271 [clangd] Allow to override contents of the file during completion.
Summary:
This is a reapplied r305280 with a fix to the crash found by build bots
(StringRef to an out-of-scope local std::string).

Reviewers: krasimir

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D34146

llvm-svn: 305291
2017-06-13 14:15:56 +00:00
Ilya Biryukov 8cc09eb8dd Revert "[clangd] Allow to override contents of the file during completion."
This caused buildbots failures, reverting until we'll find out what's
wrong.

llvm-svn: 305283
2017-06-13 10:01:11 +00:00
Ilya Biryukov b23ff10c67 [clangd] Allow to override contents of the file during completion.
Reviewers: krasimir

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D34107

llvm-svn: 305280
2017-06-13 08:32:27 +00:00
Ilya Biryukov e36035b121 [clangd] Use 'std::string' for VFSTag instead of 'int'
Reviewers: krasimir

Reviewed By: krasimir

Subscribers: klimek, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D34106

llvm-svn: 305279
2017-06-13 08:24:48 +00:00
Miklos Vajna a0ce632079 clang-rename: add new -force option
Summary:
    The use-case is when renaming a widely used name, like a lower-level
    class in a codebase and clang-rename is simply invoked for each
    translation unit based on the compile database. In this case it's not
    interesting to show errors: not finding the symbol means there is
    simply nothing to do.

Reviewers: klimek

Reviewed By: klimek

Differential Revision: https://reviews.llvm.org/D32403

llvm-svn: 304534
2017-06-02 09:32:28 +00:00
Ilya Biryukov 2260299830 [clangd] Mark results of clangd requests with a tag provided by the FileSystemProvider.
Summary:
This allows an implementation of FileSystemProvider that can track which vfs::FileSystem
were used for each of the requests.

Reviewers: bkramer, krasimir

Reviewed By: bkramer

Subscribers: klimek, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D33678

llvm-svn: 304214
2017-05-30 15:11:02 +00:00
NAKAMURA Takumi 2efabeb933 ClangdTests.cpp: Use "clang/Config/config.h" instead of "llvm/Config/config.h".
llvm-svn: 304082
2017-05-27 23:19:28 +00:00
NAKAMURA Takumi 4455641ae0 ClangdTests.cpp: Fix a possible typo, it was "if (TmpDir2 != TmpDir2)"
It caused failures in unittests if TmpDir2 is not "/tmp" .

llvm-svn: 304067
2017-05-27 08:06:52 +00:00
Ilya Biryukov 90fecdb47c [clangd] Attempt to fix tests failing on Windows
llvm-svn: 303993
2017-05-26 14:34:34 +00:00
Ilya Biryukov 0f62ed2bbe [clangd] Allow to use vfs::FileSystem for file accesses.
Summary:
Custom vfs::FileSystem is currently used for unit tests.
This revision depends on https://reviews.llvm.org/D33397.

Reviewers: bkramer, krasimir

Reviewed By: bkramer, krasimir

Subscribers: klimek, cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D33416

llvm-svn: 303977
2017-05-26 12:26:51 +00:00
Haojian Wu 03c8963c32 [clang-move] Find template class forward declarations more precisely.
Reviewers: ioeric

Reviewed By: ioeric

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D32741

llvm-svn: 301914
2017-05-02 12:15:11 +00:00
Haojian Wu 0bc5e02799 Fix compiler warnings: "ISO c99 requires rest arguments to be used" on
the test file.

llvm-svn: 299764
2017-04-07 12:37:32 +00:00
David Blaikie 0c0c8c4d38 Fix -Wmissing-field-initializer warnings to unbreak the -Werror build
llvm-svn: 299561
2017-04-05 16:50:19 +00:00
Haojian Wu 74f823a045 [clang-rename] Support renaming qualified symbol
Summary:
The patch adds a new feature for renaming qualified symbol references.
Unlike orginal clang-rename behavior, when renaming a qualified symbol to a new
qualified symbol (e.g "A::Foo" => "B::Bar"), this new rename behavior will
consider the prefix qualifiers of the symbol, and calculate the new prefix
qualifiers.  It aims to add as few additional qualifiers as possible.

As this is an early version (only supports renaming classes), I don't change
current clang-rename interfaces at the moment, and would like to keep its
(command-line tool) behavior. So I added new interfaces for the prototype.
In the long run, these interfaces should be unified.

No functionality changes in original clang-rename command-line tool.

This patch also contains a few bug fixes of clang-rename which are discovered by
the new unittest:

* fix a potential nullptr accessment when class declaration doesn't have definition.
* add USRs of nested declartaions in "getNamedDeclFor".

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: alexfh, cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D31176

llvm-svn: 299419
2017-04-04 09:30:06 +00:00
Marek Kurdej 92ecb51fae [clang-tidy] Fix treating non-space whitespaces in checks list.
Summary:
This furtherly improves r295303: [clang-tidy] Ignore spaces between globs in the Checks option.
Trims all whitespaces and not only spaces and correctly computes the offset of the checks list (taking the size before trimming).

Reviewers: alexfh

Reviewed By: alexfh

Subscribers: cfe-commits, JDevlieghere

Differential Revision: https://reviews.llvm.org/D30567

llvm-svn: 298621
2017-03-23 16:32:06 +00:00
Eric Liu 8bc2416414 [change-namespace] avoid adding leading '::' when possible.
Summary:
When changing namespaces, the tool adds leading "::" to references that need to
be fully-qualified, which would affect readability.

We avoid adding "::" when the symbol name does not conflict with the new
namespace name. For example, a symbol name "na::nb::X" conflicts with "ns::na"
since it would be resolved to "ns::na::nb::X" in the new namespace.

Reviewers: hokein

Reviewed By: hokein

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30493

llvm-svn: 298363
2017-03-21 12:41:59 +00:00
Eric Liu 284b97c6bc [change-namespace] do not rename specialized template parameters.
Reviewers: hokein

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31076

llvm-svn: 298090
2017-03-17 14:05:39 +00:00
Sam McCall 9c5ebf7039 [include-fixer] Add fuzzy SymbolIndex, where identifier needn't match exactly.
Summary:
Add fuzzy SymbolIndex, where identifier needn't match exactly.

The purpose for this is global autocomplete in clangd. The query will be a
partial identifier up to the cursor, and the results will be suggestions.

It's in include-fixer because:

  - it handles SymbolInfos, actually SymbolIndex is exactly the right interface
  - it's a good harness for lit testing the fuzzy YAML index
  - (Laziness: we can't unit test clangd until reorganizing with a tool/ dir)

Other questionable choices:

  - FuzzySymbolIndex, which just refines the contract of SymbolIndex. This is
    an interface to allow extension to large monorepos (*cough*)
  - an always-true safety check that Identifier == Name is removed from
    SymbolIndexManager, as it's not true for fuzzy matching
  - exposing -db=fuzzyYaml from include-fixer is not a very useful feature, and
    a non-orthogonal ui (fuzziness vs data source). -db=fixed is similar though.

Reviewers: bkramer

Subscribers: cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D30720

llvm-svn: 297630
2017-03-13 15:55:59 +00:00
Haojian Wu b7da0eb50e [clang-rename] Introduce an unittest skeleton for clang-rename.
Summary: This will make references rename tests easier.

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: cfe-commits, alexfh, mgorny

Differential Revision: https://reviews.llvm.org/D30813

llvm-svn: 297450
2017-03-10 10:30:14 +00:00
Sam McCall 573050e703 [include-fixer] Remove line number from Symbol identity
Summary:
Remove line number from Symbol identity.

For our purposes (include-fixer and clangd autocomplete), function overloads
within the same header should mostly be treated as a single combined symbol.

We may want to track individual occurrences (line number, full type info)
and aggregate this during mapreduce, but that's not done here.

Reviewers: hokein, bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30685

llvm-svn: 297371
2017-03-09 10:47:44 +00:00
Eric Liu 413671507f [change-namespace] get insertion points of forward declarations correct.
Summary:
Previously, the insertion points would conflict with the old namespace
deletion.

Reviewers: hokein

Reviewed By: hokein

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30490

llvm-svn: 296604
2017-03-01 10:29:39 +00:00
Sam McCall b27dc2245f [include-fixer] Add usage count to find-all-symbols.
Summary:
Add usage count to find-all-symbols.

FindAllSymbols now finds (most!) main-file usages of the discovered symbols.
The per-TU map output has NumUses=0 or 1 (only one use per file is counted).
The reducer aggregates these to find the number of files that use a symbol.

The NumOccurrences is now set to 1 in the mapper rather than being inferred by
the reducer, for consistency.

The idea here is to use NumUses for ranking: intuitively number of files that
use a symbol is more meaningful than number of files that include the header.

Reviewers: hokein, bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30210

llvm-svn: 296446
2017-02-28 08:13:15 +00:00
Haojian Wu 4a92050ce2 [clang-move] Extend clang-move to support moving global variable.
Summary: Also support dumping global variables.

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30337

llvm-svn: 296337
2017-02-27 13:19:13 +00:00
Eric Liu 7fccc9995a [change-namepsace] make it possible to whitelist symbols so they don't get updated.
Reviewers: hokein

Reviewed By: hokein

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30328

llvm-svn: 296110
2017-02-24 11:54:45 +00:00
Benjamin Kramer ba5df6dea5 [clang-tidy] Reword the "code outside header guard" warning.
The check doesn't really know if the code it is warning about came before
or after the header guard, so phrase it more neutral instead of complaining
about code before the header guard. The location for the warning is still
not optimal, but I don't think fixing that is worth the effort, the
preprocessor doesn't give us a better location.

Differential Revision: https://reviews.llvm.org/D30191

llvm-svn: 295715
2017-02-21 11:25:45 +00:00
Krasimir Georgiev 8fcdd5ab96 Fix clang-move test after clang-format update r295312
llvm-svn: 295317
2017-02-16 13:17:38 +00:00
Alexander Kornienko 762adef1a9 [clang-tidy] Ignore spaces between globs in the Checks option.
llvm-svn: 295303
2017-02-16 10:23:18 +00:00
Eric Liu 0325a77c5b [change-namespace] fix unscoped enum constant references.
Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D29460

llvm-svn: 293909
2017-02-02 17:40:38 +00:00
Eric Liu ae7de7117a [change-namespace] check using shadow decl correctly when shortening namespace specifiers.
Summary:
This fixes mismatch between template decls and template specialization decls.

Also added a few more test cases.

Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D29447

llvm-svn: 293897
2017-02-02 15:29:54 +00:00
Eric Liu 9122916ee5 [change-namespace] correctly shorten namespace when references have leading '::'
Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D29182

llvm-svn: 293187
2017-01-26 16:31:32 +00:00
Eric Liu bc715504da [change-namespace] add leading '::' to references in new namespace when name conflict is possible.
Summary:
For example, when we change 'na' to "nb::nc", we need to add leading '::' to
references "::nc::X" in the changed namespace.

Reviewers: bkramer

Reviewed By: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D29176

llvm-svn: 293182
2017-01-26 15:08:44 +00:00
Haojian Wu 85867727fa [clang-move] Dump enum and type alias declarations.
Reviewers: ioeric

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D28293

llvm-svn: 292098
2017-01-16 09:34:07 +00:00
Malcolm Parsons e65b1029ca Remove unused lambda captures. NFC
llvm-svn: 291941
2017-01-13 19:02:50 +00:00