Commit Graph

1165 Commits

Author SHA1 Message Date
Kadir Cetinkaya 748211a81f [clangd] Only report explicitly typed symbols during code navigation
Summary:
Clangd was reporting implicit symbols, like results of implicit cast
expressions during code navigation, which is not desired. For example:

```
struct Foo{ Foo(int); };
void bar(Foo);
vod foo() {
  int x;
  bar(^x);
}
```
Performing a GoTo on the point specified by ^ would give two results one
pointing to line `int x` and the other for definition of `Foo(int);`

Reviewers: ilya-biryukov, sammccall

Subscribers: ioeric, MaskRay, jkorous, mgrang, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58495

llvm-svn: 354585
2019-02-21 14:48:33 +00:00
Kadir Cetinkaya 84b584be62 [clangd] Enable indexing of template type parameters
Summary: Fixes https://bugs.llvm.org/show_bug.cgi?id=36285

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58294

llvm-svn: 354561
2019-02-21 09:55:00 +00:00
Eric Liu 4971436b21 [clangd] Handle another incomplete-type diagnostic case in IncludeFixer.
llvm-svn: 354558
2019-02-21 09:33:49 +00:00
Ilya Biryukov fc83aea6a9 [clangd] Fix a crash in Selection
Summary:
The assertion checking that a range of a node is a token range does
not hold in case of "split" tokens, e.g. between two closing template
argument lists (`vector<vector<int>>`).

Reviewers: kadircet, sammccall

Reviewed By: kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58447

llvm-svn: 354507
2019-02-20 19:26:39 +00:00
Kadir Cetinkaya 592e7c9b0e [clangd] Revert r354442 and r354444
Looks like sysroot is only working on linux.

llvm-svn: 354453
2019-02-20 11:45:20 +00:00
Kadir Cetinkaya 60ba9b4460 [clangd] Try to fix windows build bots
llvm-svn: 354444
2019-02-20 10:32:04 +00:00
Kadir Cetinkaya 30d340839c [clangd] Testcase for bug 39811
Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58133

llvm-svn: 354442
2019-02-20 09:41:26 +00:00
Eric Liu 4df070a5e3 [clangd] Handle unresolved scope specifier when fixing includes.
Summary:
In the following examples, "clangd" is unresolved, and the fixer will try to fix
include for `clang::clangd`; however, clang::clangd::X is usually intended. So
when handling a qualifier that is unresolved, we change the unresolved name and
scopes so that the fixer will fix "clang::clangd::X" in the following example.
```
  namespace clang {
  	clangd::X
	~~~~~~
  }
  // or
  clang::clangd::X
         ~~~~~~
```

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58185

llvm-svn: 354330
2019-02-19 14:32:22 +00:00
Kadir Cetinkaya 9b4db445ce [clangd] Add tests for template specializations
Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58190

llvm-svn: 354272
2019-02-18 14:23:19 +00:00
Eric Liu b355802910 [clangd] Cache include fixes for diagnostics caused by the same unresolved name or incomplete type.
Summary:
Multiple diagnostics can be caused by the same unresolved name or incomplete type,
especially if the code is copy-pasted without #includes. The cache can avoid making
repetitive index requests, and thus reduce latency and allow more diagnostics to be
fixed (we limit the number of index requests for each parse).

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58239

llvm-svn: 354268
2019-02-18 13:12:10 +00:00
Eric Liu da2ed56fea [clangd] Handle a few more diag kinds in include fixer.
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58135

llvm-svn: 353926
2019-02-13 08:58:54 +00:00
Haojian Wu e64ee7c645 Revamp the "[clangd] Format tweak's replacements"
Summary:
This patch contains two parts:

1) reverts commit r353306.
2) move the format logic out from tweaks, keep tweaks API unchanged.

Reviewers: sammccall, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58051

llvm-svn: 353712
2019-02-11 15:18:11 +00:00
Eric Liu d11fbf523d [clangd] Prefer location from codegen files when merging symbols.
Summary:
For example, if an index symbol has location in a .proto file and an AST symbol
has location in a generated .proto.h file, then we prefer location in .proto
which is more meaningful to users.

Also use `mergeSymbols` to get the preferred location between AST location and index location in go-to-def.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58037

llvm-svn: 353708
2019-02-11 15:05:29 +00:00
Kadir Cetinkaya 429022995b [clangd] Index parameters in function decls
Reviewers: hokein

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57950

llvm-svn: 353696
2019-02-11 13:03:08 +00:00
Kadir Cetinkaya cf13bfee00 [clangd] Fix broken windows build bots.
llvm-svn: 353694
2019-02-11 13:01:47 +00:00
Kadir Cetinkaya db7fbcb038 [clangd] Make system header mappings available for PreambleParsedCallback
Summary:
SystemHeaderMappings were added only after takeIncludes call, which
resulted in getting mapping on main file ast updates but not on preamble ast
updates.
Fixes https://github.com/clangd/clangd/issues/8

Reviewers: hokein

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58029

llvm-svn: 353687
2019-02-11 10:31:13 +00:00
Eric Liu 37b45dc22c [clangd] Fix an assertion in TypoCorrection.
Summary: https://github.com/clangd/clangd/issues/7

Reviewers: sammccall, hokein

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57944

llvm-svn: 353514
2019-02-08 13:27:47 +00:00
Haojian Wu ddd64a4f50 [clangd] Fix an assertion failure in Selection.
Summary:
The assertion is triggered when the Decl is null.

Details for the assertion:

F0207 09:55:09.069385   47308 logging.cc:84] assert.h assertion failed at llvm/include/llvm/Support/Casting.h:105 in static bool llvm::isa_impl_cl<clang::TranslationUnitDecl, const clang::      Decl *>::doit(const From *) [To = clang::TranslationUnitDecl, From = const clang::Decl *]: Val && "isa<> used on a null pointer"
 15 *** Check failure stack trace: ***
 19     @     0x55615c1f7e06  __assert_fail
 20     @     0x55615a6297d8  clang::clangd::(anonymous namespace)::SelectionVisitor::TraverseDecl()
 21     @     0x55615a62f48d  clang::RecursiveASTVisitor<>::TraverseTemplateTemplateParmDecl()
 22     @     0x55615a62b264  clang::RecursiveASTVisitor<>::TraverseDecl()
 23     @     0x55615a62979c  clang::clangd::(anonymous namespace)::SelectionVisitor::TraverseDecl()
 24     @     0x55615a63060c  clang::RecursiveASTVisitor<>::TraverseClassTemplatePartialSpecializationDecl()
 25     @     0x55615a62ae45  clang::RecursiveASTVisitor<>::TraverseDecl()

Reviewers: sammccall

Subscribers: javed.absar, kristof.beyls, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57879

llvm-svn: 353421
2019-02-07 16:00:44 +00:00
Eric Liu a9e9c506aa [clangd] Suggest adding missing includes for typos (like include-fixer).
Summary:
This adds include-fixer feature into clangd based on D56903. Clangd now captures
diagnostics caused by typos and attach include insertion fixes to potentially
fix the typo.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits, kadircet, arphaman, mgrang, jkorous, MaskRay, javed.absar, ilya-biryukov, mgorny

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57021

llvm-svn: 353380
2019-02-07 09:23:22 +00:00
Eric Liu d7cb5b8a33 [clangd] Add type boost to fuzzy find in Dex.
Summary:
No noticeable impact on code completions overall except some improvement on
cross-namespace completion.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57815

llvm-svn: 353310
2019-02-06 15:36:23 +00:00
Haojian Wu 12e194cbb7 [clangd] Format tweak's replacements.
llvm-svn: 353306
2019-02-06 15:24:50 +00:00
Haojian Wu 5dcc66d0b8 [clangd] Some minor fixes.
Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57755

llvm-svn: 353283
2019-02-06 09:08:26 +00:00
Kadir Cetinkaya c3dcd2673e [clangd] Enable include insertion for static index
Summary:
This enables include insertion by adding canonical includes into
preambledata.

Reviewers: ioeric, ilya-biryukov

Subscribers: javed.absar, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D57508

llvm-svn: 353054
2019-02-04 16:19:57 +00:00
Eric Liu 44d6f9e131 [clangd] Fix heap-use-after-free after r352868
llvm-svn: 352957
2019-02-02 10:35:39 +00:00
Sam McCall 9c8f432617 [clangd] Expose SelectionTree to code tweaks, and use it for swap if branches.
Summary:
This reduces the per-check implementation burden and redundant work.
It also makes checks range-aware by default (treating the commonAncestor
as if it were a point selection should be good baseline behavior).

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57570

llvm-svn: 352876
2019-02-01 15:09:47 +00:00
Sam McCall 3186e3ceb8 [clangd] Lib to compute and represent selection under cursor.
Summary:
The primary problem this solves is to expose the codeAction selection to
AST-based refactorings in a way that makes it easy and efficient for them to
bind to the right parts of the AST.

It should also allow us to make XRefs based features (textDocument/definition)
more robust, more easily implement textDocument/typeDefinition etc.
As an example, template parameter references can be identified without special
handling.
There should be slight speedup too: we can prune most of the AST traversal
in most cases.

Elephant in the room: this is similar-but-different to Tooling/Refactoring/ASTSelection.
That captures a smaller set of AST nodes, has a slightly different way of
representing selections, and generally has mare features and does more work.
The overall shape is pretty similar, and yet I can't quite get to behave as I
expect.

Reviewers: ilya-biryukov, kadircet

Subscribers: mgorny, ioeric, MaskRay, jkorous, mgrang, arphaman

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57562

llvm-svn: 352874
2019-02-01 15:05:11 +00:00
Sam McCall a4cf26b499 [clangd] Penalize file-scope symbols in the ranking for non-completion queries
Patch by Nathan Ridge!

Differential Revision: https://reviews.llvm.org/D56653

llvm-svn: 352868
2019-02-01 13:07:37 +00:00
Sam McCall 866ba2c924 [clangd] Implement textDocument/declaration from LSP 3.14
Summary:
LSP now reflects the declaration/definition distinction.

Language server changes:
 - textDocument/definition now returns a definition if one is found, otherwise
   the declaration. It no longer returns declaration + definition if they are
   distinct.
 - textDocument/declaration returns the best declaration we can find.
 - For macros, the active macro definition is returned for both methods.
 - For include directive, the top of the target file is returned for both.
There doesn't appear to be a discovery mechanism (we can't return everything to
clients that only know about definition), so this changes existing behavior.
In practice, it should greatly reduce the fraction of the time we need to show
the user a menu of options.

C++ API changes:
 - findDefinitions is replaced by locateSymbolAt, which returns a
   vector<LocatedSymbol> - one for each symbol under the cursor.
 - this contains the preferred declaration, the definition (if found), and
   the symbol name
This API enables some potentially-neat extensions, like swapping between decl
and def, and exposing the symbol name to the UI in the case of multiple symbols.

Reviewers: hokein

Subscribers: ilya-biryukov, javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D57388

llvm-svn: 352864
2019-02-01 11:26:13 +00:00
Sam McCall 8a32f3b7a0 [clangd] Fix crash in applyTweak, remove TweakID alias.
Strings are complicated, giving them opaque names makes us forget
they're complicated.

llvm-svn: 352837
2019-02-01 05:41:50 +00:00
Ilya Biryukov 4399878082 [clangd] A code action to swap branches of an if statement
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: llvm-commits, mgorny, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D56611

llvm-svn: 352796
2019-01-31 21:30:05 +00:00
Eric Liu 00eaf6732e [clangd] Append "(fix available)" to diagnostic message when fixes are present.
Summary:
This would make diagnostic fixits more discoverable, especially for
plugins like YCM.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D57509

llvm-svn: 352764
2019-01-31 16:09:25 +00:00
Ilya Biryukov 060f7fe3e3 [clangd] Unit test for sourceLocationInMainFile.
This should have been part of r352494, which added the corresponding
function. The unit test ended up as a separate commit accidentally.

llvm-svn: 352501
2019-01-29 14:31:19 +00:00
Kadir Cetinkaya 249ea33ddf [clangd] Make USRs for macros to be position independent
Summary:
USRs for macros were not cannonical due to usage of cursor location
instead of definition location.

Reviewers: jkorous

Subscribers: ilya-biryukov, ioeric, MaskRay, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D57228

llvm-svn: 352481
2019-01-29 11:19:15 +00:00
Haojian Wu 7b6f874717 [clangd] Index main-file macros (bug 39761)
Patch by Nathan Ridge!

Differential Revision: https://reviews.llvm.org/D55739

llvm-svn: 352367
2019-01-28 14:11:49 +00:00
Eric Liu dd66277c36 [clangd] Suggest adding missing includes for incomplete type diagnostics.
Summary:
This enables clangd to intercept compiler diagnostics and attach fixes (e.g. by
querying index). This patch adds missing includes for incomplete types e.g.
member access into class with only forward declaration. This would allow adding
missing includes for user-typed symbol names that are missing declarations
(e.g. typos) in the future.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56903

llvm-svn: 352361
2019-01-28 14:01:55 +00:00
Haojian Wu aa3ed5a983 [clangd] NFC: fix clang-tidy warnings.
Most are about llvm code style violation (found via
readability-identifier-naming check).

llvm-svn: 352205
2019-01-25 15:14:03 +00:00
Ilya Biryukov c514adef05 [CodeComplete] [clangd] Fix crash on ValueDecl with a null type
Reviewers: kadircet

Reviewed By: kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D57093

llvm-svn: 352040
2019-01-24 10:41:43 +00:00
Ilya Biryukov 88d7d296f4 [clangd] Workaround a test failure after r351941
This should fix failing buildbots.

llvm-svn: 351943
2019-01-23 11:32:07 +00:00
Ilya Biryukov 4e0c400a47 [clangd] Fix crash due to ObjCPropertyDecl
With ObjCPropertyDecl, ASTNode.OrigD can be a ObjCPropertyImplDecl
which is not a NamedDecl, leading to a crash since the code
incorrectly assumes ASTNode.OrigD will always be a NamedDecl.

Change by dgoldman (David Goldman)!

Differential Revision: https://reviews.llvm.org/D56916

llvm-svn: 351941
2019-01-23 10:35:12 +00:00
Haojian Wu 87b0e3fda9 [clangd] Link clangTidy into clangd tests
Patch by Nathan Ridge!

Differential Revision: https://reviews.llvm.org/D57077

llvm-svn: 351929
2019-01-23 08:04:17 +00:00
Haojian Wu 1ca0c58c81 [clangd] Support clang-tidy configuration in clangd.
Summary:
This patch adds some basic supports for clang-tidy configurations in clangd:
      - clangd will respect .clang-tidy configurations for each file
      - we don't aim to support all clang-tidy options in clangd, only a
        small subset of condfigurations (options related to which checks will be
        enabled) are supported.
      - add a `clang-tidy-checks` CLI option that can override options from
        .clang-tidy file

Reviewers: ilya-biryukov, sammccall

Reviewed By: sammccall

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55256

llvm-svn: 351792
2019-01-22 09:39:05 +00:00
Kadir Cetinkaya be6b35dac4 [clangd] Filter out plugin related flags and move all commandline manipulations into OverlayCDB.
Summary:
Some projects make use of clang plugins when building, but clangd is
not aware of those plugins therefore can't work with the same compile command
arguments.

There were multiple places clangd performed commandline manipulations,
 this one also moves them all into OverlayCDB.

Reviewers: ilya-biryukov

Subscribers: klimek, sammccall, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D56841

llvm-svn: 351788
2019-01-22 09:10:20 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Jan Korous dca9c7cf24 [clangd] XPC transport layer
- New transport layer for macOS.
- XPC Framework
- Test client

Framework and client were written by Alex Lorenz.

Differential Revision: https://reviews.llvm.org/D54428

llvm-svn: 351280
2019-01-16 00:24:22 +00:00
Kadir Cetinkaya 226af75a02 [clangd] Fix updated file detection logic in indexing
Summary:
Files without any symbols were never marked as updated during indexing, which resulted in failure while writing shards for these files.

This patch fixes the logic to mark files that are seen for the first time but don't contain any symbols as updated.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D56592

llvm-svn: 351170
2019-01-15 09:03:33 +00:00
Haojian Wu c34f022bfe [clangd] Add Limit parameter for xref.
Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56597

llvm-svn: 351081
2019-01-14 18:11:09 +00:00
Sam McCall 0e93b076c4 [clangd] Index main-file symbols (bug 39761)
Patch by Nathan Ridge!

Differential Revision: https://reviews.llvm.org/D55185

llvm-svn: 351041
2019-01-14 10:01:17 +00:00
Kadir Cetinkaya 99b060e447 [clangd] Introduce loading of shards within auto-index
Summary:
Whenever a change happens on a CDB, load shards associated with that
CDB before issuing re-index actions.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D55224

llvm-svn: 350847
2019-01-10 17:03:04 +00:00
Haojian Wu 8f85b9f867 [clangd] Don't store completion info if the symbol is not used for code completion.
Summary:
This would save us some memory and disk space:
  - Dex usage (261 MB vs 266 MB)
  - Disk (75 MB vs 76 MB)

It would save more when we index the main file symbol D55185.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: nridge, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56314

llvm-svn: 350803
2019-01-10 09:22:40 +00:00
Haojian Wu 0b097b1a8a [clangd] Add a test for SignatureHelp on dynamic index.
Summary: This would catch regressions caused by future changes of the index.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56483

llvm-svn: 350720
2019-01-09 13:42:03 +00:00
Haojian Wu 073d184ee3 [clangd] Fix a crash when reading an empty index file.
Summary:
Unfortunately, yaml::Input::setCurrentDocument() and yaml::Input::nextDocument() are
internal APIs, the way we use them may cause a nullptr accessing when
processing an empty YAML file.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56442

llvm-svn: 350633
2019-01-08 15:24:47 +00:00
Ilya Biryukov 9d96c3c7f2 [clangd] Include <cstdio> instead of <stdio.h>. NFC
This fixes the only clang-tidy check currently enabled by clangd.

llvm-svn: 350540
2019-01-07 16:55:59 +00:00
Ilya Biryukov f2001aa743 [clangd] Remove 'using namespace llvm' from .cpp files. NFC
The new guideline is to qualify with 'llvm::' explicitly both in
'.h' and '.cpp' files. This simplifies moving the code between
header and source files and is easier to keep consistent.

llvm-svn: 350531
2019-01-07 15:45:19 +00:00
Haojian Wu e1ced5c926 [clangd] Fix a regression issue caused by r348365.
Summary:
With r348365, we now detect libc++ dir using the actual compiler path
(from the compilation command), rather than the resource-dir.

This new behavior will cause clangd couldn't find libc++ dir (even the libc++ is
built from the source) when using a fallback compilation command (`clang xxx`)

The fix is to use `<clangd_install_dir>/clang` as the actual compiler path.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56380

llvm-svn: 350515
2019-01-07 12:35:02 +00:00
Ilya Biryukov 082072873f [clangd] Disable BackgroundIndexTest.PeriodicalIndex
It sometimes fails on AArch64.

llvm-svn: 350512
2019-01-07 11:18:11 +00:00
Ilya Biryukov 22fa465a8c [clangd] clang-format everything. NFC
llvm-svn: 350303
2019-01-03 13:28:05 +00:00
Haojian Wu 744eb3c3be [clangd] Cleanup syntax errors in the test, NFC.
llvm-svn: 349893
2018-12-21 09:32:49 +00:00
Ilya Biryukov d363f08426 [clangd] Try to workaround test failure by increasing the timeouts
Ideally we'd figure out a way to run this test without any sleeps, this
workaround is only there to avoid annoying people with test failures
around the holiday period when everyone is on vacation.

llvm-svn: 349769
2018-12-20 16:27:19 +00:00
Haojian Wu b2d7e269d5 [clangd] Don't miss the expected type in merge.
Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55918

llvm-svn: 349750
2018-12-20 13:05:46 +00:00
Haojian Wu bbcfa7c1e2 [clangd] Fix a syntax error on the test.
llvm-svn: 349632
2018-12-19 14:51:07 +00:00
Eric Liu ae7ac3ca5b [clangd] Try to fix buildbot failure after r349496
Increase timeout from 10ms to 100ms.
See http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/27959

llvm-svn: 349502
2018-12-18 16:06:29 +00:00
Eric Liu 667e8ef7e1 [clangd] BackgroundIndex rebuilds symbol index periodically.
Summary:
Currently, background index rebuilds symbol index on every indexed file,
which can be inefficient. This patch makes it only rebuild symbol index periodically.
As the rebuild no longer happens too often, we could also build more efficient
dex index.

Reviewers: ilya-biryukov, kadircet

Reviewed By: kadircet

Subscribers: dblaikie, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D55770

llvm-svn: 349496
2018-12-18 15:39:33 +00:00
Haojian Wu 941ce02deb [clangd] Use buildCompilerInvocation to simplify the HeadersTests, NFC.
llvm-svn: 349148
2018-12-14 13:49:00 +00:00
Haojian Wu 23ae22b669 [clangd] Fix memory leak in ClangdTests.
Summary:
createInvocationFromCommandLine sets DisableFree to true by default,
which leads memory leak in clangd. The fix is to  use the `BuildCompilationInvocation`
to create CI with the correct options (DisableFree is false).

Fix https://bugs.llvm.org/show_bug.cgi?id=39991.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D55702

llvm-svn: 349145
2018-12-14 13:19:38 +00:00
Haojian Wu d5a78e6e59 [clangd] Fix an assertion failure in background index.
Summary:
When indexing a file which contains an uncompilable error, we will
trigger an assertion failure -- the IndexFileIn data is not set, but we
access them in the backgound index.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D55650

llvm-svn: 349144
2018-12-14 12:39:08 +00:00
Haojian Wu bf28533726 [clangd] Refine the way of checking a declaration is referenced by the written code.
Summary:
The previous solution (checking the AST) is not a reliable way to
determine whether a declaration is explicitly referenced by the source
code, we are still missing a few cases.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55191

llvm-svn: 349033
2018-12-13 13:17:04 +00:00
Haojian Wu e58e0903e2 [clangd] Avoid emitting Queued status when we are able to acquire the Barrier.
Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55359

llvm-svn: 349032
2018-12-13 13:09:50 +00:00
Fangrui Song 41c249adc4 Add explicit dependency on clangSerialization after rC348911
llvm-svn: 348916
2018-12-12 08:25:16 +00:00
Ilya Biryukov f30c4ad325 [clangd] Remove the test that sometimes deadlocks
Will figure out how to properly rewrite it and recommit.

llvm-svn: 348490
2018-12-06 15:14:11 +00:00
Haojian Wu a44eb93a4b [clangd] Update the test code
I forgot to update it in the last round of code review.

llvm-svn: 348478
2018-12-06 10:22:48 +00:00
Haojian Wu ec3d02013f [clangd] C++ API for emitting file status.
Introduce clangd C++ API to emit the current status of file.

llvm-svn: 348475
2018-12-06 09:41:04 +00:00
Haojian Wu e3fa82fcde [clangd] Fix a typo in TUSchedulerTests
Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55312

llvm-svn: 348467
2018-12-06 08:55:24 +00:00
Kadir Cetinkaya 6f9b204261 [clangd] Dont provide locations for non-existent files.
Summary:
We were getting assertion errors when we had bad file names, instead we
should skip those.

Reviewers: hokein

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D55275

llvm-svn: 348359
2018-12-05 11:57:15 +00:00
Mikael Holmen 18e800f0dd Fix compilation error when using clang 3.6.0
llvm-svn: 348357
2018-12-05 11:29:27 +00:00
Kadir Cetinkaya 219c0fae5c [clangd] Partition include graph on auto-index.
Summary:
Partitions include graphs in auto-index so that each shards contains
only part of the include graph related to itself.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D55062

llvm-svn: 348252
2018-12-04 11:31:57 +00:00
Kadir Cetinkaya 5399552da1 [clangd] Populate include graph during static indexing action.
Summary:
This is the second part for introducing include hierarchy into index
files produced by clangd. You can see the base patch that introduces structures
and discusses the future of the patches in D54817

Reviewers: ilya-biryukov

Subscribers: mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54999

llvm-svn: 348005
2018-11-30 16:59:00 +00:00
Eric Liu 5ac37f495a [clangd] Penalize destructor and overloaded operators in code completion.
Reviewers: hokein

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55061

llvm-svn: 347983
2018-11-30 11:17:15 +00:00
Eric Liu b1317fa8df [clangd] Drop injected class name when class scope is not explicitly specified.
Summary: E.g. allow injected "A::A" in `using A::A^` but not in "A^".

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D55065

llvm-svn: 347982
2018-11-30 11:12:40 +00:00
Eric Liu 4d814a93e5 [clangd] Canonicalize file path in URIForFile.
Summary:
File paths in URIForFile can come from index or local AST. Path from
index goes through URI transformation and the final path is resolved by URI
scheme and could be potentially different from the original path. Hence, we
should do the same transformation for all paths. We do this in URIForFile, which
now converts a path to URI and back to a canonicalized path.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54845

llvm-svn: 347739
2018-11-28 10:30:42 +00:00
Jan Korous b406701f66 [clangd] textDocument/SymbolInfo extension
New method returning symbol info for given source position.

Differential Revision: https://reviews.llvm.org/D54799

rdar://problem/46050281

llvm-svn: 347675
2018-11-27 16:40:46 +00:00
Kadir Cetinkaya d08eab4281 [clangd] Put direct headers into srcs section.
Summary:
Currently, there's no way of knowing about header files
using compilation database, since it doesn't contain header files as entries.

Using this information, restoring from cache using compile commands becomes
possible instead of doing directory traversal. Also, we can issue indexing
actions for out-of-date headers even if source files depending on them haven't
changed.

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54817

llvm-svn: 347669
2018-11-27 16:08:53 +00:00
Sam McCall 032f3e7fd8 [clangd] Prevent thread starvation in tests on loaded systems.
Summary:
Background index deliberately runs low-priority, but for tests this may stop
them making progress.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D54938

llvm-svn: 347655
2018-11-27 12:09:13 +00:00
Ilya Biryukov e8ccb8238d [clangd] Do not drop diagnostics from macros
if they still end up being in the main file.

llvm-svn: 347574
2018-11-26 17:05:13 +00:00
Sam McCall 422c828dfc [clangd] Enable auto-index behind a flag.
Summary:
Ownership and configuration:
The auto-index (background index) is maintained by ClangdServer, like Dynamic.
(This means ClangdServer will be able to enqueue preamble indexing in future).
For now it's enabled by a simple boolean flag in ClangdServer::Options, but
we probably want to eventually allow injecting the storage strategy.

New 'sync' command:
In order to meaningfully test the integration (not just unit-test components)
we need a way for tests to ensure the asynchronous index reads/writes occur
before a certain point.
Because these tests and assertions are few, I think exposing an explicit "sync"
command for use in tests is simpler than allowing threading to be completely
disabled in the background index (as we do for TUScheduler).

Bugs:
I fixed a couple of trivial bugs I found while testing, but there's one I can't.
JSONCompilationDatabase::getAllFiles() may return relative paths, and currently
we trigger an assertion that assumes they are absolute.
There's no efficient way to resolve them (you have to retrieve the corresponding
command and then resolve against its directory property). In general I think
this behavior is broken and we should fix it in JSONCompilationDatabase and
require CompilationDatabase::getAllFiles() to be absolute.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54894

llvm-svn: 347567
2018-11-26 16:00:11 +00:00
Ilya Biryukov d360b2984e [clangd] Initial implementation of expected types
Summary:
Provides facilities to model the C++ conversion rules without the AST.
The introduced representation can be stored in the index and used to
implement type-based ranking improvements for index-based completions.

Reviewers: sammccall, ioeric

Reviewed By: sammccall

Subscribers: malaperle, mgorny, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52273

llvm-svn: 347559
2018-11-26 15:25:20 +00:00
Haojian Wu 43efc6ba65 [clangd] Use testPath in the test.
llvm-svn: 347547
2018-11-26 11:18:16 +00:00
Sam McCall 6e2d2a33b6 [clangd] Auto-index watches global CDB for changes.
Summary:
Instead of receiving compilation commands, auto-index is triggered by just
filenames to reindex, and gets commands from the global comp DB internally.
This has advantages:
 - more of the work can be done asynchronously (fetching compilation commands
   upfront can be slow for large CDBs)
 - we get access to the CDB which can be used to retrieve interpolated commands
   for headers (useful in some cases where the original TU goes away)
 - fits nicely with the filename-only change observation from r347297

The interface to GlobalCompilationDatabase gets extended: when retrieving a
compile command, the GCDB can optionally report the project the file belongs to.
This naturally fits together with getCompileCommand: it's hard to implement one
without the other. But because most callers don't care, I've ended up with an
awkward optional-out-param-in-virtual method pattern - maybe there's a better
one.

This is the main missing integration point between ClangdServer and
BackgroundIndex, after this we should be able to add an auto-index flag.

Reviewers: ioeric, kadircet

Subscribers: MaskRay, jkorous, arphaman, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D54865

llvm-svn: 347538
2018-11-26 09:51:50 +00:00
Ilya Biryukov 19d75608f8 [clangd] Add support for hierarchical documentSymbol
Reviewers: ioeric, sammccall, simark

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52311

llvm-svn: 347498
2018-11-23 15:21:19 +00:00
Ilya Biryukov 16cc195e8d [clangd] Cleanup: make the diags callback global in TUScheduler
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54760

llvm-svn: 347474
2018-11-22 17:27:08 +00:00
Ilya Biryukov 27b83d230d [clangd] Cleanup error consumption code. NFC
- Remove reimplementations of llvm::consumeError.
- Simplify test code by using EXPECT_ERROR where it fits.

llvm-svn: 347472
2018-11-22 16:20:12 +00:00
Ilya Biryukov c76394bc4b [clangd] Cleanup: make diagnostics callbacks from TUScheduler non-racy
Summary:
Previously, removeDoc followed by an addDoc to TUScheduler resulted in
racy diagnostic responses, i.e. the old dianostics could be delivered
to the client after the new ones by TUScheduler.

To workaround this, we tracked a version number in ClangdServer and
discarded stale diagnostics. After this commit, the TUScheduler will
stop delivering diagnostics for removed files and the workaround in
ClangdServer is not required anymore.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, jfb, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54829

llvm-svn: 347468
2018-11-22 15:39:54 +00:00
Eric Liu c0ac4bb17c [clangd] Cleanup: stop passing around list of supported URI schemes.
Summary:
Instead of passing around a list of supported URI schemes in clangd, we
expose an interface to convert a path to URI using any compatible scheme
that has been registered. It favors customized schemes and falls
back to "file" when no other scheme works.

Changes in this patch are:
- URI::create(AbsPath, URISchemes) -> URI::create(AbsPath). The new API finds a
compatible scheme from the registry.
- Remove URISchemes option everywhere (ClangdServer, SymbolCollecter, FileIndex etc).
- Unit tests will use "unittest" by default.
- Move "test" scheme from ClangdLSPServer to ClangdMain.cpp, and only
register the test scheme when lit-test or enable-lit-scheme is set.
(The new flag is added to make lit protocol.test work; I wonder if there
is alternative here.)

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54800

llvm-svn: 347467
2018-11-22 15:02:05 +00:00
Benjamin Kramer 2c3477596f [clangd] Fix use-after-scope in unit test
The scheduler holds a reference to `Proceed`, so it has to be destroyed
after the scheduler. Found by asan.

llvm-svn: 347460
2018-11-22 12:54:25 +00:00
Sam McCall a2b048bc4b [clangd] Respect task cancellation in TUScheduler.
Summary:
- Reads are never executed if canceled before ready-to run.
  In practice, we finalize cancelled reads eagerly and out-of-order.
- Cancelled reads don't prevent prior updates from being elided, as they don't
  actually depend on the result of the update.
- Updates are downgraded from WantDiagnostics::Yes to WantDiagnostics::Auto when
  cancelled, which allows them to be elided when all dependent reads are
  cancelled and there are subsequent writes. (e.g. when the queue is backed up
  with cancelled requests).

The queue operations aren't optimal (we scan the whole queue for cancelled
tasks every time the scheduler runs, and check cancellation twice in the end).
However I believe these costs are still trivial in practice (compared to any
AST operation) and the logic can be cleanly separated from the rest of the
scheduler.

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54746

llvm-svn: 347450
2018-11-22 10:22:16 +00:00
Sam McCall 991e316126 [clangd] Replay preamble #includes to clang-tidy checks.
Summary:
This is needed to correctly handle checks that use IncludeInserter,
which is very common.

I couldn't find a totally safe example of a check to enable for testing,
I picked modernize-deprecated-headers which some will probably hate.
We should get configuration working...

This depends on D54691 which ensures our calls to getFile(open=false)
don't break subsequent accesses via the FileManager.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54694

llvm-svn: 347298
2018-11-20 10:58:48 +00:00
Sam McCall 2bebc3d060 [clangd] Allow observation of changes to global CDBs.
Summary:
Currently, changes *within* CDBs are not tracked (CDB has no facility to do so).
However, discovery of new CDBs are tracked (all files are marked as modified).
Also, files whose compilation commands are explicitly set are marked modified.

The intent is to use this for auto-index. Newly discovered files will be indexed
with low priority.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54475

llvm-svn: 347297
2018-11-20 10:56:03 +00:00
Kadir Cetinkaya ca9e5dc714 [clangd] Store source file hash in IndexFile{In,Out}
Summary:
Puts the digest of the source file that generated the index into
serialized index and stores them back on load, if exists.

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54693

llvm-svn: 347235
2018-11-19 18:06:29 +00:00
Marc-Andre Laperle f975400838 [clangd] Fix crash hovering on non-decltype trailing return
Summary:
More specifically, hovering on "auto" in
auto main() -> int {
    return 0;
}

Signed-off-by: Marc-Andre Laperle <malaperle@gmail.com>

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54553

llvm-svn: 347119
2018-11-17 00:41:14 +00:00
Haojian Wu 22c9f7b296 [clangd] Truncate SymbolID to 8 bytes.
Summary:
This is our goal. It has a non-zero rick, but so far we haven't see any
collision (externally and internally).

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54622

llvm-svn: 347044
2018-11-16 10:58:40 +00:00
Kadir Cetinkaya 06553bfe96 Introduce shard storage to auto-index.
Reviewers: sammccall, ioeric

Reviewed By: sammccall

Subscribers: llvm-commits, mgorny, Eugene.Zelenko, ilya-biryukov, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54269

llvm-svn: 347038
2018-11-16 09:03:56 +00:00
Sam McCall 98ae975187 [clangd] Initial clang-tidy diagnostics support.
Summary:
This runs checks over a restricted subset of the TU:
 - preprocessor callbacks just receive the truncated PP events that
   occur when a preamble is used.
 - ASTMatchers run only over the top-level decls in the main-file

This patch just turns on one simple check (bugprone-sizeof-expression)
with no configuration. Configuration is complex enough to warrant a separate patch

This depends on a patch allowing traversal to be restricted to a scope.

Reviewers: hokein

Subscribers: srhines, mgorny, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54204

llvm-svn: 347036
2018-11-16 08:32:23 +00:00
Kadir Cetinkaya 5a9b92ca75 Revert "Introduce shard storage to auto-index."
This reverts commit 6dd1f24aead10a8d375d0311001987198d26e900.

llvm-svn: 346945
2018-11-15 10:34:47 +00:00
Kadir Cetinkaya bd2441c887 Revert "clang-format"
This reverts commit 0a37e9c3d88a2e21863657df2f7735fb7e5f746e.

llvm-svn: 346944
2018-11-15 10:34:43 +00:00
Kadir Cetinkaya ed18e788f0 Revert "Address comments"
This reverts commit 19a39b14eab2b5339325e276262b177357d6b412.

llvm-svn: 346943
2018-11-15 10:34:39 +00:00
Kadir Cetinkaya 8b9fed3e8d Revert "Address comments."
This reverts commit b43c4d1c731e07172a382567f3146b3c461c5b69.

llvm-svn: 346942
2018-11-15 10:34:35 +00:00
Kadir Cetinkaya 2bed2cf791 Address comments.
llvm-svn: 346941
2018-11-15 10:31:23 +00:00
Kadir Cetinkaya 89a7691fd9 Address comments
llvm-svn: 346940
2018-11-15 10:31:19 +00:00
Kadir Cetinkaya cb8407ca89 clang-format
llvm-svn: 346939
2018-11-15 10:31:15 +00:00
Kadir Cetinkaya 3e5a47560c Introduce shard storage to auto-index.
Reviewers: sammccall, ioeric

Subscribers: ilya-biryukov, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54269

llvm-svn: 346938
2018-11-15 10:31:10 +00:00
Scott Linder c0830f5577 [Support] Teach YAMLIO about polymorphic types
Add support for "polymorphic" types to YAMLIO.

PolymorphicTraits can dynamically switch between other traits (Scalar, Map, or
Sequence). When inputting, the PolymorphicTraits type is told which type to
become, and when outputting the PolymorphicTraits type is asked which type it
currently is.

Also add support for TaggedScalarTraits to allow dynamically differentiating
between multiple scalar types using YAML tags.

Serialize empty maps as "{}" and empty sequences as "[]", so that types
are preserved when round-tripping PolymorphicTraits. This change has
equivalent semantics, but may break e.g. tests which compare output
verbatim.

Differential Revision: https://reviews.llvm.org/D48144

llvm-svn: 346884
2018-11-14 19:39:59 +00:00
Kadir Cetinkaya c9b1b100e4 [clangd] Delete unused includes.
llvm-svn: 346872
2018-11-14 17:07:39 +00:00
Haojian Wu ee54a2b501 [clangd] Replace StringRef in SymbolLocation with a char pointer.
Summary:
This would save us 8 bytes per ref, and buy us ~40MB in total
for llvm index (from ~300MB to ~260 MB).

The char pointer must be null-terminated, and llvm::StringSaver
guarantees it.

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53427

llvm-svn: 346852
2018-11-14 11:55:45 +00:00
Sam McCall 39e86fa5e4 [clangd] Improve code completion for ObjC methods
Summary:
Previously code completion did not work well for Objective-C methods
which contained multiple arguments as clangd did not expect to see
multiple typed-text chunks when handling code completion.

Note that even with this change, we do not consider selector fragments
from previous arguments to be part of the signature (although we
could in the future).

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, jfb, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53934

llvm-svn: 346836
2018-11-14 09:05:19 +00:00
Kadir Cetinkaya 883a2c0519 [clangd] Fix clang-tidy warnings.
llvm-svn: 346524
2018-11-09 17:33:48 +00:00
Sam McCall 9ac2f7a1f8 [clangd] Don't treat top-level decls as "local" if they are from the preamble.
Summary:
These get passed to HandleTopLevelDecl() if they happen to have been
deserialized for any reason. We don't want to treat them as part of the
main file.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54303

llvm-svn: 346503
2018-11-09 15:35:00 +00:00
Sam McCall 72ccf00b17 [clangd] Make TestTU build with preamble, and fix the fallout.
Our testing didn't reflect reality: live clangd almost always uses a
preamble, and sometimes the preamble behaves differently.
This patch fixes a common test helper to be more realistic.

Preamble doesn't preserve information about which tokens come from the
command-line (this gets inlined into a source file). So remove logic
that attempts to treat symbols with such names differently.

A SymbolCollectorTest tries to verify that locals in headers are not
indexed, with preamble enabled this is only meaningful for locals of
auto-typed functions (otherwise the bodies aren't parsed).

Tests were relying on the fact that the findAnyDecl helper actually did expose
symbols from headers. Resolve by making all these functions consistently
able to find symbols in headers/preambles.

llvm-svn: 346488
2018-11-09 12:56:49 +00:00
Sam McCall 3c1f4903b7 [clang-tidy] Untangle layering in ClangTidyDiagnosticConsumer somewhat. NFC
Summary:
Clang's hierarchy is CompilerInstance -> DiagnosticsEngine -> DiagnosticConsumer.
(Ownership is optional/shared, but this structure is fairly clear).

Currently ClangTidyDiagnosticConsumer *owns* the DiagnosticsEngine:
 - this inverts the hierarchy, which is confusing
 - this means ClangTidyDiagnosticConsumer() mutates the passed-in context, which
   is both surprising and limits flexibility
 - it's not possible to use a different DiagnosticsEngine with ClangTidy

This means a little bit more code in the places ClangTidy is used standalone,
but more flexibility in using ClangTidy with other diagnostics configurations.

Reviewers: hokein

Subscribers: xazax.hun, cfe-commits

Differential Revision: https://reviews.llvm.org/D54033

llvm-svn: 346418
2018-11-08 17:42:16 +00:00
Haojian Wu f761a2c620 [clangd] Drop namespace references in the index.
Summary:
Namespace references is less useful compared with other symbols, and
they contribute large part of the index. This patch drops them.
The number of refs is reduced from 5.4 million to 4.7 million.

|           |  Before | After |
|file size  |  78 MB  |  71MB |
|memory     |  330MB  |  300MB|

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54202

llvm-svn: 346319
2018-11-07 14:59:24 +00:00
Kadir Cetinkaya f84a7d8d4f [clangd] [NFC] Fix clang-tidy warnings.
Reviewers: ioeric, sammccall, ilya-biryukov, hokein

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54157

llvm-svn: 346308
2018-11-07 12:25:27 +00:00
Eric Liu abbd713eb7 [clangd] Deduplicate query scopes.
Summary:
For example, when anonymous namespace is present, duplicated namespaces might be
generated for the enclosing namespace.

Reviewers: ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54105

llvm-svn: 346224
2018-11-06 11:17:40 +00:00
Eric Liu b04869a4aa [clangd] Get rid of QueryScopes.empty() == AnyScope special case.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53933

llvm-svn: 346223
2018-11-06 11:08:17 +00:00
Eric Liu ad588af2d6 [clangd] auto-index stores symbols per-file instead of per-TU.
Summary:
This allows us to deduplicate header symbols across TUs. File digests
are collects when collecting symbols/refs. And the index store deduplicates
file symbols based on the file digest.

Reviewers: sammccall, hokein

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53433

llvm-svn: 346221
2018-11-06 10:55:21 +00:00
Sam McCall c55d09a00e [clangd] Make in-memory CDB always available as an overlay, refactor.
Summary:
The new implementation is a GlobalCompilationDatabase that overlays a base.
Normally this is the directory-based CDB.
To preserve the behavior of compile_args_from=LSP, the base may be null.

The OverlayCDB is always present, and so the extensions to populate it
are always supported.

It also allows overriding the flags of the fallback command. This is
just unit-tested for now, but the plan is to expose this as an extension
on the initialize message. This addresses use cases like
https://github.com/thomasjo/atom-ide-cpp/issues/16

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53687

llvm-svn: 345970
2018-11-02 13:09:36 +00:00
Sam McCall cb50e23ba8 [clang-tidy] Get ClangTidyContext out of the business of storing diagnostics. NFC
Summary:
Currently ClangTidyContext::diag() sends the diagnostics to a
DiagnosticsEngine, which probably delegates to a ClangTidyDiagnosticsConsumer,
which is supposed to go back and populate ClangTidyContext::Errors.

After this patch, the diagnostics are stored in the ClangTidyDiagnosticsConsumer
itself and can be retrieved from there.

Why?
 - the round-trip from context -> engine -> consumer -> context is confusing
   and makes it harder to establish layering between these things.
 - context does too many things, and makes it hard to use clang-tidy as a library
 - everyone who actually wants the diagnostics has access to the ClangTidyDiagnosticsConsumer

The most natural implementation (ClangTidyDiagnosticsConsumer::take()
finalizes diagnostics) causes a test failure: clang-tidy-run-with-database.cpp
asserts that clang-tidy exits successfully when trying to process a file
that doesn't exist.
In clang-tidy today, this happens because finish() is never called, so the
diagnostic is never flushed. This looks like a bug to me.
For now, this patch carefully preserves that behavior, but I'll ping the
authors to see whether it's deliberate and worth preserving.

Reviewers: hokein

Subscribers: xazax.hun, cfe-commits, alexfh

Differential Revision: https://reviews.llvm.org/D53953

llvm-svn: 345961
2018-11-02 10:01:59 +00:00
Stephen Kelly a49fe5d878 [clang-query] Add non-exclusive output API
Summary:
Add granular options for AST dumping, text printing and diagnostics.

This makes it possible to

* Have both diag and dump active at once
* Extend the output with other queryable content in the future.

Reviewers: aaron.ballman, pcc, ioeric, ilya-biryukov, klimek, sammccall

Reviewed By: aaron.ballman

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D52857

llvm-svn: 345522
2018-10-29 18:59:56 +00:00
Sam McCall 6d4eb46d0c [clangd] workspace/symbol should be async, it reads from the index.
Summary:
To enable this, TUScheduler has to provide a way to run async tasks without
needing a preamble or AST!

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, jfb, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53644

llvm-svn: 345268
2018-10-25 14:19:14 +00:00
Douglas Yung 617eba1b1b Fix test to work on Windows.
llvm-svn: 345229
2018-10-25 01:21:08 +00:00
Stephen Kelly 70d771714e [clang-query] Refactor Output settings to booleans
Summary: This will make it possible to add non-exclusive mode output.

Reviewers: aaron.ballman

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D53501

llvm-svn: 345194
2018-10-24 20:33:55 +00:00
Stephen Kelly 51707b21a0 [clang-query] Add 'detailed-ast' output as an alias for 'dump'
Summary: Future development can then dump other content than AST.

Reviewers: aaron.ballman

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D53500

llvm-svn: 345193
2018-10-24 20:33:45 +00:00
Kadir Cetinkaya b915790385 [clangd] Do not query index for new name completions.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D53192

llvm-svn: 345153
2018-10-24 15:24:29 +00:00
Sam McCall e8437cbf5e [clangd] Don't show base class versions of members as completions.
Summary:
These are available via qualifiers, but signal to noise level is low.
Keep required quailifier machinery around though, for cross-ns completion.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53571

llvm-svn: 345141
2018-10-24 13:51:44 +00:00
Eric Liu 52a11b5662 [clangd] Downrank members from base class
Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53638

llvm-svn: 345140
2018-10-24 13:45:17 +00:00
Ilya Biryukov 5a74a9cf5f [clangd] Simplify auto hover
Summary:
Use helper from clang. Also fixes some weird corner cases, e.g.
    auto (*foo)() = bar;

Reviewers: kadircet, hokein

Reviewed By: kadircet, hokein

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D53347

llvm-svn: 345128
2018-10-24 10:09:34 +00:00
Sam McCall 16e7070e3e [clangd] Embed fixes as CodeAction, instead of clangd_fixes. Clean up serialization.
Summary:
CodeAction provides us with a standard way of representing fixes inline, so
use it, replacing our existing ad-hoc extension.

After this, it's easy to serialize diagnostics using the structured
toJSON/Protocol.h mechanism rather than assembling JSON ad-hoc.

Reviewers: hokein, arphaman

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53391

llvm-svn: 345119
2018-10-24 07:59:38 +00:00
Sam McCall 668ac94ba4 [clangd] Truncate SymbolID to 16 bytes.
Summary:
The goal is 8 bytes, which has a nonzero risk of collisions with huge indexes.
This patch should shake out any issues with truncation at all, we can lower
further later.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53587

llvm-svn: 345113
2018-10-24 06:58:42 +00:00
Sam McCall 7189112951 Fix range length comparison in DraftStore::UpdateDraft when Unicode characters are removed from the document
Summary:
See http://lists.llvm.org/pipermail/clangd-dev/2018-October/000171.html for context.

I kept the error (instead of downgrading to a log message) since the range lengths differing does indicate either a bug in the client or server range calculation or the buffers being out of sync (which both seems serious enough to me to be an error). If any existing clients aside from VSCode break they should only break when accidentally typing a Unicode character which should only be a minor nuisance for a little while until the bug is fixed in the respective client.

Patch by Daan De Meyer!

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D53527

llvm-svn: 345020
2018-10-23 11:51:53 +00:00
Eric Liu c6c894b88f [change-namespace] Enhance detection of conflicting namespaces.
Summary:
For example:
```
namespace util { class Base; }

namespace new {
namespace util { class Internal; }
}

namespace old {
util::Base b1;
}
```

When changing `old::` to `new::`, `util::` in namespace "new::" will conflict
with "new::util::" unless a leading "::" is added.

Reviewers: hokein

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D53489

llvm-svn: 344897
2018-10-22 12:48:49 +00:00
Akira Hatanaka aca3a9f03f Check that __MAC_OS_X_VERSION_MIN_REQUIRED is defined before checking
whether it is too old.

llvm-svn: 344856
2018-10-20 17:35:50 +00:00
Sam McCall c008af6466 [clangd] Namespace style cleanup in cpp files. NFC.
Standardize on the most common namespace setup in our *.cpp files:
  using namespace llvm;
  namespace clang {
  namespace clangd {
  void foo(StringRef) { ... }
And remove redundant llvm:: qualifiers. (Except for cases like
make_unique where this causes problems with std:: and ADL).

This choice is pretty arbitrary, but some broad consistency is nice.
This is going to conflict with everything. Sorry :-/

Squash the other configurations:

A)
  using namespace llvm;
  using namespace clang;
  using namespace clangd;
  void clangd::foo(StringRef);
This is in some of the older files. (It prevents accidentally defining a
new function instead of one in the header file, for what that's worth).

B)
  namespace clang {
  namespace clangd {
  void foo(llvm::StringRef) { ... }
This is fine, but in practice the using directive often gets added over time.

C)
  namespace clang {
  namespace clangd {
  using namespace llvm; // inside the namespace
This was pretty common, but is a bit misleading: name lookup preferrs
clang::clangd::foo > clang::foo > llvm:: foo (no matter where the using
directive is).

llvm-svn: 344850
2018-10-20 15:30:37 +00:00
Stephen Kelly 4a5b01ddd7 [clang-query] Add option to print matcher expression
Summary:
This is useful if using clang-query -f with a file containing multiple
matchers.

Reviewers: aaron.ballman

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D52859

llvm-svn: 344840
2018-10-20 09:13:59 +00:00
Akira Hatanaka 2f7dc72525 Disable unittests/clangd/JSONTransportTests.cpp on versions of macosx
earlier than 10.13.

rdar://problem/45310173

llvm-svn: 344827
2018-10-20 02:11:45 +00:00
Sam McCall 0d9b40f583 [clangd] Set workspace root when initializing ClangdServer, disallow mutation.
Summary:
Rename instance variable to WorkspaceRoot to match what we call it internally.

Add fixme to set it automatically. Don't do it yet, clients have assumptions
that the constructor won't access the FS.

Don't second-guess the provided root.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53404

llvm-svn: 344787
2018-10-19 15:42:23 +00:00
Haojian Wu 812b6c51c3 [clangd] Remove the overflow log.
Summary:
LLVM codebase has generated files (all are build/Target/XXX/*.inc) that
exceed the MaxLine & MaxColumn. Printing these log would be noisy.

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53400

llvm-svn: 344777
2018-10-19 08:35:24 +00:00
Eric Liu 4859738cfe [clangd] Names that are not spelled in source code are reserved.
Summary:
These are often not expected to be used directly e.g.
```
TEST_F(Fixture, X) {
  ^  // "Fixture_X_Test" expanded in the macro should be down ranked.
}
```

Only doing this for sema for now, as such symbols are mostly coming from sema
e.g. gtest macros expanded in the main file. We could also add a similar field
for the index symbol.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53374

llvm-svn: 344736
2018-10-18 12:23:05 +00:00
Haojian Wu b515fabb3b [clangd] Encode Line/Column as a 32-bits integer.
Summary:
This would buy us more memory. Using a 32-bits integer is enough for
most human-readable source code (up to 4M lines and 4K columns).

Previsouly, we used 8 bytes for a position, now 4 bytes, it would save
us 8 bytes for each Ref and each Symbol instance.

For LLVM-project binary index file, we save ~13% memory.

| Before | After |
| 412MB  | 355MB |

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53363

llvm-svn: 344735
2018-10-18 10:43:50 +00:00
Julie Hockett bf445b87de [clang-doc] Bringing bitcode tests in line
Makes bitcode tests line up with what's actually called in the tool.
Should fix the failing bot.

Also fixes a warning that was being thrown about initialization braces.

Differential Revision: https://reviews.llvm.org/D53381

llvm-svn: 344707
2018-10-17 20:16:05 +00:00
Eric Liu 3fac4ef1fd [clangd] Support scope proximity in code completion.
Summary:
This should make all-scope completion more usable. Scope proximity for
indexes will be added in followup patch.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53131

llvm-svn: 344688
2018-10-17 11:19:02 +00:00
Haojian Wu 7dd4950ea5 [clangd] Collect refs from headers.
Summary:
Add a flag to SymbolCollector to collect refs fdrom headers.

Note that we collect refs from headers in static index, and we don't do it for
dynamic index because of the preamble (we skip function body in preamble,
collecting it will result incomplete results).

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53322

llvm-svn: 344678
2018-10-17 08:38:36 +00:00
Sam McCall dc8f3cf8b0 [clangd] Refactor JSON-over-stdin/stdout code into Transport abstraction. (re-land r344620)
Summary:
This paves the way for alternative transports (mac XPC, maybe messagepack?),
and also generally improves layering: testing ClangdLSPServer becomes less of
a pipe dream, we split up the JSONOutput monolith, etc.

This isn't a final state, much of what remains in JSONRPCDispatcher can go away,
handlers can call reply() on the transport directly, JSONOutput can be renamed
to StreamLogger and removed, etc. But this patch is sprawling already.

The main observable change (see tests) is that hitting EOF on input is now an
error: the client should send the 'exit' notification.
This is defensible: the protocol doesn't spell this case out. Reproducing the
current behavior for all combinations of shutdown/exit/EOF clutters interfaces.
We can iterate on this if desired.

Reviewers: jkorous, ioeric, hokein

Subscribers: mgorny, ilya-biryukov, MaskRay, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53286

llvm-svn: 344672
2018-10-17 07:32:05 +00:00
Julie Hockett 7f87ca62a2 [clang-doc] Add unit tests for Markdown generation
Add unit tests for Markdown generation.

This is part of a move to convert clang-doc's tests to a more
maintainable unit test framework, with a smaller number of integration
tests to maintain and more granular failure feedback.

Differential Revision: https://reviews.llvm.org/D53085

llvm-svn: 344654
2018-10-16 23:07:26 +00:00
Julie Hockett 55974d3b93 [clang-doc] Add unit tests for YAML generation
Adds unit tests for the YAML generator library.

This is part of a move to convert clang-doc's tests to a more
maintainable unit test framework, with a smaller number of integration
tests to maintain and more granular failure feedback.

Differential Revision: https://reviews.llvm.org/D53084

llvm-svn: 344653
2018-10-16 23:07:16 +00:00
Julie Hockett 8fc72fcb31 [clang-doc] Add unit tests for merging
Adds unit tests for the merging logic in Respresentation.cpp.

This is part of a move to convert clang-doc's tests to a more
maintainable unit test framework, with a smaller number of integration
tests to maintain and more granular failure feedback.

Differential Revision: https://reviews.llvm.org/D53083

llvm-svn: 344652
2018-10-16 23:07:04 +00:00
Julie Hockett f276624e87 [clang-doc] Add unit tests for bitcode
Adds unit tests for the BitcodeWriter and BitcodeReader libraries.

This is part of a move to convert clang-doc's tests to a more
maintainable unit test framework, with a smaller number of integration
tests to maintain and more granular failure feedback.

Differential Revision: https://reviews.llvm.org/D53082

llvm-svn: 344651
2018-10-16 23:06:53 +00:00
Julie Hockett 68049b1a8e [clang-doc] Add unit tests for serialization
Adds unit tests for the Serialize library.

This is part of a move to convert clang-doc's tests to a more
maintainable unit test framework, with a smaller number of integration
tests to maintain and more granular failure feedback.

Differential Revision: https://reviews.llvm.org/D53081

llvm-svn: 344650
2018-10-16 23:06:42 +00:00
Krasimir Georgiev ae4e75fd6e Revert "[clangd] Refactor JSON-over-stdin/stdout code into Transport abstraction."
This reverts commit r344620.
Breaks upstream bots.

llvm-svn: 344637
2018-10-16 18:44:41 +00:00
Sam McCall 818ec5e965 [clangd] Refactor JSON-over-stdin/stdout code into Transport abstraction.
Summary:
This paves the way for alternative transports (mac XPC, maybe messagepack?),
and also generally improves layering: testing ClangdLSPServer becomes less of
a pipe dream, we split up the JSONOutput monolith, etc.

This isn't a final state, much of what remains in JSONRPCDispatcher can go away,
handlers can call reply() on the transport directly, JSONOutput can be renamed
to StreamLogger and removed, etc. But this patch is sprawling already.

The main observable change (see tests) is that hitting EOF on input is now an
error: the client should send the 'exit' notification.
This is defensible: the protocol doesn't spell this case out. Reproducing the
current behavior for all combinations of shutdown/exit/EOF clutters interfaces.
We can iterate on this if desired.

Reviewers: jkorous, ioeric, hokein

Subscribers: mgorny, ilya-biryukov, MaskRay, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53286

llvm-svn: 344620
2018-10-16 16:48:06 +00:00
Eric Liu 0c29722c60 [clangd] Allow disble down traversals from root.
Summary:
This is useful for symbo scope proximity, where down traversals from
the global scope if not desired.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53317

llvm-svn: 344604
2018-10-16 10:41:17 +00:00
Sam McCall bca624ab03 [clangd] Fix threading bugs in (not-yet-used) BackgroundIndex, re-enable test.
Summary:
One relatively boring bug: forgot to notify the CV after enqueue.

One much more fun bug: the thread member could access instance variables before
they were initialized. Although the thread was last in the init list, QueueCV
etc were listed after Thread in the class, so their default constructors raced
with the thread itself.
We have to get very unlucky to lose this race, I saw it 0.02% of the time.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D53313

llvm-svn: 344595
2018-10-16 09:05:13 +00:00
Sam McCall 96f2489557 [clangd] Optionally use dex for the preamble parts of the dynamic index.
Summary:
Reuse the old -use-dex-index experiment flag for this.

To avoid breaking the tests, make Dex deduplicate symbols, addressing an old FIXME.

Reviewers: hokein

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53288

llvm-svn: 344594
2018-10-16 08:53:52 +00:00
Sam McCall f12a43d38a [clangd] Disable timeouting test while investigating
llvm-svn: 344586
2018-10-16 06:32:14 +00:00
Sam McCall 2b24ce61a0 [clangd] Use SyncAPI in more places in tests. NFC
llvm-svn: 344520
2018-10-15 15:04:03 +00:00
Sam McCall 8dc9dbb61a [clangd] Minimal implementation of automatic static index (not enabled).
Summary:
See tinyurl.com/clangd-automatic-index for design and goals.

Lots of limitations to keep this patch smallish, TODOs everywhere:
 - no serialization to disk
 - no changes to dynamic index, which now has a much simpler job
 - no partitioning of symbols by file to avoid duplication of header symbols
 - no reindexing of edited files
 - only a single worker thread
 - compilation database is slurped synchronously (doesn't scale)
 - uses memindex, rebuilds after every file (should be dex, periodically)

It's not hooked up to ClangdServer/ClangdLSPServer yet: the layering
isn't clear (it should really be in ClangdServer, but ClangdLSPServer
has all the CDB interactions).

Reviewers: ioeric

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D53032

llvm-svn: 344513
2018-10-15 13:34:10 +00:00
Haojian Wu e83caccb58 [clangd] Fix some references missing in dynamic index.
Summary:
Previously, SymbolCollector postfilters all references at the end to
find all references of interesting symbols.
It was incorrect when indxing main AST where we don't see locations
of symbol declarations and definitions in the main AST (as those are in
preamble AST).

The fix is to do earily check during collecting references.

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53273

llvm-svn: 344507
2018-10-15 11:46:26 +00:00
Haojian Wu 903f2a71a4 [clangd] Support hover on "aut^o *".
Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D53186

llvm-svn: 344330
2018-10-12 10:11:02 +00:00
Jonas Devlieghere fc51490baf Lift VFS from clang to llvm (NFC)
This patch moves the virtual file system form clang to llvm so it can be
used by more projects.

Concretely the patch:
 - Moves VirtualFileSystem.{h|cpp} from clang/Basic to llvm/Support.
 - Moves the corresponding unit test from clang to llvm.
 - Moves the vfs namespace from clang::vfs to llvm::vfs.
 - Formats the lines affected by this change, mostly this is the result of
   the added llvm namespace.

RFC on the mailing list:
http://lists.llvm.org/pipermail/llvm-dev/2018-October/126657.html

Differential revision: https://reviews.llvm.org/D52783

llvm-svn: 344140
2018-10-10 13:27:25 +00:00
Sam McCall 26fac469d2 [clangd] Make FSProvider const-correct. NFC
llvm-svn: 344118
2018-10-10 07:46:15 +00:00
Kadir Cetinkaya e324027b72 [clangd] Mark colon as a safe character when percent-encoding.
Summary: Also change output of percent-encoding to use upper-case letters.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D53016

llvm-svn: 344033
2018-10-09 10:29:54 +00:00
Sam McCall c92e4f35fd [clangd] Fix nondeterministic test
llvm-svn: 344030
2018-10-09 10:02:02 +00:00
Kadir Cetinkaya 84bf607eac [clangd] Revert back to previous heuristic for diagnostic range extraction.
Summary: Also add a few new test cases and a special case into handling of empty fixit ranges that  collides with location of a diag.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D52889

llvm-svn: 344025
2018-10-09 08:41:12 +00:00
Eric Liu 7734e63ad8 [clangd] Avoid cache main file status in preamble.
Summary: Main file can certainly change when reusing preamble.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52991

llvm-svn: 344024
2018-10-09 08:27:31 +00:00
Eric Liu bce181c64f [clang-move] Dump whether a declaration is templated.
llvm-svn: 343982
2018-10-08 17:22:50 +00:00
Sam McCall 75dd87bed1 [clangd] Remove debugging output in test
llvm-svn: 343845
2018-10-05 12:22:40 +00:00
Sam McCall b2d530e035 [clangd] Fix a subtle case for GetBeginningOfIdentifier.
Calling getMacroArgExpansionLocation too early was causing
Lexer::getRawToken to do the wrong thing - lexing the macro name instead
of the arg contents.

Differential Revision: https://reviews.llvm.org/D52928

llvm-svn: 343844
2018-10-05 12:08:06 +00:00
Sam McCall 50b89f0a9b [clangd] Simplify Dex query tree logic and fix missing-posting-list bug
Summary:
The bug being fixed: when a posting list doesn't exist in the index, it
was previously just dropped from the query rather than being treated as
empty. Now that we have the FALSE iterator, we can use it instead.

The query tree logic previously had a bunch of special cases to detect whether
subtrees are empty. Now we just naively build the whole tree, and rely
on the query optimizations to drop the trivial parts.

Finally, there was a bug in trigram generation: the empty query would
generate a single trigram "$$$" instead of no trigrams.
This had no effect (there was no posting list, so the other bug
cancelled it out). But we now have to fix this bug too.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52796

llvm-svn: 343802
2018-10-04 17:18:55 +00:00
Sam McCall bd7718ec26 [clangd] Add std::move for converting-return to satisfy older compilers
llvm-svn: 343800
2018-10-04 17:15:41 +00:00
Jonas Toth 0451440f89 [clangd] fix another ambigous constructor in DexTest
llvm-svn: 343796
2018-10-04 16:29:58 +00:00
Sam McCall a4c17dd7f3 [clangd] Fix ambiguous constructor in DexTest
llvm-svn: 343793
2018-10-04 16:05:22 +00:00
Sam McCall 422f724618 [clangd] expose MergedIndex class
Summary:
This allows inheriting from it, so index() can ga away and allowing
TestTU::index) to be fixed.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52250

llvm-svn: 343780
2018-10-04 14:20:22 +00:00
Sam McCall cc21779c3c [clangd] clangd-indexer gathers refs and stores them in index files.
Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52531

llvm-svn: 343778
2018-10-04 14:09:55 +00:00
Sam McCall 2ec5a10db3 [clangd] Remove one-segment-skipping from Dex trigrams.
Summary:
Currently queries like "ab" can match identifiers like a_yellow_bee.
The value of allowing this for exactly one segment but no more seems dubious.
It costs ~3% of overall ram (~9% of posting list ram) and some quality.

Reviewers: ilya-biryukov, ioeric

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52885

llvm-svn: 343777
2018-10-04 14:08:11 +00:00
Sam McCall b5bbfef6cd [cland] Dex: fix/simplify short-trigram generation
Summary:
1) Instead of x$$ for a short-query trigram, just use x
2) Make rules more coherent: prefixes of length 1-2, and first char + next head
3) Fix Dex::fuzzyFind to mark results as incomplete, because
   short-trigram rules only yield a subset of results.

Reviewers: ioeric

Subscribers: ilya-biryukov, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52808

llvm-svn: 343775
2018-10-04 14:01:55 +00:00
Sam McCall 87f69eaf4e [clangd] Dex: FALSE iterator, peephole optimizations, fix AND bug
Summary:
The FALSE iterator will be used in a followup patch to fix a logic bug in Dex
(currently, tokens that don't have posting lists in the index are simply dropped
from the query, changing semantics).

It can usually be optimized away, so added the following opmitizations:
 - simplify booleans inside AND/OR
 - replace effectively-empty AND/OR with booleans
 - flatten nested AND/ORs

While working on this, found a bug in the AND iterator: its constructor sync()
assumes that ReachedEnd is set if applicable, but the constructor never sets it.
This crashes if a non-first iterator is nonempty.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52789

llvm-svn: 343774
2018-10-04 13:12:23 +00:00
Haojian Wu d592498978 [clangd] Use canonical declarations in ReferenceFinder.
Summary:
handleDeclOccurrencce reports a canonical declartion, so stick to use
canonical declarations to determine whether a declaration is in the
target set.

Also fix a previous ref test which misses a matched label (it fails without this
patch).

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52871

llvm-svn: 343763
2018-10-04 09:56:08 +00:00
Sam McCall d9eae39800 [clangd] Support refs() in dex. Largely cloned from MemIndex.
Reviewers: hokein

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52726

llvm-svn: 343760
2018-10-04 09:16:12 +00:00
Stephen Kelly 2c4079c090 [clang-query] Add comment token handling
Summary:
It is possible to pass a file of commands to clang-query using the
command line option -f or --preload.  Make it possible to write comments
in such files.

Reviewers: aaron.ballman

Reviewed By: aaron.ballman

Subscribers: mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D52752

llvm-svn: 343666
2018-10-03 08:21:54 +00:00
Stephen Kelly 42668a4d03 [clang-query] Add single-letter 'q' alias for 'quit'
Reviewers: aaron.ballman, pcc

Reviewed By: aaron.ballman

Subscribers: Szelethus, cfe-commits

Differential Revision: https://reviews.llvm.org/D52746

llvm-svn: 343664
2018-10-03 07:52:44 +00:00
Eric Liu 16922d0718 [clangd] Temporarily disable VFS stats cache test for windows.
llvm-svn: 343637
2018-10-02 21:47:41 +00:00
Eric Liu 9a3af5a4f1 [clangd] Try to fix windows buildbot after r343576
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/20347/steps/test/logs/stdio

llvm-svn: 343623
2018-10-02 20:00:32 +00:00
Sam McCall a659d779f8 Reland r343589 "[clangd] Dex: add Corpus factory for iterators, rename, fold constant. NFC""
This reverts commit r343610.

llvm-svn: 343622
2018-10-02 19:59:23 +00:00
Reid Kleckner 2b5259afb3 Revert r343589 "[clangd] Dex: add Corpus factory for iterators, rename, fold constant. NFC"
Declaring a field with the same name as a type causes GCC to error out:

Dex.h:104:10: error: declaration of 'clang::clangd::dex::Corpus clang::clangd::dex::Dex::Corpus' [-fpermissive]
   Corpus Corpus;
          ^
Iterator.h:127:7: error: changes meaning of 'Corpus' from 'class clang::clangd::dex::Corpus' [-fpermissive]
 class Corpus {

llvm-svn: 343610
2018-10-02 17:31:43 +00:00
Haojian Wu 1c6b6deb33 [clangd] Add a #include completion test that triggers an assertion.
Summary: Test for https://reviews.llvm.org/D52774.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52775

llvm-svn: 343593
2018-10-02 14:46:08 +00:00
Sam McCall a1e7385d5c [clangd] Dex: add Corpus factory for iterators, rename, fold constant. NFC
Summary:
- Corpus avoids having to pass size to the true iterator, and (soon) any
  iterator that might optimize down to true.
- Shorten names of factory functions now they're scoped to the Corpus.
  intersect() and unionOf() rather than createAnd() or createOr() as this
  seems to read better to me, and fits with other short names. Opinion wanted!
- DEFAULT_BOOST_SCORE --> 1. This is a multiplier, don't obfuscate identity.
- Simplify variadic templates in Iterator.h

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52711

llvm-svn: 343589
2018-10-02 13:44:26 +00:00
Sam McCall 7402836042 [clangd] Dex iterator printer shows query structure, not iterator state.
Summary:
This makes it suitable for logging (which immediately found a bug, to
be fixed in the next patch...)

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52715

llvm-svn: 343580
2018-10-02 11:51:36 +00:00
Eric Liu b1d7542dde [clangd] Cache FS stat() calls when building preamble.
Summary:
The file stats can be reused when preamble is reused (e.g. code
completion). It's safe to assume that cached status is not outdated as we
assume preamble files to remain unchanged.

On real file system, this made code completion ~20% faster on a measured file
(with big preamble). The preamble build time doesn't change much.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: mgorny, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52419

llvm-svn: 343576
2018-10-02 10:43:55 +00:00
Stephen Kelly 12ae3e678d Sort expected test output after previous commit
llvm-svn: 343538
2018-10-01 21:10:30 +00:00
Stephen Kelly c87e99f4a4 [clang-query] Test non-code-completion on single letter shortcuts
llvm-svn: 343536
2018-10-01 20:45:44 +00:00
Stephen Kelly 98a38f53e7 [clang-query] Add missing quit test
llvm-svn: 343535
2018-10-01 20:45:39 +00:00
Stephen Kelly 9af02f2028 [clang-query] Add missing 'l' command handling
The `let` command was added in commit 045c15ba (Add new 'let' command to
bind arbitrary values into constants., 2014-04-23).

The `let` command and the non-existant `l` command were documented in
commit 233092a0 (Add 'let' to the help message., 2015-02-27).

Implement the `l` command now for completeness.

llvm-svn: 343533
2018-10-01 20:34:21 +00:00
Stephen Kelly a3c4206e41 [clang-tidy] Build it even without static analyzer
Conditionally compile the parts of clang-tidy which depend on the static
analyzer.

Funnily enough, I made the patch to exclude this from the build in 2013,
and it was committed with the comment that the tool should not be fully
excluded, but only the parts of it which depend on the analyzer should
be excluded.

 http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20130617/081797.html

This commit implements that idea.

Reviewed By: aaron.ballman

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D52334

llvm-svn: 343528
2018-10-01 20:24:22 +00:00
Aaron Ballman 873a50901b Reverting r343415 as it breaks at least one of the bots.
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/37336

llvm-svn: 343418
2018-09-30 17:39:39 +00:00
Aaron Ballman bc70c26ef4 Allow clang-tidy to be built without a dependency on the clang static analyzer.
Patch by Stephen Kelly.

llvm-svn: 343415
2018-09-30 17:22:58 +00:00
Eric Liu 670c147d83 [clangd] Initial supoprt for cross-namespace global code completion.
Summary:
When no scope qualifier is specified, allow completing index symbols
from any scope and insert proper automatically. This is still experimental and
hidden behind a flag.

Things missing:
- Scope proximity based scoring.
- FuzzyFind supports weighted scopes.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: kbobyrev, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52364

llvm-svn: 343248
2018-09-27 18:46:00 +00:00
Kadir Cetinkaya 0ed5d29b44 Tell whether file/folder for include completions.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, ioeric, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D52547

llvm-svn: 343221
2018-09-27 14:21:07 +00:00
Kadir Cetinkaya 2e3f1f13b5 Improve diagnostics range reporting.
Summary:
If we have some range information coming from clang diagnostic, promote
that one even if it doesn't contain diagnostic location inside.

Reviewers: sammccall, ioeric

Reviewed By: ioeric

Subscribers: ilya-biryukov, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D52544

llvm-svn: 343197
2018-09-27 12:12:42 +00:00
Ilya Biryukov 4f98470dea [clangd] Handle template args for disabled function arg snippets
Reviewers: kadircet, ioeric, sammccall

Reviewed By: kadircet

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D52422

llvm-svn: 343066
2018-09-26 05:45:31 +00:00
Fangrui Song cfac25c3bd [clangd] Remove unused using-declaration testing::AllOf
llvm-svn: 343039
2018-09-25 22:32:11 +00:00
Sam McCall 02d600d267 [clangd] Merge binary + YAML serialization behind a (mostly) common interface.
Summary:
Interface is in one file, implementation in two as they have little in common.
A couple of ad-hoc YAML functions left exposed:
 - symbol -> YAML I expect to keep for tools like dexp
 - YAML -> symbol is used for the MR-style indexer, I think we can eliminate
   this (merge-on-the-fly, else use a different serialization)

Reviewers: kbobyrev

Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52453

llvm-svn: 342999
2018-09-25 18:06:43 +00:00
Kirill Bobyrev 6c2f5bd0f1 [clangd] Implement VByte PostingList compression
This patch implements Variable-length Byte compression of `PostingList`s
to sacrifice some performance for lower memory consumption.

`PostingList` compression and decompression was extensively tested using
fuzzer for multiple hours and runnning significant number of realistic
`FuzzyFindRequests`. AddressSanitizer and UndefinedBehaviorSanitizer
were used to ensure the correct behaviour.

Performance evaluation was conducted with recent LLVM symbol index (292k
symbols) and the collection of user-recorded queries (7751
`FuzzyFindRequest` JSON dumps):

| Metrics | Before| After | Change (%)
| -----  | -----  | -----   | -----
| Memory consumption (posting lists only), MB  |  54.4 | 23.5 | -60%
| Time to process queries, sec | 7.70 | 9.4 | +25%

Reviewers: sammccall, ioeric

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D52300

llvm-svn: 342965
2018-09-25 11:54:51 +00:00
Eric Liu e631d09fc5 [clangd] Check that scheme is valid when parsing URI.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52455

llvm-svn: 342961
2018-09-25 10:47:46 +00:00
Kirill Bobyrev 029aa8ec7f [clangd] NFC: Remove test duplicate
`FuzzyMatchQ` test was a duplicate of `FuzzyMatch` pulled from MemIndex
tests.

llvm-svn: 342957
2018-09-25 09:47:01 +00:00
Kirill Bobyrev 94af0612e0 [clangd] Force Dex to respect symbol collector flags
`Dex` should utilize `FuzzyFindRequest.RestrictForCodeCompletion` flags
and omit symbols not meant for code completion when asked for it.

The measurements below were conducted with setting
`FuzzyFindRequest.RestrictForCodeCompletion` to `true` (so that it's
more realistic). Sadly, the average latency goes down, I suspect that is
mostly because of the empty queries where the number of posting lists is
critical.

| Metrics  | Before | After | Relative difference
| -----  | -----  | -----   | -----
| Cumulative query latency (7000 `FuzzyFindRequest`s over LLVM static index)  | 6182735043 ns    | 7202442053 ns | +16%
| Whole Index size | 81.24 MB    | 81.79 MB | +0.6%

Out of 292252 symbols collected from LLVM codebase 136926 appear to be
restricted for code completion.

Reviewers: ioeric

Differential Revision: https://reviews.llvm.org/D52357

llvm-svn: 342866
2018-09-24 08:45:18 +00:00
Eric Liu c275fb2a5d [clangd] Remember to serialize symbol origin in YAML.
llvm-svn: 342730
2018-09-21 13:04:57 +00:00
Eric Liu 467c5f9ce0 [clangd] Store preamble macros in dynamic index.
Summary:
Pros:
o Loading macros from preamble for every completion is slow (see profile).
o Calculating macro USR is also slow (see profile).
o Sema can provide a lot of macro completion results (e.g. when filter is empty,
60k for some large TUs!).

Cons:
o Slight memory increase in dynamic index (~1%).
o Some extra work during preamble build (should be fine as preamble build and
indexAST is way slower).

Before:
{F7195645}

After:
{F7195646}

Reviewers: ilya-biryukov, sammccall

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52078

llvm-svn: 342529
2018-09-19 09:35:04 +00:00
Eric Liu 764f461f9c [clangd] Get rid of Decls parameter in indexMainDecls. NFC
It's already available in ParsedAST.

llvm-svn: 342473
2018-09-18 13:35:16 +00:00
Eric Liu 821a116818 [clangd] Merge ClangdServer::DynamicIndex into FileIndex. NFC.
Summary:
FileIndex now provides explicit interfaces for preamble and main file updates.
This avoids growing parameter list when preamble and main symbols diverge
further (e.g. D52078). This also gets rid of the hack in `indexAST` that
inferred main file index based on `TopLevelDecls`.

Also separate `indexMainDecls` from `indexAST`.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52222

llvm-svn: 342460
2018-09-18 10:30:44 +00:00
Shuai Wang 02bfd89c93 Fix build failure caused by D52157
llvm-svn: 342408
2018-09-17 20:10:33 +00:00
Jonas Toth ad51519668 [NFC][clangd] silence pedantic extra ';' warning
llvm-svn: 342267
2018-09-14 19:42:37 +00:00
Sam McCall 0be9c4e8d6 [clangd] Work around compiler macro expansion bugs(?) in completion tests
llvm-svn: 342261
2018-09-14 18:49:16 +00:00
Sam McCall ebef81227b [clangd] Don't override the preamble while completing inside it, it doesn't work.
Summary:
To stay fast, enable single-file-mode instead. This is fine since completions
in the preamble are simple.

The net effect for now is to suppress the spurious TopLevel completions when
completing inside the preamble.
Once Sema has include directive completion, this will be more important.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52071

llvm-svn: 342230
2018-09-14 12:36:06 +00:00
Kirill Bobyrev 249c5864cf [clangd] Introduce PostingList interface
This patch abstracts `PostingList` interface and reuses existing
implementation. It will be used later to test different `PostingList`
representations.

No functionality change is introduced, this patch is mostly refactoring
so that the following patches could focus on functionality while not
being too hard to review.

Reviewed By: sammccall, ioeric

Differential Revision: https://reviews.llvm.org/D51982

llvm-svn: 342155
2018-09-13 17:11:03 +00:00
Kirill Bobyrev e6dd0806c7 [clangd] Cleanup FuzzyFindRequest filtering limit semantics
As discussed during D51860 review, it is better to use `llvm::Optional`
here as it has clear semantics which reflect intended behavior.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D52028

llvm-svn: 342138
2018-09-13 14:27:03 +00:00
Sam McCall c8758406b5 [clangd] Simplify cancellation public API
Summary:
Task is no longer exposed:
 - task cancellation is hidden as a std::function
 - task creation returns the new context directly
 - checking is via free function only, with no way to avoid the context lookup
The implementation is essentially the same, but a bit terser as it's hidden.

isCancelled() is now safe to use outside any task (it returns false).
This will leave us free to sprinkle cancellation in e.g. TUScheduler without
needing elaborate test setup, and lets callers that don't cancel "just work".

Updated the docs to describe the new expected use pattern.
One thing I noticed: there's nothing async-specific about the cancellation.
Async tasks can be cancelled from any thread (typically the one that created
them), sync tasks can be cancelled from any *other* thread in the same way.
So the docs now refer to "long-running" tasks instead of async ones.

Updated usage in code complete, without any structural changes.
I didn't update all the names of the helpers in ClangdLSPServer (these will
likely be moved to JSONRPCDispatcher anyway).

Reviewers: ilya-biryukov, kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D51996

llvm-svn: 342130
2018-09-13 11:47:48 +00:00
Shuai Wang f21e8ebb91 [clangtidy] Remove old copy of ExprMutationAnalyzer
Summary:
This is 2/2 of moving ExprMutationAnalyzer from clangtidy to clang/Analysis.
ExprMutationAnalyzer is moved to clang/Analysis in D51948.
This diff migrates existing usages within clangtidy to point to the new
location and remove the old copy of ExprMutationAnalyzer.

Reviewers: george.karpenkov, JonasToth

Reviewed By: george.karpenkov

Subscribers: mgorny, a.sidorin, Szelethus, cfe-commits

Differential Revision: https://reviews.llvm.org/D51950

llvm-svn: 342006
2018-09-11 22:59:46 +00:00
Shuai Wang 277b808ad3 [clang-tidy] Handle sugared reference types in ExprMutationAnalyzer
Summary:
This handles cases like this:
```
typedef int& IntRef;
void mutate(IntRef);
void f() {
  int x;
  mutate(x);
}
```
where the param type is a sugared type (`TypedefType`) instead of a
reference type directly.

Note that another category of similar but different cases are already
handled properly before:
```
typedef int Int;
void mutate(Int&);
void f() {
  int x;
  mutate(x);
}
```

Reviewers: aaron.ballman, alexfh, george.karpenkov

Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits

Differential Revision: https://reviews.llvm.org/D50953

llvm-svn: 341986
2018-09-11 20:05:37 +00:00
Shuai Wang ea85b52732 [clang-tidy] Handle unique owning smart pointers in ExprMutationAnalyzer
Summary:
For smart pointers like std::unique_ptr which uniquely owns the
underlying object, treat the mutation of the pointee as mutation of the
smart pointer itself.

This gives better behavior for cases like this:
```
void f(std::vector<std::unique_ptr<Foo>> v) { // undesirable analyze result of `v` as not mutated.
  for (auto& p : v) {
      p->mutate(); // only const member function `operator->` is invoked on `p`
  }
}
```

Reviewers: hokein, george.karpenkov

Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits

Differential Revision: https://reviews.llvm.org/D50883

llvm-svn: 341967
2018-09-11 17:33:12 +00:00
Kadir Cetinkaya 873cae5ae7 [clangd] Add unittests for D51917
Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D51924

llvm-svn: 341950
2018-09-11 15:12:10 +00:00
Jonas Toth b88eebebbe [NFC][clangd] fix warning for extra semicolon
llvm-svn: 341933
2018-09-11 13:01:49 +00:00
Shuai Wang 5066ab369d Revert "Revert "[clang-tidy] Handle unresolved expressions in ExprMutationAnalyzer""
This is the same as D50619 plus fixes for buildbot failures on windows.
The test failures on windows are caused by -fdelayed-template-parsing
and is fixed by forcing -fno-delayed-template-parsing on test cases that
requires AST for uninstantiated templates.

llvm-svn: 341891
2018-09-11 02:23:35 +00:00
Shuai Wang cec7d3a055 Revert "[clang-tidy] Handle unresolved expressions in ExprMutationAnalyzer"
Summary:
Tests somehow break on windows (and only on windows)
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/13003
http://lab.llvm.org:8011/builders/clang-x86-windows-msvc2015/builds/13747

I have yet figure out why so reverting to unbreak first.

Reviewers: george.karpenkov

Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits

Differential Revision: https://reviews.llvm.org/D51898

llvm-svn: 341886
2018-09-10 23:58:04 +00:00
Roman Lebedev a3dc9484e3 [clang-tidy] ExprMutationAnalyzer: construct from references. Fixes PR38888
Summary:
I have hit this the rough way, while trying to use this in D51870.

There is no particular point in storing the pointers, and moreover
the pointers are assumed to be non-null, and that assumption is not
enforced. If they are null, it won't be able to do anything good
with them anyway.

Initially i thought about simply adding asserts() that they are
not null, but taking/storing references looks like even cleaner solution?

Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=38888 | PR38888 ]]

Reviewers: JonasToth, shuaiwang, alexfh, george.karpenkov

Reviewed By: shuaiwang

Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits

Tags: #clang-tools-extra

Differential Revision: https://reviews.llvm.org/D51884

llvm-svn: 341854
2018-09-10 19:59:18 +00:00
Shuai Wang bef0941b6b [clang-tidy] Handle unresolved expressions in ExprMutationAnalyzer
Summary:
- If a function is unresolved, assume it mutates its arguments
- Follow unresolved member expressions for nested mutations

Reviewers: aaron.ballman, JonasToth, george.karpenkov

Subscribers: xazax.hun, a.sidorin, cfe-commits

Differential Revision: https://reviews.llvm.org/D50619

llvm-svn: 341848
2018-09-10 18:05:13 +00:00
Kadir Cetinkaya 0b77d03d80 [clangd] Add unittests for D51038
Reviewers: ilya-biryukov, ioeric, hokein

Reviewed By: ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D51039

llvm-svn: 341830
2018-09-10 14:22:42 +00:00
Kirill Bobyrev 38a889c185 [clangd] Add symbol slab size to index memory consumption estimates
Currently, `SymbolIndex::estimateMemoryUsage()` returns the "overhead"
estimate, i.e. the estimate of the Index data structure excluding
backing data (such as Symbol Slab and Reference Slab). This patch
propagates information about paired data size where necessary.

Reviewed By: ioeric, sammccall

Differential Revision: https://reviews.llvm.org/D51539

llvm-svn: 341800
2018-09-10 11:46:07 +00:00
Kirill Bobyrev 5abe478a3d [clangd] NFC: Rename DexIndex to Dex
Also, cleanup some redundant includes.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D51774

llvm-svn: 341784
2018-09-10 08:23:53 +00:00
Eric Liu 6df66001ee [clangd] Add "Deprecated" field to Symbol and CodeCompletion.
Summary: Also set "deprecated" field in LSP CompletionItem.

Reviewers: sammccall, kadircet

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D51724

llvm-svn: 341576
2018-09-06 18:52:26 +00:00
Kirill Bobyrev 19a9461e5f [clangd] Implement proximity path boosting for Dex
This patch introduces `PathURI` Search Token kind and utilizes it to
uprank symbols which are defined in files with small distance to the
directory where the fuzzy find request is coming from (e.g. files user
is editing).

Reviewed By: ioeric

Reviewers: ioeric, sammccall

Differential Revision: https://reviews.llvm.org/D51481

llvm-svn: 341542
2018-09-06 12:54:43 +00:00
Ilya Biryukov 5c4d6e66f0 [clangd] Fix data race in async fuzzyFind tests.
llvm-svn: 341538
2018-09-06 11:04:56 +00:00
Eric Liu d25f1214a8 [clangd] Set SymbolID for sema macros so that they can be merged with index macros.
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51688

llvm-svn: 341534
2018-09-06 09:59:37 +00:00
Haojian Wu 53e91c1b08 [clangd] Sort GoToDefinition results.
Summary:
GoToDefinition returns all declaration results (implicit/explicit) that are
in the same location, and the results are returned in arbitrary order.

Some LSP clients defaultly take the first result as the final result, which
might present a bad result (implicit decl) to users.

This patch ranks the result based on whether the declarations are
referenced explicitly/implicitly. We put explicit declarations first.

This also improves the "hover" (which just take the first result) feature
in some cases.

Reviewers: ilya-biryukov

Subscribers: kadircet, ioeric, MaskRay, jkorous, mgrang, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D50438

llvm-svn: 341463
2018-09-05 12:00:15 +00:00
Sam McCall d445f17614 [clangd] Implement findReferences function
clangd will use findReferences to provide LSP's reference feature.

llvm-svn: 341458
2018-09-05 10:33:36 +00:00
Sam McCall 50f3631057 [clangd] Define a compact binary serialization fomat for symbol slab/index.
Summary:
This is intended to replace the current YAML format for general use.
It's ~10x more compact than YAML, and ~40% more compact than gzipped YAML:
  llvmidx.riff = 20M, llvmidx.yaml = 272M, llvmidx.yaml.gz = 32M
It's also simpler/faster to read and write.

The format is a RIFF container (chunks of (type, size, data)) with:
 - a compressed string table
 - simple binary encoding of symbols (with varints for compactness)
It can be extended to include occurrences, Dex posting lists, etc.

There's no rich backwards-compatibility scheme, but a version number is included
so we can detect incompatible files and do ad-hoc back-compat.

Alternatives considered:
 - compressed YAML or JSON: bulky and slow to load
 - llvm bitstream: confusing model and libraries are hard to use. My attempt
   produced slightly larger files, and the code was longer and slower.
 - protobuf or similar: would be really nice (esp for back-compat) but the
   dependency is a big hassle
 - ad-hoc binary format without a container: it seems clear we're going
   to add posting lists and occurrences here, and that they will benefit
   from sharing a string table. The container makes it easy to debug
   these pieces in isolation, and make them optional.

Reviewers: ioeric

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51585

llvm-svn: 341375
2018-09-04 16:16:50 +00:00
Sam McCall b0138317d6 [clangd] SymbolOccurrences -> Refs and cleanup
Summary:
A few things that I noticed while merging the SwapIndex patch:
 - SymbolOccurrences and particularly SymbolOccurrenceSlab are unwieldy names,
   and these names appear *a lot*. Ref, RefSlab, etc seem clear enough
   and read/format much better.
 - The asymmetry between SymbolSlab and RefSlab (build() vs freeze()) is
   confusing and irritating, and doesn't even save much code.
   Avoiding RefSlab::Builder was my idea, but it was a bad one; add it.
 - DenseMap<SymbolID, ArrayRef<Ref>> seems like a reasonable compromise for
   constructing MemIndex - and means many less wasted allocations than the
   current DenseMap<SymbolID, vector<Ref*>> for FileIndex, and none for
   slabs.
 - RefSlab::find() is not actually used for anything, so we can throw
   away the DenseMap and keep the representation much more compact.
 - A few naming/consistency fixes: e.g. Slabs,Refs -> Symbols,Refs.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51605

llvm-svn: 341368
2018-09-04 14:39:56 +00:00
Sam McCall 046557bc03 [clangd] Some nitpicking around the new split (preamble/main) dynamic index
Summary:
- DynamicIndex doesn't implement ParsingCallbacks, to make its role clearer.
  ParsingCallbacks is a separate object owned by the receiving TUScheduler.
  (I tried to get rid of the "index-like-object that doesn't implement index"
  but it was too messy).
- Clarified(?) docs around DynamicIndex - fewer details up front, more details
  inside.
- Exposed dynamic index from ClangdServer for memory monitoring and more
  direct testing of its contents (actual tests not added here, wanted to get
  this out for review)
- Removed a redundant and sligthly confusing filename param in a callback

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51221

llvm-svn: 341325
2018-09-03 16:37:59 +00:00
Ilya Biryukov 5a79d1e377 [clangd] Avoid crashes in override completions
Summary: NamedDecl::getName cannot be called on non-identifier names.

Reviewers: kadircet, ioeric, hokein, sammccall

Reviewed By: ioeric

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D51598

llvm-svn: 341322
2018-09-03 15:25:27 +00:00
Sam McCall b466e11c39 [clangd] Fix ambiguous make_unique with c++17. NFC
llvm-svn: 341321
2018-09-03 15:23:01 +00:00
Sam McCall 9c7624e14b [clangd] Factor out the data-swapping functionality from MemIndex/DexIndex.
Summary:
This is now handled by a wrapper class SwapIndex, so MemIndex/DexIndex can be
immutable and focus on their job.

Old and busted:
 I have a MemIndex, which holds a shared_ptr<vector<Symbol*>>, which keeps the
 symbol slab alive. I update by calling build(shared_ptr<vector<Symbol*>>).

New hotness: I have a SwapIndex, which holds a unique_ptr<SymbolIndex>, which
 holds a MemIndex, which holds a shared_ptr<void>, which keeps backing
 data alive.
 I update by building a new MemIndex and calling SwapIndex::reset().

Reviewers: kbobyrev, ioeric

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51422

llvm-svn: 341318
2018-09-03 14:37:43 +00:00
Eric Liu 83f63e42b2 [clangd] Support multiple #include headers in one symbol.
Summary:
Currently, a symbol can have only one #include header attached, which
might not work well if the symbol can be imported via different #includes depending
on where it's used. This patch stores multiple #include headers (with # references)
for each symbol, so that CodeCompletion can decide which include to insert.

In this patch, code completion simply picks the most popular include as the default inserted header. We also return all possible includes and their edits in the `CodeCompletion` results.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: mgrang, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51291

llvm-svn: 341304
2018-09-03 10:18:21 +00:00
Haojian Wu e8064b6f6d [clangd] Implement findOccurrences interface in dynamic index.
Summary:
Implement the interface in
  - FileIndex
  - MemIndex
  - MergeIndex

Depends on https://reviews.llvm.org/D50385.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: mgrang, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51279

llvm-svn: 341242
2018-08-31 19:53:37 +00:00
Sam McCall 2e5700f038 [clangd] Flatten out Symbol::Details. It was ill-conceived, sorry.
Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51504

llvm-svn: 341211
2018-08-31 13:55:01 +00:00
Haojian Wu d81e3146e3 [clangd] Collect symbol occurrences in SymbolCollector.
SymbolCollector will be used for two cases:
 - collect Symbol type only, used for indexing preamble AST.
 - collect Symbol and SymbolOccurrences, used for indexing main AST.

For finding local references from the AST, we will implement it in other ways.

llvm-svn: 341208
2018-08-31 12:54:13 +00:00