Commit Graph

762 Commits

Author SHA1 Message Date
Kadir Cetinkaya fc2fb60bab
[clangd] clang-format TweakTests, NFC 2020-10-14 18:14:27 +02:00
Kadir Cetinkaya 82a71822a5
[clangd] Disable extract variable for RHS of assignments
Differential Revision: https://reviews.llvm.org/D89307
2020-10-14 14:22:47 +02:00
Haojian Wu 3fcca804b2 [clangd] Refine recoveryAST flags in clangd
so that we could start experiment for C.

Previously, these flags in clangd were only meaningful for C++. We need
to flip them for C, this patch repurpose these flags.

- if true, just set it.
- if false, just respect the value in clang.

this would allow us to keep flags on for C++, and optionally flip them on for C.

Differential Revision: https://reviews.llvm.org/D89233
2020-10-14 13:42:11 +02:00
Nathan Ridge b764edc59f [clangd] Try harder to get accurate ranges for documentSymbols in macros
Fixes https://github.com/clangd/clangd/issues/500

Differential Revision: https://reviews.llvm.org/D88463
2020-10-12 19:26:36 -04:00
Nathan Ridge 1b962fdd5f [clangd] Heuristic resolution for dependent type and template names
Fixes https://github.com/clangd/clangd/issues/543

Differential Revision: https://reviews.llvm.org/D88469
2020-10-12 13:37:22 -04:00
Haojian Wu 16a4b0f0e3 [clangd] Disable a failure TopLevelDecls test.
The test fails on clang-ppc64le-rhel buildbot, needs further
investigation.
2020-10-12 16:03:52 +02:00
Haojian Wu b144cd867b Dump decl when the test matcher fails. 2020-10-12 15:42:18 +02:00
Kadir Cetinkaya 35871fde55
[clangd] Record memory usages after each notification
Depends on D88415

Differential Revision: https://reviews.llvm.org/D88417
2020-10-12 15:25:29 +02:00
Kadir Cetinkaya 23a53301c5
[clangd] Introduce memory usage dumping to TUScheduler, for Preambles and ASTCache
File-granular information is considered details.

Depends on D88411

Differential Revision: https://reviews.llvm.org/D88415
2020-10-12 15:25:29 +02:00
Kadir Cetinkaya a74d594948
[clangd] Introduce memory dumping to FileIndex, FileSymbols and BackgroundIndex
File-granular information is considered details.

Depends on D88411

Differential Revision: https://reviews.llvm.org/D88414
2020-10-12 15:25:29 +02:00
Kadir Cetinkaya c9d2876da9
[clangd] Add a metric for tracking memory usage
Differential Revision: https://reviews.llvm.org/D88413
2020-10-12 15:25:29 +02:00
Kadir Cetinkaya f9317f7bf6
[clangd] Introduce MemoryTrees
A structure that can be used to represent memory usage of a nested
set of systems.

Differential Revision: https://reviews.llvm.org/D88411
2020-10-12 15:25:29 +02:00
Sam McCall 8f1de22c76 [clangd] Stop capturing trace args if the tracer doesn't need them.
The tracer is now expected to allocate+free the args itself.

Differential Revision: https://reviews.llvm.org/D89135
2020-10-12 13:43:05 +02:00
Haojian Wu f1bf41e433 Fix buildbot failure for 702529d899. 2020-10-12 12:04:44 +02:00
Haojian Wu 702529d899 [clang] Fix returning the underlying VarDecl as top-level decl for VarTemplateDecl.
Given the following VarTemplateDecl AST,

```
VarTemplateDecl col:26 X
|-TemplateTypeParmDecl typename depth 0 index 0
`-VarDecl X 'bool' cinit
  `-CXXBoolLiteralExpr 'bool' true
```

previously, we returned the VarDecl as the top-level decl, which was not
correct, the top-level decl should be VarTemplateDecl.

Differential Revision: https://reviews.llvm.org/D89098
2020-10-12 10:46:18 +02:00
Nathan Ridge f82346fd73 [clangd] Avoid relations being overwritten in a header shard
Fixes https://github.com/clangd/clangd/issues/510

Differential Revision: https://reviews.llvm.org/D87256
2020-10-11 15:32:54 -04:00
Kadir Cetinkaya 6f1a56d37a
[clangd] Enable partial namespace matches for workspace symbols
This will enable queries like "clangd::" to find symbols under clangd
namespace, without requiring full "clang::clangd::" qualification.

Since Fuzzyfind performs the search under all scopes and only boosts the symbols
from relevant namespaces, we might get symbols from non-matching namespaces.
This patch chooses to drop those as they clearly do not match the query.

Fixes https://github.com/clangd/clangd/issues/550.

Differential Revision: https://reviews.llvm.org/D88814
2020-10-09 10:20:41 +02:00
Kadir Cetinkaya 2ff44935a5
[clangd] Reduce availability of extract function
This patch introduces hoisting detection logic into prepare state with
a partial AST traversal of the enclosing function.

We had some complaints from the users about this code action being almost always
available but failing most of the time. Hopefully this should reduce that noise.

The latency/correctness tradeoff is a bunch of hand-waving, but at least today
we don't have any other actions that are available on selection of statements,
so when we get to do the traversal it is quite likely that all the other checks
will bail out early. But this is still up for discussion, I am happy to abandon
the patch if you believe this is not practical.

Differential Revision: https://reviews.llvm.org/D85354
2020-10-09 10:12:48 +02:00
Haojian Wu efd8c9ed72 [clangd] Add more incomplete_type diagnostics that could be fixed by include-fixer.
Differential Revision: https://reviews.llvm.org/D89036
2020-10-09 08:51:18 +02:00
Haojian Wu a4e27220c2 [clangd] Add a missing include-fixer test for incomplete_type, NFC.
Also sort the list to make it easier to verify with the implementation
code.

Differential Revision: https://reviews.llvm.org/D88964
2020-10-08 13:33:38 +02:00
Haojian Wu 9c09e2055e [clangd] Add a NewName optional parameter to clangdServer::prepareRename.
If the NewName is provided, prepareRename would perform a name
validation.

The motivation is to allow our internal embeder implement the customized
"canRenameInto" functionality on top of prepareRename.

Differential Revision: https://reviews.llvm.org/D88881
2020-10-07 21:18:51 +02:00
Sam McCall 69daa368ca [clangd] Disambiguate overloads of std::move for header insertion.
Up until now, we relied on matching the filename.
This depends on unstable details of libstdc++ and doesn't work well on other
stdlibs. Also we'd like to remove it (see D88204).

Differential Revision: https://reviews.llvm.org/D88885
2020-10-07 19:42:41 +02:00
Haojian Wu f24649b77d [clangd] Don't set the Underlying bit on targets of UsingDecls.
With this patch, we don't treat `using ns::X` as a first-class declaration like `using Z = ns::Y`, reference to X that goes through this using-decl is considered a direct reference (without the Underlying bit).

Fix the workaround in https://reviews.llvm.org/D87225 and https://reviews.llvm.org/D74054.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D88472
2020-10-07 10:01:04 +02:00
Haojian Wu 8a3cbb1535 [clangd] Add basic keyword-name-validation in rename.
Differential Revision: https://reviews.llvm.org/D88875
2020-10-06 15:47:57 +02:00
Haojian Wu 48a82c4245 [clangd] Verify the diagnostic code in include-fixer diagnostic tests, NFC.
Make it easier to spot which diagnostics in the include-fixer list are tested.

Differential Revision: https://reviews.llvm.org/D88828
2020-10-06 11:02:22 +02:00
Haojian Wu 1425c72236 [clangd] Add isKeyword function.
This will be used in rename for doing basic name validation.

Differential Revision: https://reviews.llvm.org/D88810
2020-10-05 15:11:24 +02:00
Haojian Wu 96c8a17c80 [clangd] Remove unused using-decls in TypeHierarchyTests, NFC. 2020-10-05 13:14:53 +02:00
Haojian Wu 0f0cbcc4b1 [clangd] Extend the rename API.
several changes:
- return a structure result in rename API;
- prepareRename now returns more information (main-file occurrences);
- remove the duplicated detecting-touch-identifier code in prepareRename (which is implemented in rename API);

Differential Revision: https://reviews.llvm.org/D88634
2020-10-02 16:03:44 +02:00
Sam McCall 216af81c39 [clangd] Fix invalid UTF8 when extracting doc comments.
Differential Revision: https://reviews.llvm.org/D88567
2020-09-30 16:05:12 +02:00
Sam McCall 7ba0779fbb [clangd] Extract options struct for ClangdLSPServer. NFC
In preparation for making moving TweakFilter from ClangdServer::Options to
a ClangdLSPServer option, and letting it vary per-request.
(In order to implement CodeActionParams.only)

Also a general overdue cleanup.

Differential Revision: https://reviews.llvm.org/D88470
2020-09-30 10:09:52 +02:00
Nathan James 01a30fa678
[clangd] Trivial setter support when moving items to fields
Extend the Trivial setter documentation to support cases where the value is moved into a field using `std::move`.

Reviewed By: sammccall, kadircet

Differential Revision: https://reviews.llvm.org/D88297
2020-09-29 21:51:15 +01:00
Aleksandr Platonov d8ba6b4ab3 [clangd] findNearbyIdentifier(): guaranteed to give up after 2^N lines
As @kadircet mentions in D84912#2184144, `findNearbyIdentifier()` traverses the whole file if there is no identifier for the word.
This patch ensures give up after 2^N lines in any case.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D87891
2020-09-29 19:54:55 +03:00
Tadeo Kondrak 4fb303f340 [clangd] Improve PopulateSwitch tweak to work on non-empty switches
Improve the recently-added PopulateSwitch tweak to work on non-empty switches.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D88434
2020-09-29 16:37:51 +02:00
Nathan Ridge cc6d1f8029 [clangd] When finding refs for a renaming alias, do not return refs to underlying decls
Fixes https://github.com/clangd/clangd/issues/515

Differential Revision: https://reviews.llvm.org/D87225
2020-09-28 21:18:31 -04:00
Utkarsh Saxena 9b1666f3ce [clangd] Rename evaluate() to evaluateHeuristics()
Since we have 2 scoring functions (heuristics and decision forest),
renaming the existing evaluate() function to be more descriptive of the
Heuristics being evaluated in it.

Differential Revision: https://reviews.llvm.org/D88431
2020-09-28 20:05:01 +02:00
Utkarsh Saxena a8b55b6939 [clangd] Use Decision Forest to score code completions.
By default clangd will score a code completion item using heuristics model.

Scoring can be done by Decision Forest model by passing `--ranking_model=decision_forest` to
clangd.

Features omitted from the model:
- `NameMatch` is excluded because the final score must be multiplicative in `NameMatch` to allow rescoring by the editor.
- `NeedsFixIts` is excluded because the generating dataset that needs 'fixits' is non-trivial.

There are multiple ways (heuristics) to combine the above two features with the prediction of the DF:
- `NeedsFixIts` is used as is with a penalty of `0.5`.

Various alternatives of combining NameMatch `N` and Decision forest Prediction `P`
- N * scale(P, 0, 1): Linearly scale the output of model to range [0, 1]
- N * a^P:
  - More natural: Prediction of each Decision Tree can be considered as a multiplicative boost (like NameMatch)
  - Ordering is independent of the absolute value of P. Order of two items is proportional to `a^{difference in model prediction score}`. Higher `a` gives higher weightage to model output as compared to NameMatch score.

Baseline MRR = 0.619
MRR for various combinations:
N * P = 0.6346, advantage%=2.5768
N * 1.1^P = 0.6600, advantage%=6.6853
N * **1.2**^P = 0.6669, advantage%=**7.8005**
N * **1.3**^P = 0.6668, advantage%=**7.7795**
N * **1.4**^P = 0.6659, advantage%=**7.6270**
N * 1.5^P = 0.6646, advantage%=7.4200
N * 1.6^P = 0.6636, advantage%=7.2671
N * 1.7^P = 0.6629, advantage%=7.1450
N * 2^P = 0.6612, advantage%=6.8673
N * 2.5^P = 0.6598, advantage%=6.6491
N * 3^P = 0.6590, advantage%=6.5242
N * scaled[0, 1] = 0.6465, advantage%=4.5054

Differential Revision: https://reviews.llvm.org/D88281
2020-09-28 18:59:29 +02:00
Tadeo Kondrak 018066d947 [clangd] Add a tweak for filling in enumerators of a switch statement.
Add a tweak that populates an empty switch statement of an enumeration type with all of the enumerators of that type.

Before:
```
enum Color { RED, GREEN, BLUE };
void f(Color color) {
  switch (color) {}
}
```

After:
```
enum Color { RED, GREEN, BLUE };
void f(Color color) {
  switch (color) {
  case RED:
  case GREEN:
  case BLUE:
    break;
  }
}
```

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D88383
2020-09-28 13:37:18 +02:00
Kadir Cetinkaya 64168c6d99
[clangd] Disable suffix matching fallback for C during include insertion
Clangd currently doesn't respect language and breaks the builds with
include insertion for C. This patch aims to stop the bleeding by not mapping
back to CPP standard library headers.

Improves https://github.com/clangd/clangd/issues/376.

Differential Revision: https://reviews.llvm.org/D88144
2020-09-24 10:46:10 +02:00
Nathan Ridge aa3c7638ba [clang] Traverse init-captures while indexing
Fixes https://github.com/clangd/clangd/issues/496

Differential Revision: https://reviews.llvm.org/D87257
2020-09-22 02:32:09 -04:00
Utkarsh Saxena 985deba931 Revert "Temporarily Revert "[clangd] Add Random Forest runtime for code completion.""
We intend to replace heuristics based code completion ranking with a Decision Forest Model.

This patch introduces a format for representing the model and an inference runtime that is code-generated at build time.
- Forest.json contains all the trees as an array of trees.
- Features.json describes the features to be used.
- Codegen file takes the above two files and generates CompletionModel containing Feature struct and corresponding Evaluate function.
   The Evaluate function maps a feature to a real number describing the relevance of this candidate.
- The codegen is part of build system and these files are generated at build time.
- Proposes a way to test the generated runtime using a test model.
  - Replicates the model structure in unittests.
  - unittest tests both the test model (for correct tree traversal) and the real model (for sanity).

This reverts commit 549e55b3d5.
2020-09-19 10:54:04 +02:00
Eric Christopher 549e55b3d5 Temporarily Revert "[clangd] Add Random Forest runtime for code completion."
as a header doesn't appear to have made it into the commit.

This reverts commit 9b6765e784 and followup
2020-09-18 14:47:43 -07:00
Utkarsh Saxena 9b6765e784 [clangd] Add Random Forest runtime for code completion.
Summary:
[WIP]
- Proposes a json format for representing Random Forest model.
- Proposes a way to test the generated runtime using a test model.

TODO:
- Add generated source code snippet for easier review.
- Fix unused label warning.
- Figure out required using declarations for CATEGORICAL columns from Features.json.
- Necessary Google3 internal modifications for blaze before landing.
- Add documentation for format of the model.
- Document more.

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83814
2020-09-18 19:25:56 +02:00
Adam Czachorowski c894bfd1f5 [clangd] Add option for disabling AddUsing tweak on some namespaces.
For style guides forbid "using" declarations for namespaces like "std".
With this new config option, AddUsing can be selectively disabled on
those.

Differential Revision: https://reviews.llvm.org/D87775
2020-09-18 16:46:09 +02:00
Adam Czachorowski 7029e5d4ca [clangd] Actually parse Index section of the YAML file.
This fixes a bug in dbf486c0de, which
introduced the Index section of the config, but did not register the
parse method, so it didn't work in a YAML file (but did in a test).

Differential Revision: https://reviews.llvm.org/D87710
2020-09-16 13:11:02 +02:00
Sam McCall 687e1d7121 [clangd] makeStringError,make_error<StringError> -> error() 2020-09-14 11:48:31 +02:00
Sam McCall 30667c967d [clangd] Add error() function for creating formatv-style llvm::Errors. NFC
Summary:
This is considerably terser than the makeStringError and friends, and
avoids verbosity cliffs that discourage adding log information.

It follows the syntax used in log/elog/vlog/dlog that have been successful.

The main caveats are:
 - it's strictly out-of-place in logger.h, though kind of fits thematically and
   in implementation
 - it claims the "error" identifier, which seems a bit too opinionated
   to put higher up in llvm

I've updated some users of StringError mostly at random - there are lots
more mechanical changes but I'd like to get this reviewed before making
them all.

Reviewers: kbobyrev, hokein

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83419
2020-09-14 10:43:42 +02:00
Nathan Ridge ca842c825a [clangd] Handle templates more consistently in type hierarchy
If the tree includes types derived from all specializations of
a template, do not misleadingly label the root node with the
name of a single specialization.

Fixes https://github.com/clangd/clangd/issues/507

Differential Revision: https://reviews.llvm.org/D86861
2020-09-01 19:18:37 -04:00
Haojian Wu 0aaa2acc4c [clangd] Compute the inactive code range for semantic highlighting.
Differential Revision: https://reviews.llvm.org/D85635
2020-08-26 10:50:31 +02:00
Adam Czachorowski 4d90ff59ac [clangd] When inserting "using", add "::" in front if that's the style.
We guess the style based on the existing using declarations. If there
are any and they all start with ::, we add it to the newly added one
too.

Differential Revision: https://reviews.llvm.org/D86473
2020-08-25 14:07:49 +02:00
Adam Czachorowski b488935320 [clangd] Discard diagnostics from another SourceManager.
This can happen when building implicit modules, as demonstrated in test.
The CompilerInstance uses the same StoredDiags, but different
SourceManager. This used to crash clangd when it tried to relocate the
diagnostic to the main file, which, according to SourceManager from the
diagnostic, is a fake <module-includes> file.

Differential Revision: https://reviews.llvm.org/D85753
2020-08-21 13:11:21 +02:00