The `let` command was added in commit 045c15ba (Add new 'let' command to
bind arbitrary values into constants., 2014-04-23).
The `let` command and the non-existant `l` command were documented in
commit 233092a0 (Add 'let' to the help message., 2015-02-27).
Implement the `l` command now for completeness.
llvm-svn: 343533
Conditionally compile the parts of clang-tidy which depend on the static
analyzer.
Funnily enough, I made the patch to exclude this from the build in 2013,
and it was committed with the comment that the tool should not be fully
excluded, but only the parts of it which depend on the analyzer should
be excluded.
http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20130617/081797.html
This commit implements that idea.
Reviewed By: aaron.ballman
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D52334
llvm-svn: 343528
check-clangd is included via check-clang-tools, otherwise we will run two
instances of check-clangd in parallel when running 'check-all',
which may have race condition.
llvm-svn: 343526
Summary:
So we don't have to run "check-clang-tools" (which builds and tests all
clang tools) to verify our clangd-related change. It'd save waiting time for
clangd developers.
check-clangd (build ~1000 files, run ~340 tests) vs check-clang-tools (build
~3000 files, run ~1000 tests).
In the future, we probably want to add similar target for other
clang-tools (e.g. clang-tidy).
Reviewers: sammccall
Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52710
llvm-svn: 343474
Summary:
When no scope qualifier is specified, allow completing index symbols
from any scope and insert proper automatically. This is still experimental and
hidden behind a flag.
Things missing:
- Scope proximity based scoring.
- FuzzyFind supports weighted scopes.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: kbobyrev, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52364
llvm-svn: 343248
Summary:
If we have some range information coming from clang diagnostic, promote
that one even if it doesn't contain diagnostic location inside.
Reviewers: sammccall, ioeric
Reviewed By: ioeric
Subscribers: ilya-biryukov, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D52544
llvm-svn: 343197
* With the current implementation, `sizeof(std::vector<Chunk>)` is added
twice to the `Dex` memory estimate which is incorrect
* `Dex` logs memory usage estimation before `BackingDataSize` is set and
hence the log report excludes size of the external `SymbolSlab` which is
coupled with `Dex` instance
Reviewed By: ioeric
Differential Revision: https://reviews.llvm.org/D52503
llvm-svn: 343117
Because `PostingList` objects are compressed, it is now impossible to
see elements other than the current one and the documentation doesn't
match implementation anymore.
Reviewed By: ioeric
Differential Revision: https://reviews.llvm.org/D52545
llvm-svn: 343116
Summary: Soon we can drop support for MR-via-YAML.
I need to modify some out-of-tree versions to use the library, first.
Reviewers: kadircet
Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D52465
llvm-svn: 343019
Summary:
Finds instances of namespaces concatenated using explicit syntax, such as `namespace a { namespace b { [...] }}` and offers fix to glue it to `namespace a::b { [...] }`.
Properly handles `inline` and unnamed namespaces. ~~Also, detects empty blocks in nested namespaces and offers to remove them.~~
Test with common use cases included.
I ran the check against entire llvm repository. Except for expected `nested namespace definitions only available with -std=c++17 or -std=gnu++17` warnings I noticed no issues when the check was performed.
Example:
```
namespace a { namespace b {
void test();
}}
```
can become
```
namespace a::b {
void test();
}
```
Patch by wgml!
Reviewers: alexfh, aaron.ballman, hokein
Reviewed By: aaron.ballman
Subscribers: JonasToth, Eugene.Zelenko, lebedev.ri, mgorny, xazax.hun, cfe-commits
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D52136
llvm-svn: 343000
Summary:
Interface is in one file, implementation in two as they have little in common.
A couple of ad-hoc YAML functions left exposed:
- symbol -> YAML I expect to keep for tools like dexp
- YAML -> symbol is used for the MR-style indexer, I think we can eliminate
this (merge-on-the-fly, else use a different serialization)
Reviewers: kbobyrev
Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52453
llvm-svn: 342999
For consistency, functional-style code pieces are replaced with their
simple counterparts to improve readability.
Also, file headers are fixed to comply with LLVM Coding Standards.
`static` member of anonymous namespace is not marked `static` anymore,
because it is redundant.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D52466
llvm-svn: 342974
This patch implements Variable-length Byte compression of `PostingList`s
to sacrifice some performance for lower memory consumption.
`PostingList` compression and decompression was extensively tested using
fuzzer for multiple hours and runnning significant number of realistic
`FuzzyFindRequests`. AddressSanitizer and UndefinedBehaviorSanitizer
were used to ensure the correct behaviour.
Performance evaluation was conducted with recent LLVM symbol index (292k
symbols) and the collection of user-recorded queries (7751
`FuzzyFindRequest` JSON dumps):
| Metrics | Before| After | Change (%)
| ----- | ----- | ----- | -----
| Memory consumption (posting lists only), MB | 54.4 | 23.5 | -60%
| Time to process queries, sec | 7.70 | 9.4 | +25%
Reviewers: sammccall, ioeric
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D52300
llvm-svn: 342965
Summary:
After r329813, clang-apply-replacements stopped deduplicating identical
replacements; however, tools like clang-tidy relies on the deduplication of
identical dignostics replacements from different TUs to apply fixes correctly.
This change partially roll back the behavior by deduplicating changes from
diagnostics. Ideally, we should deduplicate on diagnostics level, but we need to
figure out an effecient way.
Reviewers: bkramer
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D52264
llvm-svn: 342951
`Dex` should utilize `FuzzyFindRequest.RestrictForCodeCompletion` flags
and omit symbols not meant for code completion when asked for it.
The measurements below were conducted with setting
`FuzzyFindRequest.RestrictForCodeCompletion` to `true` (so that it's
more realistic). Sadly, the average latency goes down, I suspect that is
mostly because of the empty queries where the number of posting lists is
critical.
| Metrics | Before | After | Relative difference
| ----- | ----- | ----- | -----
| Cumulative query latency (7000 `FuzzyFindRequest`s over LLVM static index) | 6182735043 ns | 7202442053 ns | +16%
| Whole Index size | 81.24 MB | 81.79 MB | +0.6%
Out of 292252 symbols collected from LLVM codebase 136926 appear to be
restricted for code completion.
Reviewers: ioeric
Differential Revision: https://reviews.llvm.org/D52357
llvm-svn: 342866
Summary:
Currently LSP clients cannot directly change EnableFunctionArgSnippets parameter.
This patch is to provide them with a way to enable/disable that functionality.
Reviewers: hokein, ioeric, ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: sammccall, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D51214
llvm-svn: 342533
Summary:
Pros:
o Loading macros from preamble for every completion is slow (see profile).
o Calculating macro USR is also slow (see profile).
o Sema can provide a lot of macro completion results (e.g. when filter is empty,
60k for some large TUs!).
Cons:
o Slight memory increase in dynamic index (~1%).
o Some extra work during preamble build (should be fine as preamble build and
indexAST is way slower).
Before:
{F7195645}
After:
{F7195646}
Reviewers: ilya-biryukov, sammccall
Reviewed By: sammccall
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52078
llvm-svn: 342529
findStyleKind is only called if D is an explicit identifier with a name,
so the checks for operators will never return true. The explicit assert()
enforces this invariant.
Differential Revision: https://reviews.llvm.org/D52179
llvm-svn: 342514
Summary:
FileIndex now provides explicit interfaces for preamble and main file updates.
This avoids growing parameter list when preamble and main symbols diverge
further (e.g. D52078). This also gets rid of the hack in `indexAST` that
inferred main file index based on `TopLevelDecls`.
Also separate `indexMainDecls` from `indexAST`.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52222
llvm-svn: 342460
Summary:
This patch uses CHECK-NOTES for the tests.
Its part of an effort to test *ALL* generated diagnostics in clang-tidy,
as emitted notes were previously ignored.
Reviewers: alexfh, aaron.ballman, hokein
Reviewed By: alexfh
Subscribers: xazax.hun, cfe-commits
Differential Revision: https://reviews.llvm.org/D52178
llvm-svn: 342458
Summary:
We can use cl::ResetCommandLineParser() to support different types of
command-lines, as long as we're careful about option lifetimes.
(I tried using subcommands, but the error messages were bad)
I found a mostly-reasonable pattern to isolate the fiddly parts.
Added -scope and -limit flags to the `find` command to demonstrate.
(Note that scope support seems to be broken in dex?)
Fixed symbol lookup to parse symbol IDs.
Caveats:
- with command help (e.g. `find -help`), you also get some spam
about required arguments. This is a bug in llvm::cl, which prints
these to errs() rather than the designated stream.
Reviewers: kbobyrev
Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51989
llvm-svn: 342456
Test failed as D52120 made ExprMutationAnalyzer smarter, fixed by:
- Add move-ctor for `Mutable` to make it actually movable.
- Properly implement `remove_reference`.
The failed test case is:
void negativeVarIsMoved() {
for (auto M : View<Iterator<Mutable>>()) {
auto Moved = std::move(M);
}
}
Before D52120, `std::move(M)` itself is considered as a mutation to `M`,
while after D52120 it's only considered as a cast to rvalue, the
move-assignment is what causes the actual mutation. The test case didn't
mock things properly so the intended move-assignement was actually a
copy-assignment.
llvm-svn: 342417
Summary:
PR37913 documents wrong behaviour for a templated exception factory function.
The check does misidentify dependent types as not derived from std::exception.
The fix to this problem is to ignore dependent types, the analysis works correctly
on the instantiated function.
Reviewers: aaron.ballman, alexfh, hokein, ilya-biryukov
Reviewed By: alexfh
Subscribers: lebedev.ri, nemanjai, mgorny, kbarton, xazax.hun, cfe-commits
Differential Revision: https://reviews.llvm.org/D48714
llvm-svn: 342393
Summary:
Hello, i would like to suggest a fix for one of the checks in clang-tidy.The bug was reported in https://bugs.llvm.org/show_bug.cgi?id=32575 where you can find more information.
For example:
```
template <typename T0>
struct S {
template <typename T>
void g() const {
int a;
(void)a;
}
};
void f() {
S<int>().g<int>();
}
```
this piece of code should not trigger any warning by the check modernize-redundant-void-arg but when we execute the following command
```
clang_tidy -checks=-*,modernize-redundant-void-arg test.cpp -- -std=c++11
```
we obtain the following warning:
/Users/eco419/Desktop/clang-tidy.project/void-redundand_2/test.cpp:6:6: warning: redundant void argument list in function declaration [modernize-redundant-void-arg]
(void)a;
^~~~
Reviewers: aaron.ballman, hokein, alexfh, JonasToth
Reviewed By: aaron.ballman, JonasToth
Subscribers: JonasToth, lebedev.ri, cfe-commits
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D52135
llvm-svn: 342388
Summary:
To stay fast, enable single-file-mode instead. This is fine since completions
in the preamble are simple.
The net effect for now is to suppress the spurious TopLevel completions when
completing inside the preamble.
Once Sema has include directive completion, this will be more important.
Reviewers: ilya-biryukov
Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52071
llvm-svn: 342230
This patch abstracts `PostingList` interface and reuses existing
implementation. It will be used later to test different `PostingList`
representations.
No functionality change is introduced, this patch is mostly refactoring
so that the following patches could focus on functionality while not
being too hard to review.
Reviewed By: sammccall, ioeric
Differential Revision: https://reviews.llvm.org/D51982
llvm-svn: 342155
As discussed during D51860 review, it is better to use `llvm::Optional`
here as it has clear semantics which reflect intended behavior.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D52028
llvm-svn: 342138
After `FuzzyFindRequest` JSON (de)serialization was introduced, it
should replace ad-hoc fuzzy-find request parsing implemented in the
IndexBenchmark driver.
Reviewed By: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D51971
llvm-svn: 342137
Summary:
The cancelable scopes are managed by JSONRPCDispatcher so that all Handlers
run in cancelable contexts.
(Previously ClangdServer did this, for code completion only).
Cancellation request processing is therefore also in JSONRPCDispatcher.
(Previously it was in ClangdLSPServer).
This doesn't actually make any new commands *respect* cancellation - they'd
need to check isCancelled() and bail out. But it opens the door to doing
this incrementally, and putting such logic in common machinery like TUScheduler.
I also rewrote the ClangdServer class/threading comments because I wanted to
add to it and I got carried away.
Reviewers: ilya-biryukov, kadircet
Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D52004
llvm-svn: 342135
Summary:
The wording implies global index support, which is confusing.
As most users shouldn't care about this flag, also make it hidden to avoid
further confusion.
Reviewers: sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51977
llvm-svn: 342134
Summary:
Task is no longer exposed:
- task cancellation is hidden as a std::function
- task creation returns the new context directly
- checking is via free function only, with no way to avoid the context lookup
The implementation is essentially the same, but a bit terser as it's hidden.
isCancelled() is now safe to use outside any task (it returns false).
This will leave us free to sprinkle cancellation in e.g. TUScheduler without
needing elaborate test setup, and lets callers that don't cancel "just work".
Updated the docs to describe the new expected use pattern.
One thing I noticed: there's nothing async-specific about the cancellation.
Async tasks can be cancelled from any thread (typically the one that created
them), sync tasks can be cancelled from any *other* thread in the same way.
So the docs now refer to "long-running" tasks instead of async ones.
Updated usage in code complete, without any structural changes.
I didn't update all the names of the helpers in ClangdLSPServer (these will
likely be moved to JSONRPCDispatcher anyway).
Reviewers: ilya-biryukov, kadircet
Subscribers: ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits
Differential Revision: https://reviews.llvm.org/D51996
llvm-svn: 342130
Many editors provide extensions and plugins with LSP Client
functionality. Many of these are known to work with Clangd, this patch
points users to the relevant resources for better experience.
Reviewed By: ioeric, ilya-biryukov
Differential Revision
llvm-svn: 342129
Summary:
Given that the indexer binary is put directly into ./bin directory
when built, 'clangd-' prefix seems to provide better context to the
reader than 'global-'.
The new name is also shorter and easier to type.
Reviewers: ioeric, sammccall, kadircet
Reviewed By: ioeric, sammccall
Subscribers: kbobyrev, ilya-biryukov, mgorny, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D51987
llvm-svn: 342123
This patch introduces index benchmarks on top of the proposed LLVM
benchmark pull.
Reviewed By: sammccall, lebedev.ri
Differential Revision: https://reviews.llvm.org/D51090
llvm-svn: 342026
Summary:
This is 2/2 of moving ExprMutationAnalyzer from clangtidy to clang/Analysis.
ExprMutationAnalyzer is moved to clang/Analysis in D51948.
This diff migrates existing usages within clangtidy to point to the new
location and remove the old copy of ExprMutationAnalyzer.
Reviewers: george.karpenkov, JonasToth
Reviewed By: george.karpenkov
Subscribers: mgorny, a.sidorin, Szelethus, cfe-commits
Differential Revision: https://reviews.llvm.org/D51950
llvm-svn: 342006
Summary:
This handles cases like this:
```
typedef int& IntRef;
void mutate(IntRef);
void f() {
int x;
mutate(x);
}
```
where the param type is a sugared type (`TypedefType`) instead of a
reference type directly.
Note that another category of similar but different cases are already
handled properly before:
```
typedef int Int;
void mutate(Int&);
void f() {
int x;
mutate(x);
}
```
Reviewers: aaron.ballman, alexfh, george.karpenkov
Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits
Differential Revision: https://reviews.llvm.org/D50953
llvm-svn: 341986
Summary:
For smart pointers like std::unique_ptr which uniquely owns the
underlying object, treat the mutation of the pointee as mutation of the
smart pointer itself.
This gives better behavior for cases like this:
```
void f(std::vector<std::unique_ptr<Foo>> v) { // undesirable analyze result of `v` as not mutated.
for (auto& p : v) {
p->mutate(); // only const member function `operator->` is invoked on `p`
}
}
```
Reviewers: hokein, george.karpenkov
Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits
Differential Revision: https://reviews.llvm.org/D50883
llvm-svn: 341967
This is the same as D50619 plus fixes for buildbot failures on windows.
The test failures on windows are caused by -fdelayed-template-parsing
and is fixed by forcing -fno-delayed-template-parsing on test cases that
requires AST for uninstantiated templates.
llvm-svn: 341891
Summary:
I have hit this the rough way, while trying to use this in D51870.
There is no particular point in storing the pointers, and moreover
the pointers are assumed to be non-null, and that assumption is not
enforced. If they are null, it won't be able to do anything good
with them anyway.
Initially i thought about simply adding asserts() that they are
not null, but taking/storing references looks like even cleaner solution?
Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=38888 | PR38888 ]]
Reviewers: JonasToth, shuaiwang, alexfh, george.karpenkov
Reviewed By: shuaiwang
Subscribers: xazax.hun, a.sidorin, Szelethus, cfe-commits
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D51884
llvm-svn: 341854
Summary:
- If a function is unresolved, assume it mutates its arguments
- Follow unresolved member expressions for nested mutations
Reviewers: aaron.ballman, JonasToth, george.karpenkov
Subscribers: xazax.hun, a.sidorin, cfe-commits
Differential Revision: https://reviews.llvm.org/D50619
llvm-svn: 341848
JSON (de)serialization of `FuzzyFindRequest` might be useful for both
D51090 and D51628. Also, this allows precise logging of the fuzzy find
requests.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D51852
llvm-svn: 341802
Currently, `SymbolIndex::estimateMemoryUsage()` returns the "overhead"
estimate, i.e. the estimate of the Index data structure excluding
backing data (such as Symbol Slab and Reference Slab). This patch
propagates information about paired data size where necessary.
Reviewed By: ioeric, sammccall
Differential Revision: https://reviews.llvm.org/D51539
llvm-svn: 341800
Summary:
This wasn't actually async (due to std::future destructor blocking).
If it were, we would have clean shutdown issues if main returned
and destroyed Placeholder before the thread is done with it.
We could attempt to avoid any blocking by using shared_ptr or weak_ptr tricks so
the thread can detect Placeholder's destruction, but there are other potential
issues (e.g. loadIndex does tracing, and we'll destroy the tracer...)
Instead, once LSPServer::run returns, we wait for the index to finish loading
before exiting. Performance is not critical in this situation.
Reviewers: ilya-biryukov
Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51674
llvm-svn: 341797
If the current element is already beyond advanceTo()'s DocID, just
return instead of doing binary search. This simple optimization saves up
to 6-7% performance,
Reviewed By: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D51802
llvm-svn: 341781
Summary: This adds a few common acronyms we found were missing from PropertyDeclarationCheck.
Reviewers: Wizard, hokein
Reviewed By: hokein
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D51819
llvm-svn: 341721
Summary: This adds a few common acronyms we found were missing from PropertyDeclarationCheck.
Reviewers: Wizard, hokein
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D51819
llvm-svn: 341720
`URIDistance` constructor should mention that `Sources` must contain
*absolute paths*, not URIs. This is not very clear when looking at the
interface, especially given that `distance(...)` accepts `URI`, not an
absolute path which can give the wrong impression.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D51691
llvm-svn: 341639
This patch sets URI schemes of Dex to SymbolCollector's default schemes
in case callers tried to pass empty list of schemes. This was the case
for initialization in Clangd main and was a reason of incorrect
behavior.
Also, it fixes a bug with missed `continue;` after spotting invalid URI
scheme conversion.
llvm-svn: 341552
Quality.cpp defines a structure for convenient storage of Top N items,
it should be used instead of the `std::priority_queue` with slightly
obscure semantics.
This patch does not affect functionality.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D51676
llvm-svn: 341544
This patch introduces `PathURI` Search Token kind and utilizes it to
uprank symbols which are defined in files with small distance to the
directory where the fuzzy find request is coming from (e.g. files user
is editing).
Reviewed By: ioeric
Reviewers: ioeric, sammccall
Differential Revision: https://reviews.llvm.org/D51481
llvm-svn: 341542
This missing directory is not yet released, but is causing some problems
internally. It's gonna be released eventually and received permission to
include it here. This matcher will also be periodically updated by my
team as we have more releases and or problems internally.
Patch by Hugo Gonzalez!
Differential Revision: https://reviews.llvm.org/D51699
llvm-svn: 341488
Summary:
GoToDefinition returns all declaration results (implicit/explicit) that are
in the same location, and the results are returned in arbitrary order.
Some LSP clients defaultly take the first result as the final result, which
might present a bad result (implicit decl) to users.
This patch ranks the result based on whether the declarations are
referenced explicitly/implicitly. We put explicit declarations first.
This also improves the "hover" (which just take the first result) feature
in some cases.
Reviewers: ilya-biryukov
Subscribers: kadircet, ioeric, MaskRay, jkorous, mgrang, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D50438
llvm-svn: 341463
x0.2 seems to be too much penalty, macros might be wanted in some cases;
changing to 0.5x instead. The tuning didn't affect ranking for non-macro
completions.
llvm-svn: 341449
Summary:
Like D51475 but simplified based on recent patches.
While here, clarify that loadIndex() takes a filename, not file content.
Reviewers: ioeric
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51638
llvm-svn: 341376
Summary:
This is intended to replace the current YAML format for general use.
It's ~10x more compact than YAML, and ~40% more compact than gzipped YAML:
llvmidx.riff = 20M, llvmidx.yaml = 272M, llvmidx.yaml.gz = 32M
It's also simpler/faster to read and write.
The format is a RIFF container (chunks of (type, size, data)) with:
- a compressed string table
- simple binary encoding of symbols (with varints for compactness)
It can be extended to include occurrences, Dex posting lists, etc.
There's no rich backwards-compatibility scheme, but a version number is included
so we can detect incompatible files and do ad-hoc back-compat.
Alternatives considered:
- compressed YAML or JSON: bulky and slow to load
- llvm bitstream: confusing model and libraries are hard to use. My attempt
produced slightly larger files, and the code was longer and slower.
- protobuf or similar: would be really nice (esp for back-compat) but the
dependency is a big hassle
- ad-hoc binary format without a container: it seems clear we're going
to add posting lists and occurrences here, and that they will benefit
from sharing a string table. The container makes it easy to debug
these pieces in isolation, and make them optional.
Reviewers: ioeric
Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51585
llvm-svn: 341375
`buildStaticIndex()` is used by two other tools that I'm building, now
it's useful outside of `tool/ClangdMain.cpp`.
Also, slightly refactor the code while moving it to the different source
file.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D51626
llvm-svn: 341369
Summary:
A few things that I noticed while merging the SwapIndex patch:
- SymbolOccurrences and particularly SymbolOccurrenceSlab are unwieldy names,
and these names appear *a lot*. Ref, RefSlab, etc seem clear enough
and read/format much better.
- The asymmetry between SymbolSlab and RefSlab (build() vs freeze()) is
confusing and irritating, and doesn't even save much code.
Avoiding RefSlab::Builder was my idea, but it was a bad one; add it.
- DenseMap<SymbolID, ArrayRef<Ref>> seems like a reasonable compromise for
constructing MemIndex - and means many less wasted allocations than the
current DenseMap<SymbolID, vector<Ref*>> for FileIndex, and none for
slabs.
- RefSlab::find() is not actually used for anything, so we can throw
away the DenseMap and keep the representation much more compact.
- A few naming/consistency fixes: e.g. Slabs,Refs -> Symbols,Refs.
Reviewers: ioeric
Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51605
llvm-svn: 341368
Summary:
- DynamicIndex doesn't implement ParsingCallbacks, to make its role clearer.
ParsingCallbacks is a separate object owned by the receiving TUScheduler.
(I tried to get rid of the "index-like-object that doesn't implement index"
but it was too messy).
- Clarified(?) docs around DynamicIndex - fewer details up front, more details
inside.
- Exposed dynamic index from ClangdServer for memory monitoring and more
direct testing of its contents (actual tests not added here, wanted to get
this out for review)
- Removed a redundant and sligthly confusing filename param in a callback
Reviewers: ilya-biryukov
Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51221
llvm-svn: 341325
Summary:
This is now handled by a wrapper class SwapIndex, so MemIndex/DexIndex can be
immutable and focus on their job.
Old and busted:
I have a MemIndex, which holds a shared_ptr<vector<Symbol*>>, which keeps the
symbol slab alive. I update by calling build(shared_ptr<vector<Symbol*>>).
New hotness: I have a SwapIndex, which holds a unique_ptr<SymbolIndex>, which
holds a MemIndex, which holds a shared_ptr<void>, which keeps backing
data alive.
I update by building a new MemIndex and calling SwapIndex::reset().
Reviewers: kbobyrev, ioeric
Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51422
llvm-svn: 341318
Summary:
Currently, a symbol can have only one #include header attached, which
might not work well if the symbol can be imported via different #includes depending
on where it's used. This patch stores multiple #include headers (with # references)
for each symbol, so that CodeCompletion can decide which include to insert.
In this patch, code completion simply picks the most popular include as the default inserted header. We also return all possible includes and their edits in the `CodeCompletion` results.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: mgrang, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51291
llvm-svn: 341304
SymbolCollector will be used for two cases:
- collect Symbol type only, used for indexing preamble AST.
- collect Symbol and SymbolOccurrences, used for indexing main AST.
For finding local references from the AST, we will implement it in other ways.
llvm-svn: 341208
* Use consistent assertion messages in iterators implementations
* Silence a bunch of clang-tidy warnings: use `emplace_back` instead of
`push_back` where possible, make sure arguments have the same name in
header and implementation file, use for loop over ranges where possible
Reviewed by: ioeric
Differential Revision: https://reviews.llvm.org/D51528
llvm-svn: 341190
This is adjustment to allow the logic to work even if implicit UsingDirectiveDecls get actual source locations.
There should be no functionality change.
llvm-svn: 341161
Summary: This allows loading a file with pre-defined let commands for example.
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D51261
llvm-svn: 341145
It's always replaced with the same (short) static string, so just put that
there directly.
No intended behavior change.
https://reviews.llvm.org/D51357
llvm-svn: 341130
Summary:
After code completion inserts a header, running signature help using the old
preamble will usually fail. So we add support for consistent preamble reads.
Reviewers: ilya-biryukov
Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51438
llvm-svn: 341076
Summary: Only accessible via the C++ API at the moment.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51437
llvm-svn: 341065
Instead of parsing and compiling the `llvm::Regex` each time, it's
faster to use basic string matching for filename prefix check.
Reviewed by: hokein
Differential Revision: https://reviews.llvm.org/D51360
llvm-svn: 341061
This patch introduces iterator cost concept to improve the performance
of Dex query iterators (mainly, AND iterator). Benchmarks show that the
queries become ~10% faster.
Before
```
-------------------------------------------------------
Benchmark Time CPU Iteration
-------------------------------------------------------
DexAdHocQueries 5883074 ns 5883018 ns 117
DexRealQ 959904457 ns 959898507 ns 1
```
After
```
-------------------------------------------------------
Benchmark Time CPU Iteration
-------------------------------------------------------
DexAdHocQueries 5238403 ns 5238361 ns 130
DexRealQ 873275207 ns 873269453 ns 1
```
Reviewed by: sammccall
Differential Revision: https://reviews.llvm.org/D51310
llvm-svn: 341057
Summary:
The check_clang_tidy.py script would allow mixing of `CHECK-NOTES` and `CHECK-MESSAGES` but running `FileCheck` for that would implicitly fail, because `CHECK-NOTES` bails out if there is a warning.
That means a clang-tidy test can not mix these constructs to check warnings with `CHECK-MESSAGES` and notes with `CHECK-NOTES`. The script gives now a clear error if that happens.
Reviewers: alexfh, aaron.ballman, lebedev.ri, hokein
Reviewed By: lebedev.ri
Subscribers: xazax.hun, cfe-commits
Differential Revision: https://reviews.llvm.org/D51381
llvm-svn: 341039
Finds instances where the user depends on internal details and warns them against doing so.
Should not be run on internal Abseil files or Abseil source code.
Patch by hugoeg!
Differential Revision: https://reviews.llvm.org/D50542
llvm-svn: 340928
This flags redundant calls to absl::StrCat where the result is being passed to another call to absl::StrCat or absl::StrAppend. Patch by Hugo Gonzalez and Samuel Benzaquen.
llvm-svn: 340918
Dex is now mature enough to be used as the default static index. This
patch performs the switch but introduces a hidden flag to allow users
fallback to Mem in case something happens.
Reviewed by: ioeric
Differential Revision: https://reviews.llvm.org/D51352
llvm-svn: 340828
This check ensures that users of Abseil do not open namespace absl in their code, as that violates our compatibility guidelines.
AbseilMatcher.h written by Hugo Gonzalez.
Patch by Deanna Garcia!
llvm-svn: 340800
Since the clangd-dev is intended to be the place for clangd-related
discussions, we should point new users to this mailing list while
probably mentioning cfe-dev, too.
Reviewed by: ioeric
Differential Revision: https://reviews.llvm.org/D51293
llvm-svn: 340749
Stop using `$$$` (empty) trigram and generating a posting list with all
items. Since TRUE iterator is already implemented and correctly inserted
when there are no real trigram posting lists, this is a valid
transformation.
Benchmarks show that this simple change allows ~30% speedup on dataset
of real completion queries.
Before
```
-------------------------------------------------------
Benchmark Time CPU Iterations
-------------------------------------------------------
DexAdHocQueries 5640321 ns 5640265 ns 120
DexRealQ 939835603 ns 939830296 ns 1
```
After
```
-------------------------------------------------------
Benchmark Time CPU Iterations
-------------------------------------------------------
DexAdHocQueries 3452014 ns 3451987 ns 203
DexRealQ 667455912 ns 667455750 ns 1
```
Reviewed by: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D51287
llvm-svn: 340729
This patch introduces LIMIT iterator, which is very important for
improving the quality of search query. LIMIT iterators can be applied on
top of BOOST iterators to prevent populating query request with a huge
number of low-quality symbols.
Reviewed by: sammccall
Differential Revision: https://reviews.llvm.org/D51029
llvm-svn: 340605
Summary:
For index-based code completion, send an asynchronous speculative index
request, based on the index request for the last code completion on the same
file and the filter text typed before the cursor, before sema code completion
is invoked. This can reduce the code completion latency (by roughly latency of
sema code completion) if the speculative request is the same as the one
generated for the ongoing code completion from sema. As a sequence of code
completions often have the same scopes and proximity paths etc, this should be
effective for a number of code completions.
Trace with speculative index request:{F6997544}
Reviewers: hokein, ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: javed.absar, jfb, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D50962
llvm-svn: 340604
This patch prints information about built index size estimation to
verbose logs. This is useful for optimizing memory usage of DexIndex and
comparisons with MemIndex.
Reviewed by: sammccall
Differential Revision: https://reviews.llvm.org/D51154
llvm-svn: 340601
Summary:
The new mode avoids serializing and deserializing YAML.
This results in better performance and less memory usage. Reduce phase
is now almost instant.
The default is to use the old mode going through YAML serialization to
allow migrating MapReduce clients that require the old mode to operate
properly. After we migrate the clients, we can switch the default to
the new mode.
Reviewers: hokein, ioeric, kbobyrev, sammccall
Reviewed By: ioeric
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D51155
llvm-svn: 340600
Summary:
Currently we match an include only if we are inside filename, with this patch we
will match whenever we are on the starting line of the include.
Reviewers: ilya-biryukov, hokein, ioeric
Reviewed By: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D51163
llvm-svn: 340539
Summary:
Whenever a code-completion is triggered within a class/struct/union looks at
base classes and figures out non-overriden virtual functions. Than suggests
completions for those.
Reviewers: ilya-biryukov, hokein, ioeric
Reviewed By: hokein
Subscribers: MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D50898
llvm-svn: 340530
Summary:
We were handling the EnableFunctionArgSnippets only when we are producing LSP
response. Move that code into CompletionItem generation so that internal clients
can benefit from that as well.
Reviewers: ilya-biryukov, ioeric, hokein
Reviewed By: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D51102
llvm-svn: 340527
Some of them timeout on our buildbots in certain configurations.
The timeouts are there to avoid hanging indefinitely on deadlocks, so
the exact number we put there does not matter.
llvm-svn: 340523
capability was given by the client
After r339738 Clangd started sending categories with each diagnostic, but that
broke the eglot client. This commit puts the categories behind a capability to
fix that breakage.
Differential Revision: https://reviews.llvm.org/D51077
llvm-svn: 340449
Summary: Adds the Abseil prefix to the list of prefixes in the documentation
Patch by Deanna Garcia!
Reviewers: aaron.ballman, hokein
Reviewed By: hokein
Subscribers: xazax.hun, cfe-commits
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D51100
llvm-svn: 340412
This check is an abseil specific check that checks for code using single character string literals as delimiters and transforms the code into characters.
The check was developed internally and has been running at google, this is just
a move to open source the check. It was originally written by @sbenza.
Patch by Deanna Garcia!
llvm-svn: 340411
This patch introduces BOOST iterator - a substantial block for efficient
and high-quality symbol retrieval. The concept of boosting allows
performing computationally inexpensive scoring on the query side so that
the final (expensive) scoring can only be applied on the items with the
highest preliminary score while eliminating the need to score too many
items.
Reviewed by: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D50970
llvm-svn: 340409
Summary:
It was previously only indexing the preamble decls. The new
implementation will index both the preamble and the main AST and
report both sets of symbols, preferring the ones from the main AST
whenever the symbol is present in both.
The symbols in the main AST slab always store all information
available in the preamble symbols, possibly adding more,
e.g. definition locations.
Reviewers: hokein, ioeric
Reviewed By: ioeric
Subscribers: kadircet, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D50889
llvm-svn: 340404
Summary:
Will be used for updating the dynamic index on updates to the open files.
Currently we collect only information coming from the preamble
AST. This has a bunch of limitations:
- Dynamic index misses important information from the body of the
file, e.g. locations of definitions.
- XRefs cannot be collected at all, since we can only obtain full
information for the current file (preamble is parsed with skipped
function bodies, therefore not reliable).
This patch only adds the new callback, actually updates to the index
will be done in a follow-up patch.
Reviewers: hokein
Reviewed By: hokein
Subscribers: kadircet, javed.absar, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D50847
llvm-svn: 340401
Stephen Kelly