Commit Graph

25 Commits

Author SHA1 Message Date
Kadir Cetinkaya c746781f50
[clangd] Fix data race in BackgroundIndex test
MockFSProvider is not thread-safe. Make sure we don't modify it while
background index is working.
2020-05-09 18:15:27 +02:00
Benjamin Kramer 777180a32b [ADT] Make StringRef's std::string conversion operator explicit
This has the same behavior as converting std::string_view to
std::string. This is an expensive conversion, so explicit conversions
are helpful for avoiding unneccessary string copies.
2020-01-28 23:47:07 +01:00
Sam McCall 7d20e80225 [clangd] Show background index status using LSP 3.15 work-done progress notifications
Summary:
It simply shows the completed/total items on the background queue, e.g.
 indexing: 233/1000
The denominator is reset to zero every time the queue goes idle.

The protocol is fairly complicated here (requires creating a remote "progress"
resource before sending updates). We implement the full protocol, but I've added
an extension allowing it to be skipped to reduce the burden on clients - in
particular the lit test takes this shortcut.

The addition of background index progress to DiagnosticConsumer seems ridiculous
at first glance, but I believe that interface is trending in the direction of
"ClangdServer callbacks" anyway. It's due for a rename, but otherwise actually
fits.

Reviewers: kadircet, usaxena95

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D73218
2020-01-24 12:21:08 +01:00
Sam McCall 99768b243c [clangd] (take 2) Try harder to find a plausible `clang` as argv0, particularly on Mac.
Summary:
This was originally committed in 88bccded8f,
and reverted in 93f77617ab.

This version is now much more testable: the "detect toolchain properties" part
is still not tested but also not active in tests.
All the command manipulation based on the detected properties is
directly tested, and also not active in other tests.

Fixes https://github.com/clangd/clangd/issues/211
Fixes https://github.com/clangd/clangd/issues/178

Reviewers: kbobyrev, ilya-biryukov

Subscribers: mgorny, ormris, cfe-commits, usaxena95, kadircet, arphaman, jkorous, MaskRay

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71029
2019-12-06 09:47:03 +01:00
Sam McCall 93f77617ab Revert "[clangd] repair mac tests for 88bccded8fa1"
Revert "[clangd] Try harder to find a plausible `clang` as argv0, particularly on Mac."
2019-12-02 22:13:29 +01:00
Sam McCall 82039cbc8d [clangd] repair mac tests for 88bccded8f 2019-12-02 21:55:54 +01:00
Sam McCall 88bccded8f [clangd] Try harder to find a plausible `clang` as argv0, particularly on Mac.
Summary:
Fixes https://github.com/clangd/clangd/issues/211
Fixes https://github.com/clangd/clangd/issues/178

No tests - this is hard to test, and basically impossible to verify what we want
(this produces compile commands that work on a real mac with recent toolchain)

(Need someone on mac to verify it actually fixes these!)

Reviewers: kbobyrev, ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70863
2019-12-02 16:17:40 +01:00
Haojian Wu c8e3f43ab5 [clangd] Use our own relation kind.
Summary:
Move the RelationKind from Serialization.h to Relation.h. This patch doesn't
introduce any breaking changes.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68981

llvm-svn: 375117
2019-10-17 14:08:28 +00:00
Jonas Devlieghere 1c705d9c53 [clang-tools-extra] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368944
2019-08-14 23:52:23 +00:00
Kadir Cetinkaya 91e5f4b46b Revert "Revert r366458, r366467 and r366468"
This reverts commit 9c377105da.

[clangd][BackgroundIndexLoader] Directly store DependentTU while loading shard

Summary:
We were deferring the population of DependentTU field in LoadedShard
until BackgroundIndexLoader was consumed. This actually triggers a use after
free since the shards FileToTU was pointing at could've been moved while
consuming the Loader.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64980

llvm-svn: 366559
2019-07-19 10:18:52 +00:00
Azharuddin Mohammed 9c377105da Revert r366458, r366467 and r366468
r366458 is causing test failures. r366467 and r366468 had to be reverted as
they were casuing conflict while reverting r366458.

r366468 [clangd] Remove dead code from BackgroundIndex
r366467 [clangd] BackgroundIndex stores shards to the closest project
r366458 [clangd] Refactor background-index shard loading

llvm-svn: 366551
2019-07-19 09:26:33 +00:00
Kadir Cetinkaya 40073f922a [clangd] Refactor background-index shard loading
Reviewers: sammccall

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64712

llvm-svn: 366458
2019-07-18 16:25:36 +00:00
Sam McCall 06377ae2e5 [clangd] Don't rebuild background index until we indexed one TU per thread.
Summary:
This increases the odds that the boosted file (cpp file matching header)
will be ready. (It always enqueues first, so it'll be present unless
another thread indexes *two* files before the first thread indexes one.)

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64682

llvm-svn: 366199
2019-07-16 10:17:06 +00:00
Sam McCall 0f7146db9b [clangd] Prioritize indexing of files that share a basename with the open file.
Summary:
In practice, opening Foo.h will still often result in Foo.cpp making the
second index build instead of the first, as the rebuild policy doesn't
know to wait.

Reviewers: kadircet

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64575

llvm-svn: 365888
2019-07-12 10:18:42 +00:00
Sam McCall 7e27d86afb [clangd] Add priorities to background index queue, extract to separate class
Reviewers: kadircet

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64560

llvm-svn: 365773
2019-07-11 13:34:38 +00:00
Sam McCall 9145f265b0 [clangd] fix assert in test after r365531.
Unverified because CMake/ninja seems to be broken...

llvm-svn: 365576
2019-07-09 23:05:20 +00:00
Sam McCall 2f760c44e6 [clangd] Rewrite of logic to rebuild the background index serving structures.
Summary:
Previously it was rebuilding every 5s by default, which was much too frequent
in the long run - the goal was to provide an early build. There were also some
bugs. There were also some bugs, and a dedicated thread was used in production
but not tested.

 - rebuilds are triggered by #TUs built, rather than time. This should scale
   more sensibly to fast vs slow machines.
 - there are two separate indexed-TU thresholds to trigger index build: 5 TUs
   for the first build, 100 for subsequent rebuilds.
 - rebuild is always done on the regular indexing threads, and is affected by
   blockUntilIdle. This means unit/lit tests run the production configuration.
 - fixed a bug where we'd rebuild after attempting to load shards, even if there
   were no shards.
 - the BackgroundIndexTests don't really test the subtleties of the rebuild
   policy (for determinism, we call blockUntilIdle, so rebuild-on-idle is enough
   to pass the tests). Instead, we expose the rebuilder as a separate class and
   have fine-grained tests for it.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64291

llvm-svn: 365531
2019-07-09 18:30:49 +00:00
Kadir Cetinkaya 12b14869dc [clangd] Make HadErrors part of background index's internal state
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64147

llvm-svn: 365123
2019-07-04 09:52:12 +00:00
Kadir Cetinkaya adbb347ffe [clangd] Add HadErrors field into shards
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64133

llvm-svn: 365122
2019-07-04 09:52:04 +00:00
Kadir Cetinkaya 11e1c50b08 [clangd] Store hash of command line in index shards.
Summary: This is to enable cache invalidation when command line flags changes.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64018

llvm-svn: 365121
2019-07-04 09:51:53 +00:00
Kadir Cetinkaya a6fedc8bd6 [clangd] Also cache failures while indexing
Summary:
Clangd currently doesn't cache any indexing failures, which results in
retrying those failed files even if their contents haven't changed.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63986

llvm-svn: 365120
2019-07-04 09:51:43 +00:00
Nathan Ridge f1e6f5713c [clangd] Index API and implementations for relations
Summary:
This builds on the relations support added in D59407, D62459,
and D62471 to expose relations in SymbolIndex and its
implementations.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62839

llvm-svn: 363481
2019-06-15 02:26:47 +00:00
Kadir Cetinkaya 70674549f1 [clangd] Count number of references while merging RefSlabs inside FileIndex
Summary:
For counting number of references clangd was relying on merging every
duplication of a symbol. Unfortunately this does not apply to FileIndex(and one
of its users' BackgroundIndex), since we get rid of duplication by simply
dropping symbols coming from non-canonical locations. So only one or two(coming
from canonical declaration header and defined source file, if exists)
replications of the same symbol reaches merging step.

This patch changes reference counting logic to rather count number of different
RefSlabs a given SymbolID exists.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, mgrang, arphaman, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59481

llvm-svn: 360344
2019-05-09 14:22:07 +00:00
Ilya Biryukov 3ab77491dd [clangd] Qualify uses of ::testing everywhere. NFC
Add an initial '::' qualifier to all usages of 'testing' namespace that
did not have one.

The goal is to make our code style in tests more consistent.

llvm-svn: 360026
2019-05-06 10:08:47 +00:00
Sam McCall b804eef090 [clangd] Move clangd tests to clangd directory. check-clangd is no longer part of check-clang-tools.
Summary:
Motivation:
 - this layout is a pain to work with
 - without a common root, it's painful to express things like "disable clangd" (D61122)
 - CMake/lit configs are a maintenance hazard, and the more the one-off hacks
   for various tools are entangled, the more we see apathy and non-ownership.

This attempts to use the bare-minimum configuration needed (while still
supporting the difficult cases: windows, standalone clang build, dynamic libs).
In particular the lit.cfg.py and lit.site.cfg.py.in are merged into lit.cfg.in.
The logic in these files is now minimal.

(Much of clang-tools-extra's lit configs can probably be cleaned up by reusing
lit.llvm.llvm_config.use_clang(), and every llvm project does its own version of
LDPATH mangling. I haven't attempted to fix any of those).

Docs are still in clang-tools-extra/docs, I don't have any plans to touch those.

Reviewers: gribozavr

Subscribers: mgorny, javed.absar, MaskRay, jkorous, arphaman, kadircet, jfb, cfe-commits, ilya-biryukov, thakis

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61187

llvm-svn: 359424
2019-04-29 08:44:01 +00:00