Commit Graph

21 Commits

Author SHA1 Message Date
Kadir Cetinkaya 375c54fd1e [clangd] Only reduce priority of a thread for indexing.
Summary:
We'll soon have tasks pending for reading shards from disk, we want
them to have normal priority. Because:
- They are not CPU intensive, mostly IO bound.
- Give a good coverage for the project at startup, therefore it is worth
  spending some cycles.
- We have only one task per whole CDB rather than one task per file.

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D55315

llvm-svn: 349345
2018-12-17 12:30:27 +00:00
Haojian Wu e58e0903e2 [clangd] Avoid emitting Queued status when we are able to acquire the Barrier.
Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55359

llvm-svn: 349032
2018-12-13 13:09:50 +00:00
Sam McCall 032f3e7fd8 [clangd] Prevent thread starvation in tests on loaded systems.
Summary:
Background index deliberately runs low-priority, but for tests this may stop
them making progress.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D54938

llvm-svn: 347655
2018-11-27 12:09:13 +00:00
Sam McCall 8443f881bf [clangd] Fix compile on very old glibc
llvm-svn: 346638
2018-11-12 08:17:49 +00:00
Kadir Cetinkaya 8e422d6775 Delete dependency on config.h
Summary:
Since llvm/Config/config.h is not available on standalone builds,
use __USE_POSIX instead of HAVE_PTHREAD_H and get rid of the include.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: lebedev.ri, krytarowski, ilya-biryukov, ioeric, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D53935

llvm-svn: 345729
2018-10-31 15:37:09 +00:00
David Carlier 13727da72f [clangd] fix non linux build
There is no SCHED_IDLE semantic equivalent in BSD systems.

Reviewers: kadircet, sammccall

Revieweed By: sammccall

Differential Revision: https://reviews.llvm.org/D53922

llvm-svn: 345700
2018-10-31 09:04:15 +00:00
Kadir Cetinkaya 6675be8747 [clangd] Use thread pool for background indexing.
Reviewers: sammccall, ioeric

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D53651

llvm-svn: 345590
2018-10-30 12:13:27 +00:00
Sam McCall c008af6466 [clangd] Namespace style cleanup in cpp files. NFC.
Standardize on the most common namespace setup in our *.cpp files:
  using namespace llvm;
  namespace clang {
  namespace clangd {
  void foo(StringRef) { ... }
And remove redundant llvm:: qualifiers. (Except for cases like
make_unique where this causes problems with std:: and ADL).

This choice is pretty arbitrary, but some broad consistency is nice.
This is going to conflict with everything. Sorry :-/

Squash the other configurations:

A)
  using namespace llvm;
  using namespace clang;
  using namespace clangd;
  void clangd::foo(StringRef);
This is in some of the older files. (It prevents accidentally defining a
new function instead of one in the header file, for what that's worth).

B)
  namespace clang {
  namespace clangd {
  void foo(llvm::StringRef) { ... }
This is fine, but in practice the using directive often gets added over time.

C)
  namespace clang {
  namespace clangd {
  using namespace llvm; // inside the namespace
This was pretty common, but is a bit misleading: name lookup preferrs
clang::clangd::foo > clang::foo > llvm:: foo (no matter where the using
directive is).

llvm-svn: 344850
2018-10-20 15:30:37 +00:00
Ilya Biryukov 1d5e640428 [clangd] Use 'const Twine&' instead of 'Twine'. NFC
To fix clang-tidy warning

llvm-svn: 338037
2018-07-26 16:13:52 +00:00
Benjamin Kramer c36c09fe8e [clangd] Replace UniqueFunction with llvm::unique_function.
One implementation of this ought to be enough for everyone.

llvm-svn: 336228
2018-07-03 20:59:33 +00:00
Ilya Biryukov 9d8f61ad60 [clangd] Trace time the operations wait on Semaphore.
The Semaphore is currently used to limit the number of concurrently
running tasks. Tracing the wait times will allow to find out how much
time is wasted waiting on other operations to complete.

llvm-svn: 334495
2018-06-12 11:56:21 +00:00
Sam McCall d1e0deb271 [clangd] Debounce streams of updates.
Summary:
Don't actually start building ASTs for new revisions until either:
- 500ms have passed since the last revision, or
- we actually need the revision for something (or to unblock the queue)

In practice, this avoids the "first keystroke results in diagnostics" problem.
This is kind of awkward to test, and the test is pretty bad.
It can be observed nicely by capturing a trace, though.

Reviewers: hokein, ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43648

llvm-svn: 326546
2018-03-02 08:56:37 +00:00
Sam McCall 568e17f1e6 [clangd] Allow embedders some control over when diagnostics are generated.
Summary:
Through the C++ API, we support for a given snapshot version:
 - Yes: make sure we generate diagnostics for exactly this version
 - Auto: generate eventually-consistent diagnostics for at least this version
 - No: don't generate diagnostics for this version
Eventually auto should be debounced for better UX.

Through LSP, we force diagnostics for initial load (bypassing future debouncing)
and all updates follow the "auto" policy.

This is complicated to implement under the CancellationFlag design, so
rewrote that part to just inspect the queue instead.

It turns out we never pass None to the diagnostics callback, so remove Optional
from the signature. The questionable behavior of not invoking the callback at
all if CppFile::rebuild fails is not changed.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43518

llvm-svn: 325774
2018-02-22 13:11:12 +00:00
Sam McCall c901c5db78 [clangd] Tracing: name worker threads, and enforce naming scheduled async tasks
Summary:
This has a bit of a blast radius, but I think there's enough value in "forcing"
us to give names to these async tasks for debugging. Guessing about what
multithreaded code is doing is so unfun...

The "file" param attached to the tasks may seem to be redundant with the thread
names, but note that thread names are truncated to 15 chars on linux!
We'll be lucky to get the whole basename...

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43388

llvm-svn: 325480
2018-02-19 09:56:28 +00:00
Sam McCall 0bb24cd4fa [clangd] Stop exposing Futures from ClangdServer operations.
Summary:
LSP has asynchronous semantics, being able to block on an async operation
completing is unneccesary and leads to tighter coupling with the threading.

In practice only tests depend on this, so we add a general-purpose "block until
idle" function to the scheduler which will work for all operations.

To get this working, fix a latent condition-variable bug in ASTWorker, and make
AsyncTaskRunner const-correct.

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43127

llvm-svn: 324990
2018-02-13 08:59:23 +00:00
Ilya Biryukov 7e5ee26d1a Resubmit "[clangd] The new threading implementation"
Initially submitted as r324356 and reverted in r324386.

This change additionally contains a fix to crashes of the buildbots.
The source of the crash was undefined behaviour caused by
std::future<> whose std::promise<> was destroyed without calling
set_value().

llvm-svn: 324575
2018-02-08 07:37:35 +00:00
Ilya Biryukov 3693f5941a Revert "[clangd] The new threading implementation" (r324356)
And the follow-up changes r324361 and r324363.
These changes seem to break two buildbots:
  - http://lab.llvm.org:8011/builders/clang-atom-d525-fedora-rel/builds/14091
  - http://lab.llvm.org:8011/builders/clang-x86_64-linux-selfhost-modules-2/builds/16001

We will need to investigate what went wrong and resubmit the changes
afterwards.

llvm-svn: 324386
2018-02-06 19:22:40 +00:00
Ilya Biryukov 0c904861e6 [clangd] Fixed a bug in the new threading implementation.
This should fix the buildbots.

llvm-svn: 324363
2018-02-06 17:22:58 +00:00
Ilya Biryukov cce8883094 [clangd] The new threading implementation
Summary:
In the new threading model clangd creates one thread per file to manage
the AST and one thread to process each of the incoming requests.
The number of actively running threads is bounded by the semaphore to
avoid overloading the system.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, jkorous-apple, ioeric, hintonda, cfe-commits

Differential Revision: https://reviews.llvm.org/D42573

llvm-svn: 324356
2018-02-06 15:53:42 +00:00
Sam McCall d1a7a37c22 [clangd] Pass Context implicitly using TLS.
Summary:
Instead of passing Context explicitly around, we now have a thread-local
Context object `Context::current()` which is an implicit argument to
every function.
Most manipulation of this should use the WithContextValue helper, which
augments the current Context to add a single KV pair, and restores the
old context on destruction.

Advantages are:
- less boilerplate in functions that just propagate contexts
- reading most code doesn't require understanding context at all, and
  using context as values in fewer places still
- fewer options to pass the "wrong" context when it changes within a
  scope (e.g. when using Span)
- contexts pass through interfaces we can't modify, such as VFS
- propagating contexts across threads was slightly tricky (e.g.
  copy vs move, no move-init in lambdas), and is now encapsulated in
  the threadpool

Disadvantages are all the usual TLS stuff - hidden magic, and
potential for higher memory usage on threads that don't use the
context. (In practice, it's just one pointer)

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D42517

llvm-svn: 323872
2018-01-31 13:40:48 +00:00
Ilya Biryukov 75f1dd9b98 [clangd] Refactored threading in ClangdServer
Summary:
We now provide an abstraction of Scheduler that abstracts threading
and resource management in ClangdServer.
No changes to behavior are intended with an exception of changed error
messages.
This patch is preliminary work to allow a revamped threading
implementation that will move the threading code out of CppFile.

Reviewers: sammccall, bkramer, jkorous-apple

Reviewed By: sammccall

Subscribers: hokein, mgorny, hintonda, ioeric, jkorous-apple, cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D42174

llvm-svn: 323851
2018-01-31 08:51:16 +00:00