This diff addresses the issue of the ever increasing memory usage of clangd. The key to understand what happens is to use `malloc_stats()`: malloc arenas keep getting bigger, although the actual memory used does not. It seems some operations while bulding the indices (both dynamic and background) create this problem. Specifically, 'FileSymbols::update' and 'FileSymbols::buildIndex' seem especially affected.
This diff adds a call to `malloc_trim()` periodically in
ClangdLSPServer.
Fixes: https://github.com/clangd/clangd/issues/251
Fixes: https://github.com/clangd/clangd/issues/115
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D93452
When querying the CDB, we stat the underlying file to check it hasn't changed.
We don't do this every time, but only if we didn't check within 5 seconds.
This behavior only exists for compile_commands.json and compile_flags.txt.
The CDB plugin system doesn't expose enough information to handle others.
Slight behavior change: we now only look for `build/compile_commands.json`
rather than trying every CDB strategy under `build` subdirectories.
Differential Revision: https://reviews.llvm.org/D92663
This is a mass-market version of the "dump AST" tweak we have behind
-hidden-features.
I think in this friendlier form it'll be useful for people outside clang
developers, which would justify making it a real feature.
It could be useful as a step towards lightweight clang-AST tooling in clangd
itself (like matcher-based search).
Advantages over the tweak:
- simplified information makes it more accessible, likely somewhat useful
without learning too much clang internals
- can be shown in a tree view
- structured information gives some options for presentation (e.g.
icon + two text colors + tooltip in vscode)
- clickable nodes jump to the corresponding code
Disadvantages:
- a bunch of code to handle different node types
- likely missing some important info vs dump-ast due to brevity/oversight
- may end up chasing/maintaining support for the long tail of nodes
Demo with VSCode support: https://imgur.com/a/6gKfyIV
Differential Revision: https://reviews.llvm.org/D89571
Performs a detailed profiling of clangd lsp server and conveys the
result to the client as a json object. It is of the form:
{
"_self": 0,
"_total": 8,
"child1": {
"_self": 4,
"_total": 4,
}
"child2": {
"_self": 2,
"_total": 4,
"child_deep": {
"_self": 2,
"_total": 2,
}
}
}
Differential Revision: https://reviews.llvm.org/D89277
If the NewName is provided, prepareRename would perform a name
validation.
The motivation is to allow our internal embeder implement the customized
"canRenameInto" functionality on top of prepareRename.
Differential Revision: https://reviews.llvm.org/D88881
several changes:
- return a structure result in rename API;
- prepareRename now returns more information (main-file occurrences);
- remove the duplicated detecting-touch-identifier code in prepareRename (which is implemented in rename API);
Differential Revision: https://reviews.llvm.org/D88634
Intent was a nice idea but it ends up being a bit awkward/heavyweight
without adding much.
In particular, it makes it hard to implement `CodeActionParams.only` properly
(there's an inheritance hierarchy for kinds).
Differential Revision: https://reviews.llvm.org/D88427
In preparation for making moving TweakFilter from ClangdServer::Options to
a ClangdLSPServer option, and letting it vary per-request.
(In order to implement CodeActionParams.only)
Also a general overdue cleanup.
Differential Revision: https://reviews.llvm.org/D88470
Translating between JSON objects and C++ strutctures is common.
From experience in clangd, fromJSON/ObjectMapper work well and save a lot of
code, but aren't adopted elsewhere at least partly due to total lack of error
reporting beyond "ok"/"bad".
The recently-added error model should be rich enough for most applications.
It requires tracking the path within the root object and reporting local
errors at appropriate places.
To do this, we exploit the fact that the call graph of recursive
parse functions mirror the structure of the JSON itself.
The current path is represented as a linked list of segments, each of which is
on the stack as a parameter. Concretely, fromJSON now looks like:
bool fromJSON(const Value&, T&, Path);
Beyond the signature change, this is reasonably unobtrusive: building
the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON.
However the root caller of fromJSON must now create a Root object to
store the errors, which is a little clunky.
I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not
general enough to be the primary interface I think (at least, not usable in
clangd).
All existing users (mostly just clangd) are updated in this patch,
making this change backwards-compatible is a bit hairy.
Differential Revision: https://reviews.llvm.org/D88103
Summary:
This is considerably terser than the makeStringError and friends, and
avoids verbosity cliffs that discourage adding log information.
It follows the syntax used in log/elog/vlog/dlog that have been successful.
The main caveats are:
- it's strictly out-of-place in logger.h, though kind of fits thematically and
in implementation
- it claims the "error" identifier, which seems a bit too opinionated
to put higher up in llvm
I've updated some users of StringError mostly at random - there are lots
more mechanical changes but I'd like to get this reviewed before making
them all.
Reviewers: kbobyrev, hokein
Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D83419
Summary:
This patch introduces basic textDocument/foldingRange support. It relies on
textDocument/documentSymbols to collect all symbols and uses takes ranges
to create folds.
The next steps for textDocument/foldingRange support would be:
* Implementing FoldingRangeClientCapabilities and respecting respect client
preferences
* Specifying folding range kind
* Migrating from DocumentSymbol implementation to custom RecursiveASTVisitor flow that will allow more flexibility
* Supporting more folding range types: comments, PP conditional regions, includes and other code regions (e.g. public/private/protected sections of classes, control flow statement bodies)
Tested: (Neo)Vim (coc-clangd) and VSCode.
Related issue: https://github.com/clangd/clangd/issues/310
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: nridge, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82436
Summary:
They don't need ASTs or anything, so they should still run immediately.
These were sync for historical reasons (they predate clangd having a pervasive
threading model). This worked ok as they were "cheap".
Aside for consistency, there are a couple of reasons to make them async:
- they do IO (finding .clang-format) so aren't trivially cheap
- having TUScheduler involved in running these tasks means we can use it as
an injection point for configuration.
(TUScheduler::run will need to learn about which file is being operated on,
but that's an easy change).
- adding the config system adds more potential IO, too
Reviewers: kbobyrev
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82642
Summary:
Clangd uses FSProvider to get threadsafe views into file systems. This
patch changes naming to make that more explicit.
Depends on D81920
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D81998
Summary:
This fixes a reported bug: if clang and libc++ are installed under
/usr/lib/llvm-11/... but there'- a symlink /usr/bin/clang++-11, then a
compile_commands.json with "/usr/bin/clang++-11 -stdlib=libc++" would previously
look for libc++ under /usr/include instead of /usr/lib/llvm-11/include.
The PATH change makes this work if the compiler is just "clang++-11" too.
As this is now doing IO potentially on every getCompileCommand(), we cache
the results for each distinct driver.
While here:
- Added a Memoize helper for this as multithreaded caching is a bit noisy.
- Used this helper to simplify QueryDriverDatabase and reduce blocking there.
(This makes use of the fact that llvm::Regex is now threadsafe)
Reviewers: kadircet
Subscribers: jyknight, ormris, ilya-biryukov, MaskRay, jkorous, arphaman, jfb, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D75414
Summary:
Structure is parsed from the raw comment using the existing heuristics used
for hover.
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D79157
Summary:
Caveats:
- only works when the header is changed in the editor and the editor provides
the notification
- we revalidate preambles for all open files (stat all their headers) rather
than taking advantage of the fact that we know which file changed.
This is much simpler!
Fixes https://github.com/clangd/clangd/issues/107
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77847
Summary:
TUStatus api had a single thread in mind. This introudces a section
action to represent state of the preamble thread. In the file status extension,
we keep old behavior almost the same. We only prepend current task with a
`parsing includes` if preamble thread is working. We omit the idle thread in the
output unless both threads are idle.
Reviewers: sammccall
Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76304
Summary:
This returns incremental highlights as a set of edits against the
previous highlights.
Server-side, we compute the full set of highlights, this just saves
wire-format size.
For now, the diff used is trivial: everything from the first change to
the last change is sent as a single edit.
The wire format is grungy - the replacement offset/length refer to
positions in the encoded array instead of the logical list of tokens.
We use token-oriented structs and translating to LSP forms when serializing.
This departs from LSP (but is consistent with semanticTokens today).
Tested in VSCode insiders (with a patched client to enable experimental
features).
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77225
Summary: This allows the standard mechanism to gracefully displace the old one.
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77206
Summary:
One change: because there's no way to signal failure individually for
each cursor, we now "succeed" with an empty range with no parent if a
cursor doesn't point at anything.
Reviewers: usaxena95
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76741
Summary:
This ties to an LSP feature (diagnostic versioning) but really a lot
of the value is in being able to log what's happening with file versions
and queues more descriptively and clearly.
As such it's fairly invasive, for a logging patch :-\
Key decisions:
- at the LSP layer, we don't reqire the client to provide versions (LSP
makes it mandatory but we never enforced it). If not provided,
versions start at 0 and increment. DraftStore handles this.
- don't propagate magically using contexts, but rather manually:
addDocument -> ParseInputs -> (ParsedAST, Preamble, various callbacks)
Context-propagation would hide the versions from ClangdServer, which
would make producing good log messages hard
- within ClangdServer, treat versions as opaque and unordered.
std::string is a convenient type for this, and allows richer versions
for embedders. They're "mandatory" but "null" is a reasonable default.
Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D75582
Summary:
Make it more convinient for the clients to select completion items by
providing a set of default characters (punctuation).
Related issue: https://github.com/clangd/clangd/issues/284
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74606
Summary:
- This option forces a preamble rebuild to handle the odd case
of a missing header file being added
Reviewers: sammccall
Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73916
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Quentin Chateau