Summary:
Dependencies ensure that Protobufs are generated before all libraries
depending on the headers are **built**, not linked.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D79085
It's not clear whether clangd should really consider #import to mark a
header as self-contained or not, but this change preserves the old (and
unit-tested) behavior.
This reverts commit ad38f4b371.
As it broke building the unittests:
.../sources/llvm-project/llvm/unittests/Support/Path.cpp:334:5: error: use of undeclared identifier 'set'
set(Value);
^
1 error generated.
Summary:
This patch adds a function that is similar to `llvm::sys::path::home_directory`, but provides access to the system cache directory.
For Windows, that is %LOCALAPPDATA%, and applications should put their files under %LOCALAPPDATA%\Organization\Product\.
For *nixes, it adheres to the XDG Base Directory Specification, so it first looks at the XDG_CACHE_HOME environment variable and falls back to ~/.cache/.
Subsequently, the Clangd Index storage leverages this new API to put index files somewhere else than the users home directory.
Fixes https://github.com/clangd/clangd/issues/341
Reviewers: sammccall, chandlerc, Bigcheese
Reviewed By: sammccall
Subscribers: hiraditya, ilya-biryukov, MaskRay, jkorous, dexonsmith, arphaman, kadircet, ormris, usaxena95, cfe-commits, llvm-commits
Tags: #clang-tools-extra, #clang, #llvm
Differential Revision: https://reviews.llvm.org/D78501
Summary:
Generated Protobuf library has to be in CLANG_EXPORTS and should also be
installed appropriately. The easiest way to do that is via CMake's
add_clang_library. That unfortunately applies "one directory - one
clang_(library|tool)" policy so .proto files should be in a separate directory
and complicates the layout.
This setup works both in shared and static libs mode.
Resolves: https://github.com/clangd/clangd/issues/351
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D78885
Summary: Add a command to dexp that exports index data in chosen format (e.g. YAML).
Reviewers: sammccall
Subscribers: kbobyrev, mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang-tools-extra, #clang
Differential Revision: https://reviews.llvm.org/D77385
Summary:
This patch allows using installed gRPC to build two simple tools which
currently provide the functionality of looking up the symbol by name.
remote-index-client is a simplified version of dexp which connects to
remote-index-server passes lookup requests.
I also significantly reduced the scope of this patch to prevent large changelist
and more bugs. The next steps would be:
* Extending Protocol for deep copies of Symbol and inherit RemoteIndex from
Index to unify the interfaces
* Make remote-index-server more generic and merge the remote index client with
dexp
* Modify Clangd to allow using remote index instead of the local one for all
global index requests
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77794
Summary:
This reduces memory usage by dynamic index from more than 400MB to 32MB
when all files in clang-tools-extra/clangd/*.cpp are active in clangd.
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77732
Summary:
This ties to an LSP feature (diagnostic versioning) but really a lot
of the value is in being able to log what's happening with file versions
and queues more descriptively and clearly.
As such it's fairly invasive, for a logging patch :-\
Key decisions:
- at the LSP layer, we don't reqire the client to provide versions (LSP
makes it mandatory but we never enforced it). If not provided,
versions start at 0 and increment. DraftStore handles this.
- don't propagate magically using contexts, but rather manually:
addDocument -> ParseInputs -> (ParsedAST, Preamble, various callbacks)
Context-propagation would hide the versions from ClangdServer, which
would make producing good log messages hard
- within ClangdServer, treat versions as opaque and unordered.
std::string is a convenient type for this, and allows richer versions
for embedders. They're "mandatory" but "null" is a reasonable default.
Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D75582
Summary:
Though we don't have new changes to the index format, we have changes to
symbol collector, e.g. collect marcos, spelled references. Bump the
version to force background-index to rebuild.
Reviewers: kadircet
Reviewed By: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74127
The goal of this patch is to maximize CPU utilization on multi-socket or high core count systems, so that parallel computations such as LLD/ThinLTO can use all hardware threads in the system. Before this patch, on Windows, a maximum of 64 hardware threads could be used at most, in some cases dispatched only on one CPU socket.
== Background ==
Windows doesn't have a flat cpu_set_t like Linux. Instead, it projects hardware CPUs (or NUMA nodes) to applications through a concept of "processor groups". A "processor" is the smallest unit of execution on a CPU, that is, an hyper-thread if SMT is active; a core otherwise. There's a limit of 32-bit processors on older 32-bit versions of Windows, which later was raised to 64-processors with 64-bit versions of Windows. This limit comes from the affinity mask, which historically is represented by the sizeof(void*). Consequently, the concept of "processor groups" was introduced for dealing with systems with more than 64 hyper-threads.
By default, the Windows OS assigns only one "processor group" to each starting application, in a round-robin manner. If the application wants to use more processors, it needs to programmatically enable it, by assigning threads to other "processor groups". This also means that affinity cannot cross "processor group" boundaries; one can only specify a "preferred" group on start-up, but the application is free to allocate more groups if it wants to.
This creates a peculiar situation, where newer CPUs like the AMD EPYC 7702P (64-cores, 128-hyperthreads) are projected by the OS as two (2) "processor groups". This means that by default, an application can only use half of the cores. This situation could only get worse in the years to come, as dies with more cores will appear on the market.
== The problem ==
The heavyweight_hardware_concurrency() API was introduced so that only *one hardware thread per core* was used. Once that API returns, that original intention is lost, only the number of threads is retained. Consider a situation, on Windows, where the system has 2 CPU sockets, 18 cores each, each core having 2 hyper-threads, for a total of 72 hyper-threads. Both heavyweight_hardware_concurrency() and hardware_concurrency() currently return 36, because on Windows they are simply wrappers over std:🧵:hardware_concurrency() -- which can only return processors from the current "processor group".
== The changes in this patch ==
To solve this situation, we capture (and retain) the initial intention until the point of usage, through a new ThreadPoolStrategy class. The number of threads to use is deferred as late as possible, until the moment where the std::threads are created (ThreadPool in the case of ThinLTO).
When using hardware_concurrency(), setting ThreadCount to 0 now means to use all the possible hardware CPU (SMT) threads. Providing a ThreadCount above to the maximum number of threads will have no effect, the maximum will be used instead.
The heavyweight_hardware_concurrency() is similar to hardware_concurrency(), except that only one thread per hardware *core* will be used.
When LLVM_ENABLE_THREADS is OFF, the threading APIs will always return 1, to ensure any caller loops will be exercised at least once.
Differential Revision: https://reviews.llvm.org/D71775
Summary:
DeclarationName for cxx constructor is special, it is not an identifier.
thus the "Spelled" flag are not set for all ctor references, this patch
fixes it.
Reviewers: kbobyrev
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74125
This patch allows the index does to provide a way to distinguish
implicit references (e.g. coming from macro expansions) from the spelled
ones. The corresponding flag was added to RefKind and symbols that are
referenced without spelling their name explicitly are now marked
implicit. This allows fixing incorrect behavior when renaming a symbol
that was referenced in macro expansions would try to rename macro
invocations.
Differential Revision: D72746
Reviewed by: hokein
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Summary:
It simply shows the completed/total items on the background queue, e.g.
indexing: 233/1000
The denominator is reset to zero every time the queue goes idle.
The protocol is fairly complicated here (requires creating a remote "progress"
resource before sending updates). We implement the full protocol, but I've added
an extension allowing it to be skipped to reduce the burden on clients - in
particular the lit test takes this shortcut.
The addition of background index progress to DiagnosticConsumer seems ridiculous
at first glance, but I believe that interface is trending in the direction of
"ClangdServer callbacks" anyway. It's due for a rename, but otherwise actually
fits.
Reviewers: kadircet, usaxena95
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D73218
Summary:
Adds macro references to the dynamic index.
Tests added.
Also exposed a new API to convert path to URI in URI.h
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71406
`findName` was always used in conjuction with `spellingLocIfSpelled`.
This patch replaces patterns of the form:
spellingLocIfSpelled(findName(&ND), SM)
With a new helper function:
nameLocation(ND, SM)
And removes `spellingLocIfSpelled` and `findName`. Both are never used
anywhere else and the latter is an equivalent of `Decl::getLocation` if
we ever need it again.
Summary:
Previously, xrefs has inconsistent behavior when the reference is inside
macro body:
- AST-based xrefs (for main file) uses the expansion location;
- our index uses the spelling location;
This patch makes our index use file locations for references, which is
consistent with AST-based xrefs, and kythe as well.
After this patch, memory usage of static index on LLVM increases ~5%.
Reviewers: ilya-biryukov
Subscribers: merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70480
Summary:
This adds the references for macros to the SymbolCollector (used for static index).
Enabled if `CollectMacro` option is set.
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70489
This reverts commit 3f76260dc0.
Breaks at least these tests on Windows:
Clang :: Driver/clang-offload-bundler.c
Clang :: Driver/clang-offload-wrapper.c
Builtins are rarely if ever accessed via the Preprocessor. They are
typically found on the ASTContext, so there should be no performance
penalty to using a pointer indirection to store the builtin context.
Summary:
we have a few places using `ASTCtx.getLangOpts().IsHeaderFile` to
determine a header file, but it relies on "-x c-header" compiler flag,
if the compilation command doesn't have this flag, we will get a false
positive. We are encountering this issue in bazel build system.
To solve this problem, we infer the file from file name, actual changes will
come in follow-ups.
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70235
Summary:
We use the name from the IdentifierInfo of the Macro to compute its
SymbolID. It is better to just take the Name as a parameter to avoid
storing the IdentifierInfo whenever we need the SymbolID for the Macro.
Patch by UTKARSH SAXENA!
Reviewers: hokein
Reviewed By: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69937
Summary:
There is a regression from https://reviews.llvm.org/D68467. Unlike class
forward declarations, function ducomentation is written in the declaration in
headers, the function definition doesn't contain any documentation, cases like:
```
foo.h
// this is foo.
void foo();
foo.cc
void foo() {}
```
we should still show documentation from the foo declaration.
Reviewers: ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69961
Summary:
This is used for cross-file rename. When renaming a class, we expect to
rename all related constructors/destructors.
Reviewers: kadircet, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69338
Summary:
This fixes cases like:
foo.h
class Undocumented{}
bar.h
// break an include cycle. we should refactor this!
class Undocumented;
Where the comment doesn't describe the class.
Note that a forward decl that is *visible to the definition* will still have
its doc comment used, by SymbolCollector: Merge isn't involved here.
Reviewers: ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68467
llvm-svn: 373892
Kirill Bobyrev