Commit Graph

29 Commits

Author SHA1 Message Date
Sam McCall 9c5ebf7039 [include-fixer] Add fuzzy SymbolIndex, where identifier needn't match exactly.
Summary:
Add fuzzy SymbolIndex, where identifier needn't match exactly.

The purpose for this is global autocomplete in clangd. The query will be a
partial identifier up to the cursor, and the results will be suggestions.

It's in include-fixer because:

  - it handles SymbolInfos, actually SymbolIndex is exactly the right interface
  - it's a good harness for lit testing the fuzzy YAML index
  - (Laziness: we can't unit test clangd until reorganizing with a tool/ dir)

Other questionable choices:

  - FuzzySymbolIndex, which just refines the contract of SymbolIndex. This is
    an interface to allow extension to large monorepos (*cough*)
  - an always-true safety check that Identifier == Name is removed from
    SymbolIndexManager, as it's not true for fuzzy matching
  - exposing -db=fuzzyYaml from include-fixer is not a very useful feature, and
    a non-orthogonal ui (fuzziness vs data source). -db=fixed is similar though.

Reviewers: bkramer

Subscribers: cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D30720

llvm-svn: 297630
2017-03-13 15:55:59 +00:00
Sam McCall 573050e703 [include-fixer] Remove line number from Symbol identity
Summary:
Remove line number from Symbol identity.

For our purposes (include-fixer and clangd autocomplete), function overloads
within the same header should mostly be treated as a single combined symbol.

We may want to track individual occurrences (line number, full type info)
and aggregate this during mapreduce, but that's not done here.

Reviewers: hokein, bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30685

llvm-svn: 297371
2017-03-09 10:47:44 +00:00
Sam McCall b27dc2245f [include-fixer] Add usage count to find-all-symbols.
Summary:
Add usage count to find-all-symbols.

FindAllSymbols now finds (most!) main-file usages of the discovered symbols.
The per-TU map output has NumUses=0 or 1 (only one use per file is counted).
The reducer aggregates these to find the number of files that use a symbol.

The NumOccurrences is now set to 1 in the mapper rather than being inferred by
the reducer, for consistency.

The idea here is to use NumUses for ranking: intuitively number of files that
use a symbol is more meaningful than number of files that include the header.

Reviewers: hokein, bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30210

llvm-svn: 296446
2017-02-28 08:13:15 +00:00
Manuel Klimek a47515ec4a Improve include fixer's ranking by taking the paths into account.
Instead of just using popularity, we also take into account how similar the
path of the current file is to the path of the header.
Our first approach is to get popularity into a reasonably small scale by taking
log2 (which is roughly intuitive to how humans would bucket popularity), and
multiply that with the number of matching prefix path fragments of the included
header with the current file.
Note that currently we do not take special care for unclean paths containing
"../" or "./".

Differential Revision: https://reviews.llvm.org/D28548

llvm-svn: 291664
2017-01-11 10:32:47 +00:00
Benjamin Kramer 35d53951a4 [include-fixer] Don't skip repeated lookups in plugin mode.
In this mode not all the errors are fixed so it doesn't make sense to
ignore later ones.

llvm-svn: 288244
2016-11-30 14:37:40 +00:00
Benjamin Kramer b2a346d5e4 [include-fixer] Don't eat one token too many when replacing a block of includes.
SourceRanges are inclusive token ranges, this was trying to form an
exclusive char range.

llvm-svn: 288145
2016-11-29 15:15:26 +00:00
Benjamin Kramer 7902aa79bb [include-fixer] Don't interfere with typo correction if we found nothing.
Just let the existing typo correction machinery handle that.

llvm-svn: 288043
2016-11-28 17:16:18 +00:00
Benjamin Kramer 87eb9667bf [include-fixer plugin] Make the plugin emit proper fixits in case multiple errors are found.
The standalone tool only fixes the first one and we managed to bake that
assumption into the code :(

Also fix a crash when no header is found at all.

llvm-svn: 287544
2016-11-21 15:28:50 +00:00
Benjamin Kramer 99eaf84589 [include-fixer] Add a test for the full round trip through libclang and the plugin.
llvm-svn: 287230
2016-11-17 15:23:06 +00:00
Haojian Wu cd63701245 [include-fixer] Support finding headers for the symbol under cursor.
Summary:
* Add a `query-symbol` option to query symbol without parsing the source file.
* Update Vim & Emacs integration scripts.

Reviewers: bkramer, massberg

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D24075

llvm-svn: 280824
2016-09-07 16:34:35 +00:00
Reid Kleckner 23c2eac34d Remove most instances of REQUIRES: shell from the tools/extra tests
None of these tests actually require bash, they just have quoting bugs
when paths contain backslashes and colons. Fix them with the "%/T" lit
substitution variants.

llvm-svn: 278815
2016-08-16 16:07:46 +00:00
Haojian Wu c99f72868d [include-fixer] Support processing multiple files in one run.
Summary:
Previously, if we pass multiple files or a file pattern (e.g. /path/to/*.cc) to
include-fixer, include-fixer will apply all replacements to the first argument,
which probably causes crashes.

With this patch, include-fixer can process multiple files now.

Vim and Emacs integration are tested manually.

Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D23266

llvm-svn: 278102
2016-08-09 08:26:19 +00:00
Haojian Wu 62aee528f6 [include-fixer] Add mising qualifiers to all instances of an unidentified symbol.
Reviewers: bkramer

Subscribers: ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D22567

llvm-svn: 276280
2016-07-21 13:47:09 +00:00
Haojian Wu 9e4bd0c070 [include-fixer] A refactoring of IncludeFixerContext.
Summary:
No functional changes in this patch. It is a refactoring (pull out a
structure representing the symbol being queried).

This is a preparing step for inserting missing namespace qualifiers to all
instances of an unidentified symbol.

Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D22510

llvm-svn: 275980
2016-07-19 14:49:04 +00:00
Haojian Wu 68c34a083d [include-fixer] Implement adding missing namespace qualifiers in vim integration.
Summary:
The patch extends include-fixer's "-output-headers", and "-insert-headers"
command line options to make it dump more information (e.g. QualifiedSymbol),
so that vim-integration can add missing qualifiers.

Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D22299

llvm-svn: 275279
2016-07-13 16:43:54 +00:00
Haojian Wu 13d3048ced [include-fixer] Keep dot dot in SymbolInfo file paths.
Summary:
Currently, removing dot dot in header's path doesn't make include-fixer
minimize path correctly in some cases, for example, specify a relative search
path based on the build directory("-I../include/").

Besides, removing dot dot can break symbolic link directories. So don't
removing it for now.

Reviewers: ioeric, bkramer

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D21132

llvm-svn: 272152
2016-06-08 15:10:18 +00:00
Eric Liu 81689dab6c [include-fixer] explicitly set minimize-paths in test.
llvm-svn: 272151
2016-06-08 14:57:43 +00:00
Benjamin Kramer b53452b2b1 [include-fixer] Be smarter about inserting symbols for a prefix.
If prefix search finds something where nothing can be nested under (e.g.
a variable or macro) don't add it to the result.

This is for cases like:
header.h:
  extern int a;

file.cc:
namespace a {
  SOME_MACRO
}

We will look up a::SOME_MACRO, which doesn't have any results. Then we
look up 'a' and find something before we ever look up just 'SOME_MACRO'.
With some basic filtering we can avoid this case.

Differential Revision: http://reviews.llvm.org/D20960

llvm-svn: 271671
2016-06-03 14:07:38 +00:00
Haojian Wu 17a54e3618 [include-fixer] Use YAML format in -output-headers and -insert-header mode.
Summary:
And some improvements:
* Show better error messages on unfound symbols.
* Fix a typo.

Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D20827

llvm-svn: 271382
2016-06-01 11:43:10 +00:00
Benjamin Kramer 658d28014b [include-fixer] Rank symbols based on the number of occurrences we found while merging.
This sorts based on the popularity of the header, not the symbol. If
there are mutliple matching symbols in one header we take the maximum
popularity for that header and deduplicate. If we know nothing we sort
lexicographically based on the header path.

Differential Revision: http://reviews.llvm.org/D20814

llvm-svn: 271283
2016-05-31 14:33:28 +00:00
Benjamin Kramer 03016b85b6 [find-all-symbols] Add a test to make sure merging actually works.
llvm-svn: 271270
2016-05-31 12:12:19 +00:00
Eric Liu c893070ff1 [include-fixer] collect the number of times a symbols is found in an indexing run and use it for symbols popularity ranking.
Summary:
[include-fixer] collect the number of times a symbols is found in an
indexing run and use it for symbols popularity ranking.

Reviewers: bkramer

Subscribers: cfe-commits, hokein, djasper

Differential Revision: http://reviews.llvm.org/D20804

llvm-svn: 271268
2016-05-31 12:01:48 +00:00
Haojian Wu 11e9bd2450 [include-fixer] Create a mode in vim integration to show multiple potential headers.
Summary:
Some changes in the patch:

* Add two commandline flags in clang-include-fixer.
* Introduce a IncludeFixerContext for the queried symbol.
* Pull out CreateReplacementsForHeader.

Reviewers: bkramer

Subscribers: klimek, cfe-commits, ioeric

Differential Revision: http://reviews.llvm.org/D20621

llvm-svn: 271258
2016-05-31 09:31:51 +00:00
Benjamin Kramer 6b5160a369 [include-fixer] Don't insert #includes if a fatal error occurred.
This typically happens when the user didn't setup include paths correctly
and the fixer starts adding garbage includes. Avoid that. Disable the error
limit though, as we might hit that easily with missing includes and still
want to fix those cases.

llvm-svn: 269923
2016-05-18 13:32:38 +00:00
Haojian Wu 2d07ed4530 [include-fixer] Add lit-test for relative include path.
Reviewers: bkramer

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D20159

llvm-svn: 269177
2016-05-11 12:30:45 +00:00
Benjamin Kramer 8fd85a5788 [include-fixer] Default to YAML db, it's much more useful than the fixed dummy db.
llvm-svn: 269043
2016-05-10 11:35:47 +00:00
Benjamin Kramer b6aed5f441 [include-fixer] Autodetect yaml databases in parent directories.
This looks for find_all_symbols_db.yaml in all parent directories of the
source file (like we do for compile_commands.json) so we don't have to
pass the path manually.

Differential Revision: http://reviews.llvm.org/D20066

llvm-svn: 268920
2016-05-09 14:14:55 +00:00
Haojian Wu d8c12badad [include-fixer] Add Yaml database integration.
Reviewers: bkramer

Subscribers: cfe-commits, klimek, djasper

Differential Revision: http://reviews.llvm.org/D19648

llvm-svn: 268017
2016-04-29 09:23:38 +00:00
Benjamin Kramer f412e9029d Clean up the include fixer 'driver' a bit and make the database configurable.
Also add a test for it. The library is covered by unit tests, the driver
was not.

llvm-svn: 267718
2016-04-27 14:24:32 +00:00