Commit Graph

156 Commits

Author SHA1 Message Date
Daniel Jasper abe2a36b7e Use memoization for has()-matcher.
In TUs with large classes, a matcher like

  methodDecl(ofClass(recordDecl(has(varDecl()))))

(finding all member functions of classes with static variables)
becomes unbearably slow otherwise.

llvm-svn: 187115
2013-07-25 09:32:14 +00:00
Samuel Benzaquen 7f8a5b140a Add support for Adaptative matchers on the dynamic registry.
Summary:
Add support for Adaptative matchers on the dynamic registry.
Each adaptative matcher is created with a function template. We instantiate the function N times, one for each possible From type and apply the techniques used on argument overloaded and polymorphic matchers to add them to the registry.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1201

llvm-svn: 187044
2013-07-24 14:48:01 +00:00
Samuel Benzaquen e0b2c8e478 Add support for overloaded matchers. ie different matcher function signatures with the same name.
Summary:
Add support for overloaded matchers.
This composes with other features, like supporting polymorphic matchers.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1188

llvm-svn: 186836
2013-07-22 16:13:57 +00:00
Samuel Benzaquen b837248ad4 Add support for raw_ostream on the printing methods of Diagnostics.
Summary:
Add printToStream*(llvm::raw_ostream&) methods to Diagnostics, and reimplement everything based on streams instead of concatenating strings.
Also, fix some functions to start with lowercase to match the style guide.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1187

llvm-svn: 186715
2013-07-19 20:02:35 +00:00
Samuel Benzaquen a37bb8c01a Separate the notion of 'context' when recursing down in the parser and actual errors.
Summary:
Change how error messages are constructed and stored in Diagnostics.
Separate the notion of 'context' when recursing down in the parser and actual errors.
This will simplify adding some new features, like argument overloading and error recovery.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1168

llvm-svn: 186602
2013-07-18 19:47:59 +00:00
Samuel Benzaquen 21b3da0f93 Add TemplateArgument related matchers to the registry.
Summary:
Continue adding more matchers to the dynamic registry.
This time, we add TemplateArgument matchers.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1166

llvm-svn: 186514
2013-07-17 15:11:30 +00:00
Samuel Benzaquen 06e056c4d8 Add CXXCtorInitializer related matchers to the dynamic matcher registry.
Summary: Now that CXXCtorInitializer is already supported in ASTNodeKind, add CXXCtorInitializer matchers to the dynamic matcher registry.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1158

llvm-svn: 186508
2013-07-17 14:28:00 +00:00
Manuel Klimek 6a46149cb1 Remove unnecessary assignment.
llvm-svn: 186412
2013-07-16 13:58:44 +00:00
Manuel Klimek 55d8fb56d3 Fixes another hard to test problem with iterator invalidation.
As every match call can recursively call back into the memoized match
via a nested traversal matcher (for example:
stmt(hasAncestor(stmt(hasDescendant(stmt(hasDescendant(stmt()))))))),
and every memoization step might clear the cache, we must not store
iterators into the result cache when calling match on a submatcher.

llvm-svn: 186411
2013-07-16 13:20:30 +00:00
Samuel Benzaquen 79656e19c8 Add support for type traversal matchers.
Summary:
Fixup the type traversal macros/matchers to specify the supported types.
Make the marshallers a little more generic to support any variadic function.
Update the doc script.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D1023

llvm-svn: 186340
2013-07-15 19:25:06 +00:00
Manuel Klimek 7e3d9698fd Fix use of invalidated iterator bug in AST match finder.
Pulled out the cache clearing in the case of descendant matching, too,
for consistency, also it is not technically needed there.

FIXME: Make cache size configurable and add unit test.
llvm-svn: 185820
2013-07-08 14:16:30 +00:00
Samuel Benzaquen c6f2c9b566 Add support for polymorphic matchers. Use runtime type checking to determine the right polymorphic overload to use.
llvm-svn: 184558
2013-06-21 15:51:31 +00:00
Samuel Benzaquen 81ef929b8f Enhancements for the DynTypedMatcher system.
- Added conversion routines and checks in Matcher<T> that take a DynTypedMatcher.
- Added type information on the error messages for the marshallers.
- Allows future work on Polymorphic/overloaded matchers. We should be
  able to disambiguate at runtime and choose the appropriate overload.

llvm-svn: 184429
2013-06-20 14:28:32 +00:00
Manuel Klimek a0c025f5d2 Completely revamp node binding for AST matchers.
This is in preparation for the backwards references to bound
nodes, which will expose a lot more about how matches occur.  Main
changes:
- instead of building the tree of bound nodes, we build a "set" of bound
  nodes and explode all possible match combinations while running
  through the matchers; this will allow us to also implement matchers
  that filter down the current set of matches, like "equalsBoundNode"
- take the set of bound nodes at the start of the match into
  consideration when doing memoization; as part of that, reevaluated
  that memoization gives us benefits that are large enough (it still
  does - the effect on common match patterns is up to an order of
  magnitude)
- reset the bound nodes when a node does not match, thus never leaking
  information from partial sub-matcher matches for failing matchers

Effects:
- we can now correctly "explode" combinatorial matches, for example:
  allOf(forEachDescendant(...bind("a")),
  forEachDescendant(...bind("b"))) will now trigger matches for all
  combinations of matching "a" and "b"s.
- we now never expose bound nodes from partial matches in matchers that
  did not match in the end - this fixes a long-standing issue

FIXMEs:
- rename BoundNodesTreeBuilder to BoundNodesBuilder or
  BoundNodesSetBuilder, as we don't build a tree any more; this is out
  of scope for this change, though
- we're seeing some performance regressions (around 10%), but I expect
  some performance tuning will get that back, and it's easily worth
  the increase in expressiveness for now

llvm-svn: 184313
2013-06-19 15:42:45 +00:00
Samuel Benzaquen b5dd69f00d Reduce the number of symbols on the object file.
Summary:
Some compilers where failing with this file because the number of symbols was above 2**15.
- Replace std::list<> and std::vector<> with plain arrays.
- Change VariadicMatcherCreateCallback to be a function template, and a
  single class that wraps the instantiations.
- Remove some more unnecessary template arguments and function calls.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D948

llvm-svn: 183768
2013-06-11 18:51:07 +00:00
Samuel Benzaquen c31b3524cb Parser/Registry argument enhancements.
Summary:
 Parser/Registry argument enhancements.
  - 2 argument support.
  - unsigned values support.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D915

llvm-svn: 183231
2013-06-04 15:46:22 +00:00
Samuel Benzaquen 31edb51a4f Add support for .bind("foo") expressions on the dynamic matchers.
Summary: Add support on the parser, registry, and DynTypedMatcher for binding IDs dynamically.

Reviewers: klimek

CC: cfe-commits, revane

Differential Revision: http://llvm-reviews.chandlerc.com/D911

llvm-svn: 183144
2013-06-03 19:31:08 +00:00
Peter Collingbourne 6a55bb2307 Add an overridable MatchCallback::onEndOfTranslationUnit() function.
Differential Revision: http://llvm-reviews.chandlerc.com/D745

llvm-svn: 182798
2013-05-28 19:21:51 +00:00
Dmitri Gribenko cb63bafefc Move documentation to the constructor. Fixes a -Wdocumentation warning
llvm-svn: 182134
2013-05-17 17:50:16 +00:00
Samuel Benzaquen a76d8cd05b Test commit
llvm-svn: 181915
2013-05-15 19:49:05 +00:00
Benjamin Kramer 819a3bdcaa Link dynamic ast matchers with the ast matchers library. Unbreaks shared cmake build.
llvm-svn: 181783
2013-05-14 12:41:50 +00:00
Manuel Klimek 24db0f0afd First revision of the dynamic ASTMatcher library.
This library supports all the features of the compile-time based ASTMatcher
library, but allows the user to specify and construct the matchers at runtime.
It contains the following modules:
 - A variant type, to be used by the matcher factory.
 - A registry, where the matchers are indexed by name and have a factory method
   with a generic signature.
 - A simple matcher expression parser, that can be used to convert a matcher
   expression string into actual matchers that can be used with the AST at
   runtime.

Many features where omitted from this first revision to simplify this code
review. The main ideas are still represented in this change and it already has
support working use cases.
Things that are missing:
 - Support for polymorphic matchers. These requires supporting code in the
   registry, the marshallers and the variant type.
 - Support for numbers, char and bool arguments to the matchers. This requires
   supporting code in the parser and the variant type.
 - A command line program putting everything together and providing an already
   functional tool.

Patch by Samuel Benzaquen.

llvm-svn: 181768
2013-05-14 09:13:00 +00:00
Manuel Klimek b64d6b7fee Implements memoization for ancestor matching.
This yields a log(#ast_nodes) worst-case improvement with matchers like
stmt(unless(hasAncestor(...))).

Also made the order of visitation for ancestor matches BFS, as the most
common use cases (for example finding the closest enclosing function
definition) rely on that.

llvm-svn: 177081
2013-03-14 16:33:21 +00:00
Manuel Klimek d4be4084b4 First step towards adding a parent map to the ASTContext.
This does not yet implement the LimitNode approach discussed.

The impact of this is an O(n) in the number of nodes in the AST
reduction of complexity for certain kinds of matchers (as otherwise the
parent map gets recreated for every new MatchFinder).

See FIXMEs in the comments for the direction of future work.

llvm-svn: 176251
2013-02-28 13:21:39 +00:00
Manuel Klimek 191c093af1 Re-design the convenience interfaces on MatchFinder.
First, this implements a match() method on MatchFinder; this allows us
to get rid of the findAll implementation, as findAll is really a special
case of recursive matchers on match.

Instead of findAll, provide a convenience function match() that lets
users iterate easily over the results instead of needing to implement
callbacks.

llvm-svn: 174172
2013-02-01 13:41:35 +00:00
Dmitri Gribenko f857950d39 Remove useless 'llvm::' qualifier from names like StringRef and others that are
brought into 'clang' namespace by clang/Basic/LLVM.h

llvm-svn: 172323
2013-01-12 19:30:44 +00:00
Manuel Klimek c844a46e77 Implements multiple parents in the parent map.
Previously we would match the last visited parent, which in the
case of template instantiations was the last instantiated template.

llvm-svn: 169508
2012-12-06 14:42:48 +00:00
Manuel Klimek 5472a52c20 Fixes crash in isDerivedFrom for recursive templates.
llvm-svn: 169262
2012-12-04 13:40:29 +00:00
Manuel Klimek 9d0412334e Allow matchers to access the ASTContext.
Patch by Edwin Vane.

llvm-svn: 169000
2012-11-30 13:45:19 +00:00
Daniel Jasper 94a56856d2 Fix partial-match-bind-behavior with forEachDescendant() matchers.
The problem is that a partial match of an (explicit or implicit) allOf matcher
binds results, i.e.

recordDecl(decl().bind("x"), hasName("A"))

can very well bind a record that is not named "A". With this fix, the common
cases of stumbling over this bug are fixed by the BoundNodesMap overwriting the
results of a partial match. An error can still be created with a weird
combination of anyOf and allOf (see inactive test). We need to decide whether
this is worth fixing, as the fix will have performance impact.

Review: http://llvm-reviews.chandlerc.com/D124
llvm-svn: 168177
2012-11-16 18:39:22 +00:00
Benjamin Kramer d6e8a7964d Make libASTMatchers link its dependencies. libRewriteFrontend depends on libRewriteCore.
LLVM_USED_LIBS does nothing here.

llvm-svn: 168172
2012-11-16 17:30:58 +00:00
Daniel Jasper 0f9f019ff8 Do not use data recursion in ASTMatchFinder.
The matchers rely on the complete AST being traversed as shown by the new test cases.

llvm-svn: 168022
2012-11-15 03:29:05 +00:00
Daniel Jasper a05bb6bd70 Fix AST-matcher descendant visiting for Types, TypeLocs and NestedNamespecifierLocs.
The RecursiveASTVisitor assumes that any given Traverse-method can be called with a NULL-node. So the subclass needs to handle these appropriately.

llvm-svn: 167850
2012-11-13 17:14:11 +00:00
Daniel Jasper 33806cdefc Fix binding of nodes in case of forEach..() matchers.
When recursively visiting the generated matches, the aggregated bindings need
to be copied during the recursion. Otherwise, we they might not be properly
overwritten (which is shown by the test), or there might be bound nodes present
that were bound on a different matching branch.

Review: http://llvm-reviews.chandlerc.com/D112
llvm-svn: 167695
2012-11-11 22:14:55 +00:00
Manuel Klimek bd0e2b7111 Insert interception point onStartOfTranslationUnit.
Often users of the ASTMatchers want to add tasks that are done once per
translation unit, for example, cleaning up caches. Combined with the
interception point for the end of source file one can add to the factory
creation, this covers the cases we've seen users need.

llvm-svn: 167271
2012-11-02 01:31:03 +00:00
Daniel Jasper 6fc3433b15 Implement descendant matchers for NestedNamespecifiers
This implements has(), hasDescendant(), forEach() and
forEachDescendant() for NestedNameSpecifier and NestedNameSpecifierLoc
matchers.

Review: http://llvm-reviews.chandlerc.com/D86
llvm-svn: 167017
2012-10-30 15:42:00 +00:00
Daniel Jasper d29d5fa3f2 Implement has(), hasDescendant(), forEach() and forEachDescendant() for
Types, QualTypes and TypeLocs.

Review: http://llvm-reviews.chandlerc.com/D83
llvm-svn: 166917
2012-10-29 10:14:44 +00:00
Manuel Klimek c268745011 Adds the possibility to run ASTMatchFinder over arbitrary AST nodes.
llvm-svn: 166567
2012-10-24 14:47:44 +00:00
Daniel Jasper 632aea92a5 Implement hasParent()-matcher.
llvm-svn: 166421
2012-10-22 16:26:51 +00:00
Daniel Jasper 516b02e548 First version of matchers for Types and TypeLocs.
Review: http://llvm-reviews.chandlerc.com/D47
llvm-svn: 166094
2012-10-17 08:52:59 +00:00
Daniel Jasper 83dafaf3db Fix isDerivedFrom matcher.
Without this patch, the isDerivedFrom matcher asserts in the
"assert(ClassDecl != NULL);" in the new test, as a
DependentTemplateSpecilizationType is not a sub-type of
TemplateSpecializationType and also does not offer getAsCXXRecordDecl().

I am not sure why this did not cause problems before. It is now (after
the changed implementation of isDerivedFrom) easier to write a matcher
that actually gets into this branch of the code.

llvm-svn: 164127
2012-09-18 14:17:42 +00:00
Daniel Jasper a6bc1f6d35 Create initial support for matching and binding NestedNameSpecifier(Loc)s.
Review: http://llvm-reviews.chandlerc.com/D39
llvm-svn: 163794
2012-09-13 13:11:25 +00:00
Daniel Jasper f49d1e0070 Change the behavior of the isDerivedFrom-matcher to not match on the
class itself. This caused some confusion (intuitively, a class is not
derived from itself) and makes it hard to write certain matchers, e.g.
"match and bind any pair of base and subclass".

The original behavior can be achieved with a new isA-matcher.  Similar
to all other matchers, this matcher has the same behavior and name as
the corresponding AST-entity - in this case the isa<>() function.

llvm-svn: 163385
2012-09-07 12:48:17 +00:00
Manuel Klimek 3ca12c5b54 Implements hasAncestor.
Implements the hasAncestor matcher. This builds
on the previous patch that introduced DynTypedNode to build up
a parent map for an additional degree of freedom in the AST traversal.

The map is only built once we hit an hasAncestor matcher, in order
to not slow down matching for cases where this is not needed.

We could implement some speed-ups for special cases, like building up
the parent map as we go and only building up the full map if we break
out of the already visited part of the tree, but that is probably
not going to be worth it, and would make the code significantly more
complex.

Major TODOs are:
- implement hasParent
- implement type traversal
- implement memoization in hasAncestor

llvm-svn: 163382
2012-09-07 09:26:10 +00:00
Manuel Klimek eb958ded72 Introduces DynTypedMatcher as a new concept that replaces the UntypedBaseMatcher and TypedMatcher.
Due to DynTypedNode the basic dynamically typed matcher interface can now be simplified.

Also switches the traversal interfaces to use DynTypedNode;
this is in preperation for the hasAncestor implementation, and
also allows us to need fewer changes when we want to add new
nodes to traverse, thus making the code a little more decoupled.

Main design concerns: I went back towards the original design
of getNodeAs to return a pointer, and switched DynTypedNode::get
to always return a pointer (in case of value types like QualType
the pointer points into the storage of DynTypedNode, thus allowing
us to treat all the nodes the same from the point of view of a
user of the DynTypedNodes.

Adding the QualType implementation for DynTypedNode was needed
for the recursive traversal interface changes.

llvm-svn: 163212
2012-09-05 12:12:07 +00:00
Manuel Klimek fdf98763ac Fixes a bug for binding memoized match results.
Intorduces an abstraction for DynTypedNode which makes
is impossible to create in ways that introduced the bug;
also hides the implementation details of the template
magic away from the user and prepares the code for adding
QualType and TypeLoc bindings, as well as using DynTypedNode
instead of overloads for child and ancestor matching.

getNodeAs<T> was changed towards a non-pointer type, as
we'll want QualType and TypeLoc nodes to be returned
by value (the alternative would be to create new storage
which is prohibitively costly if we want to use it for
child / ancestor matching).

DynTypedNode is moved into a new header ASTTypeTraits.h,
as it is completely independent of the rest of the matcher
infrastructure - if the need comes up, we can move it to
a more common place.

The interface for users before the introduction of the
common storage change remains the same, minus the introduced
bug, for which a regression test was added.

llvm-svn: 162936
2012-08-30 19:41:06 +00:00
Manuel Klimek 021d56f948 Modifes BoundNodes to store void* and allow casting them
into the correct types when pulling them out in the result
callback in a type safe way.

This is also the base change for multiple things that will
allow handling things more generally and thus supporting more
of the AST, especially handling Type nodes.

Patch contributed by Michael Diamond.

llvm-svn: 162804
2012-08-28 23:26:39 +00:00
Daniel Jasper 3cb72b476d Fix for ASTMatchFinder to visit a functions parameter declarations.
llvm-svn: 160947
2012-07-30 05:03:25 +00:00
Daniel Jasper 1975e03494 Move RefactoringCallbacks to Tooling to avoid dependency from
ASTMatchers (lower level abstraction) to Tooling (higher level
abstraction).

llvm-svn: 160351
2012-07-17 08:03:01 +00:00
Daniel Jasper 2b3c7d414b Make the isDerivedFrom matcher more generic.
It now accepts an arbitrary inner matcher but is fully backwards
compatible.

llvm-svn: 160348
2012-07-17 07:39:27 +00:00
Daniel Jasper 7e22282b68 Add refactoring callbacks to make common kinds of refactorings easy.
llvm-svn: 160255
2012-07-16 09:18:17 +00:00
Daniel Jasper 1dad183b38 Add more matchers and do cleanups.
Reviewers: klimek

    Differential Revision: http://ec2-50-18-127-156.us-west-1.compute.amazonaws.com/D2

llvm-svn: 160013
2012-07-10 20:20:19 +00:00
NAKAMURA Takumi d7ba69de90 [CMake] Get rid of unconditional dependency to ClangDiagnosticCommon. Only clangBasic and clangASTMatchers need it.
llvm-svn: 159931
2012-07-09 14:12:20 +00:00
NAKAMURA Takumi 17a2ff2494 ASTMatchers/CMakeLists.txt: Add dependencies to generated headers, or "make clean; make ASTMatchers" would fail.
llvm-svn: 159906
2012-07-07 23:13:30 +00:00
Manuel Klimek cb93f785de Build-fix: Remove non-existent directories from Makefiles.
llvm-svn: 159807
2012-07-06 06:00:30 +00:00
Manuel Klimek 04616e4776 Adds the AST Matcher library, which provides a in-C++ DSL to express
matches on interesting parts of the AST, and callback mechanisms to
act on them.

llvm-svn: 159805
2012-07-06 05:48:52 +00:00