Commit Graph

2 Commits

Author SHA1 Message Date
Logan Smith 08eb614e30 [NFC][testing] Return underlying strings directly instead of OS.str()
This avoids an unnecessary copy required by 'return OS.str()', allowing
instead for NRVO or implicit move. The .str() call (which flushes the
stream) is no longer required since 65b13610a5,
which made raw_string_ostream unbuffered by default.

Differential Revision: https://reviews.llvm.org/D115374
2021-12-09 16:05:46 -08:00
Balazs Benics 6e3071007b [analyzer] Introduce MacroExpansionContext to libAnalysis
Introduce `MacroExpansionContext` to track what and how macros in a translation
unit expand. This is the first element of the patch-stack in this direction.

The main goal is to substitute the current macro expansion generator in the
`PlistsDiagnostics`, but all the other `DiagnosticsConsumer` could benefit from
this.

`getExpandedText` and `getOriginalText` are the primary functions of this class.
The former can provide you the text that was the result of the macro expansion
chain starting from a `SourceLocation`.
While the latter will tell you **what text** was in the original source code
replaced by the macro expansion chain from that location.

Here is an example:

  void bar();
  #define retArg(x) x
  #define retArgUnclosed retArg(bar()
  #define BB CC
  #define applyInt BB(int)
  #define CC(x) retArgUnclosed

  void unbalancedMacros() {
    applyInt  );
  //^~~~~~~~~~^ is the substituted range
  // Original text is "applyInt  )"
  // Expanded text is "bar()"
  }

  #define expandArgUnclosedCommaExpr(x) (x, bar(), 1
  #define f expandArgUnclosedCommaExpr

  void unbalancedMacros2() {
    int x =  f(f(1))  ));  // Look at the parenthesis!
  //         ^~~~~~^ is the substituted range
  // Original text is "f(f(1))"
  // Expanded text is "((1,bar(),1,bar(),1"
  }

Might worth investigating how to provide a reusable component, which could be
used for example by a standalone tool eg. expanding all macros to their
definitions.

I borrowed the main idea from the `PrintPreprocessedOutput.cpp` Frontend
component, providing a `PPCallbacks` instance hooking the preprocessor events.
I'm using that for calculating the source range where tokens will be expanded
to. I'm also using the `Preprocessor`'s `OnToken` callback, via the
`Preprocessor::setTokenWatcher` to reconstruct the expanded text.

Unfortunately, I concatenate the token's string representation without any
whitespaces except if the token is an identifier when I emit an extra space
to produce valid code for `int var` token sequences.
This could be improved later if needed.

Patch-stack:
  1) D93222 (this one) Introduces the MacroExpansionContext class and unittests

  2) D93223 Create MacroExpansionContext member in AnalysisConsumer and pass
     down to the diagnostics consumers

  3) D93224 Use the MacroExpansionContext for macro expansions in plists
     It replaces the 'old' macro expansion mechanism.

  4) D94673 API for CTU macro expansions
     You should be able to get a `MacroExpansionContext` for each imported TU.
     Right now it will just return `llvm::None` as this is not implemented yet.

  5) FIXME: Implement macro expansion tracking for imported TUs as well.

It would also relieve us from bugs like:
  - [fixed] D86135
  - [confirmed] The `__VA_ARGS__` and other macro nitty-gritty, such as how to
    stringify macro parameters, where to put or swallow commas, etc. are not
    handled correctly.
  - [confirmed] Unbalanced parenthesis are not well handled - resulting in
    incorrect expansions or even crashes.
  - [confirmed][crashing] https://bugs.llvm.org/show_bug.cgi?id=48358

Reviewed By: martong, Szelethus

Differential Revision: https://reviews.llvm.org/D93222
2021-02-22 11:11:57 +01:00