llvm-project/clang/test/Lexer
Argyrios Kyrtzidis b4c83a13f6 [Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources
This is a commit with the following changes:

* Remove `ExcludedPreprocessorDirectiveSkipMapping` and related functionality

Removes `ExcludedPreprocessorDirectiveSkipMapping`; its intended benefit for fast skipping of excluded directived blocks
will be superseded by a follow-up patch in the series that will use dependency scanning lexing for the same purpose.

* Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources

Replaces the "source minimization" mechanism with a mechanism that produces lexed dependency directives tokens.

* Make the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`

This is bringing the following benefits:

    * Full access to the preprocessor state during dependency scanning. E.g. a component can see what includes were taken and where they were located in the actual sources.
    * Improved performance for dependency scanning. Measurements with a release+thin-LTO build shows ~ -11% reduction in wall time.
    * Opportunity to use dependency scanning lexing to speed-up skipping of excluded conditional blocks during normal preprocessing (as follow-up, not part of this patch).

For normal preprocessing measurements show differences are below the noise level.

Since, after this change, we don't minimize sources and pass them in place of the real sources, `DependencyScanningFilesystem` is not technically necessary, but it has valuable performance benefits for caching file `stat`s along with the results of scanning the sources. So the setup of using the `DependencyScanningFilesystem` during a dependency scan remains.

Differential Revision: https://reviews.llvm.org/D125486
Differential Revision: https://reviews.llvm.org/D125487
Differential Revision: https://reviews.llvm.org/D125488
2022-05-26 12:50:06 -07:00
..
Inputs Warn if using `elifdef` & `elifndef` in not C2x & C++2b mode 2022-05-12 09:26:44 -04:00
11-27-2007-FloatLiterals.c
SourceLocationsOverflow.c
aligned-allocation.cpp
asm-preproc-no-unicode.s
badstring_in_if0.c
bcpl-escaped-newline.c
bitint-constants-compat.c Implement literal suffixes for _BitInt 2022-03-14 09:24:19 -04:00
bitint-constants.c Fix _BitInt suffix width calculation 2022-03-22 10:00:05 -04:00
block_cmt_end.c PR50456: Properly handle multiple escaped newlines in a '*/'. 2021-05-24 16:21:03 -07:00
builtin_redef.c
c2x_digit_separators.c Add support for digit separators in C2x. 2021-03-12 07:21:03 -05:00
c90.c Revert "Pedantically warn about // comments in gnu89 mode" 2022-05-05 18:39:13 -04:00
case-insensitive-include-ms.c
case-insensitive-include-pr31836.sh
case-insensitive-include-win.c
case-insensitive-include.c
case-insensitive-system-include.c
char-escapes-delimited.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
char-escapes.c
char-literal-encoding-error.c
char-literal.cpp Make wide multi-character character literals ill-formed 2021-08-20 11:10:53 -04:00
char8_t.cpp
comment-escape.c
conflict-marker.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
constants.c Convert diagnostics about multi-character literals from extension to warning 2020-10-06 08:47:17 -04:00
coroutines.cpp
counter.c
cross-windows-on-linux-default.cpp
cross-windows-on-linux.cpp
cxx-features.cpp [Clang][C++2b] P2242R3: Non-literal variables [...] in constexpr 2022-03-22 19:51:19 +01:00
cxx0x_keyword_as_cxx98.cpp
cxx0x_raw_string_delim_length.cpp
cxx0x_raw_string_directives.cpp
cxx0x_raw_string_unterminated.cpp
cxx1y_binary_literal.cpp
cxx1y_digit_separators.cpp
cxx1z-trigraphs.cpp
cxx2a-spaceship.cpp
cxx2a_keyword_as_cxx17.cpp
deprecate-macro.c Warn if using `elifdef` & `elifndef` in not C2x & C++2b mode 2022-05-12 09:26:44 -04:00
digraph.c
dollar-idents.c
eof-char.c
eof-conflict-marker.c
eof-file.c
eof-include.c
eof-number.c
eof-string.c
escape_newline.c
final-macro.c Emit final macro diagnostics in system headers 2021-11-11 13:51:26 -06:00
gnu-flags.c
half-literal.cpp
has_attribute_objc_boxable.m
has_extension.c
has_extension_cxx.cpp
has_feature.cu [CUDA][HIP] support __noinline__ as keyword 2022-05-10 14:32:27 -04:00
has_feature_address_sanitizer.cpp
has_feature_boxed_nsvalue_expressions.m
has_feature_c1x.c
has_feature_coverage_sanitizer.cpp [Clang] Enable __has_feature(coverage_sanitizer) 2021-05-27 18:24:21 +02:00
has_feature_cxx0x.cpp
has_feature_cxx_abi_relative_vtable.cpp [clang][feature] Add cxx_abi_relative_vtable feature 2020-10-08 10:30:54 -07:00
has_feature_cxx_unstable.cpp [Clang] Add `-funstable` flag to enable unstable and experimental features 2022-03-01 12:35:20 +00:00
has_feature_exceptions.cpp
has_feature_leak_sanitizer.cpp
has_feature_memory_sanitizer.cpp
has_feature_memtag.cpp Reland "[MTE] Add -fsanitize=memtag* and friends." 2022-04-08 14:28:33 -07:00
has_feature_modules.m
has_feature_objc_arc.m
has_feature_rtti.cpp
has_feature_speculative_load_hardening.cpp
has_feature_thread_sanitizer.cpp
has_feature_type_traits.cpp
has_feature_undefined_behavior_sanitizer.cpp
has_feature_xray_instrument.cpp
header.cpp
hexfloat.cpp
keywords_test.c
keywords_test.cpp
long-long.cpp
minimize_source_to_dependency_directives_at_import_extra_tokens.m
minimize_source_to_dependency_directives_at_import_missing_semi.m
minimize_source_to_dependency_directives_include.c
minimize_source_to_dependency_directives_invalid_error.c
minimize_source_to_dependency_directives_invalid_macro_name.c [Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources 2022-05-26 12:50:06 -07:00
minimize_source_to_dependency_directives_pragmas.c [Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources 2022-05-26 12:50:06 -07:00
minimize_source_to_dependency_directives_utf8bom.c
modules-ts.cpp
ms-compatibility.c
ms-extensions.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
ms-extensions.cpp
msdos-cpm-eof.c
multi-char-constants.c Convert diagnostics about multi-character literals from extension to warning 2020-10-06 08:47:17 -04:00
multiple-include.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
newline-eof-c++98-compat.cpp
newline-eof.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
newline-nul.c
null-character-in-literal.c
numeric-literal-trash.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
objc_macros.m Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
opencl-half-literal.cl
pedantic-macro-interplay.c Implement #pragma clang final extension 2021-09-27 14:11:16 -05:00
pragma-mark.c
pragma-message.c [Diagnostics] Don't drop a statically set NoWarningAsError flag during option processing 2022-01-10 16:38:01 -08:00
pragma-message2.c
pragma-operators.cpp Add support for #pragma system_header with -fms-extensions 2021-06-23 13:26:03 +02:00
pragma-region.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
preamble.c
preamble2.c
rdar-8914293.c
rdr-6096838-2.c
rdr-6096838.c
size_t-literal.cpp [C++2b] Support size_t literals 2021-03-31 13:36:23 +00:00
string-literal-encoding.c
string-literal-errors.cpp
string_concat.cpp Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
token-concat.c
token-concat.cpp
unicode-strings.c
unicode.c [Lex] Don't assert when decoding invalid UCNs. 2022-05-06 08:51:42 +02:00
unknown-char.c
unsafe-macro.c Implement #pragma clang restrict_expansion 2021-08-23 09:46:38 -07:00
utf-16.c
utf-16.c.txt
utf8-char-literal.cpp [clang][preprocessor] Fix unsigned-ness of utf8 char literals 2022-05-13 07:57:10 +02:00
utf8-invalid.c
warn-date-time.c
warn_binary_literals.cpp
wchar-signedness.c [SystemZ][z/OS] Set default wchar_t type for zOS 2020-09-22 08:03:03 -04:00
wchar.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00