llvm-project/clang/test/Lexer
Argyrios Kyrtzidis b4c83a13f6 [Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources
This is a commit with the following changes:

* Remove `ExcludedPreprocessorDirectiveSkipMapping` and related functionality

Removes `ExcludedPreprocessorDirectiveSkipMapping`; its intended benefit for fast skipping of excluded directived blocks
will be superseded by a follow-up patch in the series that will use dependency scanning lexing for the same purpose.

* Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources

Replaces the "source minimization" mechanism with a mechanism that produces lexed dependency directives tokens.

* Make the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`

This is bringing the following benefits:

    * Full access to the preprocessor state during dependency scanning. E.g. a component can see what includes were taken and where they were located in the actual sources.
    * Improved performance for dependency scanning. Measurements with a release+thin-LTO build shows ~ -11% reduction in wall time.
    * Opportunity to use dependency scanning lexing to speed-up skipping of excluded conditional blocks during normal preprocessing (as follow-up, not part of this patch).

For normal preprocessing measurements show differences are below the noise level.

Since, after this change, we don't minimize sources and pass them in place of the real sources, `DependencyScanningFilesystem` is not technically necessary, but it has valuable performance benefits for caching file `stat`s along with the results of scanning the sources. So the setup of using the `DependencyScanningFilesystem` during a dependency scan remains.

Differential Revision: https://reviews.llvm.org/D125486
Differential Revision: https://reviews.llvm.org/D125487
Differential Revision: https://reviews.llvm.org/D125488
2022-05-26 12:50:06 -07:00
..
Inputs Warn if using `elifdef` & `elifndef` in not C2x & C++2b mode 2022-05-12 09:26:44 -04:00
11-27-2007-FloatLiterals.c
SourceLocationsOverflow.c Detect source location overflow due includes 2020-01-24 23:56:47 +00:00
aligned-allocation.cpp [SystemZ][z/OS] Set aligned allocation unavailable by default for z/OS 2020-09-16 14:49:03 -04:00
asm-preproc-no-unicode.s
badstring_in_if0.c
bcpl-escaped-newline.c
bitint-constants-compat.c Implement literal suffixes for _BitInt 2022-03-14 09:24:19 -04:00
bitint-constants.c Fix _BitInt suffix width calculation 2022-03-22 10:00:05 -04:00
block_cmt_end.c PR50456: Properly handle multiple escaped newlines in a '*/'. 2021-05-24 16:21:03 -07:00
builtin_redef.c
c2x_digit_separators.c Add support for digit separators in C2x. 2021-03-12 07:21:03 -05:00
c90.c Revert "Pedantically warn about // comments in gnu89 mode" 2022-05-05 18:39:13 -04:00
case-insensitive-include-ms.c Let normalize() for posix style convert backslash to slash unconditionally. 2020-05-05 13:54:55 -04:00
case-insensitive-include-pr31836.sh Fix bugs when an included file name is typo corrected. 2020-05-08 13:33:39 -04:00
case-insensitive-include-win.c Fix bugs when an included file name is typo corrected. 2020-05-08 13:33:39 -04:00
case-insensitive-include.c Fix pr31836 on Windows too, and correctly handle repeated separators. 2020-05-01 14:17:01 -04:00
case-insensitive-system-include.c
char-escapes-delimited.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
char-escapes.c
char-literal-encoding-error.c
char-literal.cpp Make wide multi-character character literals ill-formed 2021-08-20 11:10:53 -04:00
char8_t.cpp Make -fno-char8_t disable the char8_t keyword, even in C++20. 2020-04-28 23:49:35 -07:00
comment-escape.c
conflict-marker.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
constants.c Convert diagnostics about multi-character literals from extension to warning 2020-10-06 08:47:17 -04:00
coroutines.cpp
counter.c
cross-windows-on-linux-default.cpp Move normalization of `\` in #includes from -fms-compatibility to -fms-extensions 2019-09-26 17:19:22 +00:00
cross-windows-on-linux.cpp Move normalization of `\` in #includes from -fms-compatibility to -fms-extensions 2019-09-26 17:19:22 +00:00
cxx-features.cpp [Clang][C++2b] P2242R3: Non-literal variables [...] in constexpr 2022-03-22 19:51:19 +01:00
cxx0x_keyword_as_cxx98.cpp
cxx0x_raw_string_delim_length.cpp
cxx0x_raw_string_directives.cpp
cxx0x_raw_string_unterminated.cpp
cxx1y_binary_literal.cpp
cxx1y_digit_separators.cpp
cxx1z-trigraphs.cpp [SystemZ][z/OS] enable trigraphs by default on z/OS 2020-08-13 16:02:07 -04:00
cxx2a-spaceship.cpp Add -std=c++20 flag, replace C++2a with C++20 throughout the Clang 2020-02-18 16:16:37 -08:00
cxx2a_keyword_as_cxx17.cpp Add -std=c++20 flag, replace C++2a with C++20 throughout the Clang 2020-02-18 16:16:37 -08:00
deprecate-macro.c Warn if using `elifdef` & `elifndef` in not C2x & C++2b mode 2022-05-12 09:26:44 -04:00
digraph.c
dollar-idents.c
eof-char.c
eof-conflict-marker.c
eof-file.c
eof-include.c Factor out repeated code parsing and concatenating header-names from 2019-03-19 01:51:19 +00:00
eof-number.c
eof-string.c
escape_newline.c
final-macro.c Emit final macro diagnostics in system headers 2021-11-11 13:51:26 -06:00
gnu-flags.c
half-literal.cpp [NFC] Explicitly add -std=c++14 option to tests that rely on the C++14 default 2019-02-05 12:05:53 +00:00
has_attribute_objc_boxable.m
has_extension.c
has_extension_cxx.cpp
has_feature.cu [CUDA][HIP] support __noinline__ as keyword 2022-05-10 14:32:27 -04:00
has_feature_address_sanitizer.cpp
has_feature_boxed_nsvalue_expressions.m
has_feature_c1x.c
has_feature_coverage_sanitizer.cpp [Clang] Enable __has_feature(coverage_sanitizer) 2021-05-27 18:24:21 +02:00
has_feature_cxx0x.cpp
has_feature_cxx_abi_relative_vtable.cpp [clang][feature] Add cxx_abi_relative_vtable feature 2020-10-08 10:30:54 -07:00
has_feature_cxx_unstable.cpp [Clang] Add `-funstable` flag to enable unstable and experimental features 2022-03-01 12:35:20 +00:00
has_feature_exceptions.cpp
has_feature_leak_sanitizer.cpp [clang] [Basic] Enable __has_feature(leak_sanitizer) 2019-09-22 20:55:01 +00:00
has_feature_memory_sanitizer.cpp
has_feature_memtag.cpp Reland "[MTE] Add -fsanitize=memtag* and friends." 2022-04-08 14:28:33 -07:00
has_feature_modules.m
has_feature_objc_arc.m
has_feature_rtti.cpp
has_feature_speculative_load_hardening.cpp [clang][slh] Add test for SLH feature checking macro 2020-05-15 12:23:31 -07:00
has_feature_thread_sanitizer.cpp
has_feature_type_traits.cpp
has_feature_undefined_behavior_sanitizer.cpp
has_feature_xray_instrument.cpp
header.cpp
hexfloat.cpp
keywords_test.c [MSVC] Recognize `static_assert` keyword in C and C++98 2019-02-15 19:59:45 +00:00
keywords_test.cpp [Concepts] Deprecate -fconcepts-ts, enable Concepts under -std=c++2a 2020-01-24 00:48:59 +02:00
long-long.cpp
minimize_source_to_dependency_directives_at_import_extra_tokens.m Add clang source minimizer that reduces source to directives 2019-06-03 22:59:17 +00:00
minimize_source_to_dependency_directives_at_import_missing_semi.m Add clang source minimizer that reduces source to directives 2019-06-03 22:59:17 +00:00
minimize_source_to_dependency_directives_include.c [clang-scan-deps] Minimizer: Correctly skip over double slashes in angle bracket #include 2019-08-26 23:28:05 +00:00
minimize_source_to_dependency_directives_invalid_error.c Revert "Revert "As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there."" 2019-11-26 20:28:52 -08:00
minimize_source_to_dependency_directives_invalid_macro_name.c [Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources 2022-05-26 12:50:06 -07:00
minimize_source_to_dependency_directives_pragmas.c [Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources 2022-05-26 12:50:06 -07:00
minimize_source_to_dependency_directives_utf8bom.c [clang-scan-deps] Skip UTF-8 BOM in source minimizer 2019-08-27 00:13:52 +00:00
modules-ts.cpp
ms-compatibility.c
ms-extensions.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
ms-extensions.cpp
msdos-cpm-eof.c
multi-char-constants.c Convert diagnostics about multi-character literals from extension to warning 2020-10-06 08:47:17 -04:00
multiple-include.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
newline-eof-c++98-compat.cpp
newline-eof.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
newline-nul.c
null-character-in-literal.c
numeric-literal-trash.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
objc_macros.m Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
opencl-half-literal.cl
pedantic-macro-interplay.c Implement #pragma clang final extension 2021-09-27 14:11:16 -05:00
pragma-mark.c
pragma-message.c [Diagnostics] Don't drop a statically set NoWarningAsError flag during option processing 2022-01-10 16:38:01 -08:00
pragma-message2.c
pragma-operators.cpp Add support for #pragma system_header with -fms-extensions 2021-06-23 13:26:03 +02:00
pragma-region.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
preamble.c
preamble2.c
rdar-8914293.c
rdr-6096838-2.c
rdr-6096838.c
size_t-literal.cpp [C++2b] Support size_t literals 2021-03-31 13:36:23 +00:00
string-literal-encoding.c
string-literal-errors.cpp
string_concat.cpp Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00
token-concat.c
token-concat.cpp
unicode-strings.c
unicode.c [Lex] Don't assert when decoding invalid UCNs. 2022-05-06 08:51:42 +02:00
unknown-char.c
unsafe-macro.c Implement #pragma clang restrict_expansion 2021-08-23 09:46:38 -07:00
utf-16.c
utf-16.c.txt
utf8-char-literal.cpp [clang][preprocessor] Fix unsigned-ness of utf8 char literals 2022-05-13 07:57:10 +02:00
utf8-invalid.c
warn-date-time.c
warn_binary_literals.cpp
wchar-signedness.c [SystemZ][z/OS] Set default wchar_t type for zOS 2020-09-22 08:03:03 -04:00
wchar.c Use functions with prototypes when appropriate; NFC 2022-02-09 17:16:10 -05:00