This patch adds a new abstract class for frontend actions:
`PrescanAndSemaDebugAction`. It's almost identical to
`PrescanAndSemaAction`, but in the presence of semantic errors it does
not skip the corresponding `ExecuteAction` specialisation. Instead, it
runs it as if there were no semantic errors. This class is for developer
actions only (i.e. front-end driver options).
The new behaviour does not affect the return code from `flang-new -fc1`
when the input file is semantically incorrect. The return code is
inferred from the number of driver diagnostics generated in
`CompilerInstance::ExecuteAction` and this patch does not change that.
More specifically, the semantic errors are still reported and hence the
driver is able to correctly report that the compilation has failed (with
a non-zero return code).
This new base class is meant for debug actions only and
`DebugDumpAllAction` is updated to demonstrate the new behaviour. With
this change, `flang-new -fc1 -fdebug-dump-all` dumps the parse tree and
symbols for all input files, regardless of whether any semantic errors
were found.
This patch addresses https://bugs.llvm.org/show_bug.cgi?id=52097.
Differential Revision: https://reviews.llvm.org/D111308
When nonexistent linker inputs are passed to the driver, the linker
now errors out, instead of the compiler. If the linker does not run,
clang now emits a "warning: linker input unused" instead of an error
for nonexistent files.
The motivation for this change is that I noticed that
`clang-cl /winsysroot sysroot main.cc ole32.lib` emitted a
"ole32.lib not found" error, even though the linker finds it just fine when
I run `clang-cl /winsysroot sysroot main.cc /link ole32.lib`.
The same problem occurs if running `clang-cl main.cc ole32.lib` in a
non-MSVC shell.
The problem is that DiagnoseInputExistence() only looked for libs in %LIB%,
but MSVCToolChain uses much more involved techniques.
For this particular problem, we could make DiagnoseInputExistence() ask
the toolchain to see if it can find a .lib file, but in general the
driver can't know what the linker will do to find files, so it shouldn't
try. For example, if we implement PR24616, lld-link will look in the
registry to determine a good default for %LIB% if it isn't set.
This is less or a problem for the gcc driver, since .a paths there are
either passed via -l flags (which honor -L), or via a qualified path
(that doesn't honor -L) -- but for example ld.lld's --chroot flag
can also trigger this problem. Without this patch,
`clang -fuse-ld=lld -Wl,--chroot,some/dir /file.o` will complain that
`/file.o` doesn't exist, even though
`clang -fuse-ld=lld -Wl,--chroot,some/dir -Wl,/file.o` succeeds just fine.
This implements rnk's suggestion on the old bug PR27234.
Differential Revision: https://reviews.llvm.org/D109624
This patch cleans-up the file generation code in Flang's frontend
driver. It improves the layering between
`CompilerInstance::CreateDefaultOutputFile`,
`CompilerInstance::CreateOutputFile` and their various clients.
* Rename `CreateOutputFile` as `CreateOutputFileImpl` and make it
private. This method is an implementation detail.
* Instead of passing an `std::error_code` out parameter into
`CreateOutputFileImpl`, have it return Expected<>. This is a bit shorter
and idiomatic LLVM.
* Make `CreateDefaultOutputFile` (which calls `CreateOutputFileImpl`)
issue an error when file creation fails. The error code from
`CreateOutputFileImpl` is used to generate a meaningful diagnostic
message.
* Remove error reporting from `PrintPreprocessedAction::ExecuteAction`.
This is only for cases when output file generation fails. This is
handled in `CreateDefaultOutputFile` instead (see the previous point).
* Inline `AddOutputFile` into its only caller,
`CreateDefaultOutputFile`.
* Switch from `lvm::buffer_ostream` to `llvm::buffer_unique_ostream>`
for non-seekable output streams. This simplifies the logic in the driver
and was introduced for this very reason in [1]
* Moke sure that the diagnostics from the prescanner when running `-E`
(`PrintPreprocessedAction::ExecuteAction`) are printed before the actual
output is generated.
* Update comments, add test.
NOTE: This patch relands [2]. As suggested by Michael Kruse in the
post-commit/post-revert review, I've added the following:
```
config.errc_messages = "@LLVM_LIT_ERRC_MESSAGES@"
```
in Flang's `lit.site.cfg.py.in`. This way, `%errc_ENOENT` in
output-paths.f90 gets the correct value on Windows as well as on Linux.
[1] https://reviews.llvm.org/D93260
[2] fd21d1e198
Reviewed By: ashermancinelli
Differential Revision: https://reviews.llvm.org/D108390
This reverts commit fd21d1e198.
The test added in this patch [1] is failing on Windows and causing the
Windows BuildBot [2] to fail. I don't see any obvious way to fix this,
so reverting in order to investigate.
[1] llvm-project/flang/test/Driver/output-paths.f90
[2] https://lab.llvm.org/buildbot/#/builders/172/builds/2077
This patch refactors the file generation API in Flang's frontend driver.
It improves the layering between `CreateDefaultOutputFile`,
`CreateOutputFile` (`CompilerInstance` methods) and their various
clients.
List of changes:
* Rename `CreateOutputFile` as `CreateOutputFileImpl` and make it
private. This method is an implementation detail.
* Instead of passing an `std::error_code` out parameter into
`CreateOutputFileImpl`, have it return Expected<>. This is a bit shorter
and more idiomatic LLVM.
* Make `CreateDefaultOutputFile` (which calls `CreateOutputFileImpl`)
issue an error when file creation fails. The error code from
`CreateOutputFileImpl` is used to generate a meaningful diagnostic
message.
* Remove error reporting from `PrintPreprocessedAction::ExecuteAction`.
This is only for cases when output file generation fails. This is
handled in `CreateDefaultOutputFile` instead (see the previous point).
* Inline `AddOutputFile` into its only caller,
`CreateDefaultOutputFile`.
* Switch from `lvm::buffer_ostream` to `llvm::buffer_unique_ostream>`
for non-seekable output streams. This simplifies the logic in the driver
and was introduced for this very reason in [1]
* Moke sure that the diagnostics from the prescanner when running `-E`
(`PrintPreprocessedAction::ExecuteAction`) are printed before the actual
output is generated.
* Update comments, add test.
[1] https://reviews.llvm.org/D93260
Differential Revision: https://reviews.llvm.org/D108390
Currently, `printHelp` behaves differently for options that:
* do not define `HelpText` (such options _are not printed_), and
* define its `HelpText` as `HelpText<"">` (such options _are printed_).
In practice, both approaches lead to no help text and `printHelp` should
treat them consistently. This patch addresses that by making
`printHelpt` check the length of the help text to be printed.
All affected tests have been updated accordingly. The option definitions
for llvm-cvtres have been updated with a short description or "Not
implemented" for options that are ignored by the tool.
Differential Revision: https://reviews.llvm.org/D107557
Replacing Hello World example Plugin with one that counts and prints the names of
functions and subroutines.
This involves changing the `PluginParseTreeAction` Plugin base class to
inherit from `PrescanAndSemaAction` class to get access to the Parse Tree
so that the Plugin can walk it.
Additionally, there are tests of this new Plugin to check it prints the correct
things in different circumstances.
Depends on: D106137
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D107089
Introducing a plugin API and a simple HelloWorld Plugin example.
This patch adds the `-load` and `-plugin` flags to frontend driver and
the code around using custom frontend actions from within a plugin
shared library object.
It also adds to the Driver-help test to check the help option with the
updated driver flags.
Additionally, the patch creates a plugin-example test to check the
HelloWorld plugin example runs correctly. As part of this, a new CMake
flag (`FLANG_BUILD_EXAMPLES`) is added to allow the example to be built
and for the test to run.
This Plugin API has only been tested on Linux.
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D106137
This patch removes `f18`, a.k.a. the old driver. It is being replaced
with the new driver, `flang-new`, which has reached feature parity with
`f18` a while ago. This was discussed in [1] and also in [2].
With this change, `FLANG_BUILD_NEW_DRIVER` is no longer needed and is
also deleted. This means that we are making the dependency on Clang permanent
(i.e. it cannot be disabled with a CMake flag).
LIT set-up is updated accordingly. All references to `f18` or `f18.cpp`
are either updated or removed.
The `F18_FC` variable from the `flang` bash script is replaced with
`FLANG_FC`. The former is still supported for backwards compatibility.
[1] https://lists.llvm.org/pipermail/flang-dev/2021-June/000742.html
[2] https://reviews.llvm.org/D103177
Differential Revision: https://reviews.llvm.org/D105811
For boolean options, e.g. `-fxor-operator`/`-fno-xor-operator`, we ought
to be using TableGen multi-classes. This way, we only have to write one
definition to have both forms auto-generated. This patch refactors all
of Flang's boolean options to use two new multi-classes:
`OptInFC1FFOption` and `OptOutFC1FFOption`. These multi-classes are
based on `OptInFFOption`/`OptOutFFOption`, respectively. I've also
simplified the processing of the updated options in
CompilerInvocation.cpp.
With the new approach, "empty" help text (i.e. no `HelpText`) is now
replaced with an empty string (i.e. HelpText<"">). When running
flang-new --help, that's considered as non-empty help messages, which is
then printed (that's controlled by `printHelp` from
llvm/lib/Option/OptTable.cpp). This means that with this patch,
flang-new --help will start printing e.g. -fno-backslash, even though
there is no actual help text to print for this option (apart from the
empty string ""). Tests are updated accordingly.
Note that with this patch, both `-fxor-operator` and `-fno-xor-operator`
(and other boolean options refactored here) remain available in
`flang-new` and `flang-new -fc1`. In this respect, nothing changes. In a
forthcoming patch, I will refine this so that `flang-new -fc1` only
accepts `-ffoo` (`OptInFC1FFOption`) or `-fno-foo` (`OptOutCC1FFOption`).
For clarity, `OptInFFOption`/`OptOutFFOption` are renamed as
`OptInCC1FFOption`/`OptOutCC1FFOption`, respectively. Otherwise, this is
an NFC from Clang's perspective.
Differential Revision: https://reviews.llvm.org/D105881
Rename the current -E option to "-E -Xflang -fno-reformat".
Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.
The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.
Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).
An effort has been made to preserve original alphabetic character case
and source indentation.
Add -P and -fno-reformat to the new drivers.
Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.
Differential Revision: https://reviews.llvm.org/D106727
This patch adds a new option for the new Flang driver:
`-fno-analyzed-objects-for-unparse`. The semantics are similar to
`-funparse-typed-exprs-to-f18-fc` from `f18`. For consistency, the
latter is replaced with `-fno-analyzed-objects-for-unparse`.
The new option controls the behaviour of the unparser (i.e. the action
corresponding to `-fdebug-unparse`). The default behaviour is to use the
analyzed objects when unparsing. The new flag can be used to turn this
off, so that the original parse-tree objects are used. The analyzed
objects are generated during the semantic checks [1].
This patch also updates the semantics of
`-fno-analyzed-objects-for-unparse`/`-funparse-typed-exprs-to-f18-fc`
in `f18`, so that this flag is always taken into account when `Unparse`
is used (this way the semantics in `f18` and `flang-new` are identical).
The added test file is based on example from Peter Steinfeld.
[1]
https://github.com/llvm/llvm-project/blob/main/flang/docs/Semantics.md
Differential Revision: https://reviews.llvm.org/D103612
The test added in https://reviews.llvm.org/D104305 will only work with
the new driver and should be marked as such.
Sending this without a review as it's fairly straightforward and fixes
test failures for developers that don't want to build the new driver.
The new option will run the semantic checks and then dump the parse tree
and all the symbols. This is equivalent to running the driver twice,
once with `-fdebug-dump-parse-tree` and then with
the `-fdebug-dump-symbols` action flag.
Currently we wouldn't be able to achieve the same by simply running:
```
flang-new -fc1 -fdebug-dump-parse-tree -fdebug-dump-symbols <input-file>
```
That's because the new driver will only run one frontend action per
invocation (both of the flags used here are action flags). Diverging
from this design would lead to costly compromises and it's best avoided.
We may want to consider re-designing our debugging actions (and action
options) in the future so that there's more code re-use. For now, I'm
focusing on making sure that we support all the major cases requested by
our users.
Differential Revision: https://reviews.llvm.org/D104305
Allow the lit test suite to run under Windows. This encompasses the following changes:
* Define `lit_tools_dir` for flang's test configuration
* Replace `(<command> || true)` idiom with `not <command>`
* Add `REQUIRES: shell` on tests that invoke a shell script
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D89368
Adding the `-init-only` option and corresponding frontend action to
generate a diagnostic.
`-init-only` vs `-test-io`:
`-init-only` ignores the input (it never calls the prescanner)
`-test-io` is similar to `-init-only`, but does read and print the input
without calling the prescanner.
This patch also adds a Driver test to check this action.
Reviewed By: awarzynski, AMDChirag
Differential Revision: https://reviews.llvm.org/D102849
With this patch, the following invocation of the frontend driver will
return an error:
```
flang-new -fc1 input-file.f90 -o
```
Similar logic applies to other options that require arguments.
Similar checks are already available in the compiler driver, flang-new
(that's implemented in clangDriver).
Differential Revision: https://reviews.llvm.org/D103554
This option is supported in `f18`, but not yet available in `flang-new`.
It is required in order to call `flang-new` from the `flang` bash
script.
Differential Revision: https://reviews.llvm.org/D103613
`%f18` was originally introduced to represent the old Flang driver,
`f18`. With the introduction of the new driver, `flang-new`, we have
been switching to `%flang` (compiler driver) and `%flang_fc1` (frontend
driver) as more generic alternatives.
As most tests have been portend to use the new LIT variables instead of
`%f18`, this is good time to remove it from lit.cfg.py. There's only one
test left that requires the old driver to run. It's updated with:
```
! REQUIRES: old-flang-driver
```
This way we preserve its semantics while reducing the number of
variables in LIT configuration.
Differential Revision: https://reviews.llvm.org/D101281
This patch adds `-fget-definition` to `flang-new`. The semantics of this
option are identical in both drivers. The error message in the
"throwaway" driver is updated so that it matches the one from
`flang-new` (which is auto-generated and cannot be changed easily).
Tests are updated accordingly. A dedicated test for error handling was
added: get-definition.f90 (for the sake of simplicity,
getdefinition01.f90 no longer tests for errors).
The `ParseFrontendArgs` function is updated so that it can return
errors. This change is required in order to report invalid values
following `-fget-definition`.
The actual implementation of `GetDefinitionAction::ExecuteAction()` was
extracted from f18.cpp (i.e. the bit that deals with
`-fget-definition`).
Depends on: https://reviews.llvm.org/D100556
Differential Revision: https://reviews.llvm.org/D100558
`%flang-new` was introduced in the early days of the new driver to make
a clear distinction between the tests for the current and the new
driver. We have since introduced `%flang` (compiler driver) and
`%flang_fc1` (frontend driver) as the long term solution. This has allowed
us to share tests between `flang-new` and `f18`. This patch replaces
all uses of `%flang-new` with `%flang` and `%flang_fc1`.
Some tests are reformatted so that all tests look uniform and are easier
to follow. Where possible, `! REQUIRES: new-flang-driver` is deleted so
that more tests can be shared with `f18`. To facilitate this,
`f{no-}implicit-none` are introduced in `f18` with semantics identical
to `flang-new`.
Two tests are deleted rather than updated:
* flang/test/Frontend/print-preprocess-C-file.f90
* flang/test/Frontend/print-preprocessed-file.f90
Instead, there is plenty of preprocessor tests in
flang/test/Preprocessing/.
Differential Revision: https://reviews.llvm.org/D100174
This patch adds two debugging options in the new Flang driver
(flang-new):
*fdebug-unparse-no-sema
*fdebug-dump-parse-tree-no-sema
Each of these options combines two options from the "throwaway" driver
(left: f18, right: flang-new):
* `-fdebug-uparse -fdebug-no-semantics` --> `-fdebug-unparse-no-sema`
* `-fdebug-dump-parse-tree -fdebug-no-semantics` -->
`-fdebug-dump-parse-tree-no-sema`
There are no plans to implement `-fdebug-no-semantics` in the new
driver. Such option would be too powerful. Also, it would only make
sense when combined with specific frontend actions (`-fdebug-unparse`
and `-fdebug-dump-parse-tree`). Instead, this patch adds 2 specialised
options listed above. Each of these is implemented through a dedicated
FrontendAction (also added).
The new frontend actions are implemented in terms of a new abstract base
action: `PrescanAndSemaAction`. This new base class was required so that
we can have finer control over what steps within the frontend are
executed:
* `PrescanAction`: run the _prescanner_
* `PrescanAndSemaAction`: run the _prescanner_ and the _parser_ (new
in this patch)
* `PrescanAndSemaAction`: run the _prescanner_, _parser_ and run the
_semantic checks_
This patch introduces `PrescanAndParseAction::BeginSourceFileAction`.
Apart from the semantic checks removed at the end, it is similar to
`PrescanAndSemaAction::BeginSourceFileAction`.
Differential Revision: https://reviews.llvm.org/D99645
The -fdebug-dump-provenance flag is meant to be used with
needProvenanceRangeToCharBlockMappings set to true. This way, extra
mapping is generated that allows e.g. IDEs to retrieve symbol's scope
(offset into cooked character stream) based on symbol's source code
location. This patch makes sure that this option is set when using
-fdebug-dump-provenance.
With this patch, the implementation of -fdebug-dump-provenance in
`flang-new -fc1` becomes consistent with `f18`. The corresponding LIT
test is updated so that it can be shared with `f18`. I refined it a bit
so that:
* it becomes a frontend-only test
* it's stricter about the expected output
Differential Revision: https://reviews.llvm.org/D98847
This patch adds support for the `-cpp` and `-nocpp` flags. The
implemented semantics match f18 (i.e. the "throwaway" driver), but are
different to gfortran. In Flang the preprocessor is always run. Instead,
`-cpp/-nocpp` are used to control whether predefined and command-line
preprocessor macro definitions are enabled or not. In practice this is
sufficient to model gfortran`s `-cpp/-nocpp`.
In the absence of `-cpp/-nocpp`, the driver will use the extension of
the input file to decide whether to include the standard macro
predefinitions. gfortran's documentation [1] was used to decide which
file extension to use for this.
The logic mentioned above was added in FrontendAction::BeginSourceFile.
That's relatively late in the driver set-up, but this roughly where the
name of the input file becomes available. The logic for deciding between
fixed and free form works in a similar way and was also moved to
FrontendAction::BeginSourceFile for consistency (and to reduce
code-duplication).
The `-cpp/-nocpp` flags are respected also when the input is read from
stdin. This is different to:
* gfortran (behaves as if `-cpp` was used)
* f18 (behaves as if `-nocpp` was used)
Starting with this patch, file extensions are significant and some test
files had to be renamed to reflect that. Where possible, preprocessor
tests were updated so that they can be shared between `f18` and
`flang-new`. This was implemented on top of adding new test for
`-cpp/-nocpp`.
[1] https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D99292
Adds support for `-fget-symbols-sources` in the new Flang driver. All
relevant tests are updated to use the new driver when
`FLANG_BUILD_NEW_DRIVER` is set.
`RUN` lines in tests are updated so `-fsyntax-only`
comes before `-fget-symbols-sources`. That's because:
* both `-fsyntax-only` and `-fget-symbols-sources` are
action flags, and
* the new driver, flang-new, will only consider the right-most
action flag.
In other words, this change is needed so that the tests work with both
`f18` (requires both flags) and `flang-new` (only considers the last
action flag).
Differential Revision: https://reviews.llvm.org/D98191
This patch adds `-fdebug-dump-parsing-log` in the new driver. This option is
semantically identical to `-fdebug-instrumented-parse` in `f18` (the
former is added as an alias in `f18`).
As dumping the parsing log makes only sense for instrumented parses, we
set Fortran::parser::Options::instrumentedParse to `True` when
`-fdebug-dump-parsing-log` is used. This is consistent with `f18`.
To facilitate tweaking the configuration of the frontend based on the
action being requested, `setUpFrontendBasedOnAction` is introduced in
CompilerInvocation.cpp.
Differential Revision: https://reviews.llvm.org/D97457
Until now we've been maintaining 2 test directories for Flang's drivers:
* test/Driver for `f18` (the current driver)
* test/Flang-Driver for `flang-new` (the new driver)
As we have started sharing tests between the drivers, this separation is
no longer required. This patch merges the two test directories. As
suggested in the review, moving forward we'll avoid having tests
specifically for the old driver.
A few notable changes:
* Driver/version-test.f90 and Driver/no-files.f90 are deleted. The
versions for the new driver are more robust, but tricky to share.
* Driver/write-module.f90 is deleted in favour of
Flang-Driver/write-module.f90 (see https://reviews.llvm.org/D97197
for more context)
Differential Revision: https://reviews.llvm.org/D98257
Add -J to the f18 driver for compatibility with gfortran.
Add -module-dir for compatibility with the new flang driver.
They both set the output directory for .mod files and add the
directory to the search list. -module still only does the former.
Clean up the new driver test to match.
Differential Revision: https://reviews.llvm.org/D97164
Fixed some version information in flang/f18:
- fixed the behavior of the -v switch: this flag enables verbosity with used with arguments, but just displays the version when used alone (related to this bug: https://bugs.llvm.org/show_bug.cgi?id=46017)
- added __FLANG, __FLANG_MAJOR__, __FLANG_MINOR__ and __FLANG_PATCHLEVEL__ (similar to their __F18* counterparts) for compatibility purpose
Reviewed By: AlexisPerry, richard.barton.arm, tskeith
Differential Revision: https://reviews.llvm.org/D84334
Fixed some version information in flang/f18:
- fixed the behavior of the -v switch: this flag enables verbosity with used with arguments, but just displays the version when used alone (related to this bug: https://bugs.llvm.org/show_bug.cgi?id=46017)
- added __FLANG, __FLANG_MAJOR__, __FLANG_MINOR__ and __FLANG_PATCHLEVEL__ (similar to their __F18* counterparts) for compatibility purpose
Reviewed By: sscalpone, AlexisPerry, richard.barton.arm, tskeith
Differential Revision: https://reviews.llvm.org/D84334
Add a usage string and a defaults section that clarifies:
* If no input files are given, f18 reads from stdin
* If no input files are given, f18 dumps the parse tree.
* The default behaviour is to exec F18_FC.
* The fefault F18_FC setting is 'gfortran'
Adds a simple regression test which tests the top and tail of the help
screen and the exit status.
Depends on D84855
Differential Revision: https://reviews.llvm.org/D84856
When flang is invoked with no files it waits for input on stdin. Make it
print a message saying this to prevent the user being surprised.
Differential Revision: https://reviews.llvm.org/D84855
We have re-classified a subset of the regression tests as unit tests and
now we are porting the remaining ones.
Test discovery and running is now performed by lit rather than ctest.
The tests continue to use their original scripts with minor
modifications. Most of the changes were mechanical and so scripted.
A few changes were made by hand. Details
Manual:
* modfile09-*.f90 tests depend on being run together as some tests have
dependencies on modules created by other tests. This will need
separating out when porting away from test_modfile.sh, but for now,
added modfile09-*.f90 to the Inputs directory and added a single
tests modfile09.f90 to hold the run line.
* getdefinition03-a.f90 includes a non-test file getdefinition03-b.f90.
Manually edited the former to find the latter in Inputs so as to add
only one test.
* Same pattern for getsymbols03-{a,b}.f90
Auto:
* Remaining tests have a lit RUN line added to them based on the type
of test they are.
* Failing tests also have an XFAIL line added to them.
* Generic tests have their pre-existing RUN lines replaced with the
word "EXEC" to avoid conflict with the added lit RUN line.
Original-commit: flang-compiler/f18@63ec0af9f4
Reviewed-on: https://github.com/flang-compiler/f18/pull/1027
Tree-same-pre-rewrite: false
Andrzej Warzynski