When an unconnected unit number is used in a BACKSPACE statement
with ERR=, IOSTAT=, &/or IOMSG= control specifiers, don't crash,
but let the program deal with the error.
Differential Revision: https://reviews.llvm.org/D127782
The test for an overflow during decimal->binary conversion was taking
place too late, causing the data not to be rescanned from the beginning.
Differential Revision: https://reviews.llvm.org/D127427
When an I/O statement is known to be in a recoverable error state,
it shouldn't cause a crash later in execution because it's not in
an expected non-erroneous processing state. Add checking for the
ErroneousIoStatementState variant on paths that might otherwise
lead to runtime crashes.
Differential Revision: https://reviews.llvm.org/D127423
Track pending "asynchronous" I/O operation IDs so that WAIT statements can
report errors about bad ID numbers.
Lowering will need to extended to call GetAsynchronousId() for a READ or
WRITE statement with ID=n.
Differential Revision: https://reviews.llvm.org/D127421
When an external I/O statement is in a recoverable error
state before any data transfers take place (for example,
an unformatted transfer with ERR=/IOSTAT=/IOMSG= attempted on
a formatted unit), ensure that the unit's mutex is still
released at the end of the statement.
Differential Revision: https://reviews.llvm.org/D127032
Whether a unit number in an inquire-by-unit statement is valid or not,
it should be the value to which the NUMBER= variable is set, not -1.
-1 should be returned to NUMBER= only for an inquire-by-file statement
when the FILE= is not connected to any unit.
Differential Revision: https://reviews.llvm.org/D126145
Now that the requirements and implementation of asynchronous I/O are
better understood, adjust their I/O runtime APIs. In particular:
1) Remove the BeginAsynchronousOutput/Input APIs; they're not needed,
since any data transfer statement might have ASYNCHRONOUS= and
(if ASYNCHRONOUS='YES') ID= control list specifiers that need to
at least be checked.
2) Add implementations for BeginWait(All) to check for the error
case of a bad unit number and nonzero ID=.
3) Rearrange and comment SetAsynchronous so that it's clear that
it can be called for READ/WRITE as well as for OPEN.
The implementation remains completely synchronous, but should be conforming.
Where opportunities make sense for true asynchronous implementations of
some big block transfers without SIZE= in the future, we'll need to add
a GetAsynchronousId API to capture ID= on a READ or WRITE; add sourceFile
and sourceLine arguments to BeginWait(All) for good error reporting;
track pending operations in unit.h; and add code to force synchronization
to non-asynchronous I/O operations.
Lowering should call SetAsynchronous when ASYNCHRONOUS= appears as
a control list specifier. It should also set ID=x variables to 0
until such time as we support asynchronous operations, if ever.
This patch only removes the removed APIs from lowering.
Differential Revision: https://reviews.llvm.org/D126143
When editing numeric input, always skip leading spaces, even if
BZ mode (or BLANK='ZERO') is in effect; otherwise, a sign character
preceded by blanks will not be recognized.
Differential Revision: https://reviews.llvm.org/D124278
Adds flang/include/flang/Common/log2-visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most use sites in
the front-end and runtime to use common::visit().
The C++ standard mandates that std::visit() have O(1) execution
time, which forces implementations to build dispatch tables.
This new common::visit() is O(log2 N) in the number of alternatives
in a variant<>, but that N tends to be small and so this change
produces a fairly significant improvement in compiler build
memory requirements, a 5-10% improvement in compiler build time,
and a small improvement in compiler execution time.
Building with -DFLANG_USE_STD_VISIT causes common::visit()
to be an alias for std::visit().
Calls to common::visit() with multiple variant arguments
are referred to std::visit(), pending further work.
This change is enabled only for GCC builds with GCC >= 9;
an earlier attempt (D122441) ran into bugs in some versions of
clang and was reverted rather than simply disabled; and it is
not well tested with MSVC. In non-GCC and older GCC builds,
common::visit() is simply an alias for std::visit().
When formatted input (not list-directed or NAMELIST) is in "BZ" mode,
either because a BZ control edit descriptor appeared in a FORMAT or
BLANK="ZERO" appeared in OPEN or READ, input editing must not skip
over blanks before or within the input field.
Differential Revision: https://reviews.llvm.org/D123725
Correct the implementation of non-advancing I/O after some testing
to ensure that T tab edit descriptors are not allowed to back up
into positions of a record prior to where it stood at the beginning
of the I/O statement.
Differential Revision: https://reviews.llvm.org/D123709
Adds flang/include/flang/Common/visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most use sites in
the front-end and runtime to use common::visit().
The C++ standard mandates that std::visit() have O(1) execution
time, which forces implementations to build dispatch tables.
This new common::visit() is O(log2 N) in the number of alternatives
in a variant<>, but that N tends to be small and so this change
produces a fairly significant improvement in compiler build
memory requirements, a 5-10% improvement in compiler build time,
and a small improvement in compiler execution time.
Building with -DFLANG_USE_STD_VISIT causes common::visit()
to be an alias for std::visit().
Calls to common::visit() with multiple variant arguments
are referred to std::visit(), pending further work.
Differential Revision: https://reviews.llvm.org/D122441
Implements UTF-8 encoding and decoding for external units
with OPEN(ENCODING='UTF-8'). This encoding applies to default
CHARACTER values that are not 7-bit ASCII as well as to
the wide CHARACTER kinds 2 and 4. Basic testing is in place
via direct calls to the runtime I/O APIs, but serious checkout
awaits lowering support of the wide CHARACTER kinds.
Differential Revision: https://reviews.llvm.org/D122038
Some refactoring and related fixes for more accurate
user program error recovery in the I/O runtime, especially
for error recovery with IOMSG= character values.
1) Move any work in an EndIoStatement() implementation
that may raise an error into a new CompleteOperation()
member function. This allows error handling APIs like
GetIoMsg() to complete a pending I/O statement and harvest
any errors that may result.
2) Move the pending error code from ErroneousIoStatementState
to a new pendingError_ data member in IoErrorHandler.
This allows IoErrorHandler::InError() to return a correct
result when there is a pending error that will be recovered
from so that I/O list data transfers don't crash in the meantime.
3) Don't create and leak a unit for a failed OPEN(NEWUNIT=n)
with error recovery, and don't modify 'n'. (Depends on
changes to API call ordering in lowering, in a separate patch;
code was added to ensure that OPEN statement control list
specifiers, e.g. SetFile(), must be passed before GetNewUnit().)
4) Fix the code that calls a form of strerror to fill an
IOMSG= variable so that it actually works for Fortran's
character type: blank fill with no null or newline termination.
Differential Revision: https://reviews.llvm.org/D122036
Rather than reading default character variables in formatted
input one byte at a time via NextInField(), skip and read
them via blocks of available buffer data. This eliminates
a bottleneck that affected reads of large character values.
(It also exposed a problem with sequential reads with RECL=
set on the OPEN statement, so that's fixed too.)
Differential Revision: https://reviews.llvm.org/D121144
The runtime crashes on several fundamental I/O data transfer statement
control list errors, like list I/O on a direct-access unit, or
input from a write-only unit, &c. These errors should not be fatal
when ERR= or IOSTAT= are present.
This patch creates a new ErroneousIoStatementState class and
uses it for the state of an I/O statement that is doomed to fail
from these errors. If there is no ERR= label or IOSTAT= variable,
the error will be raised at the end of the statement. Data transfer
operations along the way will be no-op failures.
Differential Revision: https://reviews.llvm.org/D120745
Corrects the runtime implementation of I/O on files with
the access mode ACCESS='STREAM'. This is a collection
of edge-case tweaks to ensure that the distinctions between
stream and direct/sequential files, unformatted or formatted,
are respected where appropriate.
Moves NextInField() from io-stmt.h to io-stmt.cpp --
it was getting too big to keep in a header.
This patch exposed a problem with the I/O runtime
on Windows and it was reverted. This version also
fixes that problem; files are now opened on Windows
in binary mode to prevent inadvertent insertions of
carriage returns before line feeds, and those line
endings (CR+LF) are now explicitly generated.
Differential Revision: https://reviews.llvm.org/D119015
Corrects the runtime implementation of I/O on files with
the access mode ACCESS='STREAM'. This is a collection
of edge-case tweaks to ensure that the distinctions between
stream and direct/sequential files, unformatted or formatted,
are respected where appropriate.
Moves NextInField() from io-stmt.h to io-stmt.cpp --
it was getting too big to keep in a header.
Differential Revision: https://reviews.llvm.org/D118834
NAMELIST I/O was inconsistent in its choice of which set of I/O modes
to set the "inNamelist" flag. The wrong choice was in the set of modes
that are part of the persistent state of an I/O connection; the right
place is the set of modes that are reinitialized at the beginning of
each I/O statement so that they can be modified by READ/WRITE control
list specifiers and FORMAT control edit descriptors. Fix.
Differential Revision: https://reviews.llvm.org/D118745
User-defined derived type editing in formatted I/O wasn't
working with repeat counts; e.g., "2DT(10)". The solution required
some code to be moved from GetNextDataEdit() to CueUpNextDataEdit() so
that a stack entry for a nonparenthesized repeated data edit
descriptor would work correctly -- all other data edit descriptors
are capable of dealing with repetition in their callees, so the bug
hadn't been exposed before.
Debugging this problem led to some improvements in error messages
for bad format strings, and those changes have been retained; also,
a dead member function was discovered and expunged.
Differential Revision: https://reviews.llvm.org/D117904
The unit number passed to a FLUSH statement is not required to
be a valid open unit; nothing happens (esp. not the creation of
an empty fort.n file) in this case.
Differential Revision: https://reviews.llvm.org/D117819
RECL= is required for direct access I/O, but is permitted
as well for sequential I/O, where it is defined by the
standard to specify a maximum record (line) length.
The standard does not say what should happen when an
sequential formatted input record appears whose length is
unequal to RECL= when it is specified.
Precedents from other compilers are unclear: one raises an error,
some honor RECL= as an effective truncation, and a few ignore the
situation. On output, all other compilers tested raised an
error when an attempt is made to emit a record longer than RECL=.
This patch treats RECL= as effective truncation on input and
as a hard limit with error on output, and also ensures that
RECL= can be set *longer* than the actual input record lengths.
Differential Revision: https://reviews.llvm.org/D115102
Profiling a basic internal real input read benchmark shows some
hot spots in the code used to prepare input for decimal-to-binary
conversion, which is of course where the time should be spent.
The library that implements decimal to/from binary conversions has
been optimized, but not the code in the Fortran runtime that calls it,
and there are some obvious light changes worth making here.
Move some member functions from *.cpp files into the class definitions
of Descriptor and IoStatementState to enable inlining and specialization.
Make GetNextInputBytes() the new basic input API within the
runtime, replacing GetCurrentChar() -- which is rewritten in terms of
GetNextInputBytes -- so that input routines can have the
ability to acquire more than one input character at a time
and amortize overhead.
These changes speed up the time to read 1M random reals
using internal I/O from a character array from 1.29s to 0.54s
on my machine, which on par with Intel Fortran and much faster than
GNU Fortran.
Differential Revision: https://reviews.llvm.org/D113697
When processing the devious NAMELIST input
&group logarray = t t t
= 666 /
for LOGICAL::logarray(3) and INTEGER::t, the runtime library
needs to do some look-ahead on the input stream to make sure
that the last "t" on the first line is a truth value rather than
an item name -- which in this case it is. This look-ahead
was implemented in a previous patch but only worked for internal
input cases; this patch implements look-ahead capabilities for
input from an external file, too (and also adjusts repeated
list-directed input items to use this infrastructure, too).
Differential Revision: https://reviews.llvm.org/D113311
NAMELIST array input does not need to fully define an array.
If another input item begins after at least one element,
it ends input into the array and the remaining items are
not modified.
The tricky part of supporting this feature is that it's not
always easy to determine whether the next non-blank thing in
the input is a value or the next item's name, esp. in the case
of logical data where T and F can be names. E.g.,
&group logicalArray = t f f t
= 1 /
should read three elements into "logicalArray" and then read
an integer or real variable named "t".
So the I/O runtime has to do some look-ahead to determine whether
the next thing in the input is a name followed by '=', '(', or '%'.
Since the '=' may be on a later record, possibly with intervening
NAMELIST comments, the runtime has to support a general form of
saving and restoring its current position. The infrastructure
in the I/O runtime already has to support repositioning for
list-directed repetition, even on non-positionable input sources
like terminals and sockets; this patch adds an internal RAII API
to make it easier to save a position and then do arbitrary
look-ahead.
Differential Revision: https://reviews.llvm.org/D112245
Count input characters corresponding to formatted edit descriptors
for READ(SIZE=); count output bytes for INQUIRE(IOLENGTH=).
The I/O APIs GetSize() and GetLength() were adjusted to return
std::size_t as function results.
Basic unit tests were added (and others fixed).
Differential Revision: https://reviews.llvm.org/D110291
Move the closure of the subset of flang/runtime/*.h header files that
are referenced by source files outside flang/runtime (apart from unit tests)
into a new directory (flang/include/flang/Runtime) so that relative
include paths into ../runtime need not be used.
flang/runtime/pgmath.h.inc is moved to flang/include/flang/Evaluate;
it's not used by the runtime.
Differential Revision: https://reviews.llvm.org/D109107
A recent runtime I/O change[1] was meant to improve the handling of
input from external files missing a terminal newline on their last
records; the change was "triggered" by the wrong circumstances and
causing reads that should have pulled more data into the buffer to be
treated as EOFs. So fix that, and also don't retain input data
in the buffer once an input record has been finished unless it's
known that list-directed or NAMELIST input of a repeated input item
may need to backspace a non-positionable external unit to return
to the beginning of the repeated item.
[1] 6578893a0453384346f149479f8574dfff977ace
Differential Revision: https://reviews.llvm.org/D108164
A repeated null value at the end of an input record with a count > 1
would incorrectly advance to the next record when resumed. Fix.
Improve some poor naming and code flow noticed while debugging, so
next time will be easier.
Extend a unit test to check this case.
Differential Revision: https://reviews.llvm.org/D107917
NAMELIST I/O formatting uses the runtime infrastructure for
list-directed I/O. List-directed input processing has same state
that requires reinitialization for each successive NAMELIST input
item. This patch fixes bugs with "null" items and repetition counts
on NAMELIST input items after the first in the group.
Differential Revision: https://reviews.llvm.org/D106694
Non-advancing I/O was failing; ExternalFileUnit was losing
track of what writes had been committed to the file. Fixed.
Also, support the common extension of $ and \ in a FORMAT
as being equivalent to ADVANCE=NO.
Differential Revision: https://reviews.llvm.org/D105046
With derived type description tables now available to the
runtime library, it is possible to implement the concept
of "child" I/O statements in the runtime and use them to
convert instances of derived type I/O data transfers into
calls to user-defined subroutines when they have been specified
for a type. (See Fortran 2018, subclauses 12.6.4.8 & 13.7.6).
- Support formatted, list-directed, and NAMELIST
transfers to internal parent units; support these, and unformatted
transfers, for external parent units.
- Support nested child defined derived type I/O.
- Parse DT'foo'(v-list) FORMAT data edit descriptors and passes
their strings &/or v-list values as arguments to the defined
formatted I/O routines.
- Fix problems with this feature encountered in semantics and
FORMAT valiation during development and end-to-end testing.
- Convert typeInfo::SpecialBinding from a struct to a class
after adding a member function.
Differential Revision: https://reviews.llvm.org/D104930
Add InputNamelist and OutputNamelist as I/O data transfer APIs
to be used with internal & external list-directed I/O; delete the
needless original namelist-specific Begin... calls.
Implement NAMELIST output and input; add basic tests.
Differential Revision: https://reviews.llvm.org/D101931
Debug the input path for READ statements with END= labels;
don't emit errors when the program can handle them.
BeginReadingRecord() member functions have been made
"bool" for more convenient handling of error cases,
and some code in IoErrorHandler has been cleaned up.
Differential Revision: https://reviews.llvm.org/D100421
The template `ListDirectedStatementState` was declared as a struct and then as a class.
Fix it so they match.
Differential Revision: https://reviews.llvm.org/D88711
It turns out that unformatted fixed-size output records
do need to be padded out if short, in order to avoid a
spurious EOF crash on a short record at the end of the file.
While here in AdvanceRecord(), move the unformatted
variable-length record header/footer writing code to here
from EndIoStatement().
Differential revision: https://reviews.llvm.org/D88685
Add checking to I/O statement APIs to catch cases where the formatted
I/O data item transfer routines like OutputInteger64 are being
incorrectly used for unformatted I/O, which should use the
unformatted block or descriptor-based data item interfaces.
Differential revision: https://reviews.llvm.org/D88672
The '/' control edit descriptor causes a runtime crash for an
external formatted sequential READ because the AdvanceRecord()
member function for external units implemented only the tasks
to finish reading the current record. Split those out into
a new FinishReadingRecord() member function, call that instead
from EndIoStatement(), and change AdvanceRecord() to both
finish reading the current record and to begin reading the next
one.
Differential revision: https://reviews.llvm.org/D88607
External input was detecting "end of file" conditions in
BeginExternal...Input() and BeginUnformattedInput() routines
before EnableHandlers() could have been called. Defer the
"start next record" processing to the input data item
handlers (and EndIoStatement() for when there are no data
items).
Differential Revision: https://reviews.llvm.org/D85161
Add SetConvert() to the OPEN statement's runtime API.
Add ByteswapOption() to the main program's runtime API.
Check a $FORT_CONVERT environment variable, too, for
a swapping specifier.
Reviewed By: sscalpone
Differential Revision: https://reviews.llvm.org/D84284
The runtime was requiring that STATUS='OLD' be explicitly specified
on an OPEN statement for a connected unit. There error should issue
only if a STATUS= other than 'OLD' is specified; an OPEN with no
STATUS= specifier is okay.
Reviewed By: sscalpone
Differential Revision: https://reviews.llvm.org/D84079
Anonymous Fortran unit files (e.g., "./fort.7") need to be created
O_RDWR so that they can be written, rewound, and read. Other
files opened with no ACTION= specifier need to set read/write
permissions based on the file, if it exists.
Reviewed By: sscalpone
Differential Revision: https://reviews.llvm.org/D84063
The per-I/O-statement state structures need to support missing
external I/O statements, and some bugs found in testing with
formatted input and record advancement are fixed. The effects
of these changes will not be visible until further patches to
the I/O API handlers are pushed.
Reviewed By: tskeith
Differential Revision: https://reviews.llvm.org/D83151
Scan FORMAT strings locally to avoid C++ binary runtime dependence when computing deepest parenthesis nesting
Remove a dependency on ostream from runtime
Remove remaining direct external references from runtime to C++ library binaries
Remove runtime dependences on lib/common
SetPos() and SetRec()
Instantiate templates for input
Begin input; rearrange locking, deal with CLOSE races
View()
Update error message in test to agree with compiler change
First cut at real input
More robust I/O runtime error handling
Debugging of REAL input
Add iostat.{h,cpp}
Rename runtime/numeric-* to runtime/edit-*
Move templates around, templatize integer output editing
Move LOGICAL and CHARACTER output from io-api.cpp to edit-output.cpp
Change pointer argument to reference
More list-directed input
Complex list-directed input
Use enum class Direction rather than bool for templates
Catch up with changes to master
Undo reformatting of Lower code
Use record number instead of subscripts for internal unit
Unformatted sequential backspace
Testing and debugging
Dodge bogus GCC warning
Add <cstddef> for std::size_t to fix CI build
Address review comments
Original-commit: flang-compiler/f18@50406b3496
Reviewed-on: https://github.com/flang-compiler/f18/pull/1053
Peter Klausler