A BACKSPACE statement on a unit after a READ or WRITE with ADVANCE="NO"
must reset the position to the beginning of the record, not to the
beginning of the previous one.
Differential Revision: https://reviews.llvm.org/D125057
When the last operation on a foramtted sequential or stream file (prior
to an implied or explicit ENDFILE) is a non-advancing WRITE, ensure
that any partial record data is emitted to the file without a line
terminator. Further, when that last record is read with a non-advancing
READ, ensure that it won't raise an end-of-record condition after its
data, but instead will signal an end-of-file.
Differential Revision: https://reviews.llvm.org/D124546
When an error occurs in a formatted sequential output statement
and no output was ever emitted, don't emit a blank record.
This matches the error case behavior of other Fortran compilers.
Differential Revision: https://reviews.llvm.org/D123734
A predicate expression made ENDFILE statements significant
only for sequential files, but it's applicable to formatted
stream output as well.
Differential Revision: https://reviews.llvm.org/D123730
Correct the implementation of non-advancing I/O after some testing
to ensure that T tab edit descriptors are not allowed to back up
into positions of a record prior to where it stood at the beginning
of the I/O statement.
Differential Revision: https://reviews.llvm.org/D123709
Implements UTF-8 encoding and decoding for external units
with OPEN(ENCODING='UTF-8'). This encoding applies to default
CHARACTER values that are not 7-bit ASCII as well as to
the wide CHARACTER kinds 2 and 4. Basic testing is in place
via direct calls to the runtime I/O APIs, but serious checkout
awaits lowering support of the wide CHARACTER kinds.
Differential Revision: https://reviews.llvm.org/D122038
Some I/O error situations are current handled with fatal
runtime asserts, but should be exposed for user program
error recovery.
Differential Revision: https://reviews.llvm.org/D122049
Where possible, I added additional information to the messages to help
programmers figure out what went wrong. I also removed all uses of the word
"bad" from the messages since (to me) that implies a moral judgement rather
than a programming error. I replaced it with either "invalid" or "unsupported"
where appropriate.
Differential Revision: https://reviews.llvm.org/D121493
Rather than reading default character variables in formatted
input one byte at a time via NextInField(), skip and read
them via blocks of available buffer data. This eliminates
a bottleneck that affected reads of large character values.
(It also exposed a problem with sequential reads with RECL=
set on the OPEN statement, so that's fixed too.)
Differential Revision: https://reviews.llvm.org/D121144
A data transfer statement must have REC= in its control list
if (and only if) the unit was opened with ACCESS='DIRECT'.
The runtime wasn't catching this error, but was just silently
advancing to the next record as if the access were sequential.
Differential Revision: https://reviews.llvm.org/D120838
The runtime crashes on several fundamental I/O data transfer statement
control list errors, like list I/O on a direct-access unit, or
input from a write-only unit, &c. These errors should not be fatal
when ERR= or IOSTAT= are present.
This patch creates a new ErroneousIoStatementState class and
uses it for the state of an I/O statement that is doomed to fail
from these errors. If there is no ERR= label or IOSTAT= variable,
the error will be raised at the end of the statement. Data transfer
operations along the way will be no-op failures.
Differential Revision: https://reviews.llvm.org/D120745
Corrects the runtime implementation of I/O on files with
the access mode ACCESS='STREAM'. This is a collection
of edge-case tweaks to ensure that the distinctions between
stream and direct/sequential files, unformatted or formatted,
are respected where appropriate.
Moves NextInField() from io-stmt.h to io-stmt.cpp --
it was getting too big to keep in a header.
This patch exposed a problem with the I/O runtime
on Windows and it was reverted. This version also
fixes that problem; files are now opened on Windows
in binary mode to prevent inadvertent insertions of
carriage returns before line feeds, and those line
endings (CR+LF) are now explicitly generated.
Differential Revision: https://reviews.llvm.org/D119015
Corrects the runtime implementation of I/O on files with
the access mode ACCESS='STREAM'. This is a collection
of edge-case tweaks to ensure that the distinctions between
stream and direct/sequential files, unformatted or formatted,
are respected where appropriate.
Moves NextInField() from io-stmt.h to io-stmt.cpp --
it was getting too big to keep in a header.
Differential Revision: https://reviews.llvm.org/D118834
When RECL= is set on OPEN(), ensure that it:
1) enforces a max output record payload size
(not including header+footer or newline), and
2) causes padding of short output records only
for ACCESS='DIRECT'
The previous code was causing some false overrun errors
and applying padding to sequential/stream output files.
Differential Revision: https://reviews.llvm.org/D118630
After an ENDFILE statement, a WRITE is an error without
a prior BACKSPACE. Also fix the return value for the case
of formatted integer input with no input digits to be false
(exposed by new test).
Differential Revision: https://reviews.llvm.org/D117346
RECL= is required for direct access I/O, but is permitted
as well for sequential I/O, where it is defined by the
standard to specify a maximum record (line) length.
The standard does not say what should happen when an
sequential formatted input record appears whose length is
unequal to RECL= when it is specified.
Precedents from other compilers are unclear: one raises an error,
some honor RECL= as an effective truncation, and a few ignore the
situation. On output, all other compilers tested raised an
error when an attempt is made to emit a record longer than RECL=.
This patch treats RECL= as effective truncation on input and
as a hard limit with error on output, and also ensures that
RECL= can be set *longer* than the actual input record lengths.
Differential Revision: https://reviews.llvm.org/D115102
INQUIRE(POSITION=)'s results need to reflect the POSITION=
specifier used for the OPEN statement until the unit has been
repositioned. Preserve the POSITION= from OPEN and used it
for INQUIRE(POSITION=) until is becomes obsolete.
INQUIRE(PAD=) is implemented here in the case of an unconnected unit
with Fortran 2018 semantics; i.e., "UNDEFINED", rather than Fortran 90's
"YES"/"NO" (see 4.3.6 para 2). Apparent failures with F'90-only tests
will persist with INQUIRE(PAD=); these discrepancies don't seem to warrant
an option or environment variable.
To make the implementation of INQUIRE more closely match the language
in the standard, rename IsOpen() to IsConnected(), and use it explicitly
for the various INQUIRE specifiers.
Differential Revision: https://reviews.llvm.org/D114755
This is a near-universal language extension; external unit 0
is preconnected to the standard error output.
Differential Revision: https://reviews.llvm.org/D114298
The predefined units were not being initialized with FORM='FORMATTED',
so INQUIRE(PAD=) was failing if no I/O had already been done.
INQUIRE(POSITION=) was returning 'REWIND' on stdin/stdout (which
is somewhat defensible from the definition, and is what Intel Fortran
does), but most implementations return 'ASIS'. Change the runtime
to return 'REWIND' only for positionable external files, but 'ASIS'
for terminals, sockets, &c.
Differential Revision: https://reviews.llvm.org/D114028
1. To avoid overwriting the part of the record read in the non advancing read,
the furtherPositionInRecord field must be set to the max of the
furtherPositionInRecord and the positionInRecord at the beginning of the
IO write.
2. To allow any further read to succeed after the write, the unit
beganReadingRecord_ must be set to false when resetting the recordLength
during the write, otherwise, recordLength will not be computed in further
read and an assert is hit (at unit.cpp(398)).
The added unit test exercises both of these scenarios.
Differential Revision: https://reviews.llvm.org/D113740
Profiling a basic internal real input read benchmark shows some
hot spots in the code used to prepare input for decimal-to-binary
conversion, which is of course where the time should be spent.
The library that implements decimal to/from binary conversions has
been optimized, but not the code in the Fortran runtime that calls it,
and there are some obvious light changes worth making here.
Move some member functions from *.cpp files into the class definitions
of Descriptor and IoStatementState to enable inlining and specialization.
Make GetNextInputBytes() the new basic input API within the
runtime, replacing GetCurrentChar() -- which is rewritten in terms of
GetNextInputBytes -- so that input routines can have the
ability to acquire more than one input character at a time
and amortize overhead.
These changes speed up the time to read 1M random reals
using internal I/O from a character array from 1.29s to 0.54s
on my machine, which on par with Intel Fortran and much faster than
GNU Fortran.
Differential Revision: https://reviews.llvm.org/D113697
When processing the devious NAMELIST input
&group logarray = t t t
= 666 /
for LOGICAL::logarray(3) and INTEGER::t, the runtime library
needs to do some look-ahead on the input stream to make sure
that the last "t" on the first line is a truth value rather than
an item name -- which in this case it is. This look-ahead
was implemented in a previous patch but only worked for internal
input cases; this patch implements look-ahead capabilities for
input from an external file, too (and also adjusts repeated
list-directed input items to use this infrastructure, too).
Differential Revision: https://reviews.llvm.org/D113311
A recent change caused some variable-length sequential formatted
output statements with record positioning at the end of a FORMAT
(e.g., FORMAT('hi',10X) to append blanks at the end of the completed
record when emitting it.
Differential Revision: https://reviews.llvm.org/D112742
The runtime library was emitting unformatted record headers and
footers when an external unit had no fixed RECL=. This is wrong
for sequential files, which should have headers & footers even
with RECL. Change to omit headers & footers from unformatted
I/O only for direct access files.
Differential Revision: https://reviews.llvm.org/D112243
Count input characters corresponding to formatted edit descriptors
for READ(SIZE=); count output bytes for INQUIRE(IOLENGTH=).
The I/O APIs GetSize() and GetLength() were adjusted to return
std::size_t as function results.
Basic unit tests were added (and others fixed).
Differential Revision: https://reviews.llvm.org/D110291
When an end of record is met in non advancing IO:
- Set IOSTAT if present according to 12.11.4 (5).
- Position the file to the next record (12.11.4 (4)).
The previous code was only signaling EOR for fixed record length IO.
Reading at 12.11.4, I do not find the rational for this condition, so I
removed it.
It also does not seem the presence of padding should prevent
the EOR signaling.
The positionning to the next record was block when EOR is signaling
in FinishReadingRecord because ErrorHandler.isError() is true in this
case.
EOR in input is not an error, but I am not confident to modify
ErrorHandler.isError() to cover that. However, In FinishReadingRecord,
the code should not bail if the error is simply an end of record.
I did not check the SIZE requirements here because GetSize runtime is
not yet implemented.
Differential Revision: https://reviews.llvm.org/D109505
BACKSPACE leaves "recordLength" set, which is fine for a later READ,
but it causes a later WRITE to fail due to a misinterpretation of the
knowledge of the record length as indication of a fixed-length record
file (RECL=). Fix.
Differential Revision: https://reviews.llvm.org/D108594
A recent runtime I/O change[1] was meant to improve the handling of
input from external files missing a terminal newline on their last
records; the change was "triggered" by the wrong circumstances and
causing reads that should have pulled more data into the buffer to be
treated as EOFs. So fix that, and also don't retain input data
in the buffer once an input record has been finished unless it's
known that list-directed or NAMELIST input of a repeated input item
may need to backspace a non-positionable external unit to return
to the beginning of the repeated item.
[1] 6578893a0453384346f149479f8574dfff977ace
Differential Revision: https://reviews.llvm.org/D108164
Fix the external-io unittest under Windows.
In particular, fixes the following issues:
1. When creating a temporary file, open it with read+write permissions
using the _O_RDWR flag. _S_IREAD and _S_IWRITE are for the file
permissions of the created file.
2. _chsize returns 0 on success (just like ftruncate).
3. To set a std::optional, use its assign-operator overload instead of
getting a reference to its value and overwrite that. The latter is
invalid if the std::optional has no value, and is caught by
msvc's debug STL.
The non-GTest unittest is currently not executed under Windows because
of the added .exe extension to the output file: external-io.text.exe.
llvm-lit skips the file because .exe is not in the lists of test
suffixes (.test is). D105315 is going to change that by converting it
to a GTest-test.
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D106726
When a WRITE overwrites an endfile record, we need to forget
that there was an endfile record. When doing a BACKSPACE
after an explicit ENDFILE statement, the position afterwards
must be upon the endfile record.
Attempts to join list-directed delimited character input across
record boundaries was due to a bad reading of the standard
and has been deleted, now that the requirements are better understood.
This problem would cause a read attempt past EOF if a delimited
character input value was at the end of a record.
It turns out that delimited list-directed (and NAMELIST) character
output is required to emit contiguous doubled instances of the
delimiter character when it appears in the output value. When
fixed-size records are being emitted, as is the case with internal
output, this is not possible when the problematic character falls
on the last position of a record. No two other Fortran compilers
do the same thing in this situation so there is no good precedent
to follow.
Because it seems least wrong, with this patch we now emit one copy
of the delimiter as the last character of the current record and
another as the first character of the next record. (The
second-least-wrong alternative might be to flag a runtime error,
but that seems harsh since it's not an explicit error in the standard,
and the output may not have to be usable later as input anyway.)
Consequently, the output is not suitable for use as list-directed or
NAMELIST input.
If a later standard were to clarify this case, this behavior will of
course change as needed to conform.
Differential Revision: https://reviews.llvm.org/D106695
F18 was sigalling an end-of-file error condition when reading an
unformatted sequential input file without an ultimate newline
(or CR-LF). Other Fortran implementations can handle it, so change
the runtime to support it.
Differential Revision: https://reviews.llvm.org/D106321
Non-advancing I/O was failing; ExternalFileUnit was losing
track of what writes had been committed to the file. Fixed.
Also, support the common extension of $ and \ in a FORMAT
as being equivalent to ADVANCE=NO.
Differential Revision: https://reviews.llvm.org/D105046
With derived type description tables now available to the
runtime library, it is possible to implement the concept
of "child" I/O statements in the runtime and use them to
convert instances of derived type I/O data transfers into
calls to user-defined subroutines when they have been specified
for a type. (See Fortran 2018, subclauses 12.6.4.8 & 13.7.6).
- Support formatted, list-directed, and NAMELIST
transfers to internal parent units; support these, and unformatted
transfers, for external parent units.
- Support nested child defined derived type I/O.
- Parse DT'foo'(v-list) FORMAT data edit descriptors and passes
their strings &/or v-list values as arguments to the defined
formatted I/O routines.
- Fix problems with this feature encountered in semantics and
FORMAT valiation during development and end-to-end testing.
- Convert typeInfo::SpecialBinding from a struct to a class
after adding a member function.
Differential Revision: https://reviews.llvm.org/D104930
Add InputNamelist and OutputNamelist as I/O data transfer APIs
to be used with internal & external list-directed I/O; delete the
needless original namelist-specific Begin... calls.
Implement NAMELIST output and input; add basic tests.
Differential Revision: https://reviews.llvm.org/D101931
The code that initializes the default units 5 & 6 had
a race condition that would allow threads access to the
unit map before it had been populated.
Also add some missing calls to va_end() that will never
be called (they're in program abort situations) but might
elicit warnings if absent.
Differential Revision: https://reviews.llvm.org/D101928
What the Fortran standard calls "preconnected" external I/O units
might not be known to be connected to unformatted or formatted files
until the first I/O data transfer statement is executed.
Support this deferred determination by representing the flag as
a tri-state Boolean and adapting its points of use.
Differential Revision: https://reviews.llvm.org/D101929
Debug the input path for READ statements with END= labels;
don't emit errors when the program can handle them.
BeginReadingRecord() member functions have been made
"bool" for more convenient handling of error cases,
and some code in IoErrorHandler has been cleaned up.
Differential Revision: https://reviews.llvm.org/D100421
An OPEN statement that affects an already connected unit
without changing its external file is required to have
STATUS="OLD" or default STATUS=. The code was eliciting
spurious runtime errors in situations where an OPEN statement
pertained to an existing unit number but did not need to have
STATUS="OLD'.
Differential Revision: https://reviews.llvm.org/D100352
It turns out that unformatted fixed-size output records
do need to be padded out if short, in order to avoid a
spurious EOF crash on a short record at the end of the file.
While here in AdvanceRecord(), move the unformatted
variable-length record header/footer writing code to here
from EndIoStatement().
Differential revision: https://reviews.llvm.org/D88685
A WRITE to an unformatted sequential variable-length unit after
a BACKSPACE needs to forget its previous knowledge of the length
of the record that's about to be overwritten, and a BACKSPACE
after an ENDFILE or at the start of the file needs to be a no-op.
Differential revision: https://reviews.llvm.org/D88675
Add checking to I/O statement APIs to catch cases where the formatted
I/O data item transfer routines like OutputInteger64 are being
incorrectly used for unformatted I/O, which should use the
unformatted block or descriptor-based data item interfaces.
Differential revision: https://reviews.llvm.org/D88672
The '/' control edit descriptor causes a runtime crash for an
external formatted sequential READ because the AdvanceRecord()
member function for external units implemented only the tasks
to finish reading the current record. Split those out into
a new FinishReadingRecord() member function, call that instead
from EndIoStatement(), and change AdvanceRecord() to both
finish reading the current record and to begin reading the next
one.
Differential revision: https://reviews.llvm.org/D88607
Add SetConvert() to the OPEN statement's runtime API.
Add ByteswapOption() to the main program's runtime API.
Check a $FORT_CONVERT environment variable, too, for
a swapping specifier.
Reviewed By: sscalpone
Differential Revision: https://reviews.llvm.org/D84284
Anonymous Fortran unit files (e.g., "./fort.7") need to be created
O_RDWR so that they can be written, rewound, and read. Other
files opened with no ACTION= specifier need to set read/write
permissions based on the file, if it exists.
Reviewed By: sscalpone
Differential Revision: https://reviews.llvm.org/D84063
I/O from/to an unopened unit number needs to open &/or create
a "fort.$UNIT" file.
Fixes FCVS test fm401.f.
Reviewed By: tskeith
Differential Revision: https://reviews.llvm.org/D83809
Rework initial implementation of external I/O unit operations to
fix problems exposed in unit tests (in a later patch). Add flushing.
Reviewed By: sscalpone
Differential Revision: https://reviews.llvm.org/D83147
Peter Klausler