llvm-project/flang/lib/Parser/parsing.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

127 lines
4.4 KiB
C++
Raw Normal View History

//===-- lib/Parser/parsing.cpp --------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "flang/Parser/parsing.h"
#include "preprocessor.h"
#include "prescan.h"
#include "type-parsers.h"
#include "flang/Parser/message.h"
#include "flang/Parser/provenance.h"
#include "flang/Parser/source.h"
#include "llvm/Support/raw_ostream.h"
namespace Fortran::parser {
Parsing::Parsing(AllSources &s) : cooked_{s} {}
Parsing::~Parsing() {}
[flang] Improve module file reading and writing Fix problems with writing a mod file while another compilation is reading or writing. Write to a temp and then rename it: - compute the new contents of the .mod file - if it already exists, check if it is already correct - if not, write new contents to a temp file - rename the temp to the final destination `mkstemps()` seems to be the best way to create the temp file. It returns a file descriptor, so change the rest of the mod file writing to use POSIX open/read/write/close. This seems to set errno more reliably too. There is some extra work around creating the temp to make it have the same directory and suffix as the final file (so that if one gets left behind by a crash, "rm *.mod" still cleans it up). `mkstemps()` creates file with 0600 permissions so try to change it to what it would have been if we just wrote the file. Change module file reading to only read the file once; we used to read it to verify the checksum and then again to parse it. Instead, change `Parsing` so that we can get the file contents after `Prescan()` and use that to verify the checksum. Also, it has a mechanism for searching directories for files, so make use of that instead of duplicating that functionality in `ModFileReader`. This requires some changes to how errors are returned so they can be reported in the right place. Original-commit: flang-compiler/f18@d0d54971a5547820f8ed0271eb9818557bc8bc82 Reviewed-on: https://github.com/flang-compiler/f18/pull/758 Tree-same-pre-rewrite: false
2019-09-24 08:10:58 +08:00
const SourceFile *Parsing::Prescan(const std::string &path, Options options) {
options_ = options;
[flang] Improve module file reading and writing Fix problems with writing a mod file while another compilation is reading or writing. Write to a temp and then rename it: - compute the new contents of the .mod file - if it already exists, check if it is already correct - if not, write new contents to a temp file - rename the temp to the final destination `mkstemps()` seems to be the best way to create the temp file. It returns a file descriptor, so change the rest of the mod file writing to use POSIX open/read/write/close. This seems to set errno more reliably too. There is some extra work around creating the temp to make it have the same directory and suffix as the final file (so that if one gets left behind by a crash, "rm *.mod" still cleans it up). `mkstemps()` creates file with 0600 permissions so try to change it to what it would have been if we just wrote the file. Change module file reading to only read the file once; we used to read it to verify the checksum and then again to parse it. Instead, change `Parsing` so that we can get the file contents after `Prescan()` and use that to verify the checksum. Also, it has a mechanism for searching directories for files, so make use of that instead of duplicating that functionality in `ModFileReader`. This requires some changes to how errors are returned so they can be reported in the right place. Original-commit: flang-compiler/f18@d0d54971a5547820f8ed0271eb9818557bc8bc82 Reviewed-on: https://github.com/flang-compiler/f18/pull/758 Tree-same-pre-rewrite: false
2019-09-24 08:10:58 +08:00
AllSources &allSources{cooked_.allSources()};
if (options.isModuleFile) {
for (const auto &path : options.searchDirectories) {
allSources.PushSearchPathDirectory(path);
}
}
std::string buf;
llvm::raw_string_ostream fileError{buf};
const SourceFile *sourceFile;
if (path == "-") {
sourceFile = allSources.ReadStandardInput(fileError);
} else {
sourceFile = allSources.Open(path, fileError);
}
[flang] Improve module file reading and writing Fix problems with writing a mod file while another compilation is reading or writing. Write to a temp and then rename it: - compute the new contents of the .mod file - if it already exists, check if it is already correct - if not, write new contents to a temp file - rename the temp to the final destination `mkstemps()` seems to be the best way to create the temp file. It returns a file descriptor, so change the rest of the mod file writing to use POSIX open/read/write/close. This seems to set errno more reliably too. There is some extra work around creating the temp to make it have the same directory and suffix as the final file (so that if one gets left behind by a crash, "rm *.mod" still cleans it up). `mkstemps()` creates file with 0600 permissions so try to change it to what it would have been if we just wrote the file. Change module file reading to only read the file once; we used to read it to verify the checksum and then again to parse it. Instead, change `Parsing` so that we can get the file contents after `Prescan()` and use that to verify the checksum. Also, it has a mechanism for searching directories for files, so make use of that instead of duplicating that functionality in `ModFileReader`. This requires some changes to how errors are returned so they can be reported in the right place. Original-commit: flang-compiler/f18@d0d54971a5547820f8ed0271eb9818557bc8bc82 Reviewed-on: https://github.com/flang-compiler/f18/pull/758 Tree-same-pre-rewrite: false
2019-09-24 08:10:58 +08:00
if (!fileError.str().empty()) {
ProvenanceRange range{allSources.AddCompilerInsertion(path)};
messages_.Say(range, "%s"_err_en_US, fileError.str());
[flang] Improve module file reading and writing Fix problems with writing a mod file while another compilation is reading or writing. Write to a temp and then rename it: - compute the new contents of the .mod file - if it already exists, check if it is already correct - if not, write new contents to a temp file - rename the temp to the final destination `mkstemps()` seems to be the best way to create the temp file. It returns a file descriptor, so change the rest of the mod file writing to use POSIX open/read/write/close. This seems to set errno more reliably too. There is some extra work around creating the temp to make it have the same directory and suffix as the final file (so that if one gets left behind by a crash, "rm *.mod" still cleans it up). `mkstemps()` creates file with 0600 permissions so try to change it to what it would have been if we just wrote the file. Change module file reading to only read the file once; we used to read it to verify the checksum and then again to parse it. Instead, change `Parsing` so that we can get the file contents after `Prescan()` and use that to verify the checksum. Also, it has a mechanism for searching directories for files, so make use of that instead of duplicating that functionality in `ModFileReader`. This requires some changes to how errors are returned so they can be reported in the right place. Original-commit: flang-compiler/f18@d0d54971a5547820f8ed0271eb9818557bc8bc82 Reviewed-on: https://github.com/flang-compiler/f18/pull/758 Tree-same-pre-rewrite: false
2019-09-24 08:10:58 +08:00
return sourceFile;
}
CHECK(sourceFile);
[flang] Improve module file reading and writing Fix problems with writing a mod file while another compilation is reading or writing. Write to a temp and then rename it: - compute the new contents of the .mod file - if it already exists, check if it is already correct - if not, write new contents to a temp file - rename the temp to the final destination `mkstemps()` seems to be the best way to create the temp file. It returns a file descriptor, so change the rest of the mod file writing to use POSIX open/read/write/close. This seems to set errno more reliably too. There is some extra work around creating the temp to make it have the same directory and suffix as the final file (so that if one gets left behind by a crash, "rm *.mod" still cleans it up). `mkstemps()` creates file with 0600 permissions so try to change it to what it would have been if we just wrote the file. Change module file reading to only read the file once; we used to read it to verify the checksum and then again to parse it. Instead, change `Parsing` so that we can get the file contents after `Prescan()` and use that to verify the checksum. Also, it has a mechanism for searching directories for files, so make use of that instead of duplicating that functionality in `ModFileReader`. This requires some changes to how errors are returned so they can be reported in the right place. Original-commit: flang-compiler/f18@d0d54971a5547820f8ed0271eb9818557bc8bc82 Reviewed-on: https://github.com/flang-compiler/f18/pull/758 Tree-same-pre-rewrite: false
2019-09-24 08:10:58 +08:00
if (!options.isModuleFile) {
// For .mod files we always want to look in the search directories.
// For normal source files we don't push them until after the primary
// source file has been opened. If foo.f is missing from the current
// working directory, we don't want to accidentally read another foo.f
// from another directory that's on the search path.
for (const auto &path : options.searchDirectories) {
allSources.PushSearchPathDirectory(path);
}
}
Preprocessor preprocessor{allSources};
for (const auto &predef : options.predefinitions) {
if (predef.second) {
preprocessor.Define(predef.first, *predef.second);
} else {
preprocessor.Undefine(predef.first);
}
}
Prescanner prescanner{messages_, cooked_, preprocessor, options.features};
prescanner.set_fixedForm(options.isFixedForm)
.set_fixedFormColumnLimit(options.fixedFormColumns)
.AddCompilerDirectiveSentinel("dir$");
if (options.features.IsEnabled(LanguageFeature::OpenACC)) {
prescanner.AddCompilerDirectiveSentinel("$acc");
}
if (options.features.IsEnabled(LanguageFeature::OpenMP)) {
prescanner.AddCompilerDirectiveSentinel("$omp");
prescanner.AddCompilerDirectiveSentinel("$"); // OMP conditional line
}
ProvenanceRange range{allSources.AddIncludedFile(
*sourceFile, ProvenanceRange{}, options.isModuleFile)};
prescanner.Prescan(range);
if (cooked_.BufferedBytes() == 0 && !options.isModuleFile) {
// Input is empty. Append a newline so that any warning
// message about nonstandard usage will have provenance.
cooked_.Put('\n', range.start());
}
cooked_.Marshal();
if (options.needProvenanceRangeToCharBlockMappings) {
cooked_.CompileProvenanceRangeToOffsetMappings();
}
[flang] Improve module file reading and writing Fix problems with writing a mod file while another compilation is reading or writing. Write to a temp and then rename it: - compute the new contents of the .mod file - if it already exists, check if it is already correct - if not, write new contents to a temp file - rename the temp to the final destination `mkstemps()` seems to be the best way to create the temp file. It returns a file descriptor, so change the rest of the mod file writing to use POSIX open/read/write/close. This seems to set errno more reliably too. There is some extra work around creating the temp to make it have the same directory and suffix as the final file (so that if one gets left behind by a crash, "rm *.mod" still cleans it up). `mkstemps()` creates file with 0600 permissions so try to change it to what it would have been if we just wrote the file. Change module file reading to only read the file once; we used to read it to verify the checksum and then again to parse it. Instead, change `Parsing` so that we can get the file contents after `Prescan()` and use that to verify the checksum. Also, it has a mechanism for searching directories for files, so make use of that instead of duplicating that functionality in `ModFileReader`. This requires some changes to how errors are returned so they can be reported in the right place. Original-commit: flang-compiler/f18@d0d54971a5547820f8ed0271eb9818557bc8bc82 Reviewed-on: https://github.com/flang-compiler/f18/pull/758 Tree-same-pre-rewrite: false
2019-09-24 08:10:58 +08:00
return sourceFile;
}
void Parsing::DumpCookedChars(llvm::raw_ostream &out) const {
UserState userState{cooked_, common::LanguageFeatureControl{}};
ParseState parseState{cooked_};
parseState.set_inFixedForm(options_.isFixedForm).set_userState(&userState);
while (std::optional<const char *> p{parseState.GetNextChar()}) {
out << **p;
}
}
void Parsing::DumpProvenance(llvm::raw_ostream &out) const {
cooked_.Dump(out);
}
void Parsing::DumpParsingLog(llvm::raw_ostream &out) const {
log_.Dump(out, cooked_);
}
void Parsing::Parse(llvm::raw_ostream &out) {
UserState userState{cooked_, options_.features};
userState.set_debugOutput(out)
.set_instrumentedParse(options_.instrumentedParse)
.set_log(&log_);
ParseState parseState{cooked_};
parseState.set_inFixedForm(options_.isFixedForm).set_userState(&userState);
parseTree_ = program.Parse(parseState);
CHECK(
!parseState.anyErrorRecovery() || parseState.messages().AnyFatalError());
consumedWholeFile_ = parseState.IsAtEnd();
messages_.Annex(std::move(parseState.messages()));
finalRestingPlace_ = parseState.GetLocation();
}
void Parsing::ClearLog() { log_.clear(); }
} // namespace Fortran::parser