maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

Prevent data races in concurrent ThinLTO processes. 

Make sure ThinLTO with caching doesn't use non-atomic writes to the cache file (to prevent data races and cache files corruption).

1. Place temp file to the same place where the caching directory is (instead of creating it the directory pointed to by TMP/TEMP variable). This will help to prevent using non-atomic rename and falling back to non-atomic "direct" write to the cache file.
2. if rename failed do not write to the cache file directly (direct write to the file is non-atomic and could cause data race conditions).
3. if cache file doesn't exist (e.g., because 'rename' failed or because some other reasons), bypass using the cache altogether.

Differential Revision:  https://reviews.llvm.org/D45076

llvm-svn: 328904
This commit is contained in:
Ekaterina Romanova 2018-03-30 21:35:42 +00:00
parent d1fe360b06
commit 0b01dfbba6
1 changed files with 14 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
llvm/lib/LTO/ThinLTOCodeGenerator.cpp
View File

@ -400,9 +400,12 @@ public:
// Write to a temporary to avoid race condition
SmallString<128> TempFilename;
SmallString<128> CachePath(EntryPath);
int TempFD;
std::error_code EC =
sys::fs::createTemporaryFile("Thin", "tmp.o", TempFD, TempFilename);
llvm::sys::path::remove_filename(CachePath);
sys::path::append(TempFilename, CachePath, "Thin-%%%%%%.tmp.o");
std::error_code EC =
sys::fs::createUniqueFile(TempFilename, TempFD, TempFilename);
if (EC) {
errs() << "Error: " << EC.message() << "\n";
report_fatal_error("ThinLTO: Can't get a temporary file");
@ -411,16 +414,10 @@ public:
raw_fd_ostream OS(TempFD, /* ShouldClose */ true);
OS << OutputBuffer.getBuffer();
}
// Rename to final destination (hopefully race condition won't matter here)
// Rename temp file to final destination; rename is atomic
EC = sys::fs::rename(TempFilename, EntryPath);
if (EC) {
if (EC)
sys::fs::remove(TempFilename);
raw_fd_ostream OS(EntryPath, EC, sys::fs::F_None);
if (EC)
report_fatal_error(Twine("Failed to open ") + EntryPath +
" to save cached entry\n");
OS << OutputBuffer.getBuffer();
}
}
};
@ -1027,15 +1024,15 @@ void ThinLTOCodeGenerator::run() {
if (SavedObjectsDirectoryPath.empty()) {
// We need to generated a memory buffer for the linker.
if (!CacheEntryPath.empty()) {
// Cache is enabled, reload from the cache
// We do this to lower memory pressuree: the buffer is on the heap
// and releasing it frees memory that can be used for the next input
// file. The final binary link will read from the VFS cache
// (hopefully!) or from disk if the memory pressure wasn't too high.
// When cache is enabled, reload from the cache if possible.
// Releasing the buffer from the heap and reloading it from the
// cache file with mmap helps us to lower memory pressure.
// The freed memory can be used for the next input file.
// The final binary link will read from the VFS cache (hopefully!)
// or from disk (if the memory pressure was too high).
auto ReloadedBufferOrErr = CacheEntry.tryLoadingBuffer();
if (auto EC = ReloadedBufferOrErr.getError()) {
// On error, keeping the preexisting buffer and printing a
// diagnostic is more friendly than just crashing.
// On error, keep the preexisting buffer and print a diagnostic.
errs() << "error: can't reload cached file '" << CacheEntryPath
<< "': " << EC.message() << "\n";
} else {