[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
//===--- FSProvider.cpp - VFS provider for ClangdServer -------------------===//
|
|
|
|
//
|
2019-01-19 16:50:56 +08:00
|
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
//
|
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
|
|
|
|
#include "FSProvider.h"
|
|
|
|
#include "llvm/ADT/STLExtras.h"
|
|
|
|
#include "llvm/ADT/SmallString.h"
|
|
|
|
#include "llvm/ADT/StringRef.h"
|
|
|
|
#include "llvm/Support/Path.h"
|
|
|
|
#include "llvm/Support/VirtualFileSystem.h"
|
|
|
|
#include <memory>
|
|
|
|
|
|
|
|
namespace clang {
|
|
|
|
namespace clangd {
|
|
|
|
|
|
|
|
namespace {
|
|
|
|
/// Always opens files in the underlying filesystem as "volatile", meaning they
|
[clangd] Don't mmap source files on all platforms --> don't crash on git checkout
Summary:
Previously we mmapped on unix and not on windows: on windows mmap takes
an exclusive lock on the file and prevents the user saving changes!
The failure mode on linux is a bit more subtle: if the file is changed on disk
but the SourceManager sticks around, then subsequent operations on the
SourceManager will fail as invariants are violated (e.g. null-termination).
This commonly manifests as crashes after switching git branches with many files
open in clangd.
Nominally mmap is for performance here, and we should be willing to give some
up to stop crashing. Measurements on my system (linux+desktop+SSD) at least
show no measurable regression on an a fairly IO-heavy workload: drop disk caches,
open SemaOverload.cpp, wait for first diagnostics.
for i in `seq 100`; do
for variant in mmap volatile; do
echo 3 | sudo tee /proc/sys/vm/drop_caches
/usr/bin/time --append --quiet -o ~/timings -f "%C %E" \
bin/clangd.$variant -sync -background-index=0 < /tmp/mirror > /dev/null
done
done
bin/clangd.mmap -sync -background-index=0 0:07.60
bin/clangd.volatile -sync -background-index=0 0:07.89
bin/clangd.mmap -sync -background-index=0 0:07.44
bin/clangd.volatile -sync -background-index=0 0:07.89
bin/clangd.mmap -sync -background-index=0 0:07.42
bin/clangd.volatile -sync -background-index=0 0:07.50
bin/clangd.mmap -sync -background-index=0 0:07.90
bin/clangd.volatile -sync -background-index=0 0:07.53
bin/clangd.mmap -sync -background-index=0 0:07.64
bin/clangd.volatile -sync -background-index=0 0:07.55
bin/clangd.mmap -sync -background-index=0 0:07.75
bin/clangd.volatile -sync -background-index=0 0:07.47
bin/clangd.mmap -sync -background-index=0 0:07.90
bin/clangd.volatile -sync -background-index=0 0:07.50
bin/clangd.mmap -sync -background-index=0 0:07.81
bin/clangd.volatile -sync -background-index=0 0:07.95
bin/clangd.mmap -sync -background-index=0 0:07.55
bin/clangd.volatile -sync -background-index=0 0:07.65
bin/clangd.mmap -sync -background-index=0 0:08.15
bin/clangd.volatile -sync -background-index=0 0:07.54
bin/clangd.mmap -sync -background-index=0 0:07.78
bin/clangd.volatile -sync -background-index=0 0:07.61
bin/clangd.mmap -sync -background-index=0 0:07.78
bin/clangd.volatile -sync -background-index=0 0:07.55
bin/clangd.mmap -sync -background-index=0 0:07.41
bin/clangd.volatile -sync -background-index=0 0:07.40
bin/clangd.mmap -sync -background-index=0 0:07.54
bin/clangd.volatile -sync -background-index=0 0:07.42
bin/clangd.mmap -sync -background-index=0 0:07.45
bin/clangd.volatile -sync -background-index=0 0:07.49
bin/clangd.mmap -sync -background-index=0 0:07.95
bin/clangd.volatile -sync -background-index=0 0:07.66
bin/clangd.mmap -sync -background-index=0 0:08.04
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73617
2020-01-29 19:22:03 +08:00
|
|
|
/// won't be memory-mapped. Memory-mapping isn't desirable for clangd:
|
|
|
|
/// - edits to the underlying files change contents MemoryBuffers owned by
|
|
|
|
// SourceManager, breaking its invariants and leading to crashes
|
|
|
|
/// - it locks files on windows, preventing edits
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
class VolatileFileSystem : public llvm::vfs::ProxyFileSystem {
|
|
|
|
public:
|
|
|
|
explicit VolatileFileSystem(llvm::IntrusiveRefCntPtr<FileSystem> FS)
|
|
|
|
: ProxyFileSystem(std::move(FS)) {}
|
|
|
|
|
2019-01-07 23:45:19 +08:00
|
|
|
llvm::ErrorOr<std::unique_ptr<llvm::vfs::File>>
|
|
|
|
openFileForRead(const llvm::Twine &InPath) override {
|
|
|
|
llvm::SmallString<128> Path;
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
InPath.toVector(Path);
|
|
|
|
|
|
|
|
auto File = getUnderlyingFS().openFileForRead(Path);
|
|
|
|
if (!File)
|
|
|
|
return File;
|
|
|
|
// Try to guess preamble files, they can be memory-mapped even on Windows as
|
[clangd] Don't mmap source files on all platforms --> don't crash on git checkout
Summary:
Previously we mmapped on unix and not on windows: on windows mmap takes
an exclusive lock on the file and prevents the user saving changes!
The failure mode on linux is a bit more subtle: if the file is changed on disk
but the SourceManager sticks around, then subsequent operations on the
SourceManager will fail as invariants are violated (e.g. null-termination).
This commonly manifests as crashes after switching git branches with many files
open in clangd.
Nominally mmap is for performance here, and we should be willing to give some
up to stop crashing. Measurements on my system (linux+desktop+SSD) at least
show no measurable regression on an a fairly IO-heavy workload: drop disk caches,
open SemaOverload.cpp, wait for first diagnostics.
for i in `seq 100`; do
for variant in mmap volatile; do
echo 3 | sudo tee /proc/sys/vm/drop_caches
/usr/bin/time --append --quiet -o ~/timings -f "%C %E" \
bin/clangd.$variant -sync -background-index=0 < /tmp/mirror > /dev/null
done
done
bin/clangd.mmap -sync -background-index=0 0:07.60
bin/clangd.volatile -sync -background-index=0 0:07.89
bin/clangd.mmap -sync -background-index=0 0:07.44
bin/clangd.volatile -sync -background-index=0 0:07.89
bin/clangd.mmap -sync -background-index=0 0:07.42
bin/clangd.volatile -sync -background-index=0 0:07.50
bin/clangd.mmap -sync -background-index=0 0:07.90
bin/clangd.volatile -sync -background-index=0 0:07.53
bin/clangd.mmap -sync -background-index=0 0:07.64
bin/clangd.volatile -sync -background-index=0 0:07.55
bin/clangd.mmap -sync -background-index=0 0:07.75
bin/clangd.volatile -sync -background-index=0 0:07.47
bin/clangd.mmap -sync -background-index=0 0:07.90
bin/clangd.volatile -sync -background-index=0 0:07.50
bin/clangd.mmap -sync -background-index=0 0:07.81
bin/clangd.volatile -sync -background-index=0 0:07.95
bin/clangd.mmap -sync -background-index=0 0:07.55
bin/clangd.volatile -sync -background-index=0 0:07.65
bin/clangd.mmap -sync -background-index=0 0:08.15
bin/clangd.volatile -sync -background-index=0 0:07.54
bin/clangd.mmap -sync -background-index=0 0:07.78
bin/clangd.volatile -sync -background-index=0 0:07.61
bin/clangd.mmap -sync -background-index=0 0:07.78
bin/clangd.volatile -sync -background-index=0 0:07.55
bin/clangd.mmap -sync -background-index=0 0:07.41
bin/clangd.volatile -sync -background-index=0 0:07.40
bin/clangd.mmap -sync -background-index=0 0:07.54
bin/clangd.volatile -sync -background-index=0 0:07.42
bin/clangd.mmap -sync -background-index=0 0:07.45
bin/clangd.volatile -sync -background-index=0 0:07.49
bin/clangd.mmap -sync -background-index=0 0:07.95
bin/clangd.volatile -sync -background-index=0 0:07.66
bin/clangd.mmap -sync -background-index=0 0:08.04
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73617
2020-01-29 19:22:03 +08:00
|
|
|
// clangd has exclusive access to those and nothing else should touch them.
|
2019-01-07 23:45:19 +08:00
|
|
|
llvm::StringRef FileName = llvm::sys::path::filename(Path);
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
if (FileName.startswith("preamble-") && FileName.endswith(".pch"))
|
|
|
|
return File;
|
2019-01-03 21:28:05 +08:00
|
|
|
return std::unique_ptr<VolatileFile>(new VolatileFile(std::move(*File)));
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
2019-01-07 23:45:19 +08:00
|
|
|
class VolatileFile : public llvm::vfs::File {
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
public:
|
2019-01-07 23:45:19 +08:00
|
|
|
VolatileFile(std::unique_ptr<llvm::vfs::File> Wrapped)
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
: Wrapped(std::move(Wrapped)) {
|
|
|
|
assert(this->Wrapped);
|
|
|
|
}
|
|
|
|
|
|
|
|
virtual llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>>
|
2019-01-07 23:45:19 +08:00
|
|
|
getBuffer(const llvm::Twine &Name, int64_t FileSize,
|
|
|
|
bool RequiresNullTerminator, bool /*IsVolatile*/) override {
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
return Wrapped->getBuffer(Name, FileSize, RequiresNullTerminator,
|
|
|
|
/*IsVolatile=*/true);
|
|
|
|
}
|
|
|
|
|
2019-01-07 23:45:19 +08:00
|
|
|
llvm::ErrorOr<llvm::vfs::Status> status() override {
|
|
|
|
return Wrapped->status();
|
|
|
|
}
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
llvm::ErrorOr<std::string> getName() override { return Wrapped->getName(); }
|
|
|
|
std::error_code close() override { return Wrapped->close(); }
|
|
|
|
|
|
|
|
private:
|
|
|
|
std::unique_ptr<File> Wrapped;
|
|
|
|
};
|
|
|
|
};
|
|
|
|
} // namespace
|
|
|
|
|
|
|
|
llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>
|
|
|
|
clang::clangd::RealFileSystemProvider::getFileSystem() const {
|
[clangd] Don't mmap source files on all platforms --> don't crash on git checkout
Summary:
Previously we mmapped on unix and not on windows: on windows mmap takes
an exclusive lock on the file and prevents the user saving changes!
The failure mode on linux is a bit more subtle: if the file is changed on disk
but the SourceManager sticks around, then subsequent operations on the
SourceManager will fail as invariants are violated (e.g. null-termination).
This commonly manifests as crashes after switching git branches with many files
open in clangd.
Nominally mmap is for performance here, and we should be willing to give some
up to stop crashing. Measurements on my system (linux+desktop+SSD) at least
show no measurable regression on an a fairly IO-heavy workload: drop disk caches,
open SemaOverload.cpp, wait for first diagnostics.
for i in `seq 100`; do
for variant in mmap volatile; do
echo 3 | sudo tee /proc/sys/vm/drop_caches
/usr/bin/time --append --quiet -o ~/timings -f "%C %E" \
bin/clangd.$variant -sync -background-index=0 < /tmp/mirror > /dev/null
done
done
bin/clangd.mmap -sync -background-index=0 0:07.60
bin/clangd.volatile -sync -background-index=0 0:07.89
bin/clangd.mmap -sync -background-index=0 0:07.44
bin/clangd.volatile -sync -background-index=0 0:07.89
bin/clangd.mmap -sync -background-index=0 0:07.42
bin/clangd.volatile -sync -background-index=0 0:07.50
bin/clangd.mmap -sync -background-index=0 0:07.90
bin/clangd.volatile -sync -background-index=0 0:07.53
bin/clangd.mmap -sync -background-index=0 0:07.64
bin/clangd.volatile -sync -background-index=0 0:07.55
bin/clangd.mmap -sync -background-index=0 0:07.75
bin/clangd.volatile -sync -background-index=0 0:07.47
bin/clangd.mmap -sync -background-index=0 0:07.90
bin/clangd.volatile -sync -background-index=0 0:07.50
bin/clangd.mmap -sync -background-index=0 0:07.81
bin/clangd.volatile -sync -background-index=0 0:07.95
bin/clangd.mmap -sync -background-index=0 0:07.55
bin/clangd.volatile -sync -background-index=0 0:07.65
bin/clangd.mmap -sync -background-index=0 0:08.15
bin/clangd.volatile -sync -background-index=0 0:07.54
bin/clangd.mmap -sync -background-index=0 0:07.78
bin/clangd.volatile -sync -background-index=0 0:07.61
bin/clangd.mmap -sync -background-index=0 0:07.78
bin/clangd.volatile -sync -background-index=0 0:07.55
bin/clangd.mmap -sync -background-index=0 0:07.41
bin/clangd.volatile -sync -background-index=0 0:07.40
bin/clangd.mmap -sync -background-index=0 0:07.54
bin/clangd.volatile -sync -background-index=0 0:07.42
bin/clangd.mmap -sync -background-index=0 0:07.45
bin/clangd.volatile -sync -background-index=0 0:07.49
bin/clangd.mmap -sync -background-index=0 0:07.95
bin/clangd.volatile -sync -background-index=0 0:07.66
bin/clangd.mmap -sync -background-index=0 0:08.04
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73617
2020-01-29 19:22:03 +08:00
|
|
|
// Avoid using memory-mapped files.
|
|
|
|
// FIXME: Try to use a similar approach in Sema instead of relying on
|
|
|
|
// propagation of the 'isVolatile' flag through all layers.
|
2019-02-15 19:04:25 +08:00
|
|
|
return new VolatileFileSystem(
|
|
|
|
llvm::vfs::createPhysicalFileSystem().release());
|
[clangd] Avoid memory-mapping files on Windows
Summary:
Memory-mapping files on Windows leads to them being locked and prevents
editors from saving changes to those files on disk. This is fine for the
compiler, but not acceptable for an interactive tool like clangd.
Therefore, we choose to avoid using memory-mapped files on Windows.
Reviewers: hokein, kadircet
Reviewed By: kadircet
Subscribers: yvvan, zturner, nik, malaperle, mgorny, ioeric, MaskRay, jkorous, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D55139
llvm-svn: 348147
2018-12-03 23:21:49 +08:00
|
|
|
}
|
|
|
|
} // namespace clangd
|
|
|
|
} // namespace clang
|