Instead of passing file dependencies individually, assume that the
whole content of the unique test directory is a dependency. This
simplifies the test harness significantly, by making %T the directory
that contains everything required to run a test. This also removes the
need for the %{file_dependencies} substitution, which is removed by this
patch.
Furthermore, this patch also changes the harness to execute tests locally
inside %T, so as to avoid creating a separate directory for no purpose.
C++98 and C++03 are effectively aliases as far as Clang is concerned.
As such, allowing both std=c++98 and std=c++03 as Lit parameters is
just slightly confusing, but provides no value. It's similar to allowing
both std=c++17 and std=c++1z, which we don't do.
This was discovered because we had an internal bot that ran the test
suite under both c++98 AND c++03 -- one of which is redundant.
Differential Revision: https://reviews.llvm.org/D80926
Don't use std::filesystem APIs for CWDGuard, use POSIX functions
instead. This way the tests don't rely on the correctness of
the functionality they're testing.
Differential Revision: https://reviews.llvm.org/D78200
When running on remote hosts, we need the whole `echo 123 | %t.exe` command
to run on the remote host. Thus, we need to escape the pipe to make sure
the command is treated as `{ echo 123 | %t.exe } > %t.out` instead of
`{ echo 123 } | %t.exe > %t.out`m where only `echo 123` is run on the
remote host.
Summary:
Instead of storing `static_test_env` (with all the symlinks) in the repo, we create it on the fly to be cross-toolchain-friendly. The primary use case for this are Windows-hosted cross-toolchains. Windows doesn't really have a concept of symlinks. So, when the monorepo is cloned, those symlinks turn to ordinary text files. Previously, if we cross-compiled libc++ for some symlink-friendly system (e. g. Linux) and ran tests on the target system, some tests would fail. This patch makes them pass.
Reviewers: ldionne, #libc
Reviewed By: ldionne, #libc
Subscribers: EricWF, dexonsmith, libcxx-commits
Tags: #libc
Differential Revision: https://reviews.llvm.org/D78200
Instead of storing static_test_env (with all the symlinks) in the repo,
we create it on the fly to be cross-toolchain-friendly. The primary
use case for this are Windows-hosted cross-toolchains. Windows doesn't
really have a concept of symlinks. So, when the monorepo is cloned,
those symlinks turn to ordinary text files. Previously, if we
cross-compiled libc++ for some symlink-friendly system (e. g. Linux) and
ran tests on the target system, some tests would fail. This patch makes
them pass.
Differential Revision: https://reviews.llvm.org/D78200
Tests that require support for Clang-verify are already marked as such
explicitly by their extension, which is .verify.cpp. Requiring the use
of an explicit Lit feature is, after thought, not really helpful.
This is a change in design: we have been bitten in the past by tests not
being enabled when we thought they were. However, the issue was mostly
with file extensions being ignored. The fix for that is not to blindly
require explicit features all the time, but instead to report all files
that are in the suite but that don't match any known test format. This
can be implemented in a follow-up patch.
Marked unsupported for C++03 and C++11 since this test uses alias
declarations, and at least one C++03 bot was failing with
-Wc++11-extensions.
Change-Id: I8c3a579edd7eb83e0bc74e85d116b68f22400161
By renaming .fail.cpp tests that don't need clang-verify to .compile.fail.cpp,
the new test format will not try to compile these tests with clang-verify,
and the old test format will work just the same. However, this allows
removing a workaround that requires parsing each test looking for
clang-verify markup.
After this change, a .fail.cpp test should always have clang-verify markup.
When clang-verify is not supported by the compiler, we will just check that
these tests fail to compile. When clang-verify is supported, these tests
will be compiled with clang-verify whether they have markup or not (so
they should have markup, or they will fail).
This simplifies the test suite and also ensures that all of our .fail.cpp
tests provide clang-verify markup. If it's impossible for a test to have
clang-verify markup, it can be moved to a .compile.fail.cpp test, which
are unconditionally just checked for compilation failure.
The libc++ test suite currently defines several features that are not
used anywhere in the tests, or that are redundant with other features.
For the purpose of simplifying config.py and to ease the bring up of a
new configuration, this commit removes some of these features:
- rename dylib-has-no-filesystem to c++filesystem-disabled, which exists
- rename apple-darwin to just darwin, which is already set
- remove useless setting of libstdc++, which is already set correctly
- remove libcpp-abi-unstable, which is not used anywhere
- remove the glibc-XXX features, which are not used anywhere
The libc++ test suite has a lot of old Lit features used to XFAIL tests
and mark them as UNSUPPORTED. Many of them are to workaround problems on
old compilers or old platforms. As time goes by, it is good to go and
clean those up to simplify the configuration of the test suite, and also
to reflect the testing reality. It's not useful to have markup that gives
the impression that e.g. clang-3.3 is supported, when we don't really
test on it anymore (and hence several new tests probably don't have the
necessary markup on them).
Broke builders that emit different diagnostics. e.g.:
error: 'warning' diagnostics seen but not expected:
Line 13: alias declarations are a C++11 extension
Line 20: alias declarations are a C++11 extension
This reverts commit ff87813715.
The testing script used to test libc++ historically did not like directories
without any testing files, so these tests had been added. Since this is
not necessary anymore, we can now remove these files. This has the benefit
that the total number of tests reflects the real number of tests more
closely, and we also skip some unnecessary work (especially relevant when
running tests over SSH).
However, some nothing_to_do.pass.cpp tests actually serve the purpose of
documenting that an area of the Standard doesn't need to be tested, or is
tested elsewhere. These files are not removed by this commit.
Removal done with:
import os
import itertools
for (dirpath, dirnames, filenames) in itertools.chain(os.walk('./libcxx/test'),
os.walk('./libcxxabi/test')):
if len(filenames + dirnames) > 1 and \
any(p == 'nothing_to_do.pass.cpp' for p in filenames):
os.remove(os.path.join(dirpath, 'nothing_to_do.pass.cpp'))
Instead of hardcoding absolute paths on the build-host in the executables,
use relative paths from the current working directory. Also, use
FILE_DEPENDENCIES to mark the static test env as being required by
the relevant tests.
Given a SSH executor that copies the files to the remote host properly,
the tests can be run on that remote host.
This patch reimplements the dynamic filesystem helper using Posix
functionality instead of relying on Python. The primary reason for
doing this is that it allows running the libc++ test suite on devices
that do not have Python.
Differential Revision: https://reviews.llvm.org/D77140
Otherwise, trying to reproduce a failing filesystem test by copy-pasting
the command-line used and running that in the shell won't work, because
the shell will eat quoting around the define and we'll end up with a
non-stringized path in the .cpp file.
That way, local lit configuration files don't have to worry about
deep-copying the compiler instance of the test format, which is
arguably an implementation detail.
We pass the config to this method even though it is not used by the
current test format because this allows replacing the current test
format by other test formats that would require the config to add
new compile flags.
This reduces the complexity of our already complex global lit configuration,
and also avoids cluttering the compilation commands for all tests with
things that are only relevant to the filesystem tests.
Differential Revision: https://reviews.llvm.org/D76785
Forcing -Werror and other warnings means that the test suite isn't
actually testing what most people are seeing in their code -- it seems
better and less arbitrary to compile these tests as close as possible
to the compiler default instead.
Removing -Werror also means that we get to differentiate between
diagnostics that are errors and those that are warnings, which makes
the test suite more precise.
Differential Revision: https://reviews.llvm.org/D76311
Some tests do not fail at all when -verify is not supported, unless some
arbitrary warning flag is added to make them fail. We currently used
-Werror=unused-result to make them fail, but doing so makes the test
suite a lot more inscrutable. It seems better to just disable those
tests when -verify is not supported.
Differential Revision: https://reviews.llvm.org/D76256
Summary:
In `std::filesystem::proximate` tests we assume that the current working directory's name
is `fs.op.proximate`. This is fine when we're running the tests locally.
However, if we're running those tests on a remote machine via SSH, the directory layout may be
different. For example, currently we copy each test executable individually into
a temporary directory on the target board using SCP, so the assumption about the working directory name
doesn't necessarily hold.
This patch is the only thing that is necessary for all libc++ tests to pass when run remotely.
Reviewers: ldionne, EricWF, mclow.lists
Reviewed By: ldionne, EricWF
Subscribers: christof, dexonsmith, libcxx-commits
Tags: #libc
Differential Revision: https://reviews.llvm.org/D74348
This issue was detected by ASan in one of our tests. This test manually
invokes basic_filebuf::cloe(). fclose(__h.release() returned a non-zero
exit status, so __file_ wasn't set to 0. Later when basic_filebuf
destructor ran, we would enter the if (__file_) block again leading to
heap-use-after-free error.
The POSIX specification for fclose says that independently of the return
value, fclose closes the underlying file descriptor and any further
access (including another call to fclose()) to the stream results in
undefined behavior. This is exactly what happened in our test case.
To avoid this issue, we have to always set __file_ to 0 independently of
the fclose return value.
Differential Revision: https://reviews.llvm.org/D64979
llvm-svn: 366730
We fixed incorrect behavior of input streams in r357775 and tests were
added accordingly. However, older versions of macOS don't have the
change in the dylib yet, so the tests fail on those platforms.
llvm-svn: 357794
Summary:
This is a re-application of r357533 and r357531. They had been reverted
because we thought the commits broke the LLDB data formatters, but it
turns out this was because only r357531 had been included in the CI
run.
Before this patch, we would only ever throw an exception if the badbit
was set on the stream. The Standard is currently very unclear on how
exceptions should be propagated and what error flags should be set by
the input stream operations. This commit changes libc++ to behave under
a different (but valid) interpretation of the Standard. This interpretation
of the Standard matches what other implementations are doing.
This effectively implements the wording in p1264r0. It hasn't been voted
into the Standard yet, however there is wide agreement that the fix is
correct and it's just a matter of time before the fix is standardized.
PR21586
PR15949
rdar://problem/15347558
Reviewers: mclow.lists, EricWF
Subscribers: christof, dexonsmith, cfe-commits
Differential Revision: https://reviews.llvm.org/D49863
llvm-svn: 357775
This reverts commits r357533 and r357531, which broke the LLDB
data formatters. I'll hold off until we know how to fix the data
formatters accordingly.
llvm-svn: 357536
Summary:
Before this patch, we would only ever throw an exception if the badbit
was set on the stream. The Standard is currently very unclear on how
exceptions should be propagated and what error flags should be set by
the input stream operations. This commit changes libc++ to behave under
a different (but valid) interpretation of the Standard. This interpretation
of the Standard matches what other implementations are doing.
I will submit a paper in San Diego to clarify the Standard such that the
interpretation used in this commit (and other implementations) is the only
possible one.
PR21586
PR15949
rdar://problem/15347558
Reviewers: mclow.lists, EricWF
Subscribers: christof, dexonsmith, cfe-commits
Differential Revision: https://reviews.llvm.org/D49863
llvm-svn: 357531
Summary: Filesystem doesn't work on Windows, so we need a mechanism to turn it off for the time being.
Reviewers: ldionne, serge-sans-paille, EricWF
Reviewed By: EricWF
Subscribers: mstorsjo, mgorny, christof, jdoerfert, libcxx-commits
Differential Revision: https://reviews.llvm.org/D59619
llvm-svn: 356633
This fixes CI for back-deployment testers on platforms that don't have
<filesystem> support in the dylib.
This is effectively half of https://reviews.llvm.org/D59224. The other
half requires fixes in Clang.
llvm-svn: 356558
Summary:
This patch treats <filesystem> as a first-class citizen of the dylib,
like all other sub-libraries (e.g. <chrono>). As such, it also removes
all special handling for installing the filesystem library separately
or disabling part of the test suite from the lit command line.
Unlike the previous attempt (r356500), this doesn't remove all the
filesystem tests.
Reviewers: mclow.lists, EricWF, serge-sans-paille
Subscribers: mgorny, christof, jkorous, dexonsmith, jfb, jdoerfert, libcxx-commits
Differential Revision: https://reviews.llvm.org/D59152
llvm-svn: 356518
When I applied r356500 (https://reviews.llvm.org/D59152), I somehow
deleted all of filesystem's tests. I will revert r356500 and re-apply
it properly.
llvm-svn: 356505
Summary:
This patch treats <filesystem> as a first-class citizen of the dylib,
like all other sub-libraries (e.g. <chrono>). As such, it also removes
all special handling for installing the filesystem library separately
or disabling part of the test suite from the lit command line.
Reviewers: mclow.lists, EricWF, serge-sans-paille
Subscribers: mgorny, christof, jkorous, dexonsmith, jfb, jdoerfert, libcxx-commits
Differential Revision: https://reviews.llvm.org/D59152
llvm-svn: 356500
Louis Dionne