maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

[sanitizers] Make it possible to XFAIL on the effective target, not just the default. 

Summary:
The triple is not the right thing to XFAIL on since LIT only sees the default
triple and not the effective triple chosen by any -target option in the RUN
directives. This discrepancy is shown in the table below:

  Default Triple   | Options                           | XFAIL  | LIT's expected result | Desired expectation
  =================+===================================+========+=======================+====================
  mips-linux-gnu   | -target mips-linux-gnu            |        | Pass                  | Pass  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 |        | Pass                  | Pass  
  mips-linux-gnu   | -target mips-linux-gnu            | mips   | Fail                  | Fail  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips   | Fail                  | Fail/Pass* (debatable**)
  mips-linux-gnu   | -target mips-linux-gnu            | mips-  | Fail                  | Fail  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips-  | Fail                  | Pass* 
  mips-linux-gnu   | -target mips-linux-gnu            | mips64 | Pass                  | Pass  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips64 | Pass                  | Fail* 
  mips64-linux-gnu | -target mips-linux-gnu            |        | Pass                  | Pass  
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 |        | Pass                  | Pass  
  mips64-linux-gnu | -target mips-linux-gnu            | mips   | Fail                  | Fail* 
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips   | Fail                  | Fail/Pass (debatable**)
  mips64-linux-gnu | -target mips-linux-gnu            | mips-  | Pass                  | Fail* 
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips-  | Pass                  | Pass  
  mips64-linux-gnu | -target mips-linux-gnu            | mips64 | Fail                  | Pass* 
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips64 | Fail                  | Fail  
  x64_64-linux-gnu | -target i386-linux-gnu            |        | Pass                  | Pass
  x64_64-linux-gnu | -target x86_64-linux-gnu          |        | Pass                  | Pass
  x64_64-linux-gnu | -target i386-linux-gnu            | i386   | Pass                  | Fail*
  x64_64-linux-gnu | -target x86_64-linux-gnu          | i386   | Pass                  | Pass
  x64_64-linux-gnu | -target i386-linux-gnu            | x86_64 | Fail                  | Pass
  x64_64-linux-gnu | -target x86_64-linux-gnu          | x86_64 | Fail                  | Fail*
  * These all differ from LIT's current behaviour.
  ** People's expectations vary depending on whether they know that LIT does a
   substring match on the default triple or think it's an exact match on an
   architecture.

This patch adds "target-is-${target_arch}" to the available features list and
updates the mips XFAIL's to use them. XFAIL'ing on these features will
correctly account for the target being tested. Making the table:

  Options                           | XFAIL            | LIT's expected result
  ==================================+==================+======================
  -target mips-linux-gnu            |                  | Pass
  -target mips64-linux-gnu -mabi=64 |                  | Pass
  -target mips-linux-gnu            | target-is-mips   | Fail
  -target mips64-linux-gnu -mabi=64 | target-is-mips   | Pass
  -target mips-linux-gnu            | target-is-mips64 | Pass
  -target mips64-linux-gnu -mabi=64 | target-is-mips64 | Fail
  -target i386-linux-gnu            |                  | Pass
  -target x86_64-linux-gnu          |                  | Pass
  -target i386-linux-gnu            | target-is-i386   | Fail
  -target x86_64-linux-gnu          | target-is-i386   | Pass
  -target i386-linux-gnu            | target-is-x86_64 | Pass
  -target x86_64-linux-gnu          | target-is-x86_64 | Fail

Reviewers: probinson

Subscribers: probinson, kubabrecka, llvm-commits, samsonov

Differential Revision: https://reviews.llvm.org/D22802

llvm-svn: 278116
This commit is contained in:
Daniel Sanders 2016-08-09 11:50:53 +00:00
parent 16a5930803
commit dc213305e9
11 changed files with 76 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

66
compiler-rt/docs/TestingGuide.rst Normal file
View File

@ -0,0 +1,66 @@
========================================
Compiler-rt Testing Infrastructure Guide
========================================
.. contents::
:local:
Overview
========
This document is the reference manual for the compiler-rt modifications to the
testing infrastructure. Documentation for the infrastructure itself can be found at
:ref:`llvm_testing_guide`.
LLVM testing infrastructure organization
========================================
The compiler-rt testing infrastructure contains regression tests which are run
as part of the usual ``make check-all`` and are expected to always pass -- they
should be run before every commit.
Quick start
===========
The regressions tests are in the "compiler-rt" module and are normally checked
out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all``
to run the regression tests after building compiler-rt.
REQUIRES, XFAIL, etc.
---------------------
Sometimes it is necessary to restrict a test to a specific target or mark it as
an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or
``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the
behaviour of this is somewhat quirky in compiler-rt. There are two main
pitfalls to avoid.
The first pitfall is that these directives perform a substring match on the
triple and as such ``XFAIL: mips`` affects more triples than expected. For
example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and
``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a
trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but
even that has issues as described below.
The second pitfall is that the default target triple is often inappropriate for
compiler-rt tests since compiler-rt tests may be compiled for multiple targets.
For example, a typical build on an ``x86_64-linux-gnu`` host will often run the
tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark
both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386``
will have no effect at all.
To remedy both pitfalls, compiler-rt tests provide a feature string which can
be used to specify a single target. This string is of the form
``target-is-${arch}`` where ``${arch}}`` is one of the values from the
following lines of the CMake output::
-- Compiler-RT supported architectures: x86_64;i386
-- Builtin supported architectures: i386;x86_64
So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail
on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386``
will mark a test as expected to fail on i386 even if the default target triple
is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string
require exact matches so ``XFAIL: target-is-mips``,
``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and
``XFAIL: target-is-mips64el`` all refer to different MIPS targets.

2
compiler-rt/test/asan/TestCases/Linux/local_alias.cc
View File

@ -6,7 +6,7 @@
//
// FIXME: https://github.com/google/sanitizers/issues/316
// XFAIL: android
// XFAIL: mips64
// XFAIL: target-is-mips64
//
// RUN: %clangxx_asan -DBUILD_INSTRUMENTED_DSO=1 -fPIC -shared -mllvm -asan-use-private-alias %s -o %t-INSTRUMENTED-SO.so
// RUN: %clangxx -DBUILD_UNINSTRUMENTED_DSO=1 -fPIC -shared %s -o %t-UNINSTRUMENTED-SO.so

2
compiler-rt/test/asan/TestCases/Linux/odr-violation.cc
View File

@ -1,6 +1,6 @@
// FIXME: https://code.google.com/p/address-sanitizer/issues/detail?id=316
// XFAIL: android
// XFAIL: mips64
// XFAIL: target-is-mips64
//
// We use fast_unwind_on_malloc=0 to have full unwinding even w/o frame
// pointers. This setting is not on by default because it's too expensive.

1
compiler-rt/test/lit.common.configured.in
View File

@ -28,6 +28,7 @@ set_default("emulator", "@COMPILER_RT_EMULATOR@")
set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@)
set_default("has_lld", @COMPILER_RT_HAS_LLD_SOURCES_PYBOOL@)
set_default("can_symbolize", @CAN_SYMBOLIZE@)
config.available_features.add('target-is-%s' % config.target_arch)
# LLVM tools dir can be passed in lit parameters, so try to
# apply substitution.

2
compiler-rt/test/tsan/ignore_lib4.cc
View File

@ -4,7 +4,7 @@
// RUN: %env_tsan_opts=suppressions='%t.supp' %run %t 2>&1 | FileCheck %s
// Longjmp assembly has not been implemented for mips64 yet
// XFAIL: mips64
// XFAIL: target-is-mips64
// powerpc64 big endian bots failed with "FileCheck error: '-' is empty" due
// to a segmentation fault.
// UNSUPPORTED: powerpc64-unknown-linux-gnu

2
compiler-rt/test/tsan/longjmp.cc
View File

@ -1,7 +1,7 @@
// RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
// Longjmp assembly has not been implemented for mips64 yet
// XFAIL: mips64
// XFAIL: target-is-mips64
#include <stdio.h>
#include <stdlib.h>

2
compiler-rt/test/tsan/longjmp2.cc
View File

@ -1,7 +1,7 @@
// RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
// Longjmp assembly has not been implemented for mips64 yet
// XFAIL: mips64
// XFAIL: target-is-mips64
#include <stdio.h>
#include <stdlib.h>

2
compiler-rt/test/tsan/longjmp3.cc
View File

@ -1,7 +1,7 @@
// RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
// Longjmp assembly has not been implemented for mips64 yet
// XFAIL: mips64
// XFAIL: target-is-mips64
#include <pthread.h>
#include <stdio.h>

2
compiler-rt/test/tsan/longjmp4.cc
View File

@ -1,7 +1,7 @@
// RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
// Longjmp assembly has not been implemented for mips64 yet
// XFAIL: mips64
// XFAIL: target-is-mips64
#include <pthread.h>
#include <stdio.h>

2
compiler-rt/test/tsan/map32bit.cc
View File

@ -8,7 +8,7 @@
// https://github.com/google/sanitizers/issues/412
// MAP_32BIT flag for mmap is supported only for x86_64.
// XFAIL: mips64
// XFAIL: target-is-mips64
// XFAIL: aarch64
// XFAIL: powerpc64

2
compiler-rt/test/tsan/signal_longjmp.cc
View File

@ -4,7 +4,7 @@
// https://github.com/google/sanitizers/issues/482
// Longjmp assembly has not been implemented for mips64 yet
// XFAIL: mips64
// XFAIL: target-is-mips64
// This test fails on powerpc64 BE (VMA=44), a segmentation fault
// error happens at the second assignment
// "((volatile int *volatile)mem)[1] = 1".