|
| 1 | +======================================== |
| 2 | +Compiler-rt Testing Infrastructure Guide |
| 3 | +======================================== |
| 4 | + |
| 5 | +.. contents:: |
| 6 | + :local: |
| 7 | + |
| 8 | +Overview |
| 9 | +======== |
| 10 | + |
| 11 | +This document is the reference manual for the compiler-rt modifications to the |
| 12 | +testing infrastructure. Documentation for the infrastructure itself can be found at |
| 13 | +:ref:`llvm_testing_guide`. |
| 14 | + |
| 15 | +LLVM testing infrastructure organization |
| 16 | +======================================== |
| 17 | + |
| 18 | +The compiler-rt testing infrastructure contains regression tests which are run |
| 19 | +as part of the usual ``make check-all`` and are expected to always pass -- they |
| 20 | +should be run before every commit. |
| 21 | + |
| 22 | +Quick start |
| 23 | +=========== |
| 24 | + |
| 25 | +The regressions tests are in the "compiler-rt" module and are normally checked |
| 26 | +out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all`` |
| 27 | +to run the regression tests after building compiler-rt. |
| 28 | + |
| 29 | +REQUIRES, XFAIL, etc. |
| 30 | +--------------------- |
| 31 | + |
| 32 | +Sometimes it is necessary to restrict a test to a specific target or mark it as |
| 33 | +an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or |
| 34 | +``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the |
| 35 | +behaviour of this is somewhat quirky in compiler-rt. There are two main |
| 36 | +pitfalls to avoid. |
| 37 | + |
| 38 | +The first pitfall is that these directives perform a substring match on the |
| 39 | +triple and as such ``XFAIL: mips`` affects more triples than expected. For |
| 40 | +example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and |
| 41 | +``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a |
| 42 | +trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but |
| 43 | +even that has issues as described below. |
| 44 | + |
| 45 | +The second pitfall is that the default target triple is often inappropriate for |
| 46 | +compiler-rt tests since compiler-rt tests may be compiled for multiple targets. |
| 47 | +For example, a typical build on an ``x86_64-linux-gnu`` host will often run the |
| 48 | +tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark |
| 49 | +both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386`` |
| 50 | +will have no effect at all. |
| 51 | + |
| 52 | +To remedy both pitfalls, compiler-rt tests provide a feature string which can |
| 53 | +be used to specify a single target. This string is of the form |
| 54 | +``target-is-${arch}`` where ``${arch}}`` is one of the values from the |
| 55 | +following lines of the CMake output:: |
| 56 | + |
| 57 | + -- Compiler-RT supported architectures: x86_64;i386 |
| 58 | + -- Builtin supported architectures: i386;x86_64 |
| 59 | + |
| 60 | +So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail |
| 61 | +on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386`` |
| 62 | +will mark a test as expected to fail on i386 even if the default target triple |
| 63 | +is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string |
| 64 | +require exact matches so ``XFAIL: target-is-mips``, |
| 65 | +``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and |
| 66 | +``XFAIL: target-is-mips64el`` all refer to different MIPS targets. |
0 commit comments