Skip to content

Latest commit

 

History

History
91 lines (60 loc) · 2.5 KB

chapter-testsuites.pod

File metadata and controls

91 lines (60 loc) · 2.5 KB

Test Suite Wrappers

This section is about the test suites and wrappers around existing suites. These wrappers are part of our overall test infrastructure.

It's basically about the middle part in the following picture:

We have wrappers for existing test and benchmark suites.

Wrappers just run the suites as a user would manually run them but additionally extract results and produce TAP (Test Anything Protocol).

We have some specialized, small test suites that complement the general suites, e.g. for extracting meta information or parsing logs for common problems.

If the environment variables

TAPPER_REPORT_SERVER
TAPPER_REPORT_PORT

are set the wrappers report their results by piping their TAP output there, else they print to STDOUT.

Available test suite wrappers

Originally we have a lot of direct wrappers available but haven't them all published as open source. For OS testing the most important wrapper which is also publicly available is tapper-testsuite-autotest aka. Tapper-Testsuite-AutoTest. You should look at that.

tapper-testsuite-autotest

A suite that wraps the autotest client with the export of TAP and sends the resulting TAP archives to Tapper server.

That is the primary testsuite wrapper for OS testing.

Environment variables

The TAPPER automation layer provides some environment variables that the wrappers can use:

  • TAPPER_TESTRUN Currently active Testrun ID.

  • TAPPER_SERVER The controlling automation Server that initiated this testrun.

  • TAPPER_REPORT_SERVER The target server to which the tests should report their results in TAP.

  • TAPPER_REPORT_PORT The target port to which the tests should report their results in TAP. Complements TAPPER_REPORT_SERVER.

  • TAPPER_REPORT_API_PORT The port on which the more sophisticated Remote Reports API is available. It's running on the same host as TAPPER_REPORT_SERVER.

  • TAPPER_TS_RUNTIME Maximum runtime after which the testprogram will not be restarted when it runs in a loop. (This is a more passive variant than a timeout.)

  • TAPPER_GUEST_NUMBER Virtualisation guests are ordered, this is the guest number or 0 if not a guest.

  • TAPPER_NTP_SERVER The server where to request NTP dates from.

These variables should be used in the TAP of the suite as @tapper headers. Important use-case is "report groups", see next chapter.