GNOME-DESKTOP-TESTING-RUNNER(1)

NAME
gnome-desktop-testing-runner, ginsttest-runner - run "as-installed" tests

SYNOPSIS
gnome-desktop-testing-runner [--dir=DIR] [--first-root] [--list] [--log-directory=DIR] [--parallel=PROC] [--quiet] [--report-directory=DIR] [--status=yes|no|auto] [--tap] [--timeout=SECONDS] PREFIX [PREFIX...] gnome-desktop-testing-runner --log-msgid=MSGID=MESSAGE ginsttest-runner OPTIONS

DESCRIPTION
gnome-desktop-testing-runner, also known as ginsttest-runner, runs "as-installed" tests. These tests are discovered using metadata in files named installed-tests/**/*.test, and are intended to check that a library or program is functioning correctly and has been installed correctly by a system integrator such as an OS distributor or system administrator.

Tests in this format are typically provided by GNOME-related libraries, but the concept, format and tools are not GNOME-specific and can be used by any software.

OPTIONS

--dir=DIR, -d DIR
    Look for test metadata in the installed-tests subdirectory of DIR, instead of using $XDG_DATA_DIRS. If repeated, each DIR is searched in order.

--first-root
    Stop after a directory that contains installed-tests has been encountered.

--list, -l
    Don't run any tests. Instead, list what would have been run on standard output, one per line, in the format NAME (PATH).

--log-directory=DIR, -L DIR
    Write the output of each test to a file DIR/NAME.txt.

--log-msgid=MSGID=MESSAGE
    Don't run any tests. Instead, emit a log message to the systemd Journal with the unique machine-readable message ID MSGID and the human-readable message MESSAGE. This can be used to mark important points in the test log from a test written in a language where direct access to the Journal is awkward, such as shell script or JavaScript.

--parallel=PROC, -p PROC
    Run up to PROC tests in parallel. The default is 1, meaning do not run tests in parallel. If PROC is 0, detect the number of CPUs and run that many tests in parallel.

--quiet
    Don't output test results, just log them to the systemd Journal if supported.

--report-directory=DIR
    Run each test with DIR/NAME as its current working directory, and write its output to a file DIR/NAME/output.txt. If the test succeeds, the directory is deleted. If the test fails, the directory is kept for analysis, including any temporary files or logs that the test itself might have written there.

--status=yes|no|auto
    Output a status message every few seconds if a test takes a significant time to run.

--tap
    Output machine-readable test results on standard output, in the format specified by TAP (the Test Anything Protocol (https://testanything.org/) originally used by Perl's test suite).

--timeout=SECONDS, -t SECONDS
    If a test takes longer than SECONDS seconds to run, terminate it. The default is 5 minutes (300 seconds).

PREFIX [PREFIX...]
    Only list or run tests that match one of these prefixes, relative to the installed-tests directory.

EXIT STATUS

0   All tests were successful, or --list or --log-msgid was successful

1   An error occurred during options parsing, setup or testing

2   The tests were run, and least one failed

ENVIRONMENT

XDG_DATA_DIRS
    Used to discover tests if --dir, -d is not specified.

FILES

/usr/local/share/installed-tests/**/*.test
    Conventional location for metadata describing tests installed by locally-installed software.

/usr/share/installed-tests/**/*.test
    Conventional location for metadata describing tests installed by the operating system packages.

/usr/local/libexec/installed-tests/**, /usr/libexec/installed-tests/**
    Conventional location for test executables and the data files they require (although this is not required, and they can be installed in any convenient location).

./.testtmp
    Each test will be invoked in a temporary directory containing only this file. Tests can use this to avoid overwriting important files if run without using gnome-desktop-testing-runner.

EXAMPLE
To run the tests from the json-glib library:
    ginsttest-runner json-glib-1.0/

To run the tests from the dbus, glib and json-glib libraries, with up to one process per CPU, producing machine-readable output and storing results of any failed tests in a directory:
    ginsttest-runner \
        --parallel=0 \
        --report-directory=artifacts \
        --tap \
        dbus/ glib/ json-glib-1.0/

SEE ALSO
GNOME Goal: Installed Tests (https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests), Test Anything Protocol (https://testanything.org/)