Litani End-to-End Test Suite
                          ============================


Each file in the `tests` subdirectory contains a test definition. The `run`
script imports a single one of those files, and uses the test definition to
execute a complete Litani run. Once the run completes, the `run` script then
loads the resulting `run.json` file and checks that it conforms to the test
definition.

Each "test definition" is a Python module that is expected to contain certain
methods; the `run` script calls into each of these methods in order to set up,
execute, and check the Litani run. Those methods are as follows:


    get_init_args()
        Returns a dict containing command-line arguments to be passed to `litani
        init`. The dict must contain two keys, `args` and `kwargs`. `args` is a
        list of switches (command-line options that don't take an argument),
        while `kwargs` is a dict mapping command-line options to the value(s)
        that they take as an argument. The keys or list items should be equal to
        the name of the command-line switch, without the leading `--`. Values of
        the `kwargs` dict can be a string or a list of strings. If a list of
        strings, these will be joined together with whitespace.

        For example, to make the run script run

            `litani init --pools foo:1 bar:2 --project-name baz`

        this method could return the dict

            {
                "kwargs": {
                    "pools": [ "foo:1", "bar:2" ],
                    "project-name": "baz",
                }
            }

          The `kwargs` dict must not contain "output-symlink",
          "output-directory", or "output-prefix", as the `run` script will
          supply its own value for these.


    get_jobs()
        Returns a list of jobs that will be added to Litani. Each job is similar
        to the return of `get_init_args()` above, i.e. it's a dict containing
        command-line arguments. The run script will add each job before
        executing the run. For example, to make the run script run

            `litani add-job --command true --interleave-stdout-stderr`
            `litani add-job --command false --description "oh no!"`

        this method should return

            [{
                "args": [ "interleave-stdout-stderr" ],
                "kwargs": { "command": "true" }
            }, {
                "kwargs": { "command": "false", "description": "oh no!" }
            }]


    def get_run_build_args()
        Returns a dict containing the arguments to `litani run-build`, in the
        same format as `get_init_args()` above.


    check_run(run: dict)
        Takes a deserialized run.json file and returns True iff the Litani run
        was as expected. This method can check that particular keys of the run
        have certain values, etc.