This mirrors the existing archive_extract and cpio_extract helpers
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-19-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
More uncompress related code will be added shortly, so having a
separate file makes more sense.
The utils.py imports the functions from archive.py, so that
existing callers don't need to be modified. This avoids
redundant code churn until later in the series when all
calls will be adapted for other reasons.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-18-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
More archive related code will be added shortly, so having a
separate file makes more sense.
The utils.py imports the functions from archive.py, so that
existing callers don't need to be modified. This avoids
redundant code churn until later in the series when all
calls will be adapted for other reasons.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-17-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Everything in the scratch directory is automatically purged. Calling
'rmtree' again breaks the ability to optionally preserve the scratch
directory contents.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-16-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Replace any instances of
os.path.join(self.workdir, ".../...")
self.workdir + "/.../..."
with
self.scratch_file("...", "...")
which is more compact and portable
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-15-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This removes direct path manipulation to figure out the source dir
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-14-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This removes direct access of the 'BUILD_DIR' variable.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-13-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This removes direct access of the 'self.logdir' variable.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-12-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add helper methods that construct paths for
* log files - to be preserved at the end of a test
* scratch files - to be purged at the end of a test
* build files - anything relative to the build root
* data files - anything relative to the functional test source root
* socket files - a short temporary dir to avoid UNIX socket limits
These are to be used instead of direct access to the self.workdir,
or self.logdir variables, or any other place where paths are built
manually.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-11-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'which' helper is simpler, not depending on the external 'which'
binary, and is sufficient for test needs.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-10-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This ensures consistency of behaviour across all the tests, and requires
that we provide gitlab bug links when marking a test to be skipped due
to unreliability.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-9-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reduce repeated boilerplate with some helper decorators:
@skipIfNotPlatform("x86_64", "aarch64")
=> Skip unless the build host platform matches
@skipIfMissingCommands("mkisofs", "losetup")
=> Skips unless all listed commands are found in $PATH
@skipIfMissingImports("numpy", "cv2")
=> Skips unless all listed modules can be imported
@skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/NNN")
=> Skips unless env var requests flaky tests with the
reason documented in the referenced gitlab bug
@skipBigData
=> Skips unless env var permits tests creating big data files
@skipUntrustedTest
=> Skips unless env var permits tests which are potentially
dangerous to the host
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-8-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Platforms we target have new enough tesseract that it suffices to merely
check if the binary exists.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241217155953.3950506-7-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'access' check implies the file exists.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-6-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Put the 'which' function into shared code.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Allow an Asset object to be used in place of a filename but
making its string representation resolve to the cache file
path.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241217155953.3950506-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Identified using 'pylint --disable=all --enable=W0611'
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241217155953.3950506-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Tests are expected to be directly invoked when debugging so must
have execute permission.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241217155953.3950506-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Update the URLs for the binaries we use for the firmware in the
sbsa-ref functional tests.
The firmware is built using Debian 'bookworm' cross toolchain (gcc
12.2.0).
Used versions:
- Trusted Firmware v2.12.0
- Tianocore EDK2 stable202411
- Tianocore EDK2 Platforms code commit 4b3530d
This allows us to move away from "some git commit on trunk"
to a stable release for both TF-A and EDK2.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20241125125448.185504-1-marcin.juszkiewicz@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The xorriso command directly writes to 'filename', so the surrounding
'with' statement is pointless.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241129173120.761728-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241129173120.761728-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion, just the hashsums needed to be
updated to sha256 now.
These were the last tests that used image_pow2ceil_expand in
boot_linux_console.py, so we can remove that function from that
file now, too.
Message-ID: <20241206102358.1186644-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
A straight forward conversion, just the hashsums needed to be
updated to sha256 now.
Message-ID: <20241206102358.1186644-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
A pretty straight-forward conversion of the emcraft_sf2 boot
test to the functional framework.
This was the last test that used file_truncate() in
boot_linux_console.py, so we can remove that function from that
file now, too.
Message-ID: <20241206102358.1186644-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
A straight-forward conversion of the xlnx_versal_virt boot
test to the functional framework.
Message-ID: <20241206102358.1186644-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When running "make -j$(nproc) check SPEED=thorough", the sh4_tuxrun
test is timing out for me, and using TIMEOUT_MULTIPLIER I can see
that it clearly takes more than 100 seconds to finish. Thus increase
the timeout setting of this test to avoid the problem.
Message-ID: <20241204070757.663119-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This simply moves the debian boot test from the avocado testsuite to
the new functional testsuite. No changes in the test.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20241206131132.520911-8-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
This simply moves the rainier-bmc test to a new test file. No changes
in the test. The test_arm_aspeed.py is deleted.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20241206131132.520911-7-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
This moves the ast2600-evb tests to a new test file. No changes in the
test. The routines used to run the buildroot and sdk tests are removed
from the test_arm_aspeed.py file because now unused.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20241206131132.520911-6-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
This moves the ast2500-evb tests to a new test file and extends the
aspeed module with routines used to run the buildroot and sdk
tests. No changes in the test.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20241206131132.520911-5-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
This simply moves the romulus-bmc test to a new test file. No changes
in the test. The do_test_arm_aspeed routine is removed from the
test_arm_aspeed.py file because it is now unused.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20241206131132.520911-4-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
This introduces a new aspeed module for sharing code between tests and
moves the palmetto test to a new test file. No changes in the test.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/r/20241206131132.520911-3-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
When under high load the test VM does not complete running in the
default 30 second timeout. Double it to give more headroom.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241129173120.761728-2-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use self_set_machine() to set and check for the availability of the
default pc machine (so that the test is not failing if the machine
has not been included in the QEMU binary).
Message-ID: <20241128120142.593408-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
By using QemuSystemTest as a base class, we can use the set_machine()
command to check whether the required machine is available in the
binary (otherwise this test is failing when QEMU has been compiled
without the default 'pc' machine type).
Message-ID: <20241128115019.591362-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Ani Sinha <anisinha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
These were introduced in the avocado tests to workaround read issues
when interacting with console. They are no longer necessary and we can
use the expected login string instead.
Test always passes now. Remove skipUnless test on QEMU_TEST_FLAKY_TESTS.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241122141827.2039984-1-clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
There is a bug in the process of resolving the serial port base address
in the fdt of the loongarch VM UEFI. When both serial port information
and rng-seed information are chosen in the fdt, there is a probability
that the serial port base address cannot be resolved correctly.
This problem can be fixed by updating UEFI.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2686
Signed-off-by: Xianglai Li <lixianglai@loongson.cn>
Message-ID: <20241127013438.2206426-1-lixianglai@loongson.cn>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
These were introduced in the avocado tests to workaround read issues
when interacting with console. They are no longer necessary and we can
use the expected "login:" string or the command prompt now. Drop the
last use of exec_command.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20241122090322.1934697-4-clg@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Drop the SSH connection which was introduced in the avocado tests to
workaround read issues when interacting with console.
EXTRA_BOOTARGS was introduced to reduce the console output at Linux
boot time. This didn't have the desired effect as we still had issues
when trying to match patterns on the console and we had to use the ssh
connection as a workaround.
While at it, remove the U-Boot EXTRA_BOOTARGS variable which has
become useless.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20241122090322.1934697-3-clg@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Drop the SSH connection which was introduced in the avocado tests to
workaround read issues when interacting with console.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20241122090322.1934697-2-clg@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Now there are new up to date images available we should update to them.
With the new rootfs the blk I/O errors also go away on arm64be.
Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-22-alex.bennee@linaro.org>
Move the tests to a new file so that they can be run via
qemu-system-aarch64 in the functional framework.
Since these were the last tests in tests/avocado/tuxrun_baselines.py,
we can now remove that file, too.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-20-alex.bennee@linaro.org>
If a failure occurs early in the QemuBaseTest constructor, the
'log_filename' object atttribute may not exist yet. This happens
most notably if the QEMU_TEST_QEMU_BINARY is not set. We can't
initialize 'log_filename' earlier as we use the binary to identify
the architecture which is then used to build the path in which the
logs are stored.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-19-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-19-alex.bennee@linaro.org>
Support the QEMU_TEST_QMP_BACKDOOR=backdoor.sock env variable as a
way to get a QMP backdoor for debugging a stalled QEMU test. Most
typically this would be used if running the tests directly:
$ QEMU_TEST_QMP_BACKDOOR=backdoor.sock \
QEMU_TEST_QEMU_BINARY=./build/qemu-system-arm \
PYTHONPATH=./python \
./tests/functional/test_arm_tuxrun.py
And then, when the test stalls, in a second shell run:
$ ./scripts/qmp/qmp-shell backdoor.sock
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-18-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-18-alex.bennee@linaro.org>
The tuxrun tests send a series of strings to the guest to login
and then run commands. Since we have been unable to match on
console output that isn't followed by a newline, the test used
many time.sleep() statements to pretend to synchronize with
the guest.
This has proved to be unreliable for the aarch64be instance of
the tuxrun tests, with the test often hanging. The hang is a
very subtle timing problem, and it is suspected that some
(otherwise apparently harmless) I/O error messages could be
resulting in full FIFO buffers, stalling interaction with
the guest.
With the newly rewritten console interaction able to match
strings that don't have a following newline, the tux run
tests can now match directly on the login prompt, and/or
shell PS1 prompt.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2689
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-17-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-17-alex.bennee@linaro.org>
The console interaction that waits for predicted strings uses
readline(), and thus is only capable of waiting for strings
that are followed by a newline.
This is inconvenient when needing to match on some things,
particularly login prompts, or shell prompts, causing tests
to use time.sleep(...) instead, which is unreliable.
Switch to reading the console 1 byte at a time, comparing
against the success/failure messages until we see a match,
regardless of whether a newline is encountered.
The success/failure comparisons are done with the python bytes
type, rather than strings, to avoid the problem of needing to
decode partially received multibyte utf8 characters.
Heavily inspired by a patch proposed by Cédric, but written
again to work in bytes, rather than strings.
Co-developed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-16-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-16-alex.bennee@linaro.org>
When waiting for expected output, the 'success_message' is a mandatory
parameter, with 'failure_message' defaulting to None.
The code has logic which indicates it was trying to cope with
'success_message' being None and 'failure_message' being non-None but
it does not appear able to actually do anything useful. The check for
'success_message is None' will break out of the loop before any check
for 'failure_message' has been performed.
IOW, for practcal purposes 'success_message' must be non-None unless
'send_string' is set. Assert this expectation and simplify the loop
logic.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-15-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-15-alex.bennee@linaro.org>
Telling exec_command_wand_wait_for_pattern to wait for the empty
string does not make any conceptual sense, as a check for empty
string will always succeed. It makes even less sense when followed
by a call to wait_for_console_pattern() with a real match.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-14-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-14-alex.bennee@linaro.org>
