311 Commits

Author SHA1 Message Date
Pierrick Bouvier
8bdd3a0308 tests/functional/test_aarch64_replay: reenable on macos
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250410225550.46807-3-pierrick.bouvier@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2025-04-14 11:03:16 -04:00
Philippe Mathieu-Daudé
bd20bc46fe tests/functional: Skip aarch64_replay test on macOS
As of v10.0.0-rc2 this test is still failing on macos:

  $ make check-functional-aarch64 V=1
  ...
  ERROR:../../replay/replay-internal.c:235:replay_mutex_unlock: assertion failed: (replay_mutex_locked())
  Bail out! ERROR:../../replay/replay-internal.c:235:replay_mutex_unlock: assertion failed: (replay_mutex_locked())

This is tracked as https://gitlab.com/qemu-project/qemu/-/issues/2907

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-4-philmd@linaro.org>
2025-04-03 16:11:09 -07:00
Philippe Mathieu-Daudé
00f119f4c4 tests/functional: Add a decorator for skipping tests on particular OS
Since tests might be failing on some operating systems,
introduce the skipIfOperatingSystem() decorator.

Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-3-philmd@linaro.org>
2025-04-03 16:11:09 -07:00
Pierrick Bouvier
4412d71382 tests/functional/test_aarch64_rme_virt: fix sporadic failure
This test was randomly failing on our CI, and on dev machines,
especially with QEMU debug builds.

>From the information collected, it's related to an implementation choice
in edk2 QEMU virt support. The workaround is to disable KASLR, to avoid
accessing protected memory.
Note: this is *not* needed for the similar test_aarch64_rme_sbsaref.

More information is available on the associated GitLab issue.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2823
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250328183816.2687925-1-pierrick.bouvier@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250403203241.46692-2-philmd@linaro.org>
2025-04-03 16:11:09 -07:00
Thomas Huth
5a8d024f8d tests/functional/test_vnc: Skip test if VNC support is not available
These tests currently fail if VNC support has not been compiled into
the QEMU binary. Let's add some checks to skip the tests in that
case instead.

Message-ID: <20250325064715.278876-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
c3612d0b6a tests/functional/test_aarch64_virt_gpu: Skip if "dbus" display isn't available
This test currently fails if the "dbus" display has not been compiled
into the binary (which can happen when CFI has been enabled, for example).
Check for the error message to skip the test in that case.

While we're at it, also make sure that this test is covered in the
right section in the MAINTAINERS file.

Message-ID: <20250325061609.272847-1-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
dba0752f2c tests/functional: Add missing require_netdev('user') statements
A bunch of tests are using "-netdev user" but fail to check
for the availability of SLIRP in the binary, so these tests
fail if QEMU has been configured with "--disable-slirp"
(most of the tests are disabled by default with a decorator,
that's likely why nobody noticed this problem yet). Add the
missing self.require_netdev('user') statements to skip the
tests if SLIRP is not available.

Message-ID: <20250324123450.111307-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Peter Maydell
f0a6b3ec6d tests/functional/meson.build: Bump arm_aspeed_bletchley timeout
The arm_aspeed_bletchley timeout takes more than its
current 120s timeout on a debug Rust config:

$ time (cd build/rust; PYTHONPATH=../../python:../../tests/functional QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3 ../../tests/functional/test_arm_aspeed_bletchley.py)
TAP version 13
ok 1 test_arm_aspeed_bletchley.BletchleyMachine.test_arm_ast2600_bletchley_openbmc
1..1

real    2m15.536s
user    3m21.444s
sys     0m11.558s

Bump it up to 480s, same as arm_aspeed_rainier.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250321155806.1888867-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
3b9731020d tests/functional/test_ppc64_replay: Mark the e500 test as flaky
The test_ppc64_e500 occasionally fails (it just hangs and finally times
out). The same issue could also be reproduced with the former Avocado
test already (especially if the host system is under heavy load), so
it's not a new regression. It's very likely the issue that has been
filed at https://gitlab.com/qemu-project/qemu/-/issues/2523 instead (e.g.
I could not reproduce the issue in older commits before commit 578912ad),
so use this URL for the reasoning in the the decorator.

Message-ID: <20250320065012.309520-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-27 08:59:35 +01:00
Thomas Huth
f700abbbeb tests/functional/test_migration: Use "ncat" instead of "nc" in the exec test
"nc" can either be GNU netcat, OpenBSD netcat or NMap ncat. At
least GNU netcat currently does not work with this test anymore,
though the comment in the test says otherwise. GNU netcat seems
to be quite unmaintained nowadays, according to its website
(https://netcat.sourceforge.net/), the last public release is from
2004, so we should rather avoid that binary.
In our CI, we are only using "ncat" in the containers (it's the only
flavor that lcitool supports), thus to avoid silent regressions with
the other netcats, let's limit this test to "ncat" only now.

Reported-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Message-ID: <20250319105617.133191-1-thuth@redhat.com>
Reviewed-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 13:28:34 +01:00
Thomas Huth
1ddd4d3f6e tests/functional/test_x86_64_kvm_xen: Remove avocado tags
They have been forgotten to be removed when converting the
test to the functional framework. Since they are of no use
anymore, let's remove them now.

Message-ID: <20250318171530.94966-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 07:43:51 +01:00
Daniel P. Berrangé
23686dfb76 tests/functional: remove all class level fields
A number of fields are set at the class level on QemuBaseTest, even
though the exact same named field is then set at the object level
later in most cases.

The 'self.logger' initialization in ACPI bits test needs to be removed
since 'self.log' won't exist at that point in the flow. It already
initialized 'self.logger' later in the setUp() method, so the __init__
method was redundant.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250317124300.84266-1-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 07:41:32 +01:00
Niek Linnenbank
6d19d09547 tests/functional/test_arm_orangepi: rename test class to 'OrangePiMachine'
The test class in this file contains all functional test cases
for testing the Orange Pi PC board. It should be given a name
matching the Qemu machine it covers.

This commit sets the test class name to 'OrangePiMachine'.

Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Fixes: 380f7268b7b ("tests/functional: Convert the OrangePi tests to the  functional framework")
Message-ID: <20250316210232.46298-1-nieklinnenbank@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-19 07:41:32 +01:00
Stefan Hajnoczi
28ea66f6f9 * Various fixes for functional tests
* Fix the name of the "configs" directory in the documentation
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmfSjagRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbWBmA//RhAHuF/fTmQagBsZPETXjU1g8ifw9aqm
 WPZcQEXyQFlqYYQZmtV7dk3aTGEw4kBDmm+SKTSQz1yUcBGptMl8xuWaxgdpcOw0
 Bqt+lYNgwGL9/OocCdNolU3+aVbETljr5l+rzbnwsTVIqGk63Qhmtwdupb8h1nfY
 4vCXU+sY3BkvBF8HbV6Wb1aPtqC+iH/Ln8+yoKkC8UePD623dK58SsOVrhUQDfFr
 U/HUy4BZlHFCfGGmDVGBjHdEbOzQkLQ9N3ilsNSWcF87RPkWPft+qLs4RjDFW+oT
 oksXEFHcr8XQO03fwHBNTyv+NUfnrvDY8V+gl6C9ItQr58SZzse57caZKWrYppZ3
 l5iHoaLMV3juZFDNXNHkWHuveXi05+0V0UbZihzBeC4+zjNRyh3e1GuDoh5VoG8o
 XIb55RxU8eBG2/ulHZ71eAYrGpxO+tDdsdnak1coPFsU8HrC9QzRfywiAZe1Wwmx
 5t5AHbZ7RdnxgStU1lWTUT2IDVSini4DKevt/FzhKkv1aD8NbhI/ooGDC0zbS6SU
 XK6PP2G5a5OnjQ904oRCQbnhrxFa5qNfryylvvreT2bVgX0BiE4pJ9JXdgQOMYlP
 kZERZZQcv3y6VVavAT67yeNKQpyb4HSHdTDQ2irgXP1UwHRpwLpKdqB1UhzNJ8m8
 k0faA8RXir4=
 =VtGZ
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2025-03-13' of https://gitlab.com/thuth/qemu into staging

* Various fixes for functional tests
* Fix the name of the "configs" directory in the documentation

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmfSjagRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbWBmA//RhAHuF/fTmQagBsZPETXjU1g8ifw9aqm
# WPZcQEXyQFlqYYQZmtV7dk3aTGEw4kBDmm+SKTSQz1yUcBGptMl8xuWaxgdpcOw0
# Bqt+lYNgwGL9/OocCdNolU3+aVbETljr5l+rzbnwsTVIqGk63Qhmtwdupb8h1nfY
# 4vCXU+sY3BkvBF8HbV6Wb1aPtqC+iH/Ln8+yoKkC8UePD623dK58SsOVrhUQDfFr
# U/HUy4BZlHFCfGGmDVGBjHdEbOzQkLQ9N3ilsNSWcF87RPkWPft+qLs4RjDFW+oT
# oksXEFHcr8XQO03fwHBNTyv+NUfnrvDY8V+gl6C9ItQr58SZzse57caZKWrYppZ3
# l5iHoaLMV3juZFDNXNHkWHuveXi05+0V0UbZihzBeC4+zjNRyh3e1GuDoh5VoG8o
# XIb55RxU8eBG2/ulHZ71eAYrGpxO+tDdsdnak1coPFsU8HrC9QzRfywiAZe1Wwmx
# 5t5AHbZ7RdnxgStU1lWTUT2IDVSini4DKevt/FzhKkv1aD8NbhI/ooGDC0zbS6SU
# XK6PP2G5a5OnjQ904oRCQbnhrxFa5qNfryylvvreT2bVgX0BiE4pJ9JXdgQOMYlP
# kZERZZQcv3y6VVavAT67yeNKQpyb4HSHdTDQ2irgXP1UwHRpwLpKdqB1UhzNJ8m8
# k0faA8RXir4=
# =VtGZ
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 13 Mar 2025 15:47:52 HKT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2025-03-13' of https://gitlab.com/thuth/qemu:
  tests/functional: skip vulkan test if missing vulkaninfo
  tests/functional/asset: Add AssetError exception class
  tests/functional/asset: Verify downloaded size
  tests/functional/asset: Fail assert fetch when retries are exceeded
  docs/system: Fix the information on how to run certain functional tests
  tests/functional: Bump up arm_replay timeout
  tests/functional: Require 'user' netdev for ppc64 e500 test
  docs: Rename default-configs to configs

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2025-03-14 09:30:53 +08:00
Alex Bennée
b3c03666fb tests/functional: skip vulkan test if missing vulkaninfo
I could have sworn I had this is a previous iteration of the patches
but I guess it got lost in a re-base. As we are going to call
vulkaninfo to probe for "bad" drivers we need to skip if the binary
isn't available.

Fixes: 9f7e493d11 (tests/functional: skip vulkan tests with nVidia)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250312190314.1632357-1-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-13 08:18:26 +01:00
Nicholas Piggin
28adad0a4d tests/functional/asset: Add AssetError exception class
Assets are uniquely identified by human-readable-ish url, so make an
AssetError exception class that prints url with error message.

A property 'transient' is used to capture whether the client may retry
or try again later, or if it is a serious and likely permanent error.
This is used to retain the existing behaviour of treating HTTP errors
other than 404 as 'transient' and not causing precache step to fail.
Additionally, partial-downloads and stale asset caches that fail to
resolve after the retry limit are now treated as transient and do not
cause precache step to fail.

For background: The NetBSD archive is, at the time of writing, failing
with short transfer. Retrying the fetch at that position (as wget does)
results in a "503 backend unavailable" error. We would like to get that
error code directly, but I have not found a way to do that with urllib,
so treating the short-copy as a transient failure covers that case (and
seems like a reasonable way to handle it in general).

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-4-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:20:50 +01:00
Nicholas Piggin
7524e1b336 tests/functional/asset: Verify downloaded size
If the server provides a Content-Length header, use that to verify the
size of the downloaded file. This catches cases where the connection
terminates early, and gives the opportunity to retry. Without this, the
checksum will likely mismatch and fail without retry.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-3-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:20:50 +01:00
Nicholas Piggin
a5e8299d1a tests/functional/asset: Fail assert fetch when retries are exceeded
Currently the fetch code does not fail gracefully when retry limit is
exceeded, it just falls through the loop with no file, which ends up
hitting other errors.

Add a check for non-existing file, which indicates the retry limit was
exceeded.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250312130002.945508-2-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:20:50 +01:00
Peter Maydell
8c63f9aa3f tests/functional: Bump up arm_replay timeout
On my machine the arm_replay test takes over 2 minutes to run
in a config with Rust enabled and debug enabled:

$ time (cd build/rust ; PYTHONPATH=../../python:../../tests/functional
QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3
../../tests/functional/test_arm_replay.py)
TAP version 13
ok 1 test_arm_replay.ArmReplay.test_cubieboard
ok 2 test_arm_replay.ArmReplay.test_vexpressa9
ok 3 test_arm_replay.ArmReplay.test_virt
1..3

real    2m16.564s
user    2m13.461s
sys     0m3.523s

Bump up the timeout to 4 minutes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250310102830.3752440-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:15:44 +01:00
Cédric Le Goater
533b33d04b tests/functional: Require 'user' netdev for ppc64 e500 test
When commit 72cdd672e18c extended the ppc64 e500 test to add network
support, it forgot to require the 'user' netdev backend. Fix that.

Fixes: 72cdd672e18c ("tests/functional: Replace the ppc64 e500 advent calendar test")
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20250308071328.193694-1-clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-12 18:15:30 +01:00
Cédric Le Goater
e5029e28f0 ppc/ppc405: Remove tests
Since we are about to remove all support for PPC 405, start by
removing the tests referring to the ref405ep machine.

Link: https://lore.kernel.org/qemu-devel/20250110141800.1587589-2-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250204080649.836155-2-clg@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2025-03-11 22:40:47 +10:00
Pierrick Bouvier
2a8e8544b2 tests/functional: add boot error detection for RME tests
It was identified that those tests randomly fail with a synchronous
exception at boot (reported by EDK2).
While we solve this problem, report failure immediately so tests don't
timeout in CI.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20250303185745.2504842-1-pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-20-alex.bennee@linaro.org>
2025-03-10 10:30:01 +00:00
Philippe Mathieu-Daudé
ed557cc544 tests/functional: Allow running TCG plugins tests on non-Linux/BSD hosts
Not all platforms use the '.so' suffix for shared libraries,
which is how plugins are built. Use the recently introduced
dso_suffix() helper to get the proper host suffix.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2804
Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-4-philmd@linaro.org>
[AJB: moved plugin_file into testcase.py]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-11-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Philippe Mathieu-Daudé
3a7b9054b8 tests/functional: Introduce the dso_suffix() helper
Introduce a helper to get the default shared library
suffix used on the host.

Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250220080215.49165-3-philmd@linaro.org>
[AJB: dropped whitespace cmd.py damage]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-10-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Pierrick Bouvier
892b06c40e plugins: add explicit dependency in functional tests
./tests/functional/test_aarch64_tcg_plugins.py needs to have plugin
libinsn built. However, it's not listed as a dependency, so meson can't
know it needs to be built.

Thus, we keep track of all plugins, and add them as an explicit
dependency.

Fixes: 4c134d07b9e ("tests: add a new set of tests to exercise plugins")
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-9-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
d69178d370 tests/functional: update the aarch64_virg_gpu images
Update to the most recent aarch64_virt_gpu image. The principle
differences are:

  - target a v8.0 baseline CPU
  - latest vkmark (2025.1)
  - actually uses the rootfs (previously was initrd)
  - rootfs includes more testing tools for interactive use

See README.md in https://fileserver.linaro.org/s/ce5jXBFinPxtEdx for
details about the image creation and the buildroot config.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-8-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
8233f4f26d tests/functional: expand tests to cover virgl
Add two more test modes using glmark2-wayland to exercise the OpenGL
pass-through modes with virgl. Virgl can run with or without the
hostmem blob support. To avoid repeating ourselves too much we make
the initial pass a simple --validate pass.

We might want to eventually add more directed tests and individual
features later on but the glmark/vkmark tests are a good general
smoke test for accelerated 3D.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-7-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
9f7e493d11 tests/functional: skip vulkan tests with nVidia
While running the new GPU tests it was noted that the proprietary
nVidia driver barfed when run under the sanitiser:

  2025-02-20 11:13:08,226: [11:13:07.782] Output 'headless' attempts
  EOTF mode SDR and colorimetry mode default.
  2025-02-20 11:13:08,227: [11:13:07.784] Output 'headless' using color
  profile: stock sRGB color profile

  and that's the last thing it outputs.

  The sanitizer reports that when the framework sends the SIGTERM
  because of the timeout we get a write to a NULL pointer (but
  interesting not this time in an atexit callback):

  UndefinedBehaviorSanitizer:DEADLYSIGNAL
  ==471863==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address
  0x000000000000 (pc 0x7a18ceaafe80 bp 0x000000000000 sp 0x7ffe8e3ff6d0
  T471863)
  ==471863==The signal is caused by a WRITE memory access.
  ==471863==Hint: address points to the zero page.
      #0 0x7a18ceaafe80
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x16afe80)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #1 0x7a18ce9e72c0
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15e72c0)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #2 0x7a18ce9f11bb
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x15f11bb)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #3 0x7a18ce6dc9d1
  (/lib/x86_64-linux-gnu/libnvidia-eglcore.so.535.183.01+0x12dc9d1)
  (BuildId: 24b0d0b90369112e3de888a93eb8d7e00304a6db)
      #4 0x7a18e7d15326 in vrend_renderer_create_fence
  /usr/src/virglrenderer-1.0.0-1ubuntu2/obj-x86_64-linux-gnu/../src/vrend_renderer.c:10883:26
      #5 0x55bfb6621871 in virtio_gpu_virgl_process_cmd

The #dri-devel channel confirmed:

  <digetx> stsquad: nv driver is known to not work with venus, don't use
      it for testing

So lets skip running the test to avoid known failures. As we now use
vulkaninfo to probe we also need to handle the case where there is no
Vulkan driver configured for the hardware.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Cc: Dmitry Osipenko <dmitry.osipenko@collabora.com>
[AJB: also skip if vulkaninfo can't find environment]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-6-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
4e4e6986b6 tests/functional: bail early if vkmark hangs
The message:

  MESA-VIRTIO: debug: stuck in fence wait with iter at %d

Seems to occur more often on debug builds. Rather than waiting for our
long timeout to hit we might as well bail as soon as we see the
message.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-5-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
94634fbc66 tests/functional: ensure we have a GPU device for tests
It's possible to build QEMU without support for the GL enabled GPU
devices and we can catch that earlier with an explicit check.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-4-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
868c7703e0 tests/functional: factor out common code in gpu test
In preparation for handling more tests split out the common machine
setup details from the test specific stuff and add a helper for
launching the weston test. Instead of searching for "vkmark score" we
set a custom PS1 and wait for a successful completion. This ensures we
capture the score in the console log which otherwise wouldn't log
anything.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-3-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Alex Bennée
9a72282560 tests/functional: move aarch64 GPU test into own file
I want to expand the number of tests to cover a wide range of
configurations. That starts with splitting off from the normal virt
test from which it doesn't really share much code. We can also reduce
the timeout of the original virt test now it is now longer burdened
with testing the GPU.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-2-alex.bennee@linaro.org>
2025-03-10 10:30:00 +00:00
Jamin Lin
2cab06e930 tests/functional/aspeed: Add test case for AST2700 A1
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-29-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Jamin Lin
c77ec95a4b tests/functional/aspeed: Update test ASPEED SDK v09.05
In ASPEED SDK v09.05, the naming convention for pre-built images has been
updated. The pre-built image for AST2700 A0 has been renamed to
ast2700-a0-default, while ast2700-default is now used for AST2700 A1.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-28-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Jamin Lin
8241d6f8d5 tests/functional/aspeed: Update temperature hwmon path
Modified the temperature hwmon path to use a wildcard to handle different SDK
versions: "cat /sys/bus/i2c/devices/1-004d/hwmon/hwmon*/temp1_input".

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-27-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Jamin Lin
827eecb0e8 tests/functional/aspeed: Introduce start_ast2700_test API
Added a new method "start_ast2700_test" to the "AST2x00MachineSDK" class and
this method centralizes the logic for starting the AST2700 test, making it
reusable for different test cases.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250307035945.3698802-26-jamin_lin@aspeedtech.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:53 +01:00
Cédric Le Goater
b91a1d3110 tests/functional: Introduce a bletchley machine test
Use do_test_arm_aspeed_openbmc() to run the latest OpenBMC firmware
build of the bletchley BMC.

Reviewed-by: Patrick Williams <patrick@stwcx.xyz>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-6-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:52 +01:00
Cédric Le Goater
3058b634f2 tests/functional: Introduce a witherspoon machine test
Use do_test_arm_aspeed_openbmc() routine to run the latest OpenBMC
firmware build of the witherspoon BMC.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-5-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:52 +01:00
Cédric Le Goater
01050d97e1 tests/functional: Update OpenBMC image of romulus machine
Use the new do_test_arm_aspeed_openbmc() routine to run the latest
OpenBMC firmware build of the romulus BMC. Remove the older routine
which is now unused.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-4-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:52 +01:00
Cédric Le Goater
6664b3e255 tests/functional: Update OpenBMC image of palmetto machine
Use the new do_test_arm_aspeed_openbmc() routine to run the latest
OpenBMC firmware build of the palmetto BMC.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-3-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:52 +01:00
Cédric Le Goater
ae139d6e92 tests/functional: Introduce a new test routine for OpenBMC images
The OpenBMC images currently used by QEMU to test the Aspeed machines
are rather old. To prepare an update to the latest builds, we need to
adjust the console patterns. Introduce a new routine to preserve the
current tests.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-2-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
2025-03-09 14:36:52 +01:00
Stefan Hajnoczi
ffbc5e661f target-arm queue:
* hw/arm/smmu-common: Remove the repeated ttb field
  * hw/gpio: npcm7xx: fixup out-of-bounds access
  * tests/functional/test_arm_sx1: Check whether the serial console is working
  * target/arm: Fix minor bugs in generic timer register handling
  * target/arm: Implement SEL2 physical and virtual timers
  * target/arm: Correct STRD, LDRD atomicity and fault behaviour
  * target/arm: Make dummy debug registers RAZ, not NOP
  * util/qemu-timer.c: Don't warp timer from timerlist_rearm()
  * include/exec/memop.h: Expand comment for MO_ATOM_SUBALIGN
  * hw/arm/smmu: Introduce smmu_configs_inv_sid_range() helper
  * target/rx: Set exception vector base to 0xffffff80
  * target/rx: Remove TCG_CALL_NO_WG from helpers which write env
 -----BEGIN PGP SIGNATURE-----
 
 iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAmfLCzgZHHBldGVyLm1h
 eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3pwVEACgJJm1zdtRd87AnA0eY29a
 uG8M35+VS/bNbA6IXzz1hFHUFh1smrda0C7VOefRqThEhkDObh1gfKWK3YeBenDn
 FQsI6Hwu23ozTCgOniheU8SGbTtIvVxRRX4S91xNZgJ15riEATDnTisZv8iUChdr
 DcZopuH0uRiOq7TWuRjxqvhaqH6WusvHzK0mizTqr9UhbqPHVl7CZfr1/AtJLpZF
 32ix0JMofFWS52LFI19KWPlQG5Z3+lOw2ASyTf4cCaoCG6FTMv22E1x8mbMc2i96
 WrsB+NdhlBVRu7mskOP2Br09AbQZ/Fy7AGlDhgZebipOUVMlpDj1RXj/BDH3H/px
 qsjOk3V3gzM2bD+KvJuO4FlGXgEbOzGsGBwwY152C/6DYW5uTha/H1Pp+/iR8kcS
 HvAsqNLh/uF7O1Kn8qzCNvglKDC3z0C4X15Sj8SjGz8Xtn1Ign/GVkDv8ZCoR39K
 ltnXwvhzlDMkcGFFfEn33MYZZYqB15nX5a78/cStB/aOGPtZwUJ+2udLDzmug5ve
 9oY9WMqqBDVxo4+qcAeZ+aem2VD6w79mhJyy1xmqOkifhFWqQ2VbDrKtqnrqhPK/
 neyWrd2zCF6fY1wvb7vVKMy7aC5jI2K6qVU7ueZGCGKU2MtvbVaFJFByOjnVjv6o
 c65VNXkbaCIedrSlalMO4w==
 =8typ
 -----END PGP SIGNATURE-----

Merge tag 'pull-target-arm-20250307' of https://git.linaro.org/people/pmaydell/qemu-arm into staging

target-arm queue:
 * hw/arm/smmu-common: Remove the repeated ttb field
 * hw/gpio: npcm7xx: fixup out-of-bounds access
 * tests/functional/test_arm_sx1: Check whether the serial console is working
 * target/arm: Fix minor bugs in generic timer register handling
 * target/arm: Implement SEL2 physical and virtual timers
 * target/arm: Correct STRD, LDRD atomicity and fault behaviour
 * target/arm: Make dummy debug registers RAZ, not NOP
 * util/qemu-timer.c: Don't warp timer from timerlist_rearm()
 * include/exec/memop.h: Expand comment for MO_ATOM_SUBALIGN
 * hw/arm/smmu: Introduce smmu_configs_inv_sid_range() helper
 * target/rx: Set exception vector base to 0xffffff80
 * target/rx: Remove TCG_CALL_NO_WG from helpers which write env

# -----BEGIN PGP SIGNATURE-----
#
# iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAmfLCzgZHHBldGVyLm1h
# eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3pwVEACgJJm1zdtRd87AnA0eY29a
# uG8M35+VS/bNbA6IXzz1hFHUFh1smrda0C7VOefRqThEhkDObh1gfKWK3YeBenDn
# FQsI6Hwu23ozTCgOniheU8SGbTtIvVxRRX4S91xNZgJ15riEATDnTisZv8iUChdr
# DcZopuH0uRiOq7TWuRjxqvhaqH6WusvHzK0mizTqr9UhbqPHVl7CZfr1/AtJLpZF
# 32ix0JMofFWS52LFI19KWPlQG5Z3+lOw2ASyTf4cCaoCG6FTMv22E1x8mbMc2i96
# WrsB+NdhlBVRu7mskOP2Br09AbQZ/Fy7AGlDhgZebipOUVMlpDj1RXj/BDH3H/px
# qsjOk3V3gzM2bD+KvJuO4FlGXgEbOzGsGBwwY152C/6DYW5uTha/H1Pp+/iR8kcS
# HvAsqNLh/uF7O1Kn8qzCNvglKDC3z0C4X15Sj8SjGz8Xtn1Ign/GVkDv8ZCoR39K
# ltnXwvhzlDMkcGFFfEn33MYZZYqB15nX5a78/cStB/aOGPtZwUJ+2udLDzmug5ve
# 9oY9WMqqBDVxo4+qcAeZ+aem2VD6w79mhJyy1xmqOkifhFWqQ2VbDrKtqnrqhPK/
# neyWrd2zCF6fY1wvb7vVKMy7aC5jI2K6qVU7ueZGCGKU2MtvbVaFJFByOjnVjv6o
# c65VNXkbaCIedrSlalMO4w==
# =8typ
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 07 Mar 2025 23:05:28 HKT
# gpg:                using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg:                issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [full]
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>" [full]
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [full]
# gpg:                 aka "Peter Maydell <peter@archaic.org.uk>" [unknown]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* tag 'pull-target-arm-20250307' of https://git.linaro.org/people/pmaydell/qemu-arm: (21 commits)
  target/rx: Remove TCG_CALL_NO_WG from helpers which write env
  target/rx: Set exception vector base to 0xffffff80
  hw/arm/smmu: Introduce smmu_configs_inv_sid_range() helper
  include/exec/memop.h: Expand comment for MO_ATOM_SUBALIGN
  util/qemu-timer.c: Don't warp timer from timerlist_rearm()
  target/arm: Make dummy debug registers RAZ, not NOP
  target/arm: Drop unused address_offset from op_addr_{rr, ri}_post()
  target/arm: Correct STRD atomicity
  target/arm: Correct LDRD atomicity and fault behaviour
  hw/arm: enable secure EL2 timers for sbsa machine
  hw/arm: enable secure EL2 timers for virt machine
  target/arm: Document the architectural names of our GTIMERs
  target/arm: Implement SEL2 physical and virtual timers
  target/arm: Refactor handling of timer offset for direct register accesses
  target/arm: Always apply CNTVOFF_EL2 for CNTV_TVAL_EL02 accesses
  target/arm: Make CNTPS_* UNDEF from Secure EL1 when Secure EL2 is enabled
  target/arm: Don't apply CNTVOFF_EL2 for EL2_VIRT timer
  target/arm: Apply correct timer offset when calculating deadlines
  tests/functional/test_arm_sx1: Check whether the serial console is working
  hw/gpio: npcm7xx: fixup out-of-bounds access
  ...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2025-03-08 11:31:41 +08:00
Thomas Huth
5f6b9b0564 tests/functional/test_arm_sx1: Check whether the serial console is working
The kernel that is used in the sx1 test prints the usual Linux log
onto the serial console, but this test currently ignores it. To
make sure that the serial device is working properly, let's check
for some strings in the output here.

While we're at it, also add the test to the corresponding section
in the MAINTAINERS file.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20250226104833.1176253-1-thuth@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-03-07 10:08:19 +00:00
Thomas Huth
4dc11ee468 tests/functional/test_virtio_balloon: Only use KVM for running this test
The virtio_balloon test is currently hanging for unknown reasons
when being run on the shared gitlab CI runners (which don't provide
KVM, thus it's running in TCG mode there). All other functional tests
that use the same asset (the Fedora 31 kernel) have already been
marked to work only with KVM in the past, so those other tests are
skipped on the shared gitlab CI runners. As long as the problem isn't
fully understood and fixed, let's do the same with the virtio_balloon
test to avoid that the CI is failing here.

Message-ID: <20250307063904.1081961-1-thuth@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 09:24:00 +01:00
Daniel P. Berrangé
842721581f tests/functional: fix race in virtio balloon test
There are two race conditions in the recently added virtio balloon
test

 * The /dev/vda device node is not ready
 * The virtio-balloon driver has not issued the first stats refresh

To fix the former, monitor dmesg for a line about 'vda'.

To fix the latter, retry the stats query until seeing fresh data.

Adding 'quiet' to the kernel command line reduces serial output
which otherwise slows boot, making it less likely to hit the former
race too.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250304183340.3749797-1-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
[thuth: Break long line to avoid checkpatch error]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 09:23:45 +01:00
Thomas Huth
a31001b1c8 tests/functional: Increase the timeout of the mips64el_replay test
We run the gitlab-CI with the untrusted tests enabled, and
the test_replay_mips64el_malta_5KEc_cpio subtest is rather slow,
so this already hit the standard 90 seconds timeout in the CI.
Increase the timeout for more headroom.

Reported-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20250305074353.52552-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 07:50:19 +01:00
Thomas Huth
7b7f98efa7 tests/functional/test_mips_malta: Add a network test via the pcnet NIC
The kernel has a driver for the pcnet NIC included, and the initrd has
a "tftp" command, so we can test a simple network transfer here, too.

Message-ID: <20250227103915.19795-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 07:50:19 +01:00
Thomas Huth
2c92ecb678 tests/functional: Move the code for testing HTTP downloads to a common function
We are going to use this code in other tests, too, so let's move it
to the qemu_test module to be able to re-use it more easily.

Message-ID: <20250227103915.19795-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 07:50:19 +01:00
Daniel P. Berrangé
9813958892 tests/functional: stop output from zstd command when uncompressing
The zstd command will print incremental decompression progress to stderr
when running. Fortunately it is not on stdout as that would confuse the
TAP parsing, but we should still not have this printed. By switching
from 'check_call' to 'run' with the check=True and capture_output=True
we'll get the desired silence on success, and on failure the raised
exception will automatically include stdout/stderr data for diagnosis
purposes.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250228102738.3064045-8-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 07:50:19 +01:00
Daniel P. Berrangé
5ad2c8f357 tests/functional: drop unused 'get_tag' method
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250228102738.3064045-7-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2025-03-07 07:50:19 +01:00