SYS-OSS/FRET-qemu
4
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Parallels format driver changes:

Browse Source 
* Fix comments formatting inside parallels driver
   * Incorrect data end calculation in parallels_open()
   * Check if data_end greater than the file size
   * Add "explicit" argument to parallels_check_leak()
   * Add data_start field to BDRVParallelsState
   * Add checking and repairing duplicate offsets in BAT
   * Image repairing in parallels_open()
   * Use bdrv_co_getlength() in parallels_check_outside_image()
   * Add data_off check
   * Add data_off repairing to parallels_open()
   * Fix record in MAINTAINERS
 
 Parallels format driver tests:
   * Add out-of-image check test for parallels format
   * Add leak check test for parallels format
   * Add test for BAT entries duplication check
   * Refactor tests of parallels images checks (131)
   * Fix cluster size in parallels images tests (131)
   * Fix test 131 after repair was added to parallels_open()
   * Add test for data_off check
 -----BEGIN PGP SIGNATURE-----
 
 iQHDBAABCgAtFiEE9vE2f3B8+RUZInytPzClrpN3nJ8FAmT4nUgPHGRlbkBvcGVu
 dnoub3JnAAoJED8wpa6Td5yf1F4L/j4RsGv+NRJRqZb9JNn2wUm4JdWGyv6ftuuh
 hT25F44B5S6J3tR3LalDFxHpr+kCXD1Xa3ZJNK14d1G9atw7Bsp5ntxpCmzEALBk
 0PH+5fvNuhvt4ZnuYwQX70n3ZmalgzGpwf/jbs9mXUhdLinEr1RWi2f9yfCLmeZU
 x+0MSOhAdC6ZVsJOTJhGuRWWKL1q5KteuTwQlRCwDay8KF/Mc1OS/iPFqfmlWenM
 dc88PZBlg2Le15sWWNLc1AZHYguO+4xEPw6fk6RcswccILB2gCUPS6BJB0AuKNOO
 STPIgzUFMXfgIFhNUOvz58A7UnQGI4dMsRe/2UJIG+Y3qkM4DpjcZ7U/rHxhR6t0
 +GeeLS+a+aObz79TpB3gZi7leX2bpRUZ8nLkaAnL2umhtdFo5sdqD3xo4xcg4Ebk
 TbYSmgIM0eZ75d+48g7A+ddkyKYCmworGS9g9Cry6udclbs8yXhVB8KkUbYwtJlC
 HtNzgaWlw6J7n0MoSpz4OQVKq3bY0A==
 =grCk
 -----END PGP SIGNATURE-----

Merge tag 'pull-parallels-2023-09-06' of https://src.openvz.org/scm/~den/qemu into staging

Parallels format driver changes:
  * Fix comments formatting inside parallels driver
  * Incorrect data end calculation in parallels_open()
  * Check if data_end greater than the file size
  * Add "explicit" argument to parallels_check_leak()
  * Add data_start field to BDRVParallelsState
  * Add checking and repairing duplicate offsets in BAT
  * Image repairing in parallels_open()
  * Use bdrv_co_getlength() in parallels_check_outside_image()
  * Add data_off check
  * Add data_off repairing to parallels_open()
  * Fix record in MAINTAINERS

Parallels format driver tests:
  * Add out-of-image check test for parallels format
  * Add leak check test for parallels format
  * Add test for BAT entries duplication check
  * Refactor tests of parallels images checks (131)
  * Fix cluster size in parallels images tests (131)
  * Fix test 131 after repair was added to parallels_open()
  * Add test for data_off check

# -----BEGIN PGP SIGNATURE-----
#
# iQHDBAABCgAtFiEE9vE2f3B8+RUZInytPzClrpN3nJ8FAmT4nUgPHGRlbkBvcGVu
# dnoub3JnAAoJED8wpa6Td5yf1F4L/j4RsGv+NRJRqZb9JNn2wUm4JdWGyv6ftuuh
# hT25F44B5S6J3tR3LalDFxHpr+kCXD1Xa3ZJNK14d1G9atw7Bsp5ntxpCmzEALBk
# 0PH+5fvNuhvt4ZnuYwQX70n3ZmalgzGpwf/jbs9mXUhdLinEr1RWi2f9yfCLmeZU
# x+0MSOhAdC6ZVsJOTJhGuRWWKL1q5KteuTwQlRCwDay8KF/Mc1OS/iPFqfmlWenM
# dc88PZBlg2Le15sWWNLc1AZHYguO+4xEPw6fk6RcswccILB2gCUPS6BJB0AuKNOO
# STPIgzUFMXfgIFhNUOvz58A7UnQGI4dMsRe/2UJIG+Y3qkM4DpjcZ7U/rHxhR6t0
# +GeeLS+a+aObz79TpB3gZi7leX2bpRUZ8nLkaAnL2umhtdFo5sdqD3xo4xcg4Ebk
# TbYSmgIM0eZ75d+48g7A+ddkyKYCmworGS9g9Cry6udclbs8yXhVB8KkUbYwtJlC
# HtNzgaWlw6J7n0MoSpz4OQVKq3bY0A==
# =grCk
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 06 Sep 2023 11:39:52 EDT
# gpg:                using RSA key F6F1367F707CF91519227CAD3F30A5AE93779C9F
# gpg:                issuer "den@openvz.org"
# gpg: Good signature from "Denis V. Lunev <den@openvz.org>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: F6F1 367F 707C F915 1922  7CAD 3F30 A5AE 9377 9C9F

* tag 'pull-parallels-2023-09-06' of https://src.openvz.org/scm/~den/qemu:
  iotests: Add test for data_off check
  iotests: Fix test 131 after repair was added to parallels_open()
  iotests: Fix cluster size in parallels images tests (131)
  iotests: Refactor tests of parallels images checks (131)
  iotests: Add test for BAT entries duplication check
  iotests: Add leak check test for parallels format
  iotests: Add out-of-image check test for parallels format
  parallels: Add data_off repairing to parallels_open()
  parallels: Add data_off check
  parallels: Use bdrv_co_getlength() in parallels_check_outside_image()
  parallels: Image repairing in parallels_open()
  parallels: Add checking and repairing duplicate offsets in BAT
  parallels: Add data_start field to BDRVParallelsState
  parallels: Add "explicit" argument to parallels_check_leak()
  parallels: Check if data_end greater than the file size
  parallels: Incorrect data end calculation in parallels_open()
  parallels: Fix comments formatting inside parallels driver
  MAINTAINERS: add tree to keep parallels format driver changes

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
This commit is contained in:
Stefan Hajnoczi 2023-09-07 10:27:55 -04:00
commit c97d45d557
7 changed files with 571 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
MAINTAINERS
View File

@ -3693,6 +3693,7 @@ S: Supported
F: block/parallels.c
F: block/parallels-ext.c
F: docs/interop/parallels.txt
T: git https://src.openvz.org/scm/~den/qemu.git parallels
qed
M: Stefan Hajnoczi <stefanha@redhat.com>

346
block/parallels.c
View File

@ -136,6 +136,12 @@ static int cluster_remainder(BDRVParallelsState *s, int64_t sector_num,
return MIN(nb_sectors, ret);
}
static uint32_t host_cluster_index(BDRVParallelsState *s, int64_t off)
{
off -= s->data_start << BDRV_SECTOR_BITS;
return off / s->cluster_size;
}
static int64_t block_status(BDRVParallelsState *s, int64_t sector_num,
int nb_sectors, int *pnum)
{
@ -188,7 +194,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num,
idx = sector_num / s->tracks;
to_allocate = DIV_ROUND_UP(sector_num + *pnum, s->tracks) - idx;
/* This function is called only by parallels_co_writev(), which will never
/*
* This function is called only by parallels_co_writev(), which will never
* pass a sector_num at or beyond the end of the image (because the block
* layer never passes such a sector_num to that function). Therefore, idx
* is always below s->bat_size.
@ -196,7 +203,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num,
* exceed the image end. Therefore, idx + to_allocate cannot exceed
* s->bat_size.
* Note that s->bat_size is an unsigned int, therefore idx + to_allocate
* will always fit into a uint32_t. */
* will always fit into a uint32_t.
*/
assert(idx < s->bat_size && idx + to_allocate <= s->bat_size);
space = to_allocate * s->tracks;
@ -230,13 +238,15 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num,
}
}
/* Try to read from backing to fill empty clusters
/*
* Try to read from backing to fill empty clusters
* FIXME: 1. previous write_zeroes may be redundant
* 2. most of data we read from backing will be rewritten by
* parallels_co_writev. On aligned-to-cluster write we do not need
* this read at all.
* 3. it would be good to combine write of data from backing and new
* data into one write call */
* data into one write call.
*/
if (bs->backing) {
int64_t nb_cow_sectors = to_allocate * s->tracks;
int64_t nb_cow_bytes = nb_cow_sectors << BDRV_SECTOR_BITS;
@ -440,6 +450,81 @@ static void parallels_check_unclean(BlockDriverState *bs,
}
}
/*
* Returns true if data_off is correct, otherwise false. In both cases
* correct_offset is set to the proper value.
*/
static bool parallels_test_data_off(BDRVParallelsState *s,
int64_t file_nb_sectors,
uint32_t *correct_offset)
{
uint32_t data_off, min_off;
bool old_magic;
/*
* There are two slightly different image formats: with "WithoutFreeSpace"
* or "WithouFreSpacExt" magic words. Call the first one as "old magic".
* In such images data_off field can be zero. In this case the offset is
* calculated as the end of BAT table plus some padding to ensure sector
* size alignment.
*/
old_magic = !memcmp(s->header->magic, HEADER_MAGIC, 16);
min_off = DIV_ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE);
if (!old_magic) {
min_off = ROUND_UP(min_off, s->cluster_size / BDRV_SECTOR_SIZE);
}
if (correct_offset) {
*correct_offset = min_off;
}
data_off = le32_to_cpu(s->header->data_off);
if (data_off == 0 && old_magic) {
return true;
}
if (data_off < min_off || data_off > file_nb_sectors) {
return false;
}
if (correct_offset) {
*correct_offset = data_off;
}
return true;
}
static int coroutine_fn GRAPH_RDLOCK
parallels_check_data_off(BlockDriverState *bs, BdrvCheckResult *res,
BdrvCheckMode fix)
{
BDRVParallelsState *s = bs->opaque;
int64_t file_size;
uint32_t data_off;
file_size = bdrv_co_nb_sectors(bs->file->bs);
if (file_size < 0) {
res->check_errors++;
return file_size;
}
if (parallels_test_data_off(s, file_size, &data_off)) {
return 0;
}
res->corruptions++;
if (fix & BDRV_FIX_ERRORS) {
s->header->data_off = cpu_to_le32(data_off);
res->corruptions_fixed++;
}
fprintf(stderr, "%s data_off field has incorrect value\n",
fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR");
return 0;
}
static int coroutine_fn GRAPH_RDLOCK
parallels_check_outside_image(BlockDriverState *bs, BdrvCheckResult *res,
BdrvCheckMode fix)
@ -484,13 +569,13 @@ parallels_check_outside_image(BlockDriverState *bs, BdrvCheckResult *res,
static int coroutine_fn GRAPH_RDLOCK
parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res,
BdrvCheckMode fix)
BdrvCheckMode fix, bool explicit)
{
BDRVParallelsState *s = bs->opaque;
int64_t size;
int ret;
size = bdrv_getlength(bs->file->bs);
size = bdrv_co_getlength(bs->file->bs);
if (size < 0) {
res->check_errors++;
return size;
@ -499,10 +584,13 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res,
if (size > res->image_end_offset) {
int64_t count;
count = DIV_ROUND_UP(size - res->image_end_offset, s->cluster_size);
fprintf(stderr, "%s space leaked at the end of the image %" PRId64 "\n",
fix & BDRV_FIX_LEAKS ? "Repairing" : "ERROR",
size - res->image_end_offset);
res->leaks += count;
if (explicit) {
fprintf(stderr,
"%s space leaked at the end of the image %" PRId64 "\n",
fix & BDRV_FIX_LEAKS ? "Repairing" : "ERROR",
size - res->image_end_offset);
res->leaks += count;
}
if (fix & BDRV_FIX_LEAKS) {
Error *local_err = NULL;
@ -517,13 +605,148 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res,
res->check_errors++;
return ret;
}
res->leaks_fixed += count;
if (explicit) {
res->leaks_fixed += count;
}
}
}
return 0;
}
static int coroutine_fn GRAPH_RDLOCK
parallels_check_duplicate(BlockDriverState *bs, BdrvCheckResult *res,
BdrvCheckMode fix)
{
BDRVParallelsState *s = bs->opaque;
int64_t host_off, host_sector, guest_sector;
unsigned long *bitmap;
uint32_t i, bitmap_size, cluster_index, bat_entry;
int n, ret = 0;
uint64_t *buf = NULL;
bool fixed = false;
/*
* Create a bitmap of used clusters.
* If a bit is set, there is a BAT entry pointing to this cluster.
* Loop through the BAT entries, check bits relevant to an entry offset.
* If bit is set, this entry is duplicated. Otherwise set the bit.
*
* We shouldn't worry about newly allocated clusters outside the image
* because they are created higher then any existing cluster pointed by
* a BAT entry.
*/
bitmap_size = host_cluster_index(s, res->image_end_offset);
if (bitmap_size == 0) {
return 0;
}
if (res->image_end_offset % s->cluster_size) {
/* A not aligned image end leads to a bitmap shorter by 1 */
bitmap_size++;
}
bitmap = bitmap_new(bitmap_size);
buf = qemu_blockalign(bs, s->cluster_size);
for (i = 0; i < s->bat_size; i++) {
host_off = bat2sect(s, i) << BDRV_SECTOR_BITS;
if (host_off == 0) {
continue;
}
cluster_index = host_cluster_index(s, host_off);
assert(cluster_index < bitmap_size);
if (!test_bit(cluster_index, bitmap)) {
bitmap_set(bitmap, cluster_index, 1);
continue;
}
/* this cluster duplicates another one */
fprintf(stderr, "%s duplicate offset in BAT entry %u\n",
fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", i);
res->corruptions++;
if (!(fix & BDRV_FIX_ERRORS)) {
continue;
}
/*
* Reset the entry and allocate a new cluster
* for the relevant guest offset. In this way we let
* the lower layer to place the new cluster properly.
* Copy the original cluster to the allocated one.
* But before save the old offset value for repairing
* if we have an error.
*/
bat_entry = s->bat_bitmap[i];
parallels_set_bat_entry(s, i, 0);
ret = bdrv_co_pread(bs->file, host_off, s->cluster_size, buf, 0);
if (ret < 0) {
res->check_errors++;
goto out_repair_bat;
}
guest_sector = (i * (int64_t)s->cluster_size) >> BDRV_SECTOR_BITS;
host_sector = allocate_clusters(bs, guest_sector, s->tracks, &n);
if (host_sector < 0) {
res->check_errors++;
goto out_repair_bat;
}
host_off = host_sector << BDRV_SECTOR_BITS;
ret = bdrv_co_pwrite(bs->file, host_off, s->cluster_size, buf, 0);
if (ret < 0) {
res->check_errors++;
goto out_repair_bat;
}
if (host_off + s->cluster_size > res->image_end_offset) {
res->image_end_offset = host_off + s->cluster_size;
}
/*
* In the future allocate_cluster() will reuse holed offsets
* inside the image. Keep the used clusters bitmap content
* consistent for the new allocated clusters too.
*
* Note, clusters allocated outside the current image are not
* considered, and the bitmap size doesn't change.
*/
cluster_index = host_cluster_index(s, host_off);
if (cluster_index < bitmap_size) {
bitmap_set(bitmap, cluster_index, 1);
}
fixed = true;
res->corruptions_fixed++;
}
if (fixed) {
/*
* When new clusters are allocated, the file size increases by
* 128 Mb. We need to truncate the file to the right size. Let
* the leak fix code make its job without res changing.
*/
ret = parallels_check_leak(bs, res, fix, false);
}
out_free:
g_free(buf);
g_free(bitmap);
return ret;
/*
* We can get here only from places where index and old_offset have
* meaningful values.
*/
out_repair_bat:
s->bat_bitmap[i] = bat_entry;
goto out_free;
}
static void parallels_collect_statistics(BlockDriverState *bs,
BdrvCheckResult *res,
BdrvCheckMode fix)
@ -565,12 +788,22 @@ parallels_co_check(BlockDriverState *bs, BdrvCheckResult *res,
WITH_QEMU_LOCK_GUARD(&s->lock) {
parallels_check_unclean(bs, res, fix);
ret = parallels_check_data_off(bs, res, fix);
if (ret < 0) {
return ret;
}
ret = parallels_check_outside_image(bs, res, fix);
if (ret < 0) {
return ret;
}
ret = parallels_check_leak(bs, res, fix);
ret = parallels_check_leak(bs, res, fix, true);
if (ret < 0) {
return ret;
}
ret = parallels_check_duplicate(bs, res, fix);
if (ret < 0) {
return ret;
}
@ -798,10 +1031,12 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags,
BDRVParallelsState *s = bs->opaque;
ParallelsHeader ph;
int ret, size, i;
int64_t file_nb_sectors;
int64_t file_nb_sectors, sector;
uint32_t data_start;
QemuOpts *opts = NULL;
Error *local_err = NULL;
char *buf;
bool data_off_is_correct;
ret = bdrv_open_file_child(NULL, options, "file", bs, errp);
if (ret < 0) {
@ -859,15 +1094,6 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags,
ret = -ENOMEM;
goto fail;
}
s->data_end = le32_to_cpu(ph.data_off);
if (s->data_end == 0) {
s->data_end = ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE);
}
if (s->data_end < s->header_size) {
/* there is not enough unused space to fit to block align between BAT
and actual data. We can't avoid read-modify-write... */
s->header_size = size;
}
ret = bdrv_pread(bs->file, 0, s->header_size, s->header, 0);
if (ret < 0) {
@ -875,33 +1101,20 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags,
}
s->bat_bitmap = (uint32_t *)(s->header + 1);
for (i = 0; i < s->bat_size; i++) {
int64_t off = bat2sect(s, i);
if (off >= file_nb_sectors) {
if (flags & BDRV_O_CHECK) {
continue;
}
error_setg(errp, "parallels: Offset %" PRIi64 " in BAT[%d] entry "
"is larger than file size (%" PRIi64 ")",
off << BDRV_SECTOR_BITS, i,
file_nb_sectors << BDRV_SECTOR_BITS);
ret = -EINVAL;
goto fail;
}
if (off >= s->data_end) {
s->data_end = off + s->tracks;
}
if (le32_to_cpu(ph.inuse) == HEADER_INUSE_MAGIC) {
s->header_unclean = true;
}
if (le32_to_cpu(ph.inuse) == HEADER_INUSE_MAGIC) {
/* Image was not closed correctly. The check is mandatory */
s->header_unclean = true;
if ((flags & BDRV_O_RDWR) && !(flags & BDRV_O_CHECK)) {
error_setg(errp, "parallels: Image was not closed correctly; "
"cannot be opened read/write");
ret = -EACCES;
goto fail;
}
data_off_is_correct = parallels_test_data_off(s, file_nb_sectors,
&data_start);
s->data_start = data_start;
s->data_end = s->data_start;
if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) {
/*
* There is not enough unused space to fit to block align between BAT
* and actual data. We can't avoid read-modify-write...
*/
s->header_size = size;
}
opts = qemu_opts_create(&parallels_runtime_opts, NULL, 0, errp);
@ -962,10 +1175,41 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags,
bdrv_get_device_or_node_name(bs));
ret = migrate_add_blocker(s->migration_blocker, errp);
if (ret < 0) {
error_free(s->migration_blocker);
error_setg(errp, "Migration blocker error");
goto fail;
}
qemu_co_mutex_init(&s->lock);
for (i = 0; i < s->bat_size; i++) {
sector = bat2sect(s, i);
if (sector + s->tracks > s->data_end) {
s->data_end = sector + s->tracks;
}
}
/*
* We don't repair the image here if it's opened for checks. Also we don't
* want to change inactive images and can't change readonly images.
*/
if ((flags & (BDRV_O_CHECK | BDRV_O_INACTIVE)) || !(flags & BDRV_O_RDWR)) {
return 0;
}
/*
* Repair the image if it's dirty or
* out-of-image corruption was detected.
*/
if (s->data_end > file_nb_sectors || s->header_unclean
|| !data_off_is_correct) {
BdrvCheckResult res;
ret = bdrv_check(bs, &res, BDRV_FIX_ERRORS | BDRV_FIX_LEAKS);
if (ret < 0) {
error_setg_errno(errp, -ret, "Could not repair corrupted image");
migrate_del_blocker(s->migration_blocker);
goto fail;
}
}
return 0;
fail_format:
@ -973,6 +1217,12 @@ fail_format:
fail_options:
ret = -EINVAL;
fail:
/*
* "s" object was allocated by g_malloc0 so we can safely
* try to free its fields even they were not allocated.
*/
error_free(s->migration_blocker);
g_free(s->bat_dirty_bmap);
qemu_vfree(s->header);
return ret;
}

1
block/parallels.h
View File

@ -75,6 +75,7 @@ typedef struct BDRVParallelsState {
uint32_t *bat_bitmap;
unsigned int bat_size;
int64_t data_start;
int64_t data_end;
uint64_t prealloc_size;
ParallelsPreallocMode prealloc_mode;

50
tests/qemu-iotests/131
View File

@ -44,31 +44,35 @@ _supported_os Linux
inuse_offset=$((0x2c))
size=$((64 * 1024 * 1024))
CLUSTER_SIZE=64k
IMGFMT=parallels
_make_test_img $size
echo == read empty image ==
{ $QEMU_IO -c "read -P 0 32k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == write more than 1 block in a row ==
{ $QEMU_IO -c "write -P 0x11 32k 128k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == read less than block ==
{ $QEMU_IO -c "read -P 0x11 32k 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == read exactly 1 block ==
{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == read more than 1 block ==
{ $QEMU_IO -c "read -P 0x11 32k 128k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == check that there is no trash after written ==
{ $QEMU_IO -c "read -P 0 160k 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == check that there is no trash before written ==
{ $QEMU_IO -c "read -P 0 0 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
# get cluster size in sectors from "tracks" header field
CLUSTER_SIZE_OFFSET=28
CLUSTER_SIZE=$(peek_file_le $TEST_IMG $CLUSTER_SIZE_OFFSET 4)
CLUSTER_SIZE=$((CLUSTER_SIZE * 512))
CLUSTER_HALF_SIZE=$((CLUSTER_SIZE / 2))
CLUSTER_DBL_SIZE=$((CLUSTER_SIZE * 2))
echo "== Corrupt image =="
echo == read empty image ==
{ $QEMU_IO -c "read -P 0 $CLUSTER_HALF_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == write more than 1 block in a row ==
{ $QEMU_IO -c "write -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == read less than block ==
{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == read exactly 1 block ==
{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == read more than 1 block ==
{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == check that there is no trash after written ==
{ $QEMU_IO -c "read -P 0 $((CLUSTER_HALF_SIZE + CLUSTER_DBL_SIZE)) $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo == check that there is no trash before written ==
{ $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== corrupt image =="
poke_file "$TEST_IMG" "$inuse_offset" "\x59\x6e\x6f\x74"
{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
_check_test_img
_check_test_img -r all
{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== read corrupted image with repairing =="
{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== allocate with backing =="
# Verify that allocating clusters works fine even when there is a backing image.
@ -83,7 +87,7 @@ TEST_IMG="$TEST_IMG.base" _make_test_img $size
# Write some data to the base image (which would trigger an assertion failure if
# interpreted as a QEMUIOVector)
$QEMU_IO -c 'write -P 42 0 64k' "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c "write -P 42 0 $CLUSTER_SIZE" "$TEST_IMG.base" | _filter_qemu_io
# Parallels does not seem to support storing a backing filename in the image
# itself, so we need to build our backing chain on the command line
@ -99,8 +103,8 @@ QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
$QEMU_IO --image-opts "$imgopts" \
-c 'read -P 1 0 64' \
-c "read -P 42 64 $((64 * 1024 - 64))" \
-c "read -P 0 64k $((size - 64 * 1024))" \
-c "read -P 42 64 $((CLUSTER_SIZE - 64))" \
-c "read -P 0 $CLUSTER_SIZE $((size - CLUSTER_SIZE))" \
| _filter_qemu_io
# success, all done

59
tests/qemu-iotests/131.out
View File

@ -1,53 +1,42 @@
QA output created by 131
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
== read empty image ==
read 65536/65536 bytes at offset 32768
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 1048576/1048576 bytes at offset 524288
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== write more than 1 block in a row ==
wrote 131072/131072 bytes at offset 32768
128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 2097152/2097152 bytes at offset 524288
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== read less than block ==
read 32768/32768 bytes at offset 32768
32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 524288/524288 bytes at offset 524288
512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== read exactly 1 block ==
read 65536/65536 bytes at offset 65536
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== read more than 1 block ==
read 131072/131072 bytes at offset 32768
128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 2097152/2097152 bytes at offset 524288
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== check that there is no trash after written ==
read 32768/32768 bytes at offset 163840
32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 524288/524288 bytes at offset 2621440
512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== check that there is no trash before written ==
read 32768/32768 bytes at offset 0
32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== Corrupt image ==
qemu-io: can't open device TEST_DIR/t.parallels: parallels: Image was not closed correctly; cannot be opened read/write
ERROR image was not closed correctly
1 errors were found on the image.
Data may be corrupted, or further writes to the image may corrupt it.
read 524288/524288 bytes at offset 0
512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== corrupt image ==
== read corrupted image with repairing ==
Repairing image was not closed correctly
The following inconsistencies were found and repaired:
0 leaked clusters
1 corruptions
Double checking the fixed image now...
No errors were found on the image.
read 65536/65536 bytes at offset 65536
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== allocate with backing ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
wrote 65536/65536 bytes at offset 0
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 64/64 bytes at offset 0
64 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 64/64 bytes at offset 0
64 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 65472/65472 bytes at offset 64
63.938 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 67043328/67043328 bytes at offset 65536
63.938 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 1048512/1048512 bytes at offset 64
1023.938 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 66060288/66060288 bytes at offset 1048576
63 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
*** done

145
tests/qemu-iotests/tests/parallels-checks Executable file
View File

@ -0,0 +1,145 @@
#!/usr/bin/env bash
# group: rw quick
#
# Test qemu-img check for parallels format
#
# Copyright (C) 2022 Virtuozzo International GmbH
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# creator
owner=alexander.ivanov@virtuozzo.com
seq=`basename $0`
echo "QA output created by $seq"
status=1 # failure is the default!
_cleanup()
{
_cleanup_test_img
}
trap "_cleanup; exit \$status" 0 1 2 3 15
# get standard environment, filters and checks
. ../common.rc
. ../common.filter
_supported_fmt parallels
_supported_proto file
_supported_os Linux
SIZE=$((4 * 1024 * 1024))
IMGFMT=parallels
CLUSTER_SIZE_OFFSET=28
DATA_OFF_OFFSET=48
BAT_OFFSET=64
_make_test_img $SIZE
CLUSTER_SIZE=$(peek_file_le $TEST_IMG $CLUSTER_SIZE_OFFSET 4)
CLUSTER_SIZE=$((CLUSTER_SIZE * 512))
LAST_CLUSTER_OFF=$((SIZE - CLUSTER_SIZE))
LAST_CLUSTER=$((LAST_CLUSTER_OFF/CLUSTER_SIZE))
echo "== TEST OUT OF IMAGE CHECK =="
echo "== write pattern =="
{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== corrupt image =="
cluster=$(($LAST_CLUSTER + 2))
poke_file "$TEST_IMG" "$BAT_OFFSET" "\x$cluster\x00\x00\x00"
echo "== read corrupted image with repairing =="
{ $QEMU_IO -c "read -P 0x00 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
# Clear image
_make_test_img $SIZE
echo "== TEST LEAK CHECK =="
echo "== write pattern to last cluster =="
echo "write -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE"
{ $QEMU_IO -c "write -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
file_size=`stat --printf="%s" "$TEST_IMG"`
echo "file size: $file_size"
echo "== extend image by 1 cluster =="
fallocate -xl $((file_size + CLUSTER_SIZE)) "$TEST_IMG"
file_size=`stat --printf="%s" "$TEST_IMG"`
echo "file size: $file_size"
echo "== repair image =="
_check_test_img -r all
file_size=`stat --printf="%s" "$TEST_IMG"`
echo "file size: $file_size"
echo "== check last cluster =="
{ $QEMU_IO -c "read -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
# Clear image
_make_test_img $SIZE
echo "== TEST DUPLICATION CHECK =="
echo "== write pattern to whole image =="
{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== write another pattern to second cluster =="
{ $QEMU_IO -c "write -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== check second cluster =="
{ $QEMU_IO -c "read -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== corrupt image =="
poke_file "$TEST_IMG" "$(($BAT_OFFSET + 4))" "\x01\x00\x00\x00"
echo "== check second cluster =="
{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== repair image =="
_check_test_img -r all
echo "== check second cluster =="
{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== check first cluster on host =="
printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1`
echo "== check second cluster on host =="
printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1`
# Clear image
_make_test_img $SIZE
echo "== TEST DATA_OFF CHECK =="
echo "== write pattern to first cluster =="
{ $QEMU_IO -c "write -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
echo "== spoil data_off field =="
poke_file "$TEST_IMG" "$DATA_OFF_OFFSET" "\xff\xff\xff\xff"
echo "== check first cluster =="
{ $QEMU_IO -c "read -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir
# success, all done
echo "*** done"
rm -f $seq.full
status=0

75
tests/qemu-iotests/tests/parallels-checks.out Normal file
View File

@ -0,0 +1,75 @@
QA output created by parallels-checks
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304
== TEST OUT OF IMAGE CHECK ==
== write pattern ==
wrote 4194304/4194304 bytes at offset 0
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== corrupt image ==
== read corrupted image with repairing ==
Repairing cluster 0 is outside image
read 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304
== TEST LEAK CHECK ==
== write pattern to last cluster ==
write -P 0x11 3145728 1048576
wrote 1048576/1048576 bytes at offset 3145728
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
file size: 2097152
== extend image by 1 cluster ==
file size: 3145728
== repair image ==
Repairing space leaked at the end of the image 1048576
The following inconsistencies were found and repaired:
1 leaked clusters
0 corruptions
Double checking the fixed image now...
No errors were found on the image.
file size: 2097152
== check last cluster ==
read 1048576/1048576 bytes at offset 3145728
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304
== TEST DUPLICATION CHECK ==
== write pattern to whole image ==
wrote 4194304/4194304 bytes at offset 0
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== write another pattern to second cluster ==
wrote 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== check second cluster ==
read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== corrupt image ==
== check second cluster ==
read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== repair image ==
Repairing duplicate offset in BAT entry 1
The following inconsistencies were found and repaired:
0 leaked clusters
1 corruptions
Double checking the fixed image now...
No errors were found on the image.
== check second cluster ==
read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== check first cluster on host ==
content: 0x11
== check second cluster on host ==
content: 0x11
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304
== TEST DATA_OFF CHECK ==
== write pattern to first cluster ==
wrote 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== spoil data_off field ==
== check first cluster ==
Repairing data_off field has incorrect value
read 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
*** done