381d2c36e1
Unfortunately, the definition of the footnote syntax requires the author to use the awkward escaped space "\ " in the really common case of "footnote marker at end of word or sentence"; and in fact the rST documentation's examples of footnote syntax contain only artificial examples that do *not* use the syntax. This resulted in ugly rendering of footnotes throughout QEMU's documentation. Ensure the space is escaped whenever the footnote must attach to the preceding word, and also use a named reference for clarity. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
143 lines
5.9 KiB
ReStructuredText
143 lines
5.9 KiB
ReStructuredText
Mapped-ram
|
|
==========
|
|
|
|
Mapped-ram is a new stream format for the RAM section designed to
|
|
supplement the existing ``file:`` migration and make it compatible
|
|
with ``multifd``. This enables parallel migration of a guest's RAM to
|
|
a file.
|
|
|
|
The core of the feature is to ensure that RAM pages are mapped
|
|
directly to offsets in the resulting migration file. This enables the
|
|
``multifd`` threads to write exclusively to those offsets even if the
|
|
guest is constantly dirtying pages (i.e. live migration). Another
|
|
benefit is that the resulting file will have a bounded size, since
|
|
pages which are dirtied multiple times will always go to a fixed
|
|
location in the file, rather than constantly being added to a
|
|
sequential stream. Having the pages at fixed offsets also allows the
|
|
usage of O_DIRECT for save/restore of the migration stream as the
|
|
pages are ensured to be written respecting O_DIRECT alignment
|
|
restrictions.
|
|
|
|
Usage
|
|
-----
|
|
|
|
On both source and destination, enable the ``multifd`` and
|
|
``mapped-ram`` capabilities:
|
|
|
|
``migrate_set_capability multifd on``
|
|
|
|
``migrate_set_capability mapped-ram on``
|
|
|
|
Use a ``file:`` URL for migration:
|
|
|
|
``migrate file:/path/to/migration/file``
|
|
|
|
Mapped-ram migration is best done non-live, i.e. by stopping the VM on
|
|
the source side before migrating.
|
|
|
|
For best performance enable the ``direct-io`` parameter as well:
|
|
|
|
``migrate_set_parameter direct-io on``
|
|
|
|
Use-cases
|
|
---------
|
|
|
|
The mapped-ram feature was designed for use cases where the migration
|
|
stream will be directed to a file in the filesystem and not
|
|
immediately restored on the destination VM\ [#alternatives]_. These could be
|
|
thought of as snapshots. We can further categorize them into live and
|
|
non-live.
|
|
|
|
- Non-live snapshot
|
|
|
|
If the use case requires a VM to be stopped before taking a snapshot,
|
|
that's the ideal scenario for mapped-ram migration. Not having to
|
|
track dirty pages, the migration will write the RAM pages to the disk
|
|
as fast as it can.
|
|
|
|
Note: if a snapshot is taken of a running VM, but the VM will be
|
|
stopped after the snapshot by the admin, then consider stopping it
|
|
right before the snapshot to take benefit of the performance gains
|
|
mentioned above.
|
|
|
|
- Live snapshot
|
|
|
|
If the use case requires that the VM keeps running during and after
|
|
the snapshot operation, then mapped-ram migration can still be used,
|
|
but will be less performant. Other strategies such as
|
|
background-snapshot should be evaluated as well. One benefit of
|
|
mapped-ram in this scenario is portability since background-snapshot
|
|
depends on async dirty tracking (KVM_GET_DIRTY_LOG) which is not
|
|
supported outside of Linux.
|
|
|
|
.. [#alternatives] While this same effect could be obtained with the usage of
|
|
snapshots or the ``file:`` migration alone, mapped-ram provides
|
|
a performance increase for VMs with larger RAM sizes (10s to
|
|
100s of GiBs), specially if the VM has been stopped beforehand.
|
|
|
|
RAM section format
|
|
------------------
|
|
|
|
Instead of having a sequential stream of pages that follow the
|
|
RAMBlock headers, the dirty pages for a RAMBlock follow its header
|
|
instead. This ensures that each RAM page has a fixed offset in the
|
|
resulting migration file.
|
|
|
|
A bitmap is introduced to track which pages have been written in the
|
|
migration file. Pages are written at a fixed location for every
|
|
ramblock. Zero pages are ignored as they'd be zero in the destination
|
|
migration as well.
|
|
|
|
::
|
|
|
|
Without mapped-ram: With mapped-ram:
|
|
|
|
--------------------- --------------------------------
|
|
| ramblock 1 header | | ramblock 1 header |
|
|
--------------------- --------------------------------
|
|
| ramblock 2 header | | ramblock 1 mapped-ram header |
|
|
--------------------- --------------------------------
|
|
| ... | | padding to next 1MB boundary |
|
|
--------------------- | ... |
|
|
| ramblock n header | --------------------------------
|
|
--------------------- | ramblock 1 pages |
|
|
| RAM_SAVE_FLAG_EOS | | ... |
|
|
--------------------- --------------------------------
|
|
| stream of pages | | ramblock 2 header |
|
|
| (iter 1) | --------------------------------
|
|
| ... | | ramblock 2 mapped-ram header |
|
|
--------------------- --------------------------------
|
|
| RAM_SAVE_FLAG_EOS | | padding to next 1MB boundary |
|
|
--------------------- | ... |
|
|
| stream of pages | --------------------------------
|
|
| (iter 2) | | ramblock 2 pages |
|
|
| ... | | ... |
|
|
--------------------- --------------------------------
|
|
| ... | | ... |
|
|
--------------------- --------------------------------
|
|
| RAM_SAVE_FLAG_EOS |
|
|
--------------------------------
|
|
| ... |
|
|
--------------------------------
|
|
|
|
where:
|
|
- ramblock header: the generic information for a ramblock, such as
|
|
idstr, used_len, etc.
|
|
|
|
- ramblock mapped-ram header: the information added by this feature:
|
|
bitmap of pages written, bitmap size and offset of pages in the
|
|
migration file.
|
|
|
|
Restrictions
|
|
------------
|
|
|
|
Since pages are written to their relative offsets and out of order
|
|
(due to the memory dirtying patterns), streaming channels such as
|
|
sockets are not supported. A seekable channel such as a file is
|
|
required. This can be verified in the QIOChannel by the presence of
|
|
the QIO_CHANNEL_FEATURE_SEEKABLE.
|
|
|
|
The improvements brought by this feature apply only to guest physical
|
|
RAM. Other types of memory such as VRAM are migrated as part of device
|
|
states.
|