Switchover-ack is a mechanism to synchronize between source and
destination QEMU during migration to prevent the source from switching
over prematurely.
VFIO uses switchover-ack to ensure switchover happens only after
destination side has loaded the precopy initial bytes. This is important
for VFIO, as otherwise downtime could be impacted and be higher.
In its current state, switchover-ack is a one-time mechanism, meaning
that switchover is acked only once and past that another ACK cannot be
requested again. This was sufficient until now, as VFIO precopy initial
bytes was defined to be monotonically decreasing. Thus, when precopy
initial bytes reached zero for all VFIO devices, a single ACK would be
sent and its validity would hold.
However, now the new VFIO_PRECOPY_INFO_REINIT feature allows precopy
initial bytes to be re-initialized during precopy. Specifically, it
means that initial bytes can grow after reaching zero, which would
invalidate a previously sent switchover ACK.
To solve this, make switchover-ack reusable and allow devices to request
switchover ACKs when needed via the save_query_pending SaveVMHandler.
Since now switchover ACK can be requested for a specific device and in
different times, make switchover ACK per-device (instead of a single ACK
for all devices) and let source side do the pending ACKs accounting.
Keep the legacy switchover-ack mechanism for backward compatibility and
turn it on by a compatibility property for older machines. Enable the
property until VFIO implements the new switchover-ack.
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Avihai Horon <avihaih@nvidia.com>
Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-8-avihaih@nvidia.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
A new switchover-ack mechanism will be added in the following patches.
However, the old mechanism must still be kept for backward
compatibility.
Rename existing code that will be used only for old switchover-ack
mechanism as legacy. This will help to distinguish legacy code from new
code and make it more readable and easier for removal later when no
longer needed.
No functional change intended.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Avihai Horon <avihaih@nvidia.com>
Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-7-avihaih@nvidia.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
A new switchover-ack mechanism that will replace the existing one will
be added in the following patches. The new mechanism will not use
switchover_ack_needed SaveVMHandler, however, the old mechanism must
still be kept for backward compatibility.
To keep things clear and decrease API surface of old code, replace
switchover_ack_needed SaveVMHandler with a regular function
migration_request_switchover_ack().
No functional changes intended.
Acked-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Avihai Horon <avihaih@nvidia.com>
Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-6-avihaih@nvidia.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Before switchover, the source needs one last exact pending query so
modules can flush dirty state. This is currently done ad hoc in modules
handlers. For example, RAM syncs its dirty bitmap in its save_complete
handler.
This should be a general concept relevant for any module, so extract it
to migration core instead by running a final save_query_pending before
switchover.
The final query requires special handling by modules (e.g., it's called
with BQL locked, during VM stop), so extend save_query_pending
SaveVMHandlers callback and qemu_savevm_query_pending() with a "final"
flag so migration modules can tell the last pending query during
switchover from periodic iteration queries.
Call final pending query also in COLO checkpoint, which needs to flush
dirty state before the checkpoint's live state is saved. Unlike a
regular switchover, COLO reaches completion repeatedly for every
checkpoint, so this must be done on each one.
Signed-off-by: Avihai Horon <avihaih@nvidia.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260706085211.13905-4-avihaih@nvidia.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Operations that read guest RAM (dump-guest-memory, memsave, pmemsave)
must refuse to run while the destination of a migration is still
receiving that RAM: during precopy it is incomplete, and during postcopy
a read faults the page in from the source. Provide a single predicate
they can share instead of open-coding the runstate and postcopy checks.
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20260619101834.228432-2-den@openvz.org>
In GLib, GByteArray is an object managed by the library. Currently,
migrating a GByteArray requires treating it as a raw C struct and using
VMSTATE_VBUFFER_ALLOC_UINT32. For example, see vmstate_vdba in
ui/vdagent.c
QEMU cannot pretend that GByteArray is a C struct and simply use
VMS_ALLOC to g_malloc() the buffer. This is because, VMS_ALLOC blindly
overwrites the data pointer with a newly allocated buffer, thereby
leaking the previous memory. Besides, GLib tracks the array's capacity
in a hidden alloc field. Bypassing GLib APIs leave this capacity out of
sync with the newly allocated buffer, potentially leading to heap buffer
overflows during subsequent g_byte_array_append() calls.
This commit introduces VMSTATE_GBYTEARRAY which uses specific library
API calls (g_byte_array_set_size()) to safely resize and populate the
buffer.
Signed-off-by: Arun Menon <armenon@redhat.com>
Suggested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260423105733.113046-2-armenon@redhat.com
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Marc-André reported an issue on QEMU crash when retrying a cancelled
migration during early setup phase, see "Link:" for more information, and
also easy way to reproduce.
This patch is a replacement of the prior fix proposed by not only switching
to migration_cleanup(), but also fixing it from CPR side, so that we track
hup_source properly to know if src QEMU is waiting or the HUP signal.
To put it simple: this chunk of special casing in migration_cancel() should
not affect normal migration, but only cpr-transfer migration to cover the
small window when the src QEMU is waiting for a HUP signal on cpr
channel (so that src QEMU can continue the migration on the main channel).
To achieve that, we'll also need to remember to detach the hup_source
whenenver invoked: after that point, we should always be able to cleanup
the migration.
It's not a generic operation to explicitly detach a gsource from its
context while in its dispatch() function. But it should be safe, because
gsource disptch() will only happen with a boosted refcount for the
dispatcher so that the gsource will not be freed until the callback
completes. It's also safe to return G_SOURCE_REMOVE after the gsource is
detached, as glib will simply ignore the G_SOURCE_REMOVE.
One can refer to latest 2.86.5 glib code in g_main_dispatch() for that:
https://github.com/GNOME/glib/blob/2.86.5/glib/gmain.c#L3592
When at this, add a bunch of assertions to make sure nothing surprises us.
After this patch applied, the 2nd migration will not crash QEMU, instead
it'll be in CANCELLING until the socket connection times out (it will take
~2min on my Fedora default kernel). During this process no 2nd migration
will be allowed, and after it timed out migration can be restarted.
It's because so far we don't have control over socket_connect_outgoing(),
or anything yet managed by a task executed in qio_task_run_in_thread().
Speeding up the cancellation to be left for future.
I also tested cpr-transfer by only providing cpr channel not the main
channel (with -incoming defer), kickoff migration on source, then cancel it
on source directly without providing the main channel. It keeps working.
I wanted to add an unit test for that but it'll need to refactor current
cpr-transfer tests first; let's leave it for later.
Link: https://lore.kernel.org/r/20260417184742.293061-1-marcandre.lureau@redhat.com
Reported-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20260421175820.302795-1-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Various patches related to single binary effort:
- Reduce "exec/cpu-defs.h" inclusions
- Build various target specific files once
- Remove need of per-target monitor handlers
- Reduce target_ulong uses in migration code
- Reduce uses of legacy native endianness & ld/st_phys APIs
- Removed MIPSCPU::mvp memory leak
- Clear dangling GLib event source tag
- Remove pointless variable initialization in *FOREACH*() macro uses
- Few checkpatch.pl updates
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmn7S/EACgkQ4+MsLN6t
# wN5bCBAAu05saLnGWy3s9aJ7mrrs2yqSOHRMskrV8/ULr9beJhqLCH7xobuHoSO0
# c3o6BSU9QnBB04Ps3uM2SKcWx2JrhN0em09CjxUW6QRk7pLhvyRWD+JK/JBc8mU+
# kew5ul9udpPeRo8JjyQQzB8qI8ZZZtFQl5iRp4vLxWWjaf3qHOtgnfl+86L9OLCy
# KLuzt2ppppwBOQuOl/i/yZ5JXyx+2cy21m9CmYIX0ApWYC8FngmNTtSgFCTFVu5a
# BZ21EaPGwvvw7OiE0pzY1424BvYKcR6EFQ5NOS1WNl1YvYsq8XeTONRFAFlVR6XC
# OVsaLnQpIMOhi5V7kTCsS5/OH5iNJNK5LVPF2R3e6F9ShcWiNVhR0RONkjpu3e/5
# OIXBfw4swO7rCDZvHg5dSW/up1KRs3XN+jGgvj0CONzxEcXJ/2VJTVdVdv6TqQUn
# dg8q0yfwxHbQZ/lfNH8wRhOmQw35gI3sZk/rGtV+0dTwK2ohQFBewCq6x5aLTKkY
# gMrZ5UvFntpTUBJUEw6L+56qLp3yQk3t5Wn43oQ04iRO6rAeGmHEw9IiVbF/jkxo
# ohe5DzZiUNErd8jbuFNgd/xMRvjKvLgxfwdqnos8yT0wyhhNa88pBUDcFP0diE0j
# sJGFjziBhfLhmrs+2VEsknY1I/Y0yJGN7ENyA5/+VnrW3Hlaa8A=
# =rwvm
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 06 May 2026 10:10:57 EDT
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE
* tag 'single-binary-20260506' of https://github.com/philmd/qemu: (110 commits)
system/vl: inline qemu_opts_parse_noisily() result checks
scripts/checkpatch: Avoid false positive on empty blocks
cocci: Do not initialize variable used by RAMBLOCK_FOREACH* macro
cocci: Do not initialize variable used by QTAILQ_FOREACH macro
cocci: Do not initialize variable used by QSIMPLEQ_FOREACH macro
cocci: Do not initialize variable used by QSLIST_FOREACH macro
cocci: Do not initialize variable used by QLIST_FOREACH macro
scripts/checkpatch: Reject another license boilerplate pattern
io: use g_clear_handle_id() for GSource cleanup
io: Clear dangling GLib event source tag
target/xtensa/core: register types using type_init
target/s390x: Do not compile KVM stubs for linux-user binary
target/mips: Do not initialize variable used by CPU_FOREACH macro
target/mips: Reduce CPUState scope when used with CPU_FOREACH()
target/riscv: Iterate vCPUs using CPU_FOREACH() macro
target/s390x: Replace cpu_stb_data_ra -> cpu_stb_mmu in STFLE opcode
target/s390x: Compile crypto_helper.c as common unit
target/s390x: Have MSA helper pass a mmu_idx argument
target/s390x: Compile vec_helper.c as common unit
target/s390x: Compile translate.c as common unit
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
VMSTATE_VARRAY_MULTIPLY() seems to be an "optimized" version
of VMSTATE_UINTTL_ARRAY(), and is only used once. Prefer the
generic macro. Bump the SPARC CPU migration stream version.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-Id: <20260325211728.89204-10-philmd@linaro.org>
Use TargetInfo target_long_bits() helper at runtime to migrate
the VPN register as 32 or 64-bits.
The "migration/qemu-file-types.h" isn't required anymore in
"migration/cpu.h", however it was missing in "ppc/machine.c".
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-Id: <20260325211728.89204-7-philmd@linaro.org>
Last use of VMSTATE_UINTTL_TEST() was removed in commit 445d3facff
("ppc/spapr: remove deprecated machine pseries-2.7").
The following definitions were added in commit b6c4f71f50 ("Resurrect
the safe part of r5274") but never used:
- qemu_put_sbetl qemu_put_sbe64
- qemu_get_sbetl qemu_get_sbe64
- qemu_put_sbetls qemu_put_sbe64s
- qemu_get_sbetls qemu_get_sbe64s
Remove all as unused.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20260325211728.89204-5-philmd@linaro.org>
Allow modules to report data that can only be migrated after VM is stopped.
When this concept is introduced, we will need to account stopcopy size to
be part of pending_size as before.
However, when there're data only can be migrated in stopcopy phase, it
means the old "pending_size" may not always be able to reach low enough to
kickoff an slow version of query sync.
It used to be almost guaranteed to happen as all prior iterative modules
doesn't have stopcopy only data. VFIO may change that fact by having some
data that must be copied during stop phase.
So we need to make sure QEMU will kickoff a synchronized version of query
pending when all precopy data is migrated. This might be important to VFIO
to keep making progress even if the downtime cannot yet be satisfied.
So far, this patch should introduce no functional change, as no module yet
report stopcopy size.
This paves way for VFIO to properly report its pending data sizes, which
will start to include stop-only data.
Reviewed-by: Avihai Horon <avihaih@nvidia.com>
Reviewed-by: Juraj Marcin <jmarcin@redhat.com>
Link: https://lore.kernel.org/r/20260421202110.306051-8-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Introduce a new flag, VMS_ARRAY_OF_POINTER_AUTO_ALLOC, for VMSD field. It
must be used together with VMS_ARRAY_OF_POINTER.
It can be used to allow migration of an array of pointers where the
pointers may point to NULLs.
Note that we used to allow migration of a NULL pointer within an array that
is being migrated. That corresponds to the code around vmstate_info_nullptr
where we may get/put one byte showing that the element of an array is NULL.
That usage is fine but very limited, it's because even if it will migrate a
NULL pointer with a marker, it still works in a way that both src and dest
QEMUs must know exactly which elements of the array are non-NULL, so
instead of dynamically loading an array (which can have NULL pointers), it
actually only verifies the known NULL pointers are still NULL pointers
after migration.
Also, in that case since dest QEMU knows exactly which element is NULL,
which is not NULL, dest QEMU's device code will manage all allocations for
the elements before invoking vmstate_load_vmsd().
That's not enough per evolving needs of new device states that may want to
provide real dynamic array of pointers, like what Alexander proposed here
with the NVMe device migration:
https://lore.kernel.org/r/20260317102708.126725-1-alexander@mihalicyn.com
This patch is an alternative approach to address the problem.
Along with the flag, introduce two new macros:
VMSTATE_VARRAY_OF_POINTER_TO_STRUCT_UINT{8|32}_ALLOC()
Which will be used very soon in the NVMe series.
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Tested-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juraj Marcin <jmarcin@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260401202844.673494-10-peterx@redhat.com
Signed-off-by: Fabiano Rosas <farosas@suse.de>
We used to have one vmstate called "nullptr" which is only used to generate
one-byte hint to say one pointer is NULL.
Let's extend its use so that it will generate another byte to say the
pointer is non-NULL.
With that, the name of the info struct (or functions) do not apply anymore.
Update correspondingly.
Update analyze-migration.py to work with the new layout.
No functional change intended yet.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juraj Marcin <jmarcin@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260401202844.673494-8-peterx@redhat.com
Signed-off-by: Fabiano Rosas <farosas@suse.de>
When VMS_ARRAY_OF_POINTER is specified, it means the vmstate field is an
array of pointers.
The size of the element is not relevant to whatever it is stored inside: it
is always the host pointer size.
Let's reserve the "size" field in this case for future use, update
vmstate_size() so as to make it still work for array of pointers properly.
When at this, provide rich documentation on how size / size_offset works in
vmstate.
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juraj Marcin <jmarcin@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260401202844.673494-4-peterx@redhat.com
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Introduce new APIs, returning bool.
The analysis
https://lore.kernel.org/qemu-devel/aQDdRn8t0B8oE3gf@x1.local/
shows, that vmstate_load_state() return value actually only
used to check for success, specific errno values doesn't make
sense.
With this commit we introduce new functions with modern bool
interface, and in following commits we'll update the
code base to use them, starting from migration/ code, and
finally we will remove old vmstate_load_state() and
vmstate_save_state().
This patch reworks existing functions to new one, so that
old interfaces are simple wrappers, which will be easy to
remove later.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260304212303.667141-14-vsementsov@yandex-team.ru
Signed-off-by: Fabiano Rosas <farosas@suse.de>
All other handlers now have _errp() variants. Should we go this way
for .post_save()? Actually it's rather strange, when the vmstate do
successful preparations in .pre_save(), then successfully save all
sections and subsections, end then fail when all the state is
successfully transferred to the target.
Happily, we have only three .post_save() realizations, all always
successful. Let's make this a rule.
Also note, that we call .post_save() in two places, and handle
its (theoretical) failure inconsistently. Fix that too.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com> #rust
Link: https://lore.kernel.org/qemu-devel/20260304212303.667141-4-vsementsov@yandex-team.ru
Signed-off-by: Fabiano Rosas <farosas@suse.de>
All three events are shared between precopy and postcopy, rather than
precopy specific.
For example, both precopy and postcopy will go through a SETUP process.
Meanwhile, both FAILED and DONE notifiers will be notified for either
precopy or postcopy on completions / failures.
Rename them to make them match what they do, and shorter.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20260126213614.3815900-6-peterx@redhat.com
[fixed-up entry in scsi-disk.c that got merged first]
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Migration notifiers will notify at any of three places: (1) SETUP
phase, (2) migration completes, (3) migration fails.
There's actually a special case for spice: one can refer to
b82fc321bf ("Postcopy+spice: Pass spice migration data earlier"). It
doesn't need another 4th event because in commit 9d9babf78d ("migration:
MigrationEvent for notifiers") we merged it together with the DONE event.
The merge makes some sense if we treat "switchover" of postcopy as "DONE",
however that also means for postcopy we'll notify DONE twice.. The other
one at the end of postcopy when migration_cleanup().
In reality, the current code base will also notify FAILED for postcopy
twice. It's because an (maybe accidental) change in commit
4af667f87c ("migration: notifier error checking").
First of all, we still need that notification when switchover as stated in
Dave's commit, however that's only needed for spice. To fix it, introduce
POSTCOPY_START event to differenciate it from DONE. Use that instead in
postcopy_start(). Then spice will need to capture this event too.
Then we remove the extra FAILED notification in postcopy_start().
If one wonder if other DONE users should also monitor POSTCOPY_START
event.. We have two more DONE users:
- kvm_arm_gicv3_notifier
- cpr_exec_notifier
Both of them do not need a notification for POSTCOPY_START, but only when
migration completed. Actually, both of them are used in CPR, which doesn't
support postcopy.
When at this, update the notifier transition graph in the comment, and move
it from migration_add_notifier() to be closer to where the enum is defined.
I didn't attach Fixes: because I am not aware of any real bug on such
double reporting. I'm wildly guessing the 2nd notify might be silently
ignored in many cases. However this is still worth fixing.
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Dr. David Alan Gilbert <dave@treblig.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/qemu-devel/20260126213614.3815900-3-peterx@redhat.com
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Next commit will re-use VMSTATE_BUFFER_POINTER_UNSAFE().
This reverts commit 58341158d0.
Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
No code actually depend on specific errno values returned by
vmstate_load_state. The only use of it is to check for success,
and sometimes inject numeric error values into error messages
in migration code. The latter is not a stopper for gradual
conversion to "errp + bool return value" APIs.
Big analysis of vmstate_load_state() callers, showing that
specific errno values are not actually used, is done by Peter
here:
https://lore.kernel.org/qemu-devel/aQDdRn8t0B8oE3gf@x1.local/
Converting of vmstate_load_state() itself will follow in
another series.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Link: https://lore.kernel.org/r/20251028170926.77219-2-vsementsov@yandex-team.ru
Signed-off-by: Peter Xu <peterx@redhat.com>
Migration/Memory Pull for 10.2
- PeterX's fix on tls warning for preempt channel when migratino completes
- Arun's series to enhance error reporting for vTPM and migration framework
- PeterX's patch to cleanup multifd send TLS BYE messages
- Juraj's fix on postcopy start state transition when switchover failed
- Yanfei's fix to migrate APIC before VFIO-PCI to avoid irq fallbacks
- Dan's cleanup to simplify error reporting in qemu_fill_buffer()
- PeterM's fix on address space leak when cpu hot plug / unplug
- Steve's cpr-exec wholeset
# -----BEGIN PGP SIGNATURE-----
#
# iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCaN/uIhIccGV0ZXJ4QHJl
# ZGhhdC5jb20ACgkQO1/MzfOr1wZ+mAEA1l2RS9sZS1W3vXQMCNb+Nu8Uo2p+e5Qj
# Uu6J0WVV+XsBANtzGZk2UM/frqlABywW3/ozJ4qBvIPKo758Mr6/lqUH
# =asUv
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 03 Oct 2025 08:39:14 AM PDT
# gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706
# gpg: issuer "peterx@redhat.com"
# gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [unknown]
# gpg: aka "Peter Xu <peterx@redhat.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706
* tag 'staging-pull-request' of https://gitlab.com/peterx/qemu: (45 commits)
migration-test: test cpr-exec
vfio: cpr-exec mode
migration: cpr-exec docs
migration: cpr-exec mode
migration: cpr-exec save and load
migration: cpr-exec-command parameter
oslib: qemu_clear_cloexec
migration: add cpr_walk_fd
migration: multi-mode notifier
migration: simplify error reporting after channel read
physmem: Destroy all CPU AddressSpaces on unrealize
memory: New AS helper to serialize destroy+free
include/system/memory.h: Clarify address_space_destroy() behaviour
migration: ensure APIC is loaded prior to VFIO PCI devices
migration: Fix state transition in postcopy_start() error handling
migration/multifd/tls: Cleanup BYE message processing on sender side
migration: HMP: Adjust the order of output fields
migration: Make migration_has_failed() work even for CANCELLING
io/crypto: Move tls premature termination handling into QIO layer
backends/tpm: Propagate vTPM error on migration failure
...
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Add the cpr-exec migration mode. Usage:
qemu-system-$arch -machine aux-ram-share=on ...
migrate_set_parameter mode cpr-exec
migrate_set_parameter cpr-exec-command \
<arg1> <arg2> ... -incoming <uri-1> \
migrate -d <uri-1>
The migrate command stops the VM, saves state to uri-1,
directly exec's a new version of QEMU on the same host,
replacing the original process while retaining its PID, and
loads state from uri-1. Guest RAM is preserved in place,
albeit with new virtual addresses.
The new QEMU process is started by exec'ing the command
specified by the @cpr-exec-command parameter. The first word of
the command is the binary, and the remaining words are its
arguments. The command may be a direct invocation of new QEMU,
or may be a non-QEMU command that exec's the new QEMU binary.
This mode creates a second migration channel that is not visible
to the user. At the start of migration, old QEMU saves CPR state
to the second channel, and at the end of migration, it tells the
main loop to call cpr_exec. New QEMU loads CPR state early, before
objects are created.
Because old QEMU terminates when new QEMU starts, one cannot
stream data between the two, so uri-1 must be a type,
such as a file, that accepts all data before old QEMU exits.
Otherwise, old QEMU may quietly block writing to the channel.
Memory-backend objects must have the share=on attribute, but
memory-backend-epc is not supported. The VM must be started with
the '-machine aux-ram-share=on' option, which allows anonymous
memory to be transferred in place to the new process. The memfds
are kept open across exec by clearing the close-on-exec flag, their
values are saved in CPR state, and they are mmap'd in new QEMU.
Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Link: https://lore.kernel.org/r/1759332851-370353-7-git-send-email-steven.sistare@oracle.com
Signed-off-by: Peter Xu <peterx@redhat.com>
The load procedure of VFIO PCI devices involves setting up IRT
for each VFIO PCI devices. This requires determining whether an
interrupt is single-destination interrupt to decide between
Posted Interrupt(PI) or remapping mode for the IRTE. However,
determining this may require accessing the VM's APIC registers.
For example:
ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, irqs)
...
kvm_arch_irq_bypass_add_producer
kvm_x86_call(pi_update_irte)
vmx_pi_update_irte
kvm_intr_is_single_vcpu
If the LAPIC has not been loaded yet, interrupts will use remapping
mode. To prevent the fallback of interrupt mode, keep APIC is always
loaded prior to VFIO PCI devices.
Signed-off-by: Yicong Shen <shenyicong.1023@bytedance.com>
Signed-off-by: Yanfei Xu <yanfei.xu@bytedance.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Link: https://lore.kernel.org/r/20250818131127.1021648-1-yanfei.xu@bytedance.com
Signed-off-by: Peter Xu <peterx@redhat.com>
- We need to have good error reporting in the callbacks in
VMStateDescription struct. Specifically pre_save, pre_load
and post_load callbacks.
- It is not possible to change these functions everywhere in one
patch, therefore, we introduce a duplicate set of callbacks
with Error object passed to them.
- So, in this commit, we implement 'errp' variants of these callbacks,
introducing an explicit Error object parameter.
- This is a functional step towards transitioning the entire codebase
to the new error-parameterized functions.
- Deliberately called in mutual exclusion from their counterparts,
to prevent conflicts during the transition.
- New impls should preferentally use 'errp' variants of
these methods, and existing impls incrementally converted.
The variants without 'errp' are intended to be removed
once all usage is converted.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Arun Menon <armenon@redhat.com>
Tested-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
Link: https://lore.kernel.org/r/20250918-propagate_tpm_error-v14-26-36f11a6fb9d3@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
