[SeaBIOS] [Qemu-devel] [RFC PATCH v4 00/30] ACPI memory hotplug

[Erlon Cruz]

On Tue, Dec 18, 2012 at 10:41 AM, Vasilis Liaskovitis <
vasilis.liaskovitis at profitbricks.com> wrote:

> This is v4 of the ACPI memory hotplug functionality. Only x86_64 target is
> supported (both i440fx and q35). There are still several issues, but it's
> been a while since v3 and I wanted to get some more feedback on the current
> state of the patchseries.
>
>
We are working in memory hotplug functionality on pSeries machine. I'm
wondering whether and how we can better integrate things. Do you think the
DIMM abstraction is generic enough to be used in other machine types?


> Overview:
>
> Dimm device layout is modeled with a normal qemu device:
>
> "-device dimm,id=name,size=sz,node=pxm,populated=on|off,bus=membus.0"
>
>
 How does this will handle the no-hotplugable memory for example the memory
passed in '-m' parameter?


> The starting physical address for all dimms is calculated from top of
> memory,
> during memory controller init, skipping the pci hole at [PCI_HOLE_START,
> 4G).
> e.g.
> "-device dimm,id=dimm0,size=512M,node=0,populated=off,bus=membus.0"
> will define a 512M memory dimm belonging to numa node 0, on bus membus.0.
>
> Because dimm layout needs to be configured on machine-boot, all dimm
> devices
> need to be specified on startup command line (either with populated=on or
> with
> populated=off). The dimm information is stored in dimm configuration
> structures.
>
> After machine startup, dimms are hot-added or removed with normal
> device_add
> and device_del operations e.g.:
> Hot-add syntax: "device_add dimm,id=mydimm0,bus=membus.0"
> Hot-remove syntax: "device_del dimm,id=mydimm0"
>
> Changes v3->v4
>
> - Dimms added with normal -device argument (extra -dimm arg dropped).
> - multiple memory buses can be registered. Memory buses of the real
> hw/chipset
>   or a paravirtual memory bus can be added.
> - acpi implementation uses memory API instead of old ioports.
> - Support for q35/ich9 added (still buggy, see patch 12/31).
> - piix4/i440fx initialization code has been refactored to resemble q35.
> This
> will allow memory map initialization at chipset qdev init time for both
> machines, as well as more similar code.
> - Hot-remove functionality has been moved to separate patches. Hot-remove
> no
> longer frees memory but unmaps the dimm/qdev device from the guest's view.
> Freeing the memory should happen when the last user unrefs/unmaps the
> memory,
> see also (work in progress):
> https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg00728.html
> https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg02697.html
> - new qmp/hmp command for the state of each dimm (on/off)
>
> Changes v2->v3
>
> - qdev integration. Dimms are attached to a dimmbus. The dimmbus is a child
>   of i440fx device in the pc machine. Hot-add and remove are done with
> normal
>   device_add / device_del operations on the dimmbus. New commands
> "dimm_add" and
>   "dimm_del" are obsolete.
> - Add _PS3 method to allow OSPM-induced hot operations.
> - pci-window calculation in Seabios takes dimms into account(for both
> 32-bit and
>   64-bit windows)
> - rename new qmp commands: query-memory-total and query-memory-hotplug
> - balloon driver can see the hotplugged memory
>
> Changes v1->v2
>
> - memory map is automatically calculated for hotplug dimms. Dimms are
> added from
> top-of-memory skipping the pci hole at [PCI_HOLE_START, 4G).
> - Renamed from "-memslot" to "-dimm". Commands changed to "dimm_add",
> "dimm_del"
> - Seabios ejection array reduced to a byte. Use extraction macros for dimm
> ssdt.
> - additional SRAT paravirt info does not break previous SRAT fw_cfg layout.
> - Documentation of new acpi_piix4 registers and paravirt data.
> - add ACPI _OST support for _OST enabled guests. This allows qemu to
> receive
> notification for success / failure of memory hot-add and hot-remove
> operations.
> Guest needs to support _OST (https://lkml.org/lkml/2012/6/25/321)
> - add monitor info command to report total guest memory (initial +
> hot-added)
>
> Issues:
>
> - hot-remove needs to only unmap the dimm device from guest's view.
> Freeing the
> memory should happen when the last user of the device (e.g. virtio-blk)
> unrefs
> the device. A testcase is needed for this.
>
> - Live Migration: Ramblocks are migrated before qdev VMStates are
> migrated. So
> the DimmDevice is handled diferrently than other devices. Should this be
> reworked ?( DimmDevice structure currently does not define a
> VMStateDescription)
> Live migration works as long as the dimm layout (command line args) are
> identical at the source and destination qemu command line, and destination
> takes
> into account hot-operations that have occured on source. (v3 patch 10/19
> created the DimmDevice that corresponds to an unknown incoming ramblock,
> e.g.
> for a dimm that was hot-added on source. but has been dropped for the
> moment).
>
> - A main blocker issue is windows guest functionality. The patchset does
> not
> work for windows currently.  Testing on win2012 server RC or windows2008
> consumer prerelease, when adding a DIMM, there is a BSOD with
> ACPI_BIOS_ERROR
> message. After this, the VM keeps rebooting with ACPI_BIOS_ERROR. The
> windows
> pnpmem driver obviosuly has a problem with the seabios dimm implementation
> (or the seabios dimm implementation is not fully ACPI-compliant). If
> someone
> can review the seabios patches or has any ideas to debug this, let me know.
>
> - hot-operation notification lists need to be added to migration state.
>
> series is based on:
> - qemu master (commit a8a826a3) + patch:
> https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg02699.html
> - seabios master (commit a810e4e7)
>
> Can also be found at:
>
> http://github.com/vliaskov/qemu-kvm/commits/memhp-v4
> http://github.com/vliaskov/seabios/commits/memhp-v4
>
> Vasilis Liaskovitis (21):
>   qapi: make visit_type_size fallback to type_int
>   Add SIZE type to qdev properties
>   qemu-option: export parse_option_number
>   Implement dimm device abstraction
>   vl: handle "-device dimm"
>   acpi_piix4 : Implement memory device hotplug registers
>   acpi_ich9 : Implement memory device hotplug registers
>   piix_pci and pc_piix: refactor
>   piix_pci: Add i440fx dram controller initialization
>   q35: Add i440fx dram controller initialization
>   pc: Add dimm paravirt SRAT info
>   Introduce paravirt interface QEMU_CFG_PCI_WINDOW
>   Implement "info memory-total" and "query-memory-total"
>   balloon: update with hotplugged memory
>   Implement dimm-info
>   dimm: add hot-remove capability
>   acpi_piix4: add hot-remove capability
>   acpi_ich9: add hot-remove capability
>   Implement qmp and hmp commands for notification lists
>   Add _OST dimm support
>   Implement _PS3 for dimm
>
>  docs/specs/acpi_hotplug.txt |   54 ++++++
>  docs/specs/fwcfg.txt        |   28 +++
>  hmp-commands.hx             |    6 +
>  hmp.c                       |   41 ++++
>  hmp.h                       |    3 +
>  hw/Makefile.objs            |    2 +-
>  hw/acpi.h                   |    5 +
>  hw/acpi_ich9.c              |  115 +++++++++++-
>  hw/acpi_ich9.h              |   12 +-
>  hw/acpi_piix4.c             |  126 ++++++++++++-
>  hw/dimm.c                   |  444
> +++++++++++++++++++++++++++++++++++++++++++
>  hw/dimm.h                   |  102 ++++++++++
>  hw/fw_cfg.h                 |    1 +
>  hw/lpc_ich9.c               |    2 +-
>  hw/pc.c                     |   28 +++-
>  hw/pc.h                     |    1 +
>  hw/pc_piix.c                |   74 ++++++--
>  hw/pc_q35.c                 |   18 ++-
>  hw/piix_pci.c               |  249 ++++++++-----------------
>  hw/q35.c                    |   27 +++
>  hw/q35.h                    |    5 +
>  hw/qdev-properties.c        |   60 ++++++
>  hw/qdev-properties.h        |    3 +
>  hw/virtio-balloon.c         |   13 +-
>  monitor.c                   |   21 ++
>  qapi-schema.json            |   63 ++++++
>  qapi/qapi-visit-core.c      |   11 +-
>  qemu-option.c               |    4 +-
>  qemu-option.h               |    4 +
>  qmp-commands.hx             |   57 ++++++
>  sysemu.h                    |    1 +
>  vl.c                        |   60 ++++++
>  32 files changed, 1432 insertions(+), 208 deletions(-)
>  create mode 100644 docs/specs/acpi_hotplug.txt
>  create mode 100644 docs/specs/fwcfg.txt
>  create mode 100644 hw/dimm.c
>  create mode 100644 hw/dimm.h
>
>
> Vasilis Liaskovitis (9):
>   Add ACPI_EXTRACT_DEVICE* macros
>   Add SSDT memory device support
>   acpi-dsdt: Implement functions for memory hotplug
>   acpi: generate hotplug memory devices
>   q35: Add memory hotplug handler
>   pci: Use paravirt interface for pcimem_start and pcimem64_start
>   acpi: add _EJ0 operation and eject port for memory devices
>   Add _OST dimm method
>   Implement _PS3 method for memory device
>
>  Makefile                      |    2 +-
>  src/acpi-dsdt-mem-hotplug.dsl |  136 +++++++++++++++++++++++++++++++++++
>  src/acpi-dsdt.dsl             |    5 +-
>  src/acpi.c                    |  158
> +++++++++++++++++++++++++++++++++++++++--
>  src/paravirt.c                |    6 ++
>  src/paravirt.h                |    2 +
>  src/pciinit.c                 |    9 +++
>  src/q35-acpi-dsdt.dsl         |    6 +-
>  src/ssdt-mem.dsl              |   73 +++++++++++++++++++
>  tools/acpi_extract.py         |   28 +++++++
>  10 files changed, 415 insertions(+), 10 deletions(-)
>  create mode 100644 src/acpi-dsdt-mem-hotplug.dsl
>  create mode 100644 src/ssdt-mem.dsl
>
> --
> 1.7.9
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.seabios.org/pipermail/seabios/attachments/20130219/a2e3d208/attachment-0001.html>