On Fri, Sep 21, 2012 at 11:17 AM, Vasilis Liaskovitis vasilis.liaskovitis@profitbricks.com wrote:
This is v3 of the ACPI memory hotplug functionality. Only x86_64 target is supported for now.
Overview:
Dimm device layout is modeled with a new qemu command line
"-dimm id=name,size=sz,node=pxm,populated=on|off"
The starting physical address for all dimms is calculated automatically from top of memory, skipping the pci hole at [PCI_HOLE_START, 4G). Node is defining numa proximity for this dimm. When not defined it defaults to zero. "-dimm id=dimm0,size=512M,node=0,populated=off" will define a 512M memory slot belonging to numa node 0.
Dimms are added or removed with normal device_add, device_del operations: Hot-add syntax: "device_add dimm,id=mydimm0" Hot-remove syntax: "dimm_del dimm,id=mydimm0"
Changes v2->v3
- qdev integration. Dimms are attached to a dimmbus. The dimmbus is a child of i440fx device in the pc machine. Hot-add and hot-remove are done with normal device_add / device_del operations on the dimmbus. New commands "dimm_add" and "dimm_del" are obsolete. (In previous versions, dimms were always present on the qdev tree, and dimm_add/del simply meant allocating or deallocating memory for the devices. This version actually does hot-operations on the qdev tree)
- Add _PS3 method to allow OSPM-induced hot operations.
- pci-window calculation in Seabios takes dimms into account(for both 32-bit and 64-bit windows)
- rename new qmp commands: query-memory-total and query-memory-hotplug
- balloon driver can see the hotplugged memory
Changes v1->v2
- memory map is automatically calculated for hotplug dimms. Dimms are added from
top-of-memory skipping the pci hole at [PCI_HOLE_START, 4G).
- Renamed from "-memslot" to "-dimm". Commands changed to "dimm_add", "dimm_del".
- Seabios ejection array reduced to a byte. Use extraction macros for dimm ssdt.
- additional SRAT paravirt info does not break previous SRAT fw_cfg layout.
- Documentation of new acpi_piix4 registers and paravirt data.
- add ACPI _OST support for _OST enabled guests. This allows qemu to receive
notification for success / failure of memory hot-add and hot-remove operations. Guest needs to support _OST (https://lkml.org/lkml/2012/6/25/321)
- add monitor info command to report total guest memory (initial + hot-added)
- add command line options and monitor commands for batch dimm
creation/population (obsolete from v3 onwards)
Issues:
- A main blocker issue is windows guest functionality. The patchset does not work for
windows currently. My guess is the windows pnpmem driver does not like the seabios dimm device implementation (or the seabios dimm implementation is not fully ACPI-compliant). If someone can review the seabios patches or has any ideas to debug this, let me know.
Testing on win2012 server RC or windows2008 consumer prerelease. When adding a DIMM, the device shows up in DeviceManager but does not work. Relevant messages:
" This device cannot start. (Code 10) Device configured(memory.inf) (UserPnP eventID 400) Device installed (memory.inf) ACPI/PNP0C80\2&daba3ff&1 was configured Device not started(PNPMEM) (Kernel-PnP eventID 411, kernelID) Device ACPI\PNP0C80\2&daba3ff&1 had a problem starting Driver Name: memory.inf (c:\Windows\system32\DRIVERS\pnpmem.sys 6.2.8400 winmain_win8rc)) Memory range:0x80000000 - 0x90000000 (Initial memory of VM is 2GB. The hotplugged DIMM was a 256GB with physical address range starting at 2GB ) Conflicting device list: No conflicts. "
Adding a 2nd or more dimms causes a crash (PNP_DETECTED_FATAL_ERROR with blue screen of death) and makes windows reboot. After this, the VM keeps rebooting with ACPI_BIOS_ERROR. The VM refuses to boot anymore once a 2nd (or more) extra dimm is plugged-in.
- Is the dimmbus the correct way to go about integrating into qdev/qom? In a v1
comment, Anthony mentioned attaching dimms directly to an i440fx device as children. Is this possible without a bus?
- Live migration works as long as the dimm layout (-dimm command line args) are
identical at the source and destination qemu command line. Patch 10/19 creates the DimmDevice that corresponds to the unknown incoming ramblock. Ramblocks are migrated before qdev VMStates are migrated (the DimmDevice structure currently does not define a VMStateDescription). So the DimmDevice is handled diferrently than other devices. If this is not acceptable, any suggestions on how should it be reworked?
- Hot-operation notification lists need to be added to migration state.
Please review. Could people state which other issues they consider blocker for including this upstream?
Please check the patches with checkpatch.pl, there are plenty of missing braces and uses of __FUNCTION__ etc.
I also have other comments to specific patches, especially the architecture does not look correct.
Does this patchset need to wait for 1.4 or could this be considered for 1.3 (assuming blockers are resolved)? The patchset has been revised every few months, but I will provide quicker version updates onwards. I can also bring this up on a weekly meeting agenda if needed.
series is based on uq/master for qemu-kvm, and master for seabios. Can be found also at:
http://github.com/vliaskov/qemu-kvm/commits/memhp-v3 http://github.com/vliaskov/seabios/commits/memhp-v3
Vasilis Liaskovitis (12): Implement dimm device abstraction Implement "-dimm" command line option acpi_piix4: Implement memory device hotplug registers pc: calculate dimm physical addresses and adjust memory map pc: Add dimm paravirt SRAT info fix live-migration when "populated=on" is missing Implement qmp and hmp commands for notification lists Implement "info memory-total" and "query-memory-total" balloon: update with hotplugged memory Add _OST dimm support Update dimm state on reset Implement _PS3 for dimm
arch_init.c | 24 ++- docs/specs/acpi_hotplug.txt | 54 ++++++ docs/specs/fwcfg.txt | 28 +++ hmp-commands.hx | 4 + hmp.c | 24 +++ hmp.h | 2 + hw/Makefile.objs | 2 +- hw/acpi_piix4.c | 114 +++++++++++- hw/dimm.c | 435 +++++++++++++++++++++++++++++++++++++++++++ hw/dimm.h | 101 ++++++++++ hw/pc.c | 55 ++++++- hw/pc.h | 6 + hw/pc_piix.c | 20 ++- hw/virtio-balloon.c | 13 +- monitor.c | 14 ++ qapi-schema.json | 37 ++++ qemu-config.c | 25 +++ qemu-options.hx | 5 + qmp-commands.hx | 57 ++++++ sysemu.h | 1 + vl.c | 51 +++++ 21 files changed, 1051 insertions(+), 21 deletions(-) create mode 100644 docs/specs/acpi_hotplug.txt create mode 100644 docs/specs/fwcfg.txt create mode 100644 hw/dimm.c create mode 100644 hw/dimm.h
Vasilis Liaskovitis (7): Add ACPI_EXTRACT_DEVICE* macros Subject: [PATCH 02/18] Add SSDT memory device support acpi-dsdt: Implement functions for memory hotplug acpi: generate hotplug memory devices Add _OST dimm method Implement _PS3 method for memory device Calculate pcimem_start and pcimem64_start from SRAT entries
Makefile | 2 +- src/acpi-dsdt.dsl | 135 ++++++++++++++++++++++++++++++- src/acpi.c | 216 ++++++++++++++++++++++++++++++++++++++++++++---- src/acpi.h | 3 + src/pciinit.c | 6 +- src/post.c | 3 + src/smp.c | 4 + src/ssdt-mem.dsl | 73 +++++++++++++++++ tools/acpi_extract.py | 28 +++++++ 9 files changed, 447 insertions(+), 23 deletions(-) create mode 100644 src/ssdt-mem.dsl
-- 1.7.9
-
Blue Swirl