On Thu, Apr 19, 2012 at 04:08:38PM +0200, Vasilis Liaskovitis wrote:
This is a prototype for ACPI memory hotplug on x86_64 target. Based on some earlier work and comments from Gleb.
Memslot devices are modeled with a new qemu command line
"-memslot id=name,start=start_addr,size=sz,node=pxm"
user is responsible for defining memslots with meaningful start/size values, e.g. not defining a memory slot over a PCI-hole. Alternatively, the start size could also be handled/assigned automatically from the specific emulated hardware (for hw/pc.c PCI hole is currently [below_4g_mem_size, 4G), so hotplugged memory should start from max(4G, above_4g_mem_size).
Node is defining numa proximity for this memslot. When not defined it defaults to zero.
e.g. "-memslot id=hot1,start=4294967296,size=536870912,node=0" will define a 512M memory slot starting at physical address 4G, belonging to numa node 0.
Memory slots are added or removed with a new hmp command "memslot": Hot-add syntax: "memslot id add" Hot-remove syntax: "memslot id delete"
- All memslots are initially unpopulated. Memslots are currently modeling only
We can add ,populated=true to -memslot to make slot populated from the start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
- Memslots are abstracted as qdevices attached to the main system bus. However,
memory hotplugging has its own side channel ignoring main_system_bus's hotplug incapability. A cleaner integration would be needed. What's the preferred way of modeling memory devices in qom? Would it be better to attach memory devices as children-links of an acpi-capable device (in the pc case acpi_piix4) instead of the system bus?
- Refcounting memory slots has been discussed (but is not included in this
series yet). Depopulating a memory region happens on a guestOS _EJ callback, which means the guestOS will not be using the region anymore. However, guest addresses from the depopulated region need to also be unmapped from the qemu address space using cpu_physical_memory_unmap(). Does memory_region_del_subregion() or some other memory API call guarantee that a memoryregion has been unmapped from qemu's address space?
- What is the expected behaviour of hotplugged memory after a reboot? Is it
supposed to be persistent after reboots? The last 2 patches in the series try to make hotplugged memslots persistent after reboot by creating and consulting e820 map entries. A better solution is needed for hot-remove after a reboot, because e820 entries can be merged.
series is based on uq/master for qemu-kvm, and master for seabios. Can be found also at:
Vasilis Liaskovitis (9): Seabios: Add SSDT memory device support Seabios, acpi: Implement acpi-dsdt functions for memory hotplug. Seabios, acpi: generate hotplug memory devices. Implement memslot device abstraction acpi_piix4: Implement memory device hotplug registers and handlers. pc: pass paravirt info for hotplug memory slots to BIOS Implement memslot command-line option and memslot hmp monitor command pc: adjust e820 map on hot-add and hot-remove Seabios, acpi: enable memory devices if e820 entry is present
Makefile.objs | 2 +- hmp-commands.hx | 15 ++++ hw/acpi_piix4.c | 103 +++++++++++++++++++++++++++- hw/memslot.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/memslot.h | 44 ++++++++++++ hw/pc.c | 87 ++++++++++++++++++++++-- hw/pc.h | 1 + monitor.c | 8 ++ monitor.h | 1 + qemu-config.c | 25 +++++++ qemu-options.hx | 8 ++ sysemu.h | 1 + vl.c | 44 ++++++++++++- 13 files changed, 528 insertions(+), 12 deletions(-) create mode 100644 hw/memslot.c create mode 100644 hw/memslot.h
create mode 100644 src/ssdt-mem.dsl src/acpi-dsdt.dsl | 68 ++++++++++++++++++++++- src/acpi.c | 155 +++++++++++++++++++++++++++++++++++++++++++++++++++-- src/memmap.c | 15 +++++ src/ssdt-mem.dsl | 66 ++++++++++++++++++++++ 4 files changed, 298 insertions(+), 6 deletions(-) create mode 100644 src/ssdt-mem.dsl
-- 1.7.9
-- Gleb.
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
On Sun, Apr 22, 2012 at 05:06:43PM +0300, Avi Kivity wrote:
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
Because my feeling is that people that want to use this kind of feature what to start using it with VMs smaller than 4G. Of course not all memory have to be hot unpluggable. Making first 1M or event first 128M not unpluggable make perfect sense.
-- Gleb.
On 04/22/2012 05:09 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:06:43PM +0300, Avi Kivity wrote:
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
Because my feeling is that people that want to use this kind of feature what to start using it with VMs smaller than 4G. Of course not all memory have to be hot unpluggable. Making first 1M or event first 128M not unpluggable make perfect sense.
Can't you achieve this with -m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false?
(I don't think hotplugging below 512MB is needed, but I don't have any real data on this).
On Sun, Apr 22, 2012 at 05:13:27PM +0300, Avi Kivity wrote:
On 04/22/2012 05:09 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:06:43PM +0300, Avi Kivity wrote:
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
Because my feeling is that people that want to use this kind of feature what to start using it with VMs smaller than 4G. Of course not all memory have to be hot unpluggable. Making first 1M or event first 128M not unpluggable make perfect sense.
Can't you achieve this with -m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false?
From this:
(for hw/pc.c PCI hole is currently [below_4g_mem_size, 4G), so hotplugged memory should start from max(4G, above_4g_mem_size).
I understand that hotpluggable memory can start from above 4G only. With the config above we will have memory hole from 1G to PCI memory hole. May be not a big problem, but I do not see technical reason for the constrain.
(I don't think hotplugging below 512MB is needed, but I don't have any real data on this).
512MB looks like a reasonable limitation too, but again if there is not technical reason for having the limitation why have it?
-- Gleb.
Hi,
On Sun, Apr 22, 2012 at 05:20:59PM +0300, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:13:27PM +0300, Avi Kivity wrote:
On 04/22/2012 05:09 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:06:43PM +0300, Avi Kivity wrote:
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
Because my feeling is that people that want to use this kind of feature what to start using it with VMs smaller than 4G. Of course not all memory have to be hot unpluggable. Making first 1M or event first 128M not unpluggable make perfect sense.
Can't you achieve this with -m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false?
From this:
(for hw/pc.c PCI hole is currently [below_4g_mem_size, 4G), so hotplugged memory should start from max(4G, above_4g_mem_size).
I understand that hotpluggable memory can start from above 4G only. With the config above we will have memory hole from 1G to PCI memory hole. May be not a big problem, but I do not see technical reason for the constrain.
The 440fx spec mentions: "The address range from the top of main DRAM to 4 Gbytes (top of physical memory space supported by the 440FX PCIset) is normally mapped to PCI. The PMC forwards all accesses within this address range to PCI."
What we probably want is that the initial memory map creation takes into account all dimms specified (both populated/unpopulated) So "-m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false" would create a system map with top of memory and start of PCI-hole at 2G.
This may require some shifting of physical address offsets around 3.5GB-4GB - is this the minimum PCI hole allowed?
E.g. if we specify 4x1GB DIMMs (onlt the first initially populated) -m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false -device dimm,size=1G,populated=false -device dimm,size=1G,populated=false
we create the following memory map: dimm0: [0,1G) dimm1: [1G, 2G) dimm2: [2G, 3G) dimm3: [4G, 5G) or dimm3 is split into [3G, 3.5G) and [4G, 4.5G)
does either of these options sound reasonable?
thanks,
- Vasilis
On 04/22/2012 05:20 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:13:27PM +0300, Avi Kivity wrote:
On 04/22/2012 05:09 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:06:43PM +0300, Avi Kivity wrote:
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
hotplug-able memory slots i.e. initial system memory is not modeled with memslots. The concept could be generalized to include all memory though, or it could more closely follow kvm-memory slots.
OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
Because my feeling is that people that want to use this kind of feature what to start using it with VMs smaller than 4G. Of course not all memory have to be hot unpluggable. Making first 1M or event first 128M not unpluggable make perfect sense.
Can't you achieve this with -m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false?
From this:
(for hw/pc.c PCI hole is currently [below_4g_mem_size, 4G), so hotplugged memory should start from max(4G, above_4g_mem_size).
I understand that hotpluggable memory can start from above 4G only. With the config above we will have memory hole from 1G to PCI memory hole. May be not a big problem, but I do not see technical reason for the constrain.
(I don't think hotplugging below 512MB is needed, but I don't have any real data on this).
512MB looks like a reasonable limitation too, but again if there is not technical reason for having the limitation why have it?
I was thinking about not having tons of 128MB slots, so we don't have a configuration that is far from reality. But maybe this thinking is too conservative.
On Mon, Apr 23, 2012 at 04:31:04PM +0300, Avi Kivity wrote:
On 04/22/2012 05:20 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:13:27PM +0300, Avi Kivity wrote:
On 04/22/2012 05:09 PM, Gleb Natapov wrote:
On Sun, Apr 22, 2012 at 05:06:43PM +0300, Avi Kivity wrote:
On 04/22/2012 04:56 PM, Gleb Natapov wrote:
start. We will need it for migration anyway.
> hotplug-able memory slots i.e. initial system memory is not modeled with > memslots. The concept could be generalized to include all memory though, or it > could more closely follow kvm-memory slots. OK, I hope final version will allow for memory < 4G to be hot-pluggable.
Why is that important?
Because my feeling is that people that want to use this kind of feature what to start using it with VMs smaller than 4G. Of course not all memory have to be hot unpluggable. Making first 1M or event first 128M not unpluggable make perfect sense.
Can't you achieve this with -m 1G, -device dimm,size=1G,populated=true -device dimm,size=1G,populated=false?
From this:
(for hw/pc.c PCI hole is currently [below_4g_mem_size, 4G), so hotplugged memory should start from max(4G, above_4g_mem_size).
I understand that hotpluggable memory can start from above 4G only. With the config above we will have memory hole from 1G to PCI memory hole. May be not a big problem, but I do not see technical reason for the constrain.
(I don't think hotplugging below 512MB is needed, but I don't have any real data on this).
512MB looks like a reasonable limitation too, but again if there is not technical reason for having the limitation why have it?
I was thinking about not having tons of 128MB slots, so we don't have a configuration that is far from reality. But maybe this thinking is too conservative.
I think it is good interface to make memory that is specified with -m to be one big unpluggable slot, but slots defined with -device should start just above what -m specifies (after proper alignment). Memory hot-plug granularity is controlled by slot's size parameter.
-- Gleb.
On 04/24/2012 10:21 AM, Gleb Natapov wrote:
I was thinking about not having tons of 128MB slots, so we don't have a configuration that is far from reality. But maybe this thinking is too conservative.
I think it is good interface to make memory that is specified with -m to be one big unpluggable slot, but slots defined with -device should start just above what -m specifies (after proper alignment). Memory hot-plug granularity is controlled by slot's size parameter.
Agree.
-
Avi Kivity -
Gleb Natapov -
Vasilis Liaskovitis