On Wed, 09 Oct 2013 15:12:08 +0200 Gerd Hoffmann kraxel@redhat.com wrote:
On Mi, 2013-10-09 at 14:23 +0200, Igor Mammedov wrote:
I'm posting it to get an oppinion on one of possible approaches on where to map a hotplug memory.
This patch assumes that a space for hotplug memory is located right after RamSizeOver4G region and QEMU will provide romfile to specify where it ends so that BIOS could know from what base to start 64-bit PCI devices mapping.
We should think about both pci hotplug and memory hotplug while being at it.
Today the 64bit pci window is mapped right above high memory and is sized (in acpi tables) according to what is needed to map the devices present at boot.
Effect is that there is no extra address space for 64bit bars of hotplugged pci devices. And the window is also in the way when it comes to memory hotplug.
Given that some windows versions don't like the large 64bit windows we should make the window size configurable.
So far from QEMU side it's partially (only memory region mapping and not ACPI window) configurable via {i440FX-pcihost|q35-pcihost}.pci-hole64-size property
The window location can either be made configurable too, or we simply place it at the top of the address space, with "address space" being what the cpu can address according to cpuinfo.
An earlier attempt by Michael to push complete PCI window placement info via "etc/pci-info" romfile to Seabios was rejected in favor of letting Seabios to program windows at hardcoded(32-bit/behind high mem) locations with a 64-bit window size (in ACPI) that covers all present devices but doesn't account for future PCI hotplug either.
That behavior maintained in his "ACPI in QEMU" series, see: http://patchwork.ozlabs.org/patch/281032/ acpi_get_pci_info()->i440fx_pcihost_get_pci_hole64_end()->pci_bus_get_w64_range() which is then embedded in ACPI table. So end result stays the same as before (no usable 64-bit PCI window for hotlug).
But 64-bit PCI window size, which is capped by QEMU to insane legacy 62 bits (memory region size), is a bit of orthogonal to freeing space for memory hotplug before it.
Current qemu reports this by default:
$ cat /proc/cpuinfo model name : QEMU Virtual CPU version 1.5.3 address sizes : 40 bits physical, 48 bits virtual
40 address lines allow 1TB, so we would place the window just below 1TB.
Comments?
More to the point if OS supports/enforces 1Tb physical address space,the RAM and 64-bit PCI hole are going to contend for it, QEMU could abort on startup if they both do not fit in CPU supported address space but I don't see what else it could do.
Proposed patch favors RAM vs 64-bit PCI hole and moves the hole behind the possible RAM, which in present state of QEMU potentially leaves the rest of address space up to 62 bits for hole. It has drawback that one can't get a working VM if QEMU is started in memory hotlug mode with old BIOS + PCI devices that require 64-bit bars, otherwise it's backward compatible.
PS: As for remedying BSODs because of huge CRS sizes of particular RAM device/PCI window, it might be solved by splitting one big chunk in several smaller, at least it works for RAM device.
Gerd