On 07/11/18 05:12, Liu, Jing2 wrote:
Hi,
Recently, we tried some hotplug issues. The case is: when hotplug a device (e.g. iGPU) onto pci-bridge after guest booting up, guest reports "BAR 2: no space for [mem size 0x40000000 64bit pref]" etc.
Seabios checks all the devices under the pci-bridge when qemu launching the guest, and only allocates "size=ALIGN(sum, align)" of memory space for pci-bridge mem and pref-mem windows. So if we hotplug a big pci device like Intel GPU which needs 256M mem/pref-mem or bigger, it will fail.
If my understanding is right, we may need some other logic of the memory allocation in seabios?
Looking forward to the advice.
I suggest using the Q35 machine type, and hot-plugging the device into a PCI Express Root Port ("pcie-root-port"). The latter has properties dedicated to reserving various PCI resources specifically for hotplug purposes:
pcie-root-port.mem-reserve=size pcie-root-port.pref32-reserve=size pcie-root-port.bus-reserve=uint32 pcie-root-port.pref64-reserve=size pcie-root-port.io-reserve=size
In order to address the issue you report at the top, you would use
-device pcie-root-port,bus=pcie.0,id=root-port-XXX,pref64-reserve=1G
then hot-plug the device into "root-port-XXX".
Please see the following two files in the QEMU tree: - docs/pcie.txt - docs/pcie_pci_bridge.txt
The first provides guidelines on the PCI Express hierarchy in general, and also on hotplug in particular (see section 5).
The second is relevant here because it describes the vendor capability that QEMU and SeaBIOS (and OVMF) use, for passing the resource reservation hints from QEMU to guest firmware.
The 2nd document mainly focuses on hot-plugging a PCIE-PCI bridge into a PCIE root port, and on reserving a bus number range for the sub-hierarchy behind said hot-plugged PCIE-PCI bridge. However, the reservation mechanism is the same for other types of PCI resources, and for other types of child devices that are hot-plugged into root ports.
HTH, Laszlo