Hi All,
With QEMU 7.0.0 and SeaBIOS 1.16.0, virtual machines with a PCIe graphics card connected to a PCIe Root Port, rather than the Root Complex, as recommended by the QEMU PCI Express Guidelines[1], are unable to use VESA BIOS Extensions display modes. For example,
qemu-system-x86_64 \ -no-user-config \ -nodefaults \ -machine q35,accel=kvm \ -m 1G \ -cdrom "$iso" \ -device VGA
works well and can use many VBE display modes, while
qemu-system-x86_64 \ -no-user-config \ -nodefaults \ -machine q35,accel=kvm \ -m 1G \ -cdrom "$iso" \ -device pcie-root-port,id=pci.1,bus=pcie.0 \ -device VGA,bus=pci.1
only shows a few text display modes. This is the case for VGA, qxl-vga, and virtio-vga devices. `info qtree` shows the device as `type: PCIE` in both configurations. High-resolution graphics modes are available when using UEFI (OVMF) in both configurations.
Am I correct in assuming this is a limitation of SeaVGABIOS? Is this expected behavior, or a bug? If it's unlikely to change soon, would it make sense for me to propose documenting this limitation in the QEMU PCI Express Guidelines?
I'd appreciate any insights you might have.
Thanks, Kevin
On Tue, Jun 07, 2022 at 09:37:21AM -0600, Kevin Locke wrote:
Hi All,
With QEMU 7.0.0 and SeaBIOS 1.16.0, virtual machines with a PCIe graphics card connected to a PCIe Root Port, rather than the Root Complex, as recommended by the QEMU PCI Express Guidelines[1], are unable to use VESA BIOS Extensions display modes. For example,
Am I correct in assuming this is a limitation of SeaVGABIOS?
Well, sort of. There are two ways to reach the registers needed for the extended display modes, via ioports 0x01ce,0x01cf or via mmio bar (see docs/specs/standard-vga.txt in qemu source tree).
The ioport access is not forwarded over pci(e) bridges, the special exception bit for vga register access applies to the classic vga registers only not the vbe registers.
vgabios functions must be able to run in real mode, so it uses the ioports, because they are easily reachable.
ovmf driver runs in 32bit or 64bit anyway, so using the mmio bar is no problem and the driver prefers that if available. Therefore it is possible to plug vga devices into a pcie root port (although I think it doesn't work for qxl due to mmio bar not being available there).
If it's unlikely to change soon, would it make sense for me to propose documenting this limitation in the QEMU PCI Express Guidelines?
Yes, it makes sense to add an exception note there. VGA devices should be placed on the root bus to make sure they are fully functional (libvirt does that by default btw).
When using UEFI it is possible to pick a device without vga compatibility (i.e. bochs-display or virtio-gpu-pci). The legacy vga stuff (text mode and lowres graphics) wouldn't be used anyway ...
take care, Gerd
On Wed, 2022-06-08 at 12:49 +0200, Gerd Hoffmann wrote:
On Tue, Jun 07, 2022 at 09:37:21AM -0600, Kevin Locke wrote:
With QEMU 7.0.0 and SeaBIOS 1.16.0, virtual machines with a PCIe graphics card connected to a PCIe Root Port, rather than the Root Complex, as recommended by the QEMU PCI Express Guidelines[1], are unable to use VESA BIOS Extensions display modes. For example,
Am I correct in assuming this is a limitation of SeaVGABIOS?
Well, sort of. There are two ways to reach the registers needed for the extended display modes, via ioports 0x01ce,0x01cf or via mmio bar (see docs/specs/standard-vga.txt in qemu source tree).
The ioport access is not forwarded over pci(e) bridges, the special exception bit for vga register access applies to the classic vga registers only not the vbe registers.
vgabios functions must be able to run in real mode, so it uses the ioports, because they are easily reachable.
ovmf driver runs in 32bit or 64bit anyway, so using the mmio bar is no problem and the driver prefers that if available. Therefore it is possible to plug vga devices into a pcie root port (although I think it doesn't work for qxl due to mmio bar not being available there).
If it's unlikely to change soon, would it make sense for me to propose documenting this limitation in the QEMU PCI Express Guidelines?
Yes, it makes sense to add an exception note there. VGA devices should be placed on the root bus to make sure they are fully functional (libvirt does that by default btw).
When using UEFI it is possible to pick a device without vga compatibility (i.e. bochs-display or virtio-gpu-pci). The legacy vga stuff (text mode and lowres graphics) wouldn't be used anyway ...
Thanks Gerd, I really appreciate the explanation! I've tried to incorporate your explanation in a patch to the QEMU PCI Express Guidelines that I posted to qemu-devel for discussion.[1] Feel free to follow up there with any improvements or if I've misunderstood anything.
Re: libvirt. Interestingly, libvirt only appears to do so for the first/primary display device.[2] I ran into this issue after adding a second video device to a VM in virt-manager, then removing the first, which left the only video device on a Root Port leading to the missing display modes. I'm not sure if it's a bug worth filing/fixing. Let me know if you have any thoughts on it.
Thanks again, Kevin
[1]: https://lore.kernel.org/qemu-devel/922cc3081ff9c986188f881ef4d1cf15bd3adf48.... [2]: https://gitlab.com/libvirt/libvirt/-/blob/v8.4.0/src/qemu/qemu_domain_addres...
-
Gerd Hoffmann -
Kevin Locke