Hi,
After an update some time ago, a VM wit 8GB RAM cannot boot from nvme anymore.
QEMU cmdline: /usr/bin/qemu-system-x86_64 \ -chardev ringbuf,id=serial0,logfile=serial0,logappend=on \ -serial chardev:serial0 \ -m 8192 \ -enable-kvm \ -vnc :2,share=force-shared \ -blockdev driver=file,node-name=hd0-file,filename=disk.qcow2,cache.no-flush=on,discard=unmap \ -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on,discard=unmap \ -device nvme,id=hd0-device,drive=hd0,serial=hd0 \ -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios \ -bios /home/user/seabios/out/bios.bin
There is nothing fancy on the disk, just standard grub2 setup, but also I believe it doesn't matter, as it doesn't even reach grub.
When it fails, the seabios log contains: PCI: 64: 0000038000000000 - 0000038040000000 PCI: map device bdf=00:04.0 bar 0, addr 38000000000, size 00004000 [mem] ... Searching bootorder for: /pci@i0cf8/isa@1/fdc@03f0/floppy@0 ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9) ATA controller 2 at 170/374/0 (irq 15 dev 9) Searching bootorder for: HALT Can not map memory bar over 4Gig ... Boot failed: could not read the boot disk
Working version has this instead:
PCI: map device bdf=00:04.0 bar 0, addr febf0000, size 00004000 [mem] ... Booting from Hard Disk... Booting from 0000:7c00
00:04.0 is the nvme.
I'm attaching full logs for both cases.
Bisect shows:
96a8d130a8c2e908e357ce62cd713f2cc0b0a2eb is the first bad commit commit 96a8d130a8c2e908e357ce62cd713f2cc0b0a2eb (HEAD) Author: Gerd Hoffmann kraxel@redhat.com Date: Fri Sep 9 10:17:15 2022 +0200
be less conservative with the 64bit pci io window
Current seabios code will only enable and use the 64bit pci io window in case it runs out of space in the 32bit pci mmio window below 4G.
This patch will also enable the 64bit pci io window when (a) RAM above 4G is present, and (b) the physical address space size is known, and (c) seabios is running on a 64bit capable processor.
This operates with the assumption that guests which are ok with memory above 4G most likely can handle mmio above 4G too.
In case the 64bit pci io window is enabled also assign more memory to prefetchable pci bridge windows and the complete 64bit pci io window.
The total mmio window size is 1/8 of the physical address space. Minimum bridge windows size is 1/256 of the total mmio window size.
Signed-off-by: Gerd Hoffmann kraxel@redhat.com
I've tried the recent "[PATCH] limit address space used for pci devices, part two" patch from the ML, but it doesn't help. OTOH, the series with "only enable 64bit pci io window when RAM >64G" does help.