Hello,
I am experimenting with video card PCI pass-through under KVM.
I am trying to get a KVM guest to work with (up to) TWO video cards assigned to it. The goal is to have flexibility of using a single quest with two monitors, or two guests with a single monitor each, etc. Note that using a single “dual-headed” video card is not a preferred option, since it can be assigned only to one guest.
So far I was successful (Debian guest OS booting ok, graphics works fine) when using a single video card (see point 3 below).
However, when assigning two video cards to a guest, SeaBIOS fails with "PCI: out of address space" error during PCI new allocation pass #1 (see point 4 below). Notice that both video cards were properly detected and assigned to requested guest PCI slots 00:06.0 and 00:07.0.
Any ideas how to fix or get around this problem?
My coding expertise is rudimentary, but I am willing to experiment with any suggestions, try patches you might provide, etc. Is there additional info I could provide to help in the testing process?
Thanks, Marek Dudek
--------------------------
My setup is a follows:
1) Using latest version of Seabios (rel-1.6.3); compiled with debug level 8. As an experiment, I also tried recent “PCI init code refactoring” patch by Kevin O'Connor, with same result.
2) Both video cards are of same type (old NVidia GeForce 7300); the cards are located in the host PCI slots 02:00.0 and 03:00.0.
$ sudo lspci -v -s 02:00.0 02:00.0 VGA compatible controller: nVidia Corporation G72 [GeForce 7300 SE/7200 GS] (rev a1) (prog-if 00 [VGA controller]) Subsystem: eVga.com. Corp. Device c429 Flags: bus master, fast devsel, latency 0, IRQ 36 Memory at f8000000 (32-bit, non-prefetchable) [size=16M] Memory at b0000000 (64-bit, prefetchable) [size=256M] Memory at f7000000 (64-bit, non-prefetchable) [size=16M] Expansion ROM at f6fe0000 [disabled] [size=128K] Capabilities: [60] Power Management version 2 Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [78] Express Endpoint, MSI 00 Capabilities: [100] Virtual Channel Capabilities: [128] Power Budgeting <?> Kernel driver in use: pci-stub
3) When using a SINGLE video card, SeaBIOS reports as follows:
$sudo kvm -drive file=/dev/sdc -device pci-assign,host=02:00.0,addr=0x6 -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios 2>&1
------------------------ Start bios (version 1.6.3-20111012_174214-rock) enabling shadow ram qemu_cfg_present=1 Find memory size Add to e820 map: 00000000 08000000 1 Add to e820 map: 00000000 00000000 1 Add to e820 map: fffc0000 00040000 2 Add to e820 map: 000a0000 00050000 -1 Add to e820 map: 000f0000 00010000 2 Add to e820 map: feffc000 00004000 2 Ram Size=0x08000000 (0x0000000000000000 high) malloc setup Add to e820 map: 07ff0000 00010000 2 pmm_malloc zone=0x000f0930 handle=ffffffff size=63248 align=10 ret=0x07fe07d0 (detail=0x07fefee0) Relocating init from 0x000e12d0 to 0x07fe07d0 (size 63248) malloc fixup reloc init ivt init bda Add to e820 map: 0009fc00 00000400 2 init pic init timer tsc calibrate start=1004667100 end=1011214380 diff=6547280 CPU Mhz=3814 math cp init init mtrr pci setup === PCI bus & bridge init === PCI: pci_bios_init_bus_rec bus = 0x0 === PCI device probing === PCI probe pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0700 (detail=0x07fe0770) PCI device 00:00.0 (vd=8086:1237 c=0600) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0660 (detail=0x07fe06d0) PCI device 00:01.0 (vd=8086:7000 c=0601) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe05c0 (detail=0x07fe0630) PCI device 00:01.1 (vd=8086:7010 c=0101) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0520 (detail=0x07fe0590) PCI device 00:01.3 (vd=8086:7113 c=0680) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0480 (detail=0x07fe04f0) PCI device 00:02.0 (vd=1013:00b8 c=0300) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe03e0 (detail=0x07fe0450) PCI device 00:03.0 (vd=10ec:8139 c=0200) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0340 (detail=0x07fe03b0) PCI device 00:06.0 (vd=10de:01d3 c=0300) Found 7 PCI devices (max PCI bus is 00) === PCI new allocation pass #1 === pmm_malloc zone=0x07fefe30 handle=ffffffff size=528 align=10 ret=0x07fe0100 (detail=0x07fe0310) PCI: check devices bus 0 === PCI new allocation pass #2 === PCI: init bases bus 0 (primary) type io max 100 sum 110 base c000 size 100: 1 bar(s), c000 -> c0ff size 10: 1 bar(s), c100 -> c10f type mem max 10000000 sum 12022000 base e0000000 size 10000000: 1 bar(s), e0000000 -> efffffff size 1000000: 2 bar(s), f0000000 -> f1ffffff size 10000: 2 bar(s), f2000000 -> f201ffff size 1000: 2 bar(s), f2020000 -> f2021fff type prefmem max 2000000 sum 2000000 base fc000000 size 2000000: 1 bar(s), fc000000 -> fdffffff PCI: map device bus 0, bfd 0x0 PCI: map device bus 0, bfd 0x8 PCI: map device bus 0, bfd 0x9 bar 4, addr c100, size 10 [io] PCI: map device bus 0, bfd 0xb PCI: map device bus 0, bfd 0x10 bar 0, addr fc000000, size 2000000 [mem] bar 1, addr f2020000, size 1000 [mem] bar 6, addr f2000000, size 10000 [mem] PCI: map device bus 0, bfd 0x18 bar 0, addr c000, size 100 [io] bar 1, addr f2021000, size 100 [mem] bar 6, addr f2010000, size 10000 [mem] PCI: map device bus 0, bfd 0x30 bar 0, addr f0000000, size 1000000 [mem] bar 1, addr e0000000, size 10000000 [mem] bar 3, addr f1000000, size 1000000 [mem] PCI: bus=0 devfn=0x00: vendor_id=0x8086 device_id=0x1237 PCI: bus=0 devfn=0x08: vendor_id=0x8086 device_id=0x7000 PCI: bus=0 devfn=0x09: vendor_id=0x8086 device_id=0x7010 PCI: bus=0 devfn=0x0b: vendor_id=0x8086 device_id=0x7113 PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8 PCI: bus=0 devfn=0x18: vendor_id=0x10ec device_id=0x8139 PCI: bus=0 devfn=0x30: vendor_id=0x10de device_id=0x01d3 PIIX3/PIIX4 init: elcr=00 0c pmm_free 0x07fe0100 (detail=0x07fe0310) init smm Found 1 cpu(s) max supported 1 cpu(s) init bios32 init PMM init PNPBIOS table init keyboard init mouse init PIR table init MPTable pmm_malloc zone=0x07fefe30 handle=ffffffff size=32768 align=10 ret=0x07fd8310 (detail=0x07fe0310) pmm_malloc zone=0x07fefe2c handle=ffffffff size=224 align=10 ret=0x000fdb10 (detail=0x07fd82e0) pmm_malloc zone=0x07fefe2c handle=ffffffff size=16 align=10 ret=0x000fdb00 (detail=0x07fd82b0) pmm_free 0x07fd8310 (detail=0x07fe0310) MP table addr=0x000fdb00 MPC table addr=0x000fdb10 size=224 init SMBIOS tables pmm_malloc zone=0x07fefe30 handle=ffffffff size=32768 align=10 ret=0x07fd8310 (detail=0x07fe0310) pmm_malloc zone=0x07fefe2c handle=ffffffff size=31 align=10 ret=0x000fdae0 (detail=0x07fd8280) pmm_malloc zone=0x07fefe2c handle=ffffffff size=263 align=10 ret=0x000fd9d0 (detail=0x07fd8250) SMBIOS ptr=0x000fdae0 table=0x000fd9d0 size=263 pmm_free 0x07fd8310 (detail=0x07fe0310) init ACPI tables pmm_malloc zone=0x07fefe34 handle=ffffffff size=116 align=10 ret=0x07ffff80 (detail=0x07fe0310) pmm_malloc zone=0x07fefe34 handle=ffffffff size=64 align=40 ret=0x07ffff40 (detail=0x07fe02e0) pmm_malloc zone=0x07fefe34 handle=ffffffff size=9609 align=10 ret=0x07ffd9b0 (detail=0x07fe02b0) pmm_malloc zone=0x07fefe34 handle=ffffffff size=158 align=10 ret=0x07ffd910 (detail=0x07fe0280) pmm_malloc zone=0x07fefe34 handle=ffffffff size=224 align=10 ret=0x07ffd830 (detail=0x07fe0250) pmm_malloc zone=0x07fefe34 handle=ffffffff size=56 align=10 ret=0x07ffd7f0 (detail=0x07fe0220) pmm_malloc zone=0x07fefe34 handle=ffffffff size=52 align=10 ret=0x07ffd7b0 (detail=0x07fe01f0) pmm_malloc zone=0x07fefe2c handle=ffffffff size=36 align=10 ret=0x000fd9a0 (detail=0x07fe01c0) ACPI tables: RSDP=0x000fd9a0 RSDT=0x07ffd7b0 Scan for VGA option rom Attempting to init PCI bdf 00:02.0 (vd 1013:00b8) Attempting to map option rom on dev 00:02.0 Option rom sizing returned f2000000 ffff0000 Inspecting possible rom at 0xf2000000 (vd=1013:00b8 bdf=00:02.0) Copying option rom (size 35840) from 0xf2000000 to c0000 Checking rom 0x000c0000 (sig aa55 size 70) Running option rom at c000:0003 Turning on vga text mode console SeaBIOS (version 1.6.3-20111012_174214-rock)
(..cut here..)
4) When using TWO video cards, SeaBIOS reports as follows:
$ sudo kvm -drive file=/dev/sdc -device pci-assign,host=02:00.0,addr=0x6 -device pci-assign,host=03:00.0,addr=0x7 -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios 2>&1
-------------------- Start bios (version 1.6.3-20111012_174214-rock) enabling shadow ram qemu_cfg_present=1 Find memory size Add to e820 map: 00000000 08000000 1 Add to e820 map: 00000000 00000000 1 Add to e820 map: fffc0000 00040000 2 Add to e820 map: 000a0000 00050000 -1 Add to e820 map: 000f0000 00010000 2 Add to e820 map: feffc000 00004000 2 Ram Size=0x08000000 (0x0000000000000000 high) malloc setup Add to e820 map: 07ff0000 00010000 2 pmm_malloc zone=0x000f0930 handle=ffffffff size=63248 align=10 ret=0x07fe07d0 (detail=0x07fefee0) Relocating init from 0x000e12d0 to 0x07fe07d0 (size 63248) malloc fixup reloc init ivt init bda Add to e820 map: 0009fc00 00000400 2 init pic init timer tsc calibrate start=2127316692 end=2133865272 diff=6548580 CPU Mhz=3815 math cp init init mtrr pci setup === PCI bus & bridge init === PCI: pci_bios_init_bus_rec bus = 0x0 === PCI device probing === PCI probe pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0700 (detail=0x07fe0770) PCI device 00:00.0 (vd=8086:1237 c=0600) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0660 (detail=0x07fe06d0) PCI device 00:01.0 (vd=8086:7000 c=0601) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe05c0 (detail=0x07fe0630) PCI device 00:01.1 (vd=8086:7010 c=0101) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0520 (detail=0x07fe0590) PCI device 00:01.3 (vd=8086:7113 c=0680) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0480 (detail=0x07fe04f0) PCI device 00:02.0 (vd=1013:00b8 c=0300) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe03e0 (detail=0x07fe0450) PCI device 00:03.0 (vd=10ec:8139 c=0200) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe0340 (detail=0x07fe03b0) PCI device 00:06.0 (vd=10de:01d3 c=0300) pmm_malloc zone=0x07fefe30 handle=ffffffff size=112 align=10 ret=0x07fe02a0 (detail=0x07fe0310) PCI device 00:07.0 (vd=10de:01d3 c=0300) Found 8 PCI devices (max PCI bus is 00) === PCI new allocation pass #1 === pmm_malloc zone=0x07fefe30 handle=ffffffff size=528 align=10 ret=0x07fe0060 (detail=0x07fe0270) PCI: check devices bus 0 PCI: out of address space