Re: [SeaBIOS] Qemu-kvm VM halt at allocate VGA stack when continuously reset VM

[Xulei (Stone)]

Thanks for Gamal's and Kevin's reply.
Here i'd like to provide more information:
- My VM is running under KVM (KMOD 3.6);
- qmp returns:
virsh # qemu-monitor-command hrg_sles11_sp3_64_2U_ide --pretty '{"execute":"query-kvm"}'
{
    "return": {
        "enabled": true,
        "present": true
    },
    "id": "libvirt-18099"
}
- kmod ftrace returns:

<...>-113789 [014] 858323.684996: kvm_emulate_insn: f0000:9:ff ff (real)
           <...>-113789 [014] 858323.684996: kvm_inj_exception: #UD (0x0)
           <...>-113789 [014] 858323.684996: kvm_entry: vcpu 0
           <...>-113789 [014] 858323.684997: kvm_exit: reason EXCEPTION_NMI rip 0x9 info 0 80000306

<...>-113789 [014] 858323.684997: kvm_emulate_insn: f0000:9:ff ff (real)
           <...>-113789 [014] 858323.684997: kvm_inj_exception: #UD (0x0)
           <...>-113789 [014] 858323.684998: kvm_entry: vcpu 0
           <...>-113789 [014] 858323.684998: kvm_exit: reason EXCEPTION_NMI rip 0x9 info 0 80000306

So, it seems at some points, guest executes an invalid opcode triggering kmod injects exception UD 
and seabios handle this exception through handle_hwpic1.

Any idea?
> Hi Xulei,
>
> i have a few questions about the problem you're experiencing, since I do see something similar:
> Is your VM running under KVM?
> If yes, does the KVM process show any register dumps or failure messages?
> What's the output of:
>
> $ virsh qemu-monitor-command devstack  --pretty '{"execute":"query-kvm"}'
> when seabios fails to load?
>
> Regards,
> Mohammed
>
> On Thu, Dec 17, 2015 at 6:28 AM, Xulei (Stone)  wrote:
> > On Wed, Dec 02, 2015 at 07:09:36AM +0000, Xulei (Stone) wrote:
> > > I move HaveRunPost = 1 to handle_post() (after make_bios_writable()), and
> > > I have tested for 1 day with continuously resetting, it seems works well!
> > > Does following patch have some side effects?
> >
> > Thanks.  I'll be traveling for the next two weeks.  I'll take a look
> > when I return.
>
> Hi, Kevin:
>    Welcome journey back! I keep on this problem for about 1 month. Here,
>    I provide some detail information and wish you could pay a little time
>    thinking about this.
>
> My test script is very easy:
>
> #!/bin/bash
>    vmname=$1
>    while [ 1 ]
>    do
>        virsh reset $vmname &
>        virsh reset $vmname &
>       sleep 5
>    done
> This problem seems have relations with PIC irq0 and host CPU feature:
>
> On one of my host, this problem never happens while on another, it's
>    very easy to happen (both of hosts have the same kmod,qemu,libvirt and
>    SeaBIOS, SeaVGABIOS, except for the CPU feature).
> SeaBIOS log tells me once VM halts at allocate VGA stack, it also has something
>
> wrong printing handle_smp log.
> If i disconfig CONFIG_VGA_ALLOCATE_EXTRA_STACK, the VM will halt at Grub stage,
>
> and SeaVGABIOS log stop at printing "set VGA mode 114", then BIOS will loop handle_pwhic1.
>
> For now, i guess whether if SeaBIOS can not handle the hardware interrupt correctly
>    when a host CPU has some advanced feature (x2apic? avx? xsave? tsc-deadline?).
>
> ==========bad SeaBIOS log=========
>    [2015-12-17 12:37:30] In 32bit resume
>    [2015-12-17 12:37:30] =====Attempting a hard reboot====
>    [2015-12-17 12:37:30] SeaBIOS (version rel-1.8.1-0-g4adadbd-20151217_104405-linux-emBwNn)
>    [2015-12-17 12:37:30] No Xen hypervisor found.
>    [2015-12-17 12:37:30] Running on QEMU (i440fx)
>    [2015-12-17 12:37:30] Running on KVM
>    [2015-12-17 12:37:30] RamSize: 0x80000000 [cmos]
>    [2015-12-17 12:37:30] Relocating init from 0x000db230 to 0x7ffad360 (size 76768)
>    [2015-12-17 12:37:30] Found QEMU fw_cfg
>    [2015-12-17 12:37:30] RamBlock: addr 0x0000000000000000 len 0x0000000080000000 [e820]
>    [2015-12-17 12:37:30] Moving pm_base to 0x600
>    [2015-12-17 12:37:30] boot order:
>    [2015-12-17 12:37:30] 1: /pci@i0cf8/ide@1,1/drive@0/disk@0
>    [2015-12-17 12:37:30] 2: HALT
>    [2015-12-17 12:37:30] maininit
>    [2015-12-17 12:37:30] platform_hardware_setup
>    [2015-12-17 12:37:30] init pic
>    [2015-12-17 12:37:30] pic_setup
>    [2015-12-17 12:37:30] pic_reset
>    [2015-12-17 12:37:30] enable_hwirq
>    [2015-12-17 12:37:30] CPU Mhz=3304
>    [2015-12-17 12:37:30] enable_hwirq
>    [2015-12-17 12:37:30] enable_hwirq
>    [2015-12-17 12:37:30] === PCI bus & bridge init ===
>    [2015-12-17 12:37:30] PCI: pci_bios_init_bus_rec bus = 0x0
>    [2015-12-17 12:37:30] === PCI device probing ===
>    [2015-12-17 12:37:30] Found 6 PCI devices (max PCI bus is 00)
>    [2015-12-17 12:37:30] === PCI new allocation pass #1 ===
>    [2015-12-17 12:37:30] PCI: check devices
>    [2015-12-17 12:37:30] === PCI new allocation pass #2 ===
>    [2015-12-17 12:37:30] PCI: IO: c000 - c02f
>    [2015-12-17 12:37:30] PCI: 32: 0000000080000000 - 00000000fec00000
>    [2015-12-17 12:37:30] PCI: map device bdf=00:01.2  bar 4, addr 0000c000, size 00000020 [io]
>    [2015-12-17 12:37:30] PCI: map device bdf=00:01.1  bar 4, addr 0000c020, size 00000010 [io]
>    [2015-12-17 12:37:30] PCI: map device bdf=00:02.0  bar 6, addr febe0000, size 00010000 [mem]
>    [2015-12-17 12:37:30] PCI: map device bdf=00:02.0  bar 1, addr febf0000, size 00001000 [mem]
>    [2015-12-17 12:37:30] PCI: map device bdf=00:02.0  bar 0, addr fc000000, size 02000000 [prefmem]
>    [2015-12-17 12:37:30] PCI: init bdf=00:00.0 id=8086:1237
>    [2015-12-17 12:37:30] PCI: init bdf=00:01.0 id=8086:7000
>    [2015-12-17 12:37:30] PIIX3/PIIX4 init: elcr=00 0c
>    [2015-12-17 12:37:30] PCI: init bdf=00:01.1 id=8086:7010
>    [2015-12-17 12:37:30] PCI: init bdf=00:01.2 id=8086:7020
>    [2015-12-17 12:37:30] PCI: init bdf=00:01.3 id=8086:7113
>    [2015-12-17 12:37:30] Using pmtimer, ioport 0x608
>    [2015-12-17 12:37:30] PCI: init bdf=00:02.0 id=1013:00b8
>    [2015-12-17 12:37:30] PCI: Using 00:02.0 for primary VGA
>    [2015-12-17 12:37:30] handle_hshamanpnd:dl leae_p_sismcmp_p:i: d a=ap3               <<======= here, seems abnormal!
>    [2015-12-17 12:37:30] ièf[cf_^ifd_=f3
>    [2015-12-17 12:37:30] èf[f^f_f]fÃÍ^XË<90>Found 4 cpu(s) max supported 4 cpu(s)
>    [2015-12-17 12:37:30] Copying PIR from 0x7ffbea18 to 0x000f5700
>    [2015-12-17 12:37:30] Copying MPTABLE from 0x00006e30/7ffa42c0 to 0x000f55e0
>    [2015-12-17 12:37:30] Copying SMBIOS entry point from 0x00006e11 to 0x000f55c0
>    [2015-12-17 12:37:31] Scan for VGA option rom
>    [2015-12-17 12:37:31] Running option rom at c000:0003
>    [2015-12-17 12:37:31] Start SeaVGABIOS (version rel-1.8.1-0-g4adadbd-20150316_085902-nilsson.home.kraxel.org)
>    [2015-12-17 12:37:31] enter vga_post:
>    [2015-12-17 12:37:31]    a=00000010  b=0000ffff  c=00000000  d=0000ffff ds=0000 es=f000 ss=0000
>    [2015-12-17 12:37:31]   si=00000000 di=000057e0 bp=00000000 sp=00006dbe cs=f000 ip=d1fb  f=0000
>    [2015-12-17 12:37:31] cirrus init
>    [2015-12-17 12:37:31] cirrus init 2
>    [2015-12-17 12:37:31] Attempting to allocate VGA stack via pmm call to f000:d2a0   <<====== here stuck, loop handle PIC irq0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    [2015-12-17 12:37:35] handle_hwpic1 irq=0
>    ... always hanle_hwpic1 irq=0, never ends anymore...
> > -Kevin
>
> SeaBIOS mailing list
>    [e-mail]
>    http://www.seabios.org/mailman/listinfo/seabios
>
> Mohammed Gamal
> Software Engineer
>
> ProfitBricks GmbH
> Greifswalder Straße 207 
> D - 10405 Berlin
>
> Tel: [phone]
> Email:       [e-mail]
>
> Web:         http://www.profitbricks.de
>
> Sitz der Gesellschaft: Berlin.
> Registergericht: Amtsgericht Charlottenburg, HRB 125506B.
> Geschäftsführer: Andreas Gauger, Achim Weiss.