<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style>body { line-height: 1.5; }body { font-size: 10.5pt; font-family: ????; color: rgb(0, 0, 0); line-height: 1.5; }</style>
</head>
<body>
<div><span></span>Hello all,</div>
<div>Recently, I met a werid question when i run a VM in the following platfrom:</div>
<div><br>
</div>
<div>Vmware Vsphere 6.0/6.5</div>
<div> |-- centos 7.3 nested VM (with qemu 2.8, kmod 4.4.11, seabios 1.10)</div>
<div> |-- VM (with virtio-scsi controller, modern mode)</div>
<div><br>
</div>
<div><span style="font-size: 10.5pt; line-height: 1.5; background-color: window;">VM
<b>MUST</b> hang in seabios when try to mmio write during virtio-scsi init.</span></div>
<div>I deeply analyze the code, and i found that:</div>
<div><br>
</div>
<div>Seabios calltrace: </div>
<div style="font-size: 15px;">scsi_drive_setup</div>
<div style="font-size: 15px;"> |-- cdb_get_inquiry</div>
<div style="font-size: 15px;"> |-- virtio_scsi_process_op</div>
<div style="font-size: 15px;"> |-- vring_kick</div>
<div style="font-size: 15px;"> |-- writew (hang here, mmio write, never return...)</div>
<div style="font-size: 15px;"><br>
</div>
<div style="font-size: 15px;">Kmod calltrace:</div>
<div style="font-size: 15px;">handle_EPT_MISCONFIG</div>
<div style="font-size: 15px;"> |--kvm_io_bus_write</div>
<div style="font-size: 15px;"> |-- kvm_iodevice_write (fast mmio through ioeventfd to trigger qemu)</div>
<div style="font-size: 15px;"> |-- vcpu_run</div>
<div style="font-size: 15px;"> |-- vcpu_enter_guest</div>
<div style="font-size: 15px;"><br>
</div>
<div style="font-size: 15px;">Qemu calltrace:</div>
<div style="font-size: 15px;">os_host_main_loop_wait</div>
<div style="font-size: 15px;"> |--glib_pollfds_poll</div>
<div style="font-size: 15px;"> |-- aio_dispatch</div>
<div style="font-size: 15px;"> |-- vritio_scsi_handle_cmd_vq</div>
<div style="font-size: 15px;"> |--virtio_scsi_pop_req</div>
<div style="font-size: 15px;"> |-- virtqueue_pop</div>
<div style="font-size: 15px;"> |--virtio_queue_empty </div>
<div style="font-size: 15px;"><br>
</div>
<div style="font-size: 15px;">Then, kmod falls in infinite loop in handle EPT_MISCONFIG.</div>
<div style="font-size: 15px;">As far as i know, when kvm enters guest after handling EPT_MISCONFIG, seabios should return</div>
<div style="font-size: 15px;">from mmio write and wait for virtio backend(qemu) to handle this mmio writing.</div>
<div style="font-size: 15px;">What puzzles me:</div>
<div style="font-size: 15px;">1) i can not understand why kvm runs in infinite loop and seabios does not return from writew.</div>
<div style="font-size: 15px;">2) kvm nested kvm is ok. But vmware nested kvm is <b>
not</b> ok. This problem has anything related</div>
<div style="font-size: 15px;"> with vmware?</div>
<div style="font-size: 15px;"><br>
</div>
<div style="font-size: 15px;">Looking forward for directions...</div>
<div style="font-size: 15px;"><span style="line-height: 1.5; background-color: window;"><br>
</span></div>
<div style="font-size: 15px;"><span style="line-height: 1.5; background-color: window;">Attached Kmod ftrace:</span></div>
<span style="font-size: 15px; color: rgb(0, 0, 0); background-color: rgba(0, 0, 0, 0);"> CPU 0/KVM-9435 [008] .... 241968.479572: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479573: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479573: kvm_entry: vcpu 0<br>
CPU 0/KVM-9435 [008] .... 241968.479583: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479589: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479589: kvm_entry: vcpu 0<br>
CPU 0/KVM-9435 [008] .... 241968.479599: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479600: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479601: kvm_entry: vcpu 0<br>
CPU 0/KVM-9435 [008] .... 241968.479610: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479616: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479617: kvm_entry: vcpu 0<br>
CPU 0/KVM-9435 [008] .... 241968.479627: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479628: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479628: kvm_entry: vcpu 0<br>
CPU 0/KVM-9435 [008] .... 241968.479638: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479644: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479644: kvm_entry: vcpu 0<br>
CPU 0/KVM-9435 [008] .... 241968.479655: kvm_exit: reason EPT_MISCONFIG rip 0xee696 info 0 0<br>
CPU 0/KVM-9435 [008] .... 241968.479656: kvm_fast_mmio: fast mmio at gpa 0xfe003008<br>
CPU 0/KVM-9435 [008] d... 241968.479656: kvm_entry: vcpu 0<br>
</span>
<div style="font-size: 15px;"> </div>
<hr style="width: 210px; height: 1px; display: none;" color="#b5c4df" size="1" align="left">
<div><span></span></div>
</body>
</html>