On 08/02/2017 01:58 PM, Marcel Apfelbaum wrote:
On 02/08/2017 19:26, Michael S. Tsirkin wrote:
On Wed, Aug 02, 2017 at 06:36:29PM +0300, Marcel Apfelbaum wrote:
>> Can dmi-pci support shpc? why doesn't it? For compatibility? > > I don't know why, but the fact that it doesn't is the reason libvirt > settled on auto-creating a dmi-pci bridge and a pci-pci bridge under > that for Q35. The reasoning was (IIRC Laine's words correctly) > that the > dmi-pci bridge cannot receive hotplugged devices, while the pci-pci > bridge cannot be connected to the root complex. So both were needed.
At least that's what I was told :-) (seriously, 50% of the convoluted rules encoded into libvirt's PCI bus topology construction and connection rules come from trial and error, and the other 50% come from advice and recommendations from others who (unlike me) actually know something about PCI.)
Of course the whole setup of plugging a pci-bridge into a dmi-to-pci-bridge was (at the time at least) an exercise in futility, since hotplug didn't work properly on pci-bridge+Q35 anyway (that initially wasn't explained to me; it was only after I had constructed the odd bus topology and it was in released code that someone told me "Oh, by the way, hotplug to pci-bridge doesn't work on Q35". At first it was described as a bug, then later reclassified as a future feature.)
(I guess the upside is that all of the horrible complex/confusing code needed to automatically add two controllers just to plug in a single endpoint is now already in the code, and will "just work" if/when needed).
Now that I go back to look at this thread (qemu-devel is just too much for me to try and read unless something has been Cc'ed to me - I really don't know how you guys manage it!), I see that pcie-pci-bridge has been implemented, and we (libvirt) will want to use that instead of dmi-to-pci-bridge when available. And pcie-pci-bridge itself can have endpoints hotplugged into it, correct? This means there will need to be patches for libvirt that check for the presence of pcie-pci-bridge, and if it's found they will replace any auto-added dmi-to-pci-bridge+pci-bridge with a long pcie-pci-bridge.
> > Thanks > Laszlo
OK. Is it true that dmi-pci + pci-pci under it will allow hotplug on Q35 if we just flip the bit in _OSC?
Marcel, what say you?... :)
Good news, works with: -device i82801b11-bridge,id=b1 -device pci-bridge,id=b2,bus=b1,chassis_nr=1,msi=off
And presumably it works for modern windows? OK, so it looks like patch 1 is merely a bugfix, I'll merge it for 2.10.
Tested with Win10, I think is OK to merge if for 2.10.
Notice bridge's msi=off until the following kernel bug will be merged: https://www.spinics.net/lists/linux-pci/msg63052.html
Does libvirt support msi=off as a work-around?
We have no explicit setting for msi on pci controllers. The only place we explicitly set that is on the ivshmem device.
That doesn't mean that we couldn't add it. However, if we were going to do it manually, that would mean adding another knob that we have to support forever. And even if we wanted to do it automatically, we would not only need to find something in qemu to key off of when deciding whether or not to set it, but we would *still* have to explicitly store the setting in the config so that migrations between hosts using differing versions of qemu would preserve guest ABI. Are there really enough people demanding (with actual concrete plans of *using*) hotplug of legacy PCI devices on Q35 guests *immediately* that we want to permanently pollute libvirt's code in this manner just for an interim workaround?
I didn't have enough time/energy to fully parse all the rest of this thread - is msi=off currently required for pcie-pci-bridge hotplug as well? (not that it changes my opinion - just as we can tell people "upgrade to a new qemu and libvirt if you want to hotplug legacy PCI devices on Q35 guests", we can also tell them "Oh, and wait X weeks and upgrade to a new kernel too".