Patch Set 9:
coreboot's current resource allocation does not work at all for modern
hardware. This is not just advanced server platforms, but also mobile platforms
will be affected soon.In https://review.coreboot.org/c/coreboot/+/35946, I describe the need to
pre-allocate resources for hotpluggable PCIe bridges. On some server systems,
there are 8 U.2 ports, each supporting hot pluggable drives, and with the usage
of adapter cards, potentially hot pluggable devices of any kind. A conservative
allocation for each port would be around 256 MiB of prefetch memory. If this is
done in 32-bit space, we have on such a system, 2 GiB of prefetch memory
allocated.This problem is not unique to server platforms, with lots of PCIe buses. Intel
introduced integrated Thunderbolt with Ice Lake, and will be providing it
across all mobile platforms with Tiger Lake. The common U-class platform will
have up to four Thunderbolt ports, each one providing daisy chained PCIe
support. USB 4.0 will standardize such a setup, so it will be something that
will be encountered on most new hardware soon, not just these new Intel mobile
platforms.coreboot's allocator could be worked on to provide 64-bit allocation of these
resources, which would be one way of addressing the problem. However, the
difficulty of doing so compared to the benefit is questionable. Right now, the
Linux and Tianocore payloads are fully capable of doing their own PCIe
resource allocation. However, with devices being initialized by coreboot, there
is no easy way to allow the devices to be re-initialized by the payload.
Could you please elaborate on your assessment of difficulty of adding 64-bit resource allocation? I don't think it's that hard in practice, but there are some current gotchas:
coreboot on x86 arch doesn't utilize x86-64 isa by default. If we didn't target the devices that we want to hoist to 64-bit resource allocation we could run into an issue that coreboot drivers wouldn't work or even the payloads. I think your summary is fairly accurate, but it's also assuming 64-bit being pervasive. That's certainly a great desire, but I think there are use-cases where wanting 64-bit allocation may not be warranted. In short, we just need a way to turn it off and/or steer devices to fall within the 32-bit or 64-bit address space.
For any resource intensive setup, yes, the IO hole is problematic. Instead of performing multiple reboots to settle on a layout we've typically split 32-bit address space in 2GiB/2GiB for RAM vs mmio. And, yes, that is constraining if you are assuming people need and demand 256MiB of mmio space for large number of devices (and assume the OS can't correctly reallocate).
What this patch does is provide a quick and easy to understand way for an
advanced platform to do PCIe resource allocation in the payload. No massive
rewrite of the coreboot allocator has to take place. The payloads that do not
support this can continue using the simple allocator coreboot has. coreboot
pushes advanced initialization to the payload where it probably should be.
For my own curiosity, what's your definition of advanced? Doing full USB4/thunderbolt enumeration in firmware?
At System76, we are shipping products with Thunderbolt using coreboot next
week. We are doing this by using a patched version of coreboot with PCIe
hotplug resource allocation support. We are lucky to only have one Thunderbolt
port, so we can continue using the 32-bit allocator. We will not be so lucky
with our coming products. We are also working with Intel on MinPlatform, which
provides an alternative to boot initialization with coreboot. The ability to
do PCIe resource allocation in the payload will soon become a deciding factor
in our ability to continue using and contributing to coreboot.
I think we'll be working on/looking for solutions to these problems shortly on our end because we are pursuing usb4 on our products. I don't think the desire is controversial. While acknowledging we can improve the resource allocator, our current plans are to assume we *only* want USB4 enumeration and topology in the OS once it's booted, and I'm taking the assumption that Linux will be able to accomplish the enumeration of devices and resource assignment within its environment.
FWIW, if one believes the payload/OS is capable of enumerating and assigning resources, one can also accomplish this fairly easily w/o even touching device tree. Make a pci driver in coreboot that has an empty read_resources() callback that matches on the pci dids. The allocation is skipped and it's straight forward.
To view, visit change 36221. To unsubscribe, or for help writing mail filters, visit settings.