Aaron Durbin has posted comments on this change. ( https://review.coreboot.org/c/coreboot/+/36221 )
Change subject: Add configurable ramstage support for minimal PCI scanning ......................................................................
Patch Set 9:
Patch Set 9:
coreboot's current resource allocation does not work at all for modern hardware. This is not just advanced server platforms, but also mobile platforms will be affected soon.
In https://review.coreboot.org/c/coreboot/+/35946, I describe the need to pre-allocate resources for hotpluggable PCIe bridges. On some server systems, there are 8 U.2 ports, each supporting hot pluggable drives, and with the usage of adapter cards, potentially hot pluggable devices of any kind. A conservative allocation for each port would be around 256 MiB of prefetch memory. If this is done in 32-bit space, we have on such a system, 2 GiB of prefetch memory allocated.
This problem is not unique to server platforms, with lots of PCIe buses. Intel introduced integrated Thunderbolt with Ice Lake, and will be providing it across all mobile platforms with Tiger Lake. The common U-class platform will have up to four Thunderbolt ports, each one providing daisy chained PCIe support. USB 4.0 will standardize such a setup, so it will be something that will be encountered on most new hardware soon, not just these new Intel mobile platforms.
coreboot's allocator could be worked on to provide 64-bit allocation of these resources, which would be one way of addressing the problem. However, the difficulty of doing so compared to the benefit is questionable. Right now, the Linux and Tianocore payloads are fully capable of doing their own PCIe resource allocation. However, with devices being initialized by coreboot, there is no easy way to allow the devices to be re-initialized by the payload.
Could you please elaborate on your assessment of difficulty of adding 64-bit resource allocation? I don't think it's that hard in practice, but there are some current gotchas:
coreboot on x86 arch doesn't utilize x86-64 isa by default. If we didn't target the devices that we want to hoist to 64-bit resource allocation we could run into an issue that coreboot drivers wouldn't work or even the payloads. I think your summary is fairly accurate, but it's also assuming 64-bit being pervasive. That's certainly a great desire, but I think there are use-cases where wanting 64-bit allocation may not be warranted. In short, we just need a way to turn it off and/or steer devices to fall within the 32-bit or 64-bit address space.
For any resource intensive setup, yes, the IO hole is problematic. Instead of performing multiple reboots to settle on a layout we've typically split 32-bit address space in 2GiB/2GiB for RAM vs mmio. And, yes, that is constraining if you are assuming people need and demand 256MiB of mmio space for large number of devices (and assume the OS can't correctly reallocate).
What this patch does is provide a quick and easy to understand way for an advanced platform to do PCIe resource allocation in the payload. No massive rewrite of the coreboot allocator has to take place. The payloads that do not support this can continue using the simple allocator coreboot has. coreboot pushes advanced initialization to the payload where it probably should be.
For my own curiosity, what's your definition of advanced? Doing full USB4/thunderbolt enumeration in firmware?
At System76, we are shipping products with Thunderbolt using coreboot next week. We are doing this by using a patched version of coreboot with PCIe hotplug resource allocation support. We are lucky to only have one Thunderbolt port, so we can continue using the 32-bit allocator. We will not be so lucky with our coming products. We are also working with Intel on MinPlatform, which provides an alternative to boot initialization with coreboot. The ability to do PCIe resource allocation in the payload will soon become a deciding factor in our ability to continue using and contributing to coreboot.
I think we'll be working on/looking for solutions to these problems shortly on our end because we are pursuing usb4 on our products. I don't think the desire is controversial. While acknowledging we can improve the resource allocator, our current plans are to assume we *only* want USB4 enumeration and topology in the OS once it's booted, and I'm taking the assumption that Linux will be able to accomplish the enumeration of devices and resource assignment within its environment.
FWIW, if one believes the payload/OS is capable of enumerating and assigning resources, one can also accomplish this fairly easily w/o even touching device tree. Make a pci driver in coreboot that has an empty read_resources() callback that matches on the pci dids. The allocation is skipped and it's straight forward.