[coreboot] Multi domain PCI resource allocation: How to deal with multiple root busses on one domain

[Arthur Heymans]

Hi

I've recently tried to improve the soc/intel/xeon_sp codebase.
I want to make it use more native coreboot structures and codeflows instead
of parsing the FSP HOB again and again to do things. Ideally the HOB is
parsed only once in ramstage, parsed into adequate native coreboot
structures (struct device, struct bus, chip_info, ...) and used later on.

The lowest hanging fruit in that effort is resource allocation.
Currently the coreboot allocator is sort of hijacked by the soc code and
done over again.
The reason for this is that xeon_sp platforms operate a bit differently
than most AMD and Intel Client hardware: there are multiple root busses.
This means that there are PCI busses that are in use, but are not
downstream from PCI bus 0. In hardware terminology those are the IIO and
other type of Stacks.

Each Stack has its own range of usable PCI Bus numbers and decoded IO and
MEM spaces below and above 4G. I tried to map these hardware concepts to
the existing coreboot 'domain' structure. Each domain has resource windows
that are used to allocate children devices on, which would be the PCI
devices on the stacks.
The allocator needs some tweaks to allow for multiple resources of a type
(MEM or IO), but nothing major. See
https://review.coreboot.org/c/coreboot/+/62353/ and
https://review.coreboot.org/c/coreboot/+/62865 (allocator
rewrite/improvement based on Nico's excellent unmerged v4.5 work) This
seems to work really well and arguably even better than how it is now with
more elegant handling of above and below 4G resources.

Now my question is the following:
On some Stacks there are multiple root busses, but the resources need to be
allocated on the same window. My initial idea was to add those root busses
as separate struct bus in the domain->link_list. However currently the
allocator assumes only one bus on domains (and bridges).
In the code you'll see a lot of things like

for (child = domain->link_list->children; child; child = child->sibling)
      ....

This is fine if there is only one bus on the domain.
Looping over link_list->next, struct bus'ses is certainly an option here,
but I was told that having only one bus here was a design decision on the
allocator v4 rewrite. I'm not sure how common that assumption is in the
tree, so things could be broken in awkward ways.

Do you have any suggestions to move forward?

Kind regards

Arthur Heymans