[coreboot] Rebuilding coreboot image generation

[Julius Werner]

Okay, looks like we're continuing the discussion over here after all.

> Rationale: The build system is rather stupid about the order in which
> to add files, and while adding a proper scheduler that prevents
> unsuitable ordering (eg adding
> flexibly laid out files first, which may block files with stricter
> requirements later), people are rightfully complaining about too much
> magic in the build system already.
> (something to work on, but not as part of this effort)

I assume you're talking about things like crosreview.com/289491 as an
example for strict requirements. I'm not really familiar with the
details there so please correct me if I'm wrong, but if this is a blob
that needs to be placed in one exact offset (which it looks like?),
then I think we're approaching it from the completely wrong angle
trying to stuff it into CBFS anyway. Maybe it was originally
shoehorned into CBFS because that was the only thing we had at the
time, but if we're rearchitecting things from the ground up to make
FMAP a first class citizen this seems to be the right time to
reconsider that.

The whole point of FMAP is to allow exact placement in the ROM,
whereas the whole point of CBFS is to conveniently group files with
variable lengths and no strict placement requirements together in an
easily manageable way. I think the only placement restriction CBFS
should support in that new design is a (mild) minimum alignment, and
anything that needs to be placed more exactly than that (including the
bootblock and magic Intel blobs) should instead get its own FMAP
section.

>> HOST_FIRMWARE at 0xff800000 8M {
>>         SI_ALL 2M {
>>                 SI_DESC 4K
>>                 SI_ME 0x1ff000
>>         }
>>         SI_BIOS {
>>                 RW_A 0xf0000 {
>>                         VBLOCK_A 64K
>>                         RW_MAIN_A
>>                         RW_FWID_A 0x40
>>                 }
>>                 RW_B 0xf0000 {
>>                         VBLOCK_B 64K
>>                         RW_MAIN_B
>>                         RW_FWID_B 0x40
>>                 }
>>                 RO at 4M {
>>                         RO_VPD 16K
>>                         FMAP at 0x10000 2K
>>                         RO_FRID 0x40
>>                         GBB at 0x11000
>>                         RO_MAIN 1M
>>                 }
>>         }
>> }
> Just wanted to spell out the issues with this approach:
> First, it doesn't tell you where to put various build collateral. Is
> ramstage.elf to be added to SI_ME?

This was just an example of an existing (and I think oversimplified)
Chrome OS ROM that I pulled from an old discussion, to demonstrate the
format. We totally can (and should) redesign and rename some existing
parts for the new layouts, and we could also add new features to the
existing FMD parser if we deem them useful (e.g. some sort of
inclusion or macro mechanism to help deduplicate more). Stuff like
RO_MAIN and RW_MAIN_B are probably better named RO_CBFS and RW_CBFS_B,
and I think the bootblock should be split out into its own section
(since it is not a proper CBFS file, and it has strict placement
requirements).

In my current vision, you would then group all of coreboot's output
files (both stages and separate files) into "pre-boot-path-split" and
"post-boot-path-split" groups. I think the existing Makefile system
would be fine for that, but we could discuss that part separately.
Then, whatever controls the Chrome OS specific parts of the build
process (I think a Makefile would be enough, but again, that's a
separate issue from the ROM layout itself) will 'cbfstool add -r
RO_CBFS' the pre-boot-path-split files, 'cbfstool add -r
RO_CBFS,RW_CBFS_A,RW_CBFS_B' the post-boot-path-split files, and
finally 'cbfstool add -r RW_CBFS_A,RW_CBFS_B' the EC-RW images (which
are Chrome OS specific and should therefore probably only come into
play at that time).

The advantage is that the layout (which may need to be very flexible
due to all the weird corner cases that we keep encountering, and may
be duplicated a little bit in practice) only tells you *where* the
well-known sections are in a single, very easily readable and
modifiable place that can be parsed at a glance, while another (more
generic) part of the system can decide *what* to put in those
sections. I think this would be much easier to use in practice than a
solution where the mere decision which section is placed where gets
spread out through unordered lines in X different files that all
somehow work together to cook up the final layout in a process hardly
anyone will really understand (or be able to adjust).

> Second, this requires more or less one such description file per board
> and boot method configuration, which is even worse than Alexandru's
> concern about having a reenactment of horribly maintained cmos.layout
> files (they're "merely" per board, but otherwise stable).

I think it probably requires less than two layouts per SoC in practice
(one for Chrome OS, one for normal coreboot). In fact most of our
current ARM boards could probably share a "default" ARM layout, and
for x86 there might be two or three (based on differences like that
microcode placement). We could move common differences (like bootblock
size) out into a separate Kconfig and throw the .fmd files through CPP
to allow even more reuse.

Yes, we may still copy a certain pattern in two or three places in the
end, and I think that's perfectly fine. These are small, simple to
understand files... we have much worse duplication issues in other
parts already. The huge upside is that any board *can* easily override
the whole thing completely if it has to, just by copying and adjusting
the default file into its board directory. It doesn't need to
understand which manifests play into which section of the image and
how each and every one of them can be overridden (if that's possible
at all) to tweak the layout around a certain hardware requirement. It
also makes it much easier for an external project that has some
special requirements (imagine a new Chrome OS, but not yet integrated
into upstream) to easily inject their own layouts to test something
new on top of coreboot.

Yes, in the end we certainly don't want every board to have their own
layout file... but that should be rare in practice, and if we see a
certain pattern of common deviations emerge we can find a way to
re-unify them after the fact (as it often happens with code as well).
I think that's better than forcing everyone into a complex and rigid
framework spread out over three parts of the codebase right from the
start, because you can never cover all edge cases (especially future
ones we don't know yet) in one framework, and because people can only
contribute new things if the system is simple enough for everyone to
understand and expand.