[coreboot] Rebuilding coreboot image generation

[Patrick Georgi]

2015-11-06 3:57 GMT+01:00 Alex G. <mr.nuke.me at gmail.com>:
> My main concern with this is that we're introducing another language,
> with something that may just as well end up being cmos.layout 2.0.
>
> I see where the manifest language is coming from. It's sort of like a
> FMAP dts, but then it really isn't.  I understand that we don't want to
> use a per-board fmap.dts in order to prevent the duplication we ended up
> having last time we did this (cmos.layout).
>
> From reading the manifest examples, these are the few things that jumped
> out at as being unwanted:
> (1) mixing region descriptors with rules. As soon as we agree to move
> some rules to manifests and some to makefiles, we risk ending up with a
> mess that's inconsistent between platforms, and hard to track down/debug.
All rules end up in manifests. The build system wouldn't add a file to
a cbfs _ever_, cbfs-files-y would create the "coreboot" manifest, to
be consumed by the later tool.

Rationale: The build system is rather stupid about the order in which
to add files, and while adding a proper scheduler that prevents
unsuitable ordering (eg adding
flexibly laid out files first, which may block files with stricter
requirements later), people are rightfully complaining about too much
magic in the build system already.
(something to work on, but not as part of this effort)

> (2) Undermining some of the existing infrastructure. We can extend the
> concept of cbfs-files-y, to fmap-regions-y, whereas the FMAP regions are
> dynamically declared. Of course, this is in stark contrast to the
> per-module descriptors (which I also like, assuming they are pure
> fmap.dts format). However, this inconsistency worries me.
I deliberately didn't go for using dts, because that's a gross hack:
When trying to define files to add to CBFS in dts (which I do as a
temporary measure on the CrOS build system side), I now use dts
properties (key/value pairs) with arbitrary unique keys. Bleh.

And as stated above, gnu make is the wrong tool for the job, even
though it _would_ be possible to force it to do the right thing with
lots of effort. Readability will suffer, as will execution speed, and
in the end, like with certain other parts of the build system the only
one touching that stuff without making a mess would be me.

_That_ is something I don't want to see a repeat of, no matter of nice
it is to see that I can coerce gnu make to do things it wasn't
designed for on an ego-stroking level.

> (3) It changes everything we've learnt so far about building a firmware
> image. Some platforms will end up using the new manifests, others will
> use the existing build infrastructure. I'm worried we'll end up with an
> inconsistent build system. Who is going to convert all the old hardware
> to manifests?
I'll take care of upstream and CrOS. If only because I want to cut
back on the mess we have in the tree now.
Most coreboot images right now are pretty simple to convert:
They're either CBFS_SIZE == ROM_SIZE, an IFD layout, or one or two
boards with some special handling where neither of the other two hold
true.

That I'd not have to build an fmap-regions-y for each and every of the
boards individually is a plus.

> Now to the good parts. I mentioned briefly the possibility of a
> fmap-regions-y macro. Quite honestly, I much better like seeing the
> fmap.dts in one place in order to get an idea of the flash layout.
Consider we're going for an fmap description like that, and use that
and cbfs-files-y for the layout.
We still need a way to allocate files going into the various CBFS
regions (where current configurations have one or three, and there's a
pretty obvious two-CBFS configuration out there). That's another
variable to take care of, and there's nothing around really that can
make obvious any inconsistencies between those.

The proposal will, in effect, have three manifests on the coreboot
side, plus payload and additional projects that may contribute files
(eg Chrome EC in the Chrome OS use case). Ignoring the external
manifests for a moment, that's
- the chipset/board specific descriptor for flash layouts, chosen at
build time (and there will be a bunch of common ones, with the option
to override for special cases);
- the boot method manifest. Chosen by the same family of Kconfig
options that choose between simple or fallback/normal bootblocks.
These two are static and reside somewhere in the repo.
The third one is (what I called) the coreboot manifest, which lists
the files. That one is dependent on the build system, and is pretty
much a serialization of cbfs-files-y (just that cbfs-files-y now
includes all files, instead of some files in cbfs-files-y and some
manually added through $(CBFSTOOL) add* invocations in
src/arch/*/Makefile.inc).

That's the coreboot "build" side. Other projects can contribute their
own manifests for the final "link" step.

With all that stuff around, the final step is about reading the files
and generating the image.
Right now, we don't have such a clear description in our build system
of what will happen to the final image.

> What I'd like to see come out of this is a dts-like set of flash
> descriptors, with the rules left in the makefiles. At the very least,
> dts is not a new language.
It's also pretty unusable for describing cbfs files.

> Do you think we can do that without making a mess out of it?
I tried to use the opportunity to cut down on the mess we're already making.


Patrick
-- 
Google Germany GmbH, ABC-Str. 19, 20354 Hamburg
Registergericht und -nummer: Hamburg, HRB 86891, Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle