Okay, looks like we're continuing the discussion over here after all.
Rationale: The build system is rather stupid about the order in which to add files, and while adding a proper scheduler that prevents unsuitable ordering (eg adding flexibly laid out files first, which may block files with stricter requirements later), people are rightfully complaining about too much magic in the build system already. (something to work on, but not as part of this effort)
I assume you're talking about things like crosreview.com/289491 as an example for strict requirements. I'm not really familiar with the details there so please correct me if I'm wrong, but if this is a blob that needs to be placed in one exact offset (which it looks like?), then I think we're approaching it from the completely wrong angle trying to stuff it into CBFS anyway. Maybe it was originally shoehorned into CBFS because that was the only thing we had at the time, but if we're rearchitecting things from the ground up to make FMAP a first class citizen this seems to be the right time to reconsider that.
The whole point of FMAP is to allow exact placement in the ROM, whereas the whole point of CBFS is to conveniently group files with variable lengths and no strict placement requirements together in an easily manageable way. I think the only placement restriction CBFS should support in that new design is a (mild) minimum alignment, and anything that needs to be placed more exactly than that (including the bootblock and magic Intel blobs) should instead get its own FMAP section.
HOST_FIRMWARE@0xff800000 8M { SI_ALL 2M { SI_DESC 4K SI_ME 0x1ff000 } SI_BIOS { RW_A 0xf0000 { VBLOCK_A 64K RW_MAIN_A RW_FWID_A 0x40 } RW_B 0xf0000 { VBLOCK_B 64K RW_MAIN_B RW_FWID_B 0x40 } RO@4M { RO_VPD 16K FMAP@0x10000 2K RO_FRID 0x40 GBB@0x11000 RO_MAIN 1M } } }
Just wanted to spell out the issues with this approach: First, it doesn't tell you where to put various build collateral. Is ramstage.elf to be added to SI_ME?
This was just an example of an existing (and I think oversimplified) Chrome OS ROM that I pulled from an old discussion, to demonstrate the format. We totally can (and should) redesign and rename some existing parts for the new layouts, and we could also add new features to the existing FMD parser if we deem them useful (e.g. some sort of inclusion or macro mechanism to help deduplicate more). Stuff like RO_MAIN and RW_MAIN_B are probably better named RO_CBFS and RW_CBFS_B, and I think the bootblock should be split out into its own section (since it is not a proper CBFS file, and it has strict placement requirements).
In my current vision, you would then group all of coreboot's output files (both stages and separate files) into "pre-boot-path-split" and "post-boot-path-split" groups. I think the existing Makefile system would be fine for that, but we could discuss that part separately. Then, whatever controls the Chrome OS specific parts of the build process (I think a Makefile would be enough, but again, that's a separate issue from the ROM layout itself) will 'cbfstool add -r RO_CBFS' the pre-boot-path-split files, 'cbfstool add -r RO_CBFS,RW_CBFS_A,RW_CBFS_B' the post-boot-path-split files, and finally 'cbfstool add -r RW_CBFS_A,RW_CBFS_B' the EC-RW images (which are Chrome OS specific and should therefore probably only come into play at that time).
The advantage is that the layout (which may need to be very flexible due to all the weird corner cases that we keep encountering, and may be duplicated a little bit in practice) only tells you *where* the well-known sections are in a single, very easily readable and modifiable place that can be parsed at a glance, while another (more generic) part of the system can decide *what* to put in those sections. I think this would be much easier to use in practice than a solution where the mere decision which section is placed where gets spread out through unordered lines in X different files that all somehow work together to cook up the final layout in a process hardly anyone will really understand (or be able to adjust).
Second, this requires more or less one such description file per board and boot method configuration, which is even worse than Alexandru's concern about having a reenactment of horribly maintained cmos.layout files (they're "merely" per board, but otherwise stable).
I think it probably requires less than two layouts per SoC in practice (one for Chrome OS, one for normal coreboot). In fact most of our current ARM boards could probably share a "default" ARM layout, and for x86 there might be two or three (based on differences like that microcode placement). We could move common differences (like bootblock size) out into a separate Kconfig and throw the .fmd files through CPP to allow even more reuse.
Yes, we may still copy a certain pattern in two or three places in the end, and I think that's perfectly fine. These are small, simple to understand files... we have much worse duplication issues in other parts already. The huge upside is that any board *can* easily override the whole thing completely if it has to, just by copying and adjusting the default file into its board directory. It doesn't need to understand which manifests play into which section of the image and how each and every one of them can be overridden (if that's possible at all) to tweak the layout around a certain hardware requirement. It also makes it much easier for an external project that has some special requirements (imagine a new Chrome OS, but not yet integrated into upstream) to easily inject their own layouts to test something new on top of coreboot.
Yes, in the end we certainly don't want every board to have their own layout file... but that should be rare in practice, and if we see a certain pattern of common deviations emerge we can find a way to re-unify them after the fact (as it often happens with code as well). I think that's better than forcing everyone into a complex and rigid framework spread out over three parts of the codebase right from the start, because you can never cover all edge cases (especially future ones we don't know yet) in one framework, and because people can only contribute new things if the system is simple enough for everyone to understand and expand.