As some of you may know, I've been working on a way to add verification directly into CBFS (as opposed to the current vboot that can only verify images as a whole) for a while (I have presented at OSFC 2019: https://www.youtube.com/watch?v=Hs_EhewBgtM and published a general design document: https://osfc.io/uploads/talk/paper/47/The_future_of_firmware_verification_in...). As part of this work I have been very careful to keep all these changes fully backwards-compatible so that existing payloads with their own CBFS implementations will continue to work.
I have now come across a fundamental problem that I cannot solve without introducing a small backwards-incompatible change: the CBFS "stage" format that is used to store the code for coreboot stages (e.g. romstage, ramstage) consists of a small "stage header" containing things like load address and size in front of the actual binary code. From the point of view of the generic CBFS structure, both that header and the code are part of the "file data" (and not the generic CBFS "file header" that contains things like the file name or generic CBFS attributes).
This creates a tough chicken-and-egg problem when verifying pre-RAM stages, because (for platforms that actually load pre-RAM stages and don't execute in-place, like Arm platforms running from SRAM) the stage needs to be directly loaded into the final location it should execute out of, but the load address containing that location is in the stage header. My verification design works on whole files, so the stage header cannot be trusted before the whole file was loaded and hashed. But I can't load it without information from the stage header, and there's generally not enough scratch space in these pre-RAM environments to first load it somewhere else and then copy it over to its final destination after verification.
My solution to this problem is to move the stage header out of the file data into a CBFS attribute that is stored in the generic CBFS file header. File headers are verified separately so they can already be trusted by the time the file data needs to be loaded. I think this is arguably a somewhat cleaner approach anyway (the concept of CBFS attributes just didn't exist yet when the stage format was originally designed, so it wasn't written that way to start with), but of course these new stages will be incompatible to the old ones. I believe this is okay because the "stage" format is generally only used for parts of coreboot itself (e.g. romstage, ramstage), whereas the payload and other binaries the payload may want to chain-load should be using the "simple ELF" (SELF) format anyway. Since coreboot is always built as a whole, when you're gonna build the new version all your stages will be in the new format and all code that loads them will know how to load the new format, so you should be fine. Tying this into a payload build system that is not aware of these changes should be okay because that payload shouldn't try to add extra files using the "stage" format, and shouldn't try to load any stages (if it wants to chain-load things it should be using the SELF format with cbfstool add-payload, not add-stage).
But this file format has remained untouched for the whole 10+ years CBFS existed, so I thought I should ask before I do it. If you know any payloads that do use the "stage" format for files outside of coreboot itself and that couldn't adapt to this change for some reason, or have any other concerns about unexpected problems this could be causing you, please let me know here or on the CL: https://review.coreboot.org/46484
Note that for cbfstool, this will also mean that newer versions of cbfstool will only support the new format. I think this should also be okay because coreboot is built as a whole, i.e. all your stages are added with a cbfstool version that was built from the same code base version as the coreboot code trying to load these stages. But it does mean that if you have an old version of cbfstool stashed away somewhere and try to use it to cbfstool print -v or cbfstool extract a new coreboot image, it won't be able to show load address or entry point for the new stages or convert them back into an ELF file on extraction (or vice versa for new versions of cbfstool on old images). So if you do a lot of after-the-fact cbfstool manipulation on different versions of coreboot images (which is probably only done by people trying to debug something?), you may need to keep two cbfstool versions to switch back-and-forth between for a while. It would be possible to make the new cbfstool version support both formats for reading/extracting but that's a bunch of extra work that I hope I can avoid unless people really feel like this is gonna be a widespread problem.
Also note that I'm not planning to touch the SELF (payload) format (which also has a header in the "file data" part) -- I will keep that as it is because I think it's much more likely that external payloads are using it for their own purposes that I don't want to break. It's also not necessary because SELFs are mostly just used in post-RAM environments and the SELF loader code already requires it (on non-memory-mapped platforms) to be loaded to a separate scratch buffer before copying the individual pieces to where they need to go. When we have that buffer anyway, it's easy to verify the whole file in there before examining the header.