On 5/6/21 2:43 PM, Patrick Georgi wrote:
Am Do., 6. Mai 2021 um 14:03 Uhr schrieb Piotr Król <piotr.krol@3mdeb.com mailto:piotr.krol@3mdeb.com>:
If 3mdeb maintains some boards, we already testing those and would be glad to hook, in secure way, to patch testing system, but I would like to know where is interface documentation so I can evaluate cost of integration and convince customers to go that path. This was expressed many times in various communication channels (conferences, slack).We're at the ~5th or so public test infrastructure project by now and it's still not nailed down. Part of it is that it's simply a hard problem.
Wow, I'm surprised this is 5th time.
I agree problem is hard, but not impossible to solve. During my corporate career I was involved couple times in building validation infrastructure for storage controllers validation (including UEFI apps and Option ROMs) as well as UEFI/PI implementations for Intel servers. With past experience we built 3mdeb validation infrastructure, which we will move forward.
Another problem is that whoever pulls this off needs to be in the very narrow intersection of having time (i.e. not a product driven coreboot developer) and having money and a few other resources (i.e. not a hobbyist), so they can run enough of the infrastructure by themselves that others who could hook into the infrastructure see the benefit.
I'm not sure to which group 3mdeb fall, but I don't understand argument about running significant ("enough") of the infrastructure. Why maintainers of platforms do not run their part of infrastructure which support those platforms?
I think it can be expensive to go all-in in that direction, but if we could go in that direction it would be great.If you want to maintain any particular release as a long term branch, announce your intent and we'll set up a branch!
I'm not sure what benefit it would give to community or to us, but we have to maintain v4.0.x probably until PC Engines hardware EOL.
Question is why coreboot change so dramatically in all mentioned areas? Does projects with similar lifetime also change in so significant way?One reason is that we're dealing with the guts of an industry that is changing around us _very_ quickly.
I think other projects like hypervisors, Linux or highly hardware related one feel the same.
But I disagree that we're changing dramatically: on the contrary, we're pretty careful about remaining compatible in various important ways for long stretches of time to help everybody move at their own pace.
Some examples off the top of my head:
- We used to compile out strings for log levels we didn't print for
space reasons. Space is now no concern and there's the cbmem console, so we leave everything in for better debugging. The remnants of the "compile out" approach, gone for ~10 years, have only been removed within the last two months.
- We used to have CBFS with a master header that defined "off limit"
regions at the start and end of flash. That's fine as long as you don't regularly write to flash (where you risk blasting away parts of the CBFS structure), but these days we do write to flash, sadly, so there's now a partitioning scheme (fmap), making the master header obsolete. The header is still around, SeaBIOS still can't read FMAP.
- We added per-file metadata to CBFS in a compatible way even though the
structure is a bit more complicated than it could have been if we hadn't cared about compatibility.
- The "read/write registers" code had to change because C compilers like
to tighten up their rules around aliasing and volatile types and stuff like that. So we rewrite our macros into functions, with proper types, just so that a newer compiler doesn't break our entire code base.
- All that vboot/mboot/bootguard security stuff just was not a thing
when coreboot started. It brought in tons of complexity: more flash partitioning, more boot stages, just "tons more code", more memory management (for example, we now have some funky "free last two malloced objects" free() implementation. We got by without free() for 15 years)
- Thunderbolt (and USB4) have some pretty arcane requirements on
configuration buses. Originally LinuxBoot was supposed to set up only the bare minimum to jump into a kernel. With TBT/USB4 you can forget about that.
- More and more external complexity brought in: IOMMUs seem to add a new
data structure with every chip generation, ACPI is getting ever more complex (and we can't opt out of that madness or OSes won't boot), ...
Ok, agree using drama here is exaggeration, but this is just reflection on feeling we get when we suddenly get 20 emails coming yelling at us that platform would be removed from tree and we should go into panic mode to prevent that. Or on the other side complains that upstream behaves different then our releases.
I know that changes coming to tree have important reason and I'm not arguing there should be no tree-wide changes. What I really try is to highlight various problems 3mdeb see over 6 years of coreboot development.
So everything changes around us, sometimes in unexpected ways: compilers, interfaces, hardware. It would be a miracle if we didn't have to change to go along with that.
Toolchain stability and reproducibility is something I discussed. I even started to write something here: https://docs.dasharo.com/osf-trolling-list/build_process/
The main reason why you notice that with coreboot but not other firmware is that ancient-UEFI never gets uprevved (and while other firmware, like u-boot, collects firmware build support in their main repo, in products they're still used mostly in a copy&forget model). I just turned down a couple of older (non-coreboot) firmware build processes because they rely on python 2. They're dead, while coreboot isn't.
Yeah, our coreboot-sdk has problem having only python3, SeaBIOS do not understand that: https://github.com/coreboot/seabios/blob/master/Makefile#L25
At least this was problem on 4.13 tag.
The main reason why it's slightly less painful with Linux (but ask any out-of-tree module maintainer!) is that chip vendors provide open source code for Linux and maintain it whenever they raise the complexity bar a notch or two, and compiler vendors (gcc, clang) are coordinating against Linux (while they usually don't really care about coreboot).
tl;dr: All things considered, we're a pretty small project punching _way_ above our weight, working sometimes against the interests of other parties in the ecosystem who'd prefer to keep things closed that we open. Since (at this time) we can't offload the pain to those who inflict it on us the way Linux is doing, we'll have to bear it.
This last comment touch very important thing IMO. It is 10k vs small player case, if we could fill the gap between by growing the ecosystem we would be in way better position, and I believe there is a lot of place to grow considering UEFI market. Question is how has motivation to fill the gap.
Best Regards,