Hi folks,
something I've been thinking back and forth about since the hackathon a week ago: Should we give people who promote coreboot something to refer to, something that more clearly states, what coreboot is?
I know, some of you who know me might feel the urge to check if it isn't April the 1st. Nico suggesting a spec? What's happening here? I've always neglected the idea. Probably because I was too focused on fellow developers. But coreboot is not only discussed by develo- pers anymore. And we all know too well, developers don't make all the decisions. I believe having a document that basically says "that's coreboot, that's what people ask for" might help in, well, higher-level discussions.
So what do I suggest? To have some high-level description of what coreboot does, where it starts, where it ends. And also, to be com- prehensive, the things that are set in stone: cb-tables, SELF for the payload. It's in C header files. But I guess the information could be imported for the last chapters. Overall, the spec should not be long. I imagine something like 2~4 pages plus the cb-tables.
About the high-level part: Generally, I don't want to rush this. But I'll draft some things below that I already have on my mind. My current thoughts are mostly motivated by the "new bootflow for ARM64" thread[1] and Martin's excellent idea to rather call that "with coreboot technology" than "coreboot". Many people seem to agree. I guess, because it's not what we usually do. But what do we usually do? Shouldn't we write that down?
coreboot boot process ---------------------
coreboot's bootstrapping usually happens in two phases: 1. When a DRAM controller is part of the platform, it will be configured by coreboot to allow access to the main memory. 2. When the main memory is available, the platform is further initialized into an abstract state that allows generic opera- ting systems to run.
(some simple picture here; it's too late in the evening for ascii-art on my end)
coreboot starts with the first instruction on the main application processor (AP), or earlier when another processor runs platform initialization code from writable memory before the AP starts.
Exceptions are made, when * the first instructions run from a boot ROM, or * the first instructions are pre-defined by the silicon vendor and the hardware doesn't allow execution of other code, i.e. due to cryptographic signature checks. In this case, coreboot starts with the first instruction from writable memory that can be controlled by the platform owner. Future hardware iterations should strive to allow coreboot to run earlier.
After the hardware initialization, coreboot will write tables (cbtables, ACPI, SMBIOS and alike) with chip- and board-specific information. With this information, a generic operating system should be able to run without any further, board-specific know- ledge.
Finally, coreboot loads and executes an embedded payload program from the same memory that holds coreboot. If possible, coreboot will cease all other execution at this point. It is the payload program's responsibility to continue the boot strapping on its own, without relying on any services provided by coreboot but the in-memory tables.
Cheers, Nico
[1] https://mail.coreboot.org/hyperkitty/list/coreboot@coreboot.org/thread/BRW3Z...
Sounds to me like what you're asking for is really documentation, not a spec? Or maybe project-internal rules about what individual platform code may and may not do (but that's still not really a spec)?
In my understanding, a specification is always something defining a standard that allows interoperability _between_ different implementations. Something that only applies to a single implementation (i.e. a single code base) can't really be a specification. Making a "coreboot specification" would mean that someone else could then take that and implement their own "coreboot" in a completely separate cleanroom code base from scratch, and I don't think that makes sense. (We could create a specification for the coreboot payload handoff interface (i.e. coreboot tables) so that other people could write their own firmware stack that can run coreboot payloads. I don't think that would make much sense, though. If we wanted payload interoperability we should probably rather attach to one of the various other "universal payload" proposals that were going around, although we've had discussions about that before where I at least argued that I don't think that makes much sense for coreboot.)
Hi Julius,
On 28.11.23 03:31, Julius Werner wrote:
Sounds to me like what you're asking for is really documentation, not a spec?
well, yes and no. It would be documenting what we did all along. But also serve as a blueprint for coreboot.
I'm not all set on the term. I think it fits, even if it isn't what low-level developers would expect. But it seems more impor- tant to define (if not specify) what we think a coreboot should look like.
Or maybe project-internal rules about what individual platform code may and may not do (but that's still not really a spec)?
In my understanding, a specification is always something defining a standard that allows interoperability _between_ different implementations.
It can be, but it can also be about interchangeability. When you specify a product, for instance, you may want different manufacturers to produce it, without making a difference for the consumer.
Something that only applies to a single implementation (i.e. a single code base) can't really be a specification.
But there are different implementations of coreboot. Every time some- body does a bigger experiment on a local branch, that's a different coreboot. Or would you say it's still a single implementation when it shares 50% of the code base? 20%? 10%?
Making a "coreboot specification" would mean that someone else could then take that and implement their own "coreboot" in a completely separate cleanroom code base from scratch, and I don't think that makes sense.
I don't like superlatives. I don't think it needs to be "completely separate". For instance, when somebody discusses coreboot for a new platform behind closed doors[1]. And they implement something on the same code base. If they did that according to spec, it would be more likely to get accepted upstream, wouldn't it?
Nico
(We could create a specification for the coreboot payload handoff interface (i.e. coreboot tables) so that other people could write their own firmware stack that can run coreboot payloads. I don't think that would make much sense, though. If we wanted payload interoperability we should probably rather attach to one of the various other "universal payload" proposals that were going around, although we've had discussions about that before where I at least argued that I don't think that makes much sense for coreboot.)
[1] Let's not discuss if that should happen.
I don't like superlatives. I don't think it needs to be "completely separate". For instance, when somebody discusses coreboot for a new platform behind closed doors[1]. And they implement something on the same code base. If they did that according to spec, it would be more likely to get accepted upstream, wouldn't it?
I think whether stuff gets accepted upstream or not depends on whether it follows coreboot coding conventions, fits in with our existing APIs and general architecture, etc. I don't think it's really feasible to write everything that goes into the code review and design discussion process down in advance, and even if we could I'm not sure we'd really want to either. Do we want to create a situation where someone uploads code they developed in the dark and then tries to force us to take it because "it matches the spec", even though we don't like it for some good reason that we didn't anticipate in advance when writing the spec? I think developing in the open and seeking consensus among all upstream reviewers should continue to remain the officially recommended way to develop in coreboot, and anyone who for whatever reason develops stuff behind closed doors instead needs to understand that they're responsible for dealing with other opinions when they eventually decide to upload, including the risk that the majority of the community says "we don't want this at all" or "we want a complete redesign from the ground up".
If you want to add documentation that explains how coreboot code should fit together and where to integrate certain new features, or just lists general best practices, I'm all for that. We have a bit of that already and we could certainly always use more, or try to make it more organized and discoverable. I would just be wary of calling it anything that makes it sound official and authoritative. The term "specification" usually implies that as long as you follow this thing to the letter, you can _demand_ that your implementation should be considered correct and everyone else who doesn't accept it is wrong. I don't think we want anything like that for the coreboot development process. Helping people do the right thing from the start is great, but it should always remain understood that not everything that goes into the process can be written down in advance and that the final decision is done for each individual patch at review time.
Or is the question more about "up to what point are people allowed to say 'it runs coreboot' when they have out-of-tree code"? I'd say that's an entirely different thing (that I'm less interested in tbh, but maybe others are). I don't think we really have that problem in practice yet, and if we ever get to the point where we do I think just drawing the line at "runs 100% upstream code" should be good enough? coreboot is GPL so if people don't upstream their code there's really only two possible reasons, either they're lazy and unreciprocating, or their code is junk that wouldn't get accepted upstream.
On 29.11.23 02:01, Julius Werner wrote:
I don't like superlatives. I don't think it needs to be "completely separate". For instance, when somebody discusses coreboot for a new platform behind closed doors[1]. And they implement something on the same code base. If they did that according to spec, it would be more likely to get accepted upstream, wouldn't it?
I think whether stuff gets accepted upstream or not depends on whether it follows coreboot coding conventions, fits in with our existing APIs and general architecture, etc. I don't think it's really feasible to write everything that goes into the code review and design discussion process down in advance, and even if we could I'm not sure we'd really want to either.
Agreed. And that's why I want to describe things only at a rough, high level. I'd rather see compliance as a basic requirement than a suffi- cient condition. Direct things into the right direction early so there's more likely something worth reviewing.
Do we want to create a situation where someone uploads code they developed in the dark and then tries to force us to take it because "it matches the spec", even though we don't like it for some good reason that we didn't anticipate in advance when writing the spec?
Somebody could try that, sure, but I don't see how that would be worse than the "it's already written" we often hear today. More important, IMO, is if such a conflict would be more likely. If I'd get one "it matches the spec" conflict instead of five "it's already written" conflicts, that would make me quite happy.
I think developing in the open and seeking consensus among all upstream reviewers should continue to remain the officially recommended way to develop in coreboot, and anyone who for whatever reason develops stuff behind closed doors instead needs to understand that they're responsible for dealing with other opinions when they eventually decide to upload, including the risk that the majority of the community says "we don't want this at all" or "we want a complete redesign from the ground up".
I just want to level that risk. And it's not just the development. When somebody starts talking to a silicon vendor behind closed doors, that can set certain expectations about coreboot. If those turn out to be wrong later, that can make future work harder.
If you want to add documentation that explains how coreboot code should fit together and where to integrate certain new features, or just lists general best practices, I'm all for that. We have a bit of that already and we could certainly always use more, or try to make it more organized and discoverable. I would just be wary of calling it anything that makes it sound official and authoritative. The term "specification" usually implies that as long as you follow this thing to the letter, you can _demand_ that your implementation should be considered correct and everyone else who doesn't accept it is wrong. I don't think we want anything like that for the coreboot development process. Helping people do the right thing from the start is great, but it should always remain understood that not everything that goes into the process can be written down in advance and that the final decision is done for each individual patch at review time.
Like I said, I'm not set on calling it specification. And of course whatever we call it, it should have a defined scope, like an intro- duction that says what it applies for and what to expect.
Or is the question more about "up to what point are people allowed to say 'it runs coreboot' when they have out-of-tree code"? I'd say that's an entirely different thing (that I'm less interested in tbh, but maybe others are).
Same here, I guess. I'm mostly concerned about the development process upstream. Not about what people do downstream; if they don't bother me I don't want to bother them.
I don't think we really have that problem in practice yet, and if we ever get to the point where we do I think just drawing the line at "runs 100% upstream code" should be good enough?
That sounds rather infeasible, TBH. You may not know how hard it is to get things upstream when one isn't the first to work on a platform. Working around workarounds of others, maintaining undocumented boards that are already in the tree, having no word in what options a blob offers, etc. I believe coreboot has to go a long way before we can say that 100% upstream is possible for everybody.
coreboot is GPL so if people don't upstream their code there's really only two possible reasons, either they're lazy and unreciprocating, or their code is junk that wouldn't get accepted upstream.
Or maybe the code that is already upstream is junk and it's impossible to fix without breaking some boards? Or maybe we let blobs in that make hackish workarounds necessary that maybe shouldn't even get upstream?
Julius, I think in case of an ideal GPL project, I would agree with you. coreboot is just far from that.
Nico
-
Julius Werner -
Nico Huber