[RFC] POC: Use FDT as payload handoff instead lb_tables

[Paul Menzel]

Dear coreboot folks,

Arthur pushed the proof of concept *Use FDT as payload handoff instead 
lb_tables* [1] to pass information to the payload via FDT instead of 
coreboot tables.

Julius asked for more background, and Arthur replied. I am moving the 
discussion to the list for broader visibility, and better way of 
discussing these things than Gerrit comments.
> > Were there any discussions about this that I missed (if so would
> > you mind linking to them)? Can you explain who needs this and why,
> > what's the overall goal? (I know there have been vague discussions
> > about various new firmware handoff formats from Arm and Intel and
> > others, but if this is related to those I missed the parts where
> > there was suddenly consensus on how it should look, and why that is
> > FDT.)
>
> Most of the discussion happened verbally at osfc, so I can't point to
> any written communication. There is
> https://www.osfc.io/2022/talks/collaborative-firmware-payload-handoff-design...
> if you have the time to listen to a talk.
> So if I have to summarize the idea's/goals:
> Intel Universal scalable firmware suggested using UEFI HOBs as a handoff structure
> coreboot won't do that as it's no improvement over coreboot tables
> USF won't make coreboot tables a standard either
> Maybe something new can be created that would fit both the needs of payload handoff and other binary information passing between programs (e.g. FSP)?
> USF people suggested using CBOR to encode data to so that serialization/deserialization of data is explicit instead of using structs.
> CBOR is an improvement for the data format, but not what coreboot/(other project) should tell the payload: i.e. what entries are needed
> FDT was suggested as an alternative as it covers both explicit serialization/deserialization + what payload tell each other since the spec already exists, with a lot of existing compatible node strings. (e.g. I reuse simple-framebuffer from Linux here to hand of the framebuffer) Also most payloads and firmware have code for working with FDT.
> (unrelated to payload handof but part of the discussions) Another idea is to also use FDT as an interface between coreboot and FSP.
> a lot of payloads already have code to work with FDT so having FDT as a handoff structure would not be a lot of work
> FDT should make compatibility between coreboot and payload easier when handoff entries are added or changed (plain structs are painful if they are updated as both payload and coreboot definitions need to match)
> debugging the handoff structure is easier with FDT as you can use DTC in the OS userspace or coreboot code can also print out the full structure in human readable format.
>
> What are your thoughts?
> Is it a good idea to use FDT and eventually replace coreboot tables.
> If so, do we keep both for a while to make transitioning easier? I think maintaining two structures is not sustainable long term.
> How should the 'entries' be structured? I made compatible nodes that map somewhat to existing coreboot table structs.

Kind regards,

Paul

[1]: https://review.coreboot.org/c/coreboot/+/68719

[Krystian Hebel]

Hello,

This would fit nicely for POWER9 boards using Skiboot as a payload. I 
started a discussion related to that some time ago [1], since then we 
decided to switch to generating FDT in code. Basically, we did some 
parts of mentioned POC but at SoC level, to not break existing code. 
Having it as common code would make it much easier to maintain.
> > What are your thoughts?
> >
> > *   Is it a good idea to use FDT and eventually replace coreboot tables.

I think FDT is widely supported, from embedded platforms to servers, on 
many different architectures. I've got a feeling that whenever Intel 
talks about "universal", it usually means "x86-only", or at least "lets 
make it work for x86 and others can worry about their architectures later".

FDT is usually exposed by kernel so you don't need `cbmem` util for 
simplest cases, but I guess it would still be nice to have to decode 
specific tables. However:
> > *   If so, do we keep both for a while to make transitioning easier? 
> > I think maintaining two structures is not sustainable long term.

Persuading kernel to parse FDT on x86 platforms and propagating that 
change to kernels included in distros can take some time. Meanwhile, 
`cbmem` could be used to read the data instead, but it also would have 
to be made aware of the transition. IIUC currently it chooses between 
FDT and coreboot tables based on architecture it is built for. Users 
would have to update `cbmem` together with firmware, which may not 
always be the case.
> > *   How should the 'entries' be structured? I made compatible nodes 
> > that map somewhat to existing coreboot table structs.

As for standard coreboot tables like timestamps or console, for me 
existing format looks good. Just make sure to explicitly define 
endianness of fields :)

Based on experience with POWER9, we need to be able to both create 
entries specific to given SoC (e.g. XSCOM ranges, which is something 
between MSR and root complex "PCIe" devices) and to modify entries 
generated by common FDT code (Skiboot expects chip IDs on some of the 
entries when there are more CPUs on board). This can be easily done by 
having callouts to platform- and arch-specific code. Current FIT code 
doesn't work for the latter case, because call to `fit_update_memory()` 
happens after platform is allowed to modify the tree.

[1] 
https://mail.coreboot.org/hyperkitty/list/[e-mail]/thread/6EZWU...

[Julius Werner]

Hi Arthur,

Thanks for the detailed response. I went ahead and watched your
presentation as well. FWIW, I think there were a few technical
inaccuracies about the current state of coreboot tables in your
presentation that might be good to get out of the way first:

1. You said that coreboot tables were GPLv2 licensed, and this could
be an adoption problem for payloads... however, while
src/commonlib/include/commonlib/coreboot_tables.h has a GPLv2 header,
the same definitions for all structures have always been available in
payloads/libpayload/include/coreboot_tables.h with the BSD license, so
there's no actual issue for payloads. This is just a consequence of
the historic duplication we always had between coreboot and
libpayload. Now that we have the infrastructure to share commonlib/bsd
code between the two, I hope sooner or later someone will find the
time to delete coreboot's old (GPLv2) headers and move the libpayload
(BSD) headers into commonlib/bsd to be shared by both.

2. You said that there was an ABI problem due to different alignment
requirements for 64-bit types on different architectures. This has
actually never been true (unless someone made a mistake and slipped a
raw uint64_t into the table definitions, but we are generally pretty
vigilant in reviews about this so if it did happen it must have been
fixed quickly). If you look at commit 0d304c18 (so old that I can't
even link to Gerrit for it), we used to use a struct solution with
custom accessor functions to create 32-bit-aligned 64-bit types in the
past. More recently in https://review.coreboot.org/63494 we replaced
that with __attribute__((aligned, 4)) which does the same thing in a
less cumbersome way.

3. You also said that the coreboot tables aren't versioned, which is
of course technically true. However, the general guideline that I
think we have been trying to maintain in reviews is that coreboot
tables should be backwards-compatible, meaning that once a tag has
been widely used its structure layout should not change anymore (at
most new fields might be added at the end so that the size field can
be used to differentiate between both versions). Older versions of
libpayload should always be compatible with newer versions of coreboot
and as far as I'm aware we have done a pretty good job of maintaining
that standard. When a structure needs to be updated, a simple solution
is to just create a new tag and stop using the old one, and this has
been done in the past (e.g. LB_TAG_VBOOT_HANDOFF has been deprecated
and LB_TAG_VBOOT_WORKBUF basically contains the equivalent stuff in a
different format today). Fundamentally, this is pretty similar to how
FDT is used by Linux -- there's no explicit versioning for nodes or
properties, but bindings are meant to be stable, and when you really
need to change stuff up you can just make a new binding with different
compatible string and deprecate the old one.

I think the fundamental question here is still: what do we want to
achieve, and how does replacing coreboot tables with something else
help with that. One of the things you mentioned was so that we might
be able to use the same handoff format for FSP and the payload, and
the first question that comes to my mind is... why? What does that
actually give us? I think it's easy to sometimes fall into the
xkcd.com/927 trap of unifying things just for unification's sake,
because it feels "right", but ending up with a solution that's less
suited for either target than the sum of the burden of carrying all
individual existing solutions. The information that needs to be passed
to the FSP is (according to my understanding, I'm admittedly not an
FSP expert) completely different from the information that needs to be
passed to the payload (except maybe for one or two tiny specks like
console information?), so it's not like you'd actually want to reuse
the actual data structure you built and pass both things the union of
the stuff they each need to know, when the intersection of that is
almost empty. So what other reason is there to unify the format, then?
It's not like we have a ton of coreboot table support libraries or
tooling that we could drop if we unified them with something else --
that's the beauty about a very simple C structure format like that,
it's basically free to use.

The other aspect is the payload itself, and that whole "universal
payload" idea (I assume that's what Intel is calling "USF" now, or is
that something else?). I have to admit I'm just fundamentally very
skeptical of that whole concept, and I just don't believe it's gonna
work out in a way that's actually going to generate more benefit than
problems for anybody. The way we use payloads nowadays is often very
deeply tied to coreboot, with things like CBFS, verification, console,
timestamps, VPD, etc. all implemented in a very coreboot-specific way
(not just the handoff format but the very implementation of the thing
itself). You can't unify those things without unifying those
implementations which is practically impossible (i.e. if you unify
every aspect of how everything works between different firmware
projects then there's little point in having separate projects at all
anymore, because they'd all be doing the same thing anyway). So in
practice, I think the only thing a universal handoff would accomplish
is that e.g. you can then have a payload that will boot from coreboot
and start parsing coreboot's handoff, but then hangs because the flash
access bindings it is looking for don't exist in coreboot's handoff
and it doesn't know what to do with CBFS bindings. If you want a
payload that can support more than one firmware, with or without
universal handoff you'll have to implement specific support for each
anyway (e.g. like GRUB has implemented CBFS support), and once you're
doing that the extra cost of implementing a parser for a very simple,
dependency-less handoff format like the coreboot tables is not really
noticeable on top.

That said, I understand Intel wants to push this and there are people
who want to try this idea out, and I don't want to block them just
because I don't believe it'll work. So if you want to implement this
as an experimental, default-off Kconfig option that can hopefully stay
pretty isolated and self-contained in the code base, I'm fine with
that (and I don't particularly care about CBOR vs FDT... I think both
are pretty similar, both have tiny advantages and disadvantages for
your goals, and both add a similarly large overhead compared to
coreboot tables). But I wouldn't want this added as a replacement for
the coreboot tables either right away or with some implicit agreement
that we will transition there eventually. We don't even have any
practical use cases for universal payloads yet, but we have plenty of
big existing use cases that only combine coreboot with a single,
coreboot-specific payload and don't have any plans to require any more
flexibility on either of those sides in the future (ChromeOS is one of
them), and I wouldn't want those use cases to suffer from being
preemptively forward-ported onto a new solution that just adds a lot
of extra weight without having any proven benefits as of yet.

So basically, I'd like to make sure that we can agree to keep coreboot
tables as a first-class citizen now and in the future, and don't force
a new format on anyone. I'm not trying to stand in the way of
experimenting with new alternatives if people want to do that.

[Arthur Heymans]

Hi Julius

So FDT has a few advantages over C structs for passing information between
2 separate programs:
- reflection is easy
- no problem of 'header mismatch' to have the right definition of a struct
- easy to update with a text editor without needing to write code
- No need to be careful about ABI, writing structs e.g. 64bit alignment

I agree with you that these, while nice, are not very compelling arguments
to replace coreboot tables, which are very established by now.
For instance the header mismatch problem does not really exist given how
stable coreboot tables have actually been.
There is also no use case of wanting to edit the payload handoff by hand.
At last x86 Linux can be launched in around 200 bytes of stackless assembly
code... That's not possible with FDT or at least I'm not aware of any
stackless assembly written FDT parser.

For FSP it makes a lot more sense to improve the binary interface over the
existing C struct:
- reflection is a problem as FSP comes with default and printing out a
struct in C is a pain
- You never know whether the FSP binary will match a given header
- FSP validation is a manual process afaict. The people doing it don't want
to write code. That is why coreboot is not used in FSP validation.
Intel uses UEFI so that there is a menu to change FSP values for validation.

So it's not that FSP would need the same information as the payload. The
intersection is indeed almost 0.
It's just that FDT is actually a nice format for 2 separate programs to
talk to each other.
For instance u-boot also uses it to pass information from spl to u-boot
(bootloader) afaik.

I think Intel only wanted to try FDT as an interface for FSP if it's also
used in the payload (read EDK2) handoff (don't ask me why).
Maybe someone needs to convince Intel that using FDT for just FSP is a good
idea regardless of how payload handoff is handled.

So basically, I'd like to make sure that we can agree to keep coreboot
> tables as a first-class citizen now and in the future, and don't force
> a new format on anyone. I'm not trying to stand in the way of
> experimenting with new alternatives if people want to do that.

Having multiple ways to do the same thing is a maintenance problem. It was
a fun experiment, but it has no place in the master branch
if the end goal is just to have FDT live side by side with coreboot tables.

Arthur

On Fri, Nov 4, 2022 at 3:41 AM Julius Werner  wrote:
> Hi Arthur,
>
> Thanks for the detailed response. I went ahead and watched your
> presentation as well. FWIW, I think there were a few technical
> inaccuracies about the current state of coreboot tables in your
> presentation that might be good to get out of the way first:
> You said that coreboot tables were GPLv2 licensed, and this could
>
> be an adoption problem for payloads... however, while
> src/commonlib/include/commonlib/coreboot_tables.h has a GPLv2 header,
> the same definitions for all structures have always been available in
> payloads/libpayload/include/coreboot_tables.h with the BSD license, so
> there's no actual issue for payloads. This is just a consequence of
> the historic duplication we always had between coreboot and
> libpayload. Now that we have the infrastructure to share commonlib/bsd
> code between the two, I hope sooner or later someone will find the
> time to delete coreboot's old (GPLv2) headers and move the libpayload
> (BSD) headers into commonlib/bsd to be shared by both.
> You said that there was an ABI problem due to different alignment
>
> requirements for 64-bit types on different architectures. This has
> actually never been true (unless someone made a mistake and slipped a
> raw uint64_t into the table definitions, but we are generally pretty
> vigilant in reviews about this so if it did happen it must have been
> fixed quickly). If you look at commit 0d304c18 (so old that I can't
> even link to Gerrit for it), we used to use a struct solution with
> custom accessor functions to create 32-bit-aligned 64-bit types in the
> past. More recently in https://review.coreboot.org/63494 we replaced
> that with __attribute__((aligned, 4)) which does the same thing in a
> less cumbersome way.
> You also said that the coreboot tables aren't versioned, which is
>
> of course technically true. However, the general guideline that I
> think we have been trying to maintain in reviews is that coreboot
> tables should be backwards-compatible, meaning that once a tag has
> been widely used its structure layout should not change anymore (at
> most new fields might be added at the end so that the size field can
> be used to differentiate between both versions). Older versions of
> libpayload should always be compatible with newer versions of coreboot
> and as far as I'm aware we have done a pretty good job of maintaining
> that standard. When a structure needs to be updated, a simple solution
> is to just create a new tag and stop using the old one, and this has
> been done in the past (e.g. LB_TAG_VBOOT_HANDOFF has been deprecated
> and LB_TAG_VBOOT_WORKBUF basically contains the equivalent stuff in a
> different format today). Fundamentally, this is pretty similar to how
> FDT is used by Linux -- there's no explicit versioning for nodes or
> properties, but bindings are meant to be stable, and when you really
> need to change stuff up you can just make a new binding with different
> compatible string and deprecate the old one.
>
> I think the fundamental question here is still: what do we want to
> achieve, and how does replacing coreboot tables with something else
> help with that. One of the things you mentioned was so that we might
> be able to use the same handoff format for FSP and the payload, and
> the first question that comes to my mind is... why? What does that
> actually give us? I think it's easy to sometimes fall into the
> xkcd.com/927 trap of unifying things just for unification's sake,
> because it feels "right", but ending up with a solution that's less
> suited for either target than the sum of the burden of carrying all
> individual existing solutions. The information that needs to be passed
> to the FSP is (according to my understanding, I'm admittedly not an
> FSP expert) completely different from the information that needs to be
> passed to the payload (except maybe for one or two tiny specks like
> console information?), so it's not like you'd actually want to reuse
> the actual data structure you built and pass both things the union of
> the stuff they each need to know, when the intersection of that is
> almost empty. So what other reason is there to unify the format, then?
> It's not like we have a ton of coreboot table support libraries or
> tooling that we could drop if we unified them with something else --
> that's the beauty about a very simple C structure format like that,
> it's basically free to use.
>
> The other aspect is the payload itself, and that whole "universal
> payload" idea (I assume that's what Intel is calling "USF" now, or is
> that something else?). I have to admit I'm just fundamentally very
> skeptical of that whole concept, and I just don't believe it's gonna
> work out in a way that's actually going to generate more benefit than
> problems for anybody. The way we use payloads nowadays is often very
> deeply tied to coreboot, with things like CBFS, verification, console,
> timestamps, VPD, etc. all implemented in a very coreboot-specific way
> (not just the handoff format but the very implementation of the thing
> itself). You can't unify those things without unifying those
> implementations which is practically impossible (i.e. if you unify
> every aspect of how everything works between different firmware
> projects then there's little point in having separate projects at all
> anymore, because they'd all be doing the same thing anyway). So in
> practice, I think the only thing a universal handoff would accomplish
> is that e.g. you can then have a payload that will boot from coreboot
> and start parsing coreboot's handoff, but then hangs because the flash
> access bindings it is looking for don't exist in coreboot's handoff
> and it doesn't know what to do with CBFS bindings. If you want a
> payload that can support more than one firmware, with or without
> universal handoff you'll have to implement specific support for each
> anyway (e.g. like GRUB has implemented CBFS support), and once you're
> doing that the extra cost of implementing a parser for a very simple,
> dependency-less handoff format like the coreboot tables is not really
> noticeable on top.
>
> That said, I understand Intel wants to push this and there are people
> who want to try this idea out, and I don't want to block them just
> because I don't believe it'll work. So if you want to implement this
> as an experimental, default-off Kconfig option that can hopefully stay
> pretty isolated and self-contained in the code base, I'm fine with
> that (and I don't particularly care about CBOR vs FDT... I think both
> are pretty similar, both have tiny advantages and disadvantages for
> your goals, and both add a similarly large overhead compared to
> coreboot tables). But I wouldn't want this added as a replacement for
> the coreboot tables either right away or with some implicit agreement
> that we will transition there eventually. We don't even have any
> practical use cases for universal payloads yet, but we have plenty of
> big existing use cases that only combine coreboot with a single,
> coreboot-specific payload and don't have any plans to require any more
> flexibility on either of those sides in the future (ChromeOS is one of
> them), and I wouldn't want those use cases to suffer from being
> preemptively forward-ported onto a new solution that just adds a lot
> of extra weight without having any proven benefits as of yet.
>
> So basically, I'd like to make sure that we can agree to keep coreboot
> tables as a first-class citizen now and in the future, and don't force
> a new format on anyone. I'm not trying to stand in the way of
> experimenting with new alternatives if people want to do that.

[ron minnich]

I was involved in the creation of coreboot tables. They were the best we
could come up with at the time.

Over time, their limits have been more and more apparent. This is why, in
oreboot, we removed the initial support for coreboot tables and moved to
FDT.

FDT is better than coreboot tables. I'll repeat Arthur's points, the ones I
think are important:
"
- reflection is easy
- no problem of 'header mismatch' to have the right definition of a struct
- easy to update with a text editor without needing to write code
"

These are pretty critical. Further, Linux (and many if not most kernels in
fact) have built-in support for FDT, where they may not for coreboot tables.

Finally, FDT exists outside coreboot; unlike coreboot tables. There are
parsers in every language I've looked at (including Rust and Go) for fdt. I
think that's good. I think any time we can replace a coreboot-specific
standard with a standard that exists in a broader context than "just
coreboot", that's a good thing.

ron

On Fri, Nov 4, 2022 at 12:58 AM Arthur Heymans  wrote:
> Hi Julius
>
> So FDT has a few advantages over C structs for passing information between
> 2 separate programs:
> reflection is easy
> no problem of 'header mismatch' to have the right definition of a struct
> easy to update with a text editor without needing to write code
> No need to be careful about ABI, writing structs e.g. 64bit alignment
>
> I agree with you that these, while nice, are not very compelling arguments
> to replace coreboot tables, which are very established by now.
> For instance the header mismatch problem does not really exist given how
> stable coreboot tables have actually been.
> There is also no use case of wanting to edit the payload handoff by hand.
> At last x86 Linux can be launched in around 200 bytes of stackless assembly
> code... That's not possible with FDT or at least I'm not aware of any
> stackless assembly written FDT parser.
>
> For FSP it makes a lot more sense to improve the binary interface over the
> existing C struct:
> reflection is a problem as FSP comes with default and printing out a
>
> struct in C is a pain
> You never know whether the FSP binary will match a given header
> FSP validation is a manual process afaict. The people doing it don't
>
> want to write code. That is why coreboot is not used in FSP validation.
> Intel uses UEFI so that there is a menu to change FSP values for
> validation.
>
> So it's not that FSP would need the same information as the payload. The
> intersection is indeed almost 0.
> It's just that FDT is actually a nice format for 2 separate programs to
> talk to each other.
> For instance u-boot also uses it to pass information from spl to u-boot
> (bootloader) afaik.
>
> I think Intel only wanted to try FDT as an interface for FSP if it's also
> used in the payload (read EDK2) handoff (don't ask me why).
> Maybe someone needs to convince Intel that using FDT for just FSP is a
> good idea regardless of how payload handoff is handled.
>
> So basically, I'd like to make sure that we can agree to keep coreboot
> > tables as a first-class citizen now and in the future, and don't force
> > a new format on anyone. I'm not trying to stand in the way of
> > experimenting with new alternatives if people want to do that.
>
> Having multiple ways to do the same thing is a maintenance problem. It was
> a fun experiment, but it has no place in the master branch
> if the end goal is just to have FDT live side by side with coreboot tables.
>
> Arthur
>
> On Fri, Nov 4, 2022 at 3:41 AM Julius Werner  wrote:
> > Hi Arthur,
> >
> > Thanks for the detailed response. I went ahead and watched your
> > presentation as well. FWIW, I think there were a few technical
> > inaccuracies about the current state of coreboot tables in your
> > presentation that might be good to get out of the way first:
> > You said that coreboot tables were GPLv2 licensed, and this could
> >
> > be an adoption problem for payloads... however, while
> > src/commonlib/include/commonlib/coreboot_tables.h has a GPLv2 header,
> > the same definitions for all structures have always been available in
> > payloads/libpayload/include/coreboot_tables.h with the BSD license, so
> > there's no actual issue for payloads. This is just a consequence of
> > the historic duplication we always had between coreboot and
> > libpayload. Now that we have the infrastructure to share commonlib/bsd
> > code between the two, I hope sooner or later someone will find the
> > time to delete coreboot's old (GPLv2) headers and move the libpayload
> > (BSD) headers into commonlib/bsd to be shared by both.
> > You said that there was an ABI problem due to different alignment
> >
> > requirements for 64-bit types on different architectures. This has
> > actually never been true (unless someone made a mistake and slipped a
> > raw uint64_t into the table definitions, but we are generally pretty
> > vigilant in reviews about this so if it did happen it must have been
> > fixed quickly). If you look at commit 0d304c18 (so old that I can't
> > even link to Gerrit for it), we used to use a struct solution with
> > custom accessor functions to create 32-bit-aligned 64-bit types in the
> > past. More recently in https://review.coreboot.org/63494 we replaced
> > that with __attribute__((aligned, 4)) which does the same thing in a
> > less cumbersome way.
> > You also said that the coreboot tables aren't versioned, which is
> >
> > of course technically true. However, the general guideline that I
> > think we have been trying to maintain in reviews is that coreboot
> > tables should be backwards-compatible, meaning that once a tag has
> > been widely used its structure layout should not change anymore (at
> > most new fields might be added at the end so that the size field can
> > be used to differentiate between both versions). Older versions of
> > libpayload should always be compatible with newer versions of coreboot
> > and as far as I'm aware we have done a pretty good job of maintaining
> > that standard. When a structure needs to be updated, a simple solution
> > is to just create a new tag and stop using the old one, and this has
> > been done in the past (e.g. LB_TAG_VBOOT_HANDOFF has been deprecated
> > and LB_TAG_VBOOT_WORKBUF basically contains the equivalent stuff in a
> > different format today). Fundamentally, this is pretty similar to how
> > FDT is used by Linux -- there's no explicit versioning for nodes or
> > properties, but bindings are meant to be stable, and when you really
> > need to change stuff up you can just make a new binding with different
> > compatible string and deprecate the old one.
> >
> > I think the fundamental question here is still: what do we want to
> > achieve, and how does replacing coreboot tables with something else
> > help with that. One of the things you mentioned was so that we might
> > be able to use the same handoff format for FSP and the payload, and
> > the first question that comes to my mind is... why? What does that
> > actually give us? I think it's easy to sometimes fall into the
> > xkcd.com/927 trap of unifying things just for unification's sake,
> > because it feels "right", but ending up with a solution that's less
> > suited for either target than the sum of the burden of carrying all
> > individual existing solutions. The information that needs to be passed
> > to the FSP is (according to my understanding, I'm admittedly not an
> > FSP expert) completely different from the information that needs to be
> > passed to the payload (except maybe for one or two tiny specks like
> > console information?), so it's not like you'd actually want to reuse
> > the actual data structure you built and pass both things the union of
> > the stuff they each need to know, when the intersection of that is
> > almost empty. So what other reason is there to unify the format, then?
> > It's not like we have a ton of coreboot table support libraries or
> > tooling that we could drop if we unified them with something else --
> > that's the beauty about a very simple C structure format like that,
> > it's basically free to use.
> >
> > The other aspect is the payload itself, and that whole "universal
> > payload" idea (I assume that's what Intel is calling "USF" now, or is
> > that something else?). I have to admit I'm just fundamentally very
> > skeptical of that whole concept, and I just don't believe it's gonna
> > work out in a way that's actually going to generate more benefit than
> > problems for anybody. The way we use payloads nowadays is often very
> > deeply tied to coreboot, with things like CBFS, verification, console,
> > timestamps, VPD, etc. all implemented in a very coreboot-specific way
> > (not just the handoff format but the very implementation of the thing
> > itself). You can't unify those things without unifying those
> > implementations which is practically impossible (i.e. if you unify
> > every aspect of how everything works between different firmware
> > projects then there's little point in having separate projects at all
> > anymore, because they'd all be doing the same thing anyway). So in
> > practice, I think the only thing a universal handoff would accomplish
> > is that e.g. you can then have a payload that will boot from coreboot
> > and start parsing coreboot's handoff, but then hangs because the flash
> > access bindings it is looking for don't exist in coreboot's handoff
> > and it doesn't know what to do with CBFS bindings. If you want a
> > payload that can support more than one firmware, with or without
> > universal handoff you'll have to implement specific support for each
> > anyway (e.g. like GRUB has implemented CBFS support), and once you're
> > doing that the extra cost of implementing a parser for a very simple,
> > dependency-less handoff format like the coreboot tables is not really
> > noticeable on top.
> >
> > That said, I understand Intel wants to push this and there are people
> > who want to try this idea out, and I don't want to block them just
> > because I don't believe it'll work. So if you want to implement this
> > as an experimental, default-off Kconfig option that can hopefully stay
> > pretty isolated and self-contained in the code base, I'm fine with
> > that (and I don't particularly care about CBOR vs FDT... I think both
> > are pretty similar, both have tiny advantages and disadvantages for
> > your goals, and both add a similarly large overhead compared to
> > coreboot tables). But I wouldn't want this added as a replacement for
> > the coreboot tables either right away or with some implicit agreement
> > that we will transition there eventually. We don't even have any
> > practical use cases for universal payloads yet, but we have plenty of
> > big existing use cases that only combine coreboot with a single,
> > coreboot-specific payload and don't have any plans to require any more
> > flexibility on either of those sides in the future (ChromeOS is one of
> > them), and I wouldn't want those use cases to suffer from being
> > preemptively forward-ported onto a new solution that just adds a lot
> > of extra weight without having any proven benefits as of yet.
> >
> > So basically, I'd like to make sure that we can agree to keep coreboot
> > tables as a first-class citizen now and in the future, and don't force
> > a new format on anyone. I'm not trying to stand in the way of
> > experimenting with new alternatives if people want to do that.
>
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Julius Werner]

> FDT is better than coreboot tables. I'll repeat Arthur's points, the ones I think are important:
> "
> reflection is easy
> no problem of 'header mismatch' to have the right definition of a struct
> easy to update with a text editor without needing to write code
>
> "
>
> These are pretty critical. Further, Linux (and many if not most kernels in fact) have built-in support for FDT, where they may not for coreboot tables.

Well, Arthur also admitted that none of these points are really
relevant to coreboot's payload handoff, ;) which I fully agree with.
When was the last time you actually had to look at coreboot tables in
a hexdump? I think I have done that maybe once or twice total in my
~10 years of working with coreboot. We just don't need reflection, or
updating with a text editor. It's not a persistent structure that's
stored as a file on anyone's disk, it gets written out at runtime and
then consumed again immediately after. There are no environments with
text editors between coreboot and the payload.

I also disagree that FDTs are somehow more robust to "mismatches" than
C structures. Yes, the data format is self-describing, but what that
data means still has to be agreed upon by both sides. Where's the
difference between changing a member in a C structure from an integer
to a pointer, and changing a property in an FDT from an integer cell
to a phandle? Either way it's going to be broken if encountered by an
old reader -- maybe one leads to a segfault and the other "just" to a
clean parsing error, but that still doesn't mean it's not a problem
and you still need conventions to prevent it. So the convention is
that you just keep the structure layouts / bindings stable after
you've started using them, and that's what we're doing with coreboot
tables, just like projects using FDT are doing that with their
bindings. (FWIW, I'm not aware of any instances where we broke
coreboot table backwards compatibility in the past.)

I don't want to say things about FSP because I honestly don't know
enough about how that works and what people struggle with in practice
to have an informed opinion. If most of the people working with FSP
platforms think that using FDT as a handoff format there is a good
idea, then sure, do that. But that doesn't really make a difference
for what's better for payload handoff.

[Nico Huber]

Hi Ron,

On 04.11.22 16:58, ron minnich wrote:
> I was involved in the creation of coreboot tables. They were the best we
> could come up with at the time.
>
> Over time, their limits have been more and more apparent. This is why, in
> oreboot, we removed the initial support for coreboot tables and moved to
> FDT.

quick question, please bear my ignorance, I really don't know:
Does oreboot do payloads or does it only do LinuxBoot?

Nico

[Nico Huber]

Hi all,

On 04.11.22 08:57, Arthur Heymans wrote:
> So FDT has a few advantages over C structs for passing information between
> 2 separate programs:
> reflection is easy
> no problem of 'header mismatch' to have the right definition of a struct
> easy to update with a text editor without needing to write code
> No need to be careful about ABI, writing structs e.g. 64bit alignment

this is something I've been seeing a lot during the whole universal
payload discussions: The arguments often miss the point. These are
actually good arguments. But they are about the implementation that
handles the table data and not about the table format. One could
describe the coreboot table data in some equivalent of FDT bindings
and one could also write C structs for FDT bindings. I've personally
often argued for explicit serialization code instead of using C
structs in coreboot. It's doable. In case of the coreboot tables,
there's not that much benefit because the structs are well defined
and make no trouble with alignment (compared to weird `packed` structs).

(I hope I'm using the terms right as I'm not yet used to FDT things.)
> > I agree with you that these, while nice, are not very compelling arguments
>
> to replace coreboot tables, which are very established by now.
> For instance the header mismatch problem does not really exist given how
> stable coreboot tables have actually been.
> There is also no use case of wanting to edit the payload handoff by hand.
> At last x86 Linux can be launched in around 200 bytes of stackless assembly
> code... That's not possible with FDT or at least I'm not aware of any
> stackless assembly written FDT parser.

If there is no such thing yet, I'd argue to write a compiler for it.
One that takes the bindings and a description where to put the data,
and writes the assembler for you :)
> I think Intel only wanted to try FDT as an interface for FSP if it's also
> used in the payload (read EDK2) handoff (don't ask me why).
> Maybe someone needs to convince Intel that using FDT for just FSP is a good
> idea regardless of how payload handoff is handled.

They probably assume that FSP would talk to their payload directly
with only a shim in between. Remember, the whole discussion started
with somebody wanting to hand over HOBs to edk2 (which are currently
produced by FSP).

And if we are going to talk about the special-purpose payload (I think
that is what Intel means when they say `universal`) topic... one impor-
tant point that was discussed at the OSFC was that we could have a
neutral place to host the bindings (osfw.foundation was mentioned
IIRC). So that Intel would have to agree to an interface that they
couldn't dictate nor change whenever they like. IMO, this is the
most important point of the whole endeavor.

If we can't get to some common ground there, we can still talk about
FDT in coreboot. Although, IMHO, it would boil down to the simple
question if we want to use FDT so we can use already written libraries
and tools around it. Unless we wanted to replace ACPI with FDT¹ xD
> So basically, I'd like to make sure that we can agree to keep coreboot
> > tables as a first-class citizen now and in the future, and don't force
> > a new format on anyone. I'm not trying to stand in the way of
> > experimenting with new alternatives if people want to do that.
>
> Having multiple ways to do the same thing is a maintenance problem. It was
> a fun experiment, but it has no place in the master branch
> if the end goal is just to have FDT live side by side with coreboot tables.

Don't remember who said it. Somebody already said at the OSFC how a
future could look like: We should keep the coreboot table for compati-
bility in any case. We could freeze the format (structs) though and
say new things should be described in FDT. But that would mean that
we should keep two implementations in the tree, at least for a while.
Alternatively, keep one in a shim layer payload. But that's still two
things to maintain.

After all, I actually like the idea to use FDT. We just need to find
a good use case for it :)

Nico

¹ I wouldn't mind that btw., replacing ACPI. Right now FDT seems to me
like the better choice, it's just not what we have started with on x86.
As it's less complex (not a programming language), assuming we'd gene-
rate FDT info everywhere where we have ACPI now, it would be simple to
write a compatibility layer, i.e. something that knows the FDT bindings
we use and generates ACPI tables for them. Then one would only need ACPI
in coreboot (or a shim payload?) if they want compatibility to OSs that
don't understand FDT.

[Peter Stuge]

Julius Werner wrote:
> You said that coreboot tables were GPLv2 licensed, and this could
>
> be an adoption problem for payloads...

The only generator code I know is GPLv2.

One could argue that only coreboot should ever generate this format,
but I think one goal here is to enable or even encourage new situations.
> I think the fundamental question here is still: what do we want to
> achieve, and how does replacing coreboot tables with something else
> help with that.

This isn't just a technical change, for me there is also the more
political goal to make more of what Intel does with USF going forward
(please see https://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over... )
maximally useful for and within the coreboot context, ideally while
encouraging reuse.

Since Intel has planned to use CBOR and are unlikely to use coreboot
tables for anything outside of coreboot I see using FDT as a way to
improve the situation for everyone while making players come together
more - that's an unusually rare win-win.

I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
find important to not lose the simplicity of coreboot tables. FDT can
easily be parsed without RAM, CBOR not so much.
> use the same handoff format for FSP and the payload,
> What does that actually give us?

More common (data) patterns, more code re-use and potentially even
some future innovation once common patterns are more broadly used.
> ending up with a solution that's less suited for either target

I don't think FDT is less suited than coreboot tables where it
matters the most - simplicity on byte level.
> information that needs to be passed to the FSP is (according to my
> understanding, I'm admittedly not an FSP expert) completely different
> from the information that needs to be passed to the payload (except
> maybe for one or two tiny specks like console information?), so it's
> not like you'd actually want to reuse the actual data structure you
> built and pass both things the union of the stuff they each need to
> know, when the intersection of that is almost empty.

1. Already the possibility to maintain only a single dataset in a
single format is quite nice, both conceptually and for usability.

2. Both of these software packages change over time, given the above
possibility the intersection may expand.
> The other aspect is the payload itself, and that whole "universal
> payload" idea (I assume that's what Intel is calling "USF" now, or is
> that something else?).

Please do look at the diagram on the last page of the USF PDF.

I think it makes clear what Intel is working towards.
> The way we use payloads nowadays is often very deeply tied to
> coreboot,

More so if "often" means number of shipped systems, less so if "often"
means the number of individual payloads.
> You can't unify those things without unifying those implementations

Yes, but some compatibility is already a win in my opinion.
> the extra cost of implementing a parser for a very simple,
> dependency-less handoff format like the coreboot tables is
> not really noticeable on top.

This is similar enough for coreboot tables and FDT, that's why I
consider them really comparable.
> That said, I understand Intel wants to push this

Intel wants to push USF and CBOR.

"Our" people want that energy to fit well with coreboot.

If we manage to improve some interfaces within UEFI in the process
that would be even better.
> can hopefully stay pretty isolated and self-contained in the code base

I think there should be just one source of truth, from which other
formats are translated/derived.
> I don't particularly care about CBOR vs FDT... I think both
> are pretty similar, both have tiny advantages and disadvantages
> for your goals, and both add a similarly large overhead compared to
> coreboot tables).

On the byte level only FDT and coreboot tables are really comparable.
> I'd like to make sure that we can agree to keep coreboot tables as
> a first-class citizen now and in the future, and don't force a new
> format on anyone.

I don't think anyone proposed to not generate coreboot tables anymore.

Julius Werner wrote:
> > reflection is easy
> > no problem of 'header mismatch' to have the right definition of a struct
> > easy to update with a text editor without needing to write code
>
> Well, Arthur also admitted that none of these points are really
> relevant to coreboot's payload handoff, ;) which I fully agree with.

Currently that may be true, but reflection would pretty much be
required for a new payload for manual parameter tweaking during boot.
> (FWIW, I'm not aware of any instances where we broke
> coreboot table backwards compatibility in the past.)

I agree that coreboot tables have also been pretty stable.

Nico Huber wrote:
> I've personally often argued for explicit serialization code
> instead of using C structs in coreboot. It's doable. In case of the
> coreboot tables, there's not that much benefit because the structs
> are well defined and make no trouble with alignment (compared to
> weird `packed` structs).

I think the most important improvement we could make to coreboot
tables is to add explicit (de)serialization code, while others value
reflection more. I agree however that reflection would create opportunity
for new runtime UIs that are less doable or nonsensical without it.

As I understand it, cbtable member alignment is currently undefined.
That matters when building firmware and payload with different compilers.
"Just use the coreboot compiler" is not a good answer.
> > There is also no use case of wanting to edit the payload handoff by hand.
> > At last x86 Linux can be launched in around 200 bytes of stackless assembly
> > code... That's not possible with FDT or at least I'm not aware of any
> > stackless assembly written FDT parser.
>
> If there is no such thing yet, I'd argue to write a compiler for it.
> One that takes the bindings and a description where to put the data,
> and writes the assembler for you :)

Fun idea! It reminds me of romcc though.

Parsing FDT in assembler is anyway eminently doable, CBOR much less so.
> Unless we wanted to replace ACPI with FDT¹
> ¹ I wouldn't mind that btw., replacing ACPI.

That would be fantastic! But maybe it's more of a stretch goal. :)
> > Having multiple ways to do the same thing is a maintenance problem. It
> > was a fun experiment, but it has no place in the master branch if the
> > end goal is just to have FDT live side by side with coreboot tables.
>
> Don't remember who said it. Somebody already said at the OSFC how a
> future could look like: We should keep the coreboot table for compati-
> bility in any case. We could freeze the format (structs) though and
> say new things should be described in FDT. But that would mean that
> we should keep two implementations in the tree, at least for a while.
> Alternatively, keep one in a shim layer payload. But that's still two
> things to maintain.

I talked a bit about the idea to have a single origin format and
translation to other formats. Generic translators would not need
much maintainance.

//Peter

[Julius Werner]

> > You said that coreboot tables were GPLv2 licensed, and this could
> >
> > be an adoption problem for payloads...
>
> The only generator code I know is GPLv2.
>
> One could argue that only coreboot should ever generate this format,
> but I think one goal here is to enable or even encourage new situations.

Yes, I think only coreboot should ever generate this format. I'm
certainly not trying to make a case for "we should get other firmware
projects to adopt coreboot tables" here, I don't think that makes any
sense. I'm trying to make the case for "unifying payload handoff
formats between different firmwares is generally not a useful idea and
not worth the cost".
> > I think the fundamental question here is still: what do we want to
> > achieve, and how does replacing coreboot tables with something else
> > help with that.
>
> This isn't just a technical change, for me there is also the more
> political goal to make more of what Intel does with USF going forward
> (please see https://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over... )
> maximally useful for and within the coreboot context, ideally while
> encouraging reuse.
>
> Since Intel has planned to use CBOR and are unlikely to use coreboot
> tables for anything outside of coreboot I see using FDT as a way to
> improve the situation for everyone while making players come together
> more - that's an unusually rare win-win.
>
> I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
> find important to not lose the simplicity of coreboot tables. FDT can
> easily be parsed without RAM, CBOR not so much.

I'm a bit confused by the hate for CBOR here to be honest, I think
it's a very simple format and not much different from FDT in terms of
complexity (maybe the CBOR tag parsing is slightly more complicated,
but in return FDT has weird quirks like the memory reserve map,
separate strings table and phandles you have to deal with), and I
don't think parsing one of them without RAM would be much more
cumbersome than the other. But I don't want to replace the coreboot
tables with either so it's probably not worth going off on that
tangent.

I understand that some people here want to instrumentalize coreboot to
try to force Intel to change course on its USF specification, and
honestly I don't really care about that one way or the other, I'm just
asking to please not let the fallout of that negatively affect those
of us who still just care about using coreboot as a firmware and not
as a political cudgel.
> > ending up with a solution that's less suited for either target
>
> I don't think FDT is less suited than coreboot tables where it
> matters the most - simplicity on byte level.

FDT is a tokenized hierarchical key-value store with string keys (and
a ton of extra quirks), whereas the coreboot tables are just a tagged
list of C structures. That's an order of magnitude of difference in
simplicity (and binary size).
> > The way we use payloads nowadays is often very deeply tied to
> > coreboot,
>
> More so if "often" means number of shipped systems, less so if "often"
> means the number of individual payloads.

Well, if we want to open that line of argument we also have to
consider what's left in the handoff for the payloads where this isn't
true. If your payload isn't even accessing CBFS or printing logs to
the CBMEM console (both of which are supported directly within
libpayload), then what do you still need a handoff for at all? Just to
parse the mainboard vendor string?
> > can hopefully stay pretty isolated and self-contained in the code base
>
> I think there should be just one source of truth, from which other
> formats are translated/derived.

I mean I generally agree with that sentiment, but then we are at an
impasse because I disagree with abandoning coreboot tables or forcing
FDT generator code into coreboot images for those who aren't
interested in this project. I think keeping these things side-by-side
in the code base with a Kconfig to switch between them is the only way
we're going to let everyone achieve what they want here, even if it is
going to add extra maintenance burden.
> > I'd like to make sure that we can agree to keep coreboot tables as
> > a first-class citizen now and in the future, and don't force a new
> > format on anyone.
>
> I don't think anyone proposed to not generate coreboot tables anymore.

I'm talking about where this is eventually supposed to be headed, not
just the first experimental patch. I think we can all agree that
there's no point in having two firmware handoff structures that both
need to be parsed to get the full picture, and since the argument for
coreboot tables is mostly simplicity / lean code it's not a solution
for that side to just generate both.
> Julius Werner wrote:
> > > reflection is easy
> > > no problem of 'header mismatch' to have the right definition of a struct
> > > easy to update with a text editor without needing to write code
> >
> > Well, Arthur also admitted that none of these points are really
> > relevant to coreboot's payload handoff, ;) which I fully agree with.
>
> Currently that may be true, but reflection would pretty much be
> required for a new payload for manual parameter tweaking during boot.

Yeah, but what's the use case? When have you ever had a situation
where you'd want to edit coreboot table structures at runtime? (FWIW,
even if you wanted to do that for some reason it would still not be
very hard to write such a thing for coreboot tables -- the surrounding
UI and stuff of this shim payload would probably take the lion's share
of the effort, compared to the code to be able to delete/add/change
table entries for the known structures.)
> As I understand it, cbtable member alignment is currently undefined.
> That matters when building firmware and payload with different compilers.
> "Just use the coreboot compiler" is not a good answer.

Can we please stop perpetuating this myth? It's just wrong, like I
already explained in my first email here.
> > Unless we wanted to replace ACPI with FDT¹
> > ¹ I wouldn't mind that btw., replacing ACPI.
>
> That would be fantastic! But maybe it's more of a stretch goal. :)

FWIW I think this is a completely separate discussion and has nothing
to do with what we should do for payload handoff (again, just like
with the FSP thing, kernel handoff is a completely different thing
that would have very little to share with it). We already have support
for FDT for kernel handoff in coreboot today, after all.

For the record, I just met with Lean from 9elements about this as well
and basically discussed that same position -- that I don't want to
block anyone from doing this but that I would like it to remain a
compile-time option and would like coreboot tables to continue being
available and supported as a first-party citizen going forward (and
not "soft deprecated" in favor of FDT, or have the FDT stuff seep into
something that can not be compiled out). As I understood, Lean (and
Arthur?) want to continue pursuing this implementation under that
premise, and then we'll have to see in practice how well we can make
them coexist side-by-side without stepping on each other's toes.

[Arthur Heymans]

Hi

I appreciate all the solid input on this one!

If coreboot had to be rewritten, I'd argue FDT would be a nicer way of
structuring a payload handoff.
However that is not the case, and coreboot tables are well established in
many different payloads, supporting very different and specialized use
cases.
I also share the scepticism about the universal payload concept: the
reality is that hardware init and payload are tightly coupled.
The increased complexity for doing it otherwise may not be worth it, since
it involves a lot of runtime decisions based on the payload.
Is it really worth overhauling coreboot tables, which only pass limited
information like memory map, board id, framebuffer,... ?
Maybe...
Is it worth maintaining 2 payload handoffs in the tree? My answer is pretty
solid to that: *no*.
So I won't be pursuing this FDT as a payload handoff as something that
lives side by side with coreboot tables, as I've had my fill with multiple
codepaths doing the same thing in the tree:
romcc vs. C env bootblock, CAR teardown in ramstage vs. postcar, resource
allocation v3 vs. v4, legacy LAPIC_CPU_INIT vs. parallel_mp, ...
It's always a big mess to maintain multiple solutions.  I'd hate to
introduce more of that...

Currently EDK2 as a USF payload is built with a layer (shimlayer) that
translates coreboot tables and then loads the DXE stage as a separate
program.
That shimlayer is just very poorly implemented: it tries to guess the flash
memory mapping and then loads ELF at runtime.
A better way would be to use libpayload to parse coreboot tables and pass
whatever format EDK2 wants and have cbfstool process the DXE stage ELF (the
universal one ;-) ) so that it's easy to find and load.
As Libpayload is maintained with coreboot, using it to create a new
specialized payload loader is the proper way to deal with payloads
requiring a different format.

And completely unrelated to the payload handoff (so it indeed should not
impact that decision/discussion)
- FDT as an interface to FSP would be a big improvement for reusability.
Currently FSP is totally not reusable and you need code for each SOC, which
is exactly the opposite of what Intel USF claims to want to achieve.
- FDT to replace ACPI... For static information that might work, but ACPI
has a lot of real code in there that would require actual drivers in Linux
to replace it with. It's a lot of work, so it would best be set up as a
collaborative effort: let's pick a platform and go for it?

Kind regards
Arthur

On Tue, Nov 8, 2022 at 1:14 AM Julius Werner  wrote:
> > > You said that coreboot tables were GPLv2 licensed, and this could
> > >
> > > be an adoption problem for payloads...
> >
> > The only generator code I know is GPLv2.
> >
> > One could argue that only coreboot should ever generate this format,
> > but I think one goal here is to enable or even encourage new situations.
>
> Yes, I think only coreboot should ever generate this format. I'm
> certainly not trying to make a case for "we should get other firmware
> projects to adopt coreboot tables" here, I don't think that makes any
> sense. I'm trying to make the case for "unifying payload handoff
> formats between different firmwares is generally not a useful idea and
> not worth the cost".
> > > I think the fundamental question here is still: what do we want to
> > > achieve, and how does replacing coreboot tables with something else
> > > help with that.
> >
> > This isn't just a technical change, for me there is also the more
> > political goal to make more of what Intel does with USF going forward
> > (please see
>
> https://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over...
> )
> > maximally useful for and within the coreboot context, ideally while
> > encouraging reuse.
> >
> > Since Intel has planned to use CBOR and are unlikely to use coreboot
> > tables for anything outside of coreboot I see using FDT as a way to
> > improve the situation for everyone while making players come together
> > more - that's an unusually rare win-win.
> >
> > I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
> > find important to not lose the simplicity of coreboot tables. FDT can
> > easily be parsed without RAM, CBOR not so much.
>
> I'm a bit confused by the hate for CBOR here to be honest, I think
> it's a very simple format and not much different from FDT in terms of
> complexity (maybe the CBOR tag parsing is slightly more complicated,
> but in return FDT has weird quirks like the memory reserve map,
> separate strings table and phandles you have to deal with), and I
> don't think parsing one of them without RAM would be much more
> cumbersome than the other. But I don't want to replace the coreboot
> tables with either so it's probably not worth going off on that
> tangent.
>
> I understand that some people here want to instrumentalize coreboot to
> try to force Intel to change course on its USF specification, and
> honestly I don't really care about that one way or the other, I'm just
> asking to please not let the fallout of that negatively affect those
> of us who still just care about using coreboot as a firmware and not
> as a political cudgel.
> > > ending up with a solution that's less suited for either target
> >
> > I don't think FDT is less suited than coreboot tables where it
> > matters the most - simplicity on byte level.
>
> FDT is a tokenized hierarchical key-value store with string keys (and
> a ton of extra quirks), whereas the coreboot tables are just a tagged
> list of C structures. That's an order of magnitude of difference in
> simplicity (and binary size).
> > > The way we use payloads nowadays is often very deeply tied to
> > > coreboot,
> >
> > More so if "often" means number of shipped systems, less so if "often"
> > means the number of individual payloads.
>
> Well, if we want to open that line of argument we also have to
> consider what's left in the handoff for the payloads where this isn't
> true. If your payload isn't even accessing CBFS or printing logs to
> the CBMEM console (both of which are supported directly within
> libpayload), then what do you still need a handoff for at all? Just to
> parse the mainboard vendor string?
> > > can hopefully stay pretty isolated and self-contained in the code base
> >
> > I think there should be just one source of truth, from which other
> > formats are translated/derived.
>
> I mean I generally agree with that sentiment, but then we are at an
> impasse because I disagree with abandoning coreboot tables or forcing
> FDT generator code into coreboot images for those who aren't
> interested in this project. I think keeping these things side-by-side
> in the code base with a Kconfig to switch between them is the only way
> we're going to let everyone achieve what they want here, even if it is
> going to add extra maintenance burden.
> > > I'd like to make sure that we can agree to keep coreboot tables as
> > > a first-class citizen now and in the future, and don't force a new
> > > format on anyone.
> >
> > I don't think anyone proposed to not generate coreboot tables anymore.
>
> I'm talking about where this is eventually supposed to be headed, not
> just the first experimental patch. I think we can all agree that
> there's no point in having two firmware handoff structures that both
> need to be parsed to get the full picture, and since the argument for
> coreboot tables is mostly simplicity / lean code it's not a solution
> for that side to just generate both.
> > Julius Werner wrote:
> > > > reflection is easy
> > > > no problem of 'header mismatch' to have the right definition of a
>
> struct
> > > > easy to update with a text editor without needing to write code
> > >
> > > Well, Arthur also admitted that none of these points are really
> > > relevant to coreboot's payload handoff, ;) which I fully agree with.
> >
> > Currently that may be true, but reflection would pretty much be
> > required for a new payload for manual parameter tweaking during boot.
>
> Yeah, but what's the use case? When have you ever had a situation
> where you'd want to edit coreboot table structures at runtime? (FWIW,
> even if you wanted to do that for some reason it would still not be
> very hard to write such a thing for coreboot tables -- the surrounding
> UI and stuff of this shim payload would probably take the lion's share
> of the effort, compared to the code to be able to delete/add/change
> table entries for the known structures.)
> > As I understand it, cbtable member alignment is currently undefined.
> > That matters when building firmware and payload with different compilers.
> > "Just use the coreboot compiler" is not a good answer.
>
> Can we please stop perpetuating this myth? It's just wrong, like I
> already explained in my first email here.
> > > Unless we wanted to replace ACPI with FDT¹
> > > ¹ I wouldn't mind that btw., replacing ACPI.
> >
> > That would be fantastic! But maybe it's more of a stretch goal. :)
>
> FWIW I think this is a completely separate discussion and has nothing
> to do with what we should do for payload handoff (again, just like
> with the FSP thing, kernel handoff is a completely different thing
> that would have very little to share with it). We already have support
> for FDT for kernel handoff in coreboot today, after all.
>
> For the record, I just met with Lean from 9elements about this as well
> and basically discussed that same position -- that I don't want to
> block anyone from doing this but that I would like it to remain a
> compile-time option and would like coreboot tables to continue being
> available and supported as a first-party citizen going forward (and
> not "soft deprecated" in favor of FDT, or have the FDT stuff seep into
> something that can not be compiled out). As I understood, Lean (and
> Arthur?) want to continue pursuing this implementation under that
> premise, and then we'll have to see in practice how well we can make
> them coexist side-by-side without stepping on each other's toes.
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Jonathan Zhang (Infra)]

Hi, following is my perspective:

On FDT as a coreboot payload interface: I fully agree that coreboot table is well established, and we want to maintain the stability of the coreboot payload interface. That being said, FDT offers more than coreboot table does. Ron illustrated that very well, I would like to add that:
a. FDT is what Linux kernel expects on most of the embedded devices.
b. FDT is one of the two methods that ARM Linux server kernel expects. The other method is UEFI/ACPI. For coreboot to boot LinuxBoot on ARM server, coreboot need to provide either FDT or UEFI interface.
So my proposal is to add FDT as a parallel path, but do not retire coreboot table until FDT is very robust. I agree that 2 ways of doing same thing is generally not desired; I suppose that applies to more to coreboot internal, but not necessarily for external facing interfaces.

On FDT as an interface with FSP: such a discussion will not be effective without silicon vendor in the loop. When coreboot community and silicon vendor have trust and rapport with each other, such a discussion will go smoothly.

On FDT vs. ACPI: At least for target OS kernel on server, ACPI will be expected.

Thanks,
Jonathan
> On Nov 8, 2022, at 1:19 AM, Arthur Heymans  wrote:
>
> This Message Is From an External Sender
> Hi
>
> I appreciate all the solid input on this one!
>
> If coreboot had to be rewritten, I'd argue FDT would be a nicer way of structuring a payload handoff.
> However that is not the case, and coreboot tables are well established in many different payloads, supporting very different and specialized use cases.
> I also share the scepticism about the universal payload concept: the reality is that hardware init and payload are tightly coupled.
> The increased complexity for doing it otherwise may not be worth it, since it involves a lot of runtime decisions based on the payload.
> Is it really worth overhauling coreboot tables, which only pass limited information like memory map, board id, framebuffer,... ?
> Maybe...
> Is it worth maintaining 2 payload handoffs in the tree? My answer is pretty solid to that: *no*.
> So I won't be pursuing this FDT as a payload handoff as something that lives side by side with coreboot tables, as I've had my fill with multiple codepaths doing the same thing in the tree:
> romcc vs. C env bootblock, CAR teardown in ramstage vs. postcar, resource allocation v3 vs. v4, legacy LAPIC_CPU_INIT vs. parallel_mp, ...
> It's always a big mess to maintain multiple solutions.  I'd hate to introduce more of that...
>
> Currently EDK2 as a USF payload is built with a layer (shimlayer) that translates coreboot tables and then loads the DXE stage as a separate program.
> That shimlayer is just very poorly implemented: it tries to guess the flash memory mapping and then loads ELF at runtime.
> A better way would be to use libpayload to parse coreboot tables and pass whatever format EDK2 wants and have cbfstool process the DXE stage ELF (the universal one ;-) ) so that it's easy to find and load.
> As Libpayload is maintained with coreboot, using it to create a new specialized payload loader is the proper way to deal with payloads requiring a different format.
>
> And completely unrelated to the payload handoff (so it indeed should not impact that decision/discussion)
> FDT as an interface to FSP would be a big improvement for reusability. Currently FSP is totally not reusable and you need code for each SOC, which is exactly the opposite of what Intel USF claims to want to achieve.
> FDT to replace ACPI... For static information that might work, but ACPI has a lot of real code in there that would require actual drivers in Linux to replace it with. It's a lot of work, so it would best be set up as a collaborative effort: let's pick a platform and go for it?
>
> Kind regards
> Arthur
>
> On Tue, Nov 8, 2022 at 1:14 AM Julius Werner  wrote:
> > > You said that coreboot tables were GPLv2 licensed, and this could
> > >
> > > be an adoption problem for payloads...
> >
> > The only generator code I know is GPLv2.
> >
> > One could argue that only coreboot should ever generate this format,
> > but I think one goal here is to enable or even encourage new situations.
>
> Yes, I think only coreboot should ever generate this format. I'm
> certainly not trying to make a case for "we should get other firmware
> projects to adopt coreboot tables" here, I don't think that makes any
> sense. I'm trying to make the case for "unifying payload handoff
> formats between different firmwares is generally not a useful idea and
> not worth the cost".
> > > I think the fundamental question here is still: what do we want to
> > > achieve, and how does replacing coreboot tables with something else
> > > help with that.
> >
> > This isn't just a technical change, for me there is also the more
> > political goal to make more of what Intel does with USF going forward
> > (please see https://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over... )
> > maximally useful for and within the coreboot context, ideally while
> > encouraging reuse.
> >
> > Since Intel has planned to use CBOR and are unlikely to use coreboot
> > tables for anything outside of coreboot I see using FDT as a way to
> > improve the situation for everyone while making players come together
> > more - that's an unusually rare win-win.
> >
> > I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
> > find important to not lose the simplicity of coreboot tables. FDT can
> > easily be parsed without RAM, CBOR not so much.
>
> I'm a bit confused by the hate for CBOR here to be honest, I think
> it's a very simple format and not much different from FDT in terms of
> complexity (maybe the CBOR tag parsing is slightly more complicated,
> but in return FDT has weird quirks like the memory reserve map,
> separate strings table and phandles you have to deal with), and I
> don't think parsing one of them without RAM would be much more
> cumbersome than the other. But I don't want to replace the coreboot
> tables with either so it's probably not worth going off on that
> tangent.
>
> I understand that some people here want to instrumentalize coreboot to
> try to force Intel to change course on its USF specification, and
> honestly I don't really care about that one way or the other, I'm just
> asking to please not let the fallout of that negatively affect those
> of us who still just care about using coreboot as a firmware and not
> as a political cudgel.
> > > ending up with a solution that's less suited for either target
> >
> > I don't think FDT is less suited than coreboot tables where it
> > matters the most - simplicity on byte level.
>
> FDT is a tokenized hierarchical key-value store with string keys (and
> a ton of extra quirks), whereas the coreboot tables are just a tagged
> list of C structures. That's an order of magnitude of difference in
> simplicity (and binary size).
> > > The way we use payloads nowadays is often very deeply tied to
> > > coreboot,
> >
> > More so if "often" means number of shipped systems, less so if "often"
> > means the number of individual payloads.
>
> Well, if we want to open that line of argument we also have to
> consider what's left in the handoff for the payloads where this isn't
> true. If your payload isn't even accessing CBFS or printing logs to
> the CBMEM console (both of which are supported directly within
> libpayload), then what do you still need a handoff for at all? Just to
> parse the mainboard vendor string?
> > > can hopefully stay pretty isolated and self-contained in the code base
> >
> > I think there should be just one source of truth, from which other
> > formats are translated/derived.
>
> I mean I generally agree with that sentiment, but then we are at an
> impasse because I disagree with abandoning coreboot tables or forcing
> FDT generator code into coreboot images for those who aren't
> interested in this project. I think keeping these things side-by-side
> in the code base with a Kconfig to switch between them is the only way
> we're going to let everyone achieve what they want here, even if it is
> going to add extra maintenance burden.
> > > I'd like to make sure that we can agree to keep coreboot tables as
> > > a first-class citizen now and in the future, and don't force a new
> > > format on anyone.
> >
> > I don't think anyone proposed to not generate coreboot tables anymore.
>
> I'm talking about where this is eventually supposed to be headed, not
> just the first experimental patch. I think we can all agree that
> there's no point in having two firmware handoff structures that both
> need to be parsed to get the full picture, and since the argument for
> coreboot tables is mostly simplicity / lean code it's not a solution
> for that side to just generate both.
> > Julius Werner wrote:
> > > > reflection is easy
> > > > no problem of 'header mismatch' to have the right definition of a struct
> > > > easy to update with a text editor without needing to write code
> > >
> > > Well, Arthur also admitted that none of these points are really
> > > relevant to coreboot's payload handoff, ;) which I fully agree with.
> >
> > Currently that may be true, but reflection would pretty much be
> > required for a new payload for manual parameter tweaking during boot.
>
> Yeah, but what's the use case? When have you ever had a situation
> where you'd want to edit coreboot table structures at runtime? (FWIW,
> even if you wanted to do that for some reason it would still not be
> very hard to write such a thing for coreboot tables -- the surrounding
> UI and stuff of this shim payload would probably take the lion's share
> of the effort, compared to the code to be able to delete/add/change
> table entries for the known structures.)
> > As I understand it, cbtable member alignment is currently undefined.
> > That matters when building firmware and payload with different compilers.
> > "Just use the coreboot compiler" is not a good answer.
>
> Can we please stop perpetuating this myth? It's just wrong, like I
> already explained in my first email here.
> > > Unless we wanted to replace ACPI with FDT¹
> > > ¹ I wouldn't mind that btw., replacing ACPI.
> >
> > That would be fantastic! But maybe it's more of a stretch goal. :)
>
> FWIW I think this is a completely separate discussion and has nothing
> to do with what we should do for payload handoff (again, just like
> with the FSP thing, kernel handoff is a completely different thing
> that would have very little to share with it). We already have support
> for FDT for kernel handoff in coreboot today, after all.
>
> For the record, I just met with Lean from 9elements about this as well
> and basically discussed that same position -- that I don't want to
> block anyone from doing this but that I would like it to remain a
> compile-time option and would like coreboot tables to continue being
> available and supported as a first-party citizen going forward (and
> not "soft deprecated" in favor of FDT, or have the FDT stuff seep into
> something that can not be compiled out). As I understood, Lean (and
> Arthur?) want to continue pursuing this implementation under that
> premise, and then we'll have to see in practice how well we can make
> them coexist side-by-side without stepping on each other's toes.
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Christian Walter]

Hi,

I wasn't sure if I should also shim in here, but I just wanted to leave 
my opinion here. I agree with adding FDT as another method to handoff 
data to a payload. Despite all the advantages that Ron and others 
pointed out, I think something that passed data into another project 
e.g. payload here - whatever that is, should never be specific to one 
project. It should not be the status-quo that other projects, if they 
want to be loaded as a payload after coreboot need to implement other 
project specific implementations. What I mean by this is that 
hands-off's between different projects should always rely on an open 
standard that is *not* specific to the project.

In this particular case, my opinion is that other payloads implement 
coreboot tables should not be the defacto standard. coreboot should be 
encouraged to have a flexible option i.e. FDT to pass data into a 
payload - so that the payload would not care if that data is coming form 
coreboot - or any other underlying solution (oreboot?).

Of course I do understand the need for being backwards compatible, so I 
also would propose the approach to have these two solutions living 
site-by-site next to each other for some time - and then we as a 
community can decide how we want to move forward. However, we should 
never block those kind of approaches to make the open-source firmware 
would a bit more unified - and we should definitely not do it "just 
because Intel wants us to do it".

Best,

Chris

On 11/8/22 18:25, Jonathan Zhang (Infra) via coreboot wrote:
> Hi, following is my perspective:
>
> On FDT as a coreboot payload interface: I fully agree that coreboot table is well established, and we want to maintain the stability of the coreboot payload interface. That being said, FDT offers more than coreboot table does. Ron illustrated that very well, I would like to add that:
> a. FDT is what Linux kernel expects on most of the embedded devices.
> b. FDT is one of the two methods that ARM Linux server kernel expects. The other method is UEFI/ACPI. For coreboot to boot LinuxBoot on ARM server, coreboot need to provide either FDT or UEFI interface.
> So my proposal is to add FDT as a parallel path, but do not retire coreboot table until FDT is very robust. I agree that 2 ways of doing same thing is generally not desired; I suppose that applies to more to coreboot internal, but not necessarily for external facing interfaces.
>
> On FDT as an interface with FSP: such a discussion will not be effective without silicon vendor in the loop. When coreboot community and silicon vendor have trust and rapport with each other, such a discussion will go smoothly.
>
> On FDT vs. ACPI: At least for target OS kernel on server, ACPI will be expected.
>
> Thanks,
> Jonathan
> > On Nov 8, 2022, at 1:19 AM, Arthur Heymans  wrote:
> >
> > This Message Is From an External Sender
> > Hi
> >
> > I appreciate all the solid input on this one!
> >
> > If coreboot had to be rewritten, I'd argue FDT would be a nicer way of structuring a payload handoff.
> > However that is not the case, and coreboot tables are well established in many different payloads, supporting very different and specialized use cases.
> > I also share the scepticism about the universal payload concept: the reality is that hardware init and payload are tightly coupled.
> > The increased complexity for doing it otherwise may not be worth it, since it involves a lot of runtime decisions based on the payload.
> > Is it really worth overhauling coreboot tables, which only pass limited information like memory map, board id, framebuffer,... ?
> > Maybe...
> > Is it worth maintaining 2 payload handoffs in the tree? My answer is pretty solid to that: *no*.
> > So I won't be pursuing this FDT as a payload handoff as something that lives side by side with coreboot tables, as I've had my fill with multiple codepaths doing the same thing in the tree:
> > romcc vs. C env bootblock, CAR teardown in ramstage vs. postcar, resource allocation v3 vs. v4, legacy LAPIC_CPU_INIT vs. parallel_mp, ...
> > It's always a big mess to maintain multiple solutions.  I'd hate to introduce more of that...
> >
> > Currently EDK2 as a USF payload is built with a layer (shimlayer) that translates coreboot tables and then loads the DXE stage as a separate program.
> > That shimlayer is just very poorly implemented: it tries to guess the flash memory mapping and then loads ELF at runtime.
> > A better way would be to use libpayload to parse coreboot tables and pass whatever format EDK2 wants and have cbfstool process the DXE stage ELF (the universal one ;-) ) so that it's easy to find and load.
> > As Libpayload is maintained with coreboot, using it to create a new specialized payload loader is the proper way to deal with payloads requiring a different format.
> >
> > And completely unrelated to the payload handoff (so it indeed should not impact that decision/discussion)
> > FDT as an interface to FSP would be a big improvement for reusability. Currently FSP is totally not reusable and you need code for each SOC, which is exactly the opposite of what Intel USF claims to want to achieve.
> > FDT to replace ACPI... For static information that might work, but ACPI has a lot of real code in there that would require actual drivers in Linux to replace it with. It's a lot of work, so it would best be set up as a collaborative effort: let's pick a platform and go for it?
> >
> > Kind regards
> > Arthur
> >
> > On Tue, Nov 8, 2022 at 1:14 AM Julius Werner  wrote:
> > > > You said that coreboot tables were GPLv2 licensed, and this could
> > > >
> > > > be an adoption problem for payloads...
> > >
> > > The only generator code I know is GPLv2.
> > >
> > > One could argue that only coreboot should ever generate this format,
> > > but I think one goal here is to enable or even encourage new situations.
> >
> > Yes, I think only coreboot should ever generate this format. I'm
> > certainly not trying to make a case for "we should get other firmware
> > projects to adopt coreboot tables" here, I don't think that makes any
> > sense. I'm trying to make the case for "unifying payload handoff
> > formats between different firmwares is generally not a useful idea and
> > not worth the cost".
> > > > I think the fundamental question here is still: what do we want to
> > > > achieve, and how does replacing coreboot tables with something else
> > > > help with that.
> > >
> > > This isn't just a technical change, for me there is also the more
> > > political goal to make more of what Intel does with USF going forward
> > > (please seehttps://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over...  )
> > > maximally useful for and within the coreboot context, ideally while
> > > encouraging reuse.
> > >
> > > Since Intel has planned to use CBOR and are unlikely to use coreboot
> > > tables for anything outside of coreboot I see using FDT as a way to
> > > improve the situation for everyone while making players come together
> > > more - that's an unusually rare win-win.
> > >
> > > I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
> > > find important to not lose the simplicity of coreboot tables. FDT can
> > > easily be parsed without RAM, CBOR not so much.
> >
> > I'm a bit confused by the hate for CBOR here to be honest, I think
> > it's a very simple format and not much different from FDT in terms of
> > complexity (maybe the CBOR tag parsing is slightly more complicated,
> > but in return FDT has weird quirks like the memory reserve map,
> > separate strings table and phandles you have to deal with), and I
> > don't think parsing one of them without RAM would be much more
> > cumbersome than the other. But I don't want to replace the coreboot
> > tables with either so it's probably not worth going off on that
> > tangent.
> >
> > I understand that some people here want to instrumentalize coreboot to
> > try to force Intel to change course on its USF specification, and
> > honestly I don't really care about that one way or the other, I'm just
> > asking to please not let the fallout of that negatively affect those
> > of us who still just care about using coreboot as a firmware and not
> > as a political cudgel.
> > > > ending up with a solution that's less suited for either target
> > >
> > > I don't think FDT is less suited than coreboot tables where it
> > > matters the most - simplicity on byte level.
> >
> > FDT is a tokenized hierarchical key-value store with string keys (and
> > a ton of extra quirks), whereas the coreboot tables are just a tagged
> > list of C structures. That's an order of magnitude of difference in
> > simplicity (and binary size).
> > > > The way we use payloads nowadays is often very deeply tied to
> > > > coreboot,
> > >
> > > More so if "often" means number of shipped systems, less so if "often"
> > > means the number of individual payloads.
> >
> > Well, if we want to open that line of argument we also have to
> > consider what's left in the handoff for the payloads where this isn't
> > true. If your payload isn't even accessing CBFS or printing logs to
> > the CBMEM console (both of which are supported directly within
> > libpayload), then what do you still need a handoff for at all? Just to
> > parse the mainboard vendor string?
> > > > can hopefully stay pretty isolated and self-contained in the code base
> > >
> > > I think there should be just one source of truth, from which other
> > > formats are translated/derived.
> >
> > I mean I generally agree with that sentiment, but then we are at an
> > impasse because I disagree with abandoning coreboot tables or forcing
> > FDT generator code into coreboot images for those who aren't
> > interested in this project. I think keeping these things side-by-side
> > in the code base with a Kconfig to switch between them is the only way
> > we're going to let everyone achieve what they want here, even if it is
> > going to add extra maintenance burden.
> > > > I'd like to make sure that we can agree to keep coreboot tables as
> > > > a first-class citizen now and in the future, and don't force a new
> > > > format on anyone.
> > >
> > > I don't think anyone proposed to not generate coreboot tables anymore.
> >
> > I'm talking about where this is eventually supposed to be headed, not
> > just the first experimental patch. I think we can all agree that
> > there's no point in having two firmware handoff structures that both
> > need to be parsed to get the full picture, and since the argument for
> > coreboot tables is mostly simplicity / lean code it's not a solution
> > for that side to just generate both.
> > > Julius Werner wrote:
> > > > > reflection is easy
> > > > > no problem of 'header mismatch' to have the right definition of a struct
> > > > > easy to update with a text editor without needing to write code
> > > >
> > > > Well, Arthur also admitted that none of these points are really
> > > > relevant to coreboot's payload handoff, ;) which I fully agree with.
> > >
> > > Currently that may be true, but reflection would pretty much be
> > > required for a new payload for manual parameter tweaking during boot.
> >
> > Yeah, but what's the use case? When have you ever had a situation
> > where you'd want to edit coreboot table structures at runtime? (FWIW,
> > even if you wanted to do that for some reason it would still not be
> > very hard to write such a thing for coreboot tables -- the surrounding
> > UI and stuff of this shim payload would probably take the lion's share
> > of the effort, compared to the code to be able to delete/add/change
> > table entries for the known structures.)
> > > As I understand it, cbtable member alignment is currently undefined.
> > > That matters when building firmware and payload with different compilers.
> > > "Just use the coreboot compiler" is not a good answer.
> >
> > Can we please stop perpetuating this myth? It's just wrong, like I
> > already explained in my first email here.
> > > > Unless we wanted to replace ACPI with FDT¹
> > > > ¹ I wouldn't mind that btw., replacing ACPI.
> > >
> > > That would be fantastic! But maybe it's more of a stretch goal. :)
> >
> > FWIW I think this is a completely separate discussion and has nothing
> > to do with what we should do for payload handoff (again, just like
> > with the FSP thing, kernel handoff is a completely different thing
> > that would have very little to share with it). We already have support
> > for FDT for kernel handoff in coreboot today, after all.
> >
> > For the record, I just met with Lean from 9elements about this as well
> > and basically discussed that same position -- that I don't want to
> > block anyone from doing this but that I would like it to remain a
> > compile-time option and would like coreboot tables to continue being
> > available and supported as a first-party citizen going forward (and
> > not "soft deprecated" in favor of FDT, or have the FDT stuff seep into
> > something that can not be compiled out). As I understood, Lean (and
> > Arthur?) want to continue pursuing this implementation under that
> > premise, and then we'll have to see in practice how well we can make
> > them coexist side-by-side without stepping on each other's toes.
> > _______________________________________________
> > coreboot mailing list [e-mail]
> > To unsubscribe send an email [e-mail]
> > _______________________________________________
> > coreboot mailing list [e-mail]
> > To unsubscribe send an email [e-mail]
>
> coreboot mailing list [e-mail]
> To unsubscribe send an email [e-mail]

[ron minnich]

I agree with the "side by side" idea as well. Thanks to this community,
coreboot has always been good at growing with new ideas. This was a really
good discussion.

Thanks

Ron

On Tue, Nov 8, 2022 at 11:35 AM Christian Walter <
[e-mail]> wrote:
> Hi,
>
> I wasn't sure if I should also shim in here, but I just wanted to leave my
> opinion here. I agree with adding FDT as another method to handoff data to
> a payload. Despite all the advantages that Ron and others pointed out, I
> think something that passed data into another project e.g. payload here -
> whatever that is, should never be specific to one project. It should not be
> the status-quo that other projects, if they want to be loaded as a payload
> after coreboot need to implement other project specific implementations.
> What I mean by this is that hands-off's between different projects should
> always rely on an open standard that is *not* specific to the project.
>
> In this particular case, my opinion is that other payloads implement
> coreboot tables should not be the defacto standard. coreboot should be
> encouraged to have a flexible option i.e. FDT to pass data into a payload -
> so that the payload would not care if that data is coming form coreboot -
> or any other underlying solution (oreboot?).
>
> Of course I do understand the need for being backwards compatible, so I
> also would propose the approach to have these two solutions living
> site-by-site next to each other for some time - and then we as a community
> can decide how we want to move forward. However, we should never block
> those kind of approaches to make the open-source firmware would a bit more
> unified - and we should definitely not do it "just because Intel wants us
> to do it".
>
> Best,
>
> Chris
> On 11/8/22 18:25, Jonathan Zhang (Infra) via coreboot wrote:
>
> Hi, following is my perspective:
>
> On FDT as a coreboot payload interface: I fully agree that coreboot table is well established, and we want to maintain the stability of the coreboot payload interface. That being said, FDT offers more than coreboot table does. Ron illustrated that very well, I would like to add that:
> a. FDT is what Linux kernel expects on most of the embedded devices.
> b. FDT is one of the two methods that ARM Linux server kernel expects. The other method is UEFI/ACPI. For coreboot to boot LinuxBoot on ARM server, coreboot need to provide either FDT or UEFI interface.
> So my proposal is to add FDT as a parallel path, but do not retire coreboot table until FDT is very robust. I agree that 2 ways of doing same thing is generally not desired; I suppose that applies to more to coreboot internal, but not necessarily for external facing interfaces.
>
> On FDT as an interface with FSP: such a discussion will not be effective without silicon vendor in the loop. When coreboot community and silicon vendor have trust and rapport with each other, such a discussion will go smoothly.
>
> On FDT vs. ACPI: At least for target OS kernel on server, ACPI will be expected.
>
> Thanks,
> Jonathan
>
> On Nov 8, 2022, at 1:19 AM, Arthur Heymans   wrote:
>
> This Message Is From an External Sender
> Hi
>
> I appreciate all the solid input on this one!
>
> If coreboot had to be rewritten, I'd argue FDT would be a nicer way of structuring a payload handoff.
> However that is not the case, and coreboot tables are well established in many different payloads, supporting very different and specialized use cases.
> I also share the scepticism about the universal payload concept: the reality is that hardware init and payload are tightly coupled.
> The increased complexity for doing it otherwise may not be worth it, since it involves a lot of runtime decisions based on the payload.
> Is it really worth overhauling coreboot tables, which only pass limited information like memory map, board id, framebuffer,... ?
> Maybe...
> Is it worth maintaining 2 payload handoffs in the tree? My answer is pretty solid to that: *no*.
> So I won't be pursuing this FDT as a payload handoff as something that lives side by side with coreboot tables, as I've had my fill with multiple codepaths doing the same thing in the tree:
> romcc vs. C env bootblock, CAR teardown in ramstage vs. postcar, resource allocation v3 vs. v4, legacy LAPIC_CPU_INIT vs. parallel_mp, ...
> It's always a big mess to maintain multiple solutions.  I'd hate to introduce more of that...
>
> Currently EDK2 as a USF payload is built with a layer (shimlayer) that translates coreboot tables and then loads the DXE stage as a separate program.
> That shimlayer is just very poorly implemented: it tries to guess the flash memory mapping and then loads ELF at runtime.
> A better way would be to use libpayload to parse coreboot tables and pass whatever format EDK2 wants and have cbfstool process the DXE stage ELF (the universal one ;-) ) so that it's easy to find and load.
> As Libpayload is maintained with coreboot, using it to create a new specialized payload loader is the proper way to deal with payloads requiring a different format.
>
> And completely unrelated to the payload handoff (so it indeed should not impact that decision/discussion)
> FDT as an interface to FSP would be a big improvement for reusability. Currently FSP is totally not reusable and you need code for each SOC, which is exactly the opposite of what Intel USF claims to want to achieve.
> FDT to replace ACPI... For static information that might work, but ACPI has a lot of real code in there that would require actual drivers in Linux to replace it with. It's a lot of work, so it would best be set up as a collaborative effort: let's pick a platform and go for it?
>
> Kind regards
> Arthur
>
> On Tue, Nov 8, 2022 at 1:14 AM Julius Werner   wrote:
> You said that coreboot tables were GPLv2 licensed, and this could
>
> be an adoption problem for payloads...
>
> The only generator code I know is GPLv2.
>
> One could argue that only coreboot should ever generate this format,
> but I think one goal here is to enable or even encourage new situations.
>
> Yes, I think only coreboot should ever generate this format. I'm
> certainly not trying to make a case for "we should get other firmware
> projects to adopt coreboot tables" here, I don't think that makes any
> sense. I'm trying to make the case for "unifying payload handoff
> formats between different firmwares is generally not a useful idea and
> not worth the cost".
>
> I think the fundamental question here is still: what do we want to
> achieve, and how does replacing coreboot tables with something else
> help with that.
>
> This isn't just a technical change, for me there is also the more
> political goal to make more of what Intel does with USF going forward
> (please see https://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over... )
> maximally useful for and within the coreboot context, ideally while
> encouraging reuse.
>
> Since Intel has planned to use CBOR and are unlikely to use coreboot
> tables for anything outside of coreboot I see using FDT as a way to
> improve the situation for everyone while making players come together
> more - that's an unusually rare win-win.
>
> I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
> find important to not lose the simplicity of coreboot tables. FDT can
> easily be parsed without RAM, CBOR not so much.
>
> I'm a bit confused by the hate for CBOR here to be honest, I think
> it's a very simple format and not much different from FDT in terms of
> complexity (maybe the CBOR tag parsing is slightly more complicated,
> but in return FDT has weird quirks like the memory reserve map,
> separate strings table and phandles you have to deal with), and I
> don't think parsing one of them without RAM would be much more
> cumbersome than the other. But I don't want to replace the coreboot
> tables with either so it's probably not worth going off on that
> tangent.
>
> I understand that some people here want to instrumentalize coreboot to
> try to force Intel to change course on its USF specification, and
> honestly I don't really care about that one way or the other, I'm just
> asking to please not let the fallout of that negatively affect those
> of us who still just care about using coreboot as a firmware and not
> as a political cudgel.
>
> ending up with a solution that's less suited for either target
>
> I don't think FDT is less suited than coreboot tables where it
> matters the most - simplicity on byte level.
>
> FDT is a tokenized hierarchical key-value store with string keys (and
> a ton of extra quirks), whereas the coreboot tables are just a tagged
> list of C structures. That's an order of magnitude of difference in
> simplicity (and binary size).
>
> The way we use payloads nowadays is often very deeply tied to
> coreboot,
>
> More so if "often" means number of shipped systems, less so if "often"
> means the number of individual payloads.
>
> Well, if we want to open that line of argument we also have to
> consider what's left in the handoff for the payloads where this isn't
> true. If your payload isn't even accessing CBFS or printing logs to
> the CBMEM console (both of which are supported directly within
> libpayload), then what do you still need a handoff for at all? Just to
> parse the mainboard vendor string?
>
> can hopefully stay pretty isolated and self-contained in the code base
>
> I think there should be just one source of truth, from which other
> formats are translated/derived.
>
> I mean I generally agree with that sentiment, but then we are at an
> impasse because I disagree with abandoning coreboot tables or forcing
> FDT generator code into coreboot images for those who aren't
> interested in this project. I think keeping these things side-by-side
> in the code base with a Kconfig to switch between them is the only way
> we're going to let everyone achieve what they want here, even if it is
> going to add extra maintenance burden.
>
> I'd like to make sure that we can agree to keep coreboot tables as
> a first-class citizen now and in the future, and don't force a new
> format on anyone.
>
> I don't think anyone proposed to not generate coreboot tables anymore.
>
> I'm talking about where this is eventually supposed to be headed, not
> just the first experimental patch. I think we can all agree that
> there's no point in having two firmware handoff structures that both
> need to be parsed to get the full picture, and since the argument for
> coreboot tables is mostly simplicity / lean code it's not a solution
> for that side to just generate both.
>
> Julius Werner wrote:
> reflection is easy
> no problem of 'header mismatch' to have the right definition of a struct
> easy to update with a text editor without needing to write code
>
> Well, Arthur also admitted that none of these points are really
> relevant to coreboot's payload handoff, ;) which I fully agree with.
>
> Currently that may be true, but reflection would pretty much be
> required for a new payload for manual parameter tweaking during boot.
>
> Yeah, but what's the use case? When have you ever had a situation
> where you'd want to edit coreboot table structures at runtime? (FWIW,
> even if you wanted to do that for some reason it would still not be
> very hard to write such a thing for coreboot tables -- the surrounding
> UI and stuff of this shim payload would probably take the lion's share
> of the effort, compared to the code to be able to delete/add/change
> table entries for the known structures.)
>
> As I understand it, cbtable member alignment is currently undefined.
> That matters when building firmware and payload with different compilers.
> "Just use the coreboot compiler" is not a good answer.
>
> Can we please stop perpetuating this myth? It's just wrong, like I
> already explained in my first email here.
>
> Unless we wanted to replace ACPI with FDT¹
> ¹ I wouldn't mind that btw., replacing ACPI.
>
> That would be fantastic! But maybe it's more of a stretch goal. :)
>
> FWIW I think this is a completely separate discussion and has nothing
> to do with what we should do for payload handoff (again, just like
> with the FSP thing, kernel handoff is a completely different thing
> that would have very little to share with it). We already have support
> for FDT for kernel handoff in coreboot today, after all.
>
> For the record, I just met with Lean from 9elements about this as well
> and basically discussed that same position -- that I don't want to
> block anyone from doing this but that I would like it to remain a
> compile-time option and would like coreboot tables to continue being
> available and supported as a first-party citizen going forward (and
> not "soft deprecated" in favor of FDT, or have the FDT stuff seep into
> something that can not be compiled out). As I understood, Lean (and
> Arthur?) want to continue pursuing this implementation under that
> premise, and then we'll have to see in practice how well we can make
> them coexist side-by-side without stepping on each other's toes.
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>
> 9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany
> Email:  [e-mail]
> Phone:  *[phone] <[phone]>*
> Mobile:  *[phone] <[phone]>*
>
> Sitz der Gesellschaft: Bochum
> Handelsregister: Amtsgericht Bochum, HRB 17519
> Geschäftsführung: Sebastian Deutsch, Eray Basar
>
> Datenschutzhinweise nach Art. 13 DSGVO https://9elements.com/privacy

[Nico Huber]

Hi Chris,

On 08.11.22 20:35, Christian Walter wrote:
> I wasn't sure if I should also shim in here, but I just wanted to leave
> my opinion here. I agree with adding FDT as another method to handoff
> data to a payload. Despite all the advantages that Ron and others
> pointed out, I think something that passed data into another project
> e.g. payload here - whatever that is, should never be specific to one
> project. It should not be the status-quo that other projects, if they
> want to be loaded as a payload after coreboot need to implement other
> project specific implementations. What I mean by this is that
> hands-off's between different projects should always rely on an open
> standard that is *not* specific to the project.

this, having an open standard, is theoretically the ideal solution.
However, depending on the size of the problem, it might be much much
more effort to achieve this compared to the effort to work around its
absence. For instance, how many payload projects actually want to be
loaded by different firmware frameworks? is it 10%, 20%, 90%? I have
no idea. If it's few, the projects may lose interest before the new
standard is ready. Also, how much trouble is it to adapt a payload?
Let's not forget that the goal should be to make the work of firmware
developers easier, not harder.

And of course, such a standard shouldn't be decided by a single project
or group. If the goal is to get as many projects as possible to imple-
ment it, we should get many firmware framework and payload projects
together before making a decision. Otherwise, the risk is very high
that we work towards an instance of the famous xkcd/927.

Then about FDT: It can only cover a small part of the standard. There
is one important point that is often forgotten in this discussion:
  coreboot tables specify two things!
1. the binary format (this is what FDT can replace)
2. what data is passed (this is what bindings can replace)

Virtually nobody seems to be talking about 2. While IMO it would
be the much more important part of an open standard. Replacing 1.
is the part that creates churn; agreeing on 2. is mandatory to
actually achieve payload compatibility.

I know experimenting with the code is much more fun than creating
a standard. But replacing 1. alone won't make an open standard. It
would leave us with the 2. part of coreboot tables. Which IIRC,
was actually the part that Intel folks didn't want to agree with
in the first place.

Nico

[Lean Sheng Tan]

Thanks all for the good discussion. There will always be better ideas from
time to time, I am very happy to see it happens as long as someone is
willing to be in the driving seat on the effort to:
- come out with proper specification design
- communicate and with other firmware communities and silicon vendors for
discussions
- align and finalize all inputs together
- set a timeline and follow up the implementation to the end

Meanwhile in parallel, as mentioned, I will continue to work on the FDT
design with an incremental approach. Still drafting the specification and
the code structure, happy to share the spec later for wider feedback.
Please also let me know if you are interested in helping out :)

Best Regards,
*Lean Sheng Tan*

On Fri, 11 Nov 2022 at 15:31, Nico Huber  wrote:
> Hi Chris,
>
> On 08.11.22 20:35, Christian Walter wrote:
> > I wasn't sure if I should also shim in here, but I just wanted to leave
> > my opinion here. I agree with adding FDT as another method to handoff
> > data to a payload. Despite all the advantages that Ron and others
> > pointed out, I think something that passed data into another project
> > e.g. payload here - whatever that is, should never be specific to one
> > project. It should not be the status-quo that other projects, if they
> > want to be loaded as a payload after coreboot need to implement other
> > project specific implementations. What I mean by this is that
> > hands-off's between different projects should always rely on an open
> > standard that is *not* specific to the project.
>
> this, having an open standard, is theoretically the ideal solution.
> However, depending on the size of the problem, it might be much much
> more effort to achieve this compared to the effort to work around its
> absence. For instance, how many payload projects actually want to be
> loaded by different firmware frameworks? is it 10%, 20%, 90%? I have
> no idea. If it's few, the projects may lose interest before the new
> standard is ready. Also, how much trouble is it to adapt a payload?
> Let's not forget that the goal should be to make the work of firmware
> developers easier, not harder.
>
> And of course, such a standard shouldn't be decided by a single project
> or group. If the goal is to get as many projects as possible to imple-
> ment it, we should get many firmware framework and payload projects
> together before making a decision. Otherwise, the risk is very high
> that we work towards an instance of the famous xkcd/927.
>
> Then about FDT: It can only cover a small part of the standard. There
> is one important point that is often forgotten in this discussion:
>   coreboot tables specify two things!
> the binary format (this is what FDT can replace)
> what data is passed (this is what bindings can replace)
>
> Virtually nobody seems to be talking about 2. While IMO it would
> be the much more important part of an open standard. Replacing 1.
> is the part that creates churn; agreeing on 2. is mandatory to
> actually achieve payload compatibility.
>
> I know experimenting with the code is much more fun than creating
> a standard. But replacing 1. alone won't make an open standard. It
> would leave us with the 2. part of coreboot tables. Which IIRC,
> was actually the part that Intel folks didn't want to agree with
> in the first place.
>
> Nico
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Martin Roth]

've added the topic to the agenda of the next coreboot leadership meeting.  Hopefully some sort of decision can be made there.  I'm not trying to push any particular solution, but we need to be able to agree on something and then work towards implementinga single, or even a couple different options

Options:
- Mulitboot
- FDT
- CBOR
- Universal Payload
- coreboot tables.

I've made a document to capture pros & cons of each:
https://docs.google.com/document/d/1rN9PcrGliRCE-q2sa_k49WCInTELyyudN9RQvez5...

I'll try to go through the mailing list and capture arguments for and against each option, but if people could help out and fill things in as well, I'd appreciate it.

We need to be able to reach a decision and stick with it for a while, even if it's not the perfect solution and doesn't make everyone happy.

Martin

Nov 15, 2022, 05:17 by [e-mail]:
> Thanks all for the good discussion. There will always be better ideas from time to time, I am very happy to see it happens as long as someone is willing to be in the driving seat on the effort to:
> come out with proper specification design
> communicate and with other firmware communities and silicon vendors for discussions
> align and finalize all inputs together
> set a timeline and follow up the implementation to the end
>
> Meanwhile in parallel, as mentioned, I will continue to work on the FDT design with an incremental approach. Still drafting the specification and the code structure, happy to share the spec later for wider feedback. Please also let me know if you are interested in helping out :)
>
> Best Regards,
> Lean Sheng Tan
>
> On Fri, 11 Nov 2022 at 15:31, Nico Huber <> [e-mail]> > wrote:
> > Hi Chris,
> >
> > On 08.11.22 20:35, Christian Walter wrote:
> > > I wasn't sure if I should also shim in here, but I just wanted to leave
> > > my opinion here. I agree with adding FDT as another method to handoff
> > > data to a payload. Despite all the advantages that Ron and others
> > > pointed out, I think something that passed data into another project
> > > e.g. payload here - whatever that is, should never be specific to one
> > > project. It should not be the status-quo that other projects, if they
> > > want to be loaded as a payload after coreboot need to implement other
> > > project specific implementations. What I mean by this is that
> > > hands-off's between different projects should always rely on an open
> > > standard that is *not* specific to the project.
> >
> > this, having an open standard, is theoretically the ideal solution.
> >  However, depending on the size of the problem, it might be much much
> >  more effort to achieve this compared to the effort to work around its
> >  absence. For instance, how many payload projects actually want to be
> >  loaded by different firmware frameworks? is it 10%, 20%, 90%? I have
> >  no idea. If it's few, the projects may lose interest before the new
> >  standard is ready. Also, how much trouble is it to adapt a payload?
> >  Let's not forget that the goal should be to make the work of firmware
> >  developers easier, not harder.
> >
> > And of course, such a standard shouldn't be decided by a single project
> >  or group. If the goal is to get as many projects as possible to imple-
> >  ment it, we should get many firmware framework and payload projects
> >  together before making a decision. Otherwise, the risk is very high
> >  that we work towards an instance of the famous xkcd/927.
> >
> > Then about FDT: It can only cover a small part of the standard. There
> >  is one important point that is often forgotten in this discussion:
> >    coreboot tables specify two things!
> > the binary format (this is what FDT can replace)
> > what data is passed (this is what bindings can replace)
> >
> > Virtually nobody seems to be talking about 2. While IMO it would
> >  be the much more important part of an open standard. Replacing 1.
> >  is the part that creates churn; agreeing on 2. is mandatory to
> >  actually achieve payload compatibility.
> >
> > I know experimenting with the code is much more fun than creating
> >  a standard. But replacing 1. alone won't make an open standard. It
> >  would leave us with the 2. part of coreboot tables. Which IIRC,
> >  was actually the part that Intel folks didn't want to agree with
> >  in the first place.
> >
> > Nico
> >  _______________________________________________
> >  coreboot mailing list -- >> [e-mail]
> >  To unsubscribe send an email to >> [e-mail]

[Lean Sheng Tan]

Hi all,
Thanks again everyone for chipping in on this, I think that's what makes
the coreboot community healthy - that we have honest discussions while
allowing new ideas to grow.
I share the same sentiment as Ron, Nico and Peter - the goodness of
implementing FDT outweighs the cons, hence I will continue to pursue this
path.

As mentioned by Julius, we had a candid discussion yesterday on this, while
he still reserves his opinions on FDT, he will not block the
implementation. For now we will guard it with kconfig and give it a year of
trial to see how it turns out. IMHO coreboot continues to grow because it
allows people to take risks and innovate new things (that's why we are much
cooler), and the born of new things like resource allocator V2-V4
and parallel MP init, yes it introduced maintenance burden, but the
maintenance problems did not discredit the introduction of those ideas,
because the in the end the benefits of new innovations make coreboot
project better, while others complained why other firmware projects always
20 years lagging behind of the technology of OS/ kernels.

There is no way to please everyone, and there are always risks with new
innovation, and I believe coreboot is in a much more mature shape than 20
years ago to allow new ideas to flourish in the sense that it would
benefit the overall open source firmware communities.

Best Regards,
*Lean Sheng Tan*

On Tue, 8 Nov 2022 at 03:19, Arthur Heymans  wrote:
> Hi
>
> I appreciate all the solid input on this one!
>
> If coreboot had to be rewritten, I'd argue FDT would be a nicer way of
> structuring a payload handoff.
> However that is not the case, and coreboot tables are well established in
> many different payloads, supporting very different and specialized use
> cases.
> I also share the scepticism about the universal payload concept: the
> reality is that hardware init and payload are tightly coupled.
> The increased complexity for doing it otherwise may not be worth it, since
> it involves a lot of runtime decisions based on the payload.
> Is it really worth overhauling coreboot tables, which only pass limited
> information like memory map, board id, framebuffer,... ?
> Maybe...
> Is it worth maintaining 2 payload handoffs in the tree? My answer is
> pretty solid to that: *no*.
> So I won't be pursuing this FDT as a payload handoff as something that
> lives side by side with coreboot tables, as I've had my fill with multiple
> codepaths doing the same thing in the tree:
> romcc vs. C env bootblock, CAR teardown in ramstage vs. postcar, resource
> allocation v3 vs. v4, legacy LAPIC_CPU_INIT vs. parallel_mp, ...
> It's always a big mess to maintain multiple solutions.  I'd hate to
> introduce more of that...
>
> Currently EDK2 as a USF payload is built with a layer (shimlayer) that
> translates coreboot tables and then loads the DXE stage as a separate
> program.
> That shimlayer is just very poorly implemented: it tries to guess the
> flash memory mapping and then loads ELF at runtime.
> A better way would be to use libpayload to parse coreboot tables and pass
> whatever format EDK2 wants and have cbfstool process the DXE stage ELF (the
> universal one ;-) ) so that it's easy to find and load.
> As Libpayload is maintained with coreboot, using it to create a new
> specialized payload loader is the proper way to deal with payloads
> requiring a different format.
>
> And completely unrelated to the payload handoff (so it indeed should not
> impact that decision/discussion)
> FDT as an interface to FSP would be a big improvement for reusability.
>
> Currently FSP is totally not reusable and you need code for each SOC, which
> is exactly the opposite of what Intel USF claims to want to achieve.
> FDT to replace ACPI... For static information that might work, but ACPI
>
> has a lot of real code in there that would require actual drivers in Linux
> to replace it with. It's a lot of work, so it would best be set up as a
> collaborative effort: let's pick a platform and go for it?
>
> Kind regards
> Arthur
>
> On Tue, Nov 8, 2022 at 1:14 AM Julius Werner  wrote:
> > > > You said that coreboot tables were GPLv2 licensed, and this could
> > > >
> > > > be an adoption problem for payloads...
> > >
> > > The only generator code I know is GPLv2.
> > >
> > > One could argue that only coreboot should ever generate this format,
> > > but I think one goal here is to enable or even encourage new situations.
> >
> > Yes, I think only coreboot should ever generate this format. I'm
> > certainly not trying to make a case for "we should get other firmware
> > projects to adopt coreboot tables" here, I don't think that makes any
> > sense. I'm trying to make the case for "unifying payload handoff
> > formats between different firmwares is generally not a useful idea and
> > not worth the cost".
> > > > I think the fundamental question here is still: what do we want to
> > > > achieve, and how does replacing coreboot tables with something else
> > > > help with that.
> > >
> > > This isn't just a technical change, for me there is also the more
> > > political goal to make more of what Intel does with USF going forward
> > > (please see
> >
> > https://github.com/UniversalScalableFirmware/Introduction/blob/main/USF_Over...
> > )
> > > maximally useful for and within the coreboot context, ideally while
> > > encouraging reuse.
> > >
> > > Since Intel has planned to use CBOR and are unlikely to use coreboot
> > > tables for anything outside of coreboot I see using FDT as a way to
> > > improve the situation for everyone while making players come together
> > > more - that's an unusually rare win-win.
> > >
> > > I strongly prefer FDT over CBOR. FDT is simple on byte level, which I
> > > find important to not lose the simplicity of coreboot tables. FDT can
> > > easily be parsed without RAM, CBOR not so much.
> >
> > I'm a bit confused by the hate for CBOR here to be honest, I think
> > it's a very simple format and not much different from FDT in terms of
> > complexity (maybe the CBOR tag parsing is slightly more complicated,
> > but in return FDT has weird quirks like the memory reserve map,
> > separate strings table and phandles you have to deal with), and I
> > don't think parsing one of them without RAM would be much more
> > cumbersome than the other. But I don't want to replace the coreboot
> > tables with either so it's probably not worth going off on that
> > tangent.
> >
> > I understand that some people here want to instrumentalize coreboot to
> > try to force Intel to change course on its USF specification, and
> > honestly I don't really care about that one way or the other, I'm just
> > asking to please not let the fallout of that negatively affect those
> > of us who still just care about using coreboot as a firmware and not
> > as a political cudgel.
> > > > ending up with a solution that's less suited for either target
> > >
> > > I don't think FDT is less suited than coreboot tables where it
> > > matters the most - simplicity on byte level.
> >
> > FDT is a tokenized hierarchical key-value store with string keys (and
> > a ton of extra quirks), whereas the coreboot tables are just a tagged
> > list of C structures. That's an order of magnitude of difference in
> > simplicity (and binary size).
> > > > The way we use payloads nowadays is often very deeply tied to
> > > > coreboot,
> > >
> > > More so if "often" means number of shipped systems, less so if "often"
> > > means the number of individual payloads.
> >
> > Well, if we want to open that line of argument we also have to
> > consider what's left in the handoff for the payloads where this isn't
> > true. If your payload isn't even accessing CBFS or printing logs to
> > the CBMEM console (both of which are supported directly within
> > libpayload), then what do you still need a handoff for at all? Just to
> > parse the mainboard vendor string?
> > > > can hopefully stay pretty isolated and self-contained in the code base
> > >
> > > I think there should be just one source of truth, from which other
> > > formats are translated/derived.
> >
> > I mean I generally agree with that sentiment, but then we are at an
> > impasse because I disagree with abandoning coreboot tables or forcing
> > FDT generator code into coreboot images for those who aren't
> > interested in this project. I think keeping these things side-by-side
> > in the code base with a Kconfig to switch between them is the only way
> > we're going to let everyone achieve what they want here, even if it is
> > going to add extra maintenance burden.
> > > > I'd like to make sure that we can agree to keep coreboot tables as
> > > > a first-class citizen now and in the future, and don't force a new
> > > > format on anyone.
> > >
> > > I don't think anyone proposed to not generate coreboot tables anymore.
> >
> > I'm talking about where this is eventually supposed to be headed, not
> > just the first experimental patch. I think we can all agree that
> > there's no point in having two firmware handoff structures that both
> > need to be parsed to get the full picture, and since the argument for
> > coreboot tables is mostly simplicity / lean code it's not a solution
> > for that side to just generate both.
> > > Julius Werner wrote:
> > > > > reflection is easy
> > > > > no problem of 'header mismatch' to have the right definition of a
> >
> > struct
> > > > > easy to update with a text editor without needing to write code
> > > >
> > > > Well, Arthur also admitted that none of these points are really
> > > > relevant to coreboot's payload handoff, ;) which I fully agree with.
> > >
> > > Currently that may be true, but reflection would pretty much be
> > > required for a new payload for manual parameter tweaking during boot.
> >
> > Yeah, but what's the use case? When have you ever had a situation
> > where you'd want to edit coreboot table structures at runtime? (FWIW,
> > even if you wanted to do that for some reason it would still not be
> > very hard to write such a thing for coreboot tables -- the surrounding
> > UI and stuff of this shim payload would probably take the lion's share
> > of the effort, compared to the code to be able to delete/add/change
> > table entries for the known structures.)
> > > As I understand it, cbtable member alignment is currently undefined.
> > > That matters when building firmware and payload with different
> >
> > compilers.
> > > "Just use the coreboot compiler" is not a good answer.
> >
> > Can we please stop perpetuating this myth? It's just wrong, like I
> > already explained in my first email here.
> > > > Unless we wanted to replace ACPI with FDT¹
> > > > ¹ I wouldn't mind that btw., replacing ACPI.
> > >
> > > That would be fantastic! But maybe it's more of a stretch goal. :)
> >
> > FWIW I think this is a completely separate discussion and has nothing
> > to do with what we should do for payload handoff (again, just like
> > with the FSP thing, kernel handoff is a completely different thing
> > that would have very little to share with it). We already have support
> > for FDT for kernel handoff in coreboot today, after all.
> >
> > For the record, I just met with Lean from 9elements about this as well
> > and basically discussed that same position -- that I don't want to
> > block anyone from doing this but that I would like it to remain a
> > compile-time option and would like coreboot tables to continue being
> > available and supported as a first-party citizen going forward (and
> > not "soft deprecated" in favor of FDT, or have the FDT stuff seep into
> > something that can not be compiled out). As I understood, Lean (and
> > Arthur?) want to continue pursuing this implementation under that
> > premise, and then we'll have to see in practice how well we can make
> > them coexist side-by-side without stepping on each other's toes.
> > _______________________________________________
> > coreboot mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
>
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Julius Werner]

> As mentioned by Julius, we had a candid discussion yesterday on this, while he still reserves his opinions on FDT, he will not block the implementation. For now we will guard it with kconfig and give it a year of trial to see how it turns out.

So just to be clear, to make sure we weren't taking different things
away from that meeting: I said I'm okay with adding the FDT option as
a separate, isolated compile-time option for those that want to try it
out. But I care that the coreboot tables continue to be supported on
equal footing in the long term, and I don't want there to be some
pre-formed understanding that they'll be "deprecated eventually" or
that payloads will be slowly pressured to move to the FDT handoff,
particularly not with any specific timeline attached.

[ron minnich]

makes sense to me.

On Tue, Nov 8, 2022 at 2:55 PM Julius Werner  wrote:
> > As mentioned by Julius, we had a candid discussion yesterday on this,
>
> while he still reserves his opinions on FDT, he will not block the
> implementation. For now we will guard it with kconfig and give it a year of
> trial to see how it turns out.
>
> So just to be clear, to make sure we weren't taking different things
> away from that meeting: I said I'm okay with adding the FDT option as
> a separate, isolated compile-time option for those that want to try it
> out. But I care that the coreboot tables continue to be supported on
> equal footing in the long term, and I don't want there to be some
> pre-formed understanding that they'll be "deprecated eventually" or
> that payloads will be slowly pressured to move to the FDT handoff,
> particularly not with any specific timeline attached.

[Arthur Heymans]

> So just to be clear, to make sure we weren't taking different things
> away from that meeting: I said I'm okay with adding the FDT option as
> a separate, isolated compile-time option for those that want to try it
> out. But I care that the coreboot tables continue to be supported on
> equal footing in the long term, and I don't want there to be some
> pre-formed understanding that they'll be "deprecated eventually" or
> that payloads will be slowly pressured to move to the FDT handoff,
> particularly not with any specific timeline attached.
>
> With this in mind I don't see any reasons why generating the FDT handoff

needs to happen in the coreboot ramstage.
A libpayload based payload could just as well generate that FDT handoff and
chainload the next payload.
All the infrastructure is there in libpayload to cleanly do this.
The coreboot code for dealing with FDT is exactly the same as deptcharge
code (it comes from that project).
This is a better technical solution than to introduce new Kconfig guarded
code and maintenance overhead inside the coreboot tree to maintain 2
solutions for the same problem.
As a matter of fact that solution is even more 'universal' as you could
still swap out payloads without recompiling the coreboot part of things.

Arthur

On Tue, Nov 8, 2022 at 11:55 PM Julius Werner  wrote:
> > As mentioned by Julius, we had a candid discussion yesterday on this,
>
> while he still reserves his opinions on FDT, he will not block the
> implementation. For now we will guard it with kconfig and give it a year of
> trial to see how it turns out.
>
> So just to be clear, to make sure we weren't taking different things
> away from that meeting: I said I'm okay with adding the FDT option as
> a separate, isolated compile-time option for those that want to try it
> out. But I care that the coreboot tables continue to be supported on
> equal footing in the long term, and I don't want there to be some
> pre-formed understanding that they'll be "deprecated eventually" or
> that payloads will be slowly pressured to move to the FDT handoff,
> particularly not with any specific timeline attached.

[Peter Stuge]

Julius Werner wrote:
> I think only coreboot should ever generate this format. I'm certainly
> not trying to make a case for "we should get other firmware projects
> to adopt coreboot tables" here, I don't think that makes any sense.

I think it makes some sense because coreboot does what we consider best.
Of course we want others to reuse the very best parts of that.

But I also recognize that it is unlikely to happen, so to improve the
situation overall (ie. also outside coreboot) I want to consider FDT
in coreboot.
> I'm trying to make the case for "unifying payload handoff formats
> between different firmwares is generally not a useful idea

MBR called to disagree.
> and not worth the cost".

This is the only interesting discussion.
> I understand that some people here want to instrumentalize coreboot
> to try to force Intel to change course on its USF specification, and
> honestly I don't really care about that one way or the other,

I find that silly polemic. I see this not as instrumentalizing
coreboot but as coreboot accomodating Intel a little while also
getting benefit - of course with the longer term goal of Intel
and coreboot becoming a better match.

You don't have to care about USF or Intel but do consider that
there probably would be no coreboot community within industry
were coreboot not to add value to Intel hardware.

Our community is obviously not in danger over this but the better
Intel and coreboot go together the more opportunities for really
good solutions we may have in the future.

If coreboot doesn't push Intel then Intel will push coreboot (not
neccessarily on purpose, just through inertia). You may want to
ignore that for now but the coreboot will be dealing with the outcome
for some time and now is when we can act.
> I'm just asking to please not let the fallout of that negatively
> affect those of us who still just care about using coreboot as a
> firmware and not as a political cudgel.

I understand if you and others want to ignore politics, I did too for
a long time.

That said, I wouldn't worry about negative effects. I certainly have
no interest to remove coreboot tables nor do I see any reason to do
so nor have I seen anyone mention anything in that direction.

Do not make up strawmen.
> > I don't think FDT is less suited than coreboot tables where it
> > matters the most - simplicity on byte level.

..
> I'm a bit confused by the hate for CBOR here to be honest,

..
> I don't think parsing one of them without RAM would be much more
> cumbersome than the other.

..
> FDT is a tokenized hierarchical key-value store with string keys (and
> a ton of extra quirks), whereas the coreboot tables are just a tagged
> list of C structures. That's an order of magnitude of difference in
> simplicity (and binary size).

A RAM-less FDT parser probably doesn't need to load the entire tree
but merely find one value at a time, perhaps without state between
values. The complexity of that in an FDT is low since FDT is flat and
has simple primitives. CBOR, well..

--8<-- RFC8949
   The initial byte of each encoded data item contains both information
   about the major type (the high-order 3 bits, described in
   Section 3.1) and additional information (the low-order 5 bits).  With
   a few exceptions, the additional information's value describes how to
   load an unsigned integer "argument":

Less than 24:  The argument's value is the value of the additional
      information.

24, 25, 26, or 27:  The argument's value is held in the following 1,
      2, 4, or 8 bytes, respectively, in network byte order.  For major
      type 7 and additional information value 25, 26, 27, these bytes
      are not used as an integer argument, but as a floating-point value
      (see Section 3.3).

28, 29, 30:  These values are reserved for future additions to the
      CBOR format.  In the present version of CBOR, the encoded item is
      not well-formed.

31:  No argument value is derived.  If the major type is 0, 1, or 6,
      the encoded item is not well-formed.  For major types 2 to 5, the
      item's length is indefinite, and for major type 7, the byte does
      not constitute a data item at all but terminates an indefinite-
      length item; all are described in Section 3.2.
-->8--

This is bizarrely complex for storing a simple scalar.
> what do you still need a handoff for at all?
> Just to parse the mainboard vendor string?

Memory map at a minimum.
> I disagree with abandoning coreboot tables or forcing FDT generator code
> into coreboot images for those who aren't interested in this project.

So a way forward where you can ignore FDT is for coreboot tables to
stay as-is, and a Kconfig option to make coreboot use its coreboot
tables to generate an FDT.
> I think keeping these things side-by-side in the code base with a
> Kconfig to switch between them

Like Arthur I don't appreciate the "side-by-side" narrative, nor an
expectation for two independent data sets.
> > > I'd like to make sure that we can agree to keep coreboot tables as
> > > a first-class citizen now and in the future, and don't force a new
> > > format on anyone.
> >
> > I don't think anyone proposed to not generate coreboot tables anymore.
>
> I'm talking about where this is eventually supposed to be headed, not
> just the first experimental patch.

This is supposed to be headed to additional benefit for both coreboot
and USF (users). Others will use coreboot in different ways, e.g.
without FDT.
> I think we can all agree that there's no point in having two
> firmware handoff structures that both need to be parsed to get the
> full picture,

Noone suggested anything like that. Don't waste our time.
> and since the argument for coreboot tables is mostly simplicity /
> lean code it's not a solution for that side to just generate both.

If you mean "it's not a solution for coreboot to just generate both"
then I ask why not? Two things can be true at the same time: coreboot
tables are simple to generate and the code to generate FDT from
coreboot tables is also simple.
> > > Well, Arthur also admitted that none of these points are really
> > > relevant to coreboot's payload handoff, ;) which I fully agree with.
> >
> > Currently that may be true, but reflection would pretty much be
> > required for a new payload for manual parameter tweaking during boot.
>
> Yeah, but what's the use case?

There are several, both outside and within the boot process.

FDT can be meaningfully persisted and so be useful both before and after boot.
> When have you ever had a situation where you'd want to edit
> coreboot table structures at runtime?

You can also consider FDT in coreboot as plumbing to enable new
situations that were best avoided with coreboot tables, then of
course noone ever wanted to edit them. That doesn't mean that noone
would ever want to edit firmware handoff information, especially if
we also consider FSP.

Please be a bit more imaginative here.
> it would still not be very hard to write such a thing for coreboot
> tables

Of course. One goal is to have more freedom than that allows, or at a
very minimum fill a namespace not (exclusively) defined by coreboot
or Intel.
> > As I understand it, cbtable member alignment is currently undefined.
> > That matters when building firmware and payload with different compilers.
> > "Just use the coreboot compiler" is not a good answer.
>
> Can we please stop perpetuating this myth? It's just wrong, like I
> already explained in my first email here.

I don't think it's wrong. I used to, but not anymore.

This is not about 64-bit values which you mentioned in your first
email, this is about struct member offsets in the struct.

The structs are not packed so the compiler decides.

This is why both Nico and I are so strong proponents of explicit
serialization of the tables. (I'd also like to create new tags for
coreboot tables with packed values but that's another matter.)
> > > ¹ I wouldn't mind that btw., replacing ACPI.
> >
> > That would be fantastic! But maybe it's more of a stretch goal. :)
>
> FWIW I think this is a completely separate discussion and has nothing
> to do with what we should do for payload handoff (again, just like
> with the FSP thing, kernel handoff is a completely different thing
> that would have very little to share with it).

If all handoff interfaces were to use a single format (this would be
great) then obviously it can be a single data set, which would make
life simpler for a lot of different people.
> (and not "soft deprecated" in favor of FDT,

Since this your narrative is baseless it makes it look like you are
intentionally acting in bad faith. That's not a good look. Be careful.
> or have the FDT stuff seep into something that can not be compiled out).

Like vboot has been for maybe a decade now yes, I agree strongly that
this is important to avoid.
> without stepping on each other's toes.

I wouldn't worry.

//Peter

[ron minnich]

I mostly agree with Peter here:
"So a way forward where you can ignore FDT is for coreboot tables to
stay as-is, and a Kconfig option to make coreboot use its coreboot
tables to generate an FDT."

I suspect that, someday, the FDT may contain different information than is
contained in coreboot tables. We can not anticipate everything: as needs
change, we could find it is not possible to create the necessary FDT from
coreboot tables.

For that reason, I think FDT creation should be in coreboot, not in a
stage, because then the creation of the FDT will be easiest. And enabling
FDT with a kconfig option makes sense. coreboot tables are small, so,
thinking about it, it is hard to see a case where we'd ever not have
coreboot tables as well.

On Thu, Nov 10, 2022 at 8:23 AM Peter Stuge  wrote:
> Julius Werner wrote:
> > I think only coreboot should ever generate this format. I'm certainly
> > not trying to make a case for "we should get other firmware projects
> > to adopt coreboot tables" here, I don't think that makes any sense.
>
> I think it makes some sense because coreboot does what we consider best.
> Of course we want others to reuse the very best parts of that.
>
> But I also recognize that it is unlikely to happen, so to improve the
> situation overall (ie. also outside coreboot) I want to consider FDT
> in coreboot.
> > I'm trying to make the case for "unifying payload handoff formats
> > between different firmwares is generally not a useful idea
>
> MBR called to disagree.
> > and not worth the cost".
>
> This is the only interesting discussion.
> > I understand that some people here want to instrumentalize coreboot
> > to try to force Intel to change course on its USF specification, and
> > honestly I don't really care about that one way or the other,
>
> I find that silly polemic. I see this not as instrumentalizing
> coreboot but as coreboot accomodating Intel a little while also
> getting benefit - of course with the longer term goal of Intel
> and coreboot becoming a better match.
>
> You don't have to care about USF or Intel but do consider that
> there probably would be no coreboot community within industry
> were coreboot not to add value to Intel hardware.
>
> Our community is obviously not in danger over this but the better
> Intel and coreboot go together the more opportunities for really
> good solutions we may have in the future.
>
> If coreboot doesn't push Intel then Intel will push coreboot (not
> neccessarily on purpose, just through inertia). You may want to
> ignore that for now but the coreboot will be dealing with the outcome
> for some time and now is when we can act.
> > I'm just asking to please not let the fallout of that negatively
> > affect those of us who still just care about using coreboot as a
> > firmware and not as a political cudgel.
>
> I understand if you and others want to ignore politics, I did too for
> a long time.
>
> That said, I wouldn't worry about negative effects. I certainly have
> no interest to remove coreboot tables nor do I see any reason to do
> so nor have I seen anyone mention anything in that direction.
>
> Do not make up strawmen.
> > > I don't think FDT is less suited than coreboot tables where it
> > > matters the most - simplicity on byte level.
>
> ..
> > I'm a bit confused by the hate for CBOR here to be honest,
>
> ..
> > I don't think parsing one of them without RAM would be much more
> > cumbersome than the other.
>
> ..
> > FDT is a tokenized hierarchical key-value store with string keys (and
> > a ton of extra quirks), whereas the coreboot tables are just a tagged
> > list of C structures. That's an order of magnitude of difference in
> > simplicity (and binary size).
>
> A RAM-less FDT parser probably doesn't need to load the entire tree
> but merely find one value at a time, perhaps without state between
> values. The complexity of that in an FDT is low since FDT is flat and
> has simple primitives. CBOR, well..
>
> --8<-- RFC8949
>    The initial byte of each encoded data item contains both information
>    about the major type (the high-order 3 bits, described in
>    Section 3.1) and additional information (the low-order 5 bits).  With
>    a few exceptions, the additional information's value describes how to
>    load an unsigned integer "argument":
>
> Less than 24:  The argument's value is the value of the additional
>       information.
>
> 24, 25, 26, or 27:  The argument's value is held in the following 1,
>       2, 4, or 8 bytes, respectively, in network byte order.  For major
>       type 7 and additional information value 25, 26, 27, these bytes
>       are not used as an integer argument, but as a floating-point value
>       (see Section 3.3).
>
> 28, 29, 30:  These values are reserved for future additions to the
>       CBOR format.  In the present version of CBOR, the encoded item is
>       not well-formed.
>
> 31:  No argument value is derived.  If the major type is 0, 1, or 6,
>       the encoded item is not well-formed.  For major types 2 to 5, the
>       item's length is indefinite, and for major type 7, the byte does
>       not constitute a data item at all but terminates an indefinite-
>       length item; all are described in Section 3.2.
> -->8--
>
> This is bizarrely complex for storing a simple scalar.
> > what do you still need a handoff for at all?
> > Just to parse the mainboard vendor string?
>
> Memory map at a minimum.
> > I disagree with abandoning coreboot tables or forcing FDT generator code
> > into coreboot images for those who aren't interested in this project.
>
> So a way forward where you can ignore FDT is for coreboot tables to
> stay as-is, and a Kconfig option to make coreboot use its coreboot
> tables to generate an FDT.
> > I think keeping these things side-by-side in the code base with a
> > Kconfig to switch between them
>
> Like Arthur I don't appreciate the "side-by-side" narrative, nor an
> expectation for two independent data sets.
> > > > I'd like to make sure that we can agree to keep coreboot tables as
> > > > a first-class citizen now and in the future, and don't force a new
> > > > format on anyone.
> > >
> > > I don't think anyone proposed to not generate coreboot tables anymore.
> >
> > I'm talking about where this is eventually supposed to be headed, not
> > just the first experimental patch.
>
> This is supposed to be headed to additional benefit for both coreboot
> and USF (users). Others will use coreboot in different ways, e.g.
> without FDT.
> > I think we can all agree that there's no point in having two
> > firmware handoff structures that both need to be parsed to get the
> > full picture,
>
> Noone suggested anything like that. Don't waste our time.
> > and since the argument for coreboot tables is mostly simplicity /
> > lean code it's not a solution for that side to just generate both.
>
> If you mean "it's not a solution for coreboot to just generate both"
> then I ask why not? Two things can be true at the same time: coreboot
> tables are simple to generate and the code to generate FDT from
> coreboot tables is also simple.
> > > > Well, Arthur also admitted that none of these points are really
> > > > relevant to coreboot's payload handoff, ;) which I fully agree with.
> > >
> > > Currently that may be true, but reflection would pretty much be
> > > required for a new payload for manual parameter tweaking during boot.
> >
> > Yeah, but what's the use case?
>
> There are several, both outside and within the boot process.
>
> FDT can be meaningfully persisted and so be useful both before and after
> boot.
> > When have you ever had a situation where you'd want to edit
> > coreboot table structures at runtime?
>
> You can also consider FDT in coreboot as plumbing to enable new
> situations that were best avoided with coreboot tables, then of
> course noone ever wanted to edit them. That doesn't mean that noone
> would ever want to edit firmware handoff information, especially if
> we also consider FSP.
>
> Please be a bit more imaginative here.
> > it would still not be very hard to write such a thing for coreboot
> > tables
>
> Of course. One goal is to have more freedom than that allows, or at a
> very minimum fill a namespace not (exclusively) defined by coreboot
> or Intel.
> > > As I understand it, cbtable member alignment is currently undefined.
> > > That matters when building firmware and payload with different
>
> compilers.
> > > "Just use the coreboot compiler" is not a good answer.
> >
> > Can we please stop perpetuating this myth? It's just wrong, like I
> > already explained in my first email here.
>
> I don't think it's wrong. I used to, but not anymore.
>
> This is not about 64-bit values which you mentioned in your first
> email, this is about struct member offsets in the struct.
>
> The structs are not packed so the compiler decides.
>
> This is why both Nico and I are so strong proponents of explicit
> serialization of the tables. (I'd also like to create new tags for
> coreboot tables with packed values but that's another matter.)
> > > > ¹ I wouldn't mind that btw., replacing ACPI.
> > >
> > > That would be fantastic! But maybe it's more of a stretch goal. :)
> >
> > FWIW I think this is a completely separate discussion and has nothing
> > to do with what we should do for payload handoff (again, just like
> > with the FSP thing, kernel handoff is a completely different thing
> > that would have very little to share with it).
>
> If all handoff interfaces were to use a single format (this would be
> great) then obviously it can be a single data set, which would make
> life simpler for a lot of different people.
> > (and not "soft deprecated" in favor of FDT,
>
> Since this your narrative is baseless it makes it look like you are
> intentionally acting in bad faith. That's not a good look. Be careful.
> > or have the FDT stuff seep into something that can not be compiled out).
>
> Like vboot has been for maybe a decade now yes, I agree strongly that
> this is important to avoid.
> > without stepping on each other's toes.
>
> I wouldn't worry.
>
> //Peter
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Julius Werner]

> > I'm trying to make the case for "unifying payload handoff formats
> > between different firmwares is generally not a useful idea
>
> MBR called to disagree.

What's MBR in this context -- Master Boot Record? Kernel handoff and
firmware payload handoff are two entirely different ballparks here.
> That said, I wouldn't worry about negative effects. I certainly have
> no interest to remove coreboot tables nor do I see any reason to do
> so nor have I seen anyone mention anything in that direction.
>
> Do not make up strawmen.

I'm gonna try to ignore this and all the other slight jabs and
personal attacks afterwards because I don't think it's productive or
going to help us find a compromise. Throughout the whole rest of this
thread it seems like everyone had a friendly and respectful discussion
despite our technical differences in opinion, let's please try to keep
it that way (especially since it seems like we have basically found a
compromise that we can all live with already anyway).

Just so you understand where I'm coming from, Arthur's very first
comment on the original changelist asked whether it is "a good idea to
use FDT and eventually replace coreboot tables", and I think several
others (maybe not you) had still hinted at that as a long-term goal a
few times. So I'm not trying to "waste your time" with "strawmen", I'm
just trying to make sure we're on the same page on this because it is
important to me.
> A RAM-less FDT parser probably doesn't need to load the entire tree
> but merely find one value at a time, perhaps without state between
> values. The complexity of that in an FDT is low since FDT is flat and
> has simple primitives. CBOR, well..
>
> [...]
>
> This is bizarrely complex for storing a simple scalar.

It just takes an AND and a switch to decode? I honestly think you're
overestimating this. I've written a CBOR parser before (and most parts
of an FDT parser, for that matter) and it's really not a big deal. In
fact looking for a single value might be easier in CBOR because the
data is entirely local, whereas with FDT you at least need to juggle
the separate string table. (Then again FDT makes it a little easier to
skip over unrelated parts of the tree when looking for a single item,
so there are certainly trade-offs)
> > I disagree with abandoning coreboot tables or forcing FDT generator code
> > into coreboot images for those who aren't interested in this project.
>
> So a way forward where you can ignore FDT is for coreboot tables to
> stay as-is, and a Kconfig option to make coreboot use its coreboot
> tables to generate an FDT.

Yes, that approach is fine by me. It's not as efficient for your case
as generating the FDT directly, of course, but if you want to do it
this way I won't complain.
> > and since the argument for coreboot tables is mostly simplicity /
> > lean code it's not a solution for that side to just generate both.
>
> If you mean "it's not a solution for coreboot to just generate both"
> then I ask why not? Two things can be true at the same time: coreboot
> tables are simple to generate and the code to generate FDT from
> coreboot tables is also simple.

I meant that it wouldn't be a solution to do it unconditionally,
because my main argument against FDT is that I don't want that extra
bloat for configurations that don't use the FDT. If we have a Kconfig
to decide between coreboot-table-only and coreboot-table+FDT, that
works for me.
> This is not about 64-bit values which you mentioned in your first
> email, this is about struct member offsets in the struct.
>
> The structs are not packed so the compiler decides.
>
> This is why both Nico and I are so strong proponents of explicit
> serialization of the tables. (I'd also like to create new tags for
> coreboot tables with packed values but that's another matter.)

There is no padding in coreboot tables. We intentionally arrange
members such that they don't leave padding holes and insert explicit
`unused` members where necessary. This is a common pattern in a lot of
structures where exact data layout is important (e.g. those
representing hardware registers as well). Using
__attribute__((packed)) is usually a bad idea and should be avoided
unless really necessary (i.e. in cases where individual members need
to be explicitly misaligned), because it forces the compiler to assume
that all loads may be misaligned and leads to bad code generation on
architectures that can't tolerate unaligned accesses in all cases.
> > FWIW I think this is a completely separate discussion and has nothing
> > to do with what we should do for payload handoff (again, just like
> > with the FSP thing, kernel handoff is a completely different thing
> > that would have very little to share with it).
>
> If all handoff interfaces were to use a single format (this would be
> great) then obviously it can be a single data set, which would make
> life simpler for a lot of different people.

The information that is being passed from coreboot to payloads and
from the payload to kernels is so fundamentally different that there
would be very little overlap (maybe not in the cases where the memory
table is the only thing you have in your payload handoff, but
certainly in those that actually have enough records to make it worth
worrying about the format). Besides, kernel handoff formats tend to
differ widely between architectures which I don't think is something
we want to duplicate at the payload boundary.

[Felix Held]

Hi Arthur,
> You never know whether the FSP binary will match a given header

slightly off-topic for this thread, but it's possible to check for 
mismatches between the expected FSP binary versions and the actual FSP 
binary version during runtime by implementing soc_validate_fspm_header 
that compares the version information from the FSP binary to some 
version defines in the FSP headers. See [1] for the implementation used 
by the AMD SoCs that use FSP.

Regards,
Felix

[1] 
https://review.coreboot.org/plugins/gitiles/coreboot/+/aab7f0490419c50a1bcb5...