On Tue, Apr 28, 2020 at 04:16:59PM +0200, Paul Menzel wrote:
> Dear coreboot folks,
>
>
> Despite ever increasing flash ROM chip sizes, small images are still desired
> for faster boot times, faster flash times, and more space for
> payloads, which is sometimes needed for adding several payloads
> (including GRUB/TianoCore) or Linux payloads.
>
> Jacob Garber did great work to achieve this goal by enabling Link Time
> Optimization (LTO) for coreboot [1] and libpayload [2]. While doing
> this, he also found and fixed several bugs in the code base.
>
> Currently, it fails for AMD AGESA boards due handling of illegal globals.
>
> If somebody has a solution for that, that’d be great.
>
> It’d be great, if more people could test this, on your boards, and report
> back.
>
> I propose, to submit the change-sets before the next release, and to enable
> LTO for libpayload by default, and to disable it for coreboot by default.
>
> Big thanks again to Jacob for doing this. (My attempt doing this for GRUB
> failed. ;-))
>
>
> Kind regards,
>
> Paul
>
>
> [1]: https://review.coreboot.org/c/coreboot/+/38989
> [2]: https://review.coreboot.org/c/coreboot/+/38291
Hey Paul, thanks for the encouraging words. :) I tidied up the patches today
and think they are ready for review. Not all targets compile yet, but all of
the LTO framework is there so people can test it out with their boards. For
example, the Thinkpad T500 boots successfully, and there is about a 10%
reduction in stage size and compilation time.
Right now I think it's safest to leave LTO disabled for libpayload and
coreboot. GCC 9 is the first version where LTO is considered "production
ready", and until we have that merged I think LTO should be considered
experimental. Once people have had some time to try it out and work out any
bugs we can move to making it the default in some cases.
Cheers,
Jacob
Hi community
I picked up the good work done by Jacob Garber to implement link time
optimization. The size gains are very substantial, with sometimes up to 15%
reduction in binary size! Besides the size benefit it can make us a bit
more lazy programmers: for instance we often put static inline functions in
headers guarded by preprocessing depending on Kconfig options to not
generate code we won't use. With LTO that can be relaxed a bit and the
condition to do nothing can come from a separate linked object (at least I
presume, I haven't tested).
So to do LTO, both with clang and GCC, you use the compiler as a linker
frontend, rather than invoke the linker directly. Object files that are
optionally generated with the -flto flag to generate LTO optimized
binaries. See https://review.coreboot.org/c/coreboot/+/40811
Jacob's patches dealt with GCC. I toyed around with clang. The clang linker
frontend only works with LTO if using the GOLD or LLD linker. GCC can use
the BFD linker we currently use, so that's an easy transition. Both those
linkers, GOLD and LLD are less capable at parsing linker scripts than the
BFD linker. Especially for the x86 bootblock, in which we do a lot of
stuff, like ID, FIT pointer, ECFW pointer, early 16 bit code, ..., is
complicated and those linkers fail on the arithmetics doing all that
optimized linking.
So my question would be, whether we want to support non BFD linkers and
therefore not support all the magic in linker scripts we currently have. At
most we are a few bytes less optimal than possible by for instance setting
a good hardcoded size and offset of the early 16bit code rather than have
the linker script optimize that. As a side note, LTO size gainz are bigger
than what would be lost, by a large margin.
https://review.coreboot.org/c/coreboot/+/80724/1/src/arch/x86/bootblock.ld
is a crude example of how I got lld to be happy with the script.
If non BFD linkers are acceptable, how do you propose we deal with it? When
not using LTO you could in principle use a different linker of your
choosing. Should we just move to lld to make sure CI is happy about linker
scripts? Should it be an option, but then we won?t have CI being able to
test it except on a few boards put in configs/ ? Maybe have both, but then
change the default once LLD is known to work well enough? Sidenote: LLD is
faster than BFD but linking is not an expensive step in coreboot anyway.
Other more typical linker scripts like the ones you find on ARM and ARM64
platforms work just fine. There are some subtle differences e.g.
https://review.coreboot.org/c/coreboot/+/80735 where the heap needs to be
declared to not be loaded.
There are a few issues though. LLD does not like the RISCV relocation
symbols of libgcc.a... I read upstream bugreports about this issue, so
maybe that will resolve itself.
Same questions with LTO and CI. Should it be an option (IMO yes, as that's
what both Linux and u-boot do) and which default settings do we want. For
CI reasons LTO is a bit more strict and therefore useful, as the compiler
frontend does throw more warnings than the plain linker when linking.
What are your thoughts? Are there some caveats about linkers and LTO that
are worth knowing, before moving forward?
Kind regards
Arthur
Dear coreboot folks,
Despite ever increasing flash ROM chip sizes, small images are still
desired for faster boot times, faster flash times, and more space for
payloads, which is sometimes needed for adding several payloads
(including GRUB/TianoCore) or Linux payloads.
Jacob Garber did great work to achieve this goal by enabling Link Time
Optimization (LTO) for coreboot [1] and libpayload [2]. While doing
this, he also found and fixed several bugs in the code base.
Currently, it fails for AMD AGESA boards due handling of illegal globals.
> Yes, this is a current limitation of LTO right now. Because the
> object files are all lumped together into a single unit, all
> information about where the symbols came from is lost, so
> EXCLUDE_FILE is unable of excluding the AGESA objects from the
> illegal_globals check. Tracing where a symbol came from has been
> implemented in LLVM [0], but I'm not sure if it's on the roadmap for
> GCC. For now it's probably best to disable LTO when compiling AGESA.
>
> [0]: https://llvm.org/devmtg/2017-10/slides/LTOLinkerScriptsEdlerVonKoch.pdf
If somebody has a solution for that, that’d be great.
It’d be great, if more people could test this, on your boards, and
report back.
I propose, to submit the change-sets before the next release, and to
enable LTO for libpayload by default, and to disable it for coreboot by
default.
Big thanks again to Jacob for doing this. (My attempt doing this for
GRUB failed. ;-))
Kind regards,
Paul
[1]: https://review.coreboot.org/c/coreboot/+/38989
[2]: https://review.coreboot.org/c/coreboot/+/38291
Hi Julius and Nico,
Thanks for the feedback!
Did you find out any particular (magic) construct we are currently using
> that fails? Or is it the overall complexity of the script?
>
It trips on some arithmetics but I don't fully understand it yet, so my
attempt was trial and error to get something linking and booting.
Btw. can clang+lld with LTO still link against GCC objects, e.g.
> libgfxinit?
Linking is possible, but it cannot optimize that ofc. btw sometimes it's
necessary to skip LTO on some specific C code like the arm eabi_compat.c .
As a sidenote maybe https://github.com/AdaCore/gnat-llvm is an interesting
route to have LTO in the clang with libgfxinit combo?
Do you have a rough list of the types of things that the LLD linker
> cannot deal with (e.g. there seems to be something about not using a
> symbol before it was defined, like with BOOTBLOCK_TOP, but then it
> doesn't seem to apply everywhere, e.g. for ID_SECTION it still seems
> to work?), so we can get an idea what kind of limitations we'd be
> accepting here for both current and future linker scripting?
>
I'm not sure yet why LLD isn't happy about some arithmetics in the linker
script. I'll investigate to get a clearer picture.
I
> wouldn't mind some rewrites to the x86 bootblock script in general,
> since some of it honestly seems unnecessarily convoluted anyway, but
> it's more concerning if you need to drop features (like commenting out
> all of those asserts at the end) when there's no way to make something
> similar work with LLD.
The asserts don't work because of the LTO and LLD combination. The
_bootblock / _ebootblock symbols get optimized away and are somehow set to
0. Referring to them inside the code would fix that.
https://review.coreboot.org/c/coreboot/+/71871 is also a way to deal with
it.
Also, are you sure that all the Arm boards are fine? Did you do a full
> abuild and then also compare the images (with BUILD_TIMELESS) to make
> sure the layouts didn't actually shift? We do a bunch of complicated
> things in our linker scripts, I'm actually surprised that LLD would be
> fine with everything besides the x86 bootblock (it didn't use to be at
> all a couple of years ago, but I guess they may have improved it).
>
I only played with qemu on arm and arm64 and those still worked. Some more
in depth comparison of the elf output is indeed needed. For instance with
x86 stages it tripped the cbfstool assertion that loadable sections need to
be consecutive.
I think the other big question here is: why do we care about clang at
> all? If GCC can do LTO with BFD, why don't we just stick with that? My
> understanding was that people just added clang support to coreboot
> "for fun" to see if it was possible, and we said okay as long as you
> can do it without having to break any code. But now if we do need to
> make meaningful code changes to support it, should we reexamine why we
> want it at all? Is anyone actually using it in a production use case
> (and if so, why)?
I personally have clang set as the default compiler on my system using
site-local/Kconfig. I prefer its error messages. It also generates a bit
different errors/warnings as GCC, so that's always nice in CI. Clang trades
blows with GCC on code size. Especially with LTO clang can sometimes result
in 10% smaller binaries than GCC LTO binaries. Last time I checked Linux
only supports LTO with clang (that might not be true anymore), although I'm
not so sure why. If newer language support like rust or zig is desirable in
the future, then LTO with clang will work more easily, as the same LLVM IR
is used. One cool feature of clang is that it can do reflection on C
structs with compiler builtins:
https://review.coreboot.org/c/coreboot/+/72460 . So in my opinion it's a
tooling option worth exploring. "For fun" often precedes production use :-)
.
Arthur
On Sat, Feb 24, 2024 at 10:55 PM Nico Huber <nico.h(a)gmx.de> wrote:
> Hi Arthur,
>
> this sounds very interesting.
>
> On 23.02.24 17:47, Arthur Heymans wrote:
> > So my question would be, whether we want to support non BFD linkers and
> > therefore not support all the magic in linker scripts we currently have.
>
> Did you find out any particular (magic) construct we are currently using
> that fails? Or is it the overall complexity of the script?
>
> I was already wondering lately if we shouldn't split the complex x86
> linker script for different use cases (e.g. native, FSP, etc.). If we
> had three scripts instead of one, we would sometimes have to make the
> same change in multiple files. But, OTOH, I don't think we touch the
> linker scripts that much. And every time I look into the x86 one, it
> seems hard to find top and bottom.
>
> > If non BFD linkers are acceptable, how do you propose we deal with it?
> When
> > not using LTO you could in principle use a different linker of your
> > choosing. Should we just move to lld to make sure CI is happy about
> linker
> > scripts? Should it be an option, but then we won?t have CI being able to
> > test it except on a few boards put in configs/ ? Maybe have both, but
> then
> > change the default once LLD is known to work well enough? Sidenote: LLD
> is
> > faster than BFD but linking is not an expensive step in coreboot anyway.
>
> I wouldn't mind switching to LLD. Would prefer that we focus on a
> single linker, though. There is some soothing feeling when one knows
> that everybody is using the same toolchain, and chances that bugs are
> discovered early are higher.
>
> Btw. can clang+lld with LTO still link against GCC objects, e.g.
> libgfxinit?
>
> Nico
>
>
Hi Arthur,
this sounds very interesting.
On 23.02.24 17:47, Arthur Heymans wrote:
> So my question would be, whether we want to support non BFD linkers and
> therefore not support all the magic in linker scripts we currently have.
Did you find out any particular (magic) construct we are currently using
that fails? Or is it the overall complexity of the script?
I was already wondering lately if we shouldn't split the complex x86
linker script for different use cases (e.g. native, FSP, etc.). If we
had three scripts instead of one, we would sometimes have to make the
same change in multiple files. But, OTOH, I don't think we touch the
linker scripts that much. And every time I look into the x86 one, it
seems hard to find top and bottom.
> If non BFD linkers are acceptable, how do you propose we deal with it? When
> not using LTO you could in principle use a different linker of your
> choosing. Should we just move to lld to make sure CI is happy about linker
> scripts? Should it be an option, but then we won?t have CI being able to
> test it except on a few boards put in configs/ ? Maybe have both, but then
> change the default once LLD is known to work well enough? Sidenote: LLD is
> faster than BFD but linking is not an expensive step in coreboot anyway.
I wouldn't mind switching to LLD. Would prefer that we focus on a
single linker, though. There is some soothing feeling when one knows
that everybody is using the same toolchain, and chances that bugs are
discovered early are higher.
Btw. can clang+lld with LTO still link against GCC objects, e.g.
libgfxinit?
Nico
Hi Arthur,
First of all, thanks a lot for putting all this work into getting LTO
working. The benefits really seem promising!
Do you have a rough list of the types of things that the LLD linker
cannot deal with (e.g. there seems to be something about not using a
symbol before it was defined, like with BOOTBLOCK_TOP, but then it
doesn't seem to apply everywhere, e.g. for ID_SECTION it still seems
to work?), so we can get an idea what kind of limitations we'd be
accepting here for both current and future linker scripting? I
wouldn't mind some rewrites to the x86 bootblock script in general,
since some of it honestly seems unnecessarily convoluted anyway, but
it's more concerning if you need to drop features (like commenting out
all of those asserts at the end) when there's no way to make something
similar work with LLD.
Also, are you sure that all the Arm boards are fine? Did you do a full
abuild and then also compare the images (with BUILD_TIMELESS) to make
sure the layouts didn't actually shift? We do a bunch of complicated
things in our linker scripts, I'm actually surprised that LLD would be
fine with everything besides the x86 bootblock (it didn't use to be at
all a couple of years ago, but I guess they may have improved it).
I think the other big question here is: why do we care about clang at
all? If GCC can do LTO with BFD, why don't we just stick with that? My
understanding was that people just added clang support to coreboot
"for fun" to see if it was possible, and we said okay as long as you
can do it without having to break any code. But now if we do need to
make meaningful code changes to support it, should we reexamine why we
want it at all? Is anyone actually using it in a production use case
(and if so, why)?
# 2024-03-20 - coreboot Leadership Meeting Minutes
## Attendees
Martin Roth, Felix Held, Felix Singer, Jay Talbott, Matt DeVillier, Nicholas Chin, David Hendricks,
Arthur Heymans, Mina Asante, Julius Werner,Christian Walter, Max Brune, Jon Murphy, Karthik,
Patrick Georgi.
## Announcements
* FOSSY conference: August 1-4 2024 in Portland, Oregon, USA
[https://sfconservancy.org/fossy/].
* OSFC will be in Bochum Germany - September 3-5 - [https://www.osfc.io/].
## Open Action Items
* 2024-03-20
* [Open] Martin: Set up a new meeting link for the next leadership meeting.
* [Open] Martin: Add a note to the gerrit guidelines to email the leadership.
* 2024-03-06
* [Open] Martin: To update documentation on gerrit contributing guidelines.
* https://doc.coreboot.org/contributing/index.html
* 2024-01-10
* [Open] Werner: Push patch based on https://ticket.coreboot.org/issues/522
* Nico: https://review.coreboot.org/q/topic:enforce_region_api
* Open Daniel: Look at how we want to localize (non console) strings for coreboot. Long term project.
## Minutes
### [Martin] BigBlueButton
* The invite for this meeting still has the google chat link on it. We should have Patrick cancel his invite and set up a new one.
* The BBB version that the SFC offers is from 2021 and has issues on mobile. There could also be unfixed CFEs (speculation).
* Do we want to switch back to Google chat until the SFC can update to a newer version? Martin can set up an open invite for up to 100 users. $20 per month (I believe).
* Can also use OSFF - 250 people, record, etc - everyone can join.
* Decision: switch back to Google Meet
### [Martin] Adding Max Brune and Jon Murphy as core developers.
* | Name | Patches | Reviews | Comments |First Patch |
|------------------|---------|---------|----------|------------|
| Maximilian Brune | 67 | 88 | 43 | 2022-05-16 |
| Jon Murphy | 115 | 130 | 29 | 2022-02-24 |
* Others who could be considered, but we’d like to see more community involvement, reviews, and/or comments from before considering them.
* Frans Hendriks
* Bora Guvendik
* Keith Hui
* Maulik Vaghela
* Naresh Solanki
* Tyler Wang
* Varshit B Pandya
* Kapil Porwal
* If anyone feels that I’m mistaken about people in either group, please reach out to me or the leadership about these people.
* [FelixH] We should only look at the number of commits and things from the past 2 years instead of over the entire span that they’ve been working on the project.
* [Martin] I can recalculate who other than Max & Jon could be considered.
* Decision: Only look at the past 2 years?
* [Felix] How should we react when people don’t do a good job submitting code?
* [Martin] I think that the leadership should talk with the user.
* [MattD] No formal policy - let the leadership handle it.
* TODO: Add a note to the gerrit guidelines to email the leadership
### [Martin] RFC: Link Time Optimization [LTO]
* Jacob Garber pushed patches a few years ago to get GCC working - Arthur took these over. See Jacob’s initial email:
[https://mail.coreboot.org/hyperkitty/list/coreboot@coreboot.org/message/JII…]
* Working on adding clang - works differently & only works with specific linkers.
[https://mail.coreboot.org/hyperkitty/search?q=lto&page=1&mlist=coreboot%40c…]
* Do we want to make this a default?
* [Martin] We can decide that after testing.
* [Arthur] ARM (not arm64) is broken - maybe the toolchain is broken. Arm builds don’t even complete.
* Would there be a reason to keep non-LTO?
* Maybe to debug issues and see if they’re caused by LTO.
* To add LLVM with LTO we need to add LLD as part of the project when building LLVM.
* How does this affect our linker scripts?
* LLD doesn’t understand “=.” but setting a variable works.
### [Martin] Bugs
* Large number of outstanding bugs - how can we close these? No good solution?
* Reduce barriers
* Introducing single sign-on would help.
* Having the email to the mailing list seems to help.
* Being able to reply to the email and have it show up in the bug would help.
* Why is tracker.coreboot.org down right now?
* Is there any progress integrating with Gerrit?
* Felix would like to move away from redmine. Highly depends on plugins and there isn’t good modern
functionality. Removed support from open-connect - so people can’t use
* Propose we switch to forgejo for bug tracking.
* Can we use our current logins from gerrit, or does it require a new login? This is the biggest
issue we’ve heard from users with redmine.
* Does this help integration with gerrit?
* What are we looking for?
* Links from bugids from gerrit to the ticket.
* Yes? I think we can do this just with gerrit. The same as we do with the CB: links.
* Decision: Let’s look at Forgejo and see
### [Martin] Coverity issues
* Current open issues spreadsheet
* [chart of issues by area]
(https://docs.google.com/spreadsheets/d/e/2PACX-1vRYUlOLQTysbdPQoOKaCnRN_K4C…
6gATqtvgDlzSl1S4DaijcTyF7yAAs3qm/pubchart?oid=502535263&format=interactive).
* [chart of issues by type]
(https://docs.google.com/spreadsheets/d/e/2PACX-1vRYUlOLQTysbdPQoOKaCnRN_K4C…
6gATqtvgDlzSl1S4DaijcTyF7yAAs3qm/pubchart?oid=1369215095&format=interactive).
* Link to the coreboot project coverity:[https://scan.coverity.com/projects/coreboot].
* As an alternative to coverity, Felix is adding a code-checker Docker container, and we’ll be moving to it from the current scan-build website. This is not a replacement for coverity, just a nicer interface to the current scan-build that we’re already doing.
* [https://codechecker.readthedocs.io/en/latest/].
* coreboot.org/scan-build is currently broken. :(
# Next meeting
* April 3, 2024.
* [coreboot Calendar](https://coreboot.org/calendar.html).
# Notice
* Decisions shown here are not necessarily final, and are based
on the current information available. If there are questions or comments
about decisions made, or additional information to present, please put
it on the leadership meeting agenda and show up if possible to discuss it.
Of course items may also be discussed on the mailing list, but as it's
difficult to interpret tone over email, controversial topics frequently
do not have good progress in those discussions. For particularly
difficult issues, it may be best to try to schedule another meeting.
# coreboot leadership meeting minutes
[2024-03-20](https://docs.google.com/document/d/1NRXqXcLBp5pFkHiJbrLdv3Spqh1…
1).
The LTO patches seem to both compile and work/boot for me on the p2b.
I built it both on a debian sid x86_64 system and on the gentoo i686
setup I currently have for the p2b, both with the coreboot
crossgcc-i386 toolchain.
It looks like it uses the system linker though (or something similiar,
I don't remember exactly), at least according to the executable path
it shows in the build log. I'm not sure if that's actually what's
desired or if I'm just misinterpreting something.
It still looks like it needs more test reports yet, though I guess I'm
not helping either by not commenting on gerrit.
Branden
On 6/1/21, Angel Pons <th3fanbus(a)gmail.com> wrote:
> Hi Branden, list,
>
> On Tue, Jun 1, 2021 at 2:10 AM Branden Waldner <scruffy99(a)gmail.com> wrote:
>>
>> The LTO patches seem to both compile and work/boot for me on the p2b.
>>
>> I built it both on a debian sid x86_64 system and on the gentoo i686
>> setup I currently have for the p2b, both with the coreboot
>> crossgcc-i386 toolchain.
>
> That's great to hear. I hope you didn't need to build crossgcc-i386 on
> the P2B, though! :P
Well, it's got a gentoo install that has to build it's own updates,
including the system compiler, as well as crossgcc-i386. I have to
leave it to run for quite a while to do updates or build a new first
of crossgcc-i386 on it. It's some pretty tough hardware though, it
manages to handle it just fine, even though it is _really_ old for
computer hardware at this point.
It would probably be a lot faster if I could get a better cpu (w/
proper voltage handling) and the proper ram for it. I do have an ssd
attached via a pci sata adapter (which is a bottleneck), but didn't
really notice much of an improvement.
I guess some people would use (like?) a system like it for retro
gaming/computing, but I've never really found a use for it besides
testing coreboot on it.
>> It looks like it uses the system linker though (or something similiar,
>> I don't remember exactly), at least according to the executable path
>> it shows in the build log. I'm not sure if that's actually what's
>> desired or if I'm just misinterpreting something.
>
> It might be intentional, but it's not desired: using the system
> toolchain to build coreboot will wreak havoc when cross-compiling.
Thankfully, I misunderstood what I was seeing in the logs, it was just
while it was building tools that it was using the system toolchain.
But, it was using LTO settings for them, which I wasn't expecting. I
was assuming it would only use them for actually building the rom, not
the utils, but I guess I don't see why not.
>> It still looks like it needs more test reports yet, though I guess I'm
>> not helping either by not commenting on gerrit.
>
> Yes, it needs more test reports. It would be great if you could
> comment on the Gerrit change, so as to keep all test reports in one
> place.
I'll try to get around to commenting on the Gerrit change set yet.
Branden
Hi Branden, list,
On Tue, Jun 1, 2021 at 2:10 AM Branden Waldner <scruffy99(a)gmail.com> wrote:
>
> The LTO patches seem to both compile and work/boot for me on the p2b.
>
> I built it both on a debian sid x86_64 system and on the gentoo i686
> setup I currently have for the p2b, both with the coreboot
> crossgcc-i386 toolchain.
That's great to hear. I hope you didn't need to build crossgcc-i386 on
the P2B, though! :P
> It looks like it uses the system linker though (or something similiar,
> I don't remember exactly), at least according to the executable path
> it shows in the build log. I'm not sure if that's actually what's
> desired or if I'm just misinterpreting something.
It might be intentional, but it's not desired: using the system
toolchain to build coreboot will wreak havoc when cross-compiling.
> It still looks like it needs more test reports yet, though I guess I'm
> not helping either by not commenting on gerrit.
Yes, it needs more test reports. It would be great if you could
comment on the Gerrit change, so as to keep all test reports in one
place.
> Branden
Best regards,
Angel
