A case for branching AGESA

[Alex G.]

AGESA is a very heavy beast, at over one and a half million lines of
code. Although contributions to the AGESA codebase (boards and
supporting chipset/cpu code) have slowed down significantly over the
years, we still rebuild this behemoth for every version of a patch.

This causes quite a bit of strain on our build infrastructure, and even
with an army of four state-of-the art build machines, some people (wink,
wink, Martin) are concerned with long patch trains and verification
times. I can't speak for other machines, but after replacement hardware
costs, flying in, and a lost day of work, I've realized that maintaining
this infrastructure is not cheap.

This made me think if we can make more efficient use of the
infrastructure we have available today, and allow builders to use
less-powerful machines without increased latency.

I've run a few timing tests without AGESA being built, and took
historical data to compare build times for when AGESA is included in the
build.
> From averaging build times on nuclearis2, with a primed page cache and

ccache a compete abuild run takes on average, five to six minutes. A
build without a useful ccache cache takes around fifteen to twenty
minutes (I don't have a reliable way of timing this, hence the high
variance).

Now, if we build a tree without AGESA, we get the following:

First run:
[mrnuke@nuclearis2 coreboot]$ ccache -C #clear ccache
[mrnuke@nuclearis2 coreboot]$ time CPUS=12 make what-jenkins-does
real    6m46.778s

Second run:
[mrnuke@nuclearis2 coreboot]$ time CPUS=12 make what-jenkins-does
real    2m7.889s

7 min vs 20 min on empty ccache, and 2 min vs 6 min on primed ccache.
Those are speedups of 2x to 3x. Also note that the local builds were not
done on a tmpfs, like the actual jenkins builds. I expect similar
speedups on the other builders.

The obvious solution is to separate AGESA into its branch. Patch trains
from google and other contributions to non-AGESA code gain a 2x to 3x
speedup in server time, while users of AGESA can continue to contribute
and work on the codebase.

Of course, the AGESA branch would also have the non-AGESA boards and
code trimmed so as to make efficient use of the server resources. There
is some common code, and overlap between the two code paths, but it
doesn't seem to be anywhere near as significant as the behemoth that
AGESA is.

Since 4.2 was just released, we can do this today without much fanare.
This also allows other people to donate CPU cycles without needing to
own over-priced server-class machines, and without increasing the
verification latency (compared to the one today).

To anyone saying that putting code in a branch is a death sentence,
please don't treat this as TL;DR and read the above paragraphs.

Alex

[Patrick Georgi]

Alex,

while I'm happy that you're providing hard numbers, I completely
disagree with your conclusion.
A much better approach, that doesn't rely on figuring out which parts
happen to be inconvenient to a subset of our developers, is to build
an analysis of the impact of a commit.

For global code changes, we _want_ everything to be build (and boot)
tested, while a change in src/mainboard/$foo/$bar, in principle, only
needs to build that single mainboard. Such an approach requires, and
here comes the (somewhat) hard part, that the scope of a change can be
determined reliably.

The speed-up of such an approach is close to optimal.

2015-11-01 1:36 GMT+01:00 Alex G. :
> AGESA is a very heavy beast, at over one and a half million lines of
> code.

Just a note: the only reason why current Intel fares better is because
they're shipping binaries that we can't even distribute in
3rdparty/blobs. Otherwise that looks pretty similar.
> 7 min vs 20 min on empty ccache, and 2 min vs 6 min on primed ccache.
> Those are speedups of 2x to 3x.

Thank you for doing the measurements.
> The obvious solution is to separate AGESA into its branch.

It may be obvious to you, but there were enough good arguments brought
up by a many good people that branches for such purposes aren't going
to happen.
> Patch trains from google and other contributions to non-AGESA code gain a 2x to 3x
> speedup in server time, while users of AGESA can continue to contribute
> and work on the codebase.

... and diverge...
> Since 4.2 was just released, we can do this today without much fanare.

We have nothing to hide, so "without much fanfare" doesn't need to factor in.
> To anyone saying that putting code in a branch is a death sentence,

I'm not a fan of such attempts at framing the debate.

Patrick