AGESA is a very heavy beast, at over one and a half million lines of code. Although contributions to the AGESA codebase (boards and supporting chipset/cpu code) have slowed down significantly over the years, we still rebuild this behemoth for every version of a patch.
This causes quite a bit of strain on our build infrastructure, and even with an army of four state-of-the art build machines, some people (wink, wink, Martin) are concerned with long patch trains and verification times. I can't speak for other machines, but after replacement hardware costs, flying in, and a lost day of work, I've realized that maintaining this infrastructure is not cheap.
This made me think if we can make more efficient use of the infrastructure we have available today, and allow builders to use less-powerful machines without increased latency.
I've run a few timing tests without AGESA being built, and took historical data to compare build times for when AGESA is included in the build.
From averaging build times on nuclearis2, with a primed page cache and
ccache a compete abuild run takes on average, five to six minutes. A build without a useful ccache cache takes around fifteen to twenty minutes (I don't have a reliable way of timing this, hence the high variance).
Now, if we build a tree without AGESA, we get the following:
First run: [mrnuke@nuclearis2 coreboot]$ ccache -C #clear ccache [mrnuke@nuclearis2 coreboot]$ time CPUS=12 make what-jenkins-does real 6m46.778s
Second run: [mrnuke@nuclearis2 coreboot]$ time CPUS=12 make what-jenkins-does real 2m7.889s
7 min vs 20 min on empty ccache, and 2 min vs 6 min on primed ccache. Those are speedups of 2x to 3x. Also note that the local builds were not done on a tmpfs, like the actual jenkins builds. I expect similar speedups on the other builders.
The obvious solution is to separate AGESA into its branch. Patch trains from google and other contributions to non-AGESA code gain a 2x to 3x speedup in server time, while users of AGESA can continue to contribute and work on the codebase.
Of course, the AGESA branch would also have the non-AGESA boards and code trimmed so as to make efficient use of the server resources. There is some common code, and overlap between the two code paths, but it doesn't seem to be anywhere near as significant as the behemoth that AGESA is.
Since 4.2 was just released, we can do this today without much fanare. This also allows other people to donate CPU cycles without needing to own over-priced server-class machines, and without increasing the verification latency (compared to the one today).
To anyone saying that putting code in a branch is a death sentence, please don't treat this as TL;DR and read the above paragraphs.
while I'm happy that you're providing hard numbers, I completely disagree with your conclusion. A much better approach, that doesn't rely on figuring out which parts happen to be inconvenient to a subset of our developers, is to build an analysis of the impact of a commit.
For global code changes, we _want_ everything to be build (and boot) tested, while a change in src/mainboard/$foo/$bar, in principle, only needs to build that single mainboard. Such an approach requires, and here comes the (somewhat) hard part, that the scope of a change can be determined reliably.
The speed-up of such an approach is close to optimal.
2015-11-01 1:36 GMT+01:00 Alex G. email@example.com:
AGESA is a very heavy beast, at over one and a half million lines of code.
Just a note: the only reason why current Intel fares better is because they're shipping binaries that we can't even distribute in 3rdparty/blobs. Otherwise that looks pretty similar.
7 min vs 20 min on empty ccache, and 2 min vs 6 min on primed ccache. Those are speedups of 2x to 3x.
Thank you for doing the measurements.
The obvious solution is to separate AGESA into its branch.
It may be obvious to you, but there were enough good arguments brought up by a many good people that branches for such purposes aren't going to happen.
Patch trains from google and other contributions to non-AGESA code gain a 2x to 3x speedup in server time, while users of AGESA can continue to contribute and work on the codebase.
... and diverge...
Since 4.2 was just released, we can do this today without much fanare.
We have nothing to hide, so "without much fanfare" doesn't need to factor in.
To anyone saying that putting code in a branch is a death sentence,
I'm not a fan of such attempts at framing the debate.