Hi people,
we have had a great first year with regular coreboot releases. Exactly 12 months ago, we released coreboot 4.1, the first release after the 4.0 release in 2011 and our "classic" branch in 2014.
Since then we have had 4 successful releases, both Patrick and Martin went through the release process, and with this process we were able to move two corporations working with coreboot (Intel and Google) significantly closer to working with upstream coreboot.
However, releases take a lot of time, and we were not able to tighten the release process with each release, as we had originally hoped, and so, after I talked with Patrick and Martin, we have decided to move to a slightly slower paced release cycle, creating only two releases per year. In turn, we will try to improve the overall quality of the releases in the future. This switch will mean, that the coreboot 4.5 release will happen in fall 2016, rather than this month.
I have written up a few questions that I came across, and some answers to them.
If you are using coreboot releases, I would like to hear from you, so that we can continue to improve the release process and tune it towards its users, while we spend the majority of the time on making coreboot better.
We're continually improving the release process and raising the quality bar and look for developers interested in helping on that topic.
Questions and comments, flames and praise are, as always, welcome!
Stefan
FAQ ---
1. What kind of releases do we do?
The coreboot project is now creating a new release every six months.
Before this cycle, we created a new release every quarter, but every second release has basically been unused.
2. Why do we need releases at all?
Lots of people work "offline". You check out a given revision of coreboot, and do your work. When you are done, you will have to fix up your patch to still work with the now latest revision, before you can submit it to the upstream tree. Depending on how complex your problem is, or how busy you are with other stuff, a lot of time can pass between the start and the finish line.
Releases are a simple mechanism to make sure everybody is on the same page. A way of documenting what has changed between the start of your effort and it's availability to the community.
3. Isn't this just psychological?
In the past we have seen groups of people working together on a coreboot port to a new platform, with slightly different trees, being 1 or 2 weeks apart. This can cause all kinds of funny effects, that take away from the efficiency of the development process, and that, lacking up to date documentation, every engineer has to go through again.
With over 100 people contributing over 3000 patches per year to our code base, these little changes sum up to a significant amount.
4. Who are these releases for?
If you are a developer that wants to put coreboot on your board at home, you probably don't care much, yet, because chances are, we didn't even test your board before making the release.
If you are setting up an internal coreboot tree for your fellow coworkers to colaborate with you, and you can't work on coreboot.org directly, please use the latest released version to base your work off, rather than top of tree. This will make it much easier to support your work.
5. What about testing releases?
Right now, we are hand testing releases on a few selected machines before every release by boot testing on a given setup. We try to cover as much diversity as we can (e.g. Intel, AMD, ARM, ARM64). However, this part of the release process is less formalized as the coreboot project lacks a large scale test system setup at the moment. Mostly, what the release manager for a release has available at the time is what will be tested.
6. What's with that dropped board thing?
Short story: It's not really worth bringing forward all boards all the time, so we're trying to find a reasonable cut-off line.
Long story: Coreboot is a fast moving project with a lot (a whole lot) of boards, and even more features.
In the very early days (1999 - 2003), most boards wouldn't compile or work shortly after they've been merged, unless someone went ahead and fixed it. When we added abuild to the development process, that didn't happen anymore and so we attempted to move all boards forward to the latest feature set all of the time.
Since coreboot itself has no user interface and barely any attack surface, there are barely good reasons to update your firmware very often, especially if work on your board has generally stopped. On the same hand, a lot of design decisions made 10 or 15 years ago can only be overcome with significant effort that is not justified for obsolete hardware. In order to keep the code healthy and clean, we are retiring old boards in release branches. For example, if a board was removed right after the release of 4.4, you can still check out the "4.4" branch and build your board with that branch. If the board is broken on that branch, you can check in fixes to the branch to get it working again.
There is no hard cut-off line, but typically boards that have been in the tree for more than 10 years will be retired at some point.
7. What (else) happens before a release?
- Stabilization of the tree, review and check-in of critical fixes - Boot testing on a handful boards - Creation of a change log and release notes. This will contain some useful metrics about the last release, including the amount of new code integrated, old code removed, new contributors in the project, significant high level changes, and what needs to be done to unmerged ports to be adapted to these changes. - A new branch is cut, named after the release (e.g. "4.4"). This branch can be used to push critical fixes for a given release, albeit this has not happened very often in the past (Thanks to Patrick Rudolph and Nico Huber for being an exception) - A new tag is created, named after the release (e.g. "4.4") - Announcements on the blog
8. Why don't we have monthly releases?
Our release process, while, as Patrick called it, follows a "minimum viable" approach, the testing, reviews and documentation are a lot of manual work going into each release. At this point, it is a fair assumption that it is a 1 week (40+h) effort to create a new release, for someone who has done a coreboot release before. We would like to keep the project overhead (and the frustrations that come with that) adequately low. As right there are only a handful of entities relying on coreboot releases right now, a bi-annual release cycle seems reasonable.
9. What about board status?
The board status script (and the attached web page https://www.coreboot.org/Supported_Motherboards ) is currently the best method to determine when a board was last tested and working, other than getting in touch with the original developer. Unfortunately there is currenlty no real documentation about the process, and reports rely on community members willing to test new versions on their boards.
10. Long term plans
The longer term plan is to provide infrastructure that will allow the coreboot project to have 100% of the boards tested regularly (e.g. on every commit, but at the minimum for every release).
As we gather more metrics for the quality of releases, we will keep increasing the bar for each release.