Re: [coreboot] Binary blobs in the source tree (was: Re: New patch to review for coreboot: e4fc528 Add the memory reference code binary for sandybridge chipsets)
Am 15.04.2012 22:08 schrieb ron minnich:
On Sun, Apr 15, 2012 at 3:49 PM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Am 15.04.2012 04:57 schrieb ron minnich: It's a slippery slope, though. We call the blob from coreboot and expect it to return to coreboot (whether the return is a JMP/RET/... doesn't matter). Unless I'm mistaken, we've never done that before,
We've been doing it for almost ten years, first with vga bios and then with microcode. In the latter case, the microcode was a binary blob linked in to coreboot which we called. The Intel MRC is actually less connected, as it is a blob in cbfs.
I thought the microcode updates were not executed as binary blobs, but rather uploaded (or banged into a register) into the CPU.
and if we do this now in the official tree, it will be very hard to refuse merging chipset init blobs (well, pretty much any init blob), and for some chipset/CPU combinations coreboot may turn into a simple blob aggregator with PCI init. That's a scenario I'm very afraid of.
it's a danger. It's something we have to watch for. I've made it clear to the vendors that's not what we want. They seem to be listening.
Glad to hear that.
But we can not escape some minimal set of blobs unless we wish to drop x86 platforms (or most of them) because it's getting to be impossible to turn on an x86 without these blobs.
Is this a problem for all x86 CPU vendors? I don't believe in "name and shame", but at least knowing if there is unaffected hardware would help making an informed decision. By the way, we should really document in the wiki what's going on outside the coreboot code paths before and during startup.
The next blob I have to commit is the ME binary, and that's not even run by the x86, but by the external RISC CPU. And, you can't turn on the platform without it.
Microcode is conceptually different. It's a stream of bytes you bang into some register, you don't execute it as normal code on your CPU. CPU microcode is like a network card's internal firmware: outside the scope of coreboot.
I see no difference but I'm not really here for a debate. If we can't come to convergence I'll just make a separate repo.And, I suppose this means there's no objections to me putting the ME binary blob into the tree?
Now that you're explicitly asking me, I'd like to keep ME/IMC (Management Engine/Internal Management Controller) binary blobs out as well. Rudolf has worked quite a bit on a free IMC firmware for recent AMD chipsets, and there's a faint hope (i.e. wishful thinking) we'll get something like that for Intel as well one day. The same applies to firmware for standalone EC (Embedded Controller) in laptops/servers.
I heard rumors that the MRC blob is only a stopgap solution because there wasn't sufficient time to write+QA RAM init code for the sandybridge platform, and such code might materialize later.
That rumor is complete and total random noise (to use a polite word). You heard it here first. I don't know why people make that stuff up.
Wishful thinking maybe? I really hoped the rumor would be true. Probably a classic telephone game: "MRC blob is a hard requirement" -> "would be nice if we had a MRC blob replacement" -> "maybe someone is already working on that" -> "surely somone must be working on that, it's an obvious goal".
So, one more cycle of this discussion and then I'd like to come to a decision. We've still got lots to do so we can give you all a coreboot release at the same time the laptop is for sale!
I'm all for having a separate git tree with all sorts of helpful/necessary blobs hosted at coreboot.org, because that gives us the option to keep the trees in sync and it also makes coreboot.org the canonical source of everything you might ever want or need for a firmware build. (CPU microcode updates should remain in the main source tree, though.)
Regards, Carl-Daniel
To be honest I don't see the difference between having binary blobs in a separate repo and having them in the main coreboot repo. Apart that is from the slightly meaningless saying that the whole coreboot tree is GPL compatible. It seems more relevant to be able to say that about a particular port. So AMD chipsets will be fully GPL whereas Intel chipsets will not. This is regardless of where the blobs are actually stored. And aside of the question of where and how microcode is executed.
So the big question is really do we allow chipset support that requires the use of binary blobs in coreboot. If yes then those blobs may as well reside in the coreboot repo as anywhere else because you're going to need them to build a port.
In my view microcode and MRC are roughly equivalent. The method of executing the microcode is just different and that microcode is executed by the CPU in a different operating mode but it most certainly does go away and come back and the CPU state has changed by doing so. So it is a slippery slope and we have already started on it and I see no practical way back now.
Andrew
PS I am really excited to see SandyBridge support coming in, I may actually be able to use coreboot in a project now.
Sorry for the rant, it's not aimed at any particular person, just to blobs which I don't think should get into coreboot.
On Mon, Apr 16, 2012 at 08:53:56AM +0100, Andrew Goodbody wrote:
To be honest I don't see the difference between having binary blobs in a separate repo and having them in the main coreboot repo. Apart that is from the slightly meaningless saying that the whole coreboot tree is GPL compatible. It seems more relevant to be able to say that about a particular port. So AMD chipsets will be fully GPL whereas Intel chipsets will not. This is regardless of where the blobs are actually stored. And aside of the question of where and how microcode is executed.
Of course it is relevant to say a whole project is GPL compatible, or free, or even any other property. Software isn't just a bunch of features and a bit string. It is a work that evolves through time, that's why people spend effort on maintanability and that's why the choice of licenses sets a shared plan on what committers agree. When you use a piece of software you invest some effort because you have some expectations not only on current features but on future possibilities. There's a capital to waste when changing directions.
I've not helped this effort much so it's not for me to decide, but if coreboot starts filling with blobs it will be just another BIOS, only somewhat delayed or playing catchup with some features.
In the past I've bought hardware believing it was likely to work with free firmware by looking at what coreboot supports. Recently I thought I might buy a chromebook if they finally run coreboot. But if coreboot becomes a blob loader I won't buy any hardware based on the fact that it runs coreboot. Coreboot would no longer be what it was to me. If someone forks a coreboot-libre then I may follow that, otherwise, it will be like what linux has become, you no longer know much by the statement that linux runs on something. If android is free software why should it be so hard to replace it with something else on some hardware? Does it have something to do with it accomodating too many propietary complements? It is pervasive, it has a big growth, it has many users. Yet it doesn't give them quite enough freedom. Coreboot may choose to go that route or may try to go the way of many other free software projects (GCC, Apache, debian...)
So the big question is really do we allow chipset support that requires the use of binary blobs in coreboot.
I don't. I try to find chipsets that don't require that.
If yes then those blobs may as well reside in the coreboot repo as anywhere else because you're going to need them to build a port.
I don't think such a limited port is worth people's efforts, but that's for each one to decide, of course. There are other propietary BIOSes around for that hardware, a free port to any hardware is worthwhile, a greenwashed half baked port of something that used to be free is not a big deal. I know it's still a lot of work in the free code for intel chipsets, but if the memory controller (which I hear is one of the most difficult parts) or such or such can't be understood or repaired, and the propietary BIOS you got preinstalled from factory does the job, I don't see much point in making or using a port.
In my view microcode and MRC are roughly equivalent. The method of executing the microcode is just different and that microcode is executed by the CPU in a different operating mode but it most certainly does go away and come back and the CPU state has changed by doing so.
Well, yes, I'm not very comfortable with microcode updates either. Yet, the fact I'm not comfortable with two things does not mean they are equal. I'm not very up to date, but if microcode is now loaded from the cbfs (it wasn't when I last looked, long ago, before git), then at least it has some legal advantages. Stuff you link at runtime is susceptible to form a derived work, at least under some interpretations (IANAL, and not sure at all calling some blob code needs any linking or anything that can be seen as creating derived works). Stuff you load from flash to hardware (from one EPROM to another, in fact) might not be construed as derived work.
Besides, microcode can be redistributed even if it is not free. Does someone else besides intel and google have permission to distribute the blobs? coreboot users have usually assumed they had permission to distribute the tree freely under GPL, not necessarily sending people to a single repository because the parties with distribution rights have designated it as distribution means. I mean people can publish their git trees anywhere, can't they? Must they now prune the blobs or ask intel for permission ?
People who is used to collaborate in a certain way even if it has historically meant incomplete hardware support may feel less inclined to collaborate when it is not sure their contributions will be used in the way they expect, or when they have difficulty to grasp what the criteria is for accepting restricted code to enable hardware support, or for distributing the resulting material.
Even if this (real or imagined) legal uncertainity does not scare corporate contributions, it might degrade their contribution. This includes even hardware vendors who might have published materials under terms that they believed were necessary to get acceptance and might change their policy if they find out the requirements for participation weren't so strict given later decisions or even feel under unfair competition. Others can claim coreboot compatibility with less disclosure. So going for more hardware support now might lead to less hardware support later, or to a spiraling propietarisation of the code base that voids the attractiveness of the project.
So it is a slippery slope and we have already started on it and I see no practical way back now.
Not sure why being in a slippery slope would be a reason to stop trying to keep some balance...
Andrew
PS I am really excited to see SandyBridge support coming in, I may actually be able to use coreboot in a project now.
Yeah, I was excited too until I heard of blobs.
I guess it's better to just stop buying hardware, if you can't even rely on a FSF priority project to get free software. Conformism with blobs seems to be so pervasive...
Hi.
xdrudis wrote:
Sorry for the rant, it's not aimed at any particular person, just to blobs which I don't think should get into coreboot.
I'm afraid you seem quite disconnected from the reality of PC firmware in the industry. If you haven't already read Svante's thesis on the topic then I strongly urge you to do so. Even better is of course actual OEM and ODM experience.
capital to waste when changing directions
You imply that you experience a change in direction, but as I tried to make clear already in my last email, I do not consider coreboot to have changed direction at all. The recent progress is a leap forward; as already confirmed by Andrew, coreboot is more valuable with Sandy Bridge support.
You personally, and others with you, have a different view, and thanks to coreboot being open and free you are still able to find a solution which meets your requirements.
if coreboot starts filling with blobs it will be just another BIOS, only somewhat delayed or playing catchup with some features.
I'm not sure if you realize how absurd that is. In case you didn't notice, coreboot has been playing catchup with all hardware for the first 12 years.
The progress that has been made in the project during the last few years is amazing, and instead of ranting about blobs you really should be celebrating, because coreboot is in better shape than ever before. I'm sure you agree that this is a good thing.
Your experience from hardware vendor interaction tells you that open source and free software can be challenging for many vendors, but at the same time they find it exciting because of all the benefits it brings, both inherently by being open and free, and through features which all in all make open and free totally superior to proprietary.
You may think that 12 years is a long time for a project, but in fact coreboot is only since a year or so, when AMD announced their support, starting to grow up, and even then there are many things that desperately require your contributions to increase it's success.
Ranting? Not very helpful.
Of course - if you can not support coreboot based on your ideology then you can not contribute, and that's too bad, but maybe you will discover that at some later time things have changed to where you want them to be, and then you can join the project again. Or not.
As I've already written, you could also take the blob as a challenge to implement native memory init for the platforms. I would welcome that contribution very much. I expect the hardware to have become obsolete since many years, by the time you have finished. But go ahead, prove me wrong!
Or perhaps you can find another, more productive, way to support coreboot.
In the past I've bought hardware believing it was likely to work with free firmware by looking at what coreboot supports. Recently I thought I might buy a chromebook if they finally run coreboot. But if coreboot becomes a blob loader I won't buy any hardware based on the fact that it runs coreboot.
"becomes a blob loader" is conveniently unspecific. As always with open source or free software projects: YOU are reponsible for making it work for YOU according to YOUR requirements.
OR - you can go to a vendor and pay them for the service of doing it on your behalf. Currently there exists no such service for coreboot, but the more popular coreboot becomes the more likely it is that such a service provider (or several!) will appear.
YOU have to research details to ensure that YOUR requirements are met.
Blobs do not change this fundamental fact in any way. If there are no blobs then your "zero blobs" requirement is very easy to check off the list of requirements, while other metrics such as "computational performance" or "power consumption" remain exactly as difficult to check.
You have no right to spit on coreboot (which is what you say that you would do if it became "a blob loader") just because one part of it solves problems which you do not want to solve, while other parts solve exactly the problems you do want to solve.
Further, as Ron already pointed out - and as you know from your experience with PC bootstrapping, you can't really ignore that many coreboot systems already depend on another blob during startup - the VGA BIOS. It's not really related to coreboot, but to the way the PC platform is expected to work by PC operating systems.
AFAIK there are exactly three exceptions to this:
1. Very old ATI graphics support no longer available 2. Geode LX 3. Luc's work on native init of graphics on K8M890 and similar
I think the only mainboard with the latter is the m2vmxse.
The exceptions also only hold true for very special operating system cases. You can forget about running GNU/Hurd on a system without a VGA BIOS. Isn't that ironic?
it will be like what linux has become, you no longer know much by the statement that linux runs on something. If android is free software
Who has said that Android is free software? That's stupid. If you want to discuss Linux not being free enough then you can always try to take it up with Linus.
Coreboot may choose to go that route
Third time: coreboot has never changed direction and I do not see any reason to do so in the future.
Of course your perception of reality may be different.
So the big question is really do we allow chipset support that requires the use of binary blobs in coreboot.
I try to find chipsets that don't require that.
As a user of anything, be it hardware or software, that's the perfect attitude! And it's exactly this that makes open source and free software very very valuable: It allows users to find the solutions that fit their needs. You can pretend that coreboot actually does not support Sandy Bridge and Ivy Bridge at all, and you are still golden!
I don't think such a limited port is worth people's efforts, but that's for each one to decide, of course.
Indeed it is. What you write is extremely offensive. You are disrespecting the magnitude and excellence of the work that has been contributed to the coreboot project by the Chromebook team and I just find that to be really really poor form, regardless of your reasons.
a greenwashed half baked port of something that used to be free is not a big deal.
What the fuck are you talking about? Push your free implementation of Sandy Bridge and Ivy Bridge memory controller init to gerrit.
I don't see much point in making or using a port.
That's fine. If you feel that coreboot is not for you then please do not use it and do not complain that others do. Meanwhile, consider that coreboot is not only about your system. It supports a few hundred mainboards and I hope it will support many hundred more.
Does someone else besides intel and google have permission to distribute the blobs?
Please see http://www.coreboot.org/Development_Guidelines#Sign-off_Procedure
I expect you to actually have solid knowledge of this already, since you have already sent contributions.
coreboot users have usually assumed they had permission to distribute the tree freely under GPL,
You speak for all of them? I am sure that you realize how absurd that is. Noone can claim anything on behalf of "coreboot users."
difficulty to grasp what the criteria is for accepting restricted code to enable hardware support, or for distributing the resulting material.
Ask away about anything that is unclear! Given your experience from hardware vendor interaction you will know exactly which questions are appropriate and how to ask them, as well as which questions are embarrassing faux pas, or worse, for you, and for the project.
claim coreboot compatibility with less disclosure
Reality check. Vendors are not fighting each other over who can be more compatible with coreboot. For 12 years coreboot has been chasing hardware. You need to understand the full impact of this.
I guess it's better to just stop buying hardware,
Right, better not educate yourself about your options and better not choose what fits you, but instead choose nothing. That's the perfect way to consume.
Yes you have to make a big fucking effort to be a responsible consumer OF ANYTHING so that you can sleep well at night and look at yourself in the mirror every morning. NOONE frees you from this responsibility (mind the pun) - not even the FSF.
if you can't even rely on a FSF priority project to get free software.
As I am sure that you already know, coreboot is not an FSF project.
I will not bother biting on this last piece of flame bait. I am confident that you understand that coreboot is more free than your other options. If it's not free enough for you at this time then please either go away or please help us out. In any case you are *NOT* helping by complaining.
I'm very happy that the FSF agrees with the coreboot contributors that it's important to work toward a viable open and free PC firmware.
This was always the direction of coreboot, and it has not changed!
//Peter
Am 17.04.2012 01:48, schrieb xdrudis:
In the past I've bought hardware believing it was likely to work with free firmware by looking at what coreboot supports.
We already have some notebooks in the tree that require embedded controller firmware that you can't replace - it's not just an inconvenience that can be fixed by using the right DRM/GEM/Gallium/DDI/whatever driver, but their absense actually breaks the boot process.
Sorry, this was simply a wrong assumption, and for a long time.
otherwise, it will be like what linux has become, you no longer know much by the statement that linux runs on something.
coreboot is not a free software certification process. Never was.
It is pervasive, it has a big growth, it has many users. Yet it doesn't give them quite enough freedom.
What does that tell us?
Coreboot may choose to go that route or may try to go the way of many other free software projects (GCC, Apache, debian...)
Apache is hardly a free software project (it's open source - the Android "issue" you lament exists for a large part because they use the _Apache_ license).
So the big question is really do we allow chipset support that requires the use of binary blobs in coreboot.
I don't. I try to find chipsets that don't require that.
Then current Intel chipsets aren't for you: Sandybridge requires (besides the MRC binary) a MEI binary, for the embedded controller in the chipset.
Without a proper MEI firmware, the CPU won't even start executing x86 opcodes. The MEI firmware is signed (with some private Intel key). You (or someone) might be able to reimplement the raminit part. Good luck cracking the key to MEI firmware signatures.
It gets better: Intel TXT (some security feature) uses another signed bit of code. This time for x86 (yes, ran on the CPU. Look up Invisible Things Labs, they explain why this is a stupid idea even without regard for any notion of "freedom"). Again signed with some private Intel key. You can argue that TXT doesn't matter for you, but then why buy it (and support a company producing such things)?
There's another vendor, a bit smaller, but with slightly more sensible design decisions, that might suit your needs better.
I don't think such a limited port is worth people's efforts, but that's for each one to decide, of course. There are other propietary BIOSes around for that hardware, a free port to any hardware is worthwhile, a greenwashed half baked port of something that used to be free is not a big deal.
Sandybridge support was never "free" (it wasn't there in the first place), and I doubt that the intention was "greenwashing".
and the propietary BIOS you got preinstalled from factory does the job,
coreboot _is_ the firmware preinstalled from factory on those boards. I truly hope it does the job.
coreboot users have usually assumed they had permission to distribute the tree freely under GPL, not necessarily sending people to a single repository because the parties with distribution rights have designated it as distribution means. I mean people can publish their git trees anywhere, can't they? Must they now prune the blobs or ask intel for permission ?
That's why we discussed moving the file into a separate repository, "just to be sure" (and avoid annoying discussions about the DFSG down the road).
Others can claim coreboot compatibility
That first has to be desirable to vendors before I care.
I guess it's better to just stop buying hardware, if you can't even rely on a FSF priority project to get free software.
coreboot is no FSF project. Any such labelling is on their part. Sorry if they misled you.
Conformism with blobs seems to be so pervasive...
While mrc.bin matches pretty much every definition of the term "blob", the FSF muddying the waters wasn't exactly helpful to keep that term meaningful.
Patrick
we can stop now. The mrc.bin is gone. I actually was just about to abandon the CL when it got approved ;-)
Sorry to cause all this commotion! Feel free to continue the discussion, or, use the 'm' command in gmail :-)
ron
-
Andrew Goodbody -
Carl-Daniel Hailfinger -
Patrick Georgi -
Peter Stuge -
ron minnich -
xdrudis