Hi,
I wonder if we want to establish something like the "Designed for Windows XP" or "Yes it runs with Netware" certificates? It would certainly be a nice marketing aid for vendors, and at the same time it would promote coreboot visibility.
If there is interest in such an idea, we will have to decide which criteria have to be fulfilled to get such a certificate, and if the certificate has an expiry date and/or is bound to a specific svn revision. Off the top of my head, I can think of the following criteria: - coreboot+SeaBIOS works well enough to boot $ENTERPRISE_LINUX, $ENDUSER_LINUX and Windows 7 (Vista and XP as well?) - Nvidia and ATI graphics drivers (both free and closed) work if booted with a coreboot+SeaBIOS image? - Frequency scaling and the various suspend methods work - Soft poweroff works - IRQ routing and all PCI/PCIe/AGP/whatever slots work - Legacy ports (if present) work - Fans work well enough (temperature-based scaling if present in the "normal" BIOS) - Source for a working coreboot image (including the Kconfig settings for the board, and possibly NVRAM settings?) is available for free without NDA - Board port merged into coreboot svn - SeaBIOS source code is available - SeaBIOS code is merged into SeaBIOS git - flashrom works on the board (no lockdown) or there is a way to boot unlocked and run flashrom for your image of choice - At least some serial output (coreboot version) if a serial port (header) is present, otherwise... USB Debug? Floppy? LPC bus? POST card on port 82h?
Did I forget something? Are some criteria useless? What would Jane User expect from a normal mainboard if she didn't know coreboot was running?
Regards, Carl-Daniel
I think that a base coreboot certification should basically state that all the hardware on the board is usable with a major free OS (e.g. Linux-based OSes like Debian, Ubuntu, and Redhat maybe).
We could maybe have extended certifications for things like non-free OS and driver compatibility.
My comments below are what I would expect minimum coreboot compliance to mean.
On Sat, Oct 2, 2010 at 3:44 PM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
I wonder if we want to establish something like the "Designed for Windows XP" or "Yes it runs with Netware" certificates? It would certainly be a nice marketing aid for vendors, and at the same time it would promote coreboot visibility.
Interesting idea. I think that we'd need participation from board vendors for it to make much sense.
If there is interest in such an idea, we will have to decide which criteria have to be fulfilled to get such a certificate, and if the certificate has an expiry date and/or is bound to a specific svn revision. Off the top of my head, I can think of the following criteria:
- coreboot+SeaBIOS works well enough to boot $ENTERPRISE_LINUX,
$ENDUSER_LINUX and Windows 7 (Vista and XP as well?)
Why should Windows be important criteria? Should we really withhold a coreboot certification on the condition that a non-free OS work?
- Nvidia and ATI graphics drivers (both free and closed) work if booted
with a coreboot+SeaBIOS image?
Frankly, I think that ability to use the free drivers should be good enough. We shouldn't be hold out any kind of coreboot certification on the condition that non-free drivers work.
- Frequency scaling and the various suspend methods work
yes
- Soft poweroff works
yes
- IRQ routing and all PCI/PCIe/AGP/whatever slots work
yes
- Legacy ports (if present) work
How about any ports on the board should work, legacy or not?
- Fans work well enough (temperature-based scaling if present in the
"normal" BIOS)
I don't think that we should compare coreboot to the "normal" bios. We should decide whether this feature is needed or not in a certified system that is capable of it.
- Source for a working coreboot image (including the Kconfig settings
for the board, and possibly NVRAM settings?) is available for free without NDA
Yes.
- Board port merged into coreboot svn
Yes.
- SeaBIOS source code is available
Yes.
- SeaBIOS code is merged into SeaBIOS git
Yes. Doesn't this imply the previous item?
- flashrom works on the board (no lockdown) or there is a way to boot
unlocked and run flashrom for your image of choice
Yes.
- At least some serial output (coreboot version) if a serial port
(header) is present, otherwise... USB Debug? Floppy? LPC bus? POST card on port 82h?
I thought that POST cards showed valued outputed on port 80h. What is 82h?
Basically every coreboot system should output POST codes that a POST card can display if it's possible to insert a POST card.
Any physical ports (including headers for ports) on the board should be supported.
Thanks, wt
On 03.10.2010 01:28, Warren Turkal wrote:
I think that a base coreboot certification should basically state that all the hardware on the board is usable with a major free OS (e.g. Linux-based OSes like Debian, Ubuntu, and Redhat maybe).
We could maybe have extended certifications for things like non-free OS and driver compatibility.
Well, if we only care about Linux, you can avoid most (if not all) of ACPI on many machines, and you can avoid SeaBIOS as well. Heck, you could even avoid FILO and require a Linux kernel in flash. Whether such ab board would be usable for end users is a totally different question. IMHO being able to install a Linux distribution from CD is an absolute must even if you only target professional users in clusters etc. Windows support means a board is usable by the general population, and this is something vendors care about deeply. We renamed LinuxBIOS to coreboot exactly because people said all the time "I don't want Linux", and EFI marketing would love to make fun of us if we ever said "Linux only is good enough for certification".
My comments below are what I would expect minimum coreboot compliance to mean.
On Sat, Oct 2, 2010 at 3:44 PM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
If there is interest in such an idea, we will have to decide which criteria have to be fulfilled to get such a certificate, and if the certificate has an expiry date and/or is bound to a specific svn revision. Off the top of my head, I can think of the following criteria:
- coreboot+SeaBIOS works well enough to boot $ENTERPRISE_LINUX,
$ENDUSER_LINUX and Windows 7 (Vista and XP as well?)
Why should Windows be important criteria? Should we really withhold a coreboot certification on the condition that a non-free OS work?
Please see above.
- Nvidia and ATI graphics drivers (both free and closed) work if booted
with a coreboot+SeaBIOS image?
Frankly, I think that ability to use the free drivers should be good enough. We shouldn't be hold out any kind of coreboot certification on the condition that non-free drivers work.
There are two aspects of the problem: - We can't test everything (fact of life) - Closed-source drivers have a huge market share, and won't go away any time soon
- Legacy ports (if present) work
How about any ports on the board should work, legacy or not?
Absolutely. I just wanted to point out that many vendors don't care that much about legacy anymore.
- Fans work well enough (temperature-based scaling if present in the
"normal" BIOS)
I don't think that we should compare coreboot to the "normal" bios. We should decide whether this feature is needed or not in a certified system that is capable of it.
Fans can be loud. If all fans run at 100% non-stop, machines can be essentially unusable for noise reasons.
- SeaBIOS source code is available
Yes.
- SeaBIOS code is merged into SeaBIOS git
Yes. Doesn't this imply the previous item?
Indeed, but given that vendor code may not always be suitable for merging, do we want to withhold certification if code is available but not merged? And what happens if Kevin is on vacation?
- At least some serial output (coreboot version) if a serial port
(header) is present, otherwise... USB Debug? Floppy? LPC bus? POST card on port 82h?
I thought that POST cards showed valued outputed on port 80h. What is 82h?
Some POST cards support port 80h/82h/84h and you can decide which port they should listen to. Some multi-functional POST cards support reprogramming in a way that allows you to send text output to port 82h/84h and normal POST codes to port 80h.
Basically every coreboot system should output POST codes that a POST card can display if it's possible to insert a POST card.
Any physical ports (including headers for ports) on the board should be supported.
Yes.
Regards, Carl-Daniel
On Sat, Oct 2, 2010 at 5:06 PM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
On 03.10.2010 01:28, Warren Turkal wrote:
I think that a base coreboot certification should basically state that all the hardware on the board is usable with a major free OS (e.g. Linux-based OSes like Debian, Ubuntu, and Redhat maybe).
We could maybe have extended certifications for things like non-free OS and driver compatibility.
Well, if we only care about Linux, you can avoid most (if not all) of ACPI on many machines, and you can avoid SeaBIOS as well. Heck, you could even avoid FILO and require a Linux kernel in flash. Whether such ab board would be usable for end users is a totally different question.
Booting into a certain OS is clearly not the only bar that we should be looking for. It would need to comply with various standards like ACPI, etc.
IMHO being able to install a Linux distribution from CD is an absolute must even if you only target professional users in clusters etc.
I agree.
Windows support means a board is usable by the general population, and this is something vendors care about deeply.
Out of curiosity, are there actually non-niche vendors that ship Coreboot as their system firmware?
We renamed LinuxBIOS to coreboot exactly because people said all the time "I don't want Linux", and EFI marketing would love to make fun of us if we ever said "Linux only is good enough for certification".
I believe that most of the folks who care enough to use Coreboot at this time are probably running Linux or some other free OS. I also believe that most developers who have access to systems are probably running on or have easy access to some free OS. I also know that non-free OSes are not easily available to everyone. It's not that I believe that we can't test for non-free software support. It's just that I believe that certifying the boards for Coreboot should not gate on that having been done.
Ideally, having some OS independent test suite would be the best approach I think, but I don't see that getting developed overnight. :)
Maybe we should have multiple compliance levels like "Coreboot minimal" and "Coreboot standard"? The minimal could be some set of requirements. The standard could be the minimal requirements plus some extra set of requirements. We could also have tags for the compliance. These could be used to indicate special support like Windows or something like that.
Think of "Coreboot minimal" as enough to conveniently develop coreboot further without expensive and specialized hardware. Its requirement might look like the following: * Full initialization of the cpu and supporting chipset - Intialized RAM - Cooling systems should work enough to protect the system from burning itself up * At least one of the following: - Outputs reasonable POST codes to a POST card if one can be plugged in - USB debug ports enabled, if available on the board - RS232 port enabled, if available on board - Some other agreed method for debugging the system during Coreboot execution (e.g. debug LEDs on mobo, etc.) * At least one of the following - At least one usb hub usable for keyboard, if one exists - PS/2 port usable for keyboard, if one exists * All expansion slots initialized and usable * ACPI - DSDT should exist * Able to load VGA option ROM * Able to boot into an OS not stored as a coreboot payload (e.g. boot from CD, USB, or SATA drive). For the minimal certfication, I believe that OS should be Debian since it's a well supported environment for developing coreboot.
Think of "Coreboot Standard" as basically fully tested with free OSes. Its requirements might look like the following: * all requirements of the minimal certification * Any requirement of the minimal certification that allows a subset to be implemented should implement all the sub-items. For example, both rs232 and usb debug ports should be enabled if they exists on the same board instead of implementing one or the other. * Uses tiny bootblock * Uses cache-as-ram if the system is capable * Soft poweroff works * Extended initialization of supporting chipset - Fan uses vary speed based on temperature of CPU * All legacy io ports (i.e. rs232, parallel, etc.) usable * All USB ports usable * All other external (including header) ports usable (e.g. firewire, sound, etc.) * better ACPI support - OSPM (e.g. G, S, D, and C states, etc.) fully exposed and usable * CPU frequency scaling works * After booting into Debian Linux, the OS should be able to rely on any info provided by the Coreboot and any intervening payloads should allow the system to be fully enumerated and configured as much as the hardware will allow.
Tags could be used to identify specific extended support. For instance, a system certified to boot Windows 7 could be "Coreboot minimal+MSWin7" certified or "Coreboot standard+MSWin7" certified or something like that. If anyone wanted to display the certification, they could display it with or without the tags.
Possible tags: MSWin{XP,7} ReactOS GPXE AOE ISCSI ...
We should probably note the known revisions that are compliant for each board. This doesn't mean that every revision that would pass the requirements needs to be listed. Only tested versions should be listed. Those revisions would be the recommended revisions with users using other revisions at their own risk. I could imagine something like the following for a mobo: minimal: 11 99 103 150 standard: 50 75 135 155 minimal+Win7: 77
The numbers are the Coreboot svn revisions that are tested. These revisions should probably be listed in the wiki somewhere. There is probably some better way to visualize the data. We may also need to list SeaBIOS hashes for the certified builds.
*snip*
Frankly, I think that ability to use the free drivers should be good enough. We shouldn't be hold out any kind of coreboot certification on the condition that non-free drivers work.
There are two aspects of the problem:
- We can't test everything (fact of life)
- Closed-source drivers have a huge market share, and won't go away any
time soon
Agreed. I also think that some developers won't be able to test certain non-free software. For instance, I wouldn't be able to test that Windows 7 boots. I also don't have an Nvidia card to test their drivers.
Given the fact that we can't test everything, we should make a minimal amount of compliance only include things that can be tested by most folks. I think that minimally certified hardware should really be capable enough to develop Coreboot further on the hardware conveniently. Standard certified hardware should work well for free OSes if the OSes have appropriate driver support.
Also, minimally certified hardware can support more features. For example, a minimally certified board could have logic for adjusting fan speed intelligently. It would still be minimally certified until someone developed the rest of the functionality for the standard level certification and tested it.
*snip*
Fans can be loud. If all fans run at 100% non-stop, machines can be essentially unusable for noise reasons.
While I agree with this, I think that a minimally certified piece of hardware should not need working fan logic.
*snip*
Indeed, but given that vendor code may not always be suitable for merging, do we want to withhold certification if code is available but not merged? And what happens if Kevin is on vacation?
If the code's not merged into SeaBIOS, we shouldn't certify the build. If Kevin is the only person that can merge push the canonical SeaBIOS tree, then I suppose it would need to wait on him. It's the same way that a vendor can't really claim that their hardware with compliant with Microsoft's certifications until MS signs off.
If it really becomes a problem that vendors want the certification during a time when Kevin isn't around, that would be a good problem for the project to have. :)
*snip*
Comments would be appreciated, especially on the minimal and standard certification requirements above. Also, please take a look at the tags and see if you can think of any additions.
Thanks, wt
Hi!
I consider myself as enthusiast and would like to share my thoughts about coreboot certification and PR as a person from outside the coreboot project. I hope this will help you to see this project from another perspective and therefore helpful for your plans and further actions. Let me say I would really appreciate it, if vendors would ship their products preloaded with coreboot. ;-)
On Sunday 03 October 2010, Warren Turkal wrote:
On Sat, Oct 2, 2010 at 5:06 PM, Carl-Daniel Hailfinger
c-d.hailfinger.devel.2006@gmx.net wrote:
On 03.10.2010 01:28, Warren Turkal wrote:
I think that a base coreboot certification should basically state that all the hardware on the board is usable with a major free OS (e.g. Linux-based OSes like Debian, Ubuntu, and Redhat maybe).
We could maybe have extended certifications for things like non-free OS and driver compatibility.
Well, if we only care about Linux, you can avoid most (if not all) of ACPI on many machines, and you can avoid SeaBIOS as well. Heck, you could even avoid FILO and require a Linux kernel in flash. Whether such ab board would be usable for end users is a totally different question.
Booting into a certain OS is clearly not the only bar that we should be looking for. It would need to comply with various standards like ACPI, etc.
At first it would be helpful to define what you would like to achieve by introducing a coreboot certificate. Helping developers to find boards, which have basic support (as described earlier) for easy further development or helping users/customers to find boards which are coreboot compatible?
IMHO being able to install a Linux distribution from CD is an absolute must even if you only target professional users in clusters etc.
I agree.
Windows support means a board is usable by the general population, and this is something vendors care about deeply.
Out of curiosity, are there actually non-niche vendors that ship Coreboot as their system firmware?
To make coreboot popular, it needs (a) a feature people want, (b) major vendors to deliver it and (c) available documentation for all components to make it work with most boards. These requirements are interdependent and must be solved as whole. To get c vendors must be convinced to want b and therefore coreboot needs a. So what's a? It surely is fast boot up, but depending on what your target audience is, it might also be boot non-free OSes and works with proprietary drivers. If you want to make vendors want b it's definitly required to target the average user.
We renamed LinuxBIOS to coreboot exactly because people said all the time "I don't want Linux", and EFI marketing would love to make fun of us if we ever said "Linux only is good enough for certification".
I believe that most of the folks who care enough to use Coreboot at this time are probably running Linux or some other free OS. I also believe that most developers who have access to systems are probably running on or have easy access to some free OS. I also know that non-free OSes are not easily available to everyone. It's not that I believe that we can't test for non-free software support. It's just that I believe that certifying the boards for Coreboot should not gate on that having been done.
There is the possibility to depent on the vendors and the community. Vendors could test if non-free OSes work with coreboot (they test this anyway with their own BIOSes) and the community could assure this as well. That would require a place (e.g. the wiki) were the status of each board is tracked and an easy way to contribute. A Login would be required to keep the data on a sane level.
Ideally, having some OS independent test suite would be the best approach I think, but I don't see that getting developed overnight. :)
That's automation. Automation is the second step. ;-)
Maybe we should have multiple compliance levels like "Coreboot minimal" and "Coreboot standard"? The minimal could be some set of requirements. The standard could be the minimal requirements plus some extra set of requirements. We could also have tags for the compliance. These could be used to indicate special support like Windows or something like that.
If you think of a little certification logo/mark that vendors print on their boxes, then it has to be as simple and clear as possible.
Think of "Coreboot minimal" as enough to conveniently develop coreboot further without expensive and specialized hardware. Its requirement might look like the following:
- Full initialization of the cpu and supporting chipset
- Intialized RAM
- Cooling systems should work enough to protect the system from
burning itself up
- At least one of the following:
- Outputs reasonable POST codes to a POST card if one can be plugged in
- USB debug ports enabled, if available on the board
- RS232 port enabled, if available on board
- Some other agreed method for debugging the system during Coreboot
execution (e.g. debug LEDs on mobo, etc.)
- At least one of the following
- At least one usb hub usable for keyboard, if one exists
- PS/2 port usable for keyboard, if one exists
- All expansion slots initialized and usable
- ACPI
- DSDT should exist
- Able to load VGA option ROM
- Able to boot into an OS not stored as a coreboot payload (e.g. boot
from CD, USB, or SATA drive). For the minimal certfication, I believe that OS should be Debian since it's a well supported environment for developing coreboot.
What about calling it "Coreboot: Developer's Choice". Also freely available documentation would be a nice core requirement for that.
Think of "Coreboot Standard" as basically fully tested with free OSes. Its requirements might look like the following:
- all requirements of the minimal certification
- Any requirement of the minimal certification that allows a subset to
be implemented should implement all the sub-items. For example, both rs232 and usb debug ports should be enabled if they exists on the same board instead of implementing one or the other.
- Uses tiny bootblock
- Uses cache-as-ram if the system is capable
- Soft poweroff works
- Extended initialization of supporting chipset
- Fan uses vary speed based on temperature of CPU
- All legacy io ports (i.e. rs232, parallel, etc.) usable
- All USB ports usable
- All other external (including header) ports usable (e.g. firewire,
sound, etc.)
- better ACPI support
- OSPM (e.g. G, S, D, and C states, etc.) fully exposed and usable
- CPU frequency scaling works
- After booting into Debian Linux, the OS should be able to rely on
any info provided by the Coreboot and any intervening payloads should allow the system to be fully enumerated and configured as much as the hardware will allow.
This would target the normal user, so I would call it "Coreboot compatible" or "Coreboot approved". This mark should assure that all parts of the board work, meaning every feature, expected by the average user, hhis is important to please them. So that this mark actually means something to them. If they have a good reason to care about it, also the vendors have one, and this would give the coreboot project the authority dictate such high standards.
Things normal users care about to work: - All build-in components (net, snd, gfx) - All ports/expansion slots (exceptions: rs232, parallel, floppy) - Everything related to power management as supported by the underlying hardware and drivers (All power states, ACPI). Also needed to improve drivers. - Add-on components most importantly Nvidia/ATI cards - The OS of choice (BSD, Linux, Windows) - fast booting
Things normal users don't care about: - most legacy stuff like rs232, parallel, floppy - pro features like *PXE, AoE, iSCSI (Those could be combined under a logo like "Coreboot for Professionals") - OSes besides the ones they use
Tags could be used to identify specific extended support. For instance, a system certified to boot Windows 7 could be "Coreboot minimal+MSWin7" certified or "Coreboot standard+MSWin7" certified or something like that. If anyone wanted to display the certification, they could display it with or without the tags.
Possible tags: MSWin{XP,7} ReactOS GPXE AOE ISCSI ...
For OSes, users will expect that the major ones (BSD, Linux, Windows) will work flawlessy if labeled "Coreboot compatible". So if you focus on the average user, this should be respected. Listing the other features separately is a good idea, as normal users mostly won't care about them, but experienced users get the chance to check for them. To keep the logo itself simple, the feature list should be separated from the logo e.g. printed on the backside of a package. Or as mentioned above combined to another logo like "Coreboot for Professionals"
We should probably note the known revisions that are compliant for each board. This doesn't mean that every revision that would pass the requirements needs to be listed. Only tested versions should be listed. Those revisions would be the recommended revisions with users using other revisions at their own risk. I could imagine something like the following for a mobo: minimal: 11 99 103 150 standard: 50 75 135 155 minimal+Win7: 77
The numbers are the Coreboot svn revisions that are tested. These revisions should probably be listed in the wiki somewhere. There is probably some better way to visualize the data. We may also need to list SeaBIOS hashes for the certified builds.
Keeping track of detailed information in the wiki is a good thing. If vendors decide to deliver coreboot it should be as easy as possible for them.
*snip*
Frankly, I think that ability to use the free drivers should be good enough. We shouldn't be hold out any kind of coreboot certification on the condition that non-free drivers work.
There are two aspects of the problem:
- We can't test everything (fact of life)
- Closed-source drivers have a huge market share, and won't go away any
time soon
Agreed. I also think that some developers won't be able to test certain non-free software. For instance, I wouldn't be able to test that Windows 7 boots. I also don't have an Nvidia card to test their drivers.
That's were vendors and community could help out. See above.
Given the fact that we can't test everything, we should make a minimal amount of compliance only include things that can be tested by most folks. I think that minimally certified hardware should really be capable enough to develop Coreboot further on the hardware conveniently. Standard certified hardware should work well for free OSes if the OSes have appropriate driver support.
That's why I would go for "Coreboot: Developer's Choice" and "Coreboot compatible" (or "Coreboot powered" for actual boards preloaded with coreboot), it makes it much clearer in my opinion. If there's one think you don't won't a certification mark to do, then it's confusing people more than helping them.
Also, minimally certified hardware can support more features. For example, a minimally certified board could have logic for adjusting fan speed intelligently. It would still be minimally certified until someone developed the rest of the functionality for the standard level certification and tested it.
Agreed.
*snip*
Fans can be loud. If all fans run at 100% non-stop, machines can be essentially unusable for noise reasons.
While I agree with this, I think that a minimally certified piece of hardware should not need working fan logic.
True, developers should be able to deal with it, but for users it's a must have feature.
*snip*
Indeed, but given that vendor code may not always be suitable for merging, do we want to withhold certification if code is available but not merged? And what happens if Kevin is on vacation?
If the code's not merged into SeaBIOS, we shouldn't certify the build. If Kevin is the only person that can merge push the canonical SeaBIOS tree, then I suppose it would need to wait on him. It's the same way that a vendor can't really claim that their hardware with compliant with Microsoft's certifications until MS signs off.
True, but to respect vendors and how they work (e.g. deadlines etc.), it would be clever to keep up a communication channel to announce e.g. "no certification from X to Y (due to $REASON)". This allows vendors to plan. The point being always try to work with, never against each other. ;-)
If it really becomes a problem that vendors want the certification during a time when Kevin isn't around, that would be a good problem for the project to have. :)
Yes that's somehow true, but to some extend this could also be considered mismanagement. To prevent those situations each vendor would have to have (at least) one employee to stay in contact with coreboot project, which would have to ask for one. Also interproject communication (e.g. between flashrom, coreboot and seabios) is vital. I can't tell how this works atm., I assume quite good, but what I'm trying to point out is, that vendors (and users) are always looking for a complete package to solve their problems. Therefore I consider the free boot up infrastructure, meaning flashrom, coreboot and seabios, as a whole package. This makes working hand in hand very important.
*snip*
Comments would be appreciated, especially on the minimal and standard certification requirements above. Also, please take a look at the tags and see if you can think of any additions.
Thanks, wt
Please keep in mind that I'm neither involved in the coreboot project or any related projects nor do I work for a hardware vendor. I simply try to express how I see the whole picture with the hope this could be of any help for your project.
Thanks, phorsyon
On Sun, Oct 3, 2010 at 10:58 AM, phorsyon phorsyon@gmx.net wrote:
I consider myself as enthusiast and would like to share my thoughts about coreboot certification and PR as a person from outside the coreboot project. I hope this will help you to see this project from another perspective and therefore helpful for your plans and further actions. Let me say I would really appreciate it, if vendors would ship their products preloaded with coreboot. ;-)
We are not vendors, and AFAIK no hardware OEM vendors ship with coreboot pre-loaded.
On Sunday 03 October 2010, Warren Turkal wrote:
*snip*
At first it would be helpful to define what you would like to achieve by introducing a coreboot certificate. Helping developers to find boards, which have basic support (as described earlier) for easy further development or helping users/customers to find boards which are coreboot compatible?
I believe that is what this process is trying to do.
*snip*
To make coreboot popular, it needs (a) a feature people want, (b) major vendors to deliver it and (c) available documentation for all components to make it work with most boards. These requirements are interdependent and must be solved as whole. To get c vendors must be convinced to want b and therefore coreboot needs a. So what's a? It surely is fast boot up, but depending on what your target audience is, it might also be boot non-free OSes and works with proprietary drivers. If you want to make vendors want b it's definitly required to target the average user.
I do not believe that (c) is strictly required in that you don't need detailed bios developer guides once the code is available. It would be useful, but I don't see it as being required.
We also need to be realistic. OEMs are not taking up coreboot in mass right now. We need to set the certification bar at the point that it would be useful for the coreboot project. I believe that setting the certification bar allow a more rapid development of coreboot code would be much more productive that pie-in-the-sky arguments about what to do with all the OEMs that want to ship coreboot.
*snip*
There is the possibility to depent on the vendors and the community. Vendors could test if non-free OSes work with coreboot (they test this anyway with their own BIOSes) and the community could assure this as well. That would require a place (e.g. the wiki) were the status of each board is tracked and an easy way to contribute. A Login would be required to keep the data on a sane level.
Vendors do not ship coreboot yet. I believe that we only have the community at this point. Given that fact, we need to do what we can to get vendors interested. I believe that maturing the coreboot codebase is probably the most effective way to do that.
Ideally, having some OS independent test suite would be the best approach I think, but I don't see that getting developed overnight. :)
That's automation. Automation is the second step. ;-)
Indeed. For the record, I believe that Linux probably has a mature enough ACPI stack and other systems to be pretty good barometer of compliance short of having an independent test suite.
*snip*
If you think of a little certification logo/mark that vendors print on their boxes, then it has to be as simple and clear as possible.
Agreed.
*snip description of coreboot minimal certification level*
What about calling it "Coreboot: Developer's Choice". Also freely available documentation would be a nice core requirement for that.
I actually don't like "minimal." However, I also don't like "Coreboot: Developer's Choice." What would you think about "Coreboot beta test"?
*snip description of coreboot standard certification level*
This would target the normal user, so I would call it "Coreboot compatible" or "Coreboot approved".
"Coreboot compatible" is probably a better choice. I like that better than "standard."
This mark should assure that all parts of the board work, meaning every feature, expected by the average user, hhis is important to please them. So that this mark actually means something to them. If they have a good reason to care about it, also the vendors have one, and this would give the coreboot project the authority dictate such high standards.
Things normal users care about to work:
- All build-in components (net, snd, gfx)
I should have explicitly included this in the standard certification level.
- All ports/expansion slots (exceptions: rs232, parallel, floppy)
Already said this in the standard certification level except that I didn't make any exceptions. It's sloppy to leave physical and header ports not working.
- Everything related to power management as supported by the underlying
hardware and drivers (All power states, ACPI). Also needed to improve drivers.
For the record, I included this in the "better ACPI support" section of the standard certification. The OSPM part of ACPI includes all power states of all parts of the system.
- Add-on components most importantly Nvidia/ATI cards
I don't believe that we should bias toward some particular class of add-on card or some vendor of add-on card.
- The OS of choice (BSD, Linux, Windows)
While I agree with this sentiment, we can't test everything. I think that we should agree on a standard test OS. That OS needs to be freely obtainable to make the testing bar very low.
- fast booting
I think this might be a little too subjective for certification. What if a server vendor wanted to ship coreboot firmware that does a longish running operation everytime before booting. Would that never qualify for a coreboot certification?
Things normal users don't care about:
- most legacy stuff like rs232, parallel, floppy
As stated above, I think that leaving ports with physical or header connections nonfunctional is just sloppy, and it would not reflect well on the project to allow board in that state to get a standard certification.
- pro features like *PXE, AoE, iSCSI (Those could be combined under a logo
like "Coreboot for Professionals")
I agree about things like PXE, AoE, and iSCSI being more important to big iron. I'm not sure that we should have another certification level to support them right out of the gate, however. More certification levels is more confusing.
- OSes besides the ones they use
Of course.
Tags could be used to identify specific extended support. For instance, a system certified to boot Windows 7 could be "Coreboot minimal+MSWin7" certified or "Coreboot standard+MSWin7" certified or something like that. If anyone wanted to display the certification, they could display it with or without the tags.
Possible tags: MSWin{XP,7} ReactOS GPXE AOE ISCSI ...
For OSes, users will expect that the major ones (BSD, Linux, Windows) will work flawlessy if labeled "Coreboot compatible". So if you focus on the average user, this should be respected. Listing the other features separately is a good idea, as normal users mostly won't care about them, but experienced users get the chance to check for them. To keep the logo itself simple, the feature list should be separated from the logo e.g. printed on the backside of a package. Or as mentioned above combined to another logo like "Coreboot for Professionals"
Frankly, I don't think that we are aiming for average users at this time. We are aiming for the developers that can advocate coreboot's use in OEM hardware or folks who see enough of an advantage to pay for a port to their hardware. These are very sophisticated users.
*snip*
Keeping track of detailed information in the wiki is a good thing. If vendors decide to deliver coreboot it should be as easy as possible for them.
I am not sure this data is simple enough for wiki. However, I haven't given this too much thought.
That's were vendors and community could help out. See above.
We don't have vendors shipping coreboot that I am aware of. From a look at http://www.coreboot.org/Products, the closest we have is VIA, and they aren't shipping coreboot on any systems that I know of.
*snip*
That's why I would go for "Coreboot: Developer's Choice" and "Coreboot compatible" (or "Coreboot powered" for actual boards preloaded with coreboot), it makes it much clearer in my opinion. If there's one think you don't won't a certification mark to do, then it's confusing people more than helping them.
I agree that clarity is of the utmost importance. I also think that "Coreboot: Developer's Choice
*snip*
While I agree with this, I think that a minimally certified piece of hardware should not need working fan logic.
True, developers should be able to deal with it, but for users it's a must have feature.
That's why I included that as part of the standard certification.
*snip*
True, but to respect vendors and how they work (e.g. deadlines etc.), it would be clever to keep up a communication channel to announce e.g. "no certification from X to Y (due to $REASON)". This allows vendors to plan. The point being always try to work with, never against each other. ;-)
I think this is somewhat moot since we don't have vendors at this point, but, of course, I believe that we should make this work with vendors when they are ready.
If it really becomes a problem that vendors want the certification during a time when Kevin isn't around, that would be a good problem for the project to have. :)
Yes that's somehow true, but to some extend this could also be considered mismanagement. To prevent those situations each vendor would have to have (at least) one employee to stay in contact with coreboot project, which would have to ask for one. Also interproject communication (e.g. between flashrom, coreboot and seabios) is vital. I can't tell how this works atm., I assume quite good, but what I'm trying to point out is, that vendors (and users) are always looking for a complete package to solve their problems. Therefore I consider the free boot up infrastructure, meaning flashrom, coreboot and seabios, as a whole package. This makes working hand in hand very important.
I don't think it's mismanagement unless we think that Kevin is away enough to warrant adding others who can take this responsibility. If that happens, it's a good problem to have, and I am sure we can modify exactly what our definition of code state is to allow a certification. If we start off with a more conservative definition and expand it in the future, I think that'd be okay.
*snip*
Please keep in mind that I'm neither involved in the coreboot project or any related projects nor do I work for a hardware vendor. I simply try to express how I see the whole picture with the hope this could be of any help for your project.
I definitely appreciate your PoV. I think we'll need a lot of perspective to do something like this successfully.
Thanks, wt
On 04.10.2010 02:11, Warren Turkal wrote:
On Sun, Oct 3, 2010 at 10:58 AM, phorsyon phorsyon@gmx.net wrote:
I consider myself as enthusiast and would like to share my thoughts about coreboot certification and PR as a person from outside the coreboot project. I hope this will help you to see this project from another perspective and therefore helpful for your plans and further actions. Let me say I would really appreciate it, if vendors would ship their products preloaded with coreboot. ;-)
We are not vendors, and AFAIK no hardware OEM vendors ship with coreboot pre-loaded.
I think a few vendors ship some systems with coreboot if you ask nicely, but they are either in the server or embedded space, not general consumer space.
To make coreboot popular, it needs (a) a feature people want, (b) major vendors to deliver it and (c) available documentation for all components to make it work with most boards. These requirements are interdependent and must be solved as whole. To get c vendors must be convinced to want b and therefore coreboot needs a. So what's a? It surely is fast boot up, but depending on what your target audience is, it might also be boot non-free OSes and works with proprietary drivers. If you want to make vendors want b it's definitly required to target the average user.
I do not believe that (c) is strictly required in that you don't need detailed bios developer guides once the code is available. It would be useful, but I don't see it as being required.
We also need to be realistic. OEMs are not taking up coreboot in mass right now. We need to set the certification bar at the point that it would be useful for the coreboot project. I believe that setting the certification bar allow a more rapid development of coreboot code would be much more productive that pie-in-the-sky arguments about what to do with all the OEMs that want to ship coreboot.
OEMs want fire-and-forget solutions without functional regressions. coreboot can serve as fire-and-forget solution, but if we certify consumer mainboards which can't run Windows, the certificate will be essentially worthless.
Remember the complaints about "Designed for Windows Vista" and the machines which were too slow to be useful? People still remember that marketing disaster years after it happened. If we ever certify some hardware without caring about Windows, we should make that explicit, and call it a feature. "Coreboot certified board for the anti-closed-source movement, will refuse to run Windows". The variant which can run Windows would be "Coreboot certified board". Except for niche vendors, nobody will care about the anti-closed-source certification, making it moot.
Vendors do not ship coreboot yet.
AFAIK Artec, Tyan, Technexion, MSI and some other vendors ship coreboot for specific systems on request. Not sure about the current state, but AFAIK they did that in the past.
I believe that we only have the community at this point. Given that fact, we need to do what we can to get vendors interested. I believe that maturing the coreboot codebase is probably the most effective way to do that.
Booting Windows 7 on recent consumer boards is probably the best way to show vendors that coreboot is a viable BIOS replacement. This needs porting work, and of course it needs testing.
For the record, I believe that Linux probably has a mature enough ACPI stack and other systems to be pretty good barometer of compliance short of having an independent test suite.
Ummm... Linux will tolerate absolutely crappy ACPI code, and not even complain loudly if it has to fix stuff. Given the current market share of Linux, there is no money in verifying ACPI code under Linux for any consumer mainboard.
What about calling it "Coreboot: Developer's Choice". Also freely available documentation would be a nice core requirement for that.
I actually don't like "minimal." However, I also don't like "Coreboot: Developer's Choice." What would you think about "Coreboot beta test"?
For me, "Developer's choice" implies that the board already works perfectly fine, and has all the hardware features which make it easy to test new coreboot versions. I believe in honest advertising, and if the machine barely works, we should not certify it, but we could list it as "fun project if you want to finish a port".
- Add-on components most importantly Nvidia/ATI cards
I don't believe that we should bias toward some particular class of add-on card or some vendor of add-on card.
Nobody is going to test with Tseng Labs graphics cards because they are no longer on sale (actually, they haven't been on sale anytime in the last 12 years). Nobody is going to test with Intel graphics cards either unless you can actually show that such cards are on sale for mere mortals. So basically you have ATI(AMD) and Nvidia for external graphics, and maybe Matrox in niche markets. Intel and S3 and others are only available as onboard options.
- The OS of choice (BSD, Linux, Windows)
While I agree with this sentiment, we can't test everything. I think that we should agree on a standard test OS. That OS needs to be freely obtainable to make the testing bar very low.
So FreeDOS would be OK for you? Small, free, and it will probably run just fine even if major parts of the board are malfunctioning because the port is unfinished.
Frankly, I don't think that we are aiming for average users at this time. We are aiming for the developers that can advocate coreboot's use in OEM hardware or folks who see enough of an advantage to pay for a port to their hardware. These are very sophisticated users.
But OEMs won't care unless we can make average users happy. If you want an OEM to ship coreboot by default, you better make sure the OEM won't fall into the Linux netbook trap where people returned lots of machines because they expected to be able to run Windows applications. OEMs are also extremely cost-conscious, and if they can save a few cents per board without risk, they will do it. The key words are "save money" and "no risk". Given the per-mainboard licensing cost for BIOS, coreboot can serve as an alternative if there is no risk involved.
Regards, Carl-Daniel
On Sun, Oct 3, 2010 at 6:22 PM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
On 04.10.2010 02:11, Warren Turkal wrote:
On Sun, Oct 3, 2010 at 10:58 AM, phorsyon phorsyon@gmx.net wrote:
I consider myself as enthusiast and would like to share my thoughts about coreboot certification and PR as a person from outside the coreboot project. I hope this will help you to see this project from another perspective and therefore helpful for your plans and further actions. Let me say I would really appreciate it, if vendors would ship their products preloaded with coreboot. ;-)
We are not vendors, and AFAIK no hardware OEM vendors ship with coreboot pre-loaded.
I think a few vendors ship some systems with coreboot if you ask nicely, but they are either in the server or embedded space, not general consumer space.
This is interesting news for me. I think that's really great.
To make coreboot popular, it needs (a) a feature people want, (b) major vendors to deliver it and (c) available documentation for all components to make it work with most boards. These requirements are interdependent and must be solved as whole. To get c vendors must be convinced to want b and therefore coreboot needs a. So what's a? It surely is fast boot up, but depending on what your target audience is, it might also be boot non-free OSes and works with proprietary drivers. If you want to make vendors want b it's definitly required to target the average user.
I do not believe that (c) is strictly required in that you don't need detailed bios developer guides once the code is available. It would be useful, but I don't see it as being required.
We also need to be realistic. OEMs are not taking up coreboot in mass right now. We need to set the certification bar at the point that it would be useful for the coreboot project. I believe that setting the certification bar allow a more rapid development of coreboot code would be much more productive that pie-in-the-sky arguments about what to do with all the OEMs that want to ship coreboot.
OEMs want fire-and-forget solutions without functional regressions. coreboot can serve as fire-and-forget solution, but if we certify consumer mainboards which can't run Windows, the certificate will be essentially worthless.
I agree that OEMs would want that. However, I don't think that all the coreboot developers could commit to that level of testing right now. AFAICT, testing of any kind is very difficult to get done if I don't own the affected hardware.
Remember the complaints about "Designed for Windows Vista" and the machines which were too slow to be useful? People still remember that marketing disaster years after it happened. If we ever certify some hardware without caring about Windows, we should make that explicit, and call it a feature. "Coreboot certified board for the anti-closed-source movement, will refuse to run Windows".
First of all, I think you are attacking a straw man. My point has nothing to do with being anti-closed-source. My point is that I don't own a license for Windows and wouldn't be able to do the testing necessary if that's a gate for certification. I have no problem with a machine running Windows (or any other OS), but I don't think that should be needed for for minimum compliance.
The variant which can run Windows would be "Coreboot certified board". Except for niche vendors, nobody will care about the anti-closed-source certification, making it moot.
I just think that bar is too high to leverage the community of folks that would like to see and even help coreboot progress.
Vendors do not ship coreboot yet.
AFAIK Artec, Tyan, Technexion, MSI and some other vendors ship coreboot for specific systems on request. Not sure about the current state, but AFAIK they did that in the past.
With all due respect, I don't think this information has any bearing on my position.
I believe that we only have the community at this point. Given that fact, we need to do what we can to get vendors interested. I believe that maturing the coreboot codebase is probably the most effective way to do that.
Booting Windows 7 on recent consumer boards is probably the best way to show vendors that coreboot is a viable BIOS replacement. This needs porting work, and of course it needs testing.
How do I show booting Windows 7 for a board I am working on? If I can't because I don't have a license for Windows, should I not be able to get the coreboot certification?
For the record, I believe that Linux probably has a mature enough ACPI stack and other systems to be pretty good barometer of compliance short of having an independent test suite.
Ummm... Linux will tolerate absolutely crappy ACPI code, and not even complain loudly if it has to fix stuff. Given the current market share of Linux, there is no money in verifying ACPI code under Linux for any consumer mainboard.
Fair enough, but it's certainly got to be better than not running any OS on the machine as it does demonstrate some level of functionality.
What about calling it "Coreboot: Developer's Choice". Also freely available documentation would be a nice core requirement for that.
I actually don't like "minimal." However, I also don't like "Coreboot: Developer's Choice." What would you think about "Coreboot beta test"?
For me, "Developer's choice" implies that the board already works perfectly fine, and has all the hardware features which make it easy to test new coreboot versions. I believe in honest advertising, and if the machine barely works, we should not certify it, but we could list it as "fun project if you want to finish a port".
I agree that "Developer's choice" implies too much functionality. :) However, I still think we need a way to invite more people and orgs into the project. I think that we need some way to say that certain boards are a good place to start for those looking to contribute. Ideally, one day we won't even need this level of certification as everything will just ship with coreboot. :)
- Add-on components most importantly Nvidia/ATI cards
I don't believe that we should bias toward some particular class of add-on card or some vendor of add-on card.
Nobody is going to test with Tseng Labs graphics cards because they are no longer on sale (actually, they haven't been on sale anytime in the last 12 years). Nobody is going to test with Intel graphics cards either unless you can actually show that such cards are on sale for mere mortals. So basically you have ATI(AMD) and Nvidia for external graphics, and maybe Matrox in niche markets. Intel and S3 and others are only available as onboard options.
I have an ATI PCIe card and would likely use that to test, but requiring both ATI and Nvidia out of me is a little much.
- The OS of choice (BSD, Linux, Windows)
While I agree with this sentiment, we can't test everything. I think that we should agree on a standard test OS. That OS needs to be freely obtainable to make the testing bar very low.
So FreeDOS would be OK for you? Small, free, and it will probably run just fine even if major parts of the board are malfunctioning because the port is unfinished.
To be clear, I think you are attacking another straw man. I specifically suggested Debian of some form. I may not have been clear with my reasons, so here they are: it does have a chance of having drivers for the majority of the hardware, it uses more sophisticated hardware initialization techniques, it's a well supported dev environment for coreboot, and it's easy to find and download a legal copy for free.
If there is software that can verify ACPI and other needed functionality from FreeDOS, maybe it would make a good platform. I honestly have no idea.
Frankly, I don't think that we are aiming for average users at this time. We are aiming for the developers that can advocate coreboot's use in OEM hardware or folks who see enough of an advantage to pay for a port to their hardware. These are very sophisticated users.
But OEMs won't care unless we can make average users happy. If you want an OEM to ship coreboot by default, you better make sure the OEM won't fall into the Linux netbook trap where people returned lots of machines because they expected to be able to run Windows applications.
I am not denying that there is a demand for Windows. I am simply saying that I would like to be able to participate in the certification of board, and I don't have a copy of Windows to test with.
OEMs are also extremely cost-conscious, and if they can save a few cents per board without risk, they will do it. The key words are "save money" and "no risk". Given the per-mainboard licensing cost for BIOS, coreboot can serve as an alternative if there is no risk involved.
Makes sense. However, I think that OEMs that need Windows can be sure to certify their systems for Windows.
Also, I don't think that Windows support should be part of the base certification as it alienates a number of people who are really interested in seeing and helping coreboot succeed.
Also, I think that we should start pretty minimally as definitions can be changed as time goes on, but I don't want us to bite off more than we can chew and just abandon the effort as a result.
Thanks, wt
Am 04.10.2010 07:33, schrieb Warren Turkal:
How do I show booting Windows 7 for a board I am working on? If I can't because I don't have a license for Windows, should I not be able to get the coreboot certification?
That's the difference between a certification that's useful for vendors and one that's useful for computing enthusiasts.
Whoever produces and sells boards will have some Windows 7 license (and probably the dev builds to improve testing, as well as the Microsoft test kits) lying around.
That means, certification would be for different purposes.. a "coreboot + Linux" certificate would state that the board is actually useful beyond freedos (eg. networking works, HPET is around, ACPI is at least somewhat useful, even if Linux's ACPI interpreter is more forgiving than perl)
A "coreboot + Windows" certificate could build onto that, stating that Windows operation was tested, too. Stacking them this way would ensure that Windows support doesn't break Linux (or any other free OS).
But vendors will (except for some specialty shops or special customer requests) require the latter - with no regard for the former, except maybe to assess how much work they'd have to put in/sponsor to make a coreboot port Windows compatible.
As a side note on "Windows compatible", I'm not 100% sure on this, but I think the "Designed for Windows" set of certificates by Microsoft handle firmware behaviour, too.
Patrick
On 04.10.2010 02:11, Warren Turkal wrote: *snip*
What about calling it "Coreboot: Developer's Choice". Also freely available documentation would be a nice core requirement for that.
I actually don't like "minimal." However, I also don't like "Coreboot: Developer's Choice." What would you think about "Coreboot beta test"?
For me "beta test" sounds like almost done, but needs testing. What about simply calling it "Coreboot: Under development". I can't imagine it to be printed on a box though, but as you said your're focussing on the community, at least for now, a list in the wiki showing all certified products will be more appropriate.
*snip*
This mark should assure that all parts of the board work, meaning every feature, expected by the average user, hhis is important to please them. So that this mark actually means something to them. If they have a good reason to care about it, also the vendors have one, and this would give the coreboot project the authority dictate such high standards.
Things normal users care about to work:
- All build-in components (net, snd, gfx)
I should have explicitly included this in the standard certification level.
- All ports/expansion slots (exceptions: rs232, parallel, floppy)
Already said this in the standard certification level except that I didn't make any exceptions. It's sloppy to leave physical and header ports not working.
This probably was a little unclear. I meant to list what users expectations are, but not what criteria for certification should be. From a technical perspective I totally agree that every connector/header should be funtional.
- Everything related to power management as supported by the underlying
hardware and drivers (All power states, ACPI). Also needed to improve drivers.
For the record, I included this in the "better ACPI support" section of the standard certification. The OSPM part of ACPI includes all power states of all parts of the system.
- Add-on components most importantly Nvidia/ATI cards
I don't believe that we should bias toward some particular class of add-on card or some vendor of add-on card.
As Carl-Daniel stated, there are practically just ATI/AMD and Nvidia. And I also agree that it can't be expected every developer has both to test with. But you are not the only dev, it's possible to delegate test to each other and probalby to kreep track of that in the wiki. There is a testsystem: http://www.coreboot.org/Distributed_and_Automated_Testsystem I have not digged into it, but it's probably possible to extend it for semi- automatic testing. So that if dev A has written code which also needs testing for hardware only dev B has access to, dev A can call for dev B to test it. To scale well such a system would require information about who has which hardware to run tests on. This would allow for automatic notification.
- The OS of choice (BSD, Linux, Windows)
While I agree with this sentiment, we can't test everything. I think that we should agree on a standard test OS. That OS needs to be freely obtainable to make the testing bar very low.
If that's for the development certification, I totally agree. I always had the consumer in mind.
- fast booting
I think this might be a little too subjective for certification. What if a server vendor wanted to ship coreboot firmware that does a longish running operation everytime before booting. Would that never qualify for a coreboot certification?
As mentioned above it's meant as user expectations not certification criteria.
Things normal users don't care about:
- most legacy stuff like rs232, parallel, floppy
As stated above, I think that leaving ports with physical or header connections nonfunctional is just sloppy, and it would not reflect well on the project to allow board in that state to get a standard certification.
Agreed
- pro features like *PXE, AoE, iSCSI (Those could be combined under a
logo like "Coreboot for Professionals")
I agree about things like PXE, AoE, and iSCSI being more important to big iron. I'm not sure that we should have another certification level to support them right out of the gate, however. More certification levels is more confusing.
Probably having these listed as separate features would be best.
*snip*
Keeping track of detailed information in the wiki is a good thing. If vendors decide to deliver coreboot it should be as easy as possible for them.
I am not sure this data is simple enough for wiki. However, I haven't given this too much thought.
It defintely would be helpful for new developers and leave a good impression to verndors/OEMs. Right now it's not that easy for an outstanding person to tell what works and how well (also e.g. has freely available documentations or not). So improving infrastructure on this side will pay off, I guess.
I think of a combination of the automatic testing + semi-automatic testing as described above and a status site to keep track of this details. For example board $FOO has this components where this where tested an that not. Also additional test like with works Nvidia or ATI GPU could be introduced. This system could deliver all the status information needed, so anyone could tell where the project stands. The lazy evaluation would allow to make good use of the rare hardware around the developers. It's also an invitation for testers only, as they would always know what still needs testing and do the test if they find the time. This eliminates the need to setup the full automatic testsystem, which apparently aims for vendors only. So you may want to think that through. ;-)
*snip*
On Monday 04 October 2010, Patrick Georgi wrote:
Am 04.10.2010 07:33, schrieb Warren Turkal:
How do I show booting Windows 7 for a board I am working on? If I can't because I don't have a license for Windows, should I not be able to get the coreboot certification?
That's the difference between a certification that's useful for vendors and one that's useful for computing enthusiasts.
Carl-Daniel and myself were refering to the former one and Warrens seems to focus on the latter one. What do you think about doing one after another? Start off with minimal certificate to attract more developers and call it e.g. "Coreboot Beta Test" or "Coreboot under development". And at some later point, if OEMs got interested, go for a consumer certificate like "Coreboot compatible" or "Coreboot certified". The latter one should focus on the average user's needs/expectations, delivering a fire-and-forget solution for OEMs. The minimal certificate won't need to be printed on a box, as a product list on web would do fine. This would also prevent later confusion due to having more than one certificate logo out in the wild at the same time, as minimal and consumer certification time periods may overlap.
Whoever produces and sells boards will have some Windows 7 license (and probably the dev builds to improve testing, as well as the Microsoft test kits) lying around.
True, the OEMs are likely to help out on that matter.
That means, certification would be for different purposes.. a "coreboot
- Linux" certificate would state that the board is actually useful
beyond freedos (eg. networking works, HPET is around, ACPI is at least somewhat useful, even if Linux's ACPI interpreter is more forgiving than perl)
A "coreboot + Windows" certificate could build onto that, stating that Windows operation was tested, too. Stacking them this way would ensure that Windows support doesn't break Linux (or any other free OS).
A "Coreboot + $OS" logo could give a bad impression, as it will look like coreboot can only boot a specific OS. So a minimal and a consumer version of a certificate, where only the latter one would be printed on the box would have a more to-the-point expression to others.
But vendors will (except for some specialty shops or special customer requests) require the latter - with no regard for the former, except maybe to assess how much work they'd have to put in/sponsor to make a coreboot port Windows compatible.
As a side note on "Windows compatible", I'm not 100% sure on this, but I think the "Designed for Windows" set of certificates by Microsoft handle firmware behaviour, too.
This certaintly has the potential to threaten a coreboot certification. Even if there's no criteria now, Microsoft could add one to prevent OEMs to certify for both Windows and Coreboot at the same time. When the time comes for a coreboot consumer certificate some investigation on that matter is needed too.
Patrick
On Mon, Oct 4, 2010 at 10:50 AM, phorsyon phorsyon@gmx.net wrote:
This certaintly has the potential to threaten a coreboot certification. Even if there's no criteria now, Microsoft could add one to prevent OEMs to certify for both Windows and Coreboot at the same time. When the time comes for a coreboot consumer certificate some investigation on that matter is needed too.
MS told me five years ago they'd be happy to certify coreboot as long as we changed the name from linuxbios.
They have nothing against coreboot in principle -- quite the contrary.
ron
On Monday 04 October 2010, ron minnich wrote:
On Mon, Oct 4, 2010 at 10:50 AM, phorsyon phorsyon@gmx.net wrote:
This certaintly has the potential to threaten a coreboot certification. Even if there's no criteria now, Microsoft could add one to prevent OEMs to certify for both Windows and Coreboot at the same time. When the time comes for a coreboot consumer certificate some investigation on that matter is needed too.
MS told me five years ago they'd be happy to certify coreboot as long as we changed the name from linuxbios.
They have nothing against coreboot in principle -- quite the contrary.
ron
Well that's at least one thing less to care. LinuxBIOS wasn't a good name anyway. BIOS just sounds so "legacy". ;-)
phorsyon wrote:
a minimal and a consumer version of a certificate
As was mentioned, the more certifications there are, the less easy it is for the market to make use of them.
I don't think we can afford to try to market two different certifications. I would only like to try for one; "consumer coreboot" or rather; "coreboot complete"
Developers don't want problems any more than average users just because they may know how to deal with problems.
Anyway, if we would have criteria then we could also track them.
Any interested developer could easily discover what was missing for a board to be coreboot complete, judge if it is a good choice for them at the moment, and if not just look for completeness of other boards.
//Peter
On Mon, Oct 4, 2010 at 12:23 PM, Peter Stuge peter@stuge.se wrote:
phorsyon wrote:
a minimal and a consumer version of a certificate
As was mentioned, the more certifications there are, the less easy it is for the market to make use of them.
I don't think we can afford to try to market two different certifications. I would only like to try for one; "consumer coreboot" or rather; "coreboot complete"
Developers don't want problems any more than average users just because they may know how to deal with problems.
Anyway, if we would have criteria then we could also track them.
Any interested developer could easily discover what was missing for a board to be coreboot complete, judge if it is a good choice for them at the moment, and if not just look for completeness of other boards.
I agree with Peter here, and will add my $0.02 to the thread...
Certification is a *huge* process, at least if you want it to mean anything. I would expect that a true certification effort would rival development of the code itself. Multiple levels of certification only complicate the process and confuse users. And quite honestly, I don't expect that to happen unless a lot of dedicated resources are poured into it.
Perhaps the best thing is simply to publish a giant testing matrix for each board. Certify against the absolute bare minimum technical requirements, ie, can all on-board devices be initialized by firmware, are SMBIOS tables populated, etc. Leave it up to the system builder to decide whether or not if it's usable for the application.
Maybe someone (FSF?) wants to create a higher-level standard that includes Coreboot along with a full free software stack, but that shouldn't be key to Coreboot certification at any level or you risk alienating major commercial partners. Heck, everyone in the Coreboot community ought to be flattered if a major OEM ships a "Made for Win7" computer with Coreboot on it.
Certify stuff that you know, leave everything else up to vendors.
On Mon, Oct 4, 2010 at 10:59 PM, David Hendricks david.hendricks@gmail.com wrote:
On Mon, Oct 4, 2010 at 12:23 PM, Peter Stuge peter@stuge.se wrote:
phorsyon wrote:
a minimal and a consumer version of a certificate
As was mentioned, the more certifications there are, the less easy it is for the market to make use of them.
I don't think we can afford to try to market two different certifications. I would only like to try for one; "consumer coreboot" or rather; "coreboot complete"
Developers don't want problems any more than average users just because they may know how to deal with problems.
Anyway, if we would have criteria then we could also track them.
Any interested developer could easily discover what was missing for a board to be coreboot complete, judge if it is a good choice for them at the moment, and if not just look for completeness of other boards.
I agree with Peter here, and will add my $0.02 to the thread... Certification is a *huge* process, at least if you want it to mean anything. I would expect that a true certification effort would rival development of the code itself. Multiple levels of certification only complicate the process and confuse users. And quite honestly, I don't expect that to happen unless a lot of dedicated resources are poured into it.
Agreed.
Perhaps the best thing is simply to publish a giant testing matrix for each board.
This would certainly be a huge improvement. Maybe this should be our actual first step?
Certify against the absolute bare minimum technical requirements, ie, can all on-board devices be initialized by firmware, are SMBIOS tables populated, etc.
I actually fully agree with this. However, we would need some way to tell if things like the SMBIOS tables are populated. That means that we'd need some form of software to run after the coreboot step to verify. Since we don't have a piece of software that is only that, how do we test for certification? I see a couple options: * don't certify anything until we develop a coreboot confirmation payload of some type * certify based on some software stack that we can currently use
Given that I don't see a coreboot confirmation payload being developed overnight, I would think that we should start with some software stack that we can currently use. In my eyes, this means using some easy to obtain Linux distribution and a bit of software to test compliance with some set of standards.
How would others solve the problem of testing for compliance?
Leave it up to the system builder to decide whether or not if it's usable for the application.
Agreed.
Maybe someone (FSF?) wants to create a higher-level standard that includes Coreboot along with a full free software stack, but that shouldn't be key to Coreboot certification at any level or you risk alienating major commercial partners.
Agreed. I wasn't trying to place a higher-level standard like "Works with Linux." I was trying to find the lowest barrier way to test for some minimum level of compliance.
Heck, everyone in the Coreboot community ought to be flattered if a major OEM ships a "Made for Win7" computer with Coreboot on it. Certify stuff that you know, leave everything else up to vendors.
I think this is one of the best perspective statements in this thread so far. :)
Thanks, wt
Hi,
Maybe good start to use some existing test suite
http://www.h-online.com/open/news/item/Ubuntu-Kernel-Developer-releases-Firm...
http://smackerelofopinion.blogspot.com/2010/08/firmware-test-suite-biosacpi-...
https://launchpad.net/~firmware-testing-team http://linuxfirmwarekit.org/
This is based on http://linuxfirmwarekit.org/
Thanks, Rudolf
Does that software kit run in Linux? Is it relying on Linux's ACPI implementation some how?
Thanks, wt
On Tue, Oct 5, 2010 at 1:59 AM, Rudolf Marek r.marek@assembler.cz wrote:
Hi,
Maybe good start to use some existing test suite
http://www.h-online.com/open/news/item/Ubuntu-Kernel-Developer-releases-Firm...
http://smackerelofopinion.blogspot.com/2010/08/firmware-test-suite-biosacpi-...
https://launchpad.net/~firmware-testing-team http://linuxfirmwarekit.org/
This is based on http://linuxfirmwarekit.org/
Thanks, Rudolf
-- coreboot mailing list: coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Hi,
I used it like year ago. It is basically some linux program that is doing various checks using tools or kernel.
Someone needs to try out agin.
R.
Making your own certification is hard. Using someone else's test is easier. Maybe we should invert the problem. Have a page which shows, for each board, what suite of tests it passes under coreboot.
- boots vista - boots windows 7 - boots linux - runs intels firmware validation
and so on.
ron
Wondered if you guys have seen this:
http://www.h-online.com/open/news/item/Intel-releases-GRUB-based-BIOS-test-s...
-Anders
Den 05-10-2010 18:34, ron minnich skrev:
Making your own certification is hard. Using someone else's test is easier. Maybe we should invert the problem. Have a page which shows, for each board, what suite of tests it passes under coreboot.
- boots vista
- boots windows 7
- boots linux
- runs intels firmware validation
and so on.
ron
* Anders Jenbo anders@jenbo.dk [110301 07:27]:
Wondered if you guys have seen this:
http://www.h-online.com/open/news/item/Intel-releases-GRUB-based-BIOS-test-s...
Did anyone work on getting this to work on coreboot?
Stefan
My .02 from an end user (wannabe) pov:
In general I agree with Warren Turkal:
I'd like a certification that ensures coreboot and free sofware work with the hardware, irrelevant of what happens with non-free software. I don't mind having other certification (tags) for non-free software for whoever cares about it. What I would regret having less hardware certified because there wasn't a certification option for free software uses. Coreboot is good for many things and one of them is getting rid of legacy where possible. Requiring everyone to support legacy just because it is a current big market segment will only perpetuate this legacy burdened market. The argument works exchanging non-free software with legacy, only that non-free software is nastier than legacy. So legacy, nonfre-software should be optional for certification. Free software support should be required for any certification, then some would also tag windows7 or whatever.
It would require some credible organization doing the tests, I guess. Or maybe it would be a kind of trust system ? (the different certificates and their exact meaning would be clear, then anyone could certify some hard and customers would decide whether they believe a certificate signed by one or other party, or even self signed by a vendor).
The tests should be reproducible, so the exact versions of kernel, distribution, etc. should be stated, even if a particular choice is not a requirement, documenting the choice tested should be.
There should be non-technical requirements too: - support code GPLed and integrated - maybe security criteria ? I don't know about it - something on availability of the tested hardware ?
A problem I can see is what gets certified precisely . a mainboard? a whole computer ? whatever as long as it is clearly stated ?
My experience is that of a consumer trying to buy hardware to use with coreboot. I picked every component in my new PC according to price, the performance I wanted and the information on coreboot.org, more or less accepting the risk for the parts apparently less well supported.
And I mostly got what I expected except partly with the CPU. I would have liked more precise lists of what CPU models have been tested. I can't be sure but I suspect that my particular CPU wouldn't work on any supported mainboard right now (well, not even the shipped BIOS worked until I upgraded it). I've found out that the information on documentation of the CPU was accurate, an so it is just a matter of time/effort until it works, but the site gave the impression that any AMD CPU would work (there were lists of mainboards and chipsets, but the CPU part was not very detailed, beyond brand or family). Then looking at the code I found some revisions didn't even have a position in a bitfield.
What I mean is that if it is a board that gets certified, then a customer may buy it and put a new CPU there that doesn't work with coreboot. It may happen with other hardware, but I can't think of any example. Maybe RAM ?
On Sun, Oct 03, 2010 at 02:15:44AM -0700, Warren Turkal wrote:
On Sat, Oct 2, 2010 at 5:06 PM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
[...]
Well, if we only care about Linux, you can avoid most (if not all) of ACPI on many machines, and you can avoid SeaBIOS as well. Heck, you could even avoid FILO and require a Linux kernel in flash. Whether such ab board would be usable for end users is a totally different question.
So what ? If a consumer can buy a system with coreboot and gnu and linux that works and gives her all due freedoms why should she care for ACPI,etc?
I understand most users will want more, I just wouldn't rule out a certificate option unrelated to legacy or current standards. Maybe some vendor wants to offer some advanced features that is too cumbersome to adapt to ACPI... Or open hardware vendors may opt for just skipping ACPI and just giving the hardware details and linux drivers...
Booting into a certain OS is clearly not the only bar that we should be looking for. It would need to comply with various standards like ACPI, etc.
It might be one of the bars. On the other hand ACPI compliance could be certified by the ACPI consortium (there is one, I guess...), maybe . Likewise for any other standard.
IMHO being able to install a Linux distribution from CD is an absolute must even if you only target professional users in clusters etc.
I agree.
I agree one of the certification options should require it
Windows support means a board is usable by the general population, and this is something vendors care about deeply.
And is something we don't need to help stay the same. So I say require it only for some of the optional tags.
We renamed LinuxBIOS to coreboot exactly because people said all the time "I don't want Linux", and EFI marketing would love to make fun of us if we ever said "Linux only is good enough for certification".
Laughing is healthy. Let them laugh. I don't see the marketing problem if there are windows tags.
I believe that most of the folks who care enough to use Coreboot at this time are probably running Linux or some other free OS. I also believe that most developers who have access to systems are probably running on or have easy access to some free OS. I also know that non-free OSes are not easily available to everyone. It's not that I believe that we can't test for non-free software support. It's just that I believe that certifying the boards for Coreboot should not gate on that having been done.
Well, if A wants B to certify that coreboot supports OS W on hardware H I guess A should contribute any changes to coreboot and give B the hardware H and the necessary W licenses, if A can find some B willing to use W. If W was free software the requirement would be the same but it'd be much easier to comply with. If W license was particulary nasty it might even be impossible to use W for testing coreboot.
But A shouldn't be required to have any deal with windows to certificate coreboot works in H without windows. SO corebot+windows certification should generally be more expensive/cumbersome than coreboot+freesoftware certification only (depending on the case, maybe coreboot+hurd may be difficult to certify...). Sounds logical.
Ideally, having some OS independent test suite would be the best approach I think, but I don't see that getting developed overnight. :)
Is that feasible at all?
Maybe we should have multiple compliance levels like "Coreboot minimal" and "Coreboot standard"? The minimal could be some set of requirements. The standard could be the minimal requirements plus some extra set of requirements. We could also have tags for the compliance. These could be used to indicate special support like Windows or something like that.
Very reasonable.
- ACPI
- DSDT should exist
Is that needed to develop coreboot further ? Not sure it should be required for minimal. Maybe split minimal and minimal + DSDT
- Able to load VGA option ROM
unless they find a way not to use it, like free framebuffer drivers or so on ? If propietary software is not a requirement, propietary firmware shouldn't either, whenever it can be worked around.
- Able to boot into an OS not stored as a coreboot payload (e.g. boot
from CD, USB, or SATA drive). For the minimal certfication, I believe that OS should be Debian since it's a well supported environment for developing coreboot.
I would simply say that it should boot a system that allows coreboot development using only free software, and it should document which precise system, version, etc. it is. Of course depending on what it is it may be easier or harder to find certifiers (or customers once certified).
And then we come to the thorny issue of what does "only free software" mean, since debian for instance includes kernel blobs or similar issues...
Possible tags: MSWin{XP,7} ReactOS GPXE AOE ISCSI ...
The process for adding tags to the certification criteria should be clear and simple. We won't be able to find all useful tags from the start.
Something about power saving capabilities would be useful, not sure what or how...
We should probably note the known revisions that are compliant for each board. This doesn't mean that every revision that would pass the requirements needs to be listed. Only tested versions should be listed. Those revisions would be the recommended revisions with users using other revisions at their own risk. I could imagine something like the following for a mobo: minimal: 11 99 103 150 standard: 50 75 135 155 minimal+Win7: 77
Ok. Adding the rest of hardware and software tested, like Debian lenny , AMD Phenom II X4 910e, such and such DDR3 DIMMs, such VGA... The more diverse hardware and software tested the more work to certify it, and the more tempting for consumers, who would evaluate the risk of using it with other hardware or software. One could risk buying a certified system with different amount of RAM than tested and install a different distribution, etc.
Frankly, I think that ability to use the free drivers should be good enough. We shouldn't be hold out any kind of coreboot certification on the condition that non-free drivers work.
ACK
There are two aspects of the problem:
- We can't test everything (fact of life)
- Closed-source drivers have a huge market share, and won't go away any
time soon
Agreed. I also think that some developers won't be able to test certain non-free software. For instance, I wouldn't be able to test that Windows 7 boots. I also don't have an Nvidia card to test their drivers.
And if you test and find that it doesn't work with non-free drivers, that is an answer, but not very productive, is there a problem in some initialisation in coreboot or in the driver? You can't fix the driver, so supporting it is a tall order, so optional.
If the code's not merged into SeaBIOS, we shouldn't certify the build.
Yes. Not sure SeaBIOS is needed for the most basic certification, but in general if any support code is not commited then certification should wait. Not sure this is a huge problem, I'd say one wants to benefit from the community testing stuff before paying or otherwise causing someone to thoroughly test for certification. So waiting until a little after committed would be wise. Although I guess hardware is obsoleted quickly and vendors may want certification before they put it on the market... so I don't know how much of a burden this is, but I think it should be required. If Google Summer of Code candidates where required to send patches, vendors should be required to have support code good enough for being merged.
Sorry for being verbose.
On Sun, Oct 3, 2010 at 12:28 PM, xdrudis xdrudis@tinet.cat wrote:
Booting into a certain OS is clearly not the only bar that we should be looking for. It would need to comply with various standards like ACPI, etc.
It might be one of the bars. On the other hand ACPI compliance could be certified by the ACPI consortium (there is one, I guess...), maybe . Likewise for any other standard.
I think that we should require ACPI compliance as it is an integral part of modern computing on X86 hardware, and it makes enumeration significantly more efficient or just plain possible in some cases.
Also, I think that we need to be quite careful about overloading our users with information. Otherwise, out users will be overwhelmed and will not understand everything that we try to communicate with them.
Thanks, wt
Warren Turkal wrote:
- SeaBIOS source code is available
- SeaBIOS code is merged into SeaBIOS git
Yes. Doesn't this imply the previous item?
Heh, no. It's easy to think so, but don't fall into that trap.
Certification requires a fair amount of QA which isn't being done right now is just my 2.
//Peter
On Sat, Oct 2, 2010 at 9:20 PM, Peter Stuge peter@stuge.se wrote:
Warren Turkal wrote:
- SeaBIOS source code is available
- SeaBIOS code is merged into SeaBIOS git
Yes. Doesn't this imply the previous item?
Heh, no. It's easy to think so, but don't fall into that trap.
I mean that if we accept that the code in the SeaBIOS git repo is complete for a given board, can we assume that the code is available?
wt
-
Anders Jenbo -
Carl-Daniel Hailfinger -
David Hendricks -
Patrick Georgi -
Peter Stuge -
phorsyon -
ron minnich -
Rudolf Marek -
Stefan Reinauer -
Warren Turkal -
xdrudis