Hi Patrick, thanks for picking this up. From a high level perspective this is very simple. I try to sum it up to help find an existing framework, which I tried several times but failed. When ignoring distributing binaries or blobs and only focussing on test reports you have something similar to a review application.
We need a) User management and authentication b) Users being able to add new "products". In our case that's a mainboard variant with a specific hardware configuration (memory, hard-disk, PCIe cards, ...). Every product has a custom set of properties you want to evaluate and for which you can send in a test report. That could be - boots OS x - device Y detected and properly configured - register Z locked and secure boot is possible c) For every product in the database authenticated users could then push status reports (reviews) which supply a test result for the properties defined earlier. The status-report would be submitted for a specific coreboot version and build config. You could 'add', 'list', 'sort', 'filter', 'delete' reports for "products" as well.
Regards, Patrick Rudolph B.Sc. Electrical Engineering System Firmware Developer
9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany Email: patrick.rudolph@9elements.com Phone: +49 234 68 94 188
Sitz der Gesellschaft: Bochum Handelsregister: Amtsgericht Bochum, HRB 17519 Geschäftsführung: Sebastian Deutsch, Eray Basar
Datenschutzhinweise nach Art. 13 DSGVO
Patrick Rudolph B.Sc. Electrical Engineering System Firmware Developer
9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany Email: patrick.rudolph@9elements.com Phone: +49 234 68 94 188
Sitz der Gesellschaft: Bochum Handelsregister: Amtsgericht Bochum, HRB 17519 Geschäftsführung: Sebastian Deutsch, Eray Basar
Datenschutzhinweise nach Art. 13 DSGVO
On Wed, Jun 16, 2021 at 10:35 PM Piotr Król piotr.krol@3mdeb.com wrote:
On 6/16/21 8:46 PM, Patrick Georgi via coreboot wrote:
Hi everybody,
Hi Patrick, thank you for bringing this topic to ml.
There has been some talk recently in a smaller group where coreboot needs to improve the most in public perception, and how to get there.
Consensus has been that we're doing a pretty bad job at promoting all the hardware that we support in each coreboot version.
There's board-status which I started _years_ ago in the hope that somebody picks up the slack, but everybody has been busy, myself included. By now the collected information of the last 7.5 years is compiled into a 12MB HTML file that takes ages to render on moderate hardware (beware: https://www.coreboot.org/status/board-status.html https://www.coreboot.org/status/board-status.html), and the process to collect that data is mostly manual using pretty poor tooling. (Most of the links on that page don't even work anymore (which I'll fix) due to gitweb/cgit/gitiles changes on review.coreboot.org http://review.coreboot.org (and I only noticed by chance now).)
Meanwhile, there are several parties that boot test the hardware they care about regularly, with (often internal) information about how well coreboot does there.
On of this parties is 3mdeb and we publish regression 150 test results for v4.0.x and mainline on 6 platforms: https://docs.google.com/spreadsheets/d/1_uRhVo9eYeZONnelymonYp444zYHT_Q_qmJE...
We can't expect all those existing systems to converge into a single testing framework, but we could make it a single test result reporting framework.
This topic was discussed many times on various conferences and OSFW Slack. I believe contest aim to be that framework, please correct me if I'm misinterpreted something from OSFC'20.
Missing things from perspective of sending test reports from 3mdeb validation infrastructure is REST API definition that can receive required board status data.
For basic support maybe Qubes OS-like HCL would be good enough? https://www.qubes-os.org/doc/hcl/
Please note Qubes do not force people to do git commits as in case of old board status, what lowers the barrier for reporting. https://github.com/QubesOS/qubes-hcl
To this end, I invite people interested in that topic to chime in on this email thread so that we can discuss what we could do to provide a common place with information about which coreboot versions are bootable on which boards in a way that makes sense for everybody: users who are interested in such data as well as testers that already collect it but have no way to publish it.
As it was mentioned on coreboot leadership validation system most probably will need tweaks/modifications/improvements of build system.
Some future ideas that may came from build system could be:
- coreboot.org to host build results for given defconfig and make it
accessible to regular users - of course this would be vanilla coreboot since we agreed that in most cases production builds are different from what we have on coreboot.org. This lowers the barrier since regular users do not have to compile coreboot by themselves. Having confirmed, working binaries for given configuration would be huge win and step towards "stable releases", which I advertised in other discussions. 2. Firmware binaries could be delivered by fwupd/LVFS infrastructure This would largely help to reach more end users since by simple switch in fwupd they would be able to seamlessly deliver alternative firmware to their devices including updates.
Best Regards,
Piotr Król Embedded Systems Consultant GPG: B2EE71E967AA9E4C https://3mdeb.com | @3mdeb_com
coreboot mailing list -- coreboot@coreboot.org To unsubscribe send an email to coreboot-leave@coreboot.org