Dear LinuxBIOS community,
it's about time to release the automated and distributed test system for LinuxBIOS.
The test system creates a framework to run images created by the automatic build system (known from http://snapshots.linuxbios.org/ a.k.a http://qa.linuxbios.org/) on real hardware.
So from now on LinuxBIOS binary images can be downloaded from http://qa.linuxbios.org/ that are known to work on at least one mainboard of the given type.
I started a page gathering all the information about the test system in the Wiki. Please help completing it: http://linuxbios.org/Distributed_and_Automated_Testsystem
Get a short overview on the test system from the slides of the talk I held at the LinuxBIOS symposium 2006: http://www.coresystems.de/PDFs/LinuxBIOS-testing/Slides-LinuxBIOS-QA.pdf
Since nobody is supposed to have the 50+ mainboards supported by LinuxBIOSv2 at a single place, the system works distributed and single boards can be hooked up to the test system with relatively cheap components.
So please do understand this announcement as a call for participation. This test system is only of value if we get many boards tested on a regular basis.
Are you working for a mainboard or system vendor that supports LinuxBIOS? Are you a developer that has a spare board supported by LinuxBIOS sitting at your office? Then please have a look at the Test Integration Manual to find out how you can help the LinuxBIOS project to improve its quality: http://www.coresystems.de/PDFs/LinuxBIOS-testing/TestIntegrationManual.pdf The Test Integration Manual also explains how you can add new tests and test components to the test system.
Are you developing LinuxBIOS, and just want to know how to use the test system without setting up your own dedicated test hardware? Make sure a bios image you build works on real hardware? Then have a glance at the Developer's Manual: http://www.coresystems.de/PDFs/LinuxBIOS-testing/DevelopersManual.pdf
An exercise that some of you might know from safety critical and embedded software is test and project documentation. Basically the goal should be to have (at least) one test for each requirement of the software system.
There are two IEEE standards for writing specifications for such requirements and test specifications:
IEEE Std 829-1998: IEEE Standard for Software Test Documentation IEEE Std 830-1998: IEEE Recommended Practice for Software Requirements Specifications
I wrote a requirements specification for LinuxBIOS which is rather incomplete and only scratches the top of the iceberg. (Not to mention that not all of the requirements are properly tested, while some tests dont have a requirement yet) While I tried to keep close to the standard, there are some differences and inaccuracies that might be fine for most projects, unless LinuxBIOS is taking a relevant part in a mission critical application that requires strict compliance (in which case it should be easy to fix that)
Find the documents here: http://www.coresystems.de/PDFs/LinuxBIOS-testing/RequirementsSpecification.p... http://www.coresystems.de/PDFs/LinuxBIOS-testing/TestSpecification.pdf
Any ideas, add-ons, patches, hints, flames, questions are of course welcome.
Again: If you have a chance to participate in the LinuxBIOS testing, please have a glimpse at the Test Integration Manual and contact me to get things going.
Best regards, Stefan