Hi Piotr, Patrick et al.,
actually, there is dedicated software to achieve exactly what you are talking about. I initiated the SystemTestPortal project [1] a while ago to provide an open source solution that focuses on simplicity and low entry barriers - you can watch my 2019 talk at FrOsCon [2] to get a first overview.
Currently, the project is stalled and, thus, the software is ready for production due to the lack of maintenance. However I plan tp resume the project soon. There are also other open source projects that provide similar solutions in this space. Yet, I must confess that to date I had a really hard time convincing developers to give our tool a serious try. While this might be due to bugs in the tool itself, I think it is mainly because many developers and project leaders don't buy into such a "community testing" approach as they think that the only right way to test software is through test automation.
I think that coreboot is a perfect example of a project that could benefit from such a tool (not necessarily our tool, but one from this category).
Cheers, Daniel
[1] https://www.systemtestportal.org/ [2] https://media.ccc.de/v/froscon2019-2359-testing_software_yes_please
-
Daniel Kulesz -
Patrick Rudolph -
ron minnich