On 04/22/2015 01:28 AM, Gerd Hoffmann wrote:
Hi,
If the community would like me to stop sending these notifications I will do so; I could also adjust the maximum frequency if desired.
How about sending out messages on status changes (i.e. edge not level triggered). That should avoid annoying repeated reports on the same failure.
Also mails clearly need to be improved so they are useful to people like me who don't know your test system in detail. I'm caring about qemu support in coreboot, so I had a look at the log. On a quick scan nothing unusual was outstanding, except the big fat line saying the boot test was successful. The email itself just says "test failure" without any specifics, so I don't have an idea what to look for in the log. That is not helpful.
cheers, Gerd
Understood. I did not have any real data until now on how failures would be dealt with by the coreboot community; it appears that failures may last for a lot longer in the master tree than I originally anticipated, therefore some adjustment in how the system notifies the community is needed. I still prefer some form of nag message so that the failure is not ignored; it is my experience that issues in open source tend to languish / be forgotten without persistent reminders. Right now the system is set to 1 day minimum interval; if this is still too fast it can be dialled back further. I'd like input from others before changing this setting (or switching to "edge triggered" notification).
I have also been considering how to improve the failure messages; a failure to boot is pretty obvious in the boot log, however as you say, verification failures are poorly noted. Much of this is due to my original assumptions on the quality of the mainline GIT tree at any given instant; I will extend the system to note exactly which tests have failed in order to avoid future confusion.