[coreboot] QEMU x86_64 Q35 Automated Test Failure

[Timothy Pearson]

On 04/22/2015 01:28 AM, Gerd Hoffmann wrote:
>    Hi,
>
>> If the community would like me to stop sending these notifications I
>> will do so; I could also adjust the maximum frequency if desired.
>
> How about sending out messages on status changes (i.e. edge not level
> triggered).  That should avoid annoying repeated reports on the same
> failure.
>
> Also mails clearly need to be improved so they are useful to people like
> me who don't know your test system in detail.  I'm caring about qemu
> support in coreboot, so I had a look at the log.  On a quick scan
> nothing unusual was outstanding, except the big fat line saying the boot
> test was successful.  The email itself just says "test failure" without
> any specifics, so I don't have an idea what to look for in the log.
> That is not helpful.
>
> cheers,
>    Gerd
>
>

Understood.  I did not have any real data until now on how failures 
would be dealt with by the coreboot community; it appears that failures 
may last for a lot longer in the master tree than I originally 
anticipated, therefore some adjustment in how the system notifies the 
community is needed.  I still prefer some form of nag message so that 
the failure is not ignored; it is my experience that issues in open 
source tend to languish / be forgotten without persistent reminders. 
Right now the system is set to 1 day minimum interval; if this is still 
too fast it can be dialled back further.  I'd like input from others 
before changing this setting (or switching to "edge triggered" 
notification).

I have also been considering how to improve the failure messages; a 
failure to boot is pretty obvious in the boot log, however as you say, 
verification failures are poorly noted.  Much of this is due to my 
original assumptions on the quality of the mainline GIT tree at any 
given instant; I will extend the system to note exactly which tests have 
failed in order to avoid future confusion.

-- 
Timothy Pearson
Raptor Engineering
+1 (415) 727-8645
http://www.raptorengineeringinc.com