On 04/21/2015 11:31 AM, Patrick Georgi via coreboot wrote:
2015-04-21 17:31 GMT+02:00 Gregg Levinegregg.drwho8@gmail.com:
I suspect somehow it was supposed to be internal to hs outfit only.
Timothy provides boot testing of coreboot master with QEmu and a AMD board that he maintains as a service to the community.
And something changed with regards to the logic behind how those annoying e-mail messages being sent to us.
The system only sends mails when things look wrong. Right now coreboot is able to boot, but has issues with the cbmem console. According to Paul's analysis, that's why it's now reporting a failure.
Patrick
This is correct. Please see the thread marked "cbmem console broken on Intel hardware since ec5e5e0" for additional information on the regression in coreboot itself.
If the community would like me to stop sending these notifications I will do so; I could also adjust the maximum frequency if desired. As stated this is intended to be a service so that broken boards are detected and repaired; perhaps there should be an alternate way of reporting the test failure (e.g. the board status repository could be modified to store failed tests as well as successful ones?)
On 04/21/2015 11:44 AM, Timothy Pearson wrote:
On 04/21/2015 11:31 AM, Patrick Georgi via coreboot wrote:
2015-04-21 17:31 GMT+02:00 Gregg Levinegregg.drwho8@gmail.com:
I suspect somehow it was supposed to be internal to hs outfit only.
Timothy provides boot testing of coreboot master with QEmu and a AMD board that he maintains as a service to the community.
And something changed with regards to the logic behind how those annoying e-mail messages being sent to us.
The system only sends mails when things look wrong. Right now coreboot is able to boot, but has issues with the cbmem console. According to Paul's analysis, that's why it's now reporting a failure.
Patrick
This is correct. Please see the thread marked "cbmem console broken on Intel hardware since ec5e5e0" for additional information on the regression in coreboot itself.
If the community would like me to stop sending these notifications I will do so; I could also adjust the maximum frequency if desired. As stated this is intended to be a service so that broken boards are detected and repaired; perhaps there should be an alternate way of reporting the test failure (e.g. the board status repository could be modified to store failed tests as well as successful ones?)
After some thought I have limited the notification to a maximum of once per day. This is in line with the default nag settings of other projects such as Bugzilla. Developers wanting to verify that an issue has been fixed before the next 24 hour message window should refer to the board-status repository, which continues to update at the normal rate (within an hour or two of commit to mainline GIT).
On 21.04.2015 19:14, Timothy Pearson wrote:
On 04/21/2015 11:44 AM, Timothy Pearson wrote:
On 04/21/2015 11:31 AM, Patrick Georgi via coreboot wrote:
2015-04-21 17:31 GMT+02:00 Gregg Levinegregg.drwho8@gmail.com:
I suspect somehow it was supposed to be internal to hs outfit only.
Timothy provides boot testing of coreboot master with QEmu and a AMD board that he maintains as a service to the community.
And something changed with regards to the logic behind how those annoying e-mail messages being sent to us.
The system only sends mails when things look wrong. Right now coreboot is able to boot, but has issues with the cbmem console. According to Paul's analysis, that's why it's now reporting a failure.
Patrick
This is correct. Please see the thread marked "cbmem console broken on Intel hardware since ec5e5e0" for additional information on the regression in coreboot itself.
If the community would like me to stop sending these notifications I will do so; I could also adjust the maximum frequency if desired. As stated this is intended to be a service so that broken boards are detected and repaired; perhaps there should be an alternate way of reporting the test failure (e.g. the board status repository could be modified to store failed tests as well as successful ones?)
After some thought I have limited the notification to a maximum of once per day. This is in line with the default nag settings of other projects such as Bugzilla. Developers wanting to verify that an issue has been fixed before the next 24 hour message window should refer to the board-status repository, which continues to update at the normal rate (within an hour or two of commit to mainline GIT).
Thank you for providing this service. It is very much appreciated.
I would like to make a suggestion, though: The mail size of failure reports is increasing constantly as long as a boot problem remains unfixed. Would it make sense to only list the first 5 and last 5 commits in the failing series and replace the rest with "..." or something similar?
Regards, Carl-Daniel
On 04/21/2015 05:25 PM, Carl-Daniel Hailfinger wrote:
Thank you for providing this service. It is very much appreciated.
I would like to make a suggestion, though: The mail size of failure reports is increasing constantly as long as a boot problem remains unfixed. Would it make sense to only list the first 5 and last 5 commits in the failing series and replace the rest with "..." or something similar?
Regards, Carl-Daniel
Done. I also added a note of how many commits were in the skipped section.
On 22.04.2015 06:01, Timothy Pearson wrote:
On 04/21/2015 05:25 PM, Carl-Daniel Hailfinger wrote:
Thank you for providing this service. It is very much appreciated.
I would like to make a suggestion, though: The mail size of failure reports is increasing constantly as long as a boot problem remains unfixed. Would it make sense to only list the first 5 and last 5 commits in the failing series and replace the rest with "..." or something similar?
Done. I also added a note of how many commits were in the skipped section.
Thank you!
Regards, Carl-Daniel
Hi,
If the community would like me to stop sending these notifications I will do so; I could also adjust the maximum frequency if desired.
How about sending out messages on status changes (i.e. edge not level triggered). That should avoid annoying repeated reports on the same failure.
Also mails clearly need to be improved so they are useful to people like me who don't know your test system in detail. I'm caring about qemu support in coreboot, so I had a look at the log. On a quick scan nothing unusual was outstanding, except the big fat line saying the boot test was successful. The email itself just says "test failure" without any specifics, so I don't have an idea what to look for in the log. That is not helpful.
cheers, Gerd
On 04/22/2015 01:28 AM, Gerd Hoffmann wrote:
Hi,
If the community would like me to stop sending these notifications I will do so; I could also adjust the maximum frequency if desired.
How about sending out messages on status changes (i.e. edge not level triggered). That should avoid annoying repeated reports on the same failure.
Also mails clearly need to be improved so they are useful to people like me who don't know your test system in detail. I'm caring about qemu support in coreboot, so I had a look at the log. On a quick scan nothing unusual was outstanding, except the big fat line saying the boot test was successful. The email itself just says "test failure" without any specifics, so I don't have an idea what to look for in the log. That is not helpful.
cheers, Gerd
Understood. I did not have any real data until now on how failures would be dealt with by the coreboot community; it appears that failures may last for a lot longer in the master tree than I originally anticipated, therefore some adjustment in how the system notifies the community is needed. I still prefer some form of nag message so that the failure is not ignored; it is my experience that issues in open source tend to languish / be forgotten without persistent reminders. Right now the system is set to 1 day minimum interval; if this is still too fast it can be dialled back further. I'd like input from others before changing this setting (or switching to "edge triggered" notification).
I have also been considering how to improve the failure messages; a failure to boot is pretty obvious in the boot log, however as you say, verification failures are poorly noted. Much of this is due to my original assumptions on the quality of the mainline GIT tree at any given instant; I will extend the system to note exactly which tests have failed in order to avoid future confusion.
-
Carl-Daniel Hailfinger -
Gerd Hoffmann -
Timothy Pearson