[SeaBIOS] varlow/extrastack vs code

[Dr. David Alan Gilbert]

* Kevin O'Connor (kevin at koconnor.net) wrote:
> On Mon, Jan 23, 2017 at 06:49:07PM +0000, Dr. David Alan Gilbert wrote:
> > * Laszlo Ersek (lersek at redhat.com) wrote:
> > > On 01/23/17 16:49, Kevin O'Connor wrote:
> > > > On Mon, Jan 23, 2017 at 11:11:02AM +0100, Laszlo Ersek wrote:
> > > >> On 01/20/17 20:39, Dr. David Alan Gilbert wrote:
> > > >>> * Kevin O'Connor (kevin at koconnor.net) wrote:
> > > >>>> On Fri, Jan 20, 2017 at 06:40:44PM +0000, Dr. David Alan Gilbert wrote:
> > > >>>>> Hi,
> > > >>>>>   I turned the debug level up to 4 on our smaller (128k) ROM downstream
> > > >>>>> build and seem to have hit a case where it's been layed out so that the
> > > >>>>> 'ExtraStack' is at the same location as some code (display_uuid) which
> > > >>>>> was causing some very random behaviour;
> > > > [...]
> > > >> Would this be consistent with a stack overflow?
> > > >>
> > > >> See commit 46b82624c95b951e8825fab117d9352faeae0ec8. Perhaps
> > > >> BUILD_EXTRA_STACK_SIZE (2KB) is too small now?
> > > > 
> > > > The ExtraStack isn't used at the point Dave reports the problem -
> > > > display_uuid() is part of the init phase and that happens on the main
> > > > "post" stack.
> > > > 
> > > > [...]
> > > >> (This is based off 1.9.1)
> > > > 
> > > > I missed that earlier - there were some important fixes post 1.9.1 wrt
> > > > reboots.  Commits b837e68d / a48f602c2 could explain the issue.  I'd
> > > > make sure the issue is still present on the latest version.
> > > 
> > > That's a very promising hunch -- b837e68d explicitly mentions "reboot
> > > loop" in the subject. It seems that Dave didn't mention any RHBZ numbers
> > > in his email, but we have two somewhat similar bug reports (which I hope
> > > share a root cause) and the second report triggers the issue with a
> > > reboot loop specifically.
> > > 
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1411275
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1382906
> > > 
> > > (Apologies that the 2nd RHBZ is not public; it's currently filed for the
> > > RH kernel, and those BZs default to private. :/)
> 
> That first report mentions migration to a different QEMU version.
> When migrating, is the BIOS software migrated as well (the copy at
> 0xffff0000), or does the new instance get a potentially different
> instance of the BIOS?

This case isn't with migration; as soon as I started debugging that
I found I could hit it with just a simple reboot loop in the guest.
(and migration should give you exactly the same ROM image as the
source because the read-only blocks get migrated and used rather than
the local copy on the destination - until you power the VM down and
restart the QEMU when it rereads the local file).

> > > CC'ing DavidH too, for RHBZ#1382906.
> > 
> > Yeh, it's looking promising; I've done a build with low debug that
> > survived for 50+ reboots and turned my debug on and it's going for 20 so far,
> > so that's pretty good.
> > 
> > However, reading the commits I'm a little confused.
> > 
> > I don't seem to have hit any cases where it's taken the shutdown case after
> > failing to reboot; so it's not that path.
> > 
> > My reboots in this case are always guest triggered, so they're not very
> > early reboots.
> 
> Both of those seabios fixes are for reboots that occur while
> processing a reboot.  Any chance the guest tries multiple reboot
> signals and one of them gets delayed?

It's possible; I'll have to add some qemu debug to watch for that;
the guest I'm using is a rhel 6.9-ish image - so old kernel
with lots of patches; but I'll try and see what it's doing.
I've just got a loop that waits for the VM to boot, sees the login/password
prompt on serial console, and as soon as it gets a shell issues 'reboot'.

> > One comment in there is:
> >    +        // Some old versions of KVM don't store a pristine copy of the
> >    +        // BIOS in high memory.  Try to shutdown the machine instead.
> > 
> > do you have a definition of 'old';  in this case it's a new-ish qemu
> > on our downstream (older) kernel but it's got fairly new kvm bits in,
> > but the qemu is configured in our rhel6 compatibility mode - so hmm.
> 
> I don't have the kvm version handy, but it's really old.  You're
> definitely not on that version, or every reboot would result in a
> shutdown instead.

OK, thanks.

Dave

> -Kevin
--
Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK