[SeaBIOS] [PATCH] vgabios: Avoid extra stack if it appears a modern OS is in use

[Kevin O'Connor]

On Mon, Mar 16, 2015 at 11:47:51AM -0400, Kevin O'Connor wrote:
> If a VBE mode set call is made, stop using the vgabios extra stack.
> This works under the premise that only a modern OS would invoke the
> VBE mode changing facilities and a modern OS would always call the
> vgabios with sufficient stack space.
> 
> This is an ugly hack to work around a problem Windows Vista (and
> possibly later Windows releases) has with the VGA BIOS using a stack
> in the e-segment.
[...]
> +void
> +check_extra_stack(void)
> +{
> +    if (!CONFIG_VGA_ALLOCATE_EXTRA_STACK)
> +        return;
> +    extern void entry_10(void);
> +    if (GET_IVT(0x10).offset == (u32)entry_10)
> +        return;

That should really test if the current entry point is not
entry_10_extrastack.  New patch is below.  I've also uploaded this to:

https://github.com/KevinOConnor/seabios/tree/testing

-Kevin


commit 8b8bca2205cf52ddf7dd216826c192c98c904068
Author: Kevin O'Connor <kevin at koconnor.net>
Date:   Mon Mar 16 11:22:30 2015 -0400

    vgabios: Avoid extra stack if it appears a modern OS is in use
    
    If a VBE mode set call is made, stop using the vgabios extra stack.
    This works under the premise that only a modern OS would invoke the
    VBE mode changing facilities and a modern OS would always call the
    vgabios with sufficient stack space.
    
    This is an ugly hack to work around a problem Windows Vista (and
    possibly later Windows releases) has with the VGA BIOS using a stack
    in the e-segment.
    
    Reported-by: Richard Laager <rlaager at wiktel.com>
    Signed-off-by: Kevin O'Connor <kevin at koconnor.net>

diff --git a/vgasrc/vbe.c b/vgasrc/vbe.c
index af3d0cc..834d7b3 100644
--- a/vgasrc/vbe.c
+++ b/vgasrc/vbe.c
@@ -205,6 +205,7 @@ static void
 vbe_104f02(struct bregs *regs)
 {
     dprintf(1, "VBE mode set: %x\n", regs->bx);
+    check_extra_stack();
 
     int mode = regs->bx & ~MF_VBEFLAGS;
     int flags = regs->bx & MF_VBEFLAGS;
diff --git a/vgasrc/vgabios.h b/vgasrc/vgabios.h
index fd796f2..985bbbb 100644
--- a/vgasrc/vgabios.h
+++ b/vgasrc/vgabios.h
@@ -100,6 +100,7 @@ extern u8 vgafont16alt[];
 // vgainit.c
 extern struct video_save_pointer_s video_save_pointer_table;
 extern struct video_param_s video_param_table[29];
+void check_extra_stack(void);
 
 // vgabios.c
 extern int VgaBDF;
diff --git a/vgasrc/vgainit.c b/vgasrc/vgainit.c
index 8d12261..2e89419 100644
--- a/vgasrc/vgainit.c
+++ b/vgasrc/vgainit.c
@@ -86,6 +86,21 @@ allocate_extra_stack(void)
     }
 }
 
+void
+check_extra_stack(void)
+{
+    extern void entry_10_extrastack(void);
+    if (!CONFIG_VGA_ALLOCATE_EXTRA_STACK
+        || GET_IVT(0x10).offset != (u32)entry_10_extrastack)
+        return;
+    // Disable the extra stack if a VBE mode set call is made.  This
+    // works around a bug in Windows Vista when the stack is in the
+    // e-segment.
+    dprintf(1, "Disabling SeaVGABIOS extra stack.\n");
+    extern void entry_10(void);
+    SET_IVT(0x10, SEGOFF(get_global_seg(), (u32)entry_10));
+}
+
 
 /****************************************************************
  * Timer hook