On Wed, Jan 18, 2017 at 12:40:24PM +0100, Gerd Hoffmann wrote:
After I posted the seavgabios stuff above, it occurred to me that another way to tackle this would be to keep the code in seabios and harden the sercon assembler entry point. Specifically, the assembler code could check that the last mode set wasn't from a vesa mode set call and it could check that calll (vs callw) works.
Additionally we need to hook the sercon code only into a few subcalls, and with the exception of 00h (aka set-mode) none of them are useful for software which wants program vesa modes. So, yes, I think that could work.
One of the BSDs (I think freebsd) uses the x86emu code to emulate the vgabios at kernel startup. I'm not sure if it only makes vesa calls.
I was thinking a few instructions to test if calll worked would be sufficient to catch the x86emu case - something like:
pushl $0xffffffff calll 1f 1:popl %eax cmpl %eax, $1b jne running_on_broken_x86emu
I'll go have a look, starting with old linux xorg+x86emu. Do you remember which windows versions have problems too?
The notes at the top of scripts/vgafixup.py have:
# The x86emu code widely used in Linux distributions when running Xorg # in vesamode is known to have issues with "retl", "leavel", "entryl", # "leal", and some variants of "calll". This code modifies those # instructions that are known to be generated by gcc to avoid # triggering the x86emu bugs.
# It is also known that the Windows vgabios emulator has issues with # addressing negative offsets to the %esp register. That has been # worked around by not using the gcc parameter "-fomit-frame-pointer" # when compiling.
I believe at least WinXP had the esp problem. In addition to the above at least Vista has the skifree bug (stack can't be in high memory).
-Kevin