I've run into an interesting problem doing some "bait and switch" with my emulator.
I have machine code for an x86 program that expects to run in protected mode. It's not an ELF file, so I can't load it directly with elfboot(). What I'm doing for the time being is having elfboot load Etherboot, with the emulator set to break at the Etherboot entry address. When the breakpoint is hit I load the machine code into memory and change EIP to its entry point. (Ultimately I'll probably create an ELF wrapper for this code, but I don't have one yet).
My machine code gets a protection exception when it tries to set one of the segment registers (to the value it already has, BTW). I traced this to the fact that Etherboot was loaded on top of the GDT used by LinuxBIOS.
We can argue about what kind of assumptions payloads should make about their runtime environment, but it seems to me that being in protected mode without a GDT is a bomb waiting to go off. Some payloads are bound to do things in a sequence that causes an explosion.
Can we move the GDT within the memory LB reports as unusable, say, before the tables?
Steve www.digidescorp.com