[PATCH 00/12] Support using SMM for 32bit trampolining

[Kevin O'Connor]

      This series reworks the internal 16bit <-> 32bit trampolining that the
SeaBIOS code uses during runtime, and it adds support for using System
Management Mode (SMM) for performing a more correct switch to 32bit
mode during runtime.  Currently, QEMU v2.1 in TCG mode is needed in
order for the SMM handler to be utilized.

This series was inspired by the desire to move more hardware
processing to 32bit mode.  This change, however, does not move any
additional code to 32bit.  The series may still be useful though when
running some existing 32bit drivers (eg, ahci and xhci).

The series is also available at:
  https://github.com/KevinOConnor/seabios/tree/testing

With the completion of this series, the full 32bit hardware branch is
down to only three patches.  For reference, that branch is at:
  https://github.com/KevinOConnor/seabios/tree/testing-32bit-drivers

-Kevin


Kevin O'Connor (12):
  Move stack hop code below call32/call16 code in stacks.c
  Add need_hop_back() call that determines if stack_hop_back is needed
  Update invoke_mouse_handler() to use need_hop_back()
  Update stack_hop_back() to jump to 16bit mode if called in 32bit mode.
  Track when entering via call32() and use the same mode for
    stack_hop_back()
  Simplify farcall16 code
  Update reset() to use call16_back()
  build: Support declaring 32bit C functions that must reside in the
    f-segment
  Move call16() functions from romlayout.S to inline assembler in
    stacks.c
  Break up call32() into call32() and call32_sloppy()
  Fully restore 16bit state during call16_sloppy()
  Implement call32 mechanism using SMIs.

 scripts/layoutrom.py |  16 +-
 src/Kconfig          |   4 +
 src/fw/smm.c         |  46 ++++-
 src/mouse.c          |  15 +-
 src/romlayout.S      |  42 -----
 src/stacks.c         | 510 ++++++++++++++++++++++++++++++++++++++-------------
 src/stacks.h         |  23 ++-
 src/types.h          |   4 +
 8 files changed, 477 insertions(+), 183 deletions(-)

-- 
1.9.3

[PATCH 01/12] Move stack hop code below call32/call16 code in stacks.c

[Kevin O'Connor]

      This change is a just code movement.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c | 164 +++++++++++++++++++++++++++++------------------------------
 src/stacks.h |   2 +-
 2 files changed, 83 insertions(+), 83 deletions(-)

diff --git a/src/stacks.c b/src/stacks.c
index df719fd..dd955fb 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -18,88 +18,6 @@
 
 
 /****************************************************************
- * Extra 16bit stack
- ****************************************************************/
-
-// Space for a stack for 16bit code.
-u8 ExtraStack[BUILD_EXTRA_STACK_SIZE+1] VARLOW __aligned(8);
-u8 *StackPos VARLOW;
-
-// Test if currently on the extra stack
-static inline int
-on_extra_stack(void)
-{
-    return MODE16 && GET_SEG(SS) == SEG_LOW && getesp() >
(u32)ExtraStack;
-}
-
-// Switch to the extra stack and call a function.
-u32
-stack_hop(u32 eax, u32 edx, void *func)
-{
-    if (on_extra_stack())
-        return ((u32 (*)(u32, u32))func)(eax, edx);
-    ASSERT16();
-    u16 stack_seg = SEG_LOW;
-    u32 bkup_ss, bkup_esp;
-    asm volatile(
-        // Backup current %ss/%esp values.
-        "movw %%ss, %w3\n"
-        "movl %%esp, %4\n"
-        // Copy stack seg to %ds/%ss and set %esp
-        "movw %w6, %%ds\n"
-        "movw %w6, %%ss\n"
-        "movl %5, %%esp\n"
-        "pushl %3\n"
-        "pushl %4\n"
-        // Call func
-        "calll *%2\n"
-        "popl %4\n"
-        "popl %3\n"
-        // Restore segments and stack
-        "movw %w3, %%ds\n"
-        "movw %w3, %%ss\n"
-        "movl %4, %%esp"
-        : "+a" (eax), "+d" (edx), "+c" (func),
"=&r" (bkup_ss), "=&r" (bkup_esp)
-        : "m" (StackPos), "r" (stack_seg)
-        : "cc", "memory");
-    return eax;
-}
-
-// Switch back to original caller's stack and call a function.
-u32
-stack_hop_back(u32 eax, u32 edx, void *func)
-{
-    if (!on_extra_stack())
-        return ((u32 (*)(u32, u32))func)(eax, edx);
-    ASSERT16();
-    u16 bkup_ss;
-    u32 bkup_stack_pos, temp;
-    asm volatile(
-        // Backup stack_pos and current %ss/%esp
-        "movl %6, %4\n"
-        "movw %%ss, %w3\n"
-        "movl %%esp, %6\n"
-        // Restore original callers' %ss/%esp
-        "movl -4(%4), %5\n"
-        "movl %5, %%ss\n"
-        "movw %%ds:-8(%4), %%sp\n"
-        "movl %5, %%ds\n"
-        // Call func
-        "calll *%2\n"
-        // Restore %ss/%esp and stack_pos
-        "movw %w3, %%ds\n"
-        "movw %w3, %%ss\n"
-        "movl %6, %%esp\n"
-        "movl %4, %6"
-        : "+a" (eax), "+d" (edx), "+c" (func),
"=&r" (bkup_ss)
-          , "=&r" (bkup_stack_pos), "=&r" (temp),
"+m" (StackPos)
-        :
-        : "cc", "memory");
-    return eax;
-}
-
-
-/****************************************************************
  * 16bit / 32bit calling
  ****************************************************************/
 
@@ -189,6 +107,88 @@ call16big(u32 eax, u32 edx, void *func)
 
 
 /****************************************************************
+ * Extra 16bit stack
+ ****************************************************************/
+
+// Space for a stack for 16bit code.
+u8 ExtraStack[BUILD_EXTRA_STACK_SIZE+1] VARLOW __aligned(8);
+u8 *StackPos VARLOW;
+
+// Test if currently on the extra stack
+static inline int
+on_extra_stack(void)
+{
+    return MODE16 && GET_SEG(SS) == SEG_LOW && getesp() >
(u32)ExtraStack;
+}
+
+// Switch to the extra stack and call a function.
+u32
+stack_hop(u32 eax, u32 edx, void *func)
+{
+    if (on_extra_stack())
+        return ((u32 (*)(u32, u32))func)(eax, edx);
+    ASSERT16();
+    u16 stack_seg = SEG_LOW;
+    u32 bkup_ss, bkup_esp;
+    asm volatile(
+        // Backup current %ss/%esp values.
+        "movw %%ss, %w3\n"
+        "movl %%esp, %4\n"
+        // Copy stack seg to %ds/%ss and set %esp
+        "movw %w6, %%ds\n"
+        "movw %w6, %%ss\n"
+        "movl %5, %%esp\n"
+        "pushl %3\n"
+        "pushl %4\n"
+        // Call func
+        "calll *%2\n"
+        "popl %4\n"
+        "popl %3\n"
+        // Restore segments and stack
+        "movw %w3, %%ds\n"
+        "movw %w3, %%ss\n"
+        "movl %4, %%esp"
+        : "+a" (eax), "+d" (edx), "+c" (func),
"=&r" (bkup_ss), "=&r" (bkup_esp)
+        : "m" (StackPos), "r" (stack_seg)
+        : "cc", "memory");
+    return eax;
+}
+
+// Switch back to original caller's stack and call a function.
+u32
+stack_hop_back(u32 eax, u32 edx, void *func)
+{
+    if (!on_extra_stack())
+        return ((u32 (*)(u32, u32))func)(eax, edx);
+    ASSERT16();
+    u16 bkup_ss;
+    u32 bkup_stack_pos, temp;
+    asm volatile(
+        // Backup stack_pos and current %ss/%esp
+        "movl %6, %4\n"
+        "movw %%ss, %w3\n"
+        "movl %%esp, %6\n"
+        // Restore original callers' %ss/%esp
+        "movl -4(%4), %5\n"
+        "movl %5, %%ss\n"
+        "movw %%ds:-8(%4), %%sp\n"
+        "movl %5, %%ds\n"
+        // Call func
+        "calll *%2\n"
+        // Restore %ss/%esp and stack_pos
+        "movw %w3, %%ds\n"
+        "movw %w3, %%ss\n"
+        "movl %6, %%esp\n"
+        "movl %4, %6"
+        : "+a" (eax), "+d" (edx), "+c" (func),
"=&r" (bkup_ss)
+          , "=&r" (bkup_stack_pos), "=&r" (temp),
"+m" (StackPos)
+        :
+        : "cc", "memory");
+    return eax;
+}
+
+
+/****************************************************************
  * External 16bit interface calling
  ****************************************************************/
 
diff --git a/src/stacks.h b/src/stacks.h
index 22fb943..9d3422f 100644
--- a/src/stacks.h
+++ b/src/stacks.h
@@ -5,10 +5,10 @@
 #include "types.h" // u32
 
 // stacks.c
+u32 call32(void *func, u32 eax, u32 errret);
 extern u8 ExtraStack[], *StackPos;
 u32 stack_hop(u32 eax, u32 edx, void *func);
 u32 stack_hop_back(u32 eax, u32 edx, void *func);
-u32 call32(void *func, u32 eax, u32 errret);
 struct bregs;
 inline void farcall16(struct bregs *callregs);
 inline void farcall16big(struct bregs *callregs);
-- 
1.9.3

[PATCH 02/12] Add need_hop_back() call that determines if stack_hop_back is needed

[Kevin O'Connor]

      Also, use need_hop_back() instead of on_extra_stack() in code that
determines whether or not to call stack_hop_back().

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c |  8 ++++----
 src/stacks.h | 10 ++++++++++
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/src/stacks.c b/src/stacks.c
index dd955fb..4fd70a7 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -115,7 +115,7 @@ u8 ExtraStack[BUILD_EXTRA_STACK_SIZE+1] VARLOW __aligned(8);
 u8 *StackPos VARLOW;
 
 // Test if currently on the extra stack
-static inline int
+int
 on_extra_stack(void)
 {
     return MODE16 && GET_SEG(SS) == SEG_LOW && getesp() >
(u32)ExtraStack;
@@ -197,7 +197,7 @@ void VISIBLE16
 _farcall16(struct bregs *callregs, u16 callregseg)
 {
     ASSERT16();
-    if (on_extra_stack()) {
+    if (need_hop_back()) {
         stack_hop_back((u32)callregs, callregseg, _farcall16);
         return;
     }
@@ -384,7 +384,7 @@ fail:
 void VISIBLE16
 check_irqs(void)
 {
-    if (on_extra_stack()) {
+    if (need_hop_back()) {
         stack_hop_back(0, 0, check_irqs);
         return;
     }
@@ -416,7 +416,7 @@ yield(void)
 void VISIBLE16
 wait_irq(void)
 {
-    if (on_extra_stack()) {
+    if (need_hop_back()) {
         stack_hop_back(0, 0, wait_irq);
         return;
     }
diff --git a/src/stacks.h b/src/stacks.h
index 9d3422f..265b404 100644
--- a/src/stacks.h
+++ b/src/stacks.h
@@ -9,6 +9,7 @@ u32 call32(void *func, u32 eax, u32 errret);
 extern u8 ExtraStack[], *StackPos;
 u32 stack_hop(u32 eax, u32 edx, void *func);
 u32 stack_hop_back(u32 eax, u32 edx, void *func);
+int on_extra_stack(void);
 struct bregs;
 inline void farcall16(struct bregs *callregs);
 inline void farcall16big(struct bregs *callregs);
@@ -35,4 +36,13 @@ int wait_preempt(void);
 void check_preempt(void);
 u32 call32_params(void *func, u32 eax, u32 edx, u32 ecx, u32 errret);
 
+// Inline functions
+
+// Check if a call to stack_hop_back is needed.
+static inline int
+need_hop_back(void)
+{
+    return on_extra_stack();
+}
+
 #endif // stacks.h
-- 
1.9.3

[PATCH 03/12] Update invoke_mouse_handler() to use need_hop_back()

[Kevin O'Connor]

      Make the mouse handler stack_hop_back() code similar to the other
users that use need_stack_hop().

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/mouse.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/src/mouse.c b/src/mouse.c
index 100255d..466f55a 100644
--- a/src/mouse.c
+++ b/src/mouse.c
@@ -275,8 +275,13 @@ handle_15c2(struct bregs *regs)
 }
 
 static void
-invoke_mouse_handler(u16 ebda_seg)
+invoke_mouse_handler(void)
 {
+    if (need_hop_back()) {
+        stack_hop_back(0, 0, invoke_mouse_handler);
+        return;
+    }
+    u16 ebda_seg = get_ebda_seg();
     u16 status = GET_EBDA(ebda_seg, mouse_data[0]);
     u16 X      = GET_EBDA(ebda_seg, mouse_data[1]);
     u16 Y      = GET_EBDA(ebda_seg, mouse_data[2]);
@@ -330,5 +335,5 @@ process_mouse(u8 data)
     }
 
     SET_EBDA(ebda_seg, mouse_flag1, 0);
-    stack_hop_back(ebda_seg, 0, invoke_mouse_handler);
+    invoke_mouse_handler();
 }
-- 
1.9.3

[PATCH 04/12] Update stack_hop_back() to jump to 16bit mode if called in 32bit mode.

[Kevin O'Connor]

      Also, update callers to rely on this feature.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/mouse.c  |  8 ++++++--
 src/stacks.c | 31 +++++++++++++------------------
 src/stacks.h |  2 +-
 3 files changed, 20 insertions(+), 21 deletions(-)

diff --git a/src/mouse.c b/src/mouse.c
index 466f55a..6d1f5b7 100644
--- a/src/mouse.c
+++ b/src/mouse.c
@@ -274,13 +274,17 @@ handle_15c2(struct bregs *regs)
     }
 }
 
-static void
+void VISIBLE16
 invoke_mouse_handler(void)
 {
+    if (!CONFIG_MOUSE)
+        return;
     if (need_hop_back()) {
-        stack_hop_back(0, 0, invoke_mouse_handler);
+        extern void _cfunc16_invoke_mouse_handler(void);
+        stack_hop_back(0, 0, _cfunc16_invoke_mouse_handler);
         return;
     }
+    ASSERT16();
     u16 ebda_seg = get_ebda_seg();
     u16 status = GET_EBDA(ebda_seg, mouse_data[0]);
     u16 X      = GET_EBDA(ebda_seg, mouse_data[1]);
diff --git a/src/stacks.c b/src/stacks.c
index 4fd70a7..5a2628a 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -158,6 +158,8 @@ stack_hop(u32 eax, u32 edx, void *func)
 u32
 stack_hop_back(u32 eax, u32 edx, void *func)
 {
+    if (!MODESEGMENT)
+        return call16big(eax, edx, func);
     if (!on_extra_stack())
         return ((u32 (*)(u32, u32))func)(eax, edx);
     ASSERT16();
@@ -196,11 +198,12 @@ stack_hop_back(u32 eax, u32 edx, void *func)
 void VISIBLE16
 _farcall16(struct bregs *callregs, u16 callregseg)
 {
-    ASSERT16();
     if (need_hop_back()) {
-        stack_hop_back((u32)callregs, callregseg, _farcall16);
+        extern void _cfunc16__farcall16(void);
+        stack_hop_back((u32)callregs, callregseg, _cfunc16__farcall16);
         return;
     }
+    ASSERT16();
     asm volatile(
         "calll __farcall16\n"
         : "+a" (callregs), "+m" (*callregs), "+d"
(callregseg)
@@ -385,7 +388,8 @@ void VISIBLE16
 check_irqs(void)
 {
     if (need_hop_back()) {
-        stack_hop_back(0, 0, check_irqs);
+        extern void _cfunc16_check_irqs(void);
+        stack_hop_back(0, 0, _cfunc16_check_irqs);
         return;
     }
     asm volatile("sti ; nop ; rep ; nop ; cli ; cld" : : :"memory");
@@ -395,19 +399,14 @@ check_irqs(void)
 void
 yield(void)
 {
-    if (MODESEGMENT) {
+    if (MODESEGMENT || !CONFIG_THREADS) {
         check_irqs();
         return;
     }
-    extern void _cfunc16_check_irqs(void);
-    if (!CONFIG_THREADS) {
-        call16big(0, 0, _cfunc16_check_irqs);
-        return;
-    }
     struct thread_info *cur = getCurThread();
     if (cur == &MainThread)
         // Permit irqs to fire
-        call16big(0, 0, _cfunc16_check_irqs);
+        check_irqs();
 
     // Switch to the next thread
     switch_next(cur);
@@ -417,7 +416,8 @@ void VISIBLE16
 wait_irq(void)
 {
     if (need_hop_back()) {
-        stack_hop_back(0, 0, wait_irq);
+        extern void _cfunc16_wait_irq(void);
+        stack_hop_back(0, 0, _cfunc16_wait_irq);
         return;
     }
     asm volatile("sti ; hlt ; cli ; cld": : :"memory");
@@ -427,17 +427,12 @@ wait_irq(void)
 void
 yield_toirq(void)
 {
-    if (MODESEGMENT) {
-        wait_irq();
-        return;
-    }
-    if (have_threads()) {
+    if (!MODESEGMENT && have_threads()) {
         // Threads still active - do a yield instead.
         yield();
         return;
     }
-    extern void _cfunc16_wait_irq(void);
-    call16big(0, 0, _cfunc16_wait_irq);
+    wait_irq();
 }
 
 // Wait for all threads (other than the main thread) to complete.
diff --git a/src/stacks.h b/src/stacks.h
index 265b404..cbc5f4f 100644
--- a/src/stacks.h
+++ b/src/stacks.h
@@ -42,7 +42,7 @@ u32 call32_params(void *func, u32 eax, u32 edx, u32 ecx, u32 errret);
 static inline int
 need_hop_back(void)
 {
-    return on_extra_stack();
+    return !MODESEGMENT || on_extra_stack();
 }
 
 #endif // stacks.h
-- 
1.9.3

[PATCH 05/12] Track when entering via call32() and use the same mode for stack_hop_back()

[Kevin O'Connor]

      If 32bit mode is entered directly via transition32, then use a simple
call16() when hopping back to 16bit mode.  Use only call16big() when
entering 32bit mode via call32().

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/src/stacks.c b/src/stacks.c
index 5a2628a..5674d0a 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -22,6 +22,8 @@
  ****************************************************************/
 
 u16 StackSeg VARLOW;
+u8 Call32Method VARLOW;
+#define C32_SLOPPY 1
 
 // Call a 32bit SeaBIOS function from a 16bit SeaBIOS function.
 u32 VISIBLE16
@@ -45,6 +47,7 @@ call32(void *func, u32 eax, u32 errret)
 
     u16 oldstackseg = GET_LOW(StackSeg);
     SET_LOW(StackSeg, GET_SEG(SS));
+    SET_LOW(Call32Method, C32_SLOPPY);
     u32 bkup_ss, bkup_esp;
     asm volatile(
         // Backup ss/esp / set esp to flat stack location
@@ -71,6 +74,7 @@ call32(void *func, u32 eax, u32 errret)
         : "r" (func)
         : "ecx", "edx", "cc", "memory");
 
+    SET_LOW(Call32Method, 0);
     SET_LOW(StackSeg, oldstackseg);
 
     // Restore gdt and fs/gs
@@ -95,6 +99,7 @@ call16(u32 eax, u32 edx, void *func)
     return __call16(eax, edx, func - BUILD_BIOS_ADDR);
 }
 
+// Call a 16bit SeaBIOS function in "big real" mode.
 static inline u32
 call16big(u32 eax, u32 edx, void *func)
 {
@@ -105,6 +110,26 @@ call16big(u32 eax, u32 edx, void *func)
     return __call16big(eax, edx, func - BUILD_BIOS_ADDR);
 }
 
+// Jump back to 16bit mode while in 32bit mode from call32()
+static u32
+call16_sloppy(u32 eax, u32 edx, void *func)
+{
+    Call32Method = 0;
+    u32 ret = call16big(eax, edx, func);
+    Call32Method = C32_SLOPPY;
+    return ret;
+}
+
+// Call a 16bit SeaBIOS function, restoring the mode from last call32().
+static u32
+call16_back(u32 eax, u32 edx, void *func)
+{
+    ASSERT32FLAT();
+    if (Call32Method == C32_SLOPPY)
+        return call16_sloppy(eax, edx, func);
+    return call16(eax, edx, func);
+}
+
 
 /****************************************************************
  * Extra 16bit stack
@@ -159,7 +184,7 @@ u32
 stack_hop_back(u32 eax, u32 edx, void *func)
 {
     if (!MODESEGMENT)
-        return call16big(eax, edx, func);
+        return call16_back(eax, edx, func);
     if (!on_extra_stack())
         return ((u32 (*)(u32, u32))func)(eax, edx);
     ASSERT16();
-- 
1.9.3

[PATCH 06/12] Simplify farcall16 code

[Kevin O'Connor]

      With this change, farcall16() is only used for external API calls and
is only invoked from a 32bit mode entered directly via transition32.
farcall16big() is also only used for external API calls and is only
invoked from a 32bit mode entered directly via transition32.

call16_int() now calls _farcall16() directly, and it will use normal
16bit mode or big real mode as required.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c | 26 ++++++++++++--------------
 src/stacks.h |  6 +++---
 2 files changed, 15 insertions(+), 17 deletions(-)

diff --git a/src/stacks.c b/src/stacks.c
index 5674d0a..a36f643 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -236,34 +236,32 @@ _farcall16(struct bregs *callregs, u16 callregseg)
         : "ebx", "ecx", "esi", "edi",
"cc", "memory");
 }
 
-inline void
+void
 farcall16(struct bregs *callregs)
 {
-    if (MODE16) {
-        _farcall16(callregs, GET_SEG(SS));
-        return;
-    }
     extern void _cfunc16__farcall16(void);
-    call16((u32)callregs - StackSeg * 16, StackSeg, _cfunc16__farcall16);
+    call16((u32)callregs, 0, _cfunc16__farcall16);
 }
 
-inline void
+void
 farcall16big(struct bregs *callregs)
 {
     extern void _cfunc16__farcall16(void);
-    call16big((u32)callregs - StackSeg * 16, StackSeg, _cfunc16__farcall16);
+    call16big((u32)callregs, 0, _cfunc16__farcall16);
 }
 
 // Invoke a 16bit software interrupt.
-inline void
+void
 __call16_int(struct bregs *callregs, u16 offset)
 {
-    if (MODESEGMENT)
-        callregs->code.seg = GET_SEG(CS);
-    else
-        callregs->code.seg = SEG_BIOS;
     callregs->code.offset = offset;
-    farcall16(callregs);
+    if (!MODESEGMENT) {
+        callregs->code.seg = SEG_BIOS;
+        _farcall16((void*)callregs - StackSeg * 16, StackSeg);
+        return;
+    }
+    callregs->code.seg = GET_SEG(CS);
+    _farcall16(callregs, GET_SEG(SS));
 }
 
 // Reset the machine
diff --git a/src/stacks.h b/src/stacks.h
index cbc5f4f..c3ddc17 100644
--- a/src/stacks.h
+++ b/src/stacks.h
@@ -11,9 +11,9 @@ u32 stack_hop(u32 eax, u32 edx, void *func);
 u32 stack_hop_back(u32 eax, u32 edx, void *func);
 int on_extra_stack(void);
 struct bregs;
-inline void farcall16(struct bregs *callregs);
-inline void farcall16big(struct bregs *callregs);
-inline void __call16_int(struct bregs *callregs, u16 offset);
+void farcall16(struct bregs *callregs);
+void farcall16big(struct bregs *callregs);
+void __call16_int(struct bregs *callregs, u16 offset);
 #define call16_int(nr, callregs) do {                           \
         extern void irq_trampoline_ ##nr ();                    \
         __call16_int((callregs), (u32)&irq_trampoline_ ##nr );  \
-- 
1.9.3

[PATCH 07/12] Update reset() to use call16_back()

[Kevin O'Connor]

      Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/stacks.c b/src/stacks.c
index a36f643..9c0001c 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -269,8 +269,8 @@ void
 reset(void)
 {
     extern void reset_vector(void) __noreturn;
-    if (!MODESEGMENT)
-        call16(0, 0, reset_vector);
+    if (!MODE16)
+        call16_back(0, 0, reset_vector);
     reset_vector();
 }
 
-- 
1.9.3

[PATCH 08/12] build: Support declaring 32bit C functions that must reside in the f-segment

[Kevin O'Connor]

      Add support for a FUNCFSEG macro that will force a "32bit flat" C
function to be located in the f-segment.  This is useful for 32bit
code with inline assembler that thunks to 16bit mode.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 scripts/layoutrom.py | 16 +++++++++++-----
 src/types.h          |  4 ++++
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/scripts/layoutrom.py b/scripts/layoutrom.py
index 2454920..dd770fe 100755
--- a/scripts/layoutrom.py
+++ b/scripts/layoutrom.py
@@ -197,11 +197,14 @@ def doLayout(sections, config, genreloc):
         textsections + rodatasections + datasections, sec16_start
         , segoffset=BUILD_BIOS_ADDR)
 
-    # Determine "fseg memory" data positions
-    sections32fseg = getSectionsCategory(sections, '32fseg')
+    # Determine 32bit "fseg memory" data positions
+    sections32textfseg = getSectionsCategory(sections, '32textfseg')
+    sec32textfseg_start, sec32textfseg_align = setSectionsStart(
+        sections32textfseg, sec32seg_start, 16)
 
+    sections32fseg = getSectionsCategory(sections, '32fseg')
     sec32fseg_start, sec32fseg_align = setSectionsStart(
-        sections32fseg, sec32seg_start, 16
+        sections32fseg, sec32textfseg_start, 16
         , segoffset=BUILD_BIOS_ADDR)
 
     # Determine 32flat runtime positions
@@ -274,13 +277,14 @@ def doLayout(sections, config, genreloc):
     # Print statistics
     size16 = BUILD_BIOS_ADDR + BUILD_BIOS_SIZE - sec16_start
     size32seg = sec16_start - sec32seg_start
-    size32fseg = sec32seg_start - sec32fseg_start
+    size32textfseg = sec32seg_start - sec32textfseg_start
+    size32fseg = sec32textfseg_start - sec32fseg_start
     size32flat = sec32fseg_start - sec32flat_start
     size32init = sec32flat_start - sec32init_start
     sizelow = li.sec32low_end - li.sec32low_start
     print("16bit size:           %d" % size16)
     print("32bit segmented size: %d" % size32seg)
-    print("32bit flat size:      %d" % size32flat)
+    print("32bit flat size:      %d" % (size32flat + size32textfseg))
     print("32bit flat init size: %d" % size32init)
     print("Lowmem size:          %d" % sizelow)
     print("f-segment var size:   %d" % size32fseg)
@@ -659,6 +663,8 @@ def main():
             section.category = '32low'
         elif section.name.startswith('.data.varfseg.'):
             section.category = '32fseg'
+        elif section.name.startswith('.text.32fseg.'):
+            section.category = '32textfseg'
         elif section.name.startswith('.fixedaddr.'):
             section.category = 'fixed'
         elif section.fileid == '32flat' and section not in runtimesections:
diff --git a/src/types.h b/src/types.h
index 097372c..6dd8c43 100644
--- a/src/types.h
+++ b/src/types.h
@@ -70,6 +70,8 @@ extern void __force_link_error__only_in_16bit(void) __noreturn;
 # define VARFSEG __section(".discard.varfseg." UNIQSEC) __VISIBLE __weak
 // Designate a variable at a specific address in the f-segment.
 # define VARFSEGFIXED(addr) __section(".discard.varfixed." UNIQSEC) __VISIBLE
__weak
+// Notes a 32bit flat function that must reside in the f-segment.
+# define FUNCFSEG __section(".discard.32fseg." UNIQSEC) __VISIBLE __weak
 // Verify a variable is only accessable via 32bit "init" functions
 # define VARVERIFY32INIT __section(".discard.varinit." UNIQSEC)
 // Designate top-level assembler as 16bit only.
@@ -90,6 +92,7 @@ extern void __force_link_error__only_in_16bit(void) __noreturn;
 # define VARLOW __section(".discard.varlow." UNIQSEC) __VISIBLE __weak
 # define VARFSEG __section(".discard.varfseg." UNIQSEC) __VISIBLE __weak
 # define VARFSEGFIXED(addr) __section(".discard.varfixed." UNIQSEC) __VISIBLE
__weak
+# define FUNCFSEG __section(".discard.32fseg." UNIQSEC) __VISIBLE __weak
 # define VARVERIFY32INIT __section(".discard.varinit." UNIQSEC)
 # define ASM16(code)
 # define ASM32FLAT(code)
@@ -106,6 +109,7 @@ extern void __force_link_error__only_in_16bit(void) __noreturn;
 # define VARLOW __section(".data.varlow." UNIQSEC) __VISIBLE __weak
 # define VARFSEG __section(".data.varfseg." UNIQSEC) __VISIBLE
 # define VARFSEGFIXED(addr) __section(".fixedaddr." __stringify(addr))
__VISIBLE __aligned(1)
+# define FUNCFSEG __section(".text.32fseg." UNIQSEC) __VISIBLE
 # define VARVERIFY32INIT __section(".data.varinit." UNIQSEC)
 # define ASM16(code)
 # define ASM32FLAT(code) __ASM(code)
-- 
1.9.3

[PATCH 09/12] Move call16() functions from romlayout.S to inline assembler in stacks.c

[Kevin O'Connor]

      Use inline assembler in call16 type functions instead of using
__call16() in romlayout.S.

Since call16() and call16big() are now only called with %ss==0 they do
not need to update the stack pointer.  Only call16_sloppy() requires
the stack manipulation code.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/romlayout.S | 42 ------------------------------
 src/stacks.c    | 81 ++++++++++++++++++++++++++++++++++++++++++++++++---------
 2 files changed, 69 insertions(+), 54 deletions(-)

diff --git a/src/romlayout.S b/src/romlayout.S
index 028d1e8..0651112 100644
--- a/src/romlayout.S
+++ b/src/romlayout.S
@@ -129,48 +129,6 @@ transition16big:
         movl %ecx, %eax
         jmpl *%edx
 
-// Call a 16bit SeaBIOS function from SeaBIOS 32bit C code.
-// %ecx = calling function
-// Clobbers: %ecx, %edx, flags, segment registers, idt/gdt
-        DECLFUNC __call16
-        .global __call16big
-        .code32
-__call16:
-        pushl %edx
-        pushl %ecx
-        movl $1f, %edx
-        jmp transition16
-__call16big:
-        pushl %edx
-        pushl %ecx
-        movl $1f, %edx
-        jmp transition16big
-
-        // Make call.
-        .code16
-1:      movl $_zonelow_seg, %edx        // Adjust %ds, %ss, and %esp
-        movl %edx, %ds
-        movzwl StackSeg, %edx
-        movl %edx, %ecx
-        shll $4, %ecx
-        movl %edx, %ss
-        subl %ecx, %esp
-        movl %edx, %ds
-
-        popl %ecx                       // Call function
-        popl %edx
-        calll *%ecx
-
-        movl %ss, %edx                  // Readjust %esp
-        shll $4, %edx
-        addl %edx, %esp
-
-        // Return via transition32
-        movl $(2f + BUILD_BIOS_ADDR), %edx
-        jmp transition32
-        .code32
-2:      retl
-
 
 /****************************************************************
  * External calling trampolines
diff --git a/src/stacks.c b/src/stacks.c
index 9c0001c..82d1c5c 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -1,6 +1,6 @@
 // Code for manipulating stack locations.
 //
-// Copyright (C) 2009-2010  Kevin O'Connor <kevin(a)koconnor.net&gt;
+// Copyright (C) 2009-2014  Kevin O'Connor <kevin(a)koconnor.net&gt;
 //
 // This file may be distributed under the terms of the GNU LGPLv3 license.
 
@@ -89,35 +89,92 @@ call32(void *func, u32 eax, u32 errret)
 }
 
 // Call a 16bit SeaBIOS function from a 32bit SeaBIOS function.
-static inline u32
+u32 FUNCFSEG
 call16(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
-    if (getesp() > MAIN_STACK_MAX)
+    if (getesp() > BUILD_STACK_ADDR)
         panic("call16 with invalid stack\n");
-    extern u32 __call16(u32 eax, u32 edx, void *func);
-    return __call16(eax, edx, func - BUILD_BIOS_ADDR);
+    func -= BUILD_BIOS_ADDR;
+    asm volatile(
+        // Transition to 16bit mode
+        "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
+        "  jmp transition16\n"
+        // Call func
+        "  .code16\n"
+        "1:movl %2, %%edx\n"
+        "  calll *%1\n"
+        // Return to 32bit
+        "  movl $2f, %%edx\n"
+        "  jmp transition32\n"
+        "  .code32\n"
+        "2:\n"
+        : "+a" (eax)
+        : "r" (func), "r" (edx)
+        : "edx", "ecx", "cc", "memory");
+    return eax;
 }
 
 // Call a 16bit SeaBIOS function in "big real" mode.
-static inline u32
+u32 FUNCFSEG
 call16big(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
-    if (getesp() > MAIN_STACK_MAX)
+    if (getesp() > BUILD_STACK_ADDR)
         panic("call16big with invalid stack\n");
-    extern u32 __call16big(u32 eax, u32 edx, void *func);
-    return __call16big(eax, edx, func - BUILD_BIOS_ADDR);
+    func -= BUILD_BIOS_ADDR;
+    asm volatile(
+        // Transition to 16bit mode
+        "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
+        "  jmp transition16big\n"
+        // Call func
+        "  .code16\n"
+        "1:movl %2, %%edx\n"
+        "  calll *%1\n"
+        // Return to 32bit
+        "  movl $2f, %%edx\n"
+        "  jmp transition32\n"
+        "  .code32\n"
+        "2:\n"
+        : "+a" (eax)
+        : "r" (func), "r" (edx)
+        : "edx", "ecx", "cc", "memory");
+    return eax;
 }
 
 // Jump back to 16bit mode while in 32bit mode from call32()
-static u32
+u32 FUNCFSEG
 call16_sloppy(u32 eax, u32 edx, void *func)
 {
+    ASSERT32FLAT();
+    if (getesp() > MAIN_STACK_MAX)
+        panic("call16_sloppy with invalid stack\n");
+    func -= BUILD_BIOS_ADDR;
     Call32Method = 0;
-    u32 ret = call16big(eax, edx, func);
+    u32 stackseg = GET_LOW(StackSeg);
+    asm volatile(
+        // Transition to 16bit mode
+        "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
+        "  jmp transition16big\n"
+        // Setup ss/esp and call func
+        "  .code16\n"
+        "1:movl %3, %%ecx\n"
+        "  shll $4, %3\n"
+        "  movw %%cx, %%ss\n"
+        "  subl %3, %%esp\n"
+        "  movw %%cx, %%ds\n"
+        "  movl %2, %%edx\n"
+        "  calll *%1\n"
+        // Return to 32bit and restore esp
+        "  movl $2f, %%edx\n"
+        "  jmp transition32\n"
+        "  .code32\n"
+        "2:addl %3, %%esp\n"
+        : "+a" (eax)
+        : "r" (func), "r" (edx), "r" (stackseg)
+        : "edx", "ecx", "cc", "memory");
     Call32Method = C32_SLOPPY;
-    return ret;
+    return eax;
 }
 
 // Call a 16bit SeaBIOS function, restoring the mode from last call32().
-- 
1.9.3

[PATCH 10/12] Break up call32() into call32() and call32_sloppy()

[Kevin O'Connor]

      This separates call32() into two functions.  It also moves the
call16_sloppy() code next to the call32_sloppy() code.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c | 86 +++++++++++++++++++++++++++++++++---------------------------
 1 file changed, 47 insertions(+), 39 deletions(-)

diff --git a/src/stacks.c b/src/stacks.c
index 82d1c5c..2a7be06 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -25,16 +25,12 @@ u16 StackSeg VARLOW;
 u8 Call32Method VARLOW;
 #define C32_SLOPPY 1
 
-// Call a 32bit SeaBIOS function from a 16bit SeaBIOS function.
-u32 VISIBLE16
-call32(void *func, u32 eax, u32 errret)
+// Call a C function in 32bit mode.  This clobbers the 16bit segment
+// selector registers.
+static u32
+call32_sloppy(void *func, u32 eax)
 {
     ASSERT16();
-    u32 cr0 = getcr0();
-    if (cr0 & CR0_PE)
-        // Called in 16bit protected mode?!
-        return errret;
-
     // Backup cmos index register and disable nmi
     u8 cmosindex = inb(PORT_CMOS_INDEX);
     outb(cmosindex | NMI_DISABLE_BIT, PORT_CMOS_INDEX);
@@ -88,45 +84,65 @@ call32(void *func, u32 eax, u32 errret)
     return eax;
 }
 
-// Call a 16bit SeaBIOS function from a 32bit SeaBIOS function.
+// Jump back to 16bit mode while in 32bit mode from call32_sloppy()
 u32 FUNCFSEG
-call16(u32 eax, u32 edx, void *func)
+call16_sloppy(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
-    if (getesp() > BUILD_STACK_ADDR)
-        panic("call16 with invalid stack\n");
+    if (getesp() > MAIN_STACK_MAX)
+        panic("call16_sloppy with invalid stack\n");
     func -= BUILD_BIOS_ADDR;
+    Call32Method = 0;
+    u32 stackseg = GET_LOW(StackSeg);
     asm volatile(
         // Transition to 16bit mode
         "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
-        "  jmp transition16\n"
-        // Call func
+        "  jmp transition16big\n"
+        // Setup ss/esp and call func
         "  .code16\n"
-        "1:movl %2, %%edx\n"
+        "1:movl %3, %%ecx\n"
+        "  shll $4, %3\n"
+        "  movw %%cx, %%ss\n"
+        "  subl %3, %%esp\n"
+        "  movw %%cx, %%ds\n"
+        "  movl %2, %%edx\n"
         "  calll *%1\n"
-        // Return to 32bit
+        // Return to 32bit and restore esp
         "  movl $2f, %%edx\n"
         "  jmp transition32\n"
         "  .code32\n"
-        "2:\n"
+        "2:addl %3, %%esp\n"
         : "+a" (eax)
-        : "r" (func), "r" (edx)
+        : "r" (func), "r" (edx), "r" (stackseg)
         : "edx", "ecx", "cc", "memory");
+    Call32Method = C32_SLOPPY;
     return eax;
 }
 
-// Call a 16bit SeaBIOS function in "big real" mode.
+// Call a 32bit SeaBIOS function from a 16bit SeaBIOS function.
+u32 VISIBLE16
+call32(void *func, u32 eax, u32 errret)
+{
+    ASSERT16();
+    u32 cr0 = getcr0();
+    if (cr0 & CR0_PE)
+        // Called in 16bit protected mode?!
+        return errret;
+    return call32_sloppy(func, eax);
+}
+
+// Call a 16bit SeaBIOS function from a 32bit SeaBIOS function.
 u32 FUNCFSEG
-call16big(u32 eax, u32 edx, void *func)
+call16(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
     if (getesp() > BUILD_STACK_ADDR)
-        panic("call16big with invalid stack\n");
+        panic("call16 with invalid stack\n");
     func -= BUILD_BIOS_ADDR;
     asm volatile(
         // Transition to 16bit mode
         "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
-        "  jmp transition16big\n"
+        "  jmp transition16\n"
         // Call func
         "  .code16\n"
         "1:movl %2, %%edx\n"
@@ -142,38 +158,30 @@ call16big(u32 eax, u32 edx, void *func)
     return eax;
 }
 
-// Jump back to 16bit mode while in 32bit mode from call32()
+// Call a 16bit SeaBIOS function in "big real" mode.
 u32 FUNCFSEG
-call16_sloppy(u32 eax, u32 edx, void *func)
+call16big(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
-    if (getesp() > MAIN_STACK_MAX)
-        panic("call16_sloppy with invalid stack\n");
+    if (getesp() > BUILD_STACK_ADDR)
+        panic("call16big with invalid stack\n");
     func -= BUILD_BIOS_ADDR;
-    Call32Method = 0;
-    u32 stackseg = GET_LOW(StackSeg);
     asm volatile(
         // Transition to 16bit mode
         "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
         "  jmp transition16big\n"
-        // Setup ss/esp and call func
+        // Call func
         "  .code16\n"
-        "1:movl %3, %%ecx\n"
-        "  shll $4, %3\n"
-        "  movw %%cx, %%ss\n"
-        "  subl %3, %%esp\n"
-        "  movw %%cx, %%ds\n"
-        "  movl %2, %%edx\n"
+        "1:movl %2, %%edx\n"
         "  calll *%1\n"
-        // Return to 32bit and restore esp
+        // Return to 32bit
         "  movl $2f, %%edx\n"
         "  jmp transition32\n"
         "  .code32\n"
-        "2:addl %3, %%esp\n"
+        "2:\n"
         : "+a" (eax)
-        : "r" (func), "r" (edx), "r" (stackseg)
+        : "r" (func), "r" (edx)
         : "edx", "ecx", "cc", "memory");
-    Call32Method = C32_SLOPPY;
     return eax;
 }
 
-- 
1.9.3

[PATCH 11/12] Fully restore 16bit state during call16_sloppy()

[Kevin O'Connor]

      When transitioning back to 16bit mode from within call32(), restore
the full state (cmos index, gdt, fs/gs) in addition to restoring the
original stack.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/stacks.c | 93 ++++++++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 63 insertions(+), 30 deletions(-)

diff --git a/src/stacks.c b/src/stacks.c
index 2a7be06..eee658f 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -21,29 +21,64 @@
  * 16bit / 32bit calling
  ****************************************************************/
 
-u16 StackSeg VARLOW;
-u8 Call32Method VARLOW;
+struct {
+    u8 method;
+    u8 cmosindex;
+    u16 ss, fs, gs;
+    struct descloc_s gdt;
+} Call32Data VARLOW;
+
 #define C32_SLOPPY 1
 
-// Call a C function in 32bit mode.  This clobbers the 16bit segment
-// selector registers.
-static u32
-call32_sloppy(void *func, u32 eax)
+// Backup state in preparation for call32_sloppy()
+static void
+call32_sloppy_prep(void)
 {
-    ASSERT16();
     // Backup cmos index register and disable nmi
     u8 cmosindex = inb(PORT_CMOS_INDEX);
     outb(cmosindex | NMI_DISABLE_BIT, PORT_CMOS_INDEX);
     inb(PORT_CMOS_DATA);
+    SET_LOW(Call32Data.cmosindex, cmosindex);
 
-    // Backup fs/gs and gdt
-    u16 fs = GET_SEG(FS), gs = GET_SEG(GS);
+    // Backup ss/fs/gs and gdt
+    SET_LOW(Call32Data.ss, GET_SEG(SS));
+    SET_LOW(Call32Data.fs, GET_SEG(FS));
+    SET_LOW(Call32Data.gs, GET_SEG(GS));
     struct descloc_s gdt;
     sgdt(&gdt);
+    SET_LOW(Call32Data.gdt.length, gdt.length);
+    SET_LOW(Call32Data.gdt.addr, gdt.addr);
+
+    SET_LOW(Call32Data.method, C32_SLOPPY);
+}
 
-    u16 oldstackseg = GET_LOW(StackSeg);
-    SET_LOW(StackSeg, GET_SEG(SS));
-    SET_LOW(Call32Method, C32_SLOPPY);
+// Restore state backed up during call32_sloppy()
+static void
+call32_sloppy_post(void)
+{
+    SET_LOW(Call32Data.method, 0);
+    SET_LOW(Call32Data.ss, 0);
+
+    // Restore gdt and fs/gs
+    struct descloc_s gdt;
+    gdt.length = GET_LOW(Call32Data.gdt.length);
+    gdt.addr = GET_LOW(Call32Data.gdt.addr);
+    lgdt(&gdt);
+    SET_SEG(FS, GET_LOW(Call32Data.fs));
+    SET_SEG(GS, GET_LOW(Call32Data.gs));
+
+    // Restore cmos index register
+    outb(GET_LOW(Call32Data.cmosindex), PORT_CMOS_INDEX);
+    inb(PORT_CMOS_DATA);
+}
+
+// Call a C function in 32bit mode.  This clobbers the 16bit segment
+// selector registers.
+static u32
+call32_sloppy(void *func, u32 eax)
+{
+    ASSERT16();
+    call32_sloppy_prep();
     u32 bkup_ss, bkup_esp;
     asm volatile(
         // Backup ss/esp / set esp to flat stack location
@@ -69,21 +104,20 @@ call32_sloppy(void *func, u32 eax)
         : "=&r" (bkup_ss), "=&r" (bkup_esp), "+a"
(eax)
         : "r" (func)
         : "ecx", "edx", "cc", "memory");
-
-    SET_LOW(Call32Method, 0);
-    SET_LOW(StackSeg, oldstackseg);
-
-    // Restore gdt and fs/gs
-    lgdt(&gdt);
-    SET_SEG(FS, fs);
-    SET_SEG(GS, gs);
-
-    // Restore cmos index register
-    outb(cmosindex, PORT_CMOS_INDEX);
-    inb(PORT_CMOS_DATA);
+    call32_sloppy_post();
     return eax;
 }
 
+// 16bit handler code called from call16_sloppy()
+u32 VISIBLE16
+call16_sloppy_helper(u32 eax, u32 edx, u32 (*func)(u32 eax, u32 edx))
+{
+    call32_sloppy_post();
+    u32 ret = func(eax, edx);
+    call32_sloppy_prep();
+    return ret;
+}
+
 // Jump back to 16bit mode while in 32bit mode from call32_sloppy()
 u32 FUNCFSEG
 call16_sloppy(u32 eax, u32 edx, void *func)
@@ -92,8 +126,7 @@ call16_sloppy(u32 eax, u32 edx, void *func)
     if (getesp() > MAIN_STACK_MAX)
         panic("call16_sloppy with invalid stack\n");
     func -= BUILD_BIOS_ADDR;
-    Call32Method = 0;
-    u32 stackseg = GET_LOW(StackSeg);
+    u32 stackseg = Call32Data.ss;
     asm volatile(
         // Transition to 16bit mode
         "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%edx\n"
@@ -106,7 +139,8 @@ call16_sloppy(u32 eax, u32 edx, void *func)
         "  subl %3, %%esp\n"
         "  movw %%cx, %%ds\n"
         "  movl %2, %%edx\n"
-        "  calll *%1\n"
+        "  movl %1, %%ecx\n"
+        "  calll _cfunc16_call16_sloppy_helper\n"
         // Return to 32bit and restore esp
         "  movl $2f, %%edx\n"
         "  jmp transition32\n"
@@ -115,7 +149,6 @@ call16_sloppy(u32 eax, u32 edx, void *func)
         : "+a" (eax)
         : "r" (func), "r" (edx), "r" (stackseg)
         : "edx", "ecx", "cc", "memory");
-    Call32Method = C32_SLOPPY;
     return eax;
 }
 
@@ -190,7 +223,7 @@ static u32
 call16_back(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
-    if (Call32Method == C32_SLOPPY)
+    if (Call32Data.method == C32_SLOPPY)
         return call16_sloppy(eax, edx, func);
     return call16(eax, edx, func);
 }
@@ -322,7 +355,7 @@ __call16_int(struct bregs *callregs, u16 offset)
     callregs->code.offset = offset;
     if (!MODESEGMENT) {
         callregs->code.seg = SEG_BIOS;
-        _farcall16((void*)callregs - StackSeg * 16, StackSeg);
+        _farcall16((void*)callregs - Call32Data.ss * 16, Call32Data.ss);
         return;
     }
     callregs->code.seg = GET_SEG(CS);
-- 
1.9.3

[PATCH 12/12] Implement call32 mechanism using SMIs.

[Kevin O'Connor]

      Add support for jumping into 32bit mode using a System Management Mode
(SMM) handler.  When available, this allows SeaBIOS to transition to
32bit mode even when called in vm86 mode.  It will also prevent the
clobbering of the segment registers.

Currently, the SMM mode is only supported in QEMU when running in TCG
mode.  Also, QEMU v2.1 (or later) is needed for it to work when in
vm86 mode.

Signed-off-by: Kevin O'Connor <kevin(a)koconnor.net&gt;
---
 src/Kconfig  |   4 ++
 src/fw/smm.c |  46 +++++++++++++++++++-
 src/stacks.c | 138 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/stacks.h |   5 +++
 4 files changed, 192 insertions(+), 1 deletion(-)

diff --git a/src/Kconfig b/src/Kconfig
index a863866..a1c0c1e 100644
--- a/src/Kconfig
+++ b/src/Kconfig
@@ -292,6 +292,10 @@ menu "Hardware support"
         default y
         help
             Support System Management Mode (on emulators).
+    config CALL32_SMM
+        bool
+        depends on USE_SMM
+        default y
     config MTRR_INIT
         depends on QEMU
         bool "Initialize MTRRs"
diff --git a/src/fw/smm.c b/src/fw/smm.c
index 5b26977..dabc677 100644
--- a/src/fw/smm.c
+++ b/src/fw/smm.c
@@ -13,6 +13,7 @@
 #include "hw/pci_regs.h" // PCI_DEVICE_ID
 #include "output.h" // dprintf
 #include "paravirt.h" // PORT_SMI_STATUS
+#include "stacks.h" // HaveSmmCall32
 #include "string.h" // memcpy
 #include "util.h" // smm_setup
 #include "x86.h" // wbinvd
@@ -42,7 +43,9 @@ struct smm_state {
 };
 
 struct smm_layout {
-    u8 stack[0x8000];
+    struct smm_state backup1;
+    struct smm_state backup2;
+    u8 stack[0x7c00];
     u64 codeentry;
     u8 pad_8008[0x7df8];
     struct smm_state cpu;
@@ -69,8 +72,49 @@ handle_smi(u16 cs)
         }
         // indicate to smm_relocate_and_restore() that the SMM code was executed
         outb(0x00, PORT_SMI_STATUS);
+
+        if (CONFIG_CALL32_SMM) {
+            // Backup current cpu state for SMM trampolining
+            struct smm_layout *newsmm = (void*)BUILD_SMM_ADDR;
+            memcpy(&newsmm->backup1, &smm->cpu,
sizeof(newsmm->backup1));
+            memcpy(&newsmm->backup2, &smm->cpu,
sizeof(newsmm->backup2));
+            HaveSmmCall32 = 1;
+        }
+
         return;
     }
+
+    if (CONFIG_CALL32_SMM && cmd == CALL32SMM_CMDID) {
+        if (smm->cpu.i32.smm_rev == SMM_REV_I32) {
+            u32 regs[8];
+            memcpy(regs, &smm->cpu.i32.eax, sizeof(regs));
+            if (smm->cpu.i32.ecx == CALL32SMM_ENTERID) {
+                dprintf(9, "smm cpu call pc=%x esp=%x\n", regs[3], regs[4]);
+                memcpy(&smm->backup2, &smm->cpu, sizeof(smm->backup2));
+                memcpy(&smm->cpu, &smm->backup1, sizeof(smm->cpu));
+                memcpy(&smm->cpu.i32.eax, regs, sizeof(regs));
+                smm->cpu.i32.eip = regs[3];
+            } else if (smm->cpu.i32.ecx == CALL32SMM_RETURNID) {
+                dprintf(9, "smm cpu ret %x esp=%x\n", regs[3], regs[4]);
+                memcpy(&smm->cpu, &smm->backup2, sizeof(smm->cpu));
+                memcpy(&smm->cpu.i32.eax, regs, sizeof(regs));
+                smm->cpu.i32.eip = regs[3];
+            }
+        } else if (smm->cpu.i64.smm_rev == SMM_REV_I64) {
+            u64 regs[8];
+            memcpy(regs, &smm->cpu.i64.rdi, sizeof(regs));
+            if ((u32)smm->cpu.i64.rcx == CALL32SMM_ENTERID) {
+                memcpy(&smm->backup2, &smm->cpu, sizeof(smm->backup2));
+                memcpy(&smm->cpu, &smm->backup1, sizeof(smm->cpu));
+                memcpy(&smm->cpu.i64.rdi, regs, sizeof(regs));
+                smm->cpu.i64.rip = (u32)regs[4];
+            } else if ((u32)smm->cpu.i64.rcx == CALL32SMM_RETURNID) {
+                memcpy(&smm->cpu, &smm->backup2, sizeof(smm->cpu));
+                memcpy(&smm->cpu.i64.rdi, regs, sizeof(regs));
+                smm->cpu.i64.rip = (u32)regs[4];
+            }
+        }
+    }
 }
 
 extern void entry_smi(void);
diff --git a/src/stacks.c b/src/stacks.c
index eee658f..5ef7c55 100644
--- a/src/stacks.c
+++ b/src/stacks.c
@@ -6,6 +6,7 @@
 
 #include "biosvar.h" // GET_GLOBAL
 #include "bregs.h" // CR0_PE
+#include "fw/paravirt.h" // PORT_SMI_CMD
 #include "hw/rtc.h" // rtc_use
 #include "list.h" // hlist_node
 #include "malloc.h" // free
@@ -29,6 +30,139 @@ struct {
 } Call32Data VARLOW;
 
 #define C32_SLOPPY 1
+#define C32_SMM    2
+
+int HaveSmmCall32 VARFSEG;
+
+// Backup state in preparation for call32_smm()
+static void
+call32_smm_prep(void)
+{
+    // Backup cmos index register and disable nmi
+    u8 cmosindex = inb(PORT_CMOS_INDEX);
+    outb(cmosindex | NMI_DISABLE_BIT, PORT_CMOS_INDEX);
+    inb(PORT_CMOS_DATA);
+    SET_LOW(Call32Data.cmosindex, cmosindex);
+
+    // Backup ss
+    SET_LOW(Call32Data.ss, GET_SEG(SS));
+
+    SET_LOW(Call32Data.method, C32_SMM);
+}
+
+// Restore state backed up during call32_smm()
+static void
+call32_smm_post(void)
+{
+    SET_LOW(Call32Data.method, 0);
+    SET_LOW(Call32Data.ss, 0);
+
+    // Restore cmos index register
+    outb(GET_LOW(Call32Data.cmosindex), PORT_CMOS_INDEX);
+    inb(PORT_CMOS_DATA);
+}
+
+// Call a SeaBIOS C function in 32bit mode using smm trampoline
+static u32
+call32_smm(void *func, u32 eax)
+{
+    ASSERT16();
+    dprintf(9, "call32_smm %p %x\n", func, eax);
+    call32_smm_prep();
+    u32 bkup_esp;
+    asm volatile(
+        // Backup esp / set esp to flat stack location
+        "  movl %%esp, %0\n"
+        "  movl %%ss, %%eax\n"
+        "  shll $4, %%eax\n"
+        "  addl %%eax, %%esp\n"
+
+        // Transition to 32bit mode, call func, return to 16bit
+        "  movl $" __stringify(CALL32SMM_CMDID) ", %%eax\n"
+        "  movl $" __stringify(CALL32SMM_ENTERID) ", %%ecx\n"
+        "  movl $(" __stringify(BUILD_BIOS_ADDR) " + 1f), %%ebx\n"
+        "  outb %%al, $" __stringify(PORT_SMI_CMD) "\n"
+        "  rep; nop\n"
+        "  hlt\n"
+        "  .code32\n"
+
+        "1:movl %1, %%eax\n"
+        "  calll *%2\n"
+        "  movl %%eax, %1\n"
+
+        "  movl $" __stringify(CALL32SMM_CMDID) ", %%eax\n"
+        "  movl $" __stringify(CALL32SMM_RETURNID) ", %%ecx\n"
+        "  movl $2f, %%ebx\n"
+        "  outb %%al, $" __stringify(PORT_SMI_CMD) "\n"
+        "  rep; nop\n"
+        "  hlt\n"
+
+        // Restore esp
+        "  .code16gcc\n"
+        "2:movl %0, %%esp\n"
+        : "=&r" (bkup_esp), "+r" (eax)
+        : "r" (func)
+        : "eax", "ecx", "edx", "ebx",
"cc", "memory");
+    call32_smm_post();
+
+    dprintf(9, "call32_smm done %p %x\n", func, eax);
+    return eax;
+}
+
+// 16bit handler code called from call16_smm()
+u32 VISIBLE16
+call16_smm_helper(u32 eax, u32 edx, u32 (*func)(u32 eax, u32 edx))
+{
+    if (!CONFIG_CALL32_SMM)
+        return eax;
+    call32_smm_post();
+    u32 ret = func(eax, edx);
+    call32_smm_prep();
+    return ret;
+}
+
+u32 FUNCFSEG
+call16_smm(u32 eax, u32 edx, void *func)
+{
+    ASSERT32FLAT();
+    if (!CONFIG_CALL32_SMM)
+        return eax;
+    func -= BUILD_BIOS_ADDR;
+    dprintf(9, "call16_smm %p %x %x\n", func, eax, edx);
+    u32 stackoffset = Call32Data.ss << 4;
+    asm volatile(
+        // Restore esp
+        "  subl %0, %%esp\n"
+
+        // Transition to 16bit mode, call func, return to 32bit
+        "  movl $" __stringify(CALL32SMM_CMDID) ", %%eax\n"
+        "  movl $" __stringify(CALL32SMM_RETURNID) ", %%ecx\n"
+        "  movl $(1f - " __stringify(BUILD_BIOS_ADDR) "), %%ebx\n"
+        "  outb %%al, $" __stringify(PORT_SMI_CMD) "\n"
+        "  rep; nop\n"
+        "  hlt\n"
+
+        "  .code16gcc\n"
+        "1:movl %1, %%eax\n"
+        "  movl %3, %%ecx\n"
+        "  calll _cfunc16_call16_smm_helper\n"
+        "  movl %%eax, %1\n"
+
+        "  movl $" __stringify(CALL32SMM_CMDID) ", %%eax\n"
+        "  movl $" __stringify(CALL32SMM_ENTERID) ", %%ecx\n"
+        "  movl $2f, %%ebx\n"
+        "  outb %%al, $" __stringify(PORT_SMI_CMD) "\n"
+        "  rep; nop\n"
+        "  hlt\n"
+
+        // Set esp to flat stack location
+        "  .code32\n"
+        "2:addl %0, %%esp\n"
+        : "+r" (stackoffset), "+r" (eax), "+d" (edx)
+        : "r" (func)
+        : "eax", "ecx", "ebx", "cc",
"memory");
+    return eax;
+}
 
 // Backup state in preparation for call32_sloppy()
 static void
@@ -157,6 +291,8 @@ u32 VISIBLE16
 call32(void *func, u32 eax, u32 errret)
 {
     ASSERT16();
+    if (CONFIG_CALL32_SMM && GET_GLOBAL(HaveSmmCall32))
+        return call32_smm(func, eax);
     u32 cr0 = getcr0();
     if (cr0 & CR0_PE)
         // Called in 16bit protected mode?!
@@ -223,6 +359,8 @@ static u32
 call16_back(u32 eax, u32 edx, void *func)
 {
     ASSERT32FLAT();
+    if (CONFIG_CALL32_SMM && Call32Data.method == C32_SMM)
+        return call16_smm(eax, edx, func);
     if (Call32Data.method == C32_SLOPPY)
         return call16_sloppy(eax, edx, func);
     return call16(eax, edx, func);
diff --git a/src/stacks.h b/src/stacks.h
index c3ddc17..82c4c3c 100644
--- a/src/stacks.h
+++ b/src/stacks.h
@@ -4,7 +4,12 @@
 
 #include "types.h" // u32
 
+#define CALL32SMM_CMDID    0xb5
+#define CALL32SMM_ENTERID  0x1234
+#define CALL32SMM_RETURNID 0x5678
+
 // stacks.c
+extern int HaveSmmCall32;
 u32 call32(void *func, u32 eax, u32 errret);
 extern u8 ExtraStack[], *StackPos;
 u32 stack_hop(u32 eax, u32 edx, void *func);
-- 
1.9.3

[PATCH 12/12] Implement call32 mechanism using SMIs.

[Paolo Bonzini]

      Il 30/09/2014 20:38, Kevin O'Connor ha scritto:
  +    config CALL32_SMM
 +        bool
 +        depends on USE_SMM
 +        default y
I think it is a bit too early to make this the default.

Also, it will be false on regular hardware---any plans to change this?
Port 0xb2 is more or less universal, though enabling it requires
chipset-specific initialization.

Paolo

[PATCH 12/12] Implement call32 mechanism using SMIs.

[Paolo Bonzini]

      Il 02/10/2014 21:38, Paolo Bonzini ha scritto:
  > +   
config CALL32_SMM
 > +        bool
 > +        depends on USE_SMM
 > +        default y
 I think it is a bit too early to make this the default.
Nevermind, I cannot read C.

Paolo

[PATCH 12/12] Implement call32 mechanism using SMIs.

[Kevin O'Connor]

      On Thu, Oct 02, 2014 at 09:38:45PM +0200, Paolo Bonzini wrote:
  Il 30/09/2014 20:38, Kevin O'Connor ha scritto:
  +    config CALL32_SMM
 +        bool
 +        depends on USE_SMM
 +        default y
 
 I think it is a bit too early to make this the default.
Thanks for reviewing.  The CONFIG_CALL32_SMM is currently only present
to reduce the code size in non-qemu configs.  It could be promoted to
a user visible config choice, but I don't think many people would want
to change it (it's currently only applicable to QEMU TCG anyway).

  Also, it will be false on regular hardware---any plans
to change this?
 Port 0xb2 is more or less universal, though enabling it requires
 chipset-specific initialization.
I think it would be useful to have this (or something similar) on kvm,
coreboot, and csm.  On coreboot, I imagine the coreboot code would
have the smm handler and a coreboot table would pass in the port and
equivalent of CALL32SMM_CMDID to activate the support in seabios.  On
kvm, I think the easiest would be to get psuedo smm support working in
kvm.  Not sure the best way to do it on CSM.  I haven't looked at this
in any depth though.

-Kevin

[PATCH 12/12] Implement call32 mechanism using SMIs.

[Paolo Bonzini]

      Il 02/10/2014 22:35, Kevin O'Connor ha scritto:
  On kvm, I think the easiest would be to get psuedo smm
support
 working in kvm.
That's on the todo list.

  Not sure the best way to do it on CSM.
There's support for that; OVMF needs to provide an SmmInit hook.  It
probably would map into an extra, non-standard Compatibility16 call.

Paolo

[Kevin O'Connor]

      On Tue, Sep 30, 2014 at 02:38:34PM -0400, Kevin O'Connor wrote:
  This series reworks the internal 16bit <-> 32bit
trampolining that the
 SeaBIOS code uses during runtime, and it adds support for using System
 Management Mode (SMM) for performing a more correct switch to 32bit
 mode during runtime.  Currently, QEMU v2.1 in TCG mode is needed in
 order for the SMM handler to be utilized.
FYI, I have pushed this patch series to the SeaBIOS master branch.

-Kevin