SPARC64: Forth re-entrancy during MMU I/D TLB miss

[Mark Cave-Ayland]

      Hi everyone,

So having a poke around various bits of documentation relating to 
implementing pgmap@, I've come to the realisation that since the Solaris 
kernel needs to install it's own va>tte-data implementation into 
OpenBIOS then it's going to be necessary to call back into Forth from 
the D/I-MMU miss traps.

I've been looking around the code in arch/sparc64/switch.S but it looks 
as if this code only swaps between 2 individual contexts - am I right in 
thinking there needs to be a way of specifying the memory address of the 
context for this code to be reusable during a trap? Or would a better 
way be to duplicate this file but with another static context used only 
for MMU miss traps?

Another question for PPC users: will pgmap@/pgmap! be required there 
too? I'm trying to figure out if this is something that needs to live 
within libopenbios/ofmem_common.c or whether it's just a SPARC-specific 
hack. It looks as if these words are also required for SPARC32 so I'm 
leaning towards the former...


ATB,

Mark.

-- 
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: [phone]

Sirius Labs: http://www.siriusit.co.uk/labs

[Blue Swirl]

      On Tue, Aug 31, 2010 at 3:38 PM, Mark Cave-Ayland
<mark.cave-ayland(a)siriusit.co.uk&gt; wrote:
  Hi everyone,

 So having a poke around various bits of documentation relating to
 implementing pgmap@, I've come to the realisation that since the Solaris
 kernel needs to install it's own va>tte-data implementation into OpenBIOS
 then it's going to be necessary to call back into Forth from the D/I-MMU
 miss traps.
The thought of that gives me the shivers.

  I've been looking around the code in
arch/sparc64/switch.S but it looks as
 if this code only swaps between 2 individual contexts - am I right in
 thinking there needs to be a way of specifying the memory address of the
 context for this code to be reusable during a trap? Or would a better way be
 to duplicate this file but with another static context used only for MMU
 miss traps?
Sparc64 should use the same approach as Sparc32 (and x86): save the
context to stack. It's even easier than Sparc32 because there is a
window flush instruction.

[Mark Cave-Ayland]

      Blue Swirl wrote:

  So having a
poke around various bits of documentation relating to
 implementing pgmap@, I've come to the realisation that since the Solaris
 kernel needs to install it's own va>tte-data implementation into OpenBIOS
 then it's going to be necessary to call back into Forth from the D/I-MMU
 miss traps.
 
 The thought of that gives me the shivers.
It sounds scary, but I don't think it will be too bad. Looking at 
arch/sparc64/entry.S, it seems like the OpenBIOS memory regions are 
locked into the TLB as N x 512K entries - so as long as we don't end up 
with MMU miss trap recursion, we should be good.

The plan I have in my mind goes something like this:

- Alter the D/I-MMU trap miss handlers so they invoke a C function 
rather than iterate through the translations structure in ASM.

- Initially make the C function do exactly the same as the current ASM 
for testing.

- Create a simple Forth function to perform the address translation.

- Modify the C function to invoke feval(...) on the translation function 
instead of iterating through the translations structure directly.

  I've been
looking around the code in arch/sparc64/switch.S but it looks as
 if this code only swaps between 2 individual contexts - am I right in
 thinking there needs to be a way of specifying the memory address of the
 context for this code to be reusable during a trap? Or would a better way be
 to duplicate this file but with another static context used only for MMU
 miss traps?
 
 Sparc64 should use the same approach as Sparc32 (and x86): save the
 context to stack. It's even easier than Sparc32 because there is a
 window flush instruction.
Hmmm. SPARC64 definitely doesn't do this right at the moment - instead 
of saving context to the stack, it seems to save it back into the static 
context structure :(

Looks like the first trick is to figure out how to store the context on 
the stack instead. However, since __switch_context automatically 
restores all registers upon exit, I'm not sure even then we could use it 
  within a trap handler since we'd want the TTE data entry returned in 
one of the registers when we are finished so we can add the entry to the 
MMU. Maybe there is a better way of refactoring this code so we can 
share the majority of it?

Also I see that arch/sparc64/switch.S is marked as ".align 4" - 
shouldn't this be 8 for a 64-bit architecture? Not that it seems to 
matter at the moment...


ATB,

Mark.

-- 
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: [phone]

Sirius Labs: http://www.siriusit.co.uk/labs

[Blue Swirl]

      On Wed, Sep 1, 2010 at 12:28 PM, Mark Cave-Ayland
<mark.cave-ayland(a)siriusit.co.uk&gt; wrote:
  Blue Swirl wrote:

  So having
a poke around various bits of documentation relating to
 implementing pgmap@, I've come to the realisation that since the Solaris
 kernel needs to install it's own va>tte-data implementation into OpenBIOS
 then it's going to be necessary to call back into Forth from the D/I-MMU
 miss traps.
 The thought of that gives me the shivers.
 It sounds scary, but I don't think it will be too bad. Looking at
 arch/sparc64/entry.S, it seems like the OpenBIOS memory regions are locked
 into the TLB as N x 512K entries - so as long as we don't end up with MMU
 miss trap recursion, we should be good.

 The plan I have in my mind goes something like this:

 - Alter the D/I-MMU trap miss handlers so they invoke a C function rather
 than iterate through the translations structure in ASM.
The registers need to be saved at this point. There are also several
sets of global registers (%g1-%g7), which may need separate handling.

  - Initially make the C function do exactly the same as
the current ASM for
 testing.

 - Create a simple Forth function to perform the address translation.

 - Modify the C function to invoke feval(...) on the translation function
 instead of iterating through the translations structure directly.

  I've
been looking around the code in arch/sparc64/switch.S but it looks
 as
 if this code only swaps between 2 individual contexts - am I right in
 thinking there needs to be a way of specifying the memory address of the
 context for this code to be reusable during a trap? Or would a better way
 be
 to duplicate this file but with another static context used only for MMU
 miss traps?
 Sparc64 should use the same approach as Sparc32 (and x86): save the
 context to stack. It's even easier than Sparc32 because there is a
 window flush instruction.
 Hmmm. SPARC64 definitely doesn't do this right at the moment - instead of
 saving context to the stack, it seems to save it back into the static
 context structure :(

 Looks like the first trick is to figure out how to store the context on the
 stack instead. However, since __switch_context automatically restores all
 registers upon exit, I'm not sure even then we could use it  within a trap
 handler since we'd want the TTE data entry returned in one of the registers
 when we are finished so we can add the entry to the MMU. Maybe there is a
 better way of refactoring this code so we can share the majority of it?
The saved context could be stored into a location which would be
visible to C/Forth code, so that it could alter the return value
register.

  Also I see that arch/sparc64/switch.S is marked as
".align 4" - shouldn't
 this be 8 for a 64-bit architecture? Not that it seems to matter at the
 moment...
The instructions on Sparc64 still only require 4 byte alignment. If we
also considered cache line effects, the alignment should be 32 or 64
bytes, like Linux .S files. In QEMU, the best alignment probably
depends on host data cache line size.