Hi everyone,
I've just committed a partial implementation of the SUNW,retain word and so have moved further into booting OpenSolaris. Currently it's only a partial implementation because I can't get Qemu to perform a warm restart without crashing the guest, so I may need to revisit this later. Fortunately, the SUNW,retain word itself is there which now enables the OpenSolaris kernel to get further during boot:
0 > go Jumping to entry point 00000000010071d8 for type 0000000000000001... switching to new context: entry point 0x10071d8 stack 0x00000000ffe02a91 / panic[cpu0]/thread=180e000: portid not found
000000000180b750 unix:fill_cpu+418 (ffffffffffe2fbd8, 10b81b0, 10b83d0, 0, 10bac00, ffffffffffffffff) %l0-3: 00000000018127c0 00000000010b81b8 00000000010b8000 ffffffffffffffff %l4-7: 0000000000000000 0000000000000004 00000000ffe2fbd8 0000000000000004 000000000180b870 unix:map_wellknown+124 (ffffffffffe2fbd8, 0, 0, 180b940, 0, 10b8188) %l0-3: 0000000000000001 0000000000000000 0000000000000000 ffffffffffffffff %l4-7: 0000000000000000 00000000010b8000 0000000000000000 000000000180b92c 000000000180b960 unix:map_wellknown_devices+80 (0, f7, 0, 10bb000, 10bb000, 10b8000) %l0-3: 0000000001812c00 0000000000002000 00000000010b8000 0000000000000000 %l4-7: 000000000183ac00 0000000001812400 0000000000000010 0000000000000000 000000000180ba20 unix:mlsetup+308 (180c000, 180bec0, 10b6548, 180c000, 2000, 180e000) %l0-3: 000000000180e000 0000000070002000 0000000000000000 000000000183ac00 %l4-7: 0000000000000000 000000000180c000 0000000070002000 0000000000000000
Looking through the OpenSolaris source code[1], it appears that fill_cpu() is expecting a portid property on the CPU nodes. Also searching further within the same file, it seems have_pci() is expecting a portid property on the PCI bus node too.
Does anyone with experience with real Sun kit know what the portid property for both the CPU and PCI device should be set to? It's fairly obvious that it's an integer, but I'm not sure how it should reflect the virtual hardware.
ATB,
Mark.
[1] http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/sun4u/os/f...
On 5/2/10, Mark Cave-Ayland mark.cave-ayland@siriusit.co.uk wrote:
Hi everyone,
I've just committed a partial implementation of the SUNW,retain word and so have moved further into booting OpenSolaris. Currently it's only a partial implementation because I can't get Qemu to perform a warm restart without crashing the guest, so I may need to revisit this later.
I think OpenBIOS does not flush the I/D TLBs on reset.
Fortunately, the SUNW,retain word itself is there which now enables the OpenSolaris kernel to get further during boot:
0 > go Jumping to entry point 00000000010071d8 for type 0000000000000001... switching to new context: entry point 0x10071d8 stack 0x00000000ffe02a91 / panic[cpu0]/thread=180e000: portid not found
000000000180b750 unix:fill_cpu+418 (ffffffffffe2fbd8, 10b81b0, 10b83d0, 0, 10bac00, ffffffffffffffff) %l0-3: 00000000018127c0 00000000010b81b8 00000000010b8000 ffffffffffffffff %l4-7: 0000000000000000 0000000000000004 00000000ffe2fbd8 0000000000000004 000000000180b870 unix:map_wellknown+124 (ffffffffffe2fbd8, 0, 0, 180b940, 0, 10b8188) %l0-3: 0000000000000001 0000000000000000 0000000000000000 ffffffffffffffff %l4-7: 0000000000000000 00000000010b8000 0000000000000000 000000000180b92c 000000000180b960 unix:map_wellknown_devices+80 (0, f7, 0, 10bb000, 10bb000, 10b8000) %l0-3: 0000000001812c00 0000000000002000 00000000010b8000 0000000000000000 %l4-7: 000000000183ac00 0000000001812400 0000000000000010 0000000000000000 000000000180ba20 unix:mlsetup+308 (180c000, 180bec0, 10b6548, 180c000, 2000, 180e000) %l0-3: 000000000180e000 0000000070002000 0000000000000000 000000000183ac00 %l4-7: 0000000000000000 000000000180c000 0000000070002000 0000000000000000
Looking through the OpenSolaris source code[1], it appears that fill_cpu() is expecting a portid property on the CPU nodes. Also searching further within the same file, it seems have_pci() is expecting a portid property on the PCI bus node too.
Does anyone with experience with real Sun kit know what the portid property for both the CPU and PCI device should be set to? It's fairly obvious that it's an integer, but I'm not sure how it should reflect the virtual hardware.
Most PCI devices have 0x1f in 'upa-portid' and CPU has zero.
Debian package sparc-utils contains some device trees: http://packages.debian.org/lenny/sparc/sparc-utils/filelist
Blue Swirl wrote:
I think OpenBIOS does not flush the I/D TLBs on reset. Most PCI devices have 0x1f in 'upa-portid' and CPU has zero.
I've just seen your commits for this - thanks!
Debian package sparc-utils contains some device trees: http://packages.debian.org/lenny/sparc/sparc-utils/filelist
That's really useful to know. I used these as a reference for creating the extra CPU properties required to stop the divide-by-zero error.
This helps things get further, but in particular it shows that we're missing a whole set of architecture-specific Forth words for manipulating registers. The OpenSolaris kernel tries to execute the following Forth:
sync %tl-c %tstate h# 1860640 x! %g1 h# 1860648 x! %g2 h# 1860650 x! %g3 h# 1860658 x! %g4 h# 1860660 x! %g5 h# 1860668 x! %g6 h# 1860670 x! %g7 h# 1860678 x! %o0 h# 1860680 x! %o1 h# 1860688 x! %o2 h# 1860690 x! %o3 h# 1860698 x! %o4 h# 18606a0 x! %o5 h# 18606a8 x! %o6 h# 18606b0 x! %o7 h# 18606b8 x! %tl-c %tpc h# 18606c0 x! %tl-c %tnpc h# 18606c8 x! %y h# 18606d0 l! %tl-c %tt h# 1843cf8 x! sync ; warning !
(from http://src.opensolaris.org/source/xref/systemz/sirius/usr/src/uts/sun4/os/st...)
It seems that we need some Forth functions for creating/managing CPU state? I suspect I'll have to dig into the OpenBOOT source to find out how these are supposed to work.
ATB,
Mark.
On 2010-5-2 7:12 PM, Mark Cave-Ayland wrote:
[...] This helps things get further, but in particular it shows that we're missing a whole set of architecture-specific Forth words for manipulating registers. The OpenSolaris kernel tries to execute the following Forth:
sync %tl-c %tstate h# 1860640 x! %g1 h# 1860648 x! %g2 h# 1860650 x! %g3 h# 1860658 x! %g4 h# 1860660 x! %g5 h# 1860668 x! %g6 h# 1860670 x! %g7 h# 1860678 x! %o0 h# 1860680 x! %o1 h# 1860688 x! %o2 h# 1860690 x! %o3 h# 1860698 x! %o4 h# 18606a0 x! %o5 h# 18606a8 x! %o6 h# 18606b0 x! %o7 h# 18606b8 x! %tl-c %tpc h# 18606c0 x! %tl-c %tnpc h# 18606c8 x! %y h# 18606d0 l! %tl-c %tt h# 1843cf8 x! sync ; warning !
(from http://src.opensolaris.org/source/xref/systemz/sirius/usr/src/uts/sun4/os/st...)
It seems that we need some Forth functions for creating/managing CPU state? I suspect I'll have to dig into the OpenBOOT source to find out how these are supposed to work.
Those are all SPARC registers. What it's doing is copying the registers to a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by %tl-c - which is presumably the trap-level pointer.
Tarl Neustaedter wrote:
Those are all SPARC registers. What it's doing is copying the registers to a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by %tl-c - which is presumably the trap-level pointer.
Okay, thanks. Looks like I'll have to browse the OpenBoot source to find out the stack diagrams for these words to reimplement them in OpenBIOS.
There's also more huge chunks of Forth embedded in http://src.opensolaris.org/source/xref/systemz/sirius/usr/src/uts/sun4/os/st... to manage the TTEs aswell :( Alas since these aren't part of the official OF specification, it's probably going to take some time to figure out which words/defers are missing and reimplement them.
ATB,
Mark.
On 5/3/10, Mark Cave-Ayland mark.cave-ayland@siriusit.co.uk wrote:
Tarl Neustaedter wrote:
Those are all SPARC registers. What it's doing is copying the registers to
a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by
%tl-c - which is presumably the trap-level pointer.
Okay, thanks. Looks like I'll have to browse the OpenBoot source to find out the stack diagrams for these words to reimplement them in OpenBIOS.
There's also more huge chunks of Forth embedded in http://src.opensolaris.org/source/xref/systemz/sirius/usr/src/uts/sun4/os/st...
The memory map in the file claims that OBP's start address would be 0xf0000000. This would mean 256MB of space, solving some of the memory problems we've had.
On 2010-5-3 3:32 PM, Blue Swirl wrote:
[...]
There's also more huge chunks of Forth embedded in http://src.opensolaris.org/source/xref/systemz/sirius/usr/src/uts/sun4/os/st...
The memory map in the file claims that OBP's start address would be 0xf0000000. This would mean 256MB of space, solving some of the memory problems we've had.
0xf000.0000 is where OBP usually lives in virtual space on Sun machines. In sun4u/sun4s machines it tried to place itself at the very top of the first contiguous chunk of memory (which usually meant at the top of physical memory) - it copies itself out of PROM and into memory for performance reasons.
Still have the item about registers to look up, haven't forgotten it.
Tarl Neustaedter wrote:
Those are all SPARC registers. What it's doing is copying the registers to a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by %tl-c - which is presumably the trap-level pointer.
Another thought about this: surely the act of invoking normal functions within OpenBIOS is enough to cause the register window to shift. Does that mean that the CIF needs to take a snapshot of the registers upon entry and then map this snapshot to forth words such as %g0, %g1 etc.? Otherwise it all seems a bit useless.
ATB,
Mark.
On 2010-5-3 8:16 AM, Mark Cave-Ayland wrote:
Tarl Neustaedter wrote:
Those are all SPARC registers. What it's doing is copying the registers to a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by %tl-c - which is presumably the trap-level pointer.
Another thought about this: surely the act of invoking normal functions within OpenBIOS is enough to cause the register window to shift. Does that mean that the CIF needs to take a snapshot of the registers upon entry and then map this snapshot to forth words such as %g0, %g1 etc.? Otherwise it all seems a bit useless.
Let me look this up later today.
The registers referenced should be in a saved register block (when forth is invoked, it has to save registers from the caller), not live. The forth compiler needs registers itself, so it had to save the caller's information somewhere.
On 2010-5-3 8:16 AM, Mark Cave-Ayland wrote:
Tarl Neustaedter wrote:
Those are all SPARC registers. What it's doing is copying the registers to a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by %tl-c - which is presumably the trap-level pointer.
Another thought about this: surely the act of invoking normal functions within OpenBIOS is enough to cause the register window to shift. Does that mean that the CIF needs to take a snapshot of the registers upon entry and then map this snapshot to forth words such as %g0, %g1 etc.? Otherwise it all seems a bit useless.
This all involves stuff from (assuming you have access to the open-sourced OBP sources) obp/cpu/sparc/fentry9.fth .
forth-entry is how Solaris calls into OBP's client-interface. Notice it stashing registers away on stack.
return-to-c is how forth returns from a client-interface call, restoring stuff.
The actual register definitions are in obp/cpu/sparc/register9.fth - notice it defines them with out-regs, local-regs, and global-regs, which are complex constructs allocating space from the heap and setting up use-actions for them referring to the cpu-state blocks. References to %o7 ect. refer to these variables rather than the live registers.
Unfortunately, this is black magic that I don't understand - I've not dived into the guts of our forth/c interface much before.
On 5/3/10, Tarl Neustaedter tarl-b2@tarl.net wrote:
On 2010-5-3 8:16 AM, Mark Cave-Ayland wrote:
Tarl Neustaedter wrote:
Those are all SPARC registers. What it's doing is copying the registers
to a buffer, presumably to be restored later.
It looks like the trap registers (%tstate, %tnpc, %tt) are prefixed by
%tl-c - which is presumably the trap-level pointer.
Another thought about this: surely the act of invoking normal functions
within OpenBIOS is enough to cause the register window to shift. Does that mean that the CIF needs to take a snapshot of the registers upon entry and then map this snapshot to forth words such as %g0, %g1 etc.? Otherwise it all seems a bit useless.
This all involves stuff from (assuming you have access to the open-sourced OBP sources) obp/cpu/sparc/fentry9.fth .
forth-entry is how Solaris calls into OBP's client-interface. Notice it stashing registers away on stack.
return-to-c is how forth returns from a client-interface call, restoring stuff.
The actual register definitions are in obp/cpu/sparc/register9.fth - notice it defines them with out-regs, local-regs, and global-regs, which are complex constructs allocating space from the heap and setting up use-actions for them referring to the cpu-state blocks. References to %o7 ect. refer to these variables rather than the live registers.
Unfortunately, this is black magic that I don't understand - I've not dived into the guts of our forth/c interface much before.
Maybe we could reuse our register context handling here (arch/sparc64/context.c). Client's context could be passed by start_elf, or just make the context global.
Tarl Neustaedter wrote:
The actual register definitions are in obp/cpu/sparc/register9.fth - notice it defines them with out-regs, local-regs, and global-regs, which are complex constructs allocating space from the heap and setting up use-actions for them referring to the cpu-state blocks. References to %o7 ect. refer to these variables rather than the live registers.
Unfortunately, this is black magic that I don't understand - I've not dived into the guts of our forth/c interface much before.
After quite a bit of time studying the obp code, I think I see what's happening here. On the surface, it looks as if obp/cpu/sparc/registers9.fth only saves/restores the globals. However, if you look deeper, it looks as if the default simple trap handler in OpenBOOT references a global state area (indexed by window pointer) to store the current state before invoking the handler. This area can then be accessed from Forth using the standard functions.
So I believe the correct thing to do is to alter the window fill/spill trap handlers so that they store the current CPU state in the global area, and then add some code to OpenBIOS to allow this code to be accessed from C and Forth. Does this sound right to everyone? If this is the approach required, any ideas/help would be appreciated since SPARC is not my native ASM ;)
ATB,
Mark.
On 5/15/10, Mark Cave-Ayland mark.cave-ayland@siriusit.co.uk wrote:
Tarl Neustaedter wrote:
The actual register definitions are in obp/cpu/sparc/register9.fth -
notice it defines them with out-regs, local-regs, and global-regs, which are complex constructs allocating space from the heap and setting up use-actions for them referring to the cpu-state blocks. References to %o7 ect. refer to these variables rather than the live registers.
Unfortunately, this is black magic that I don't understand - I've not
dived into the guts of our forth/c interface much before.
After quite a bit of time studying the obp code, I think I see what's happening here. On the surface, it looks as if obp/cpu/sparc/registers9.fth only saves/restores the globals. However, if you look deeper, it looks as if the default simple trap handler in OpenBOOT references a global state area (indexed by window pointer) to store the current state before invoking the handler. This area can then be accessed from Forth using the standard functions.
So I believe the correct thing to do is to alter the window fill/spill trap handlers so that they store the current CPU state in the global area, and then add some code to OpenBIOS to allow this code to be accessed from C and Forth. Does this sound right to everyone? If this is the approach required, any ideas/help would be appreciated since SPARC is not my native ASM ;)
I don't think so, window fill/spill is triggered also inside OpenBIOS when the call stack exceeds 8 at the latest.
I'd enhance the context handling, it's used during client interface entry. The CPU state is also stored in a global area which could be accessed.
[...] So I believe the correct thing to do is to alter the window fill/spill trap handlers so that they store the current CPU state in the global area, and then add some code to OpenBIOS to allow this code to be accessed from C and Forth.
Something like that. Be aware that the fill/spill areas only get the i's/o's and l's, the g's have to be handled differently (there are three sets of globals, as I recall - default, alternate and interrupt).
-
Blue Swirl -
Mark Cave-Ayland -
Tarl Neustaedter