This patchset follows on from my previous series and gets NetBSD/OpenBSD to the same point as FreeBSD, i.e. the kernel starts to execute and we begin enumerating hardware.
OpenBSD:
OpenBIOS for Sparc64 Configuration device id QEMU version 1 machine id 0 kernel cmdline CPUs: 1 x SUNW,UltraSPARC-IIi UUID: 00000000-0000-0000-0000-000000000000 Welcome to OpenBIOS v1.1 built on May 19 2014 21:06 Type 'help' for detailed information Trying cdrom:f... Not a bootable ELF image Not a bootable a.out image
Loading FCode image... Loaded 4829 bytes entry point is 0x4000 OpenBSD IEEE 1275 Bootblock 1.3 .. Jumping to entry point 0000000000100000 for type 0000000000000001... switching to new context: entry point 0x100000 stack 0x00000000ffe8aa09
OpenBSD BOOT 1.6
Trying bsd... open /pci@1fe,0/pci-ata@5/ide1@600/cdrom@0:f/etc/random.seed: No such file or directory Booting /pci@1fe,0/pci-ata@5/ide1@600/cdrom@0:f/bsd 3901336@0x1000000+6248@0x13b8798+3261984@0x1800000+932320@0x1b1c620 symbols @ 0xffc5a300 119 start=0x1000000
Unexpected client interface exception: -1 console is /pci@1fe,0/ebus@3/su Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. Copyright (c) 1995-2014 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 5.5 (RAMDISK) #153: Tue Mar 4 15:12:10 MST 2014 deraadt@sparc64.openbsd.org:/usr/src/sys/arch/sparc64/compile/RAMDISK real mem = 134217728 (128MB) avail mem = 122011648 (116MB) mainbus0 at root: OpenBiosTeam,OpenBIOS cpu0 at mainbus0: SUNW,UltraSPARC-IIi (rev 9.1) @ 100 MHz cpu0: physical 256K instruction (64 b/l), 16K data (32 b/l), 256K external (64 b/l) psycho0 at mainbus0: SUNW,sabre, impl 0, version 0, ign 7c0 psycho0: bus range 0-2, PCI bus 0 panic: intr_establish: bad intr number 0 halted
EXIT 0 >
NetBSD:
OpenBIOS for Sparc64 Configuration device id QEMU version 1 machine id 0 kernel cmdline CPUs: 1 x SUNW,UltraSPARC-IIi UUID: 00000000-0000-0000-0000-000000000000 Welcome to OpenBIOS v1.1 built on May 19 2014 21:06 Type 'help' for detailed information Trying cdrom:f... Not a bootable ELF image Not a bootable a.out image
Loading FCode image... Loaded 7478 bytes entry point is 0x4000 NetBSD IEEE 1275 Multi-FS Bootblock Version $NetBSD: bootblk.fth,v 1.13 2010/06/24 00:54:12 eeh Exp $ .. Jumping to entry point 0000000000100000 for type 0000000000000001... switching to new context: entry point 0x100000 stack 0x00000000ffe8aa09
NetBSD/sparc64 OpenFirmware Boot, Revision 1.16
=0x8870a0 Loading netbsd: 8071888+553056+339856 [601032+393301]=0x9cd528 Unimplemented service set-symbol-lookup ([2] -- [0])
Unexpected client interface exception: -1 Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012 The NetBSD Foundation, Inc. All rights reserved. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved.
NetBSD 6.1.2 (GENERIC) total memory = 128 MB avail memory = 111 MB mainbus0 (root): OpenBiosTeam,OpenBIOS: hostid 80123456 cpu0 at mainbus0: SUNW,UltraSPARC-IIi @ 100 MHz, UPA id 0 cpu0: 256K instruction (64 b/l), 16K data (32 b/l), 256K external (64 b/l) psycho0 at mainbus0 psycho0: SUNW,sabre: impl 0, version 0: ign 7c0 bus range 0 to 2; PCI bus 0 psycho_alloc_extent: no "available" property psycho_alloc_extent: no "available" property pci0 at psycho0 ppb0 at pci0 dev 1 function 0: Sun Microsystems Simba PCI Bridge (rev. 0x11) pci1 at ppb0 bus 1 ppb1 at pci0 dev 1 function 1: Sun Microsystems Simba PCI Bridge (rev. 0x11) pci2 at ppb1 bus 2 genfb0 at pci0 dev 2 function 0: unmatched vendor 0x1234 product 0x1111 (rev. 0x00) ebus0 at pci0 dev 3 function 0 ebus0: Sun Microsystems PCIO Ebus2, revision 0x01 panic: ebus ranges: error 2 Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x4: nop db{0}>
Getting these kernels to a point where they start to execute and enumerate hardware is a big milestone. Next step will be to fix up the extra properties required by the *BSDs in the device tree.
Signed-off-by: Mark Cave-Ayland mark.cave-ayland@ilande.co.uk
Mark Cave-Ayland (4): SPARC64: mark PROM memory allocations TTEs as privileged SPARC64: reduce client interface stack usage SPARC64: defer set_trap_table effect until CIF exit SPARC64: preserve window state when invoking CIF functions
openbios-devel/arch/sparc64/call-client.S | 242 ++++++++++++++++++++++++--- openbios-devel/arch/sparc64/ofmem_sparc64.c | 8 +- openbios-devel/arch/sparc64/openbios.c | 8 +- 3 files changed, 227 insertions(+), 31 deletions(-)
This brings the behaviour in line with OBP.
Signed-off-by: Mark Cave-Ayland mark.cave-ayland@ilande.co.uk --- openbios-devel/arch/sparc64/ofmem_sparc64.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/openbios-devel/arch/sparc64/ofmem_sparc64.c b/openbios-devel/arch/sparc64/ofmem_sparc64.c index 16f9247..6e1bb05 100644 --- a/openbios-devel/arch/sparc64/ofmem_sparc64.c +++ b/openbios-devel/arch/sparc64/ofmem_sparc64.c @@ -215,14 +215,14 @@ int ofmem_arch_encode_physaddr(ucell *p, phys_addr_t value) ucell ofmem_arch_default_translation_mode( phys_addr_t phys ) { /* Writable, cacheable */ - /* not privileged and not locked */ - return SPITFIRE_TTE_CP | SPITFIRE_TTE_CV | SPITFIRE_TTE_WRITABLE; + /* Privileged and not locked */ + return SPITFIRE_TTE_CP | SPITFIRE_TTE_CV | SPITFIRE_TTE_WRITABLE | SPITFIRE_TTE_PRIVILEGED; }
ucell ofmem_arch_io_translation_mode( phys_addr_t phys ) { - /* Writable, not privileged and not locked */ - return SPITFIRE_TTE_CV | SPITFIRE_TTE_WRITABLE; + /* Writable, privileged and not locked */ + return SPITFIRE_TTE_CV | SPITFIRE_TTE_WRITABLE | SPITFIRE_TTE_PRIVILEGED; }
/* Architecture-specific OFMEM helpers */
Currently the client interface entrypoint uses the existing client stack to save state before invoking the main C implementation function. Unfortunately some clients such as OpenBSD have a very small stack available which can be exhausted by internal OpenBIOS calls.
Reduce the stack space required by just saving the globals onto the existing stack and instead switching to a separate stack when invoking the OpenBIOS client interface.
Signed-off-by: Mark Cave-Ayland mark.cave-ayland@ilande.co.uk --- openbios-devel/arch/sparc64/call-client.S | 67 ++++++++++++++++++++--------- 1 file changed, 46 insertions(+), 21 deletions(-)
diff --git a/openbios-devel/arch/sparc64/call-client.S b/openbios-devel/arch/sparc64/call-client.S index d15f177..18de704 100644 --- a/openbios-devel/arch/sparc64/call-client.S +++ b/openbios-devel/arch/sparc64/call-client.S @@ -2,8 +2,12 @@
.data .align 8 -client_globals: - .skip 64 + + .skip 16384 +openbios_stack: + +client_stack: + .xword 0
.text .align 4 @@ -19,26 +23,47 @@ client_globals: sparc64_of_client_interface: /* make sure caller's windows are on caller's stack */ flushw - save %sp, -248, %sp - stx %g1, [%sp + 2047 + 192 + 0] - stx %g2, [%sp + 2047 + 192 + 8] - stx %g3, [%sp + 2047 + 192 + 16] - stx %g4, [%sp + 2047 + 192 + 24] - stx %g5, [%sp + 2047 + 192 + 32] - stx %g6, [%sp + 2047 + 192 + 40] - stx %g7, [%sp + 2047 + 192 + 48] + + /* Save globals on callers stack */ + add %sp, -56, %sp + + stx %g1, [%sp + 2047 + 0] + stx %g2, [%sp + 2047 + 8] + stx %g3, [%sp + 2047 + 16] + stx %g4, [%sp + 2047 + 24] + stx %g5, [%sp + 2047 + 32] + stx %g6, [%sp + 2047 + 40] + stx %g7, [%sp + 2047 + 48] + + /* Save existing stack and move to openbios stack */ + setx client_stack, %g6, %g7 + stx %sp, [%g7] + setx openbios_stack - 2047 - 192, %g6, %g7 + mov %g7, %sp + + /* Call client inteface */ + save %sp, -192, %sp
call of_client_interface mov %i0, %o0 - mov %o0, %i0 - - ldx [%sp + 2047 + 192 + 0], %g1 - ldx [%sp + 2047 + 192 + 8], %g2 - ldx [%sp + 2047 + 192 + 16], %g3 - ldx [%sp + 2047 + 192 + 24], %g4 - ldx [%sp + 2047 + 192 + 32], %g5 - ldx [%sp + 2047 + 192 + 40], %g6 - ldx [%sp + 2047 + 192 + 48], %g7 - return %i7+8 - nop
+ mov %o0, %i0 + restore %sp, 192, %sp + + /* Restore stack */ + setx client_stack, %g6, %g7 + ldx [%g7], %sp + + /* Restore globals */ + ldx [%sp + 2047 + 0], %g1 + ldx [%sp + 2047 + 8], %g2 + ldx [%sp + 2047 + 16], %g3 + ldx [%sp + 2047 + 24], %g4 + ldx [%sp + 2047 + 32], %g5 + ldx [%sp + 2047 + 40], %g6 + ldx [%sp + 2047 + 48], %g7 + + add %sp, 56, %sp + + jmp %o7+8 + nop
Since OpenBIOS invokes window fill/spill traps during normal operation, make sure that we defer setting the client trap table until we return control back to the client.
Signed-off-by: Mark Cave-Ayland mark.cave-ayland@ilande.co.uk --- openbios-devel/arch/sparc64/call-client.S | 14 +++++++++++++- openbios-devel/arch/sparc64/openbios.c | 8 ++++++-- 2 files changed, 19 insertions(+), 3 deletions(-)
diff --git a/openbios-devel/arch/sparc64/call-client.S b/openbios-devel/arch/sparc64/call-client.S index 18de704..47d937f 100644 --- a/openbios-devel/arch/sparc64/call-client.S +++ b/openbios-devel/arch/sparc64/call-client.S @@ -1,4 +1,4 @@ - .globl sparc64_of_client_interface + .globl sparc64_of_client_interface, client_tba
.data .align 8 @@ -8,6 +8,8 @@ openbios_stack:
client_stack: .xword 0 +client_tba: + .xword 0
.text .align 4 @@ -35,6 +37,11 @@ sparc64_of_client_interface: stx %g6, [%sp + 2047 + 40] stx %g7, [%sp + 2047 + 48]
+ /* Save client trap table */ + setx client_tba, %g6, %g7 + rdpr %tba, %g6 + stx %g6, [%g7] + /* Save existing stack and move to openbios stack */ setx client_stack, %g6, %g7 stx %sp, [%g7] @@ -54,6 +61,11 @@ sparc64_of_client_interface: setx client_stack, %g6, %g7 ldx [%g7], %sp
+ /* Restore client trap table */ + setx client_tba, %g6, %g7 + ldx [%g7], %g6 + wrpr %g6, %tba + /* Restore globals */ ldx [%sp + 2047 + 0], %g1 ldx [%sp + 2047 + 8], %g2 diff --git a/openbios-devel/arch/sparc64/openbios.c b/openbios-devel/arch/sparc64/openbios.c index 9ce3e53..9cc1ac2 100644 --- a/openbios-devel/arch/sparc64/openbios.c +++ b/openbios-devel/arch/sparc64/openbios.c @@ -91,14 +91,18 @@ struct cpudef { /* ( addr -- ? ) */ + +extern volatile uint64_t client_tba; + static void set_trap_table(void) { unsigned long addr;
addr = POP(); - asm("wrpr %0, %%tba\n" - : : "r" (addr)); + + /* Update client_tba to be updated on CIF exit */ + client_tba = addr; }
/* Reset control register is defined in 17.2.7.3 of US IIi User Manual */
Members of the *BSD family have a restriction that no window spill/fill traps can occur in the short time between calling SUNW,set-trap-table in OF and returning back to the caller.
In order to ensure we minimise window spill/fill traps, preserve the entire window state across the CIF calls (probably similar to OBP) to ensure that we don't breach this restriction.
Signed-off-by: Mark Cave-Ayland mark.cave-ayland@ilande.co.uk --- openbios-devel/arch/sparc64/call-client.S | 171 +++++++++++++++++++++++++++-- 1 file changed, 163 insertions(+), 8 deletions(-)
diff --git a/openbios-devel/arch/sparc64/call-client.S b/openbios-devel/arch/sparc64/call-client.S index 47d937f..f365e3c 100644 --- a/openbios-devel/arch/sparc64/call-client.S +++ b/openbios-devel/arch/sparc64/call-client.S @@ -1,5 +1,153 @@ .globl sparc64_of_client_interface, client_tba
+ +/* + * SAVE_WINDOW_STATE and RESTORE_WINDOW_STATE are used to ensure + * that the CPU window state is preserved across CIF calls. This is + * to workaround a *BSD restriction that window fill/spill traps must + * be minimised during trap table takeover, and likely emulates the + * behaviour of OBP. + */ + +#define SAVE_WINDOW_STATE(type) \ + setx client_window, %g6, %g1; \ + rdpr %cwp, %g7; \ + stx %g7, [%g1]; \ + rdpr %cansave, %g7; \ + stx %g7, [%g1 + 0x8]; \ + rdpr %canrestore, %g7; \ + stx %g7, [%g1 + 0x10]; \ + rdpr %otherwin, %g7; \ + stx %g7, [%g1 + 0x18]; \ + rdpr %wstate, %g7; \ + stx %g7, [%g1 + 0x20]; \ + rdpr %cleanwin, %g7; \ + stx %g7, [%g1 + 0x28]; \ + \ + stx %o0, [%g1 + 0x30]; \ + stx %o1, [%g1 + 0x38]; \ + stx %o2, [%g1 + 0x40]; \ + stx %o3, [%g1 + 0x48]; \ + stx %o4, [%g1 + 0x50]; \ + stx %o5, [%g1 + 0x58]; \ + stx %o6, [%g1 + 0x60]; \ + stx %o7, [%g1 + 0x68]; \ + \ + rdpr %pstate, %g7; \ + stx %g7, [%g1 + 0x70]; \ + rd %y, %g7; \ + stx %g7, [%g1 + 0x78]; \ + rd %fprs, %g7; \ + stx %g7, [%g1 + 0x80]; \ + \ + /* Now iterate through all of the windows saving all l and i registers */ \ + add %g1, 0x90, %g5; \ + \ + /* Get the number of windows in %g6 */ \ + rdpr %ver, %g6; \ + and %g6, 0xf, %g6; \ + inc %g6; \ + \ +save_cpu_window_##type: \ + deccc %g6; \ + wrpr %g6, %cwp; \ + stx %l0, [%g5]; \ + stx %l1, [%g5 + 0x8]; \ + stx %l2, [%g5 + 0x10]; \ + stx %l3, [%g5 + 0x18]; \ + stx %l4, [%g5 + 0x20]; \ + stx %l5, [%g5 + 0x28]; \ + stx %l6, [%g5 + 0x30]; \ + stx %l7, [%g5 + 0x38]; \ + stx %i0, [%g5 + 0x40]; \ + stx %i1, [%g5 + 0x48]; \ + stx %i2, [%g5 + 0x50]; \ + stx %i3, [%g5 + 0x58]; \ + stx %i4, [%g5 + 0x60]; \ + stx %i5, [%g5 + 0x68]; \ + stx %i6, [%g5 + 0x70]; \ + stx %i7, [%g5 + 0x78]; \ + bne save_cpu_window_##type; \ + add %g5, 0x80, %g5; \ + \ + /* For 8 windows with 16 registers to save in the window, memory required \ + is 16*8*8 = 0x400 bytes */ \ + \ + /* Now we should be in window 0 so update the other window registers */ \ + rdpr %ver, %g6; \ + and %g6, 0xf, %g6; \ + dec %g6; \ + wrpr %g6, %cansave; \ + \ + wrpr %g0, %cleanwin; \ + wrpr %g0, %canrestore; \ + wrpr %g0, %otherwin; + + +#define RESTORE_WINDOW_STATE(type) \ + setx client_window, %g6, %g1; \ + \ + /* Get the number of windows in %g6 */ \ + rdpr %ver, %g6; \ + and %g6, 0xf, %g6; \ + inc %g6; \ + \ + /* Now iterate through all of the windows restoring all l and i registers */ \ + add %g1, 0x90, %g5; \ + \ +restore_cpu_window_##type: \ + deccc %g6; \ + wrpr %g6, %cwp; \ + ldx [%g5], %l0; \ + ldx [%g5 + 0x8], %l1; \ + ldx [%g5 + 0x10], %l2; \ + ldx [%g5 + 0x18], %l3; \ + ldx [%g5 + 0x20], %l4; \ + ldx [%g5 + 0x28], %l5; \ + ldx [%g5 + 0x30], %l6; \ + ldx [%g5 + 0x38], %l7; \ + ldx [%g5 + 0x40], %i0; \ + ldx [%g5 + 0x48], %i1; \ + ldx [%g5 + 0x50], %i2; \ + ldx [%g5 + 0x58], %i3; \ + ldx [%g5 + 0x60], %i4; \ + ldx [%g5 + 0x68], %i5; \ + ldx [%g5 + 0x70], %i6; \ + ldx [%g5 + 0x78], %i7; \ + bne restore_cpu_window_##type; \ + add %g5, 0x80, %g5; \ + \ + /* Restore the window registers to their original value */ \ + ldx [%g1], %g7; \ + wrpr %g7, %cwp; \ + ldx [%g1 + 0x8], %g7; \ + wrpr %g7, %cansave; \ + ldx [%g1 + 0x10], %g7; \ + wrpr %g7, %canrestore; \ + ldx [%g1 + 0x18], %g7; \ + wrpr %g7, %otherwin; \ + ldx [%g1 + 0x20], %g7; \ + wrpr %g7, %wstate; \ + ldx [%g1 + 0x28], %g7; \ + wrpr %g7, %cleanwin; \ + \ + ldx [%g1 + 0x30], %o0; \ + ldx [%g1 + 0x38], %o1; \ + ldx [%g1 + 0x40], %o2; \ + ldx [%g1 + 0x48], %o3; \ + ldx [%g1 + 0x50], %o4; \ + ldx [%g1 + 0x58], %o5; \ + ldx [%g1 + 0x60], %o6; \ + ldx [%g1 + 0x68], %o7; \ + \ + ldx [%g1 + 0x70], %g7; \ + wrpr %g7, %pstate; \ + ldx [%g1 + 0x78], %g7; \ + wr %g7, 0, %y; \ + ldx [%g1 + 0x80], %g7; \ + wr %g7, 0, %fprs + + .data .align 8
@@ -10,6 +158,9 @@ client_stack: .xword 0 client_tba: .xword 0 +client_window: + .skip 2048 +
.text .align 4 @@ -23,8 +174,6 @@ client_tba: */
sparc64_of_client_interface: - /* make sure caller's windows are on caller's stack */ - flushw
/* Save globals on callers stack */ add %sp, -56, %sp @@ -42,20 +191,26 @@ sparc64_of_client_interface: rdpr %tba, %g6 stx %g6, [%g7]
- /* Save existing stack and move to openbios stack */ + /* Save existing stack */ setx client_stack, %g6, %g7 stx %sp, [%g7] + + /* Save windows */ + SAVE_WINDOW_STATE(cif) + + /* Move to OpenBIOS stack */ setx openbios_stack - 2047 - 192, %g6, %g7 mov %g7, %sp
/* Call client inteface */ - save %sp, -192, %sp - call of_client_interface - mov %i0, %o0 + ldx [%g1 + 0x30], %o0 + + setx client_window, %g6, %g1 + stx %o0, [%g1 + 0x30]
- mov %o0, %i0 - restore %sp, 192, %sp + /* Restore windows */ + RESTORE_WINDOW_STATE(cif)
/* Restore stack */ setx client_stack, %g6, %g7
-
Mark Cave-Ayland