[OpenBIOS] Running client with MMU off

[BALATON Zoltan]

On Mon, 9 Jun 2014, BALATON Zoltan wrote:
>> Therefore for a load/store not to fault on real hardware then that means 
>> the entry is already in the TLB, i.e. it must have already been accessed 
>> AND also not been previously evicted by hash collision (see hash_page32).
>
> Unfortunately PPC has many other ways to translate an address not only via 
> the TLB so this is not that simple. I suspect the TLB is not used on Macs 
> during boot but BAT translation is because MorphOS only disables the MMU 
> during replacing the BAT registers.

In the tests I wrote about here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-06/msg02828.html

I've also tried to find answers to these questions. The results so far are 
here:

http://goliat.eik.bme.hu/~balaton/oftest/results/

It seems BAT registers are not used only a TLB which is near the end of 
the memory like in OpenBIOS.

>> Hence you need to trace through previous accesses to the addresses that 
>> fault and determine why the entry is no longer in the TLB in QEMU/OpenBIOS 
>> when it should be there on real hardware.
>
> The vectors are only accessed by OpenBIOS during the inital copying of the 
> vectors before setting up the MMU then by MorphOS when copying it's vectors 
> that cause the exception which crashes. No other access inbetween so it's not 
> evicted from TLB as it never gets into it.
>
>>> Actually after writing the sr0-sr15 and sdr1 registers it clears the MMU
>>> bits and then writes to the ibat and dbat registers and seems to set up
>>> the TLB and then enable the bits in the MSR. So maybe Apple relies on
>>> translations in these registers and that's why it works there.
>> 
>> Not that I have the definitive answer, but I can't see why on earth Apple 
>> would map the trap table with an IBAT/DBAT. My understanding is that these 
>> are designed for optimally mapping large blocks of memory, e.g. kernels, so 
>> why would you implement a second form of translation in a BIOS environment 
>> when you already have limited resources? It just doesn't make any sense. 
>> Plus you still get a fault once you effectively "preload" the trap table 
>> into the TLB with your fake access above, so something else is still 
>> missing from the picture.

The fake access would still be needed I think because otherwise a 
translation for the page starting at 0x1000 would not be in the TLB. The 
fault I got with the fake access was not when writing to 0x1000 like 
without it but accessing the stack that the fake access did not preload.

> With 128MB it will overwrite the stack as I've found earlier here:
> http://www.openfirmware.info/pipermail/openbios/2014-March/008186.html
>
> With 512MB it will freeze after clearing the TLB and replacing BAT registers 
> with it's own values like this:
>
> Set IBAT0l to 00000000 (0041ce54)
> Set IBAT0u to 00000000 (0041ce54)
> Flush BAT from 00000000 to 00020000 (00000000)
> Flush done
> Flush BAT from 00000000 to 00020000 (00000000)
> Flush done
> Set IBAT1l to f0000012 (0041ce54)
> Set IBAT1u to f0001ffe (0041ce54)
> Flush BAT from 00000000 to 10000000 (0ffe0000)
> Flush done
> Flush BAT from f0000000 to 00000000 (0ffe0000)
> Flush done
> Set IBAT2l to 00000000 (0041ce54)
> Set IBAT2u to 00000000 (0041ce54)
> Set IBAT3l to 00000012 (0041ce54)
> Set IBAT3u to 00000ffe (0041ce54)
> Flush BAT from 00000000 to 08000000 (07fe0000)
> Flush done
> Flush BAT from 00000000 to 08000000 (07fe0000)
> Flush done
> Set DBAT0l to f000002a (0041ce54)
> Set DBAT0u to f0001ffe (0041ce54)
> Flush BAT from 00000000 to 10000000 (0ffe0000)
> Flush done
> Flush BAT from f0000000 to 00000000 (0ffe0000)
> Flush done
> Set DBAT1l to 00000012 (0041ce54)
> Set DBAT1u to 00001ffe (0041ce54)
> Flush BAT from 1fdc0000 to 2fdc0000 (0ffe0000)
> Flush done
> Flush BAT from 00000000 to 10000000 (0ffe0000)
> Flush done
> Set DBAT2l to 00000000 (0041ce54)
> Set DBAT2u to 00000000 (0041ce54)
> Set DBAT3l to 8000002a (0041ce54)
> Set DBAT3u to 80001ffe (0041ce54)
>
> and then trying to access a variable on the stack:
>
> 0x0041cf40:  lmw     r13,20(r1)
> 0x0041cf44:  addi    r1,r1,96
> 0x0041cf48:  blr
>
> htab_base 0000000000000000 htab_mask 000000000000ffff hash 000000000000fde7
> 0 htab=0000000000000000/000000000000ffff vsid=0 ptem=3f hash=000000000000fde7
> 1 htab=0000000000000000/000000000000ffff vsid=0 api=3f hash=ffffffffffff0218
> Raise exception at 0041cf40 => 00000002 (00)
>
> r0             0x3030   12336
> r1             0x1fde7e70       534675056
>
> which is not covered by any translation now. So I think the stack must be in 
> the first 256MB to which a translation is added to the BAT and this is why it 
> works with 256MB of memory. Thus to be complete the stack should be moved 
> from the end of the memory where it is now (described in a comment in 
> arch/ppc/qemu/start.S) to somwhere else. But where? There's another comment 
> in arch/ppc/qemu/ofmem.c that seems to originate from here:
>
> http://opensource.apple.com/source/BootX/BootX-81/bootx.tproj/include.subproj/sl.h
>
> but seems it has changed at the origin since it was copied. Does anyone know 
> a good location that is left free by the boot loaders we care about where the 
> stack could be placed within the first 256MB? MoprhOS seems to start using 
> memory from the end so maybe the stack should not be there.
> Any more ideas?

>From the test results it seems that Apple puts the stack after the image 
(and probably clears it or otherwise adds a mapping in the TLB for it so 
accessing it will not generate exceptions). How could this be implemented 
in OpenBIOS? I've tried to look at the code to find a good place but there 
seems to be different cases for different architectures and using 
preloaded kernel image seems to be an additional complication. Could the 
file-size stored in saved-program-state be used as a base to place the 
stack in call_elf? (this would need adding an additional parameter to it 
in the ppc/qemu case because accessing Forth from assembly is not 
something I'd try).

Or are my conclusions wrong or you have a better idea? (Also what about my 
other patches on the list? This place seemed abandoned for the last week.)

Regards,
BALATON Zoltan