[OpenBIOS] Running client with MMU off

[Mark Cave-Ayland]

On 17/06/14 01:01, BALATON Zoltan wrote:

> On Mon, 9 Jun 2014, BALATON Zoltan wrote:
>>> Therefore for a load/store not to fault on real hardware then that
>>> means the entry is already in the TLB, i.e. it must have already been
>>> accessed AND also not been previously evicted by hash collision (see
>>> hash_page32).
>>
>> Unfortunately PPC has many other ways to translate an address not only
>> via the TLB so this is not that simple. I suspect the TLB is not used
>> on Macs during boot but BAT translation is because MorphOS only
>> disables the MMU during replacing the BAT registers.
>
> In the tests I wrote about here:
>
> http://lists.nongnu.org/archive/html/qemu-devel/2014-06/msg02828.html
>
> I've also tried to find answers to these questions. The results so far
> are here:
>
> http://goliat.eik.bme.hu/~balaton/oftest/results/
>
> It seems BAT registers are not used only a TLB which is near the end of
> the memory like in OpenBIOS.

I had a feeling that would be the case, however I'm glad at least we 
have some evidence to back this up. BTW you mean hash table, not TLB 
here, right?

>>> Hence you need to trace through previous accesses to the addresses
>>> that fault and determine why the entry is no longer in the TLB in
>>> QEMU/OpenBIOS when it should be there on real hardware.
>>
>> The vectors are only accessed by OpenBIOS during the inital copying of
>> the vectors before setting up the MMU then by MorphOS when copying
>> it's vectors that cause the exception which crashes. No other access
>> inbetween so it's not evicted from TLB as it never gets into it.
>>
>>>> Actually after writing the sr0-sr15 and sdr1 registers it clears the
>>>> MMU
>>>> bits and then writes to the ibat and dbat registers and seems to set up
>>>> the TLB and then enable the bits in the MSR. So maybe Apple relies on
>>>> translations in these registers and that's why it works there.
>>>
>>> Not that I have the definitive answer, but I can't see why on earth
>>> Apple would map the trap table with an IBAT/DBAT. My understanding is
>>> that these are designed for optimally mapping large blocks of memory,
>>> e.g. kernels, so why would you implement a second form of translation
>>> in a BIOS environment when you already have limited resources? It
>>> just doesn't make any sense. Plus you still get a fault once you
>>> effectively "preload" the trap table into the TLB with your fake
>>> access above, so something else is still missing from the picture.
>
> The fake access would still be needed I think because otherwise a
> translation for the page starting at 0x1000 would not be in the TLB. The
> fault I got with the fake access was not when writing to 0x1000 like
> without it but accessing the stack that the fake access did not preload.
>
>> With 128MB it will overwrite the stack as I've found earlier here:
>> http://www.openfirmware.info/pipermail/openbios/2014-March/008186.html
>>
>> With 512MB it will freeze after clearing the TLB and replacing BAT
>> registers with it's own values like this:
>>
>> Set IBAT0l to 00000000 (0041ce54)
>> Set IBAT0u to 00000000 (0041ce54)
>> Flush BAT from 00000000 to 00020000 (00000000)
>> Flush done
>> Flush BAT from 00000000 to 00020000 (00000000)
>> Flush done
>> Set IBAT1l to f0000012 (0041ce54)
>> Set IBAT1u to f0001ffe (0041ce54)
>> Flush BAT from 00000000 to 10000000 (0ffe0000)
>> Flush done
>> Flush BAT from f0000000 to 00000000 (0ffe0000)
>> Flush done
>> Set IBAT2l to 00000000 (0041ce54)
>> Set IBAT2u to 00000000 (0041ce54)
>> Set IBAT3l to 00000012 (0041ce54)
>> Set IBAT3u to 00000ffe (0041ce54)
>> Flush BAT from 00000000 to 08000000 (07fe0000)
>> Flush done
>> Flush BAT from 00000000 to 08000000 (07fe0000)
>> Flush done
>> Set DBAT0l to f000002a (0041ce54)
>> Set DBAT0u to f0001ffe (0041ce54)
>> Flush BAT from 00000000 to 10000000 (0ffe0000)
>> Flush done
>> Flush BAT from f0000000 to 00000000 (0ffe0000)
>> Flush done
>> Set DBAT1l to 00000012 (0041ce54)
>> Set DBAT1u to 00001ffe (0041ce54)
>> Flush BAT from 1fdc0000 to 2fdc0000 (0ffe0000)
>> Flush done
>> Flush BAT from 00000000 to 10000000 (0ffe0000)
>> Flush done
>> Set DBAT2l to 00000000 (0041ce54)
>> Set DBAT2u to 00000000 (0041ce54)
>> Set DBAT3l to 8000002a (0041ce54)
>> Set DBAT3u to 80001ffe (0041ce54)
>>
>> and then trying to access a variable on the stack:
>>
>> 0x0041cf40:  lmw     r13,20(r1)
>> 0x0041cf44:  addi    r1,r1,96
>> 0x0041cf48:  blr
>>
>> htab_base 0000000000000000 htab_mask 000000000000ffff hash
>> 000000000000fde7
>> 0 htab=0000000000000000/000000000000ffff vsid=0 ptem=3f
>> hash=000000000000fde7
>> 1 htab=0000000000000000/000000000000ffff vsid=0 api=3f
>> hash=ffffffffffff0218
>> Raise exception at 0041cf40 => 00000002 (00)
>>
>> r0             0x3030   12336
>> r1             0x1fde7e70       534675056
>>
>> which is not covered by any translation now. So I think the stack must
>> be in the first 256MB to which a translation is added to the BAT and
>> this is why it works with 256MB of memory. Thus to be complete the
>> stack should be moved from the end of the memory where it is now
>> (described in a comment in arch/ppc/qemu/start.S) to somwhere else.
>> But where? There's another comment in arch/ppc/qemu/ofmem.c that seems
>> to originate from here:
>>
>> http://opensource.apple.com/source/BootX/BootX-81/bootx.tproj/include.subproj/sl.h
>>
>>
>> but seems it has changed at the origin since it was copied. Does
>> anyone know a good location that is left free by the boot loaders we
>> care about where the stack could be placed within the first 256MB?
>> MoprhOS seems to start using memory from the end so maybe the stack
>> should not be there.
>> Any more ideas?
>
>> From the test results it seems that Apple puts the stack after the image
> (and probably clears it or otherwise adds a mapping in the TLB for it so
> accessing it will not generate exceptions). How could this be
> implemented in OpenBIOS? I've tried to look at the code to find a good
> place but there seems to be different cases for different architectures
> and using preloaded kernel image seems to be an additional complication.
> Could the file-size stored in saved-program-state be used as a base to
> place the stack in call_elf? (this would need adding an additional
> parameter to it in the ppc/qemu case because accessing Forth from
> assembly is not something I'd try).

FWIW I don't know much about the PPC side of this, however for SPARC the 
client stack is set as part of the context initialisation (see 
arch/sparc64/context.c for an example of this). This allows the caller 
to initialise a context (including specifying the stack) all in C before 
dropping into asm to finally execute the loaded file.

> Or are my conclusions wrong or you have a better idea? (Also what about
> my other patches on the list? This place seemed abandoned for the last
> week.)

Well, as I mentioned last week I've been away on holiday. Plus it's QEMU 
soft freeze time which means everybody is busy trying to get their 
patches applied in preparation for 2.1. This generally means people are 
very busy.


ATB,

Mark.