On Mon, 9 Jun 2014, BALATON Zoltan wrote:
Therefore for a load/store not to fault on real hardware then that means the entry is already in the TLB, i.e. it must have already been accessed AND also not been previously evicted by hash collision (see hash_page32).
Unfortunately PPC has many other ways to translate an address not only via the TLB so this is not that simple. I suspect the TLB is not used on Macs during boot but BAT translation is because MorphOS only disables the MMU during replacing the BAT registers.
In the tests I wrote about here:
http://lists.nongnu.org/archive/html/qemu-devel/2014-06/msg02828.html
I've also tried to find answers to these questions. The results so far are here:
http://goliat.eik.bme.hu/~balaton/oftest/results/
It seems BAT registers are not used only a TLB which is near the end of the memory like in OpenBIOS.
Hence you need to trace through previous accesses to the addresses that fault and determine why the entry is no longer in the TLB in QEMU/OpenBIOS when it should be there on real hardware.
The vectors are only accessed by OpenBIOS during the inital copying of the vectors before setting up the MMU then by MorphOS when copying it's vectors that cause the exception which crashes. No other access inbetween so it's not evicted from TLB as it never gets into it.
Actually after writing the sr0-sr15 and sdr1 registers it clears the MMU bits and then writes to the ibat and dbat registers and seems to set up the TLB and then enable the bits in the MSR. So maybe Apple relies on translations in these registers and that's why it works there.
Not that I have the definitive answer, but I can't see why on earth Apple would map the trap table with an IBAT/DBAT. My understanding is that these are designed for optimally mapping large blocks of memory, e.g. kernels, so why would you implement a second form of translation in a BIOS environment when you already have limited resources? It just doesn't make any sense. Plus you still get a fault once you effectively "preload" the trap table into the TLB with your fake access above, so something else is still missing from the picture.
The fake access would still be needed I think because otherwise a translation for the page starting at 0x1000 would not be in the TLB. The fault I got with the fake access was not when writing to 0x1000 like without it but accessing the stack that the fake access did not preload.
With 128MB it will overwrite the stack as I've found earlier here: http://www.openfirmware.info/pipermail/openbios/2014-March/008186.html
With 512MB it will freeze after clearing the TLB and replacing BAT registers with it's own values like this:
Set IBAT0l to 00000000 (0041ce54) Set IBAT0u to 00000000 (0041ce54) Flush BAT from 00000000 to 00020000 (00000000) Flush done Flush BAT from 00000000 to 00020000 (00000000) Flush done Set IBAT1l to f0000012 (0041ce54) Set IBAT1u to f0001ffe (0041ce54) Flush BAT from 00000000 to 10000000 (0ffe0000) Flush done Flush BAT from f0000000 to 00000000 (0ffe0000) Flush done Set IBAT2l to 00000000 (0041ce54) Set IBAT2u to 00000000 (0041ce54) Set IBAT3l to 00000012 (0041ce54) Set IBAT3u to 00000ffe (0041ce54) Flush BAT from 00000000 to 08000000 (07fe0000) Flush done Flush BAT from 00000000 to 08000000 (07fe0000) Flush done Set DBAT0l to f000002a (0041ce54) Set DBAT0u to f0001ffe (0041ce54) Flush BAT from 00000000 to 10000000 (0ffe0000) Flush done Flush BAT from f0000000 to 00000000 (0ffe0000) Flush done Set DBAT1l to 00000012 (0041ce54) Set DBAT1u to 00001ffe (0041ce54) Flush BAT from 1fdc0000 to 2fdc0000 (0ffe0000) Flush done Flush BAT from 00000000 to 10000000 (0ffe0000) Flush done Set DBAT2l to 00000000 (0041ce54) Set DBAT2u to 00000000 (0041ce54) Set DBAT3l to 8000002a (0041ce54) Set DBAT3u to 80001ffe (0041ce54)
and then trying to access a variable on the stack:
0x0041cf40: lmw r13,20(r1) 0x0041cf44: addi r1,r1,96 0x0041cf48: blr
htab_base 0000000000000000 htab_mask 000000000000ffff hash 000000000000fde7 0 htab=0000000000000000/000000000000ffff vsid=0 ptem=3f hash=000000000000fde7 1 htab=0000000000000000/000000000000ffff vsid=0 api=3f hash=ffffffffffff0218 Raise exception at 0041cf40 => 00000002 (00)
r0 0x3030 12336 r1 0x1fde7e70 534675056
which is not covered by any translation now. So I think the stack must be in the first 256MB to which a translation is added to the BAT and this is why it works with 256MB of memory. Thus to be complete the stack should be moved from the end of the memory where it is now (described in a comment in arch/ppc/qemu/start.S) to somwhere else. But where? There's another comment in arch/ppc/qemu/ofmem.c that seems to originate from here:
http://opensource.apple.com/source/BootX/BootX-81/bootx.tproj/include.subpro...
but seems it has changed at the origin since it was copied. Does anyone know a good location that is left free by the boot loaders we care about where the stack could be placed within the first 256MB? MoprhOS seems to start using memory from the end so maybe the stack should not be there. Any more ideas?
From the test results it seems that Apple puts the stack after the image
(and probably clears it or otherwise adds a mapping in the TLB for it so accessing it will not generate exceptions). How could this be implemented in OpenBIOS? I've tried to look at the code to find a good place but there seems to be different cases for different architectures and using preloaded kernel image seems to be an additional complication. Could the file-size stored in saved-program-state be used as a base to place the stack in call_elf? (this would need adding an additional parameter to it in the ppc/qemu case because accessing Forth from assembly is not something I'd try).
Or are my conclusions wrong or you have a better idea? (Also what about my other patches on the list? This place seemed abandoned for the last week.)
Regards, BALATON Zoltan