On 26/02/11 14:53, Blue Swirl wrote:
So it looks like Solaris has already taken over the IOMMU page table by setting a new base address, and appears to be flushing 2 entries for 0xfc000000 and 0xfc001000 which looks like it should be doing the right thing. If Solaris has taken over the IOMMU page table, then how could OpenBIOS affect this?
If the DMA is launched by accident before the page tables are set up, that could cause the crash.
Please enable DMA debugging as well, that would tell us what is the programming sequence.
Okay - I didn't find any DMA debugging, but I've re-enabled the debugging in hw/esp.c to give a feel for the order of things:
vac: enabled in write through mode mem = 131072K (0x8000000) avail mem = 110419968 ### Writing to iommu addr: 1 ### Setting IOMMU base addr: 6bc000 ### Writing to iommu addr: 0 ### Writing to iommu addr: 5 ### IOMMU TLB flush 0 root nexus = SUNW,SPARCstation-5 iommu0 at root: obio 0x10000000 sbus0 at iommu0: obio 0x10001000 dma0 at sbus0: SBus slot 5 0x8400000 dma0 is /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000 ### Writing to iommu addr: 6 ### IOMMU page flush fc000000 ESP: write reg[11]: 0x00 -> 0x00 ESP: write reg[11]: 0x00 -> 0x0a ESP: read reg[11]: 0x0a ESP: write reg[12]: 0x00 -> 0x00 ESP: write reg[12]: 0x00 -> 0x05 ESP: read reg[12]: 0x05 ESP: write reg[11]: 0x0a -> 0x08 ESP: write reg[12]: 0x05 -> 0x00 ESP: write reg[3]: 0x10 -> 0x03 ESP: Bus reset (03) ESP: Raise IRQ ESP: Lower enable ESP: write reg[3]: 0x00 -> 0x02 ESP: Chip reset (02) ESP: write reg[3]: 0x02 -> 0x80 ESP: NOP (80) ESP: write reg[3]: 0x80 -> 0x80 ESP: NOP (80) ESP: write reg[9]: 0x00 -> 0x00 ESP: write reg[5]: 0x00 -> 0xa3 ESP: write reg[6]: 0x00 -> 0x00 ESP: write reg[7]: 0x00 -> 0x00 ESP: read reg[14]: 0x04 ESP: read reg[14]: 0x04 ESP: write reg[8]: 0x00 -> 0x17 ESP: write reg[12]: 0x00 -> 0x01 ESP: write reg[11]: 0x00 -> 0x08 /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,8800000 (esp0): esp-options=0x46 ESP: read reg[5]: 0x00 esp0 at dma0: SBus slot 5 0x8800000 sparc ipl 4 esp0 is /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,8800000 ### Writing to iommu addr: 6 ### IOMMU page flush fc001000 ESP: write reg[4]: 0x00 -> 0x00 ESP: write reg[6]: 0x00 -> 0x00 ESP: write reg[7]: 0x00 -> 0x00 ESP: write reg[12]: 0x01 -> 0x01 ESP: write reg[8]: 0x17 -> 0x07 ESP: write reg[0]: 0x00 -> 0x07 ESP: write reg[1]: 0x00 -> 0x00 ESP: Raise enable ESP: write reg[3]: 0x80 -> 0xc2 ESP: Select with ATN (c2) ESP: get_cmd: len 7 target 0 buf 0x7fff87f4ba90 ESP: ### No such drive! ESP: ### No such drive pause over! ESP: Raise IRQ qemu: fatal: Trap 0x29 while interrupts disabled, Error state
A flush should be the removal of an entry, so I would guess that Solaris flushes the relevant entries and then adds them back into its IOMMU page table manually?
Also, what do the IOMMU_RNGE_*MB constants do? Do they control the size of the IOMMU page table? Sorry to keep asking these questions but it seems that Oracle have removed all of the documents referenced in the various IOMMU source files :(
Yes, for example with 64MB range, virtual DMA addresses between 0xfc000000 to 0xffffffff are available.
One possibility for crash is that OpenBIOS uses 64MB range, but Solaris "knows" that it is something else as used by OBP.
AFAICT OpenBIOS sets a 32MB range, but some debugging shows that Solaris sets it to 64MB just before the scan - so I think this range should be valid, there is just no mapping in the page table for it.
ATB,
Mark.