[OpenBIOS] Solaris 8: ESP device enumeration issue

[Blue Swirl]

On Wed, Feb 23, 2011 at 7:12 PM, Mark Cave-Ayland
<mark.cave-ayland at siriusit.co.uk> wrote:
> On 20/02/11 20:28, Mark Cave-Ayland wrote:
>
>> Searching on the internet seems to suggest that IRQ 15 is used for
>> distributing interrupts on multi-processor systems - so perhaps it is
>> related to OpenBIOS' SMP setup? With a conditional breakpoint, I can see
>> that they only start appearing upon activation of the first DMA transfer
>> initiated by the Solaris kernel.
>
> Ah so I think I have found the answer here:
> http://www.mail-archive.com/qemu-devel@nongnu.org/msg39793.html. So
> basically it seems this IRQ is being raised by the IOMMU because an attempt
> is made to access an unmapped I/O address.
>
> Using a debug build of qemu, I placed a breakpoint on iommu_bad_addr in
> order to get a backtrace, and sure enough just after the Select with ATN I
> get a break here:
>
> Breakpoint 1, iommu_bad_addr (s=0x1726020, addr=4227858432, is_write=0) at
> /home/build/src/qemu/git/qemu/hw/sun4m_iommu.c:279
> 279     {
> (gdb) p/x 4227858432
> $1 = 0xfc000000
> (gdb) bt
> #0  iommu_bad_addr (s=0x1726020, addr=4227858432, is_write=0) at
> /home/build/src/qemu/git/qemu/hw/sun4m_iommu.c:279
> #1  0x0000000000549919 in sparc_iommu_memory_rw (opaque=0x1726020,
> addr=4227858464, buf=0x7fffac0c4560
> "\374\377\377\377\377\377\377\377\032\324T", len=7, is_write=0)
>    at /home/build/src/qemu/git/qemu/hw/sun4m_iommu.c:303
> #2  0x000000000054ccb7 in sparc_iommu_memory_read (opaque=0x1726020,
> addr=4227858464, buf=0x7fffac0c4560
> "\374\377\377\377\377\377\377\377\032\324T", len=7)
>    at /home/build/src/qemu/git/qemu/hw/sun4m.h:15
> #3  0x000000000054d2a1 in espdma_memory_read (opaque=0x182ce70,
> buf=0x7fffac0c4560 "\374\377\377\377\377\377\377\377\032\324T", len=7) at
> /home/build/src/qemu/git/qemu/hw/sparc32_dma.c:154
> #4  0x000000000053f57b in get_cmd (s=0x1a66020, buf=0x7fffac0c4560
> "\374\377\377\377\377\377\377\377\032\324T") at
> /home/build/src/qemu/git/qemu/hw/esp.c:199
> #5  0x000000000053f92e in handle_satn (s=0x1a66020) at
> /home/build/src/qemu/git/qemu/hw/esp.c:276
> #6  0x00000000005409ec in esp_mem_writeb (opaque=0x1a66020, addr=12,
> val=194) at /home/build/src/qemu/git/qemu/hw/esp.c:631
> #7  0x00000000004f151a in subpage_writelen (mmio=0x1a6c190, addr=12,
> value=194, len=0) at /home/build/src/qemu/git/qemu/exec.c:3260
> #8  0x00000000004f15c0 in subpage_writeb (opaque=0x1a6c190, addr=12,
> value=194) at /home/build/src/qemu/git/qemu/exec.c:3271
> #9  0x0000000000522797 in io_writeb (physaddr=12, val=194 '\302',
> addr=4119879692, retaddr=0x4134d166) at ../softmmu_template.h:213
> #10 0x000000000052287c in __stb_mmu (addr=4119879692, val=194 '\302',
> mmu_idx=1) at ../softmmu_template.h:245
> #11 0x000000004134d167 in ?? ()
> #12 0x0000000000000001 in ?? ()
> #13 0x1e8dd2975dfbc400 in ?? ()
> #14 0x0000000000000000 in ?? ()
>
> So it looks like the IOMMU doesn't have an entry mapped at 0xfc000000 which
> is causing it to raise IRQ15. Now I see that OpenBIOS doesn't map that range
> in drivers/iommu.c - is that something we should be doing?

IOMMU virtual address space is completely separate from MMU virtual
address space. 0xfc000000 does not need to be mapped by MMU. This area
is determined by the mapping range bits in IOMMU control register.

> The other interesting part is why address 0xfc000000, since OpenBIOS must
> surely have pre-mapped ESP somewhere above 0xff000000 to be able to
> bootstrap the kernel from disk using DMA?

The address 0xfc000000 comes from ESP DMA controller's address
register, ESP can only supply lowest 24 bits of the address.

A very similar problem prevented NetBSD 1.6.x boot, because NetBSD
programmed the DMA controller and ESP in different order that what
QEMU expected. This is now fixed, so it shouldn't be a problem anymore
and since OBP works, the problem at hand is not in QEMU side.

Maybe Solaris also programs DMA and ESP in some different way, which
causes DMA to fire before the address is set up. To fix it, maybe
OpenBIOS should reset the DMA controller after using it or at least
disable DMA.