On 27/02/11 11:14, Mark Cave-Ayland wrote:
On 26/02/11 17:28, Blue Swirl wrote:
Blue: what debugging do I need to add to prove that this is the DMA bug you mentioned before?
Tracepoint logs with all points from both sparc32_dma.c and sun4m_iommu.c enabled?
Okay - as an alternative, here is the output of my hacked-up qemu with ESP debugging and OpenBIOS romvec debugging enabled:
vac: enabled in write through mode mem = 131072K (0x8000000) avail mem = 110419968 obp_nextnode(0x0) = 0xffd4527c obp_proplen(0xffd4527c, reg) (not found) obp_proplen(0xffd4527c, ranges) (not found) obp_proplen(0xffd4527c, intr) (not found) obp_proplen(0xffd4527c, interrupts) (not found) ### Writing to iommu addr: 1, value 6bc000 0 --- Current translation for 0xfc000000 is 7fba06 ### Setting IOMMU base addr: 6bc000 ### Writing to iommu addr: 0, value 9 0 --- Current translation for 0xfc000000 is 0 ### Writing to iommu addr: 5, value 0 0 --- Current translation for 0xfc000000 is 0 ### IOMMU TLB flush 0 obp_proplen(0xffd4527c, ttymodes) (not found) obp_proplen(0xffd4527c, device_type) (not found) root nexus = SUNW,SPARCstation-5 obp_proplen(0xffd4527c, pm-hardware-state) (not found) obp_proplen(0xffd4527c, pm-want-child-notification?) (not found) obp_proplen(0xffd4527c, pm-components) (not found) obp_proplen(0xffd455e4, reg) (not found) obp_proplen(0xffd455e4, ranges) (not found) obp_proplen(0xffd455e4, intr) (not found) obp_proplen(0xffd455e4, interrupts) (not found) obp_proplen(0xffd455e4, reg) (not found) obp_proplen(0xffd455e4, ranges) (not found) obp_proplen(0xffd455e4, intr) (not found) obp_proplen(0xffd455e4, interrupts) (not found) obp_proplen(0xffd455e4, device_type) (not found) obp_proplen(0xffd455e4, pm-hardware-state) (not found) obp_proplen(0xffd455e4, pm-want-child-notification?) (not found) obp_proplen(0xffd455e4, pm-components) (not found) obp_fortheval_v2(0 0 f024360c f59a88c8 11 ['] find-device catch if 2drop true else current-device device-end then swap l!) obp_proplen(0xffd4caa4, reg) = 12 obp_proplen(0xffd4caa4, reg) = 12 obp_getprop(0xffd4caa4, reg) = 00 00 00 00 10 00 00 00 00 00 03 00 obp_proplen(0xffd4caa4, ranges) (not found) obp_proplen(0xffd4caa4, intr) (not found) obp_proplen(0xffd4caa4, interrupts) (not found) obp_proplen(0xffd4caa4, device_type) (not found) iommu0 at root: obio 0x10000000 obp_proplen(0xffd4caa4, pm-want-child-notification?) (not found) obp_proplen(0xffd4caa4, pm-components) (not found) obp_fortheval_v2(0 0 f024360c f59a88c8 21 ['] find-device catch if 2drop true else current-device device-end then swap l!) obp_proplen(0xffd4cc14, reg) = 12 obp_proplen(0xffd4cc14, reg) = 12 obp_getprop(0xffd4cc14, reg) = 00 00 00 00 10 00 10 00 00 00 00 28 obp_proplen(0xffd4cc14, ranges) = 120 obp_proplen(0xffd4cc14, ranges) = 120 obp_getprop(0xffd4cc14, ranges) = 00 00 00 00 00 00 00 00 00 00 00 00 20 00 00 00 10 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 30 00 00 00 10 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 40 00 00 00 10 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 50 00 00 00 10 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 60 00 00 00 10 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00 70 00 00 00 10 00 00 00 obp_proplen(0xffd4cc14, intr) (not found) obp_proplen(0xffd4cc14, interrupts) (not found) obp_proplen(0xffd4cc14, device_type) = 13 obp_proplen(0xffd4cc14, device_type) = 13 obp_getprop(0xffd4cc14, device_type) = hierarchical sbus0 at iommu0: obio 0x10001000 obp_proplen(0xffd4cc14, burst-sizes) = 4 obp_proplen(0xffd4cc14, burst-sizes) = 4 obp_getprop(0xffd4cc14, burst-sizes) = 0000003f obp_proplen(0xffd4cc14, pm-hardware-state) (not found) obp_proplen(0xffd4cc14, pm-want-child-notification?) (not found) obp_proplen(0xffd4cc14, pm-components) (not found) obp_fortheval_v2(0 0 f024360c f59a88c8 32 ['] find-device catch if 2drop true else current-device device-end then swap l!) obp_proplen(0xffd4cfec, reg) = 12 obp_proplen(0xffd4cfec, reg) = 12 obp_getprop(0xffd4cfec, reg) = 00 00 00 05 08 40 00 00 00 00 00 10 obp_proplen(0xffd4cfec, ranges) (not found) obp_proplen(0xffd4cfec, intr) (not found) obp_proplen(0xffd4cfec, interrupts) (not found) obp_proplen(0xffd4cfec, device_type) (not found) dma0 at sbus0: SBus slot 5 0x8400000 dma0 is /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000 obp_proplen(0xffd4cfec, pm-hardware-state) (not found) obp_proplen(0xffd4cfec, pm-want-child-notification?) (not found) obp_proplen(0xffd4cfec, pm-components) (not found) obp_fortheval_v2(0 0 f024360c f59a88c8 40 ['] find-device catch if 2drop true else current-device device-end then swap l!) obp_proplen(0xffd4fac0, reg) = 12 obp_proplen(0xffd4fac0, reg) = 12 obp_getprop(0xffd4fac0, reg) = 00 00 00 05 08 80 00 00 00 00 00 10 obp_proplen(0xffd4fac0, ranges) (not found) obp_proplen(0xffd4fac0, intr) = 8 obp_proplen(0xffd4fac0, intr) = 8 obp_getprop(0xffd4fac0, intr) = 00 00 00 24 00 00 00 00 obp_proplen(0xffd4fac0, interrupts) (not found) obp_proplen(0xffd4fac0, device_type) = 5 obp_proplen(0xffd4fac0, device_type) = 5 obp_getprop(0xffd4fac0, device_type) = scsi obp_proplen(0xffd4cc14, slave-only) (not found) obp_proplen(0xffd4caa4, slave-only) (not found) obp_proplen(0xffd4527c, slave-only) (not found) obp_proplen(0xffd455e4, slave-only) (not found) ### Writing to iommu addr: 6, value fc000000 0 --- Current translation for 0xfc000000 is 0 ### IOMMU page flush fc000000 obp_proplen(0xffd4fac0, initiator-id) (not found) obp_proplen(0xffd4cfec, initiator-id) (not found) obp_proplen(0xffd4cc14, initiator-id) (not found) obp_proplen(0xffd4caa4, initiator-id) (not found) obp_proplen(0xffd4527c, initiator-id) (not found) obp_proplen(0xffd455e4, initiator-id) (not found) obp_proplen(0xffd4fac0, scsi-initiator-id) (not found) obp_proplen(0xffd4cfec, scsi-initiator-id) (not found) obp_proplen(0xffd4cc14, scsi-initiator-id) (not found) obp_proplen(0xffd4caa4, scsi-initiator-id) (not found) obp_proplen(0xffd4527c, scsi-initiator-id) (not found) obp_proplen(0xffd455e4, scsi-initiator-id) (not found) obp_proplen(0xffd4fac0, scsi-reset-delay) (not found) obp_proplen(0xffd4cfec, scsi-reset-delay) (not found) obp_proplen(0xffd4cc14, scsi-reset-delay) (not found) obp_proplen(0xffd4caa4, scsi-reset-delay) (not found) obp_proplen(0xffd4527c, scsi-reset-delay) (not found) obp_proplen(0xffd455e4, scsi-reset-delay) (not found) obp_proplen(0xffd4fac0, scsi-tag-age-limit) (not found) obp_proplen(0xffd4cfec, scsi-tag-age-limit) (not found) obp_proplen(0xffd4cc14, scsi-tag-age-limit) (not found) obp_proplen(0xffd4caa4, scsi-tag-age-limit) (not found) obp_proplen(0xffd4527c, scsi-tag-age-limit) (not found) obp_proplen(0xffd455e4, scsi-tag-age-limit) (not found) obp_proplen(0xffd4fac0, scsi-watchdog-tick) (not found) obp_proplen(0xffd4cfec, scsi-watchdog-tick) (not found) obp_proplen(0xffd4cc14, scsi-watchdog-tick) (not found) obp_proplen(0xffd4caa4, scsi-watchdog-tick) (not found) obp_proplen(0xffd4527c, scsi-watchdog-tick) (not found) obp_proplen(0xffd455e4, scsi-watchdog-tick) (not found) obp_proplen(0xffd4fac0, scsi-options) (not found) obp_proplen(0xffd4cfec, scsi-options) (not found) obp_proplen(0xffd4cc14, scsi-options) (not found) obp_proplen(0xffd4caa4, scsi-options) (not found) obp_proplen(0xffd4527c, scsi-options) (not found) obp_proplen(0xffd455e4, scsi-options) (not found) obp_proplen(0xffd4fac0, scsi-selection-timeout) (not found) obp_proplen(0xffd4cfec, scsi-selection-timeout) (not found) obp_proplen(0xffd4cc14, scsi-selection-timeout) (not found) obp_proplen(0xffd4caa4, scsi-selection-timeout) (not found) obp_proplen(0xffd4527c, scsi-selection-timeout) (not found) obp_proplen(0xffd455e4, scsi-selection-timeout) (not found) obp_proplen(0xffd4fac0, target0-scsi-options) (not found) obp_proplen(0xffd4cfec, target0-scsi-options) (not found) obp_proplen(0xffd4cc14, target0-scsi-options) (not found) obp_proplen(0xffd4caa4, target0-scsi-options) (not found) obp_proplen(0xffd4527c, target0-scsi-options) (not found) obp_proplen(0xffd455e4, target0-scsi-options) (not found) obp_proplen(0xffd4fac0, target1-scsi-options) (not found) obp_proplen(0xffd4cfec, target1-scsi-options) (not found) obp_proplen(0xffd4cc14, target1-scsi-options) (not found) obp_proplen(0xffd4caa4, target1-scsi-options) (not found) obp_proplen(0xffd4527c, target1-scsi-options) (not found) obp_proplen(0xffd455e4, target1-scsi-options) (not found) obp_proplen(0xffd4fac0, target2-scsi-options) (not found) obp_proplen(0xffd4cfec, target2-scsi-options) (not found) obp_proplen(0xffd4cc14, target2-scsi-options) (not found) obp_proplen(0xffd4caa4, target2-scsi-options) (not found) obp_proplen(0xffd4527c, target2-scsi-options) (not found) obp_proplen(0xffd455e4, target2-scsi-options) (not found) obp_proplen(0xffd4fac0, target3-scsi-options) (not found) obp_proplen(0xffd4cfec, target3-scsi-options) (not found) obp_proplen(0xffd4cc14, target3-scsi-options) (not found) obp_proplen(0xffd4caa4, target3-scsi-options) (not found) obp_proplen(0xffd4527c, target3-scsi-options) (not found) obp_proplen(0xffd455e4, target3-scsi-options) (not found) obp_proplen(0xffd4fac0, target4-scsi-options) (not found) obp_proplen(0xffd4cfec, target4-scsi-options) (not found) obp_proplen(0xffd4cc14, target4-scsi-options) (not found) obp_proplen(0xffd4caa4, target4-scsi-options) (not found) obp_proplen(0xffd4527c, target4-scsi-options) (not found) obp_proplen(0xffd455e4, target4-scsi-options) (not found) obp_proplen(0xffd4fac0, target5-scsi-options) (not found) obp_proplen(0xffd4cfec, target5-scsi-options) (not found) obp_proplen(0xffd4cc14, target5-scsi-options) (not found) obp_proplen(0xffd4caa4, target5-scsi-options) (not found) obp_proplen(0xffd4527c, target5-scsi-options) (not found) obp_proplen(0xffd455e4, target5-scsi-options) (not found) obp_proplen(0xffd4fac0, target6-scsi-options) (not found) obp_proplen(0xffd4cfec, target6-scsi-options) (not found) obp_proplen(0xffd4cc14, target6-scsi-options) (not found) obp_proplen(0xffd4caa4, target6-scsi-options) (not found) obp_proplen(0xffd4527c, target6-scsi-options) (not found) obp_proplen(0xffd455e4, target6-scsi-options) (not found) obp_proplen(0xffd4fac0, target7-scsi-options) (not found) obp_proplen(0xffd4cfec, target7-scsi-options) (not found) obp_proplen(0xffd4cc14, target7-scsi-options) (not found) obp_proplen(0xffd4caa4, target7-scsi-options) (not found) obp_proplen(0xffd4527c, target7-scsi-options) (not found) obp_proplen(0xffd455e4, target7-scsi-options) (not found) obp_proplen(0xffd4fac0, clock-frequency) = 4 obp_proplen(0xffd4fac0, clock-frequency) = 4 obp_getprop(0xffd4fac0, clock-frequency) = 02625a00 ESP: write reg[11]: 0x00 -> 0x00 ESP: write reg[11]: 0x00 -> 0x0a ESP: read reg[11]: 0x0a ESP: write reg[12]: 0x00 -> 0x00 ESP: write reg[12]: 0x00 -> 0x05 ESP: read reg[12]: 0x05 ESP: write reg[11]: 0x0a -> 0x08 ESP: write reg[12]: 0x05 -> 0x00 ESP: write reg[3]: 0x10 -> 0x03 ESP: Bus reset (03) ESP: Raise IRQ ESP: write reg[3]: 0x00 -> 0x02 ESP: Chip reset (02) ESP: write reg[3]: 0x02 -> 0x80 ESP: NOP (80) ESP: write reg[3]: 0x80 -> 0x80 ESP: NOP (80) ESP: write reg[9]: 0x00 -> 0x00 ESP: write reg[5]: 0x00 -> 0xa3 ESP: write reg[6]: 0x00 -> 0x00 ESP: write reg[7]: 0x00 -> 0x00 ESP: read reg[14]: 0x04 ESP: read reg[14]: 0x04 ESP: write reg[8]: 0x00 -> 0x17 ESP: write reg[12]: 0x00 -> 0x01 ESP: write reg[11]: 0x00 -> 0x08 obp_proplen(0xffd4fac0, esp-options) (not found) obp_proplen(0xffd4cfec, esp-options) (not found) obp_proplen(0xffd4cc14, esp-options) (not found) obp_proplen(0xffd4caa4, esp-options) (not found) obp_proplen(0xffd4527c, esp-options) (not found) obp_proplen(0xffd455e4, esp-options) (not found) /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,8800000 (esp0): esp-options=0x46 ESP: read reg[5]: 0x00 esp0 at dma0: SBus slot 5 0x8800000 sparc ipl 4 esp0 is /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,8800000 obp_proplen(0xffd4fac0, pm-hardware-state) (not found) obp_proplen(0xffd4fac0, pm-want-child-notification?) (not found) obp_fortheval_v2(0 0 f024360c f59a88c8 47 ['] find-device catch if 2drop true else current-device device-end then swap l!) ### Writing to iommu addr: 6, value fc001000 0 --- Current translation for 0xfc000000 is 0 ### IOMMU page flush fc001000 ESP: write reg[4]: 0x00 -> 0x00 ESP: write reg[6]: 0x00 -> 0x00 ESP: write reg[7]: 0x00 -> 0x00 ESP: write reg[12]: 0x01 -> 0x01 ESP: write reg[8]: 0x17 -> 0x07 ESP: write reg[0]: 0x00 -> 0x07 ESP: write reg[1]: 0x00 -> 0x00 ==== Enabling DMA a4240210 ESP: Raise enable ESP: write reg[3]: 0x80 -> 0xc2 ESP: Select with ATN (c2) ESP: get_cmd: len 7 target 0 buf 0x7fff253fed90 ESP: ### No such drive! ESP: ### No such drive pause over! ESP: Raise IRQ qemu: fatal: Trap 0x29 while interrupts disabled, Error state
I've altered the qemu IOMMU code so that every time a new value is written to one of the control registers, we display the current translation for 0xfc000000 which should contain a valid mapping after the IOMMU TLB flush (or why else would the kernel want to flush the TLB?)
There is also extra code to show when the DMA_EN bit is set which prints "==== Enabling DMA" and the new value of the control register. Looking at the earlier NetBSD bug reports, it looks as if this is doing the right thing in that the DMA is only triggered once DMA_EN has been set.
At the moment I'm really struggling to find out why the IOMMU mapping isn't being setup, and since this is being managed by the Solaris kernel it's hard to see how OpenBIOS is having an effect on this (unless of course it is influenced indirectly by various OBP properties). Anyone have any more ideas?
ATB,
Mark.