Re: [coreboot] Assigning PCI resources clobbers console with io-mapped / memory-mapped UART

[Aaron Durbin]

On Thu, Mar 29, 2018 at 3:36 PM, Jay Talbott
 wrote:
> We ran into an issue where we were not getting a full coreboot log on
> Denverton with the Harcuvar CRB, where it just abruptly stops the serial
> console output during the assignment of the PCI resources.
>
> Root Device assign_resources, bus 0 link: 0
>
> DOMAIN: 0000 assign_resources, bus 0 link: 0
>
> PCI: 00:09.0 1c <- [0x000000ffff - 0x000000fffe] size 0x00000000 gran 0x0c
> bus 01 io
>
> PCI: 00:09.0 24 <- [0x00dfffffff - 0x00dffffffe] size 0x00000000 gran 0x14
> bus 01 prefmem
>
> PCI: 00:09.0 20 <- [0x00dfffffff - 0x00dffffffe] size 0x00000000 gran 0x14
> bus 01 mem
>
> PCI: 00:09.0 10 <- [0x00df700000 - 0x00df71ffff] size 0x00020000 gran 0x11
> mem64
>
> PCI: 00:0e.0 1c <- [0x000000ffff - 0x000000fffe] size 0x00000000 gran 0x0c
> bus 02 io
>
> PCI: 00:0e.0 24 <- [0x00dfffffff - 0x00dffffffe] size 0x00000000 gran 0x14
> bus 02 prefmem
>
> PCI: 00:0e.0 20 <- [0x00dfffffff - 0x00dffffffe] size 0x00000000 gran 0x14
> bus 02 mem
>
> PCI: 00:0e.0 10 <- [0x00df720000 - 0x00df73ffff] size 0x00020000 gran 0x11
> mem64
>
> PCI: 00:10.0 1c <- [0x000000ffff - 0x000000fffe] size 0x00000000 gran 0x0c
> bus 03 io
>
> PCI: 00:10.0 24 <- [0x00dc000000 - 0x00ddffffff] size 0x02000000 gran 0x14
> bus 03 prefmem
>
> PCI: 00:10.0 20 <- [0x00de000000 - 0x00de8fffff] size 0x00900000 gran 0x14
> bus 03 mem
>
> PCI: 00:10.0 10 <- [0x00df740000 - 0x00df75ffff] size 0x00020000 gran 0x11
> mem64
>
> PCI: 00:10.0 assign_resources, bus 3 link: 0
>
> PCI: 03:00.0 1c <- [0x000000ffff - 0x000000fffe] size 0x00000000 gran 0x0c
> bus 04 io
>
> PCI: 03:00.0 24 <- [0x00dc000000 - 0x00ddffffff] size 0x02000000 gran 0x14
> bus 04 prefmem
>
> PCI: 03:00.0 20 <- [0x00de000000 - 0x00de8fffff] size 0x00900000 gran 0x14
> bus 04 mem
>
> PCI: 03:00.0 assign_resources, bus 4 link: 0
>
> PCI: 04:00.0 10 <- [0x00dc000000 - 0x00ddffffff] size 0x02000000 gran 0x19
> prefmem
>
> PCI: 04:00.0 14 <- [0x00de820000 - 0x00de823fff] size 0x00004000 gran 0x0e
> mem
>
> PCI: 04:00.0 18 <- [0x00de000000 - 0x00de7fffff] size 0x00800000 gran 0x17
> mem
>
> PCI: 04:00.0 30 <- [0x00de800000 - 0x00de81ffff] size 0x00020000 gran 0x11
> romem
>
> PCI: 03:00.0 assign_resources, bus 4 link: 0
>
> PCI: 00:10.0 assign_resources, bus 3 link: 0
>
> PCI: 00:12.0 10 <- [0x00df77b000 - 0x00df77b3ff] size 0x00000400 gran 0x0a
> mem64
>
> PCI: 00:14.0 10 <- [0x00df774000 - 0x00df775fff] size 0x00002000 gran 0x0d
> mem
>
> PCI: 00:14.0 14 <- [0x00df77c000 - 0x00df77c0ff] size 0x00000100 gran 0x08
> mem
>
> PCI: 00:14.0 18 <- [0x0000001c40 - 0x0000001c47] size 0x00000008 gran 0x03
> io
>
> PCI: 00:14.0 1c <- [0x0000001c60 - 0x0000001c63] size 0x00000004 gran 0x02
> io
>
> PCI: 00:14.0 20 <- [0x0000001c00 - 0x0000001c1f] size 0x00000020 gran 0x05
> io
>
> PCI: 00:14.0 24 <- [0x00df77a000 - 0x00df77a7ff] size 0x00000800 gran 0x0b
> mem
>
> PCI: 00:15.0 10 <- [0x00df760000 - 0x00df76ffff] size 0x00010000 gran 0x10
> mem64
>
> PCI: 00:16.0 1c <- [0x000000ffff - 0x000000fffe] size 0x00000000 gran 0x0c
> bus 05 io
>
> PCI: 00:16.0 24 <- [0x00dea00000 - 0x00deefffff] size 0x00500000 gran 0x14
> bus 05 prefmem
>
> PCI: 00:16.0 20 <- [0x00df500000 - 0x00df5fffff] size 0x00100000 gran 0x14
> bus 05 mem
>
> PCI: 00:16.0 assign_resources, bus 5 link: 0
>
> PCI: 05:00.0 10 <- [0x00dea00000 - 0x00debfffff] size 0x00200000 gran 0x15
> prefmem64
>
> PCI: 05:00.0 20 <- [0x00dee00000 - 0x00dee03fff] size 0x00004000 gran 0x0e
> prefmem64
>
> PCI: 05:00.0 30 <- [0x00df500000 - 0x00df57ffff] size 0x00080000 gran 0x13
> romem
>
> PCI: 05:00.1 10 <- [0x00dec00000 - 0x00dedfffff] size 0x00200000 gran 0x15
> prefmem64
>
> PCI: 05:00.1 20 <- [0x00dee04000 - 0x00dee07fff] size 0x00004000 gran 0x0e
> prefmem64
>
> PCI: 05:00.1 30 <- [0x00df580000 - 0x00df5fffff] size 0x00080000 gran 0x13
> romem
>
> PCI: 00:16.0 assign_resources, bus 5 link: 0
>
> PCI: 00:17.0 1c <- [0x000000ffff - 0x000000fffe] size 0x00000000 gran 0x0c
> bus 06 io
>
> PCI: 00:17.0 24 <- [0x00df000000 - 0x00df4fffff] size 0x00500000 gran 0x14
> bus 06 prefmem
>
> PCI: 00:17.0 20 <- [0x00df600000 - 0x00df6fffff] size 0x00100000 gran 0x14
> bus 06 mem
>
> PCI: 00:17.0 assign_resources, bus 6 link: 0
>
> PCI: 06:00.0 10 <- [0x00df000000 - 0x00df1fffff] size 0x00200000 gran 0x15
> prefmem64
>
> PCI: 06:00.0 20 <- [0x00df400000 - 0x00df403fff] size 0x00004000 gran 0x0e
> prefmem64
>
> PCI: 06:00.0 30 <- [0x00df600000 - 0x00df67ffff] size 0x00080000 gran 0x13
> romem
>
> PCI: 06:00.1 10 <- [0x00df200000 - 0x00df3fffff] size 0x00200000 gran 0x15
> prefmem64
>
> PCI: 06:00.1 20 <- [0x00df404000 - 0x00df407fff] size 0x00004000 gran 0x0e
> prefmem64
>
> PCI: 06:00.1 30 <- [0x00df680000 - 0x00df6fffff] size 0x00080000 gran 0x13
> romem
>
> PCI: 00:17.0 assign_resources, bus 6 link: 0
>
> PCI: 00:18.0 10 <- [0x00df776000 - 0x00df776fff] size 0x00001000 gran 0x0c
> mem64
>
> The boot continues, there’s just no more console output from coreboot past
> this point. The payload launches fine after coreboot is finished. If we use
> a Linux kernel payload, it still successfully sends all of its kernel
> console output to the serial console beginning immediately after where the
> coreboot console output stopped.
>
> The last device that has its resources assigned that is displayed in the log
> is D24, F0 (PCI 00:18.0). The next device in the list to have its resources
> assigned is D26, F0 (PCI 00:1a.0), which is UART 0, which is the UART used
> for the serial console. So this is starting to make some sense…
>
> Since the BARs are assigned in order, and the first BAR of the UART device
> is the I/O BAR at offset 0x10, it’s pretty clear that changing the I/O space
> resource assigned to the UART is what’s clobbering the console. I haven’t
> looked at it yet, but I’m assuming that the UART driver just caches the base
> address early on and doesn’t know that the base address changed during PCI
> resource assignment later in the boot cycle.
>
> Of course, not only do we lose the console output, but also the UART driver
> is still writing the rest of the log to an I/O address that is no longer
> associated with the UART, but now potentially assigned to some other PCI
> device.
>
> Fortunately, with Denverton, the UART mode can be changed from the default
> Non Legacy Mode to Legacy Mode, which appears to have resolved the issue in
> this particular case.
>
> However, I can see this as being a general problem on any/all platforms that
> don’t have a legacy UART (or a UART that can be put in legacy mode). Any
> platform with UARTs that strictly use I/O and/or memory mapped registers
> that are accessed via PCI BARs have the potential of having the BARs
> programmed with different addresses during PCI resource assignment, thus
> terminating the console output and otherwise potentially causing other bad
> things to happen.
>
> I thought it best to bring this up to the coreboot community, since this
> could easily be a problem on any/all platforms that don’t have a legacy UART
> (or a UART that can be put into legacy mode). Has anybody else run into this
> and/or thought about a general solution to this potential problem?

You already have the partial solution in denverton, though it's really
messy and not sure where it came from:
src/soc/intel/denverton_ns/uart.c

src/soc/intel/skylake/uart.c has the currently preferred solution
(look for pch_uart_read_resources()). Alternatively you could use a
dynamic uart_platform_baseptr() implementation which returns the
updated addresses.
> Jay
>
> Jay Talbott
> Principal Consulting Engineer
> SysPro Consulting, LLC
> 3057 E. Muirfield St.
> Gilbert, AZ 85298
> [phone]
> [phone] (FAX)
> [e-mail]
>
> http://www.sysproconsulting.com