In my quest to get POST codes to show on my Asus P8Z77-M, and fixing sb/bd82x6x [1] along the way, I'm still not quite getting them.
I have a ISA/PCI POST card, but it isn't giving me anything. I bought a little "Debug Card Expert" off Amazon that looks like a mini-PCI/mini-PCIe cross, but in today's testing I found out that it isn't wired properly (i.e. isn't going to work) for PCIe at all, but I did get it to show POST codes over LPC via the TPM header. I have another of this same little PoS that broke in half on arrival. Now that I know it only really works over LPC and only 9 signals, I'll attempt to repair the broken traces and glue it together.
My board has a PCI-PCI bridge on bus:device.func 3:0.0 (so bus 3, this will be important) that services the PCI slot. After applying [1] to have the PCH send port 80 data to PCI as I specified in Kconfig, I still don't get any POST code... and not over LPC either, so I know my patch works.
Now I think I may need to enable CONFIG_EARLY_PCI_BRIDGE. Problem is the bridge has to be a device on bus 0.
How can I enable this PCI bridge on bus 3 early so a POST card sitting on it can get codes?
Thanks Keith
Hi Keith,
On 22.12.23 08:53, Keith Hui wrote:
My board has a PCI-PCI bridge on bus:device.func 3:0.0 (so bus 3, this will be important) that services the PCI slot. After applying [1] to have the PCH send port 80 data to PCI as I specified in Kconfig, I still don't get any POST code... and not over LPC either, so I know my patch works.
Now I think I may need to enable CONFIG_EARLY_PCI_BRIDGE. Problem is the bridge has to be a device on bus 0.
you'll have to enable all bridges on the path to your PCI device. Starting with the bridge on bus 0. Note that all bus numbers except bus 0 are decided by software. The EARLY_PCI_BRIDGE code is currently not prepared for this. You'll have to * configure more than one bus (repeat what the code currently does) * configure PCI_SUBORDINATE_BUS everywhere to the highest bus number * configure a port I/O range (code currently assumes MMIO) (I don't think there's a simple port 80h switch for PCI bridges, but could be wrong)
The code currently sets bus 15 (as a comment suggests, this is an arbitrary number >0). For your case with the P8Z77-M, the procedure could look like this:
For 00:1c.6: * PCI_SECONDARY_BUS => 15 * PCI_SUBORDINATE_BUS => 16
For 0f:00.0: * PCI_SECONDARY_BUS => 16 * PCI_SUBORDINATE_BUS => 16
For both bridges: * PCI_IO_BASE => 0x00 * PCI_IO_LIMIT => 0x00 (lower bits are implied to be 1s) * PCI_IO_BASE_UPPER16 => 0x00 (registers only exist if indicated * PCI_IO_LIMIT_UPPER16 => 0x00 by PCI_IO_BASE[0..3] == 0x01) * PCI_COMMAND => PCI_COMMAND_IO
Then your device should show up as 10:00.0 (if the board uses 00 as slot number), in case you need to configure something there. That's usually done in pci_early_device_probe().
It's probably not too much work to refactor the code to support multiple bridges in a generic way. For instance, by allowing CONFIG_EARLY_PCI_BRIDGE_DEVICE and _FUNCTION to be comma- separated lists (having two arrays is a bit ugly, though...):
unsigned int bus = 0; const unsigned int max_subordinate = 255; unsigned int bridge_devs[] = { CONFIG_EARLY_PCI_BRIDGE_DEVICES }; unsigned int bridge_funcs[] = { CONFIG_EARLY_PCI_BRIDGE_FUNCTIONS }; _Static_assert (ARRAY_SIZE(bridge_devs) == ARRAY_SIZE(bridge_funcs));
for (int i = 0; i < ARRAY_SIZE(bridge_devs); ++i, ++bus) { const pci_devfn_t bridge = PCI_DEV(bus, bridge_devs[i], bridge_funcs[i]); const unsigned int secondary_bus = bus + 1; /* do thing for each bridge */ }
/* pci_early_device_probe() etc. */
Hope that helps, Nico
(Also including the list as I welcome more eyeballs.)
Hi Nico,
Thanks for your pointers.
On Fri, 22 Dec 2023 at 08:55, Nico Huber nico.h@gmx.de wrote:
Hi Keith,
On 22.12.23 08:53, Keith Hui wrote:
My board has a PCI-PCI bridge on bus:device.func 3:0.0 (so bus 3, this will be important) that services the PCI slot. After applying [1] to have the PCH send port 80 data to PCI as I specified in Kconfig, I still don't get any POST code... and not over LPC either, so I know my patch works.
Now I think I may need to enable CONFIG_EARLY_PCI_BRIDGE. Problem is the bridge has to be a device on bus 0.
you'll have to enable all bridges on the path to your PCI device. Starting with the bridge on bus 0. Note that all bus numbers except bus 0 are decided by software. The EARLY_PCI_BRIDGE code is currently not prepared for this. You'll have to
- configure more than one bus (repeat what the code currently does)
- configure PCI_SUBORDINATE_BUS everywhere to the highest bus number
- configure a port I/O range (code currently assumes MMIO) (I don't think there's a simple port 80h switch for PCI bridges, but could be wrong)
The code currently sets bus 15 (as a comment suggests, this is an arbitrary number >0). For your case with the P8Z77-M, the procedure could look like this:
For 00:1c.6:
- PCI_SECONDARY_BUS => 15
- PCI_SUBORDINATE_BUS => 16
For 0f:00.0:
- PCI_SECONDARY_BUS => 16
- PCI_SUBORDINATE_BUS => 16
For both bridges:
- PCI_IO_BASE => 0x00
- PCI_IO_LIMIT => 0x00 (lower bits are implied to be 1s)
- PCI_IO_BASE_UPPER16 => 0x00 (registers only exist if indicated
- PCI_IO_LIMIT_UPPER16 => 0x00 by PCI_IO_BASE[0..3] == 0x01)
- PCI_COMMAND => PCI_COMMAND_IO
Then your device should show up as 10:00.0 (if the board uses 00 as slot number), in case you need to configure something there. That's usually done in pci_early_device_probe().
I came up with the code at the end of this email. However it renders my board so inoperative I had to rescue myself. Can you see any obvious errors in it?
I commented out the timeout part as I thought it was hanging me up. It was not.
Thanks Keith
--------8<--------
void bootblock_mainboard_early_init(void) { nuvoton_enable_serial(SERIAL_DEV, CONFIG_TTYS0_BASE);
if (CONFIG(POST_DEVICE_PCI_PCIE)) { /* * Enable PCIe root port 7 (00:1c.6) and the ASM1083 PCIe-PCI bridge behind it * and have it accept I/O cycles. This should allow a POST card in the PCI slot * to receive POST codes. * * Below is copied and modified from device/pci_early.c because CONFIG_PCI_EARLY_BRIDGE * doesn't quite do what we need. */ //int timeout; const int secondary = 3; pci_devfn_t bridge = PCH_PCIE_DEV(6); //pci_devfn_t p2p_bridge = PCI_DEV(secondary_bus, 0, 0);
/* Trigger secondary bus reset and program its bus number. */ pci_s_assert_secondary_reset(bridge); pci_s_deassert_secondary_reset(bridge); pci_s_write_config8(bridge, PCI_SECONDARY_BUS, secondary); pci_s_write_config8(bridge, PCI_SUBORDINATE_BUS, secondary + 1);
u16 reg16;
/* Enable I/O window behind the bridge. * Base = 0x0000 * Limit = 0x0FFF */ pci_s_write_config16(bridge, PCI_IO_BASE, 0x0000); pci_or_config16(bridge, PCI_COMMAND, PCI_COMMAND_IO);
/* Enable subtractive decode (PECR3[SDE]). */ pci_or_config8(bridge, 0xec, 0x01); /* Have to do it in D30F0 too, even though Z77 doesn't natively do PCI. */ pci_or_config32(PCI_DEV(0, 30, 0), 0x4c, (1 << 28));
/* Done with root port. Now do the 1083. */ bridge = PCI_DEV(secondary, 0, 0);
/* for (timeout = 20000; timeout; timeout--) { u32 id = pci_s_read_config32(bridge, PCI_VENDOR_ID); if (id != 0 && id != 0xffffffff && id != 0xffff0001) break; udelay(10); }*/
reg16 = pci_read_config16(bridge, PCI_CLASS_DEVICE);
if (reg16 != 0x0604) /* PCI-PCI Bridge */ return;
/* Trigger secondary bus reset and program its bus number. */ pci_s_assert_secondary_reset(bridge); pci_s_deassert_secondary_reset(bridge); pci_s_write_config8(bridge, PCI_SECONDARY_BUS, secondary + 1); pci_s_write_config8(bridge, PCI_SUBORDINATE_BUS, secondary + 1); /* Set this second bridge to decode the same I/O ports. */ pci_s_write_config16(bridge, PCI_IO_BASE, 0x0000); pci_or_config16(bridge, PCI_COMMAND, PCI_COMMAND_IO); }
}
Hi again, and a happy new year!
On 03.01.24 05:44, Keith Hui wrote:
/* Trigger secondary bus reset and program its bus number. */ pci_s_assert_secondary_reset(bridge); pci_s_deassert_secondary_reset(bridge);
I don't think this is actually necessary as everything should be reset already. The actual procedure is more involved. There should be a delay between assertion and de-assertion.
pci_s_write_config8(bridge, PCI_SECONDARY_BUS, secondary); pci_s_write_config8(bridge, PCI_SUBORDINATE_BUS, secondary + 1); u16 reg16;/* Enable I/O window behind the bridge.* Base = 0x0000 * Limit = 0x0FFF */ pci_s_write_config16(bridge, PCI_IO_BASE, 0x0000); pci_or_config16(bridge, PCI_COMMAND, PCI_COMMAND_IO);
One thing I didn't consider, this will conflict with all the other legacy I/O ports (e.g. RTC, UART). And it's not necessary, as...
/* Enable subtractive decode (PECR3[SDE]). */ pci_or_config8(bridge, 0xec, 0x01);
...this should actually suffice for the root port. I didn't know Intel implemented this. But given that they did, it's the best option for your case. I don't know if you also have to set the PCI_COMMAND_IO bit. If you do, set PCI_IO_BASE to 0x00ff (which disables it to avoid conflicts by positive decoding).
/* Have to do it in D30F0 too, even though Z77 doesn'tnatively do PCI. */ pci_or_config32(PCI_DEV(0, 30, 0), 0x4c, (1 << 28));
But you should never enable this for more than one device on a given bus. If your port 80 sink is behind the root port, don't do this for the legacy PCI bridge.
/* Done with root port. Now do the 1083. */ bridge = PCI_DEV(secondary, 0, 0);/* for (timeout = 20000; timeout; timeout--) { u32 id = pci_s_read_config32(bridge, PCI_VENDOR_ID); if (id != 0 && id != 0xffffffff && id != 0xffff0001) break; udelay(10); }*/
reg16 = pci_read_config16(bridge, PCI_CLASS_DEVICE); if (reg16 != 0x0604) /* PCI-PCI Bridge */ return; /* Trigger secondary bus reset and program its bus number. */ pci_s_assert_secondary_reset(bridge); pci_s_deassert_secondary_reset(bridge); pci_s_write_config8(bridge, PCI_SECONDARY_BUS, secondary + 1); pci_s_write_config8(bridge, PCI_SUBORDINATE_BUS, secondary + 1); /* Set this second bridge to decode the same I/O ports. */ pci_s_write_config16(bridge, PCI_IO_BASE, 0x0000); pci_or_config16(bridge, PCI_COMMAND, PCI_COMMAND_IO);
You should clear the PCI_SECONDARY_BUS registers afterwards as the current early-PCI code does. This is to avoid conflicts with coreboot's later PCI enumeration. Start with PCI_DEV(secondary, 0, 0), then the root port.
Hope this helps, Nico
Happy new year to you too!
On Fri, 5 Jan 2024 at 09:58, Nico Huber nico.h@gmx.de wrote:
Hi again, and a happy new year!
With this code below I did just enough to enable subtractive decode in the root port and legacy PCI bridge (datasheet does call for doing both for legacy PCI mode) and nothing else. That ASM1083 just worked, and I finally got POST codes on PCI slot.
However, I lost integrated graphics completely. A boot log is attached. Can you see why this is happening?
Thanks Keith
----8<----
void bootblock_mainboard_early_init(void) { nuvoton_enable_serial(SERIAL_DEV, CONFIG_TTYS0_BASE);
if (CONFIG(POST_DEVICE_PCI_PCIE)) { u32 rpfn = RCBA32(RPFN); /* Unhide root port 7. */ rpfn &= ~(1 << 27); RCBA32(RPFN) = rpfn; /* Get root port 7 function number. */ unsigned int rpfunc = (rpfn >> 24) & 0x7;
pci_devfn_t bridge = PCH_PCIE_DEV(rpfunc);
/* Enable subtractive decode (PECR3[SDE]). */ pci_or_config8(bridge, 0xec, 0x01);
/* Enable I/O but don't set a window. */ pci_s_write_config16(bridge, PCI_IO_BASE, 0x00ff); pci_or_config16(bridge, PCI_COMMAND, PCI_COMMAND_IO);
/* Datasheet p.129: Have to enable subtractive decode in D30F0 too, * even though Z77 doesn't natively do PCI. */ pci_or_config32(PCI_DEV(0, 30, 0), 0x4c, (1 << 28));
/* POST card in the PCI slot should now receive POST codes. */ } }
On 11.01.24 07:03, Keith Hui wrote:
Happy new year to you too!
On Fri, 5 Jan 2024 at 09:58, Nico Huber nico.h@gmx.de wrote:
Hi again, and a happy new year!
With this code below I did just enough to enable subtractive decode in the root port and legacy PCI bridge (datasheet does call for doing both for legacy PCI mode) and nothing else. That ASM1083 just worked, and I finally got POST codes on PCI slot.
That is interesting, so the ASM1083 forwards port 80h without configuration?
However, I lost integrated graphics completely. A boot log is attached.
I fear it is not. Did you forget the attachment?
Can you see why this is happening?
At least I can't imagine anything that this little code could do to VGA registers. Have you tried a graphics framebuffer? Not sure if it can be configured without legacy i/o.
Thanks Keith
----8<----
void bootblock_mainboard_early_init(void) { nuvoton_enable_serial(SERIAL_DEV, CONFIG_TTYS0_BASE);
if (CONFIG(POST_DEVICE_PCI_PCIE)) { u32 rpfn = RCBA32(RPFN); /* Unhide root port 7. */ rpfn &= ~(1 << 27); RCBA32(RPFN) = rpfn; /* Get root port 7 function number. */ unsigned int rpfunc = (rpfn >> 24) & 0x7; pci_devfn_t bridge = PCH_PCIE_DEV(rpfunc); /* Enable subtractive decode (PECR3[SDE]). */ pci_or_config8(bridge, 0xec, 0x01); /* Enable I/O but don't set a window. */ pci_s_write_config16(bridge, PCI_IO_BASE, 0x00ff); pci_or_config16(bridge, PCI_COMMAND, PCI_COMMAND_IO); /* Datasheet p.129: Have to enable subtractive decode in D30F0 too, * even though Z77 doesn't natively do PCI. */
I still read that it says only one device at a time. It might mention D30F0 only due to copy-pasting from older documentation. OTOH, if this indeed makes it work, I would be wrong.
pci_or_config32(PCI_DEV(0, 30, 0), 0x4c, (1 << 28)); /* POST card in the PCI slot should now receive POST codes. */ }}
On 11.01.24 14:43, Nico Huber via coreboot wrote:
On 11.01.24 07:03, Keith Hui wrote:
However, I lost integrated graphics completely. A boot log is attached.
I fear it is not. Did you forget the attachment?
Found it now. It says early in ramstage:
[INFO ] PCI: Static device PCI: 00:02.0 not found, disabling it.
So that's definitely an issue. But I really can't imagine how this is happening, and why this particular device. As I understand the hardware, all the decoding settings in the PCH shouldn't affect the northbridge devices.
There's more
[DEBUG] IGD decoded, subtracting 160M UMA and 0M GTT
0M isn't a good idea. This just prints what register settings were found. Those were probably set in romstage (compiled into bootblock with your settings if I interpreted the log correctly).
It looks all very odd and alas I have no clue how it could relate to the subtractive-decode settings.
Nico
On Thu, 11 Jan 2024 at 10:52, Nico Huber nico.h@gmx.de wrote:
On Fri, 5 Jan 2024 at 09:58, Nico Huber nico.h@gmx.de wrote:
Hi again, and a happy new year!
With this code below I did just enough to enable subtractive decode in the root port and legacy PCI bridge (datasheet does call for doing both for legacy PCI mode) and nothing else. That ASM1083 just worked, and I finally got POST codes on PCI slot.
That is interesting, so the ASM1083 forwards port 80h without configuration?
Pretty much. Documentation on this chip is very minimal - it only has a register listing and their defaults, no explaining what any of them do.
It too has a subtractive decode bit that defaults to off, but no minute details around it.
On 11.01.24 14:43, Nico Huber via coreboot wrote:
On 11.01.24 07:03, Keith Hui wrote:
However, I lost integrated graphics completely. A boot log is attached.
I fear it is not. Did you forget the attachment?
Found it now. It says early in ramstage:
[INFO ] PCI: Static device PCI: 00:02.0 not found, disabling it.
So that's definitely an issue. But I really can't imagine how this is happening, and why this particular device. As I understand the hardware, all the decoding settings in the PCH shouldn't affect the northbridge devices.
There's more
[DEBUG] IGD decoded, subtracting 160M UMA and 0M GTT
0M isn't a good idea. This just prints what register settings were found. Those were probably set in romstage (compiled into bootblock with your settings if I interpreted the log correctly).
It looks all very odd and alas I have no clue how it could relate to the subtractive-decode settings.
Me neither - I didn't know where this setting resides.
After giving an arbitrary out-of-the-way bus number (10) to the root port, I finally got both POST codes and integrated graphic working.
Thanks Nico for your help so far - but wait, there's more!
Seems all the troubles I have encountered along the way: System becoming inoperative - turns out it got hung up during PCI bus scan Loss of integrated graphics Loss of ability to soft reboot and power off ... are all the results of bus number conflict.
Another takeaway is I don't really need the legacy PCI bridge device aka D30F0 at all. All my troubles old and new this guy doesn't seem to make any difference.
Leaving this setup in, now I have a Sound Blaster Live! in the PCI slot and it can't initialize.
Kernel reports: pci 0000:0a:00.0: can't derive routing for PCI INT A snd_emu10k1 0000:0b:00.0: PCI INT A: no GSI genirq: Flags mismatch irq 0. 00000080 (snd_emu10k1) vs. 00015a00 (timer) snd_emu10k1: probe of 0000:0b:00.0 failed with error -16 (that -16 is -EBUSY.)
Since I shut the I/O window on the (subtractive) root port and the PCI code in ramstage didn't touch it, the sound card didn't have I/O allocated and can't possibly work.
I also see that ramstage did not scan bus 10. Is it by design? Will I have to probe for a device and abort this attempt if one is found? My thinking is POST cards tend to be cheap designs that don't respond to bus scans nor have a config space. I have no experience with PCIe POST cards because I don't have one, and are more expensive starting at $50.
How can I convince the PCI enumerator to work on this fudged root port like any other?
Thanks Keith
-
Keith Hui -
Nico Huber