Hi Nicholas,
Thanks for the reply, and the scoop, and pardon my second email to the list that essentially asked the same question again.
The hidden register would be a boon to my goals. I have an Asus P8Z77-V LE Plus board on order that I will soon be able to test the idea on, since it is one of those boards with more devices than PCIe lanes. It is part of my plan to bring coreboot to all the P8x7x boards. Towards that goal there are a few patches I'm looking to get in such as CB:82556,82557,85834, and am still looking for volunteers to boot test my as yet incomplete ports.
Cheers Keith
Hi Keith,
I can't speak to the devicetree stuff, but I did find something interesting regarding the PCH PCIe ports:
p8z77-v and sabertooth_z77 have a third PCIe x4 slot serviced by the PCH, but the 4 lanes are shared with some other x1 slots and devices. They can configure the PCIe x4 slot to really be x4, or the 4 lanes can be configured as 4 x1 ports for various x1 slots or onboard devices. This requires changing a soft strap as well as manipulating some GPIOs in the SIO chip, and the devicetree changed to match. I have the SIO GPIO part figured out, the soft strap I can only come to one possible explanation: reflash SPI descriptor with the changed soft strap and force a platform reset. To verify I'll need to get a board and actually check the flash chip for changes.
Changing the IFD at runtime seems like an impractical way to achieve this, especially since the IFD is normally supposed to be read only. I suspect there's another, undocumented method to change the configuration. On ICH/PCH datasheets, there is an RCBA register (Root Port Configuration Register) that indicates the current port configuration. On the 7 series PCH's, that is at RCBA32(0x400), bits 3:2 for ports 5-8 and bits 1:0 for ports 1:4. However, these bitfields are documented as read only in the public datasheets. However, in the public ICH9 datasheet (and seemingly only the ICH9 datasheet; other datasheets like ICH7, ICH8, ICH10, etc don't have this detail) the equivalent register at RCBA32(0x224) lists the bitfields as R/W, with a note saying that writing to them is for debug/testing and should be treated as read only and modifiable only through hard straps (it seems like ICH9 didn't use IFD soft straps for this). Based on that note, I assume this mechanism is intended for testing different PCH PCIe configurations without reflashing the IFD. I haven't tested this yet on any platform but it does seem plausible that this mechanism exists on chipsets other than ICH9.
Cheers, Nicholas
Hi Keith,
Hi Nicholas,
Thanks for the reply, and the scoop, and pardon my second email to the list that essentially asked the same question again.
The hidden register would be a boon to my goals. I have an Asus P8Z77-V LE Plus board on order that I will soon be able to test the idea on, since it is one of those boards with more devices than PCIe lanes. It is part of my plan to bring coreboot to all the P8x7x boards. Towards that goal there are a few patches I'm looking to get in such as CB:82556,82557,85834, and am still looking for volunteers to boot test my as yet incomplete ports.
Cheers Keith
Hi Keith, I can't speak to the devicetree stuff, but I did find something interesting regarding the PCH PCIe ports: > p8z77-v and sabertooth_z77 have a third PCIe x4 slot serviced by the > PCH, but the 4 lanes are shared with some other x1 slots and devices. > They can configure the PCIe x4 slot to really be x4, or the 4 lanes > can be configured as 4 x1 ports for various x1 slots or onboard > devices. This requires changing a soft strap as well as manipulating > some GPIOs in the SIO chip, and the devicetree changed to match. I > have the SIO GPIO part figured out, the soft strap I can only come to > one possible explanation: reflash SPI descriptor with the changed soft > strap and force a platform reset. To verify I'll need to get a board > and actually check the flash chip for changes. Changing the IFD at runtime seems like an impractical way to achieve this, especially since the IFD is normally supposed to be read only. I suspect there's another, undocumented method to change the configuration. On ICH/PCH datasheets, there is an RCBA register (Root Port Configuration Register) that indicates the current port configuration. On the 7 series PCH's, that is at RCBA32(0x400), bits 3:2 for ports 5-8 and bits 1:0 for ports 1:4. However, these bitfields are documented as read only in the public datasheets. However, in the public ICH9 datasheet (and seemingly only the ICH9 datasheet; other datasheets like ICH7, ICH8, ICH10, etc don't have this detail) the equivalent register at RCBA32(0x224) lists the bitfields as R/W, with a note saying that writing to them is for debug/testing and should be treated as read only and modifiable only through hard straps (it seems like ICH9 didn't use IFD soft straps for this). Based on that note, I assume this mechanism is intended for testing different PCH PCIe configurations without reflashing the IFD. I haven't tested this yet on any platform but it does seem plausible that this mechanism exists on chipsets other than ICH9. Cheers, Nicholas
I did some experimentation on my Latitude E6430 (Ivy Bridge/7 Series Chipset) by attempting to write to RCBA32(0x400) in sb/intel/bd82x6x/early_rcba.c:southbridge_rcba_config(). Unfortunately, that register remained unchanged before and after the write. So I guess either those fields actually are read only, or there's some other thing that I'm missing. Perhaps there's some sort of lock that prevents those bits from being modified if they are indeed writable. I'll probably do some experimentation on my Latitude E6400 (GM45/ICH9) since that chipset did explicitly document those bits as read write.
Cheers, Nicholas
Hi Keith,
Hi Nicholas,
Thanks for the reply, and the scoop, and pardon my second email to the list that essentially asked the same question again.
The hidden register would be a boon to my goals. I have an Asus P8Z77- V LE Plus board on order that I will soon be able to test the idea on, since it is one of those boards with more devices than PCIe lanes. It is part of my plan to bring coreboot to all the P8x7x boards. Towards that goal there are a few patches I'm looking to get in such as CB:82556,82557,85834, and am still looking for volunteers to boot test my as yet incomplete ports.
Cheers Keith
Oh, I was going to add: When you get your P8Z77-V LE Plus, I guess it would be a good idea to set the different options for that "PCI Express X16_3 Slot (Black) bandwidth" option in the vendor firmware setup menu and then dump the IFD to see if it's actually changing any soft straps. It would also be good to collect register dumps from tools like inteltool (especially the RCBA registers) and lspci to see if there's anything else that's changing.
Cheers, Nicholas
The board is here. For some reason both my old Fedora installation and SystemRescue failed to boot on it so I can't run autoport yet.
However, I did change that PCIe bandwidth setting and took a SPI flash dump before and after. That PCH soft strap DID change.
Where do we go from here?
Thanks Keith
On Thu, Jan 9, 2025, 22:24 Nicholas Chin nic.c3.14@gmail.com wrote:
Hi Keith,
Hi Nicholas,
Thanks for the reply, and the scoop, and pardon my second email to the list that essentially asked the same question again.
The hidden register would be a boon to my goals. I have an Asus P8Z77- V LE Plus board on order that I will soon be able to test the idea on, since it is one of those boards with more devices than PCIe lanes. It is part of my plan to bring coreboot to all the P8x7x boards. Towards that goal there are a few patches I'm looking to get in such as CB:82556,82557,85834, and am still looking for volunteers to boot test my as yet incomplete ports.
Cheers Keith
Oh, I was going to add: When you get your P8Z77-V LE Plus, I guess it would be a good idea to set the different options for that "PCI Express X16_3 Slot (Black) bandwidth" option in the vendor firmware setup menu and then dump the IFD to see if it's actually changing any soft straps. It would also be good to collect register dumps from tools like inteltool (especially the RCBA registers) and lspci to see if there's anything else that's changing.
On 2025-01-16 00:48, Keith Hui wrote:
The board is here. For some reason both my old Fedora installation and SystemRescue failed to boot on it so I can't run autoport yet.
However, I did change that PCIe bandwidth setting and took a SPI flash dump before and after. That PCH soft strap DID change.
Where do we go from here?
Thanks Keith
Hmm, interesting. There are some boards that don't seem to have a UEFI option to change the bandwidth, and instead change it depending on whether a slot is populated or not. From what I understand doing that with the IFD would require booting, detecting a card being present, reflashing the IFD, and rebooting; all without any indication to the user. So hypothetically, just plugging in a PCIe card could soft brick the system if something went wrong while reflashing the IFD. At least with the UEFI setting a user has to intentionally make a change before the IFD is reflashed.
I know someone with a Haswell board that does it at runtime; I'll see if we can experiment and determine if it's actually reflashing the IFD at runtime or if it seems to be doing something else. That said, UEFItool does show a DXE called "DescriptorUpdate", which might suggest that it is indeed rewriting the descriptor. But it's hard to say without digging further into the firmware what that is actually used for. I'd hope that the behavior on Haswell chipsets is similar to Ivy Bridge chipsets, though I did notice that the Root Port Configuration Register on Haswell would always return 0, whereas on Ivy Bridge the same register would contain values that reflected what was set in the IFD.
From experimentation, it seems like the RPC register is indeed writable on ICH9 like the datasheet suggests, and that results in the PCI config registers for the root ports reporting different widths for the link capabilities depending on the value written to the RPC register. I don't have any way to test if the ports actually operate as wider widths though since all of those are routed to mPCIe slots or the Expresscard, which are all x1 links. Interestingly it also doesn't seem to hide the PCI device for the root ports that shouldn't be available due to their lanes being assigned to a different root port operating with a wider link width, and thus it appears that there are more total PCIe lanes than are physically present. As I've mentioned before, writing to that register on Ivy Bridge and Haswell seems to have no effect.
I guess this doesn't really get us anywhere other than suggesting that the PCH soft straps are indeed the only way to change things on newer chipsets.
Cheers, Nicholas
-
Keith Hui -
Nicholas Chin