-----Original Message----- From: Marc Jones [mailto:marcj303@gmail.com] Sent: Thursday, November 20, 2008 11:13 AM To: Myles Watson Cc: Coreboot Subject: Re: [coreboot] HT chains fixup
On Wed, Nov 19, 2008 at 9:16 PM, Myles Watson mylesgw@gmail.com wrote:
-----Original Message----- From: Marc Jones [mailto:marcj303@gmail.com] Sent: Wednesday, November 19, 2008 4:11 PM To: Myles Watson Subject: Re: [coreboot] HT chains fixup
Myles, Hi, sorry to take so long to look at this. I have some possibly dumb questions.
No problem. I appreciate the review.
What is gained by knowing the ht path?
When a HT chain is powered on, all the devices have the same UnitID (0) The subordinate busses get assigned bus numbers in the order that they
are
found by PCI scan, but they are found in the order of the chain. As you assign each a device number, another device is visible at device 0 until
the
end of the chain is found.
But I think that the unitid scan happens before and device/dts is used or called. This has to be done to set the HT link speed. See northbridge\amd\k8\incoherent_ht.c. By the time the device code starts they are setup and the bus can be scanned as normal pci.
The problem is HTX slots and modules that can't be initialized without some code.
Every HT bridge/tunnel under a CPU acts like a PCI device. Linux doesn't know about HT and I am not sure that the dts needs to.
That's why I made ht devices morph into pci devices at chain enumeration time. Once they're assigned UnitIDs it doesn't matter where they are in
the
chain.
Because it is a unitid bus number it is already numbered. I think that the device code should use any bus number already set. That would be equivalent to setting the bus number with pci_x@ in the dts.
That could happen. The problem is how you pick the "correct" early enumeration. It's easy to pick an enumeration that lets you see all the busses and optimize the HT links. It's not easy to make the busses get numbered correctly, which is why you need the dts information.
The exception being that coreboot needs to know the CPU that the bus comes off of. I feel like I am missing something obvious that you, Ron, and others understand.
I'm getting just as much push back from everyone else. I'm not sure
anyone
understands the code that well. I feel silly for having to rewrite so
much
of it. It probably points to my lack of understanding.
What are you describing with and ht@? For example:
ht@0,a {/config/("southbridge/amd/amd8132/pcix.dts");
};pci_a@0,1{/config/("southbridge/amd/amd8132/apic.dts");
ht@0,a is ht chain 0, UnitID a. I wanted it to be explicit where the
chains
end and what the numbers are. The way the code works is that it takes
the
HT device and all the PCI devices that it implements off the list at
once.
Without knowing which devices are HT, that's hard to tell. The code
before
used a heuristic and a lot of #if statements.
How does the code treat an ht device differently at this point?
When it sets the UnitID, it also sets the function numbers of the children, since they are offsets from the UnitID.
The reason I put pci_a here is that otherwise device 0,1 of the 8132 collides with 0,1 of the 8111 (their dts names are the same.)
The reason for specifying the chain number is that devices could be
inserted
in a lot of places, and it is nice to be able to specify which one you
mean.
Think of an 8-way box where you could replace any of the processors with
a
co-processor. It gets ugly in a hurry.
So shouldn't you treat it like any bridge? ( I think this is how bridges should be treated.)
coprocessors aren't necessarily bridges. Mine isn't.
pci@0,1{ /config/bridge dts pci@0,0{ /config/device dts }; };
Since ht bridges are getting special names should pci bridges? pcib@x,y
They don't need them unless you want to override automatic bus numbering.
Another nit I have is that pci_18_0 is the top device. Physically it is but logically it isn't. The dts doesn't make sense when you compare it with the lspci. I don't know if this is really better but maybe something like this:
My next patches will move everything from being a child of 18.0 to being
a
child of the domain. That's assuming I ever get agreement for what I'm doing :) I plan to have it look like this:
domain { /k8/domain ht@0,a ht@0,6 pci@18,0 pci@18,1 pci@18,2 pci@18,3 pci@19,0 ... };
So a ht link off of another cpu will be a new domain? Do you know how the 8151 will be defined in the Serengeti dts?
No. I'm not sure I understand domains yet, but I've been told you can implement multiple domains in a k8 or fam10.
from the lspci:
08:01.0 Host bridge [0600]: Advanced Micro Devices [AMD] AMD-8151 System Controller [1022:7454] (rev 14) 08:02.0 PCI bridge [0604]: Advanced Micro Devices [AMD] AMD-8151 AGP Bridge [1022:7455] (rev 14) 09:00.0 VGA compatible controller [0300]: ATI Technologies Inc RV280 [Radeon 9200] [1002:5961] (rev 01) 09:00.1 Display controller [0380]: ATI Technologies Inc RV280 [Radeon 9200] (Secondary) [1002:5941] (rev 01)
The 8151 will be: ht@1,1 (link 1, UnitID 1) pci_uniquifier@1,0 (8151 AGP bridge)
and the dtsnames will then be: domain_0_ht_1_1 domain_0_pci_uniquifier_1_0
domain_0_pci_18_0_ht_0_6(PCI: 00:06.0): enabled 1 have_resources 0
This will be domain_0_ht_0_6 if I get my way.
great
What if this were a real PCI to PCI bridge? Would it be domain_0_pci_18_0_pci_0_6?
Yes, but there's no such thing with an Opteron, right? It would have to
be
another level down in the hierarchy like domain_0_ht_0_6_pci_6_0
(remember
that HT UnitID and PCI dev are the same, so ht_0_6 becomes pci_6_0.)
I meant if the ht bridge that acts like a pci bridge were a real pci bridge. I think then it is domain_0_pci_18_0_pci_0_6.
? The 8111 pci is a real pci bridge.
domain_0_pci_18_0_ht_0_6_pci_0_0(PCI: 01:00.0): enabled 1have_resources
0
Why isn't this domain_0_pci_18_0_ht_0_6_pci_1_0_0?
Bus numbers get assigned as they're discovered, so they aren't mentioned
in
the dts. That's why the 8111 has to be at 0x6 and the 8132 has to be at 0xa, otherwise the 8132's busses get numbered first, and the
southbridge
ends up on bus 3. Actually the 8111 just has to have a lower number.
0x0
and 0x2 also work.
This is what I noted above. Devices under a bridge need to be in the bridge brackets so you know where they are regardless of bus numbering.
Maybe this is related to my previous misunderstanding?
domain_0_pci_18_0_ht_0_6_pci_0_1(PCI: 01:00.1): enabled 1have_resources
0
...
domain_0_pci_18_0_pci_6_1_0(PCI: 00:07.0): enabled 1
have_resources 0
I really don't understand this one. It is a device on bus 0 so what does 6_1_0 mean?
It's device 1 function 0 of the pci_6 ht device. Remember I had to add
the
_6 so that it wouldn't have the same name as pci_a_1_0. I agree it's confusing. I couldn't think of a really great way to do it. Maybe just taking out the extra underscore would help:
domain_0_pci6_0_1 or naming it explicitly domain_0_pciht6_0_1
I'm open to suggestions.
This still doesn't make sense to me. It is on the silicon that happens to be a HT device but it is just a pci device on bus 0. Think of it in terms of any other motherboard device. The device is always at the same location in the system. This one happens to be set to unitid+1. I think that you still set it in the dts as pci@0,7.
I think you meant pci@7,0. With ht@ it's link, unitid. With pci it's dev, fn (dev == unitid)
If we later have a hierarchical dts so you don't have to specify all these numbers (since they don't change), we'll want them to be offsets.
Here are some of the next steps:
- Get video working.
Do you know where the breakage is?
I think it's in resource allocation. It's really hard to follow,
though.
In the end the 8132 claims the address space that should be for the
video
buffer, and it never gets to the card.
Is the Subtractive or VGA bits set in the 8132 bridge or is the BAR claiming the area?
The BAR seems to be claiming the area.
Thanks, Myles