I need to clear up my understanding of resource allocation. Here's my understanding of it:
1. Start at the root with 2 resources (1 for memory, 1 for IO) 2. For each child of each link: A. Add resources B. Get common constraints C. Descend links D. Propagate back to root
When you're done you should have the Root device with the superset of all devices' resources.
My question is if resources ever split. In other words, lets imagine a device which implements several IO ports in the 0x3fX range and larger regions that must be mapped above 0x1000. Since the limit for the smaller regions is 0x7ff and the alignment for the larger regions is 0x1000, they both can't coexist.
What happens?
Thanks, Myles
----- Original Message ----
From: Myles Watson mylesgw@gmail.com To: Coreboot coreboot@coreboot.org Sent: Monday, November 10, 2008 2:53:57 PM Subject: [coreboot] Resource allocation
I need to clear up my understanding of resource allocation. Here's my understanding of it:
- Start at the root with 2 resources (1 for memory, 1 for IO)
- For each child of each link: A. Add resources B. Get common constraints C. Descend links D. Propagate back to root
When you're done you should have the Root device with the superset of all devices' resources.
Yes, and each bridge needs the resources grouped which is why the links(bridges) are desended as found.
My question is if resources ever split. In other words, lets imagine a device which implements several IO ports in the 0x3fX range and larger regions that must be mapped above 0x1000. Since the limit for the smaller regions is 0x7ff and the alignment for the larger regions is 0x1000, they both can't coexist.
What happens?
Everything below 0x1000 is legacy IO space and are not typically used by PCI devices (some devices/firmware sneek some stuff in around 0x800 for ACPI). As you noted there are some devices that might need to decode legacy regions to boot. If that is the case they will usually have a legacy enable bit in the header. In legacy mode the PCI BARs are ignored. The firmware, option ROM, and driver need to be aware of which mode it is in. An example is the AMD sb600 IDE controller(Device 20, Function 1). There is a legacy/PCI mode in the revision register ot offset 0x8.
http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/4615...
Marc
-----Original Message----- From: coreboot-bounces@coreboot.org [mailto:coreboot-bounces@coreboot.org] On Behalf Of Marc Jones Sent: Monday, November 10, 2008 3:28 PM To: Coreboot Subject: Re: [coreboot] Resource allocation
----- Original Message ----
From: Myles Watson mylesgw@gmail.com To: Coreboot coreboot@coreboot.org Sent: Monday, November 10, 2008 2:53:57 PM Subject: [coreboot] Resource allocation
I need to clear up my understanding of resource allocation. Here's my understanding of it:
- Start at the root with 2 resources (1 for memory, 1 for IO)
- For each child of each link: A. Add resources B. Get common constraints C. Descend links D. Propagate back to root
When you're done you should have the Root device with the superset of
all
devices' resources.
Yes, and each bridge needs the resources grouped which is why the links(bridges) are desended as found.
My question is if resources ever split. In other words, lets imagine a device which implements several IO ports in the 0x3fX range and larger regions that must be mapped above 0x1000. Since the limit for the
smaller
regions is 0x7ff and the alignment for the larger regions is 0x1000,
they
both can't coexist.
What happens?
Everything below 0x1000 is legacy IO space and are not typically used by PCI devices (some devices/firmware sneek some stuff in around 0x800 for ACPI). As you noted there are some devices that might need to decode legacy regions to boot. If that is the case they will usually have a legacy enable bit in the header. In legacy mode the PCI BARs are ignored. The firmware, option ROM, and driver need to be aware of which mode it is in. An example is the AMD sb600 IDE controller(Device 20, Function 1). There is a legacy/PCI mode in the revision register ot offset 0x8.
Thanks Marc.
All right. So the legacy IO resources shouldn't be grouped with the others. Should they be marked Subtractive? Are they declared to be special in a different way?
Thanks, Myles
----- Original Message ----
From: Myles Watson mylesgw@gmail.com To: Marc Jones marcj303@yahoo.com; Coreboot coreboot@coreboot.org; ron minnich rminnich@gmail.com Sent: Tuesday, November 11, 2008 7:42:57 AM Subject: RE: [coreboot] Resource allocation
My question is if resources ever split. In other words, lets imagine a device which implements several IO ports in the 0x3fX range and larger regions that must be mapped above 0x1000. Since the limit for the
smaller
regions is 0x7ff and the alignment for the larger regions is 0x1000,
they
both can't coexist.
What happens?
Everything below 0x1000 is legacy IO space and are not typically used by PCI devices (some devices/firmware sneek some stuff in around 0x800 for ACPI). As you noted there are some devices that might need to decode legacy regions to boot. If that is the case they will usually have a legacy enable bit in the header. In legacy mode the PCI BARs are ignored. The firmware, option ROM, and driver need to be aware of which mode it is in. An example is the AMD sb600 IDE controller(Device 20, Function 1). There is a legacy/PCI mode in the revision register ot offset 0x8.
Thanks Marc.
All right. So the legacy IO resources shouldn't be grouped with the others. Should they be marked Subtractive? Are they declared to be special in a different way?
Yes but those legacy addresses could be positively decoded anywhere along the subtractive path. They will usually go as far as the chipset with an integrated controller. So, if someone put a legacy IDE controller farther on the path it (like lpc) it would never get the IO. For completeness in the dts you could have all the addresses in the system but I don't know if it is that important. I guess we need to add legacy io reserve code to the chipsets similar to the SIOs.
Note that the exception to the subtractive decode is the VGA snoop and VGA enable (just for legacy VGA registes 0x3Cx -0x3Dx I think).
Marc
-----Original Message----- From: Marc Jones [mailto:marcj303@yahoo.com] Sent: Tuesday, November 11, 2008 10:12 AM To: Myles Watson; Coreboot; ron minnich Subject: Re: [coreboot] Resource allocation
----- Original Message ----
From: Myles Watson mylesgw@gmail.com To: Marc Jones marcj303@yahoo.com; Coreboot coreboot@coreboot.org;
ron minnich rminnich@gmail.com
Sent: Tuesday, November 11, 2008 7:42:57 AM Subject: RE: [coreboot] Resource allocation
My question is if resources ever split. In other words, lets
imagine a
device which implements several IO ports in the 0x3fX range and
larger
regions that must be mapped above 0x1000. Since the limit for the
smaller
regions is 0x7ff and the alignment for the larger regions is 0x1000,
they
both can't coexist.
What happens?
Everything below 0x1000 is legacy IO space and are not typically used by PCI devices (some devices/firmware sneek some stuff in around 0x800 for ACPI). As you noted there are some devices that might need to decode legacy regions to boot. If that is the case they will usually have a legacy enable bit in the header. In legacy mode the PCI BARs are ignored. The firmware, option ROM, and driver need to be aware of
which
mode it is in. An example is the AMD sb600 IDE controller(Device 20, Function 1). There is a legacy/PCI mode in the revision register ot offset 0x8.
Thanks Marc.
All right. So the legacy IO resources shouldn't be grouped with the
others.
Should they be marked Subtractive? Are they declared to be special in a different way?
Yes but those legacy addresses could be positively decoded anywhere along the subtractive path. They will usually go as far as the chipset with an integrated controller. So, if someone put a legacy IDE controller farther on the path it (like lpc) it would never get the IO. For completeness in the dts you could have all the addresses in the system but I don't know if it is that important. I guess we need to add legacy io reserve code to the chipsets similar to the SIOs.
Thanks for the help; sorry to be dense. Could we flesh this out a little more?
The problem I'm having is that the SuperIO is declaring several resources, some of which are below the 0x1000 threshold, and some of which are not. They are all declared to be IO, so they need to fit in the same bridge resource, and can't. If they were a different type, or if they were ignored by the resource code, it could still work.
I understand that the subtractive decode and VGA bits need to be set on the bridge (amd8111), but I don't know how we protect the legacy IDE area if someone added an IDE card to the PCI bus of the amd8111.
Note that the exception to the subtractive decode is the VGA snoop and VGA enable (just for legacy VGA registes 0x3Cx -0x3Dx I think).
OK.
Thanks, Myles
----- Original Message ----
From: Myles Watson mylesgw@gmail.com To: Marc Jones marcj303@yahoo.com; Coreboot coreboot@coreboot.org; ron minnich rminnich@gmail.com Sent: Tuesday, November 11, 2008 10:33:47 AM Subject: RE: [coreboot] Resource allocation
All right. So the legacy IO resources shouldn't be grouped with the
others.
Should they be marked Subtractive? Are they declared to be special in a different way?
Yes but those legacy addresses could be positively decoded anywhere along the subtractive path. They will usually go as far as the chipset with an integrated controller. So, if someone put a legacy IDE controller farther on the path it (like lpc) it would never get the IO. For completeness in the dts you could have all the addresses in the system but I don't know if it is that important. I guess we need to add legacy io reserve code to the chipsets similar to the SIOs.
Thanks for the help; sorry to be dense. Could we flesh this out a little more?
The problem I'm having is that the SuperIO is declaring several resources, some of which are below the 0x1000 threshold, and some of which are not. They are all declared to be IO, so they need to fit in the same bridge resource, and can't. If they were a different type, or if they were ignored by the resource code, it could still work.
I understand that the subtractive decode and VGA bits need to be set on the bridge (amd8111), but I don't know how we protect the legacy IDE area if someone added an IDE card to the PCI bus of the amd8111.
Legacy can be completely ignored by the bridges. As long as the device is on the subtractive path(which an sio should be) it will get the cycles so those cycles can also be ignored by bridges. The question is how they are handled in the dts. Do we care to track those addresses there? I think so since that is the point. So, PCI bridges ignore legacy/subtractive decode io ranges (and technically memory ranges as well) and the dts understands them, maybe check for an overlap and prints warning. No other action is needed. In the case of IDE. If a card is added in front of the 8111 and positivly decodes the legacy range, the 8111 will never get the cycles. This is not a problem for coreboot as there isn't anything we can do about it. This is very very unlikely. Most PCI addin cards don't decode legacy ranges for this reason. They assume tha there is a legacy device on the subtractive bus and do everything in PCI native mode.
Marc
Marc Jones wrote:
Legacy can be completely ignored by the bridges. As long as the device is on the subtractive path(which an sio should be) it will get the cycles so those cycles can also be ignored by bridges. The question is how they are handled in the dts. Do we care to track those addresses there? I think so since that is the point.
We must - so that we do not allocate conflicting resources to other devices.
We might also want to make sure that we never change the subtractive decoding. Maybe hardware doesn't even allow that?
So, PCI bridges ignore legacy/subtractive decode io ranges (and technically memory ranges as well) and the dts understands them, maybe check for an overlap and prints warning. No other action is needed.
Again, we should exclude them from available resources so that the allocator algorithm can not use them, even if by bug.
In the case of IDE. If a card is added in front of the 8111 and positivly decodes the legacy range, the 8111 will never get the cycles. This is not a problem for coreboot as there isn't anything we can do about it. This is very very unlikely. Most PCI addin cards don't decode legacy ranges for this reason.
But they could still be configured to do so, right? Would such an IDE card not accept config cycles?
They assume tha there is a legacy device on the subtractive bus and do everything in PCI native mode.
At least something moves away from legacy. :)
//Peter
-----Original Message----- From: coreboot-bounces@coreboot.org [mailto:coreboot-bounces@coreboot.org] On Behalf Of Peter Stuge Sent: Tuesday, November 11, 2008 11:12 AM To: Coreboot Subject: Re: [coreboot] Resource allocation
Marc Jones wrote:
Legacy can be completely ignored by the bridges. As long as the device is on the subtractive path(which an sio should be) it will get the cycles so those cycles can also be ignored by bridges. The question is how they are handled in the dts. Do we care to track those addresses there? I think so since that is the point.
We must - so that we do not allocate conflicting resources to other devices.
We might also want to make sure that we never change the subtractive decoding. Maybe hardware doesn't even allow that?
So, PCI bridges ignore legacy/subtractive decode io ranges (and technically memory ranges as well) and the dts understands them, maybe check for an overlap and prints warning. No other action is needed.
Again, we should exclude them from available resources so that the allocator algorithm can not use them, even if by bug.
In the case of IDE. If a card is added in front of the 8111 and positivly decodes the legacy range, the 8111 will never get the cycles. This is not a problem for coreboot as there isn't anything we can do about it. This is very very unlikely. Most PCI addin cards don't decode legacy ranges for this reason.
But they could still be configured to do so, right? Would such an IDE card not accept config cycles?
My understanding is that their BARs would be too large to allow them to be allocated at 0x3xx.
Can we try a concrete example?
Opteron -> 8111 -> 8111lpc -> Winbond SuperIO
The SuperIO has resources like this: PNP 2e.0: size 8 align 3 gran 3 limit 7ff flags 100 index 60 - IO resource that has to be allocated below 7ff PNP 2e.5: size 1 align 0 gran 0 limit ffffffff flags 60000100 index 60 - IO resource that has to be allocated below ffffffff
Then does the 8111lpc need to have an IO subtractive resource? the 8111? the domain?
Are either of them supposed to be positively decoded by the bridge, or should we let them both live under 7ff? What code takes care of this? What code makes sure that these resources don't conflict?
Thanks, Myles
On Tue, Nov 11, 2008 at 11:20 AM, Myles Watson mylesgw@gmail.com wrote:
Opteron -> 8111 -> 8111lpc -> Winbond SuperIO
The SuperIO has resources like this: PNP 2e.0: size 8 align 3 gran 3 limit 7ff flags 100 index 60 - IO resource that has to be allocated below 7ff PNP 2e.5: size 1 align 0 gran 0 limit ffffffff flags 60000100 index 60 - IO resource that has to be allocated below ffffffff
Then does the 8111lpc need to have an IO subtractive resource? the 8111? the domain?
Well what appears to happen on more modern k8 hardware (and I am pretty sure this did not happen in the early days, but who knows -- we did the resource map on older boards so this would all work).
The legacy chain will "just work" such that this device can be accessed. In other words, some set of hardware will work correctly so that, e.g., FLASH can be read. Since this part IIRC is also on the chain (?) it is accessible too.
Are either of them supposed to be positively decoded by the bridge, or should we let them both live under 7ff? What code takes care of this? What code makes sure that these resources don't conflict?
From memory, on the old boards, the very early resource map setup in
v2 auto.c did this. Things were set up just enough that the basic legacy devices worked. Note that config space doesn't change later -- just some basic routing. So if the subtractive path doesn't get messed up, and no other resource positively decodes this, we're ok.
Let's not bring the IOINDEX_SUBTRACTIVE resources into this discussion -- I did that already and it got me into trouble. The SUBTRACTIVE name in the resource code is misleading IMHO.
ron
-----Original Message----- From: ron minnich [mailto:rminnich@gmail.com] Sent: Tuesday, November 11, 2008 12:45 PM To: Myles Watson Cc: Coreboot Subject: Re: [coreboot] Resource allocation
On Tue, Nov 11, 2008 at 11:20 AM, Myles Watson mylesgw@gmail.com wrote:
Opteron -> 8111 -> 8111lpc -> Winbond SuperIO
The SuperIO has resources like this: PNP 2e.0: size 8 align 3 gran 3 limit 7ff flags 100 index 60 - IO resource that has to be allocated below 7ff PNP 2e.5: size 1 align 0 gran 0 limit ffffffff flags 60000100
index
60 - IO resource that has to be allocated below ffffffff
Then does the 8111lpc need to have an IO subtractive resource? the 8111?
the
domain?
Well what appears to happen on more modern k8 hardware (and I am pretty sure this did not happen in the early days, but who knows -- we did the resource map on older boards so this would all work).
The legacy chain will "just work" such that this device can be accessed. In other words, some set of hardware will work correctly so that, e.g., FLASH can be read. Since this part IIRC is also on the chain (?) it is accessible too.
Yes. The flash just works. I'm having trouble with PNP devices.
Are either of them supposed to be positively decoded by the bridge, or should we let them both live under 7ff? What code takes care of this?
What
code makes sure that these resources don't conflict?
From memory, on the old boards, the very early resource map setup in v2 auto.c did this. Things were set up just enough that the basic legacy devices worked.
So the parallel port, floppy, etc should be enabled in stage1? Otherwise their addresses will be set later.
Note that config space doesn't change later -- just some basic routing. So if the subtractive path doesn't get messed up, and no other resource positively decodes this, we're ok.
This seems to work fine. The resources that need to be set up to be below 7ff don't work.
Let's not bring the IOINDEX_SUBTRACTIVE resources into this discussion -- I did that already and it got me into trouble. The SUBTRACTIVE name in the resource code is misleading IMHO.
I'm sorry. I realize that I'm trying to go too many directions at once and I can't keep it all on the top of the stack. It's too much complexity.
Does anyone have an idea why resource allocation isn't working? My talking around the idea hasn't helped me yet.
Thanks, Myles
In theory split resources can happen. In practice it never has. The only real split is legacy IO/mem/vga. They are always supported on every chipset to date by a 'decode legacy range' bit.
So, in practice, the extensive capabilities of the resource stuff have not, as far as I know, every really been used. It always comes down to'contiguous memory range' 'contiguous prefmem range' 'contiguous IO range'
ron
-----Original Message----- From: ron minnich [mailto:rminnich@gmail.com] Sent: Monday, November 10, 2008 3:48 PM To: Myles Watson Cc: Coreboot Subject: Re: [coreboot] Resource allocation
In theory split resources can happen. In practice it never has. The only real split is legacy IO/mem/vga. They are always supported on every chipset to date by a 'decode legacy range' bit.
So, in practice, the extensive capabilities of the resource stuff have not, as far as I know, every really been used. It always comes down to'contiguous memory range' 'contiguous prefmem range' 'contiguous IO range'
Thanks Ron. After reading your answer I thought about the fact that there is only one BAR on a bridge for each type of resource, and the split resource idea didn't make as much sense.
Myles
On Tue, Nov 11, 2008 at 6:44 AM, Myles Watson mylesgw@gmail.com wrote:
Thanks Ron. After reading your answer I thought about the fact that there is only one BAR on a bridge for each type of resource, and the split resource idea didn't make as much sense.
yes, and thinking about it ... there's only one bridge anywhere in the system that is going to decode legacy IO, or mem, or vsa (not always same bridge).
I wonder if it would be simpler to have a property to mark these bridges in the dts. It can not be determined -- code has to know that a given piece of hardware is decoding the legacy range.
ron
ron minnich wrote:
yes, and thinking about it ... there's only one bridge anywhere in the system that is going to decode legacy IO, or mem, or vsa (not always same bridge).
I wonder if it would be simpler to have a property to mark these bridges in the dts. It can not be determined -- code has to know that a given piece of hardware is decoding the legacy range.
It adds constraints to resource allocation so it has to be known. And it is property of hardware so it goes into the dts. :)
I think that makes sense!
//Peter
-----Original Message----- From: coreboot-bounces@coreboot.org [mailto:coreboot-bounces@coreboot.org] On Behalf Of Peter Stuge Sent: Tuesday, November 11, 2008 9:58 AM To: Coreboot Subject: Re: [coreboot] Resource allocation
ron minnich wrote:
yes, and thinking about it ... there's only one bridge anywhere in the system that is going to decode legacy IO, or mem, or vsa (not always same bridge).
I wonder if it would be simpler to have a property to mark these bridges in the dts. It can not be determined -- code has to know that a given piece of hardware is decoding the legacy range.
It adds constraints to resource allocation so it has to be known. And it is property of hardware so it goes into the dts. :)
I think that makes sense!
I agree except that you still have to have the resources (they have to be declared by the device that implements them), so you put it in two places.
Thanks, Myles
-
Marc Jones -
Myles Watson -
Peter Stuge -
ron minnich