PCI domains

[Myles Watson]

I'm unclear on the concept of PCI domains.  I've done a Google search and
searched the books that I have (PCIe & HT).

The only place I find domains mentioned is in lspci documentation.

Has anyone ever seen an lspci with domains set to anything but 0?

Thanks,
Myles

[Carl-Daniel Hailfinger]

On 03.11.2008 17:30, Myles Watson wrote:
> I'm unclear on the concept of PCI domains.  I've done a Google search and
> searched the books that I have (PCIe & HT).
>
> The only place I find domains mentioned is in lspci documentation.
>
> Has anyone ever seen an lspci with domains set to anything but 0?

http://marc.info/?l=linux-kernel&m=109809009130342&w=4

Regards,
Carl-Daniel

[Myles Watson]

On Mon, Nov 3, 2008 at 9:51 AM, Carl-Daniel Hailfinger <
[e-mail]> wrote:
> On 03.11.2008 17:30, Myles Watson wrote:
> > I'm unclear on the concept of PCI domains.  I've done a Google search and
> > searched the books that I have (PCIe & HT).
> >
> > The only place I find domains mentioned is in lspci documentation.
> >
> > Has anyone ever seen an lspci with domains set to anything but 0?
>
> http://marc.info/?l=linux-kernel&m=109809009130342&w=4

Thanks!

So can we be safe in assuming that IBM Power machines and SGI Altix machines
have domains, but x86 don't?

Myle

[yhlu]

On Mon, Nov 3, 2008 at 11:37 AM, Myles Watson  wrote:
> On Mon, Nov 3, 2008 at 9:51 AM, Carl-Daniel Hailfinger <
> [e-mail]> wrote:
> > On 03.11.2008 17:30, Myles Watson wrote:
> > > I'm unclear on the concept of PCI domains.  I've done a Google search
> >
> > and
> > > searched the books that I have (PCIe & HT).
> > >
> > > The only place I find domains mentioned is in lspci documentation.
> > >
> > > Has anyone ever seen an lspci with domains set to anything but 0?
> >
> > http://marc.info/?l=linux-kernel&m=109809009130342&w=4
>
> Thanks!
>
> So can we be safe in assuming that IBM Power machines and SGI Altix
> machines have domains, but x86 don't?

one HP's AMD based workstation (like tyan s2895) has that ... one ht chain
==> one pci domain.
you could enable that in BIOS setup. (HP's own BIOS)

YH

[ron minnich]

On Mon, Nov 3, 2008 at 11:46 AM, yhlu  wrote:
> one HP's AMD based workstation (like tyan s2895) has that ... one ht chain
> ==> one pci domain.
> you could enable that in BIOS setup. (HP's own BIOS)

Is there any benefit to it?

ron

[yhlu]

On Mon, Nov 3, 2008 at 11:52 AM, ron minnich  wrote:
> On Mon, Nov 3, 2008 at 11:46 AM, yhlu  wrote:
> > one HP's AMD based workstation (like tyan s2895) has that ... one ht
>
> chain
> > ==> one pci domain.
> > you could enable that in BIOS setup. (HP's own BIOS)
>
> Is there any benefit to it?

more than 255 pci buses. think about if you have 32 ht chains...

amd Family 10h cpu could support 16 domains ( 4 bits for domain...)

YH

[Myles Watson]

On Mon, Nov 3, 2008 at 2:44 PM, yhlu  wrote:
> On Mon, Nov 3, 2008 at 11:52 AM, ron minnich  wrote:
> > On Mon, Nov 3, 2008 at 11:46 AM, yhlu  wrote:
> > > one HP's AMD based workstation (like tyan s2895) has that ... one ht
> >
> > chain
> > > ==> one pci domain.
> > > you could enable that in BIOS setup. (HP's own BIOS)
> >
> > Is there any benefit to it?
>
> more than 255 pci buses. think about if you have 32 ht chains...

Sorry to be dense, but how do you have more than 4 HT chains?  Aren't there
are only 8 nodes allowed in a system (3 links per node)?

If you took out the middle links in an 8-way design I guess you could have 8
HT chains.

And I guess if you made it into a chain you could have 10.

I'm obviously missing something, because it seems silly.
> amd Family 10h cpu could support 16 domains ( 4 bits for domain...)
>
> YH

Could you point me to a section of a BKDG?  I found something about
protection domains, but it looked like it was only for DMA.

I don't see how our current PCI functions could use domains.

Thanks,
Myles

[yhlu]

On Mon, Nov 3, 2008 at 2:13 PM, Myles Watson  wrote:
> On Mon, Nov 3, 2008 at 2:44 PM, yhlu  wrote:
> > On Mon, Nov 3, 2008 at 11:52 AM, ron minnich  wrote:
> > > On Mon, Nov 3, 2008 at 11:46 AM, yhlu  wrote:
> > > > one HP's AMD based workstation (like tyan s2895) has that ... one ht
> > >
> > > chain
> > > > ==> one pci domain.
> > > > you could enable that in BIOS setup. (HP's own BIOS)
> > >
> > > Is there any benefit to it?
> >
> > more than 255 pci buses. think about if you have 32 ht chains...
>
> Sorry to be dense, but how do you have more than 4 HT chains?  Aren't there
> are only 8 nodes allowed in a system (3 links per node)?
>
> If you took out the middle links in an 8-way design I guess you could have
> 8 HT chains.

for amd family 10h (HT3 ?) will have 8 sublinks...so you could build system
with more nodes and more ht chains

YH

[ron minnich]

Where do the domain bits live in the chipset?

ron

[yhlu]

On Mon, Nov 3, 2008 at 5:00 PM, ron minnich  wrote:
> Where do the domain bits live in the chipset?

MSR

they need to domain_nr + bus_nr to dicide how to direct io range...

YH

[ron minnich]

On Mon, Nov 3, 2008 at 7:43 PM, yhlu  wrote:
> On Mon, Nov 3, 2008 at 5:00 PM, ron minnich  wrote:
> > Where do the domain bits live in the chipset?
>
> MSR
>
> they need to domain_nr + bus_nr to dicide how to direct io range...

thanks
ron