nested description of chains

[Stefan Reinauer]

Hi,

looking at the big picture again and again, I don't think that
nesting hypertransport devices when doing the chain description is a
very good idea as it gives us some limitations.

* CPUs (North bridges) don't fit in the scenario, since they 
  might be arranged in a circular chain:

CPU-3 --- CPU-2
    |         |
    |         |
  CPU-0 --- CPU-1

If trying to pack this into the nested description manner as it
  is now, we can't describe the fact that it's a circular organization:

northbridge amd/amdk8 "mc0"
    pci 0:18.0
    [..]
    northbridge amd/amdk8 "mc1"
      pci 0:19.0
      northbridge amd/amdk8 "mc2"
        pci 0:1a.0
        northbridge amd/amdk8 "mc3"
          [..] (nesting ad nauseum)
        end
      end
    end
    northbridge amd/amdk8 "mc3"
      [..] (nesting ad nauseum)
    end
  end

* Non-CPU components can be described as long as they are not ring
  organized, but for Hypertransport devices the description tries 
  to hide the fact that the organization form is a chain.

Adding a nested description for components like a superio chip makes
  perfect sense, since it's unlikely that superio chips are ever
  cascaded or put into a circular chain. Also, the connection between
  the I/O hub (southbridge) and the superio can be seen as a
  peer-to-peer connection rather than a bus that allows arbitrary links
  between all components.

The reason why I am writing this is that currently we have a hardcoded
assumption in in LinuxBIOS' coherent hypertransport setup that looks
like this:

|                 |
    [2]               [2]
   CPU-3 [1] --- [1] CPU-2
    [0]               [0]
     |                 |
    [2]               [2]
   CPU-0 [1] --- [1] CPU-1
    [0]               [0] 
     |                 |

[x] means that HT link no x is used for a certain link.

But this is not the only available organization method for a
hypertransport setup. Assumed a CPU HT device looks like this:

| [LDT2] 
   ___|__
  |      |
  | CPU0 |-- [LDT1]
  |______|
      |
      | [LDT0]

A hypertransport setup like this is a little more complex, but
still perfectly valid:
   _______________         
   |    ______   |
   |   | 8111 |  |
   |   |______|  |
   |    ___|__   |
   |   |      |  |
   |   | CPU0 |__|
   |   |______|
   | ______|
   | |     ________________
   | |  ___|__    ______  |
   | | |      |  | 8131 | |
   | | | CPU1 |--|______| |
   | | |______|           |
   | |_____|              |
   |_______               |
        ___|__            |
       |      |           |
       | CPU2 |--[term]   |
       |______|           |
   ________|              |
  |                       |
  |     [term]            |
  |    ___|__             |
  |   |      |            |
  |   | CPU3 |-------------
  |   |______|
  |______|

I'll look at getting this scenario supported in the coherent_ht setup.
But it would be nice to see configurations like this happen without
actually touching the code when working a new opteron based port
somewhen in future.

Comments? Flames?

Stefan

[ron minnich]

On Fri, 12 Sep 2003, Stefan Reinauer wrote:
> looking at the big picture again and again, I don't think that
> nesting hypertransport devices when doing the chain description is a
> very good idea as it gives us some limitations.

you mean nesting them in the config file? I agree. It is not a good idea.

The config language only supports a parent-child relationship, and HT has 
a much richer interconnection structure.

Are you saying that we should reflect this richer topology that HT has in
the config language?

ron

[Stefan Reinauer]

* ron minnich  [030912 15:06]:
> The config language only supports a parent-child relationship, and HT has 
> a much richer interconnection structure.

Yes. Currently the config tool only allows an acyclic graph for
describing hardware. this does not suit the machine description that
would be needed for hypertransport.
> Are you saying that we should reflect this richer topology that HT has in
> the config language?

Yes, and I think this does not need to be really complex. See the below
example. Then the romimage class could be extended to write a new file
htlinks.h in case it detects "htlink" keywords in any device. I'm
currently fighting myself a bit through config.g to see how this can be
done.

I don't think this necessarily should go to the static device tree,
since we would have to use the device tree in romcc code then, which
is not yet possible iirc. (And might bloat romcc code up)

typedef struct htlink
{
        int direction; // 0,1,2 for LDT0,1,2
        int speed;
        int width;
        int nodenum;  // ht device number of remote component
} htlink_t;

enum {
NORTHBRIDGE,
SOUTHBRIDGE,
PCIBRIDGE,
} htdev_t;

typedev struct htdev
{
    htdev_t type; // type
    int num;  // instance, i.e. the num'th device of type "type"
    htlink_t links[3];
} htdev_t;

htdevs_t **get_htdevs() 
{
        htdevt_t htdevs[]={
          { NORTHBRIDGE, 0, {
              { 0, 800, 16, 1 },
              { 1, 800, 16, 3 },
              { 2, 800, 16, 4 }, // range 0-3 used by cpus
            }
          },
          { NORTHBRIDGE, 1, {
              { 0, 800, 16, 0 },
              { 1, 800, 16, 5 },
              { 2, 800, 16, 3 },
            }
          },
        [..]
        };

return htdevs;
}

generated from a config file like this:

northbridge amd/amdk8 "mc0"
        pci 0:18.0
        pci 0:18.0
        pci 0:18.0
        pci 0:18.1
        pci 0:18.2
        pci 0:18.3
        htlink 0 "mc1" [ speed=default|200|400.. ] [ width=default|8|16 ]
        htlink 1 "mc3" speed=default width=16
        htlink 2 "amd8111" speed=600 width=8
end

northbridge amd/amdk8 "mc1"
        pci 0:19.0
        pci 0:19.0
        pci 0:19.0
        pci 0:19.1
        pci 0:19.2
        pci 0:19.3
        htlink 0 "mc0"
        htlink 1 "amd8131" 
        htlink 2 "mc3" 
end

northbridge amd/amdk8 "mc2"
        pci 0:1a.0
        pci 0:1a.0
        pci 0:1a.0
        pci 0:1a.1
        pci 0:1a.2
        pci 0:1a.3
        htlink 0 "mc3"
        htlink 2 "mc0" 
end

northbridge amd/amdk8 "mc3"
        pci 0:1b.0
        pci 0:1b.0
        pci 0:1b.0
        pci 0:1b.1
        pci 0:1b.2
        pci 0:1b.3
        htlink 0 "mc2"
        htlink 1 "mc1" 
end

southbridge amd/amd8131 "amd8131-1"
        pci 0:0.0
        pci 0:0.1
        pci 0:1.0
        pci 0:1.1
        htlink 0 "mc0" 
        htlink 1 "amd8131-2"
end

southbridge amd/amd8131 "amd8131-2"
        pci 0:0.0
        pci 0:0.1
        pci 0:1.0
        pci 0:1.1
        htlink 0 "amd8131-1" 
end

southbridge amd/amd8111 "amd8111"
        pci 0:0.0
        pci 0:1.0
        pci 0:1.1
        pci 0:1.2
        pci 0:1.3
        pci 0:1.5
        pci 0:1.6
        superio NSC/pc87360
                pnp 1:2e.0
                pnp 1:2e.1
                pnp 1:2e.2
                pnp 1:2e.3
                pnp 1:2e.4
                pnp 1:2e.5
                pnp 1:2e.6
                pnp 1:2e.7
                pnp 1:2e.8
                pnp 1:2e.9
                pnp 1:2e.a
                register "com1" = "{1, 0, 0x3f8, 4}"
                register "lpt" = "{1}"
        end
        htlink 0 "mc0" speed=200 width=8
end

Stefan

[Greg Watson]

Stefan,

This should be a relatively straight forward change to config.g. I 
would prefer to see the config format maintained, so I would suggest 
using:

htlink 0 "mc1"
         speed = 200
         width = 8
    end

htlink 0 "amd8131-1"  end

rather than

htlink 0 "mc1"  speed=200 width=8
     htlink 0 "amd8131-1"

Let me know if you need any help in implementing it.

Regards,

Greg

At 4:57 PM +0200 12/9/03, Stefan Reinauer wrote:
> ron minnich  [030912 15:06]:
> > The config language only supports a parent-child relationship, and HT has
> >  a much richer interconnection structure.
>
> Yes. Currently the config tool only allows an acyclic graph for
> describing hardware. this does not suit the machine description that
> would be needed for hypertransport.
> > Are you saying that we should reflect this richer topology that HT has in
> >  the config language?
>
> Yes, and I think this does not need to be really complex. See the below
> example. Then the romimage class could be extended to write a new file
> htlinks.h in case it detects "htlink" keywords in any device. I'm
> currently fighting myself a bit through config.g to see how this can be
> done.
>
> I don't think this necessarily should go to the static device tree,
> since we would have to use the device tree in romcc code then, which
> is not yet possible iirc. (And might bloat romcc code up)
>
> typedef struct htlink
> {
>         int direction; // 0,1,2 for LDT0,1,2
>         int speed;
>         int width;
>         int nodenum;  // ht device number of remote component
> } htlink_t;
>
> enum {
> NORTHBRIDGE,
> SOUTHBRIDGE,
> PCIBRIDGE,
> } htdev_t;
>
> typedev struct htdev
> {
>     htdev_t type; // type
>     int num;  // instance, i.e. the num'th device of type "type"
>     htlink_t links[3];
> } htdev_t;
>
> htdevs_t **get_htdevs()
> {
>         htdevt_t htdevs[]={
>           { NORTHBRIDGE, 0, {
>               { 0, 800, 16, 1 },
>               { 1, 800, 16, 3 },
>               { 2, 800, 16, 4 }, // range 0-3 used by cpus
>             }
>           },
>           { NORTHBRIDGE, 1, {
>               { 0, 800, 16, 0 },
>               { 1, 800, 16, 5 },
>               { 2, 800, 16, 3 },
>             }
>           },
>         [..]
>         };
>     return htdevs;
>
> }
>
> generated from a config file like this:
>
> northbridge amd/amdk8 "mc0"
>         pci 0:18.0
>         pci 0:18.0
>         pci 0:18.0
>         pci 0:18.1
>         pci 0:18.2
>         pci 0:18.3
>         htlink 0 "mc1" [ speed=default|200|400.. ] [ width=default|8|16 ]
>         htlink 1 "mc3" speed=default width=16
>         htlink 2 "amd8111" speed=600 width=8
> end
>
> northbridge amd/amdk8 "mc1"
>         pci 0:19.0
>         pci 0:19.0
>         pci 0:19.0
>         pci 0:19.1
>         pci 0:19.2
>         pci 0:19.3
>         htlink 0 "mc0"
>         htlink 1 "amd8131"
>         htlink 2 "mc3"
> end
>
> northbridge amd/amdk8 "mc2"
>         pci 0:1a.0
>         pci 0:1a.0
>         pci 0:1a.0
>         pci 0:1a.1
>         pci 0:1a.2
>         pci 0:1a.3
>         htlink 0 "mc3"
>         htlink 2 "mc0"
> end
>
> northbridge amd/amdk8 "mc3"
>         pci 0:1b.0
>         pci 0:1b.0
>         pci 0:1b.0
>         pci 0:1b.1
>         pci 0:1b.2
>         pci 0:1b.3
>         htlink 0 "mc2"
>         htlink 1 "mc1"
> end
>
> southbridge amd/amd8131 "amd8131-1"
>         pci 0:0.0
>         pci 0:0.1
>         pci 0:1.0
>         pci 0:1.1
>         htlink 0 "mc0"
>         htlink 1 "amd8131-2"
> end
>
> southbridge amd/amd8131 "amd8131-2"
>         pci 0:0.0
>         pci 0:0.1
>         pci 0:1.0
>         pci 0:1.1
>         htlink 0 "amd8131-1"
> end
>
> southbridge amd/amd8111 "amd8111"
>         pci 0:0.0
>         pci 0:1.0
>         pci 0:1.1
>         pci 0:1.2
>         pci 0:1.3
>         pci 0:1.5
>         pci 0:1.6
>         superio NSC/pc87360
>                 pnp 1:2e.0
>                 pnp 1:2e.1
>                 pnp 1:2e.2
>                 pnp 1:2e.3
>                 pnp 1:2e.4
>                 pnp 1:2e.5
>                 pnp 1:2e.6
>                 pnp 1:2e.7
>                 pnp 1:2e.8
>                 pnp 1:2e.9
>                 pnp 1:2e.a
>                 register "com1" = "{1, 0, 0x3f8, 4}"
>                 register "lpt" = "{1}"
>         end
>         htlink 0 "mc0" speed=200 width=8
> end
>
> Stefan