static and dynamic devices (again)

[ron minnich]

Greg has spent the day exploring options for how to merge the static and
dynamic device trees in LinuxBIOS, per Eric's comments. He wrote a
substantial amount of code, which at this point we may not commit. The
goal of the code was to see if it makes sense to merge the static and
dynamic trees. I am not convinced such a merger makes sense.

Discussion follows.

The problem is this: on a given motherboard, there are a static set of 
devices, such as northbridges, southbridges, so on: these are soldered to 
the board. Then there are dynamic devices, plugged into PCI slots. 

On a motherboard, we know that the static devices are there, and we even
know a lot about them, such as what certain registers do.  We need on
the motherboard to initialize components of the static devices. In other
words, we have knowledge about:
o the type of the device; 
o instances ofthe device; 
o special operations we need to do to the device to modify internal 
  state (e.g. turn off the IDE controller, turn on Ethernet, etc.)
o special control information that is highly device-specific

These operations are operations that we have to do because the OS assumes
they will be finished when the OS starts, or we need to do them to
initialize things, such as DRAM.

Dynamic devices are known only in a generic fashion. In LinuxBIOS, the sum
total of our knowledge of dynamic devices is the Base Address Register
set, the Command register, and possibly (in future) the IRQ. LinuxBIOS has
no knowledge of device-specific I/O registers. For dynamic devices,
LinuxBIOS does a few generic operations and lets the OS do the rest. 

So for dynamic devics we know about:
o instances of the device
o Generic control information that is not device-specific

So, to sum up: for static devices, LinuxBIOS needs knowledge that goes
beyond what the OS knows, and will perform device-specific operations
based on internal knowledge LinuxBIOS has about the device. LinuxBIOS will
maintain information about the type of the device and all instances of the
device. In many cases, LinuxBIOS will do operations on the device that the
OS is not capable of doing.

For dynamic devices, LinuxBIOS only does the most general configuration, 
and has no knowledge of device internals. The OS manages device-specific 
operations. 

The dichotomy is pretty clear. 

It is kind of like a house. There are walls and floors and outlets and
phone jacks. These are fixed (static). Then there are chairs and lights
and phones. These are dynamic. You know all about the phone jacks, but you
don't know about what phones are plugged in, and the phones change all the
time, especially if you have children in the house. The static devices
(walls) are a fundamentally different thing than the dynamic devices
(chairs).

We are calling static devices 'chips'. Chips are structured as follows:
there is a chip_control structure for a TYPE of a chip, with generic
information about the chip and a pointer to a control function that is
specific to that chip. Then, for each instance of the chip, there is a
chip structure that contains a pointer to the chip_control structure and
information specific to the INSTANCE of the chip.

Think of chip_info as the class definition, and chip as instantiations of
the chip. 

Chips are statically declared and initialized, and form a tree with a root
at the mainboard, with sibling pointers for chips at the same level --
e.g., two devices hung off the same southbridge are siblings. These
structures are filled out at compile time.

Dynamic devices are called struct device. There is (currently) a device
struct allocated dynamically for each function on a device. 

[digression:  The PCI heritage of devices is very clear once you start
looking at them. This is reasonable, as even on HyperTransport everything
looks like a PCI device.  But it may bite us at some point.]

Devices also form a tree, with children, parents, siblings, etc. One
assumption we've noticed in the device code is that of a 'root bus'.  
Properly, this should not really exist: there can be multiple northbridges
in a system, each with a PCI bus. But the device code so far inherits from 
freebios 1.0 the PC assumption that there is one root bus, and that it is 
configured from a common point of control -- on PCs, the 0xcf8/0xcfc 
mechanism. 

Our early assumption was that we could take the static tree, build a
dynamic tree, and then populate the tree with dynamic devices. Eric would
like to see this happen. Greg and I are no longer convinced this makes
sense. Hence the source of this discussion. 

At minimum, if you build a dynamic tree from the static tree, you'll have
to mark the devices as static. You'll have to handle them differently
because the resources work differently. Some resources such as Base
Address Registers won't exist on devices in the static tree. You'll have
to note that some devices are capable of having device-specific code, and
others are not. You'll end up with structure that really contains two
structures inside: structure members for static devices only, and
structure members for dynamic devices only. It won't be a happy marriage.  
Finally, all your code will have to handle two cases: static and dynamic
devices. Once we looked at this, we saw that we ended up with a dynamic
tree with static devices grafted in, with no apparent savings on
complexity and no real code reuse.

Here is what we think makes sense. The dichotomy between static and 
dynamic devices should remain. The static devices will not be converted 
into some form of dynamic device. There will need to be a linkage from the 
static devices to dynamic devices. This linkage will be at the northbridge 
-- as it is in hardware. 

Northbridges, i.e. devices which have an attached PCI bus (or more than
one), will have a device structure pointer in the device-specific data
structure (i.e. in the chip structure). At some point, as part of the code
that walks the statically allocated northbridge, pci enumeration for that
northbridge takes place.  This has a major advantage for non-PC platforms
(such as PPC) that don't use the standard PC 0xfcf8/0xcfc mechanism for
configuring the PCI bus -- it makes sense to talk to the northbridge
device directly to configure the bus(ses) on that northbridge. It is also
a big win for systems with more than one northbridge -- these will be more
and more common. And for switched, not bus-based, systems the advantage is
apparent.

We thus have a static tree (representing the static resources) with links
at certain places to the dynamic tree (representing dynamic resources).  
LinuxBIOS can do device-specific operations on devices in the static tree,
and can attach dynamic devices to nodes in the static tree. On the
whiteboard in Greg's office, this setup makes lots of sense. We'll see how
it is in practice.

We plan to prototype this in the coming few days and try it out. 

Comments and questions are welcome. We need to open this discussion up. I 
would really like to hear from you folks. As Eric pointed out, we're 
trying to define the 2.0 core functionality, and we want to make it a good 
structure but also one that is easy for people to do new ports. It is 
essential to hear from those of you who are doing ports -- there are lots 
of you on this list.

thanks

ron