Eric has pointed out to me that we need a tree to properly describe the new complex systems such as K8. The current config language is not sufficient to do the job.
I've been thinking about mods to the language that preserve the current simplicity for the vast majority of motherboards out there, while at the same time allowing a complex hierarchy to be described. I think I have an idea.
Currently we say things like this:
southbridge vendor/model ...
superio vendor/model ...
This syntax is actually sufficient for something like 95% of the motherboards we need to support. But what if we have
southbridge vendor/model ... southbridge vendor/model ...
superio vendor/model ...
which southbridge is the superio on?
Here is my proposed change
southbridge vendor/model[=<tag>] [parent=<tag>] ...
superio vendor/model[=<tag>] [parent=<tag>] ...
(note this syntax applies to all the types of parts)
Consider the simple case. The one southbridge is given a default tag of southbridge, with the default parent of northbridge. The superio is given the default tag of superio, with the default parent of southbridge. It all works.
Consider the hard case.
southbridge intel/xyz=sb1 ... southbridge intel/xyz=sb2 ...
superio NSC/abs parent=sb1 ...
If there is more than one northbridge, then we can set the parent tag for the southbridge.
This syntax allows arbitrary trees to be described; it is as simple in the base case as the current one; and it should be fairly straightforward to implement. Computing the path to a given part is then easy.
Some flexibility in the tags should be possible. The tag could allude to an actual tag, or a part type (southbridge), or a part vendor/manufacturer pair (intel/xyz). As long as the tag specifies something unique, it should be acceptable.
Another way to do it is this:
southbridge [tag=<tag>] [parent=<tag>]
any preference for syntax?
Comments?
ron
Phew! What a can of worms you've just opened up!
Some comments:
Your proposed syntax seems a bit inconsistent, in that you use 'name=<tag>' in some places, but not in others. If you go this way then I would suggest using this syntax for all arguments. Also, if 'parent' is optional then what does it mean to leave it out? Attached to the root?
Defining the tree from bottom-up is somewhat non-intuitive and potentially confusing. It seems more natural to define the tree structure top-down as follows:
mainboard vendor/model cpu p5 southbridge vendor/model end southbridge vendor/model2 superio vendor/model ...config... end end end
This provides a much more readable description of the hierarchy, and also avoids the need to define arbitrary tags just to define the tree.
Parsing is simple: a keyword ('mainboard', etc.) starts a tree node. The 'end' keyword ends the current node. Some keywords may be terminal so they don't need 'end', eg 'cpu'. Configuration options no longer need to be supplied as arguments to a keyword, they can be terminal keywords in a node.
This works nicely for the PPC as follows:
mainboard motorola/sandpoint pmc altimus/mpc7500 northbridge motorola/mpc107 end southbridge windbond/w83c553 superio NSC/pc97307 com1={1} com2={1} floppy=0 lpt=1 keyboard=1 hwmonitor=1 end end nvram flash end
I think this also provides a nice way of conceptualizing the architecture.
Regards,
Greg
At 8:06 PM -0600 19/5/03, ron minnich wrote:
Eric has pointed out to me that we need a tree to properly describe the new complex systems such as K8. The current config language is not sufficient to do the job.
I've been thinking about mods to the language that preserve the current simplicity for the vast majority of motherboards out there, while at the same time allowing a complex hierarchy to be described. I think I have an idea.
Currently we say things like this:
southbridge vendor/model ...
superio vendor/model ...
This syntax is actually sufficient for something like 95% of the motherboards we need to support. But what if we have
southbridge vendor/model ... southbridge vendor/model ...
superio vendor/model ...
which southbridge is the superio on?
Here is my proposed change
southbridge vendor/model[=<tag>] [parent=<tag>] ...
superio vendor/model[=<tag>] [parent=<tag>] ...
(note this syntax applies to all the types of parts)
Consider the simple case. The one southbridge is given a default tag of southbridge, with the default parent of northbridge. The superio is given the default tag of superio, with the default parent of southbridge. It all works.
Consider the hard case.
southbridge intel/xyz=sb1 ... southbridge intel/xyz=sb2 ...
superio NSC/abs parent=sb1 ...
If there is more than one northbridge, then we can set the parent tag for the southbridge.
This syntax allows arbitrary trees to be described; it is as simple in the base case as the current one; and it should be fairly straightforward to implement. Computing the path to a given part is then easy.
Some flexibility in the tags should be possible. The tag could allude to an actual tag, or a part type (southbridge), or a part vendor/manufacturer pair (intel/xyz). As long as the tag specifies something unique, it should be acceptable.
Another way to do it is this:
southbridge [tag=<tag>] [parent=<tag>]
any preference for syntax?
Comments?
ron
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
Hi Greg, Ron Just to chip in my $0.02.
Greg, what you've suggested looks to lend itself towards an XML type markup.
How about something like this for your example :-
<mainboard vendor="motorola" board="sandpoint"> <cpu>p5</cpu> <pmc vendor="altimus" model="mpc7500"> <northbridge vendor="motorola" model="mpc107"/> </pmc> <southbridge vendor="winbond" model="w83c553"> <superio vendor="NSC" model="pc97307"> <com number="1"/> <com number="2"/> <lpt/> <keyboard/> <hwmonitor/> </superio> </southbridge> <nvram/> <flash/> </mainboard>
It's just a suggestion, but with a properly defined schema, the config file could be verified/edited easily.
Regards Mark Wilkinson.
-----Original Message----- From: linuxbios-admin@clustermatic.org [mailto:linuxbios-admin@clustermatic.org] On Behalf Of Greg Watson Sent: 20 May 2003 04:51 To: linuxbios@clustermatic.org Subject: Re: config language
Phew! What a can of worms you've just opened up!
Some comments:
Your proposed syntax seems a bit inconsistent, in that you use 'name=<tag>' in some places, but not in others. If you go this way then I would suggest using this syntax for all arguments. Also, if 'parent' is optional then what does it mean to leave it out? Attached to the root?
Defining the tree from bottom-up is somewhat non-intuitive and potentially confusing. It seems more natural to define the tree structure top-down as follows:
mainboard vendor/model cpu p5 southbridge vendor/model end southbridge vendor/model2 superio vendor/model ...config... end end end
This provides a much more readable description of the hierarchy, and also avoids the need to define arbitrary tags just to define the tree.
Parsing is simple: a keyword ('mainboard', etc.) starts a tree node. The 'end' keyword ends the current node. Some keywords may be terminal so they don't need 'end', eg 'cpu'. Configuration options no longer need to be supplied as arguments to a keyword, they can be terminal keywords in a node.
This works nicely for the PPC as follows:
mainboard motorola/sandpoint pmc altimus/mpc7500 northbridge motorola/mpc107 end southbridge windbond/w83c553 superio NSC/pc97307 com1={1} com2={1} floppy=0 lpt=1 keyboard=1 hwmonitor=1 end end nvram flash end
I think this also provides a nice way of conceptualizing the architecture.
Regards,
Greg
At 8:06 PM -0600 19/5/03, ron minnich wrote:
Eric has pointed out to me that we need a tree to properly
describe the
new complex systems such as K8. The current config language is not sufficient to do the job.
I've been thinking about mods to the language that preserve
the current
simplicity for the vast majority of motherboards out there, while at the same time allowing a complex hierarchy to be described.
I think I
have an idea.
Currently we say things like this:
southbridge vendor/model ...
superio vendor/model ...
This syntax is actually sufficient for something like 95% of the motherboards we need to support. But what if we have
southbridge vendor/model ... southbridge vendor/model ...
superio vendor/model ...
which southbridge is the superio on?
Here is my proposed change
southbridge vendor/model[=<tag>] [parent=<tag>] ...
superio vendor/model[=<tag>] [parent=<tag>] ...
(note this syntax applies to all the types of parts)
Consider the simple case. The one southbridge is given a
default tag of
southbridge, with the default parent of northbridge. The superio is given the default tag of superio, with the default parent of southbridge. It all works.
Consider the hard case.
southbridge intel/xyz=sb1 ... southbridge intel/xyz=sb2 ...
superio NSC/abs parent=sb1 ...
If there is more than one northbridge, then we can set the
parent tag
for the southbridge.
This syntax allows arbitrary trees to be described; it is as
simple in
the base case as the current one; and it should be fairly straightforward to implement. Computing the path to a given part is then easy.
Some flexibility in the tags should be possible. The tag
could allude
to an actual tag, or a part type (southbridge), or a part vendor/manufacturer pair (intel/xyz). As long as the tag specifies something unique, it should be acceptable.
Another way to do it is this:
southbridge [tag=<tag>] [parent=<tag>]
any preference for syntax?
Comments?
ron
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/l%3E inuxbios
On Tue, 20 May 2003, Mark Wilkinson wrote:
Greg, what you've suggested looks to lend itself towards an XML type markup.
Thanks for the suggestions and looking at the idea, but we're not going XML. Sorry, but we've got a real bias here against using text markup languages for other purposes.
Besides, Greg language is pretty, and the xml example is in my opionion kind of ugly ...
Other than that, however, sounds like you think Greg's idea is ok?
ron
Hi Ron, Greg,
-----Original Message----- From: linuxbios-admin@clustermatic.org [mailto:linuxbios-admin@clustermatic.org] On Behalf Of ron minnich Sent: 20 May 2003 13:49 To: Mark Wilkinson Cc: 'Greg Watson'; linuxbios@clustermatic.org Subject: RE: config language
On Tue, 20 May 2003, Mark Wilkinson wrote:
Greg, what you've suggested looks to lend itself towards an
XML type
markup.
Thanks for the suggestions and looking at the idea, but we're not going XML. Sorry, but we've got a real bias here against using text markup languages for other purposes.
Besides, Greg language is pretty, and the xml example is in my opionion kind of ugly ...
Other than that, however, sounds like you think Greg's idea is ok?
Yep, all for Greg's idea !
ron
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/l%3E inuxbios
Regards Mark
On Tue, May 20, 2003 at 06:49:15AM -0600, ron minnich wrote:
On Tue, 20 May 2003, Mark Wilkinson wrote:
Greg, what you've suggested looks to lend itself towards an XML type markup.
Thanks for the suggestions and looking at the idea, but we're not going XML. Sorry, but we've got a real bias here against using text markup languages for other purposes.
Besides, Greg language is pretty, and the xml example is in my opionion kind of ugly ...
Other than that, however, sounds like you think Greg's idea is ok?
ron
Ron,
Given that you just bemoaned the work neccessary to build a C-based parser for a new language, one would think that you would be interested in a solution wherein the parser comes pretty much for free. With an XML-based config file, you can have a correct, validating parser implemented in an afternoon. With some thought put into defaults, the translation of the existing config files into XML could be automated with a small amount of effort, and config file editors that do not allow the creation of invalid config files could easily be constructed.
PyRXP will vacuum up an XML document into a native Python data structure, and validate the document against a DTD if you have one: http://www.reportlab.com/xml/pyrxp.html It's fast, GPL'd, and brain-dead easy to use.
--Bob
On Tue, 20 May 2003, Bob Drzyzgula wrote:
Given that you just bemoaned the work neccessary to build a C-based parser for a new language, one would think that
yeah, I'm a whiner :-)
I'll start with yapps first and talk to greg and sung later today. They're the compiler dudes.
ron
Hi Bob,
The config language does lend itself to XML, (and it was my first thought when Ron suggested a new languge) but I would recommend against it for a couple of reasons. The first is that the language needs to serve the dual purpose of specifying *and documenting* a particular configuration. XML is great for the former, but is almost impossible to read easily, particularly for people not used to working with it. Sure, you could then add something that converts the XML into a more readable document, but then what is a simple idea is now becoming very complicated. Secondly, and more importantly, is that it would mean that LB relies on the existence of an extremely complex external component, that is out of the control of the LB community (be it pyRXP, libXML, or whatever). Arguably this is already the case, since the configuration process uses python, but if we adopt a new configuration language then the parser can be included as part of the distribution - making it effectively self-contained. So in my view the limitations of using XML outweigh any advantages that might be obtained.
Regards,
Greg
At 9:24 AM -0400 20/5/03, Bob Drzyzgula wrote:
On Tue, May 20, 2003 at 06:49:15AM -0600, ron minnich wrote:
On Tue, 20 May 2003, Mark Wilkinson wrote:
Greg, what you've suggested looks to lend itself towards an XML type markup.
Thanks for the suggestions and looking at the idea, but we're not going XML. Sorry, but we've got a real bias here against using text markup languages for other purposes.
Besides, Greg language is pretty, and the xml example is in my opionion kind of ugly ...
Other than that, however, sounds like you think Greg's idea is ok?
ron
Ron,
Given that you just bemoaned the work neccessary to build a C-based parser for a new language, one would think that you would be interested in a solution wherein the parser comes pretty much for free. With an XML-based config file, you can have a correct, validating parser implemented in an afternoon. With some thought put into defaults, the translation of the existing config files into XML could be automated with a small amount of effort, and config file editors that do not allow the creation of invalid config files could easily be constructed.
PyRXP will vacuum up an XML document into a native Python data structure, and validate the document against a DTD if you have one: http://www.reportlab.com/xml/pyrxp.html It's fast, GPL'd, and brain-dead easy to use.
--Bob _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
On Tue, May 20, 2003 at 07:56:59AM -0600, Greg Watson wrote:
Hi Bob,
The config language does lend itself to XML, (and it was my first thought when Ron suggested a new languge) but I would recommend against it for a couple of reasons. The first is that the language needs to serve the dual purpose of specifying *and documenting* a particular configuration. XML is great for the former, but is almost impossible to read easily, particularly for people not used to working with it. Sure, you could then add something that converts the XML into a more readable document, but then what is a simple idea is now becoming very complicated. Secondly, and more importantly, is that it would mean that LB relies on the existence of an extremely complex external component, that is out of the control of the LB community (be it pyRXP, libXML, or whatever). Arguably this is already the case, since the configuration process uses python, but if we adopt a new configuration language then the parser can be included as part of the distribution - making it effectively self-contained. So in my view the limitations of using XML outweigh any advantages that might be obtained.
Regards,
Greg
Greg,
Y'know, it's *OK* to say "I just don't like it". :-) I know if Ron & y'all don't *want* to do it it won't get done, which is, again, *OK*. :-)
But understanding that I don't seriously expect this to happen in LB, I do sort of want to respin a couple of your points...
It's interesting what you say about documentation, because one of the oft-cited primary advantages of XML is that it is self-documenting. Perhaps it takes some getting used to but in the end the meaning of:
<pmc vendor="altimus" model="mpc7500">
has got to be at least as clear as
pmc altimus/mpc7500
even if it is a bit more longer.
Beyond this, the formal specification for an XML document is in the DTD (or schema): an overt, external data file that all code can reference. With a config file of the sort that LB has had thus far, the real specification is embedded in the parser source code, perhaps supplemented by usage notes. Thus it is fairly easy for disconnects to occur not only between the implmentation and the user's understanding thereof, but between the written, external documentation and the implementation.
The reliance on an external library is, I would think, something of a red herring. As you point out, there are myriad external libraries that are being relied upon. A yacc/lex implementation relies on the implementation of yacc & lex and all the lexical arcania that come with them. Ron suggested YAPP, which relies on the YAPP project. In the case of PyRXP, that whole thing is GPL'd, so if the project wanted, a version of PyRXP could be frozen into the LB CVS tree. Granted, that could mean fixing it if it turns out to be broken, but that's a separate issue from the one you cited, and is probably not a significantly greater a risk than exists with many other parser generators that could be chosen.
The one other thing I'll add is that, while existing LB developers may not be well-versed in XML, the fact is that, industry-wide, XML literacy is growing if not exploding. In the long term, finding people who understand and can work with XML will be no harder, and probably easier, than finding people who can program in C. A custom configuration language, and the code to implement it, will be novel to everyone who encounters it, now and forever.
Just my $0.02.
--Bob
ok, Bob, let's do this:
send me a pointer to a simple (!) XML parser.
Send me if you can a simple example of a simple thing that uses a simple parser.
we'll take a look.
thanks
ron
I'll be tied up with some other stuff for a day or two, but I'll get back to you with something. Warning, though: If you really want it in XSD, it's less "simple" pretty much by definition. :-)
--Bob
On Tue, May 20, 2003 at 09:31:53AM -0600, ron minnich wrote:
ok, Bob, let's do this:
send me a pointer to a simple (!) XML parser.
Send me if you can a simple example of a simple thing that uses a simple parser.
we'll take a look.
thanks
ron
Ron,
FYI, I'm back and starting to mess with this. I'm digging through some stuff and tinkering with some candidate tools that will hopefully come close to your requirements ("simple" + "schema-based"). The bulk of my XML experience, what there is of it, has been DTD-based, so this takes a small bit of getting up to speed.
Validating, XSDL-aware, Python-based, simple-to-use parsers are (perhaps you knew this :) about as scarce as hen's teeth. The main Python XSDL processsor appears to be XSV http://www.ltg.ed.ac.uk/~ht/xsv-status.html, but I haven't yet made an assesment of the "simple" criteria. However, in digging around I find that recent builds of 4Suite http://www.4suite.org/ provide some support for RELAX NG (see http://www.relaxng.org/) which in fact may be superior for this application because of its extensive support for flexible content ordering.
More later.
--Bob
On Tue, May 20, 2003 at 09:31:53AM -0600, ron minnich wrote:
ok, Bob, let's do this:
send me a pointer to a simple (!) XML parser.
Send me if you can a simple example of a simple thing that uses a simple parser.
we'll take a look.
thanks
ron
The meaning may be just as clear. But I would argue that it's not as easy for the eye to read. And "altimus/mpc7500" is closer to the environment you are actually trying to debug. It's an include file at the path altimus/mpc7500.
Jordan
At 10:45 AM 5/20/2003 -0400, Bob Drzyzgula wrote:
On Tue, May 20, 2003 at 07:56:59AM -0600, Greg Watson wrote:
Hi Bob,
The config language does lend itself to XML, (and it was my first thought when Ron suggested a new languge) but I would recommend against it for a couple of reasons. The first is that the language needs to serve the dual purpose of specifying *and documenting* a particular configuration. XML is great for the former, but is almost impossible to read easily, particularly for people not used to working with it. Sure, you could then add something that converts the XML into a more readable document, but then what is a simple idea is now becoming very complicated. Secondly, and more importantly, is that it would mean that LB relies on the existence of an extremely complex external component, that is out of the control of the LB community (be it pyRXP, libXML, or whatever). Arguably this is already the case, since the configuration process uses python, but if we adopt a new configuration language then the parser can be included as part of the distribution - making it effectively self-contained. So in my view the limitations of using XML outweigh any advantages that might be obtained.
Regards,
Greg
Greg,
Y'know, it's *OK* to say "I just don't like it". :-) I know if Ron & y'all don't *want* to do it it won't get done, which is, again, *OK*. :-)
But understanding that I don't seriously expect this to happen in LB, I do sort of want to respin a couple of your points...
It's interesting what you say about documentation, because one of the oft-cited primary advantages of XML is that it is self-documenting. Perhaps it takes some getting used to but in the end the meaning of:
<pmc vendor="altimus" model="mpc7500">
has got to be at least as clear as
pmc altimus/mpc7500
even if it is a bit more longer.
Beyond this, the formal specification for an XML document is in the DTD (or schema): an overt, external data file that all code can reference. With a config file of the sort that LB has had thus far, the real specification is embedded in the parser source code, perhaps supplemented by usage notes. Thus it is fairly easy for disconnects to occur not only between the implmentation and the user's understanding thereof, but between the written, external documentation and the implementation.
The reliance on an external library is, I would think, something of a red herring. As you point out, there are myriad external libraries that are being relied upon. A yacc/lex implementation relies on the implementation of yacc & lex and all the lexical arcania that come with them. Ron suggested YAPP, which relies on the YAPP project. In the case of PyRXP, that whole thing is GPL'd, so if the project wanted, a version of PyRXP could be frozen into the LB CVS tree. Granted, that could mean fixing it if it turns out to be broken, but that's a separate issue from the one you cited, and is probably not a significantly greater a risk than exists with many other parser generators that could be chosen.
The one other thing I'll add is that, while existing LB developers may not be well-versed in XML, the fact is that, industry-wide, XML literacy is growing if not exploding. In the long term, finding people who understand and can work with XML will be no harder, and probably easier, than finding people who can program in C. A custom configuration language, and the code to implement it, will be novel to everyone who encounters it, now and forever.
Just my $0.02.
--Bob _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
On Tue, 2003-05-20 at 05:49, ron minnich wrote:
Besides, Greg language is pretty, and the xml example is in my opionion kind of ugly ...
eh. i don't program much but i've seen one or two xml config files (of which the authors boost of their ease of understanding)...and i must laugh because the html markup made it a pain to read.
Hi!
Greg, what you've suggested looks to lend itself towards an XML type markup.
Thanks for the suggestions and looking at the idea, but we're not going XML. Sorry, but we've got a real bias here against using text markup languages for other purposes.
XML is not a text markup language. It is a markup language which describes data where data can be almost anything. You would be using it here for the kind of things it was designed for.
Besides, Greg language is pretty, and the xml example is in my opionion kind of ugly ...
That is a matter of taste. Besides, XML is not meant to be read/edited by humans, even though it is perfectly possible.
The biggest drawback is the complexity of XML parsing and (therefore) the size of XML parsers.
Other than that, however, sounds like you think Greg's idea is ok?
Groetjes, Peter Busser
Over on etherboot-developers, there has been discussion about encoding a description of the available NICs in the DHCP request so that an appropriate image can be chained. This encodes the PCI IDs (and, IIRC some ISA card names).
It would be great to have a more general parseable hardware inventory available via DHCP (which can neatly encapsulate options for this kind of thing). So, whatever kind of structure you end up with, consider how a more general purpose tree fits in and how it can be flattened into DHCP encapsulated options.
On Tue, May 20, 2003 at 12:16:16PM +0100, prl-linuxbios@sychron.com wrote:
Over on etherboot-developers, there has been discussion about encoding a description of the available NICs in the DHCP request so that an appropriate image can be chained. This encodes the PCI IDs (and, IIRC some ISA card names).
It would be great to have a more general parseable hardware inventory available via DHCP (which can neatly encapsulate options for this kind of thing). So, whatever kind of structure you end up with, consider how a more general purpose tree fits in and how it can be flattened into DHCP encapsulated options.
The great thing about using XML in this kind of situation is the relative ease with which XML-based data can be parsed and transformed into nearly any other format. Were the config language encoded in XML, your application -- to the extent that the data in the config file were truely useful here -- would most likely be codable in a relatively simple script in XSL or Python, etc.
--Bob
XML parsers I have seen are not trivial. S-expressions are far better for this type of thing. For a simple, low-cost, embeddable S-expression parsing library see:
ron
On Tue, 20 May 2003 prl-linuxbios@sychron.com wrote:
It would be great to have a more general parseable hardware inventory available via DHCP (which can neatly encapsulate options for this kind of thing). So, whatever kind of structure you end up with, consider how a more general purpose tree fits in and how it can be flattened into DHCP encapsulated options.
great idea. Looks like we're going to a real parser anyway. We'll keep that in mind, and if we don't, tell us :-)
ron
prl-linuxbios@sychron.com writes:
Over on etherboot-developers, there has been discussion about encoding a description of the available NICs in the DHCP request so that an appropriate image can be chained. This encodes the PCI IDs (and, IIRC some ISA card names).
It would be great to have a more general parseable hardware inventory available via DHCP (which can neatly encapsulate options for this kind of thing). So, whatever kind of structure you end up with, consider how a more general purpose tree fits in and how it can be flattened into DHCP encapsulated options.
This is functionality we need in the LinuxBIOS table especially for devices that cannot be detected by the OS. Like temperature sensors. This should be easy to add in the 1.1 development tree.
This is information we cannot transmit via DHCP. A DHCP packet is to small. In DHCP we just need enough information to bootstrap.
On the same vein I am very tempted after those tables have been implemented to write a version of mkelfImage that will generate a pirq and an mptable from the LinuxBIOS table. Then we can remove that cruft from the LinuxBIOS.
Eric
On 20 May 2003, Eric W. Biederman wrote:
On the same vein I am very tempted after those tables have been implemented to write a version of mkelfImage that will generate a pirq and an mptable from the LinuxBIOS table. Then we can remove that cruft from the LinuxBIOS.
yes, and also eliminate error-prone $PIR tables. I am really sick of getting $PIR tables from broken BIOSes.
ron
This is functionality we need in the LinuxBIOS table especially for devices that cannot be detected by the OS. Like temperature sensors. This should be easy to add in the 1.1 development tree.
This is information we cannot transmit via DHCP. A DHCP packet is to small. In DHCP we just need enough information to bootstrap.
I was not suggesting that the tables must of necessity all be munged into a DHCP packet, though I observe that with a reasonable encoding, most descriptions probably *would* fit in a single DHCP options field of 312 bytes. In practise I agree that one would use only those parts which were relevant to (network) booting in a real DHCP packet.
But... reading through the recent discussions about XML, it took me some time to release that people were talking about parsing IN LinuxBIOS, as opposed using XML as an input file to a build or config process. XML or similar in LinuxBIOS itself seems like overkill to me.
Please consider using DHCP encapsulated option format as an internal format for LinuxBIOS in the first instance, even if DHCP (the protocol) is not used. I see several reasons...
- Compatibility with the DHCP protocol
OK, I understand that not everything wants to use DHCP for network configuration, but even the most ardent boot-from-the-bare-metal embedded fan must appreciate that many do use DHCP, even if only for testing code via DHCP / TFTP before a final product is burned in.
Or, put another way, why would you make a *point* of being different?
- Code exists
DHCP option parsing code exists in Etherboot (so Etherboot-on-LinuxBIOS users have it anyway). As I say, a hardware description has been discussed (and I *think* implemented) in Etherboot in the particular case of PCI nics.
- Compact parser and encoding
DHCP hardware descriptions (i.e. individual chipsets) should be encoded numerically. There are reasons why existing DHCP options and SNMP MIBs do it this way - 32 bits encodes a HELL of a lot of possible hardware types in only 4 bytes. Doubtless someone will tell me that DHCP isn't the most efficient method or that they have a *really* neat parser for their favourite format. Big deal - refer to "Code exists".
- Do the hard stuff outside LinuxBIOS
Register the encodings centrally held table (like other DHCP options or an SNMP MIB), so that when building, tables can be built as appropriate from XML definitions. A DHCP server or a small amount of perl in the build process can translate raw numbers to and from human readable form using boilerplate derived from the central table. Where possible (e.g. PCI IDs), use an existing registry anyway.
On Wed, 2003-05-21 at 23:03, prl-linuxbios@sychron.com wrote:
But... reading through the recent discussions about XML, it took me some time to release that people were talking about parsing IN LinuxBIOS, as opposed using XML as an input file to a build or config process. XML or similar in LinuxBIOS itself seems like overkill to me.
Maybe I can not read English. Are we really talking about parsing XML in LinuxBIOS rather than "config" tool ?
Hi, Years before XML, In perl, I developed a kind of complex structure to store user state information at the NY Stock Exchange for if/when the systems went down all the users could be restored to what they were doing beforehand.
Later I wrote a dataserver which accepts something like a URL string with a serialized data structure on the end of it, freezes it to disk and the returns an sub or super set of that structure in serialized form. I think created a way to put code in it as text, and now I have promised myself that I will actually embed code in them someday.
XML came along and showed how inefficiently this could be done. I think the parsing complexity is becasue it is a markup language and has attributes and stuff that are comical by normal language standards.
I went to a lisp meeting last night, and suddenly realized I had been using lisp to teach perl to kids... now I wish there was a C-Lisp-AN.
I think perl structures are about as readable as a config can get, though there is a slight learning curve. I am also wondering how Perl6's Parrot VM can be integrated into LinuxBIOS.
Here is a quick tutorial for perl with structures that I wrote, http://thinman.com/perl_primer/
John
On Wed, 2003-05-21 at 11:03, prl-linuxbios@sychron.com wrote:
This is functionality we need in the LinuxBIOS table especially for devices that cannot be detected by the OS. Like temperature sensors. This should be easy to add in the 1.1 development tree.
This is information we cannot transmit via DHCP. A DHCP packet is to small. In DHCP we just need enough information to bootstrap.
I was not suggesting that the tables must of necessity all be munged into a DHCP packet, though I observe that with a reasonable encoding, most descriptions probably *would* fit in a single DHCP options field of 312 bytes. In practise I agree that one would use only those parts which were relevant to (network) booting in a real DHCP packet.
But... reading through the recent discussions about XML, it took me some time to release that people were talking about parsing IN LinuxBIOS, as opposed using XML as an input file to a build or config process. XML or similar in LinuxBIOS itself seems like overkill to me.
Please consider using DHCP encapsulated option format as an internal format for LinuxBIOS in the first instance, even if DHCP (the protocol) is not used. I see several reasons...
- Compatibility with the DHCP protocol
OK, I understand that not everything wants to use DHCP for network configuration, but even the most ardent boot-from-the-bare-metal embedded fan must appreciate that many do use DHCP, even if only for testing code via DHCP / TFTP before a final product is burned in.
Or, put another way, why would you make a *point* of being different?
- Code exists
DHCP option parsing code exists in Etherboot (so Etherboot-on-LinuxBIOS users have it anyway). As I say, a hardware description has been discussed (and I *think* implemented) in Etherboot in the particular case of PCI nics.
- Compact parser and encoding
DHCP hardware descriptions (i.e. individual chipsets) should be encoded numerically. There are reasons why existing DHCP options and SNMP MIBs do it this way - 32 bits encodes a HELL of a lot of possible hardware types in only 4 bytes. Doubtless someone will tell me that DHCP isn't the most efficient method or that they have a *really* neat parser for their favourite format. Big deal - refer to "Code exists".
- Do the hard stuff outside LinuxBIOS
Register the encodings centrally held table (like other DHCP options or an SNMP MIB), so that when building, tables can be built as appropriate from XML definitions. A DHCP server or a small amount of perl in the build process can translate raw numbers to and from human readable form using boilerplate derived from the central table. Where possible (e.g. PCI IDs), use an existing registry anyway.
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
thanks, we have (for now) settled on a python-based ELL parser generator. There is a bit more error handling in python than perl objects, which I like.
But thanks for the info.
ron
On Mon, 19 May 2003, Greg Watson wrote:
Defining the tree from bottom-up is somewhat non-intuitive and potentially confusing. It seems more natural to define the tree structure top-down as follows:
mainboard vendor/model cpu p5 southbridge vendor/model end southbridge vendor/model2 superio vendor/model ...config... end end end
OH, ok, you win. I had thought of this but hate the idea of writing a parser ... string handling in C is like walking on broken glass.
mainboard motorola/sandpoint pmc altimus/mpc7500 northbridge motorola/mpc107 end southbridge windbond/w83c553 superio NSC/pc97307 com1={1} com2={1} floppy=0 lpt=1 keyboard=1 hwmonitor=1 end end nvram flash end
agreed, it's pretty. But NOBODY is allowed to suggest lexical scoping from indentation. That's not allowed :-)
Greg, parser in the usual lex/yacc/C mode or .... I'll see if anybody's got parsers written in python.
ron
ron minnich rminnich@lanl.gov writes:
Eric has pointed out to me that we need a tree to properly describe the new complex systems such as K8. The current config language is not sufficient to do the job.
Several points. 1) Any system with a pci-pci bridge needs this. The more bridges you have the more this shows through.
2) The device config tree will be grafted together with the discovered device tree during device initialization. Which means that a complete inventory of the hardware will be available only after the 2 are merged.
The truly cute case will be when we have configuration for devices that are not present. Many boards have things like a scsi option etc.
Eric
On 20 May 2003, Eric W. Biederman wrote:
- The device config tree will be grafted together with the discovered device tree during device initialization. Which means that a complete inventory of the hardware will be available only after the 2 are merged.
yes.
The truly cute case will be when we have configuration for devices that are not present. Many boards have things like a scsi option etc.
Greg thought of that, it's going in there.
ron
-
Bob Drzyzgula -
ebiederman＠lnxi.com -
Greg Watson -
John van Vlaanderen -
Mark Wilkinson -
ollie lho -
Peter Busser -
prl-linuxbios＠sychron.com -
roger -
ron minnich -
sklavo＠cybermorph.com