There has been a similar proposal on this list a while ago, but nothing happened so far, so I want to put this pack to discussion.
Objectives ----------
1.) LinuxBIOS is kind of hard to set up for project newbies, since it does not only require manually tweaking the configuration files for basically every situation, but also necessarily needs an external payload to do anything useful. LinuxBIOS currently sets a high barrier due to the modular concepts it uses.
- LinuxBIOS itself is sometimes very sensitive to the build environment. See requirement for setting LANG for example.
- For a project outsider it is hard to determine the best payload solution for a specific purpose. There is basically no information except the mailing list archives.
- Config files have to be tweaked to explicitly suit the user's directory structure. There is no proposal, nothing that works out of the box. One just _has to_ edit the config files.
- LinuxBIOS requires the user to specify a size for the payload instead of determining the required size and doing the needed calculations itself. This is very hand-crufted and can be pretty mind wasting.
2.) LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
LinuxBIOS wants to keep up it's modularity, letting each module do it's job. This possibility of not doing more than one task should be passed on to the payloads.
Solution --------
Feature plugins like "testbios" should not be merged into LinuxBIOS' core code. Instead they should be implemented as a payload. Since we want to load an operating system later on, we also need to be able to load more than one payload. Implementing ELFLOAD in each such plugin is inadequate. Instead LinuxBIOS should execute multiple payloads the same way it executes multiple "drivers" now.
* LinuxBIOS therefore needs a way to automatically determine payload sizes when building the image and later when executing payloads. Hardcoding the size values in the config files is inadequate and will lead to unnecessary overhead
* LinuxBIOS at runtime needs to go through the list of available payloads, either by having a linked list of payload start points or by scanning the flash rom.
* LinuxBIOS should, to be consequent, remove all streaming code except CONFIG_ROM_STREAM
* Payloads should have the possibility to add their own enhancements to the LinuxBIOS table.
* A least one payload should be "default payload" with the possibility to build this automatically and link it into the image.
This is why I checked the "util/extensions" directory into v2 during the last discussion. It should hold possible choices to payloads that can automatically be built and included. Potentially one could add more payloads by symlinking their source tree to this directory to make it available to LinuxBIOS without major reconfiguration. People feel a lot safer with creating a symlink than with changing config files they do not fully understand.
Since these can later be executed in row by elfboot, the minimum overhead design of LinuxBIOS itself will not be hurt.
At this point I want to put an idea to discussion: If we are going more and more modular and some of us think the current tree is too bloated: Why do we not modularize code like pci resource allocation into a payload as well. My favorite bootloader may already do this and I can't stand this bloat everywhere, you know ;) Even though this may sound funny, I am serious about this issue. I do not see why allocating PCI resources should really be part of the lowlevel code, except for the fact that the NEXT payload in row, potentially Linux itself is too stupid to do that. Bummer.
* LinuxBIOS configuration should have an easier mechanism for choosing payloads from the "default" directory, allowing them to be built automatically. Right now I am doing:
cd filo-0.4.2 linux32 make CC="gcc -m32" LD="ld -melf_i386" cd .. freebios2/targets/buildtarget amd/solo $PWD/freebios2 cd build-solo make cd .. cp build-solo/solo.rom .
My target Config.lb comes with these constructs: target ../../../../build-solo payload ../../filo-0.4.2/filo.elf
So I build everything completely out of the freebios2 tree, because building in-tree sucks. The only thing left is to get filo and the other payloads to build out-of-tree as well.
Comments? Discussion? Assassination attempts?
Stefan
Stefan,
I think this is a reasonable idea, particularly your suggestion of making linuxbios more modular. One of my main beefs with the payload strategy is that each payload has to provide it's own set of, potentially buggy, driver code. If we have 5 payloads then we have 5 sets of drivers that all do the same thing slightly differently. If the drivers were modular enough so that a payload could call them directly, then this would go a long way to addressing these concerns.
Regards,
Greg
On Jun 8, 2004, at 4:49 AM, Stefan Reinauer wrote:
There has been a similar proposal on this list a while ago, but nothing happened so far, so I want to put this pack to discussion.
Objectives
1.) LinuxBIOS is kind of hard to set up for project newbies, since it does not only require manually tweaking the configuration files for basically every situation, but also necessarily needs an external payload to do anything useful. LinuxBIOS currently sets a high barrier due to the modular concepts it uses.
LinuxBIOS itself is sometimes very sensitive to the build environment. See requirement for setting LANG for example.
For a project outsider it is hard to determine the best payload solution for a specific purpose. There is basically no information except the mailing list archives.
Config files have to be tweaked to explicitly suit the user's directory structure. There is no proposal, nothing that works out of the box. One just _has to_ edit the config files.
LinuxBIOS requires the user to specify a size for the payload
instead of determining the required size and doing the needed calculations itself. This is very hand-crufted and can be pretty mind wasting.
2.) LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
LinuxBIOS wants to keep up it's modularity, letting each module do it's job. This possibility of not doing more than one task should be passed on to the payloads.
Solution
Feature plugins like "testbios" should not be merged into LinuxBIOS' core code. Instead they should be implemented as a payload. Since we want to load an operating system later on, we also need to be able to load more than one payload. Implementing ELFLOAD in each such plugin is inadequate. Instead LinuxBIOS should execute multiple payloads the same way it executes multiple "drivers" now.
- LinuxBIOS therefore needs a way to automatically determine payload sizes when building the image and later when executing payloads. Hardcoding the size values in the config files is inadequate and
will lead to unnecessary overhead
LinuxBIOS at runtime needs to go through the list of available payloads, either by having a linked list of payload start points or by scanning the flash rom.
LinuxBIOS should, to be consequent, remove all streaming code except CONFIG_ROM_STREAM
Payloads should have the possibility to add their own enhancements
to the LinuxBIOS table.
A least one payload should be "default payload" with the possibility to build this automatically and link it into the image.
This is why I checked the "util/extensions" directory into v2 during the last discussion. It should hold possible choices to payloads
that can automatically be built and included. Potentially one could add more payloads by symlinking their source tree to this directory to make it available to LinuxBIOS without major reconfiguration. People feel a lot safer with creating a symlink than with changing config files they do not fully understand.
Since these can later be executed in row by elfboot, the minimum overhead design of LinuxBIOS itself will not be hurt.
At this point I want to put an idea to discussion: If we are going more and more modular and some of us think the current tree is too bloated: Why do we not modularize code like pci resource allocation into a payload as well. My favorite bootloader may already do this and I can't stand this bloat everywhere, you know ;) Even though this may sound funny, I am serious about this issue. I do not see why allocating PCI resources should really be part of the lowlevel code, except for the fact that the NEXT payload in row, potentially Linux itself is too stupid to do that. Bummer.
LinuxBIOS configuration should have an easier mechanism for choosing payloads from the "default" directory, allowing them to be built automatically. Right now I am doing:
cd filo-0.4.2 linux32 make CC="gcc -m32" LD="ld -melf_i386" cd .. freebios2/targets/buildtarget amd/solo $PWD/freebios2 cd build-solo make cd .. cp build-solo/solo.rom .
My target Config.lb comes with these constructs: target ../../../../build-solo payload ../../filo-0.4.2/filo.elf
So I build everything completely out of the freebios2 tree, because building in-tree sucks. The only thing left is to get filo and the other payloads to build out-of-tree as well.
Comments? Discussion? Assassination attempts?
Stefan
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
* Greg Watson gwatson@lanl.gov [040608 15:22]:
I think this is a reasonable idea, particularly your suggestion of making linuxbios more modular. One of my main beefs with the payload strategy is that each payload has to provide it's own set of, potentially buggy, driver code. If we have 5 payloads then we have 5 sets of drivers that all do the same thing slightly differently. If the drivers were modular enough so that a payload could call them directly, then this would go a long way to addressing these concerns.
As far as I can tell the only drivers involved would be output drivers, ie. video output and serial output. There the extensible LinuxBIOS table could come into play. The video driver could store a pointer to the framebuffer, the resolution and maybe even a font, to save duplicates. Serial is only a kilobyte or so of a driver i think.
Have I forgotten something?
Stefan
On Jun 8, 2004, at 7:59 AM, Stefan Reinauer wrote:
- Greg Watson gwatson@lanl.gov [040608 15:22]:
I think this is a reasonable idea, particularly your suggestion of making linuxbios more modular. One of my main beefs with the payload strategy is that each payload has to provide it's own set of, potentially buggy, driver code. If we have 5 payloads then we have 5 sets of drivers that all do the same thing slightly differently. If the drivers were modular enough so that a payload could call them directly, then this would go a long way to addressing these concerns.
As far as I can tell the only drivers involved would be output drivers, ie. video output and serial output. There the extensible LinuxBIOS table could come into play. The video driver could store a pointer to the framebuffer, the resolution and maybe even a font, to save duplicates. Serial is only a kilobyte or so of a driver i think.
Have I forgotten something?
PCI device code and resource information should be available for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
Greg
Finally, someone has recognized the lack of support with LinuxBios. About 6 months ago we decide to do an x86 based project but abandoned it because of the lack of support for this bootloader. The code is disorganized and in disarray as far as I'm concerend. We were on a tight schedule and didn't have the luxury of spending a lot of time trying to understand the code layout. This was due in part to the lack of badly needed documentation and code organization. We decided to base our design on a powerpc and go with u-boot. Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86 for everyday use.:-(
--- Greg Watson gwatson@lanl.gov wrote:
On Jun 8, 2004, at 7:59 AM, Stefan Reinauer wrote:
- Greg Watson gwatson@lanl.gov [040608 15:22]:
I think this is a reasonable idea, particularly your
suggestion of
making linuxbios more modular. One of my main beefs with
the payload
strategy is that each payload has to provide it's own set
of,
potentially buggy, driver code. If we have 5 payloads then
we have 5
sets of drivers that all do the same thing slightly
differently. If
the drivers were modular enough so that a payload could call
them
directly, then this would go a long way to addressing these concerns.
As far as I can tell the only drivers involved would be
output drivers,
ie. video output and serial output. There the extensible
LinuxBIOS
table could come into play. The video driver could store a pointer
to the
framebuffer, the resolution and maybe even a font, to save
duplicates.
Serial is only a kilobyte or so of a driver i think.
Have I forgotten something?
PCI device code and resource information should be available for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
Greg
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
IIRC, that's what Greg's addition of FILO for PPC was all about.
On Tue, 8 Jun 2004, Frank wrote:
Finally, someone has recognized the lack of support with LinuxBios. About 6 months ago we decide to do an x86 based project but abandoned it because of the lack of support for this bootloader. The code is disorganized and in disarray as far as I'm concerend. We were on a tight schedule and didn't have the luxury of spending a lot of time trying to understand the code layout. This was due in part to the lack of badly needed documentation and code organization. We decided to base our design on a powerpc and go with u-boot. Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86 for everyday use.:-(
--- Greg Watson gwatson@lanl.gov wrote:
On Jun 8, 2004, at 7:59 AM, Stefan Reinauer wrote:
- Greg Watson gwatson@lanl.gov [040608 15:22]:
I think this is a reasonable idea, particularly your
suggestion of
making linuxbios more modular. One of my main beefs with
the payload
strategy is that each payload has to provide it's own set
of,
potentially buggy, driver code. If we have 5 payloads then
we have 5
sets of drivers that all do the same thing slightly
differently. If
the drivers were modular enough so that a payload could call
them
directly, then this would go a long way to addressing these concerns.
As far as I can tell the only drivers involved would be
output drivers,
ie. video output and serial output. There the extensible
LinuxBIOS
table could come into play. The video driver could store a pointer
to the
framebuffer, the resolution and maybe even a font, to save
duplicates.
Serial is only a kilobyte or so of a driver i think.
Have I forgotten something?
PCI device code and resource information should be available for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
Greg
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
You missed the point! If I wanted a bootloader for the PPC I would use u-boot and that's what I did. But if I wanted a bootloader for the X86 platform I would have liked to use LinuxBios. U-BOOT is synonymous with the PPC. LinuxBios should be synonymous with the X86 just like PMON is with the MIPS platform. Take a long look at u-boot and you'll find support for just about every PPC and the number of drivers are abundant. I can port u-boot to a new PPC based board in about 3 days tops. There is no way I can port LinuxBios to a new X86 based board in that amount of time. You should focus your efforts on improving LinuxBios for the X86 and not waste time trying to make it work for the PPC. If I wanted a bootloader I would use u-boot not LinuxBios. I have heard of other people abandoning LinuxBios and going to u-boot in that past, just like I did... --- "Hendricks David W." dwh@lanl.gov wrote:
IIRC, that's what Greg's addition of FILO for PPC was all about.
On Tue, 8 Jun 2004, Frank wrote:
Finally, someone has recognized the lack of support with LinuxBios. About 6 months ago we decide to do an x86 based project but abandoned it because of the lack of support for
this
bootloader. The code is disorganized and in disarray as far
as
I'm concerend. We were on a tight schedule and didn't have
the
luxury of spending a lot of time trying to understand the
code
layout. This was due in part to the lack of badly needed documentation and code organization. We decided to base our design on a powerpc and go with u-boot. Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86
for
everyday use.:-(
--- Greg Watson gwatson@lanl.gov wrote:
On Jun 8, 2004, at 7:59 AM, Stefan Reinauer wrote:
- Greg Watson gwatson@lanl.gov [040608 15:22]:
I think this is a reasonable idea, particularly your
suggestion of
making linuxbios more modular. One of my main beefs
with
the payload
strategy is that each payload has to provide it's own
set
of,
potentially buggy, driver code. If we have 5 payloads
then
we have 5
sets of drivers that all do the same thing slightly
differently. If
the drivers were modular enough so that a payload could
call
them
directly, then this would go a long way to addressing these
concerns.
As far as I can tell the only drivers involved would be
output drivers,
ie. video output and serial output. There the extensible
LinuxBIOS
table could come into play. The video driver could store a
pointer
to the
framebuffer, the resolution and maybe even a font, to
save
duplicates.
Serial is only a kilobyte or so of a driver i think.
Have I forgotten something?
PCI device code and resource information should be
available
for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
Greg
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
On Tue, 8 Jun 2004, Frank wrote:
You missed the point!
Yes, I did miss the point (I should've read more closely).
at u-boot and you'll find support for just about every PPC and the number of drivers are abundant. I can port u-boot to a new PPC based board in about 3 days tops. There is no way I can port LinuxBios to a new X86 based board in that amount of time.
I haven't looked at u-boot code much, so I cannot compare ease of porting to a new board. However, LB doesn't seem all bad in that area from where I stand. Of course it's going to be difficult if you switch architectures, but once the initial work is done it's a piece of cake. I was get an Iwill DK8S2 (Dual opteron board) running in less than 3 days simply by duplicating the Arima HDAMA (Another dual Opteron board) code and tweaking some config files, and I'm just a student working here and not some l33t hacker like Eric or Ron. No hacking on assembly code (Or even C code for that matter) was required.
As far as architectures go, LinuxBIOS has run on Alphas, x86, x86-64, and now PPC. It's portable. And since freebios2 is took a huge leap in reducing the amount of assembly required to do a port, it should be easier now than it was before.
On Tue, 8 Jun 2004, Frank wrote:
not waste time trying to make it work for the PPC. If I wanted a bootloader I would use u-boot not LinuxBios. I have heard of other people abandoning LinuxBios and going to u-boot in that past, just like I did...
You ought to try FILO as a payload for LinuxBIOS. I think it accomplishes what you wanted a year ago ( http://www.clustermatic.org/pipermail/linuxbios/2003-June/003789.html ).
--- "Hendricks David W." dwh@lanl.gov wrote:
IIRC, that's what Greg's addition of FILO for PPC was all about.
On Tue, 8 Jun 2004, Frank wrote:
Finally, someone has recognized the lack of support with LinuxBios. About 6 months ago we decide to do an x86 based project but abandoned it because of the lack of support for
this
bootloader. The code is disorganized and in disarray as far
as
I'm concerend. We were on a tight schedule and didn't have
the
luxury of spending a lot of time trying to understand the
code
layout. This was due in part to the lack of badly needed documentation and code organization. We decided to base our design on a powerpc and go with u-boot. Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86
for
everyday use.:-(
--- Greg Watson gwatson@lanl.gov wrote:
On Jun 8, 2004, at 7:59 AM, Stefan Reinauer wrote:
- Greg Watson gwatson@lanl.gov [040608 15:22]:
I think this is a reasonable idea, particularly your
suggestion of
making linuxbios more modular. One of my main beefs
with
the payload
strategy is that each payload has to provide it's own
set
of,
potentially buggy, driver code. If we have 5 payloads
then
we have 5
sets of drivers that all do the same thing slightly
differently. If
the drivers were modular enough so that a payload could
call
them
directly, then this would go a long way to addressing these
concerns.
As far as I can tell the only drivers involved would be
output drivers,
ie. video output and serial output. There the extensible
LinuxBIOS
table could come into play. The video driver could store a
pointer
to the
framebuffer, the resolution and maybe even a font, to
save
duplicates.
Serial is only a kilobyte or so of a driver i think.
Have I forgotten something?
PCI device code and resource information should be
available
for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
Greg
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
Frank wrote:
Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86 for everyday use.:-(
I think the above is a little harsh.
As long as I've been on this list LinuxBIOS has always been about getting dirty with the code and making it to do what you need it to do for your specific applicaiton. Along the way many good people have put a _lot_ of effort into trying to take all these individual needs and produce some sort of API that could be re-useable by the next person. But at this level that's really hard.
We've always known the level of documentation sucks. If you ask we will tell you it sucks. Just none of the people who really know the core have time to pull off and re-hash it.
From what it started with and considering the odds stacked against it LB has make _huge_ advancements. It x86 specific becuse that's what all the core developers use. For LinuxBios to make the jump to multi-platform it will take an investment of developers such as yourself getting dirty in the core and makeing it happen. I don't think any of the developers just "assume" the world uses x86. x86 just happens to be most of what thier world is right now.
I understand your time requirements prevented you from jumping in on this project but until _someone_ takes the initiative and brings LB forward on non-x86 platforms it won't ever meet your "masses" criteria.
So I'd suggest that if you really wanted LB available with all the extra features for your platform you should revise your tone a bit. Whiners don't get the best of support from coders.
You missed the point as well! I was not complaining or whining as you put it about not having support for the PPC. I was complaining about the lack of support for new users to use it for the X86 platform. Leave the "other" processor alone and focus on what you do best, the X86 platform. As far as I'm concerned you will _never_ have as much support for the PPC as u-boot does and u-boot will never have the support for the X86 as LinuxBios does. But it speaks volumes about the readability of the code when someone switches directions just so they don't have to go thru the pain of trying understand your code base. I am not a novice, I have been doing this for over 20 years. I have ported or written bootloaders for just about every platform out there. The trick is to use the bootloader for a particular processor that has the most traction for a particular architecture.
--- Richard Smith rsmith@bitworks.com wrote:
Frank wrote:
Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86
for
everyday use.:-(
I think the above is a little harsh.
As long as I've been on this list LinuxBIOS has always been about getting dirty with the code and making it to do what you need it to do for your specific applicaiton. Along the way many good people have put a _lot_ of effort into trying to take all these individual needs and produce some sort of API that could be re-useable by the next person. But at this level that's really hard.
We've always known the level of documentation sucks. If you ask we will tell you it sucks. Just none of the people who really know the core have time to pull off and re-hash it.
From what it started with and considering the odds stacked against it LB has make _huge_ advancements. It x86 specific becuse that's what all the core developers use. For LinuxBios to make the jump to multi-platform it will take an investment of developers such as yourself getting dirty in the core and makeing it happen. I don't think any of the developers just "assume" the world uses x86. x86 just happens to be most of what thier world is right now.
I understand your time requirements prevented you from jumping in on this project but until _someone_ takes the initiative and brings LB forward on non-x86 platforms it won't ever meet your "masses" criteria.
So I'd suggest that if you really wanted LB available with all the extra features for your platform you should revise your tone a bit. Whiners don't get the best of support from coders.
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
Frank wrote:
You missed the point as well!
Yes. seems I did. If your mail read less critical to me it might have been clearer. Anyway. My apologies for going on the defensive.
support for the X86 as LinuxBios does. But it speaks volumes about the readability of the code when someone switches directions just so they don't have to go thru the pain of trying understand your code base.
I'll not disagree with you there. I haven't messed with v2 much but v1 does take a lot to navigate and grok.
Are you talking v1 or v2?
and that's another problem. Why V1 and V2... --- Richard Smith rsmith@bitworks.com wrote:
Frank wrote:
You missed the point as well!
Yes. seems I did. If your mail read less critical to me it might have been clearer. Anyway. My apologies for going on the defensive.
support for the X86 as LinuxBios does. But it speaks volumes about the readability of the code when someone switches directions just so they don't have to go thru the pain of
trying
understand your code base.
I'll not disagree with you there. I haven't messed with v2 much but v1 does take a lot to navigate and grok.
Are you talking v1 or v2?
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
Frank wrote:
and that's another problem. Why V1 and V2...
V1 was deemed un-maintainable. All the memory init code has to be done in assembly with registers only and its very difficult stuff to write. There are also several issues with the config files.
Eric wrote romcc which compiles C code that runs without RAM. And the LANL guys did a re-work on the config file parser. The merging of the 2 new major sub-systems essentially became V2. [If I left anything/anyone out please someone correct me, I was but a spectator for all this]
The V2 tree was then started with a fresh clean slate and the goal to correct some of the infrastructure problems with V1.
V1 is still around for people like me who haven't been able to make the port to V2. Like you I'm still under too much time pressure and until I can make my video work the effort to go to V2 dosen't make sense. One I can deliver a proto to my customer then I'll go look at V2.
If you are starting from scratch then don't even bother with V1 except for reference only. V2 is the future. Hopefully you can help it be a better future than V1 had.
Someone mentioned ROMCC when I first started to inquire about V2. But it wasn't in the source tree for V2 and I had to not only go searching for it I had to figure out how to use it.:-( That is another selling point for u-boot. When I download the toolchain (ELDK3) I get everything in one place. From what I know about ROMCC, it looks like the answer to the "stack in cache" trick. I am not trying to flame or put down LB. I'm just trying to point out it's shortcomings. I was only trying to provide some constructive criticism. In about 6 months we will revisit V2 and try to see if we can use it for our cost reduced version of the existing product. Hopefully I won't be pressed for time as I am now and will be able to contribute something.:-)
--- Richard Smith rsmith@bitworks.com wrote:
Frank wrote:
and that's another problem. Why V1 and V2...
V1 was deemed un-maintainable. All the memory init code has to be done in assembly with registers only and its very difficult stuff to write. There are also several issues with the config files.
Eric wrote romcc which compiles C code that runs without RAM. And the LANL guys did a re-work on the config file parser. The merging of the 2 new major sub-systems essentially became V2. [If I left anything/anyone out please someone correct me, I was but a spectator for all this]
The V2 tree was then started with a fresh clean slate and the goal to correct some of the infrastructure problems with V1.
V1 is still around for people like me who haven't been able to make the port to V2. Like you I'm still under too much time pressure and until I can make my video work the effort to go to V2 dosen't make sense. One I can deliver a proto to my customer then I'll go look at V2.
If you are starting from scratch then don't even bother with V1 except for reference only. V2 is the future. Hopefully you can help it be a better future than V1 had.
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
* Frank frannk_m1@yahoo.com [040608 21:14]:
Someone mentioned ROMCC when I first started to inquire about V2. But it wasn't in the source tree for V2 and I had to not only go searching for it I had to figure out how to use it.:-(
It is, since a very long time now. freebios2/util/romcc/ And it is built and used automatically since when building LinuxBIOS v2.
Feel free to come back with questions to this list when you are evaluating firmware for your next project. :-)
Stefan
On Tue, Jun 08, 2004 at 12:14:56PM -0700, Frank wrote:
I am not trying to flame or put down LB. I'm just trying to point out it's shortcomings.
Thanks! I think you'll be glad to see that a lot has changed since your last visit, no thanks to me though. :)
Since you're familiar with the problem I assume you've also had your share of x86 bugs and un/misdocumented "features" - which also have had some effect on LB structure, since noone really knew about the horrors in some of the mainboards/chips/configurations out there.
//Peter
On Tue, 8 Jun 2004, Frank wrote:
for new users to use it for the X86 platform. Leave the "other" processor alone and focus on what you do best, the X86 platform. As far as I'm concerned you will _never_ have as much support for the PPC as u-boot does and u-boot will never have the support for the X86 as LinuxBios does. But it speaks volumes
The objectives of the LANL people are mostly centered around clustering and HPC, not mass production. If someone wants to experiment with PPC clustering and sees a niche for LinuxBIOS that other BIOS options cannot satisfy, then we do what we can to support it. We're not competing with u-boot or trying to convert any of their users, we're merely satisfying our own (As selfish as it sounds).
--- Richard Smith rsmith@bitworks.com wrote:
Frank wrote:
Hopefully by the time we decide to do cost reduced version, LinuxBios will be usable for the masses and not just for an elite group of people who assume everybody else uses the x86
for
everyday use.:-(
I think the above is a little harsh.
As long as I've been on this list LinuxBIOS has always been about getting dirty with the code and making it to do what you need it to do for your specific applicaiton. Along the way many good people have put a _lot_ of effort into trying to take all these individual needs and produce some sort of API that could be re-useable by the next person. But at this level that's really hard.
We've always known the level of documentation sucks. If you ask we will tell you it sucks. Just none of the people who really know the core have time to pull off and re-hash it.
From what it started with and considering the odds stacked against it LB has make _huge_ advancements. It x86 specific becuse that's what all the core developers use. For LinuxBios to make the jump to multi-platform it will take an investment of developers such as yourself getting dirty in the core and makeing it happen. I don't think any of the developers just "assume" the world uses x86. x86 just happens to be most of what thier world is right now.
I understand your time requirements prevented you from jumping in on this project but until _someone_ takes the initiative and brings LB forward on non-x86 platforms it won't ever meet your "masses" criteria.
So I'd suggest that if you really wanted LB available with all the extra features for your platform you should revise your tone a bit. Whiners don't get the best of support from coders.
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
* Greg Watson gwatson@lanl.gov [040608 18:33]:
PCI device code and resource information should be available for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
I fully agree. That information should go to the LB table. OTOH it would be a nice option to leave the whole pci device probing away if one happens to have a payload that does this already.
Stefan
Stefan Reinauer stepan@openbios.org writes:
- Greg Watson gwatson@lanl.gov [040608 18:33]:
PCI device code and resource information should be available for payloads to use. A payload should not have to re-probe for devices on the PCI bus.
I fully agree. That information should go to the LB table.
That has been the tentative plan for quite a while. So far I have just had more pressing concerns and no immediate need so I haven't implemented it.
OTOH it would be a nice option to leave the whole pci device probing away if one happens to have a payload that does this already.
Probing is not the interesting thing in LinuxBIOS the interesting thing is resource allocation. And with OpenFirmware doing it I can see some point in not predoing it. But the pci enumeration is quite cheap so I don't see that it would be much of a problem, especially as it allows us to concentrate the board specific information in LinuxBIOS.
What is gained most by exporting this in the LinuxBIOS table is avoiding broken PCI devices. And possibly some information about non-standard PCI bars.
Eric
On 9 Jun 2004, Eric W. Biederman wrote:
Probing is not the interesting thing in LinuxBIOS the interesting thing is resource allocation. And with OpenFirmware doing it I can see some point in not predoing it. But the pci enumeration is quite cheap so I don't see that it would be much of a problem, especially as it allows us to concentrate the board specific information in LinuxBIOS.
The issue that concerns me is the "buggy hardware" issue, which has happened just often enough to be really painful. The worst-case was the VT8601, which if not handled properly would lock up the north bridge, or the Acer north bridge that also had a few tricks up its sleeve, and so on.
I can't expect every boot loader out there to get this one right. PCI enumeration is *almost* always simple and cheap.
ron
no, it's just that right now some payloads redo work that linuxbios already does. linuxbios walks the pci bus, then the payload does it. There's some issues here we need to work out.
I think with sufficient goodwill and willingness to compromise we can figure this out, so let's try to keep the discussion polite and technical.
thanks
ron
Greg Watson gwatson@lanl.gov writes:
Stefan,
I think this is a reasonable idea, particularly your suggestion of making linuxbios more modular. One of my main beefs with the payload strategy is that each payload has to provide it's own set of, potentially buggy, driver code. If we have 5 payloads then we have 5 sets of drivers that all do the same thing slightly differently. If the drivers were modular enough so that a payload could call them directly, then this would go a long way to addressing these concerns.
The historical perspective on this is interesting.
On the alpha there is/was a bootloader known as MILO. MILO reused kernel drivers directly from the kernel build tree. I think this was actually inspired by the design of the SRM as I have heard the SRM uses VMS drivers. Anyway in practices MILO was tied to a single kernel version from which it could uses drivers. In theory MILO was only tied to a kernel major version, but even switch to a kernel with a different minor version was a pain.
Then look at etherboot or oskit or u-boot, or the BSDs just about any project that has it's own set of native hardware drivers. They work and evolve and are not unmaintainable relics before their time. Between related projects you will often seen code sharing or being heavily influenced by other projects but never direct code reuse.
Given that code reuse is a mantra in some circles of the software world. That sharing fails and not sharing works appears non-intuitive. I can only guess about the reasons for this but here are a couple. Writing reusable generic code is at least 3X as difficult as writing code for a specific purpose, and most driver code is not designed to used directly in multiple projects. By not sharing code directly reasonable expectations about the difficulty of adding new features are set appropriately.
Another way of looking at it is that synchronization and coherency in the large is hard. Things scale better by having their own private resources instead of shared resources that can't scale. This appears to apply to software source bases as well as hard drives, memory, and the like.
Eric
Hello,
I found it clear but :)
2.) LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
Ok but How are you calling this if it's not a bootloader chaining or payload chain ?
LinuxBIOS - (ETHERBOOT - IDE PATCH) - LINUX
because this were made ... More, I've reported that is was and that this works. I've also reported some boot chains examples to illustrate a section :
LB - ETHERBOOT - ADLO - LILO - LINUX
What have i missed ?
mathieu
LinuxBIOS wants to keep up it's modularity, letting each module do it's job. This possibility of not doing more than one task should be passed on to the payloads.
* Mathieu Deschamps mdeschamps@mangrove-systems.com [040608 15:42]:
LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
Ok but How are you calling this if it's not a bootloader chaining or payload chain ?
LinuxBIOS - (ETHERBOOT - IDE PATCH) - LINUX
I don't exactly get the problem. Etherboot would probably be the end of the in-rom payload chain, since it does not return from execution but pass control to Linux, which is not stored in ROM in the above scenario.
I've reported that is was and that this works. I've also reported some boot chains examples to illustrate a section :
LB - ETHERBOOT - ADLO - LILO - LINUX
Again, at this point Etherboot would be the end of the in-rom payload chain. But possibly ADLO could be moved to ROM before Etherboot is started.
Stefan
On Tue, 8 Jun 2004, Stefan Reinauer wrote:
2.) LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
Sounds good. How would you suggest bringing up payloads in a certain order? Say you wanted a payload to boot a SCSI controller and then another payload to boot off a SCSI disk? You'd need a certain order, and you'd have to make certain one payload didn't wipe out another.
There are also instances where one might wish to keep the payload in tact even after the operating system has booted, such as using OpenBIOS as a LinuxBIOS payload.
Thoughts?
On Tue, 8 Jun 2004, Stefan Reinauer wrote:
Feature plugins like "testbios" should not be merged into LinuxBIOS' core code. Instead they should be implemented as a payload. Since we
What about VGA, which requires testbios? If it were to be run strictly as a payload, we would have practically no useful BIOS information display on the screen before the bootloader or Linux comes up.
(I have a feeling I missed the point, again)
* Hendricks David W. dwh@lanl.gov [040608 21:48]:
On Tue, 8 Jun 2004, Stefan Reinauer wrote:
LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
Sounds good. How would you suggest bringing up payloads in a certain order? Say you wanted a payload to boot a SCSI controller and then another payload to boot off a SCSI disk? You'd need a certain order, and you'd have to make certain one payload didn't wipe out another.
The order could be coded in the config file. Scenarios with getting this from cmos could exist, but I think the order is pretty constant
There are also instances where one might wish to keep the payload in tact even after the operating system has booted, such as using OpenBIOS as a LinuxBIOS payload.
In such a case, the payload should take care of staying resident itself. Everything else would probably require callbacks.
Feature plugins like "testbios" should not be merged into LinuxBIOS' core code. Instead they should be implemented as a payload. Since we
What about VGA, which requires testbios? If it were to be run strictly as a payload, we would have practically no useful BIOS information display on the screen before the bootloader or Linux comes up.
Currently, with the ati rage xl graphics card, you won't get any output before pci resource allocation is done and the init drivers get executed If PCI resource allocation would be a seperate payload, then testbios could also run directly after that.
Stefan
On Tue, Jun 08, 2004 at 12:49:38PM +0200, Stefan Reinauer wrote:
Feature plugins like "testbios" should not be merged into LinuxBIOS' core code. Instead they should be implemented as a payload. Since we want to load an operating system later on, we also need to be able to load more than one payload. Implementing ELFLOAD in each such plugin is inadequate. Instead LinuxBIOS should execute multiple payloads the same way it executes multiple "drivers" now.
Hi Stefan,
If I understand your proposal correctly, one of the major issues will be ensuring that the payloads don't step on each other or linuxbios when they run.
Wouldn't it be simpler to just link linuxbios and all the payloads together at compile time? Essentially, that is what your proposal is doing, except it does it at run time.
So, couldn't an __init tag be created (a la linux) that marks functions to be run prior to the bootloader. With this feature, modules could then be created entirely separate from the linuxbios core that just define a single __init entry point. A simple post-build script could then collect all the user's modules, scan them for __init functions, create a stub function that executes the __init functions, and then link the whole mess into the final linuxbios target..
The above seems functionally equivalent to me, but conceptually simpler; did I miss something? -Kevin
This is a reply to myself.
On Thu, Jun 10, 2004 at 12:09:05AM -0400, Kevin O'Connor wrote:
So, couldn't an __init tag be created (a la linux) that marks functions to be run prior to the bootloader. With this feature, modules could then be created entirely separate from the linuxbios core that just define a single __init entry point. A simple post-build script could then collect all the user's modules, scan them for __init functions, create a stub function that executes the __init functions, and then link the whole mess into the final linuxbios target..
Sorry, I meant s/__init/module_init/
* Kevin O'Connor kevin@koconnor.net [040610 06:09]:
If I understand your proposal correctly, one of the major issues will be ensuring that the payloads don't step on each other or linuxbios when they run.
Yes, mostly. They must not overwrite LinuxBIOS in memory. And there needs to be a mechanism within linuxbios that allows to execute many of them.
Wouldn't it be simpler to just link linuxbios and all the payloads together at compile time? Essentially, that is what your proposal is doing, except it does it at run time.
It actually would be simpler, yes. But as far as I understood the common discussion here, people prefer having a strict border between linuxbios and it's payloads.
So, couldn't an __init tag be created (a la linux) that marks functions to be run prior to the bootloader. With this feature, modules could then be created entirely separate from the linuxbios core that just define a single __init entry point. A simple post-build script could then collect all the user's modules, scan them for __init functions, create a stub function that executes the __init functions, and then link the whole mess into the final linuxbios target..
The above seems functionally equivalent to me, but conceptually simpler; did I miss something?
Functionally it is equivalent, yes.
Stefan
- Kevin O'Connor kevin@koconnor.net [040610 06:09]:
If I understand your proposal correctly, one of the major issues will be ensuring that the payloads don't step on each other or linuxbios when they run.
Yes, mostly. They must not overwrite LinuxBIOS in memory. And there needs to be a mechanism within linuxbios that allows to execute many of them.
but ADLO does just exactly that... but then ADLO is an exception not the rule here... necessiated by support of Legacy Applications.
Adam
On Fri, 2004-06-11 at 06:27, Stefan Reinauer wrote:
- Kevin O'Connor kevin@koconnor.net [040610 06:09]:
If I understand your proposal correctly, one of the major issues will be ensuring that the payloads don't step on each other or linuxbios when they run.
Yes, mostly. They must not overwrite LinuxBIOS in memory. And there needs to be a mechanism within linuxbios that allows to execute many of them.
Wouldn't it be simpler to just link linuxbios and all the payloads together at compile time? Essentially, that is what your proposal is doing, except it does it at run time.
(What I am going to say proabably will offense a lot of people. I have to say sorry in advance.)
The reason we are having this discussion is due to some POLICY, someone does not want these MECHANISMs to be linked with the CORE LinuxBIOS to jeopardize the PURITY of it. The only way to hold this POLICY and to maintain the PURITY is to seperate these MECHANISMs out and use ELFLOAD as firewall.
This project/software is going to be exactly as what I predicted a few years ago: it is going to be an OS. Well, it is not that bad as the original idea of LinuxBIOS is to use an OS to boot an OS. The question is who is who and which is which. The discussion I have read so far can be categorised into these two:
1. Monolithic v.s. Micro Kernel - do we want to put and link everything into one program or as seperated PAYLOADs (processes/servers?) and use some complicated Inter Payload Communiction (IPC) to exachange information and reuse implementation?
2. Reimplementing an OS called DOS - I read topics about the order of loading PAYLOADs and how or should they overlay each other. I also found people talking about something as Terminated but Stay Resident (TSR). Sooner or later we will face the problem of 640K and the solution would be HIMEM.SYS.
Before we going to discuss this any further we should ask ourself, do we really want to go this way? History probably repeat it self or not, but do we want to repeat the history?
Ollie
I agree with what is said here, except if I may...
I need to react to your questions and This topic is NOT a question of purity of code or of something nor only a policy... Brief : this is a stacke...
But let admitting that the question of extending the payload handling seems central and IMHO Linuxbios cannot spare this evolve.
This still lives, provided it doesn't jeopardize theses fondations (and i think these are still goals):
-LinuxBios is govern by the less possible code writing (letting Linuxbios be truely an OS would neck twist this at very short term)
-LinuxBios aims no specific target or rather potentially everyone (architecture as application field of this technology)
-LinuxBios is to answer to the lack of 'modernity' of common BIOS, to answer to nowdays applications, theses BIOS :
-support old DOS fonctions we don't want, theses are useless for non-DOS-based OS, and they are more and more present. (again shall we, not to redo what DOS have done and now we don't need nor want ?)
-are based upon the executant/orderer scheme as mirror reflection to the master/slave scheme of the hardware controler, this approch leds to ennoying limitations and has rise norms and rules which asserves rather than they freed.
This scheme *was* logical, and inspires straight in the line the old net working style, in the same relationship of communication.
Since the begining, 20 years ago, the interconnection of net , changed that old scheme to new one though at this time they couldn't figure how up this could led (or maybe could they)
My questions:
Would we return that new scheme to the oldiest part of a computer ? (further in) Is this good to be inspired by that scheme and return it back in the opposite way out ?
Considering this and letting myself, just for the fun, being affimative twice here is my reflexion :
With the networking growth and the all-communicative state of the demand and its consequence, interoperabilty; can't LinuxBios implements an upper abstraction so to take over the forsaid scheme and answer in the whole to that demand ?
Precisely, That demand is wide spread and, to get Linuxbios customized to this or that type of application (whether it's for clustering or embedding or whatever) without requiering week of port or customing is impossible as it goes.
There is another way, to the 2 mentionned there after (fore admitting the second is a one :)
As payload can be distant or local, why not handling payloads as modern network handles packets ? I know this could sound wierd or crazy but to my sense it is not.
and I too, have to say sorry if I confused the thing, but i feel like this needed to be said.
mathieu.
Le ven 11/06/2004 à 18:24, Li-Ta Lo a écrit :
On Fri, 2004-06-11 at 06:27, Stefan Reinauer wrote:
- Kevin O'Connor kevin@koconnor.net [040610 06:09]:
If I understand your proposal correctly, one of the major issues will be ensuring that the payloads don't step on each other or linuxbios when they run.
Yes, mostly. They must not overwrite LinuxBIOS in memory. And there needs to be a mechanism within linuxbios that allows to execute many of them.
Wouldn't it be simpler to just link linuxbios and all the payloads together at compile time? Essentially, that is what your proposal is doing, except it does it at run time.
(What I am going to say proabably will offense a lot of people. I have to say sorry in advance.)
The reason we are having this discussion is due to some POLICY, someone does not want these MECHANISMs to be linked with the CORE LinuxBIOS to jeopardize the PURITY of it. The only way to hold this POLICY and to maintain the PURITY is to seperate these MECHANISMs out and use ELFLOAD as firewall.
This project/software is going to be exactly as what I predicted a few years ago: it is going to be an OS. Well, it is not that bad as the original idea of LinuxBIOS is to use an OS to boot an OS. The question is who is who and which is which. The discussion I have read so far can be categorised into these two:
- Monolithic v.s. Micro Kernel - do we want to put and link
everything into one program or as seperated PAYLOADs (processes/servers?) and use some complicated Inter Payload Communiction (IPC) to exachange information and reuse implementation?
- Reimplementing an OS called DOS - I read topics about the
order of loading PAYLOADs and how or should they overlay each other. I also found people talking about something as Terminated but Stay Resident (TSR). Sooner or later we will face the problem of 640K and the solution would be HIMEM.SYS.
Before we going to discuss this any further we should ask ourself, do we really want to go this way? History probably repeat it self or not, but do we want to repeat the history?
Ollie
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
* Mathieu Deschamps mdeschamps@mangrove-systems.com [040614 11:07]:
There is another way, to the 2 mentionned there after (fore admitting the second is a one :)
As payload can be distant or local, why not handling payloads as modern network handles packets ? I know this could sound wierd or crazy but to my sense it is not.
This sounds interesting. How would that work? It's possible to load remote payloads with etherboot. Is that what you mean?
and I too, have to say sorry if I confused the thing, but i feel like this needed to be said.
I appreciate your sometimes flowery way of describing things even if it is hard for me to follow (which is due to my lack of french knowledge, when it comes down to your fine document)
Stefan
Le dim 20/06/2004 à 18:23, Stefan Reinauer a écrit :
- Mathieu Deschamps mdeschamps@mangrove-systems.com [040614 11:07]:
There is another way, to the 2 mentionned there after (fore admitting the second is a one :)
As payload can be distant or local, why not handling payloads as modern network handles packets ? I know this could sound wierd or crazy but to my sense it is not.
This sounds interesting. How would that work? It's possible to load remote payloads with etherboot. Is that what you mean?
Brief State: No but that's quite in the view, like the a modular approch.
Complete State: IMHO, with what i've seen and read, I consider not the payload simply as just an OS ...and well that the topic : extended payload handling.
So maybe you'll say that now i've seek to far in the way. Ok, that makes some reading but :
-It is wide scaled and borrows from the good suggestions/needs of every one.
-This design is really worthy when networking and if local,it needs some sort of loopback. But it is admitted it networks.
-So... not oriented... but in favor of clustering and embedding rather than others : anyway theses are the main application I think ?
Let's move in deeper:
It seems clear it to clarify the payload thing, even before saying we are going to extend the payload handling.
During my works i couln't find clearly a definition of payload or rather limitations. If I digested it well, actually, a payload is a polymorph container of what is not Linuxbios-seemingly.
Moreover, it seems underway, drafts making almost everything a payload, there is pro and cons, but i like this idea, suggest vastness...
Generally speaking, the good side is a part of project that's not defined, has illimited lifetime, because it'll morph and adapt to current needs.
The bad is in the pratical coding and designing, that swinging nature pouring into this needs/fonctions or that needs/fonctions leads to unconstance properties, and this undef is bad to 'computing' common sense :). It has to have some stablity to prevent deprecation.
The payload is some kind of concept that needs definition, to seize its properties to fix them down. Ok enough foresaying, some of my proposition are truely science fiction rather than operationnal, but other don't.
DRAFT EXTENDED PAYLOAD
I see four... five properties for now : 1- Where/Spread : internal, local or distant. its nature is pure internal to linuxbios or is local to machine or has networking capacities (single hardware component sharing : realtime payload servers)
2- Who/Nature : OS, BOOTLOADER or Core Program. A core program could be a bios hardware component init. code (linuxbios components) or to-end-finality program like a securisation/authentification protocole before booting OS.
3- When/How long/Lifetime: *Once : instant use and kill and overwrite memory, *TSR : instant use and teminated but memory reserved, *Illimited : a real bios service (e.g.IRQ) always alife, memory reserved 4- How/Ressources : pure volatile, LB tables, distant. 5- How deep/Intensity : raw i/o or bios init
This properties combines to will to suit every needs.
Basically, as suggested Ollie there is a Kernel, rather a core and...
' - do we want to put and link everything into one program or as seperated PAYLOADs (processes/servers?) and use some complicated Inter Payload Communiction (IPC) to exachange information and reuse implementation?'
... Yes shortly, that's what i like.
It needs IPC or rather a abstract payload layer with a somehow communication based upon the Linuxbios Table. Not as heavy as IPC but i feel no shame getting inspired by that strong accredited fondation?
The core has a programmed (boot sequence) tasklist definied for example in the config file (with a somewhat order). So basically, you'll asked the core to wait uppon the arrival of the payload containing the code that initialize the controler, that boot the OS, or that is the OS.
Whether it come from distant packets sendings or local. Ok for now it is pure local-local... But in the near future ? bootloader/Os chain in Etherboot is already distant and that a clue. The paylaod architecture i propose should thus integrate this idea.
Today's Pros : some piece modular, somewhat easy to set but hard to port 'Potential' Pros : interoperability, self-determined, 'intelligence'.
Look by the time the Real Time Clock is set, it could know, after a defined timeout has expired (because it cannot do it with local ressource), that this core componant (core program ?) it tries to initialize, has been done, onto another server.
Ok that lead to some fiction but that we are not that far of... so maybe if the lan controler is init, and inter lxbios request layer (to my mind here would be the real comparison with IPC) is set it could ask to the foresaid server some assistance ? Why not reduce port to the maximun and centalized it in the core ?
But back to ground, the core indeed should contain memory managment, cpu managment and basic io to pci bridge, the rest should be modules that are loaded upon linuxbios config file tweakings.
Truely (a little) less that LB's core part does for now, so that there would be low in-trash and full reuse and encapsulation. And the core will have hat abstract payload layer that understand those declinaisons.
Fo examples, some scenarii and their corresponding attributions : *You like your Linuxbios to initialize your SCSI controler : 1- Internal (a Linuxbios component) 2- Core Program (a module) 3- Once (to boot os and then the kernel redo SCSI) 4- volatile (here because once) 5- bios init *You like it to init your graph chipset in order to display a linuxbios splashscreen. 1- local 2- Core Program 3- Once/TSR depending what OS will set then 4- volatile/LB table 5- raw i/o or bios full init *You like to choose etherboot to be you bootloader it is 1- distant 2- bt ldr 3- depends if you what to have some session like handling but normally it should be Once 4- LB table/distant 5- raw i/o or bios init
ok, anyway it is a draft... again that is long ...
mathieu.
and I too, have to say sorry if I confused the thing, but i feel like this needed to be said.
I appreciate your sometimes flowery way of describing things even if it is hard for me to follow (which is due to my lack of french knowledge, when it comes down to your fine document)
ha ah.. Thanks.. i'am part of people that thinks, 'smiling' while reading a 90 pages long technical document is not a luxury... it is a compulsary ;)
Stefan
Li-Ta Lo ollie@lanl.gov writes:
On Fri, 2004-06-11 at 06:27, Stefan Reinauer wrote:
- Kevin O'Connor kevin@koconnor.net [040610 06:09]:
If I understand your proposal correctly, one of the major issues will be ensuring that the payloads don't step on each other or linuxbios when they run.
Yes, mostly. They must not overwrite LinuxBIOS in memory. And there needs to be a mechanism within linuxbios that allows to execute many of them.
Wouldn't it be simpler to just link linuxbios and all the payloads together at compile time? Essentially, that is what your proposal is doing, except it does it at run time.
(What I am going to say proabably will offense a lot of people. I have to say sorry in advance.)
The reason we are having this discussion is due to some POLICY, someone does not want these MECHANISMs to be linked with the CORE LinuxBIOS to jeopardize the PURITY of it. The only way to hold this POLICY and to maintain the PURITY is to seperate these MECHANISMs out and use ELFLOAD as firewall.
Exactly whenever you change the interface as visible to the outside world of a piece of software you need to examine very closely what you are doing and to discuss it. Things are much more serious, and the consequences are much more significant than a purely internal change.
Especially something like firmware that people are reluctant to touch after it works.
This project/software is going to be exactly as what I predicted a few years ago: it is going to be an OS. Well, it is not that bad as the original idea of LinuxBIOS is to use an OS to boot an OS. The question is who is who and which is which. The discussion I have read so far can be categorised into these two:
- Monolithic v.s. Micro Kernel - do we want to put and link
everything into one program or as seperated PAYLOADs (processes/servers?) and use some complicated Inter Payload Communiction (IPC) to exachange information and reuse implementation?
- Reimplementing an OS called DOS - I read topics about the
order of loading PAYLOADs and how or should they overlay each other. I also found people talking about something as Terminated but Stay Resident (TSR). Sooner or later we will face the problem of 640K and the solution would be HIMEM.SYS.
Before we going to discuss this any further we should ask ourself, do we really want to go this way? History probably repeat it self or not, but do we want to repeat the history?
I don't think we want to becoming an OS.
History happens to be easy to repeat because people are familiar and comfortable with it.
From what I can tell all of the messy corner cases come from people
wanting to implement some form of code reuse. And to reuse code things must share and cooperate.
I will argue that if want code reuse someone should be a library of the functions that will be reused. Then we don't need multiple PAYLOADs and all of the associated nastiness. Plus the act of putting the code in a library and cleaning it up so it can live in one is what is needed to actually make the code reusable.
The current architecture of LinuxBIOS (not counting the weird FILO thing) works and does not leave us implementing an OS. As we approach being feature complete and quite general we accumulate a lot of code. But largely it is conditional upon what hardware actually exists on a motherboard.
Another issue on this topic is what interface do we provide to the outside world to people who want to customize LinuxBIOS. The reason I am largely in favor of testbios, openbios, ADLO and the Linux kernel is those are interfaces that already exist. Commodity interfaces you might call them. We don't have to convince hardware manufacturers or software developers to support something new. That makes the option rom problem easier.
For embedded developers the question is what will work best for their hardware and use the least amount of resources (to keep the price down) while doing so. So far LinuxBIOS has kept itself inside of 64K for x86 which is doing pretty good on general purpose server boards.
As best I can tell there are two primary target configurations for LinuxBIOS. General purpose firmware that can do a lot of things. Embedded systems where things are well enough understood you can directly load your final image from flash.
Except for a few issues like romcc being a code size pig I think we are ok on the minimal configurations. For the general purpose default configuration/configurations we still have a long ways to go. Etherboot when I proposed it was a stop gap that worked. And we could use it immediately because of it's size.
Etherboot does not currently do everything we need. It works but is lousy at booting off of disks, for example. There are a lot of ways forward including enhancing etherboot, enhancing filo, or finally getting the Linux kernel small enough we can use it.
So far I have been buried alive under the task of catching with the latest and greatest chipsets and doing all of the required ports. I have not had much time to work on the bootloader problem recently. I want people to understand the reason I have not followed through on my ideas have been scheduling conflicts and not because of any intrinsic problems with the ideas. Except possibly the romcc size optimizations :)
Eric
2.) LinuxBIOS can currently execute one payload. For greater flexibility and isolated development cycles of the firmware related code parts/projects LinuxBIOS should allow payload chains, ie. executing multiple payloads one after the other.
LinuxBIOS wants to keep up it's modularity, letting each module do it's job. This possibility of not doing more than one task should be passed on to the payloads.
Stepping back a moment. I see the line at elfload is that the hardware is sufficiently initialized that we can run stand-alone programs.
Motherboard specific information should be passed between the BIOS and the OS which allows a general purpose OS to be built that does not need to know how each chip is hooked up on each motherboard.
The only argument I have seen for payload chaining is testbios. I do not see a reason to exclude this from LinuxBIOS. It fits in nicely as an init method for a generic pc compatible vga device. As long as this code can be compiled out for those boards that don't have a slot you can plug in a vga compatible card into all is good.
As the discussion has shown multiple payloads are one form of runtime service and we once we add one a whole lot more are wanted.
....
My take on runtime services is if they are provided at all they belong in their own personality/bootloader. If you want the ability to switch personalities it is simple enough to add an elf loader to load other personalities.
Eric
Stefan Reinauer stepan@openbios.org writes:
There has been a similar proposal on this list a while ago, but nothing happened so far, so I want to put this pack to discussion.
Objectives
1.) LinuxBIOS is kind of hard to set up for project newbies, since it does not only require manually tweaking the configuration files for basically every situation, but also necessarily needs an external payload to do anything useful. LinuxBIOS currently sets a high barrier due to the modular concepts it uses.
- LinuxBIOS itself is sometimes very sensitive to the build environment. See requirement for setting LANG for example.
? The only time I have heard this is with buggy perl when building etherboot.
- For a project outsider it is hard to determine the best payload solution for a specific purpose. There is basically no information except the mailing list archives.
That are documentation is sketchy is a problem. Half the problem is that many of the payloads are not terribly mature. ADLO for example has to be hacked for each board for example. That can only be fixed by fixing the payloads.
- Config files have to be tweaked to explicitly suit the user's directory structure. There is no proposal, nothing that works out of the box. One just _has to_ edit the config files.
Agree that is an issue.
- LinuxBIOS requires the user to specify a size for the payload instead of determining the required size and doing the needed calculations itself. This is very hand-crufted and can be pretty mind wasting.
I sort of agree. This is more a matter of reserving a whole you can place a payload in. Ideally the payloads can be flashed independently. In any case this should be once per board port, and not something the user needs to mess with.
And a couple more.
- Too many linuxbios options need to be specified in the top level configuration file, or they don't work.
- We don't have stable releases. You get snapshot of they day instead.
- LinuxBIOS therefore needs a way to automatically determine payload sizes when building the image and later when executing payloads. Hardcoding the size values in the config files is inadequate and will lead to unnecessary overhead
There is certainly an issue with flash device layout here. But I don't think it is quite as simple as your intuition.
- LinuxBIOS should, to be consequent, remove all streaming code except CONFIG_ROM_STREAM
On some platforms like the alpha after you have loaded yourself from an srom. Or if your primary ROM chip is a Disk-On-Chip that you need code for reading a ROM. The IDE driver is certainly interesting.
- Payloads should have the possibility to add their own enhancements to the LinuxBIOS table.
They do they can rewrite it.
- A least one payload should be "default payload" with the possibility to build this automatically and link it into the image.
To make things easier I largely agree.
This is why I checked the "util/extensions" directory into v2 during the last discussion. It should hold possible choices to payloads that can automatically be built and included. Potentially one could add more payloads by symlinking their source tree to this directory to make it available to LinuxBIOS without major reconfiguration. People feel a lot safer with creating a symlink than with changing config files they do not fully understand.
Likely but....
Since these can later be executed in row by elfboot, the minimum overhead design of LinuxBIOS itself will not be hurt.
At this point I want to put an idea to discussion: If we are going more and more modular and some of us think the current tree is too bloated: Why do we not modularize code like pci resource allocation into a payload as well. My favorite bootloader may already do this and I can't stand this bloat everywhere, you know ;) Even though this may sound funny, I am serious about this issue. I do not see why allocating PCI resources should really be part of the lowlevel code, except for the fact that the NEXT payload in row, potentially Linux itself is too stupid to do that. Bummer.
You cannot use hardware unless it has been allocated resources. You cannot do device resource allocation except by looking at all devices. So largely this is a catch 22 type situation.
The problem with implementing device resource allocation in Linux is that it requires motherboard specific knowledge. Frequently onboard devices have BARs in non-standard locations, the kernel does not know about, or there are non-enumerable devices like sio chips that have resources as well.
If a general purpose way of exporting the motherboard specific information (which devices are present and how they are hooked up) can be developed. And we can teach Linux to use it. Possibly making it a CONFIG_EMBEDDED option. I have no problem with pushing out most of the code and letting Linux handle it. And I am even willing to help with that as time permits. But until such time as we can reset OS's expectations of what to expect from the hardware/firmware combination I am not ready to remove that code.
Beyond that the pci enumeration code is totally optional. The mainboard code just needs not to call it.
There are couple of sticky points I see. On boxes like the Opteron I have a hard time seeing how to setup something that is reasonably sane with hypertransport routing without find which resources are needed by the devices. I also have a hard time seeing how much memory you can report and how to set the memory BARs without knowing how large your memory mapped pci resources are.
Absolute minimal size and trivial configuration are conflicting goals. Getting things as small as possible will require some serious user tweaking of configuration and possibly likely even the code. For example to remove the calls into the pci configuration code requires changing the motherboard code.
LinuxBIOS configuration should have an easier mechanism for choosing payloads from the "default" directory, allowing them to be built automatically. Right now I am doing:
cd filo-0.4.2 linux32 make CC="gcc -m32" LD="ld -melf_i386" cd .. freebios2/targets/buildtarget amd/solo $PWD/freebios2 cd build-solo make cd .. cp build-solo/solo.rom .
My target Config.lb comes with these constructs: target ../../../../build-solo payload ../../filo-0.4.2/filo.elf
So I build everything completely out of the freebios2 tree, because building in-tree sucks. The only thing left is to get filo and the other payloads to build out-of-tree as well.
Hmm. I always have payload set to something like:
payload ../../bootloader.ebi
So the symlink part comes naturally.
Given that building out of the tree is our encouraged state. What we likely need to do is to actually have something like configuration that sets up the environment for the user.
That way it can be as simple as: ../freebios2/configure --mainboard=arima/hdama make
What I should probably do is dig out the LinuxBIOS source rpms that I use and see if there is something that the community can adopt from them.
Eric
On 14 Jun 2004, Eric W. Biederman wrote:
- Too many linuxbios options need to be specified in the top level configuration file, or they don't work.
yes, that's on my list too.
- We don't have stable releases. You get snapshot of they day instead.
yes.
I'm not so sure I get this 'bloated' discussion. LinuxBIOS is around 32K. What's bloat? linuxbios overall is about the size of 'cat'.
ron
On Mon, 14 Jun 2004, ron minnich wrote:
I'm not so sure I get this 'bloated' discussion. LinuxBIOS is around 32K. What's bloat? linuxbios overall is about the size of 'cat'.
Closer to dog, actually :) http://jl.photodex.com/dog/dog-1.7.tar.gz
On Mon, 14 Jun 2004, Hendricks David W. wrote:
On Mon, 14 Jun 2004, ron minnich wrote:
I'm not so sure I get this 'bloated' discussion. LinuxBIOS is around 32K. What's bloat? linuxbios overall is about the size of 'cat'.
Closer to dog, actually :) http://jl.photodex.com/dog/dog-1.7.tar.gz
oh, I don't know david :-)
-rwxr-xr-x 1 root root 19154 Jul 1 2002 /bin/cat
ron
ron minnich rminnich@lanl.gov writes:
On 14 Jun 2004, Eric W. Biederman wrote:
I'm not so sure I get this 'bloated' discussion. LinuxBIOS is around 32K. What's bloat? linuxbios overall is about the size of 'cat'.
Ron can I have you version of romcc?. We have several ports that are routinely smacking up against the 64K limit during development. The last E7501 port I did in the freebios1 tree LinuxBIOS was running about 40K. With romcc inlining everything we are worse in the freebios2 tree.
As far as bloat baring Stefan's concerns about redundancy between LinuxBIOS and OpenBIOS (the resource allocation code). Is not so much about the features we have to day, but more a feeling much more will be the pebble that starts the avalanche. Or maybe we feel we have seen the pebble, and we are watching it bounce downhill with dread.
The bottom line is that to keep size under control you constantly have to be paying attention to size, just as you have to constantly watch correctness and performance. The long term trends in the industry are for very slow romsize growth.
Anyway even at 32K you are twice as big as my cat :) ls -l /bin/cat -rwxr-xr-x 1 root root 15000 2003-10-04 17:10 /bin/cat*
Eric
-
Adam Sulmicki -
ebiederman＠lnxi.com -
Frank -
Greg Watson -
Hendricks David W. -
Kevin O'Connor -
Li-Ta Lo -
Mathieu Deschamps -
Peter Stuge -
Richard Smith -
ron minnich -
Stefan Reinauer