[PROPOSAL] extended payload handling

[Stefan Reinauer]

There has been a similar proposal on this list a while ago, but nothing 
happened so far, so I want to put this pack to discussion.

Objectives
----------

1.) 
LinuxBIOS is kind of hard to set up for project newbies, since it 
does not only require manually tweaking the configuration files for
basically every situation, but also necessarily needs an external
payload to do anything useful.
LinuxBIOS currently sets a high barrier due to the modular concepts it
uses.

 - LinuxBIOS itself is sometimes very sensitive to the build
   environment. See requirement for setting LANG for example.

 - For a project outsider it is hard to determine the best payload
   solution for a specific purpose. There is basically no information
   except the mailing list archives.

 - Config files have to be tweaked to explicitly suit the user's
   directory structure. There is no proposal, nothing that works out of
   the box. One just _has to_ edit the config files.

 - LinuxBIOS requires the user to specify a size for the payload instead
   of determining the required size and doing the needed calculations
   itself. This is very hand-crufted and can be pretty mind wasting.
 
2.) 
LinuxBIOS can currently execute one payload. For greater flexibility and
isolated development cycles of the firmware related code parts/projects
LinuxBIOS should allow payload chains, ie. executing multiple payloads
one after the other.

LinuxBIOS wants to keep up it's modularity, letting each module do it's
job. This possibility of not doing more than one task should be passed 
on to the payloads. 


Solution
--------

Feature plugins like "testbios" should not be merged into LinuxBIOS'
core code. Instead they should be implemented as a payload. Since we
want to load an operating system later on, we also need to be able to
load more than one payload. Implementing ELFLOAD in each such plugin is
inadequate. Instead LinuxBIOS should execute multiple payloads the same
way it executes multiple "drivers" now.

 * LinuxBIOS therefore needs a way to automatically determine payload
   sizes when building the image and later when executing payloads.
   Hardcoding the size values in the config files is inadequate and will
   lead to unnecessary overhead

 * LinuxBIOS at runtime needs to go through the list of available
   payloads, either by having a linked list of payload start points
   or by scanning the flash rom.

 * LinuxBIOS should, to be consequent, remove all streaming code except
   CONFIG_ROM_STREAM

 * Payloads should have the possibility to add their own enhancements to 
   the LinuxBIOS table.

 * A least one payload should be "default payload" with the possibility 
   to build this automatically and link it into the image. 

   This is why I checked the "util/extensions" directory into v2 during
   the last discussion. It should hold possible choices to payloads that
   can automatically be built and included. Potentially one could add
   more payloads by symlinking their source tree to this directory to make
   it available to LinuxBIOS without major reconfiguration.
   People feel a lot safer with creating a symlink than with changing
   config files they do not fully understand.

   Since these can later be executed in row by elfboot, the minimum
   overhead design of LinuxBIOS itself will not be hurt.

   At this point I want to put an idea to discussion: If we are going
   more and more modular and some of us think the current tree is too 
   bloated: Why do we not modularize code like pci resource allocation
   into a payload as well. My favorite bootloader may already do this 
   and I can't stand this bloat everywhere, you know ;) Even though this
   may sound funny, I am serious about this issue. I do not see why 
   allocating PCI resources should really be part of the lowlevel code,
   except for the fact that the NEXT payload in row, potentially Linux
   itself is too stupid to do that. Bummer.

 * LinuxBIOS configuration should have an easier mechanism for choosing 
   payloads from the "default" directory, allowing them to be built
   automatically. Right now I am doing:

     cd filo-0.4.2
     linux32 make CC="gcc -m32" LD="ld -melf_i386"
     cd ..
     freebios2/targets/buildtarget amd/solo $PWD/freebios2
     cd build-solo
     make
     cd ..
     cp build-solo/solo.rom .

   My target Config.lb comes with these constructs:
     target  ../../../../build-solo
     payload ../../filo-0.4.2/filo.elf

   So I build everything completely out of the freebios2 tree, because
   building in-tree sucks. The only thing left is to get filo and the
   other payloads to build out-of-tree as well.
   
Comments?
Discussion?
Assassination attempts?

     Stefan