[LinuxBIOS] [FEATURE] payload compression

[Stefan Reinauer]

Hi,

while I've been working on FILO and OpenBIOS lately I really wished
that they could be used as nrv2b compressed payload, like its possible
with etherboot.

Since etherboot is probably not going to be developed beyond v5.5 and
because the zelf support in etherboot is really ugly, I thought of an
alternative to support compressed payloads easily.

So I wrote a new stream which can be enabled by using
CONFIG_COMPRESSED_ROM_STREAM instead of CONFIG_ROM_STREAM.

When you enable CONFIG_COMPRESSED_ROM_STREAM, the payload you specify
will completely automatically be compressed during the build stage, so
you don't have to do anything except safe space with a few small
changes.

Stefan

[yhlu]

sounds good.
you need to notice one thing...
check if it could cross the [0xa0000, 0xd0000)...

To make it easier, You may just set CONFIG_TOP_MEM_K to 4096, and
_RAM_START to 1M...

YH

On 4/29/06, Stefan Reinauer  wrote:
> Hi,
>
> while I've been working on FILO and OpenBIOS lately I really wished
> that they could be used as nrv2b compressed payload, like its possible
> with etherboot.
>
> Since etherboot is probably not going to be developed beyond v5.5 and
> because the zelf support in etherboot is really ugly, I thought of an
> alternative to support compressed payloads easily.
>
> So I wrote a new stream which can be enabled by using
> CONFIG_COMPRESSED_ROM_STREAM instead of CONFIG_ROM_STREAM.
>
> When you enable CONFIG_COMPRESSED_ROM_STREAM, the payload you specify
> will completely automatically be compressed during the build stage, so
> you don't have to do anything except safe space with a few small
> changes.
>
> Stefan

[yhlu]

the copy_and_run already have the same function that from nrv2b.c, So
we share the function...

YH

On 4/29/06, yhlu  wrote:
> sounds good.
> you need to notice one thing...
> check if it could cross the [0xa0000, 0xd0000)...
>
> To make it easier, You may just set CONFIG_TOP_MEM_K to 4096, and
> _RAM_START to 1M...
>
> YH
>
> On 4/29/06, Stefan Reinauer  wrote:
> > Hi,
> >
> > while I've been working on FILO and OpenBIOS lately I really wished
> > that they could be used as nrv2b compressed payload, like its possible
> > with etherboot.
> >
> > Since etherboot is probably not going to be developed beyond v5.5 and
> > because the zelf support in etherboot is really ugly, I thought of an
> > alternative to support compressed payloads easily.
> >
> > So I wrote a new stream which can be enabled by using
> > CONFIG_COMPRESSED_ROM_STREAM instead of CONFIG_ROM_STREAM.
> >
> > When you enable CONFIG_COMPRESSED_ROM_STREAM, the payload you specify
> > will completely automatically be compressed during the build stage, so
> > you don't have to do anything except safe space with a few small
> > changes.
> >
> > Stefan

[Stefan Reinauer]

* yhlu  [060429 23:41]:
> the copy_and_run already have the same function that from nrv2b.c, So
> we share the function...

Yes. I actually took it from copy_and_run.c.

Should we put it to lib/unrv2b.c or similar and reference it in both
places?

I would assume that we have to compile it twice in any case, as the one
time it is running in CAR/romcc context and the other time we're in
"late gcc" context.

Stefan

[yhlu]

Good.  lib/unrv2b.c and CAR could include it....

YH

On 4/29/06, Stefan Reinauer  wrote:
> yhlu  [060429 23:41]:
> > the copy_and_run already have the same function that from nrv2b.c, So
> > we share the function...
>
> Yes. I actually took it from copy_and_run.c.
>
> Should we put it to lib/unrv2b.c or similar and reference it in both
> places?
>
> I would assume that we have to compile it twice in any case, as the one
> time it is running in CAR/romcc context and the other time we're in
> "late gcc" context.
>
> Stefan

[Stefan Reinauer]

* yhlu  [060429 23:33]:
> sounds good.
> you need to notice one thing...
> check if it could cross the [0xa0000, 0xd0000)...

ah right, I did not think of that.

I originally rewrote the nrv2b decompressor so that it delivers 1 byte
per function call while not needing an additional decompression space.
But unfortunately this only seems to be possible with recursive calls
and it makes some 30000 recursive calls for decompressing a simple 
200byte makefile. So I figured the buffer is the better solution..
> To make it easier, You may just set CONFIG_TOP_MEM_K to 4096, and
> _RAM_START to 1M...

Why would I need to change CONFIG_LB_MEM_TOPK? It's 2048 by default, and
if it's kbytes and I put _RAM_START to 1M I'd still have a whole MByte
for the decompressed payload, or do I miss something?

Hm, trying to follow your suggestion while reading the code..

_RAMSTART vs _RAMBASE - What's the difference?

_RAMSTART is defined here, but never used anywhere in the whole source
code... Can we kill it, or is there some code in some queue who will
start using it?

src/config/Options.lb:define _RAMSTART
src/mainboard/totalimpact/briq/Options.lb:default _RAMSTART=0x00100000
src/mainboard/motorola/sandpoint/Options.lb:default _RAMSTART=0x00100000
src/mainboard/motorola/sandpointx3_altimus_mpc7410/Options.lb:default _RAMSTART=0x00100000

_RAMBASE is used in many places. I assume it is supposed to be the same
as _RAMSTART, but I am not sure.

Stefan

[yhlu]

Wait...You only need to set _RAMBASE to 1M..., and don't need to touch
CONFIG_MAX_TOP_MEM_K, because at the time all ram mtrr and etc are
initialized well...

Another problem put the payload on somewhere after may cause the
problem for final kerenal..., kernel need to start from 1M....?

So We may need to
1. _RAMBASE for linuxbios_ram ... from 1M or 0x4000
2. _PAYLOAD_BASE it should be somewhere safe for all...
a. sth place before 0xa0000, or b. last Mbytes before end of TOM...

payloads will be
1. Etherboot, zelf
2. FILO,
3. OpenBIOS
4. zelf from kernel and initram disk...less than 4Mbyte or 16Mbyte...

How about compress ratio for nrv2b?

YH

[Stefan Reinauer]

* yhlu  [060430 00:10]:
> payloads will be
> Etherboot, zelf
> FILO,
> OpenBIOS
> zelf from kernel and initram disk...less than 4Mbyte or 16Mbyte...
>
> How about compress ratio for nrv2b?

filo has 1:1.76, openbios even 1:2.12 due to the forth dictionary.
The compression rate is the same as zelf, as its the same compression
algorithm.

Stefan