[LinuxBIOS] Compressing the OLPC VSM?

[Carl-Daniel Hailfinger]

Hi,

the VSM Ron mailed yesterday to olpc-devel can be compressed
very well to under 32 kb.

65536 vsa_test.64k-0001.bin
35587 vsa_test.64k-0001.bin.nrv2b
32143 vsa_test.64k-0001.bin.lzma

Is the VSM executed after RAM is there and can we use compression
for it? 50% savings are really good.

Regards,
Carl-Daniel

[Ronald G Minnich]

Carl-Daniel Hailfinger wrote:
> Hi,
>
> the VSM Ron mailed yesterday to olpc-devel can be compressed
> very well to under 32 kb.
>
> 65536 vsa_test.64k-0001.bin
> 35587 vsa_test.64k-0001.bin.nrv2b
> 32143 vsa_test.64k-0001.bin.lzma
>
> Is the VSM executed after RAM is there and can we use compression
> for it? 50% savings are really good.
>
> Regards,
> Carl-Daniel

yes, this is great. I want to move to the new compression asap.

ron

[Carl-Daniel Hailfinger]

Ronald G Minnich wrote:
> Carl-Daniel Hailfinger wrote:
> > Hi,
> >
> > the VSM Ron mailed yesterday to olpc-devel can be compressed
> > very well to under 32 kb.
> >
> > 65536 vsa_test.64k-0001.bin
> > 35587 vsa_test.64k-0001.bin.nrv2b
> > 32143 vsa_test.64k-0001.bin.lzma
> >
> > Is the VSM executed after RAM is there and can we use compression
> > for it? 50% savings are really good.
>
> yes, this is great. I want to move to the new compression asap.

See my other patch to the list which adds the decompression code
and infrastructure. I'm preparing another patch which will make
use of all that code. The patches I sent until now do not modify
any existing code, so they should be safe to apply.

Regards,
Carl-Daniel

P.S. I have a script which will figure out the best compression
possible with lzma if you give it the memory scratch pad size
as an argument. This can yield up to 2% additional compression.

[Stefan Reinauer]

* Carl-Daniel Hailfinger  [060523 19:52]:
> Ronald G Minnich wrote:
> > Carl-Daniel Hailfinger wrote:
> > > Hi,
> > >
> > > the VSM Ron mailed yesterday to olpc-devel can be compressed
> > > very well to under 32 kb.
> > >
> > > 65536 vsa_test.64k-0001.bin
> > > 35587 vsa_test.64k-0001.bin.nrv2b
> > > 32143 vsa_test.64k-0001.bin.lzma
> > >
> > > Is the VSM executed after RAM is there and can we use compression
> > > for it? 50% savings are really good.
> >
> > yes, this is great. I want to move to the new compression asap.
>
> See my other patch to the list which adds the decompression code
> and infrastructure. I'm preparing another patch which will make
> use of all that code. The patches I sent until now do not modify
> any existing code, so they should be safe to apply.

Can you try and see whether we can exchange the assembler nrv2b code 
in crt0.S.lb with something gcc compiled? This would make the whole
compression thing nicely pluggable..

Stefan

[Carl-Daniel Hailfinger]

Stefan Reinauer wrote:
> Can you try and see whether we can exchange the assembler nrv2b code 
> in crt0.S.lb with something gcc compiled? This would make the whole
> compression thing nicely pluggable..

Sorry, I'm swamped with university work right now. Maybe later.

Regards,
Carl-Daniel

[yhlu]

1. change crt0.s.lb to include another asm (set stack in ram)
and copy_and_run.asm produced by gcc.
2. you may need to use perl to replace some section name in copy_and_run.asm

look at cache_as_ram.inc and cach_as_ram_post.c to set the stack...

YH

On 5/23/06, Stefan Reinauer  wrote:
> Carl-Daniel Hailfinger  [060523 19:52]:
> > Ronald G Minnich wrote:
> > > Carl-Daniel Hailfinger wrote:
> > > > Hi,
> > > >
> > > > the VSM Ron mailed yesterday to olpc-devel can be compressed
> > > > very well to under 32 kb.
> > > >
> > > > 65536 vsa_test.64k-0001.bin
> > > > 35587 vsa_test.64k-0001.bin.nrv2b
> > > > 32143 vsa_test.64k-0001.bin.lzma
> > > >
> > > > Is the VSM executed after RAM is there and can we use compression
> > > > for it? 50% savings are really good.
> > >
> > > yes, this is great. I want to move to the new compression asap.
> >
> > See my other patch to the list which adds the decompression code
> > and infrastructure. I'm preparing another patch which will make
> > use of all that code. The patches I sent until now do not modify
> > any existing code, so they should be safe to apply.
>
> Can you try and see whether we can exchange the assembler nrv2b code
> in crt0.S.lb with something gcc compiled? This would make the whole
> compression thing nicely pluggable..
>
> Stefan

[jardel]

JCalg compression library is possible too, isn't ? It's open source and
the source is already assembly optimized. It was available
www.collakesoftware.com.

Thanks.

Jardel.

Ronald G Minnich wrote:
> Carl-Daniel Hailfinger wrote:
> > Hi,
> >
> > the VSM Ron mailed yesterday to olpc-devel can be compressed
> > very well to under 32 kb.
> >
> > 65536 vsa_test.64k-0001.bin
> > 35587 vsa_test.64k-0001.bin.nrv2b
> > 32143 vsa_test.64k-0001.bin.lzma
> >
> > Is the VSM executed after RAM is there and can we use compression
> > for it? 50% savings are really good.
> >
> > Regards,
> > Carl-Daniel
>
> yes, this is great. I want to move to the new compression asap.
>
> ron

[Ronald G Minnich]

jardel wrote:
> JCalg compression library is possible too, isn't ? It's open source and
> the source is already assembly optimized. It was available
> www.collakesoftware.com.

you mean it contains assembly? No assembly allowed. :-)

ron

[jardel]

Dear Mr. Ronald G Minnich,

It is. You're right. The algorthim is available in c too, but the
interesting is the assembly version which is assembly optmized.

Thanks.

Jardel.

Ronald G Minnich wrote:
> jardel wrote:
> > JCalg compression library is possible too, isn't ? It's open source and
> > the source is already assembly optimized. It was available
> > www.collakesoftware.com.
>
> you mean it contains assembly? No assembly allowed. :-)
>
> ron

[Carl-Daniel Hailfinger]

jardel wrote:
> JCalg compression library is possible too, isn't ? It's open source and
> the source is already assembly optimized. It was available
> www.collakesoftware.com.

How small does JCalg compress the VSA module at the link below?

http://laptop.org/pipermail/devel/attachments/20060521/e389e3ea/vsa_test.64k...
> > > 65536 vsa_test.64k-0001.bin
> > > 35587 vsa_test.64k-0001.bin.nrv2b
> > > 32143 vsa_test.64k-0001.bin.lzma

If it is better than the lzma result, we may consider it.
But IIRC the JCalg author said that lzma should be better.

Regards,
Carl-Daniel

[jardel]

This is the result for compressing the file vsa_test.64k-0001.bin in
1 to 9 level compressions of JCalg:

23/07/1999  15:29    <DIR>          .
23/07/1999  15:29    <DIR>          ..
23/05/2006  15:28            65.536 vsa.bin
23/07/1999  15:28            36.382 vsa1.jc
23/07/1999  15:28            35.858 vsa2.jc
23/07/1999  15:28            35.414 vsa3.jc
23/07/1999  15:28            35.158 vsa4.jc
23/07/1999  15:28            34.970 vsa5.jc
23/07/1999  15:28            34.970 vsa6.jc
23/07/1999  15:28            34.970 vsa7.jc
23/07/1999  15:29            34.970 vsa8.jc
23/07/1999  15:29            34.970 vsa9.jc

Thanks.

Jardel.

Carl-Daniel Hailfinger wrote:
> jardel wrote:
> > JCalg compression library is possible too, isn't ? It's open source and
> > the source is already assembly optimized. It was available
> > www.collakesoftware.com.
>
> How small does JCalg compress the VSA module at the link below?
>
> http://laptop.org/pipermail/devel/attachments/20060521/e389e3ea/vsa_test.64k...
> > > > 65536 vsa_test.64k-0001.bin
> > > > 35587 vsa_test.64k-0001.bin.nrv2b
> > > > 32143 vsa_test.64k-0001.bin.lzma
>
> If it is better than the lzma result, we may consider it.
> But IIRC the JCalg author said that lzma should be better.
>
> Regards,
> Carl-Daniel

[Carl-Daniel Hailfinger]

jardel wrote:
> Carl-Daniel Hailfinger wrote:
> > jardel wrote:
> > > JCalg compression library is possible too, isn't ? It's open source and
> > > the source is already assembly optimized. It was available
> > > www.collakesoftware.com.
> >
> > How small does JCalg compress the VSA module at the link below?
> >
> > http://laptop.org/pipermail/devel/attachments/20060521/e389e3ea/vsa_test.64k...
> > > > > 65536 vsa_test.64k-0001.bin
> > > > > 35587 vsa_test.64k-0001.bin.nrv2b
> > > > > 32143 vsa_test.64k-0001.bin.lzma
> >
> > If it is better than the lzma result, we may consider it.
> > But IIRC the JCalg author said that lzma should be better.
>
> This is the result for compressing the file vsa_test.64k-0001.bin in
> 1 to 9 level compressions of JCalg:
>
> 36382 vsa1.jc
> [...]
> 34970 vsa9.jc

OK, so it is worse than lzma. Thanks for checking that.

Regards,
Carl-Daniel

[Stefan Reinauer]

* Carl-Daniel Hailfinger  [060523 19:58]:
> http://laptop.org/pipermail/devel/attachments/20060521/e389e3ea/vsa_test.64k...
> > > > 65536 vsa_test.64k-0001.bin
> > > > 35587 vsa_test.64k-0001.bin.nrv2b
> > > > 32143 vsa_test.64k-0001.bin.lzma
>
> If it is better than the lzma result, we may consider it.
> But IIRC the JCalg author said that lzma should be better.

Just as a comparison:

34981 2006-05-22 07:25 vsa_test.64k-0001.bin.bz2

bzip2 is bigger than lzma here..

Regards.
Stefan