On Tue, Apr 10, 2012 at 17:19, Mark Cave-Ayland mark.cave-ayland@ilande.co.uk wrote:
On 10/04/12 17:43, Blue Swirl wrote:
Then we can dramatically reduce the dictionary usage within the file too. I wonder if we could incorporate a simple decompressor based upon something like pdlzip (http://lzip.nongnu.org/pdlzip.html) to perform LZMA decompression/relocation on the dictionary early within the startup process?
ROM image size is not very interesting, but CPU cycles wasted at startup are. I think run time memory needs would not be changed by compression, the dictionary needs to be uncompressed to memory anyway before it can be used.
But we could avoid copying the dictionary when copying ROM to RAM initially. Then the dictionary could be copied to OFMEM allocated area before Forth start. Some large arrays (FSYS_BUF, dstack, rstack etc.) could also be allocated before use. Would this help?
Yes - perhaps I haven't explained this particularly clearly. I'm thinking of a 2 stage strategy - the first stage is to copy the dictionary to an OFMEM-allocated area on startup. This will save space because at the moment the entire dictionary (including space for all the definitions used by various bootloaders) is within the OpenBIOS image, which is currently 512K in the case of SPARC64. Hence we would only need ~180K within the image rather than 512K for this.
Yes, but again, image size savings are not very interesting. Savings in image loading time is easily lost during decompression.
Secondly, if we're copying the data to an OFMEM-allocated area then why don't we just compress it at build time and then decompress it during the copy using DEFLATE (see my previous email)? Then we can further reduce the dictionary payload size from ~180K to around ~37K, although as you rightly point out there could be a small delay on startup - given the small size involved (and the fact we can lock the TLB entry like we do with the Forth machine), I don't think the penalty will be too bad.
But only the decompressed result is what matters wrt memory usage, isn't it? After Forth has started, there shouldn't be any difference in RAM use, except that the decompressor code takes more space.
I'm fairly confident I can come up with an experimental patch for this - would people mind if we added zlib as a build-time dependency, and puff.c to the OpenBIOS codebase? At the very least, if the decompression appears too expensive the first stage on its own would still be a good idea.
For maximum compression, bzImage (still remember those?) style approach could be used for whole image.
ATB,
Mark.