Hi,
I was looking through the coreboot code to see if I could access the lar directly from SeaBIOS. It looks like this is pretty straight forward, so I think I might end up trying it.
However, I ran into one thing that looks like it will make things a little difficult. Currently, the lar code itself has the decompression code linked into it. If lar traversing code is added to SeaBIOS, it looks like it will also have to support the various decompress functions. That will lead to a notable increase in payload size.
I seem to recall reading somewhere that the original plan for the lar was to put the decompressors in their own lar files. That would be ideal for the SeaBIOS case, because then it could invoke the decompress routines without having to duplicate them.
Would it be possible to move the decompressors to their own lar files? Looking at the code it doesn't seem too difficult.
If I've missed something, please let me know.
-Kevin
On 21/10/08 21:20 -0400, Kevin O'Connor wrote:
Hi,
I was looking through the coreboot code to see if I could access the lar directly from SeaBIOS. It looks like this is pretty straight forward, so I think I might end up trying it.
However, I ran into one thing that looks like it will make things a little difficult. Currently, the lar code itself has the decompression code linked into it. If lar traversing code is added to SeaBIOS, it looks like it will also have to support the various decompress functions. That will lead to a notable increase in payload size.
I seem to recall reading somewhere that the original plan for the lar was to put the decompressors in their own lar files. That would be ideal for the SeaBIOS case, because then it could invoke the decompress routines without having to duplicate them.
Would it be possible to move the decompressors to their own lar files? Looking at the code it doesn't seem too difficult.
I think this is doable. To simplify everything, the code would have to be run in place on the ROM, and it would have to run with only the stack and nothing else (looking briefly at the lzma and nrv2b code, this wouldn't be a problem, but it is a design consideration for future compression schemes).
The LZMA code from v3 is 4236 bytes (off topic rant - there HAS to be a cleaner C or assembly implementation out there somewhere). I am pretty sure nrv2b code is even smaller, but I don't have a compiled version handy. Looking briefly at some of the other compression candidates (believe in change - vote LZO!), it looks like at most we would sacrifice somewhere between 4k and 8k of space (probably 1-2 erase blocks in the brave new partial rewrite world) per method. Based on current usage, I expect that we would only end up using a single decompressor per LAR. Bonus is that the decompressor code would disappear from coreboot, so our net size increase will be about zero (excepting alignment and LAR headers).
The downside is that the utility will have to live as a component in the LAR which will have an effect on performance when accessing the other components. A possibility is that we could point directly to the decompressor in the master LAR header that lives underneath the reset vector. My magic eight ball sees a LAR re-design in our future, so this is something that we should address (can anybody think of any other "utilities" we can share in the lar. ELF/multiboot loader comes to mind).
In the mean time, somebody needs to play build master - we need the LZMA and NRV2B code to be wholy contained in a single binary with an entry point. Best best for a container is a ELF with a single .text segment and nothing else. A binary blob would be fine too, assuming that we know the entry point. Coreboot and the payloads will need to know how to jump to the code and back out.
For the moment, I think that it would be best if the code lived separately from the LAR utility and then was included while the ROM was constructed. I think this will encourage other compression schemes. And we should take advantage of this to clean up our existing compression schemes as well. Maybe we can find some smaller code (fitting everything into a single typical erase block would be heavenly).
Oh, and as long as I'm talking - we need to teach LAR how to use off-the-shelf libraries to _compress_ the payloads - if we could eliminate all compression code from LAR and coreboot and libpayload - that would rock!
Jordan
Hi Jordan,
On Wed, Oct 22, 2008 at 09:36:54AM -0600, Jordan Crouse wrote:
On 21/10/08 21:20 -0400, Kevin O'Connor wrote:
Would it be possible to move the decompressors to their own lar files? Looking at the code it doesn't seem too difficult.
I think this is doable. To simplify everything, the code would have to be run in place on the ROM, and it would have to run with only the stack and nothing else (looking briefly at the lzma and nrv2b code, this wouldn't be a problem, but it is a design consideration for future compression schemes).
[...]
The downside is that the utility will have to live as a component in the LAR which will have an effect on performance when accessing the other components.
[...]
In the mean time, somebody needs to play build master - we need the LZMA and NRV2B code to be wholy contained in a single binary with an entry point. Best best for a container is a ELF with a single .text segment and nothing else. A binary blob would be fine too, assuming that we know the entry point. Coreboot and the payloads will need to know how to jump to the code and back out.
The current lar code supports running a LAR blob in ROM (execute_in_place() in lib/lar.c). I was kinda thinking that one could change process_file() to call something like execute_in_place("ulzma") instead of calling ulzma() directly.
I didn't think performance would be that important, as I thought we currently only decompress stage2 and the payload. Are extra LAR scans very expensive?
Thanks for your comments. -Kevin
On 22/10/08 20:45 -0400, Kevin O'Connor wrote:
Hi Jordan,
On Wed, Oct 22, 2008 at 09:36:54AM -0600, Jordan Crouse wrote:
On 21/10/08 21:20 -0400, Kevin O'Connor wrote:
Would it be possible to move the decompressors to their own lar files? Looking at the code it doesn't seem too difficult.
I think this is doable. To simplify everything, the code would have to be run in place on the ROM, and it would have to run with only the stack and nothing else (looking briefly at the lzma and nrv2b code, this wouldn't be a problem, but it is a design consideration for future compression schemes).
[...]
The downside is that the utility will have to live as a component in the LAR which will have an effect on performance when accessing the other components.
[...]
In the mean time, somebody needs to play build master - we need the LZMA and NRV2B code to be wholy contained in a single binary with an entry point. Best best for a container is a ELF with a single .text segment and nothing else. A binary blob would be fine too, assuming that we know the entry point. Coreboot and the payloads will need to know how to jump to the code and back out.
The current lar code supports running a LAR blob in ROM (execute_in_place() in lib/lar.c). I was kinda thinking that one could change process_file() to call something like execute_in_place("ulzma") instead of calling ulzma() directly.
Well, esentially, thats what execute_in_place() does - but thats just semantics. One problem though, execute_in_place doesn't grok arguments, which you will need to do the compression.
I didn't think performance would be that important, as I thought we currently only decompress stage2 and the payload. Are extra LAR scans very expensive?
Anything having to do with the ROM is expensive. In fact, after talking to Marc and others, I have serious doubts that this method will be very effective. So, If you don't execute from ROM, then it gets shadowed everywhere. So then coreboot and the payloads are going to copy the compressor to memory, but where should it live? Then you think that the code should stay resident, and then you are talking about services, and you are right back at the discssion about keeping a permanent service middle layer in memory, and then that distracts you from the matter at hand.
So I'll have to retract what I said yesterday - I'm back to thinking that our best bet is a generic compression library containing the LZMA and nrv2b code that payloads can consume. You still have the duplication, but at least you don't need to get your hands dirty with the code.
Jordan
On Thu, Oct 23, 2008 at 08:41:26AM -0600, Jordan Crouse wrote:
On 22/10/08 20:45 -0400, Kevin O'Connor wrote:
I didn't think performance would be that important, as I thought we currently only decompress stage2 and the payload. Are extra LAR scans very expensive?
Anything having to do with the ROM is expensive. In fact, after talking to Marc and others, I have serious doubts that this method will be very effective. So, If you don't execute from ROM, then it gets shadowed everywhere.
Hrmm - I thought the current decompress code runs from the rom today. Aren't the LAR "boot block" functions all XIP?
So then coreboot and the payloads are going to copy the compressor to memory, but where should it live? Then you think that the code should stay resident, and then you are talking about services, and you are right back at the discssion about keeping a permanent service middle layer in memory, and then that distracts you from the matter at hand.
I definitely don't want to implement services.
Why can't we run XIP from the rom? Wont the icache mask the rom latency? Also, we're only talking about uncompressing a few files - stage2, the payload, maybe one or two option roms. Is the latency going to be noticable (ie, on the order of a second)?
So I'll have to retract what I said yesterday - I'm back to thinking that our best bet is a generic compression library containing the LZMA and nrv2b code that payloads can consume. You still have the duplication, but at least you don't need to get your hands dirty with the code.
By consume, you mean link with (eg, by adding to libpayload)? Or, are you suggesting it be somewhere where payloads can extract it from flash at runtime?
Thanks, -Kevin
On Wed, Oct 22, 2008 at 8:36 AM, Jordan Crouse jordan.crouse@amd.com wrote:
In the mean time, somebody needs to play build master - we need the LZMA and NRV2B code to be wholy contained in a single binary with an entry point. Best best for a container is a ELF with a single .text segment and nothing else. A binary blob would be fine too, assuming that we know the entry point. Coreboot and the payloads will need to know how to jump to the code and back out.
why the elf? we run initram now in place just fine. Why can't we do this the same way we do initram?
ron
-
Jordan Crouse -
Kevin O'Connor -
ron minnich