[LinuxBIOS] LZMA compression of kernel image and p7zip (Carl-Daniel Hailfinger / Stefan Reinauer)
Hey everyone, I have stumbled upon posts by Carl-Daniel and Stefan while trying to get some information why when I compile a kernel image with 7zip (LZMA) the kernel image doesn't load up. The specific post is - http://www.mail-archive.com/linuxbios@linuxbios.org/msg06389.html I figured that maybe you could help diagnosing this problem as it seems that I have found the problem but I am not too sure actually. I have been given an SDK based on Linux 2.6.10 for an embedded mips 4Kec (TI Avalanche series) cpu to build an image from. When I build the image using GZIP (standard) the kernel boots fine but when I choose 7ZIP support for better compression, the 'make ram_zimage' procedure is working successfully but when the image attempts to run off the device (via nfs) it writes: Launching kernel decompressor. Starting LZMA Uncompression Algorithm. Compressed file is LZMA format. Kernel decompressor was successful ... launching kernel. and nothing happens after that. I've been attempting to figure out what is causing the kernel not to load and after reading the post by Carl-Daniel and Stefan about 7zip using different headers than "normal" LZMA sdk I thought that this could be the problem. I understand that this is not the best forum to ask, being a linuxbios mailing-list but I would appreciate your comments and any help on this issue as I am really trying to understand what is going on. Thanks, Sincerely, Liran.
Hello Liran, On 30.08.2007 17:06, liran tal wrote:
I have stumbled upon posts by Carl-Daniel and Stefan while trying to get some information why when I compile a kernel image with 7zip (LZMA) the kernel image doesn't load up. [...] I have been given an SDK based on Linux 2.6.10 for an embedded mips 4Kec (TI Avalanche series) cpu to build an image from. When I build the image using GZIP (standard) the kernel boots fine but when I choose 7ZIP support for better compression, the 'make ram_zimage' procedure is working successfully but when the image attempts to run off the device (via nfs) it writes:
Launching kernel decompressor. Starting LZMA Uncompression Algorithm. Compressed file is LZMA format. Kernel decompressor was successful ... launching kernel.
and nothing happens after that.
Ah, the classic "no logs available" problem because the kernel hangs during early boot. Since it works with gzip I have to assume there is a bug either in the decompression routine or some #define has not been adjusted to the architecture. Less likely, but still possible would be that the decompression routine is touching memory which is special on your machine.
I've been attempting to figure out what is causing the kernel not to load and after reading the post by Carl-Daniel and Stefan about 7zip using different headers than "normal" LZMA sdk I thought that this could be the problem.
Possible, but rather unlikely. What do the docs for the lzma kernel patch say? Is it supposed to be used with p7zip or the original SDK? Carl-Daniel
Hey Carl, Sorry about the late reply, I've been on and off during flights out of the country and haven't got around to check this issue. I have done some more investigating on this issue but no luck (yet). It made some sense to me as well that the problem might be with the use of 7ZIP-specific compression/decompression but I'm not sure of that anymore. I have attempted to use the LZMA SDK library to compress the image using that instead of the 7z application (p7zip) but actually the booloader/kernel denied me from that image, possibly because it was written specifically to 7z so I am sure that encoding the image must be done using 7z. That's about how to encode the image. Regarding the decompression which happens in the kernel/bootloader stage - well the print out is: Launching kernel decompressor. Starting LZMA Uncompression Algorithm. Compressed file is LZMA format. Kernel decompressor was successful ... launching kernel. So I am relying on the fact that it decompressed fine and that there is no bug in the decompression code (I've been provided with an SDK so this is not some home-brew code that might contain more errors than usual). So going over the code more thouroughly - the part that handles the decompression calls the kernel_entry function which in turn calls the start_kernel and that to _text (if I remember the ordering right) and I am assuming that the problem is there somewhere. A good thing to remember is that the only difference from this problem to a working kernel image is the CONFIG_KERNEL_COMPRESS_7ZIP option which I use. If I use the default GZIP compression then the kernel image loads just fine. I guess this could also be some sort of alignment problems, but memory alignments specific to the 7z compression is somewhat doubtful, isn't it? Any insight is welcome, Regards, Liran. On 8/31/07, Carl-Daniel Hailfinger <c-d.hailfinger.devel.2006@gmx.net> wrote:
Hello Liran,
On 30.08.2007 17:06, liran tal wrote:
I have stumbled upon posts by Carl-Daniel and Stefan while trying to get some information why when I compile a kernel image with 7zip (LZMA) the kernel image doesn't load up. [...] I have been given an SDK based on Linux 2.6.10 for an embedded mips 4Kec (TI Avalanche series) cpu to build an image from. When I build the image using GZIP (standard) the kernel boots fine but when I choose 7ZIP support for better compression, the 'make ram_zimage' procedure is working successfully but when the image attempts to run off the device (via nfs) it writes:
Launching kernel decompressor. Starting LZMA Uncompression Algorithm. Compressed file is LZMA format. Kernel decompressor was successful ... launching kernel.
and nothing happens after that.
Ah, the classic "no logs available" problem because the kernel hangs during early boot. Since it works with gzip I have to assume there is a bug either in the decompression routine or some #define has not been adjusted to the architecture. Less likely, but still possible would be that the decompression routine is touching memory which is special on your machine.
I've been attempting to figure out what is causing the kernel not to load and after reading the post by Carl-Daniel and Stefan about 7zip using different headers than "normal" LZMA sdk I thought that this could be the problem.
Possible, but rather unlikely. What do the docs for the lzma kernel patch say? Is it supposed to be used with p7zip or the original SDK?
Carl-Daniel
Hi Liran, I have thought a while about your problem but no obvious solution appeared. On 04.09.2007 17:50, liran tal wrote:
I have done some more investigating on this issue but no luck (yet). It made some sense to me as well that the problem might be with the use of 7ZIP-specific compression/decompression but I'm not sure of that anymore. I have attempted to use the LZMA SDK library to compress the image using that instead of the 7z application (p7zip) but actually the booloader/kernel denied me from that image, possibly because it was written specifically to 7z so I am sure that encoding the image must be done using 7z. That's about how to encode the image.
OK, so the format seems to be OK.
Regarding the decompression which happens in the kernel/bootloader stage - well the print out is:
Launching kernel decompressor. Starting LZMA Uncompression Algorithm. Compressed file is LZMA format. Kernel decompressor was successful ... launching kernel.
So I am relying on the fact that it decompressed fine and that there is no bug in the decompression code
At least no obvious bug.
(I've been provided with an SDK so this is not some home-brew code that might contain more errors than usual).
My experience is that SDKs may contain really hard-to-find errors because sometimes the code is tested, but not reviewed.
So going over the code more thouroughly - the part that handles the decompression calls the kernel_entry function which in turn calls the start_kernel and that to _text (if I remember the ordering right) and I am assuming that the problem is there somewhere.
A possible problem is overlap between the compressed and decompressed kernel, another possibility is decompression to the wrong address. Can you try to get both decompression routines to print out where exactly the decompressed code is located? If possible, printing the end addr of the decompressed data will be interesting as well.
A good thing to remember is that the only difference from this problem to a working kernel image is the CONFIG_KERNEL_COMPRESS_7ZIP option which I use. If I use the default GZIP compression then the kernel image loads just fine.
You could try with early serial support to get at least some logs, but I doubt you'd get that far.
I guess this could also be some sort of alignment problems, but memory alignments specific to the 7z compression is somewhat doubtful, isn't it?
Many patches for embedded systems not merged in mainline Linux have had their share of problems in the past because the review is less thorough (if any). It has been quite a lot of time since I last looked at the in-kernel LZMA decompression code and I'm not quite sure how it works anymore. Hm... another option is a compiler/linker bug. The decompression scratch pad memory is sometimes quite big and depends on lots of factors, one of them being dictionary size. Try using the default values for everything when compressing. That might help as well. Regards, Carl-Daniel
participants (2)
-
Carl-Daniel Hailfinger -
liran tal