I like CBFS and hope all boards can use it soon. I'm afraid that if we're not careful we'll bring the instability of v3 to v2 as we try to bring back what we've learned.
I'd like to see an orderly transition to CBFS, from simplest to most complex. I think fixed areas, romstraps, etc. should be last.
Simple things that don't work yet in CBFS: 1. Normal/Fallback/Failover - a. There are comments in the code that failover should disappear. Either failover should disappear or the comments should. b. Normal should be stored in CBFS, not concatenated into the bootblock. 2. Delete - Delete works on the assumption that you want contiguous files a. I'm not convinced that fixed CBFS areas will be simpler than ldscripts. I consider them both ugly. 3. PCI ROM optimization. There should be some optimization for ROMs. a. Possibilities: 1. Traverse ROM once and set a flag in devices which have ROMs 2. Traverse ROM and keep a list of ROMs 3. .... 4. Geode ROM handling in CBFS. a. It would be nice to change the code so that it doesn't have to be in a fixed location (maybe via a copy to RAM?)
I guess the short version is that I'd like to keep all the ugly details in the bootblock and peel them out one at a time. I think we'll break fewer boards that way. I feel like we've been lucky to catch some of the little glitches lately, and we've had help from people being willing to bisect.
Thanks, Myles
On Tue, May 5, 2009 at 8:13 AM, Myles Watson mylesgw@gmail.com wrote:
b. Normal should be stored in CBFS, not concatenated into the bootblock.
I would argue that fallback should also be in CBFS. Which is fine, but ...
- Delete - Delete works on the assumption that you want contiguous files
a. I'm not convinced that fixed CBFS areas will be simpler than ldscripts. I consider them both ugly.
How are we going to support code that runs at a fixed address? I can only come up with these two options. I think the PIE stuff from v3 is not an option: we've been warned about it for a while now, and I now believe the warnings.
- Geode ROM handling in CBFS.
a. It would be nice to change the code so that it doesn't have to be in a fixed location (maybe via a copy to RAM?)
I guess the short version is that I'd like to keep all the ugly details in the bootblock and peel them out one at a time. I think we'll break fewer boards that way. I feel like we've been lucky to catch some of the little glitches lately, and we've had help from people being willing to bisect.
good points. But, we're at step 1b and we need to make a decision. Our current decision, by default, is 'use ldscripts to create fallback and normal images, not cbfs'. If you look at how the normal symbol is created in the fallback image, I don't see that fixed-location cbfs files are any uglier ...
ron
On Tue, May 5, 2009 at 9:20 AM, ron minnich rminnich@gmail.com wrote:
On Tue, May 5, 2009 at 8:13 AM, Myles Watson mylesgw@gmail.com wrote:
b. Normal should be stored in CBFS, not concatenated into the bootblock.
I would argue that fallback should also be in CBFS. Which is fine, but ...
I agree. I just think that it can be a future step.
- Delete - Delete works on the assumption that you want contiguous files
a. I'm not convinced that fixed CBFS areas will be simpler than ldscripts. I consider them both ugly.
How are we going to support code that runs at a fixed address? I can only come up with these two options. I think the PIE stuff from v3 is not an option: we've been warned about it for a while now, and I now believe the warnings.
I wish I knew the best answer here.
- Geode ROM handling in CBFS.
a. It would be nice to change the code so that it doesn't have to be in a fixed location (maybe via a copy to RAM?)
I guess the short version is that I'd like to keep all the ugly details in the bootblock and peel them out one at a time. I think we'll break fewer boards that way. I feel like we've been lucky to catch some of the little glitches lately, and we've had help from people being willing to bisect.
good points. But, we're at step 1b and we need to make a decision. Our current decision, by default, is 'use ldscripts to create fallback and normal images, not cbfs'.
I think we can put off the decision if we insert CBFS code to find the normal image and jump to it.
If you look at how the normal symbol is created in the fallback image, I don't see that fixed-location cbfs files are any uglier ...
It's ugly. I hope that adding normal images in CBFS without changing fallback will help us see the pathway forward.
Myles
On 05.05.2009 17:24, Myles Watson wrote:
On Tue, May 5, 2009 at 9:20 AM, ron minnich rminnich@gmail.com wrote:
On Tue, May 5, 2009 at 8:13 AM, Myles Watson mylesgw@gmail.com wrote:
- Delete - Delete works on the assumption that you want contiguous files a. I'm not convinced that fixed CBFS areas will be simpler than
ldscripts. I consider them both ugly.
How are we going to support code that runs at a fixed address? I can only come up with these two options. I think the PIE stuff from v3 is not an option: we've been warned about it for a while now, and I now believe the warnings.
I wish I knew the best answer here.
Although some people have talked about the evilness of PIE, I believe it still is the best way forward. And if GCC ever deprecates our usage of that mode, LLVM people seem to be happy to accommodate fringe use cases if it gets them improved regression tests or some press attention or a thank you note.
Regards, Carl-Daniel
On Tue, May 5, 2009 at 8:42 AM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Although some people have talked about the evilness of PIE, I believe it still is the best way forward. And if GCC ever deprecates our usage of that mode, LLVM people seem to be happy to accommodate fringe use cases if it gets them improved regression tests or some press attention or a thank you note.
I want to hear about a PIE mode that Segher will find acceptable :-)
ron
Am 05.05.2009 17:24, schrieb Myles Watson:
I think we can put off the decision if we insert CBFS code to find the normal image and jump to it.
My next step is to make copy_and_run cbfs aware. That would push the coreboot_ram part of both stages into cbfs already, significantly decreasing the bootblock size.
It's a low hanging fruit given that copy_and_run is a single code path now, and that at this time of the boot process, RAM is available.
After that, I wanted to take a look at the location calculation stuff that determines where normal is linked to. That would fix normal and I think it can be automated in a way that works for both cbfs and "old-style". That would mean less numbers to tweak, too.
Unless you really want to put the remaining fallback/normal code into cbfs, in which case that work would be useless. I really like the idea of using %cs, if it's known to be stable.
Patrick
On Tue, May 5, 2009 at 9:17 AM, Patrick Georgi patrick@georgi-clan.de wrote:
My next step is to make copy_and_run cbfs aware. That would push the coreboot_ram part of both stages into cbfs already, significantly decreasing the bootblock size.
this is the next sensible step to make. I think I jumped a bit ahead with the XIP patch.
ron
On Tue, May 5, 2009 at 10:24 AM, ron minnich rminnich@gmail.com wrote:
On Tue, May 5, 2009 at 9:17 AM, Patrick Georgi patrick@georgi-clan.de wrote:
My next step is to make copy_and_run cbfs aware. That would push the coreboot_ram part of both stages into cbfs already, significantly decreasing the bootblock size.
Thanks Patrick. This sounds great.
this is the next sensible step to make. I think I jumped a bit ahead with the XIP patch.
It sparked a good discussion and hopefully clarified where we're going.
Thanks, Myles
On Tue, May 05, 2009 at 06:17:53PM +0200, Patrick Georgi wrote:
Am 05.05.2009 17:24, schrieb Myles Watson:
I think we can put off the decision if we insert CBFS code to find the normal image and jump to it.
My next step is to make copy_and_run cbfs aware. That would push the coreboot_ram part of both stages into cbfs already, significantly decreasing the bootblock size.
It's a low hanging fruit given that copy_and_run is a single code path now, and that at this time of the boot process, RAM is available.
Thanks - that would be great to see.
What are your plans with lzma decoding? (Will the bootblock have an lzma decoder for deploying coreboot_ram and will coreboot_ram have an lzma decoder for the payload?)
Unless you really want to put the remaining fallback/normal code into cbfs, in which case that work would be useless. I really like the idea of using %cs, if it's known to be stable.
SeaBIOS must deal with %cs modifications, so I'm familiar with using it. However, it's not clear to me what coreboot needs it for. Can someone point me to a discussion on the problems with PIE?
-Kevin
Am 06.05.2009 02:14, schrieb Kevin O'Connor:
What are your plans with lzma decoding? (Will the bootblock have an lzma decoder for deploying coreboot_ram and will coreboot_ram have an lzma decoder for the payload?)
Yes. Tests with coreboot_ram showed that lzma compressing it gains more (vs. nrv2b compressing) than the overhead of shipping the lzma decoder, so this is a net win.
Unless you really want to put the remaining fallback/normal code into cbfs, in which case that work would be useless. I really like the idea of using %cs, if it's known to be stable.
SeaBIOS must deal with %cs modifications, so I'm familiar with using it. However, it's not clear to me what coreboot needs it for. Can someone point me to a discussion on the problems with PIE?
Short version: We can't rely on the compiler to do the right thing. We already run into problems every now and then because coreboot uses gcc (and binutils) in ways not intended by its developers - unix userland and linux kernel (and probably hurd) is what they seem to support.
Patrick
Hi Myles,
I tried CBFS on M2V-MX SE. So far it worked (Seabios + gpxe + vga rom) Well done all ;)
It seems that the ROM SIP area for chipset is OK. Is there a way how to verify that it will be always so? (Its in last 128B of flash)
I only ran to one issue, did not investigated yet. I had this rom as above and I removed the VGA rom and system stopped finding the seabios.
Dont know maybe there is smth wrong with remove,
Rudolf
On Tue, May 5, 2009 at 9:22 AM, Rudolf Marek r.marek@assembler.cz wrote:
Hi Myles,
I tried CBFS on M2V-MX SE. So far it worked (Seabios + gpxe + vga rom) Well done all ;)
It seems that the ROM SIP area for chipset is OK. Is there a way how to verify that it will be always so? (Its in last 128B of flash)
As long as the compilation process stays as it is right now. Right now we take a working (fallback or failover) image and use it as the "bootblock". When that changes there will be no guarantees. Right now you should use fallback+failover or fallback-only images. Normal is guaranteed to fail.
I only ran to one issue, did not investigated yet. I had this rom as above and I removed the VGA rom and system stopped finding the seabios.
Dont know maybe there is smth wrong with remove,
Most likely. I thought I fixed remove, but I probably missed a corner case.
Thanks, Myles
On Tue, May 5, 2009 at 10:15 AM, Myles Watson mylesgw@gmail.com wrote:
On Tue, May 5, 2009 at 9:22 AM, Rudolf Marek r.marek@assembler.cz wrote:
Hi Myles,
I tried CBFS on M2V-MX SE. So far it worked (Seabios + gpxe + vga rom) Well done all ;)
It seems that the ROM SIP area for chipset is OK. Is there a way how to verify that it will be always so? (Its in last 128B of flash)
As long as the compilation process stays as it is right now. Right now we take a working (fallback or failover) image and use it as the "bootblock". When that changes there will be no guarantees. Right now you should use fallback+failover or fallback-only images. Normal is guaranteed to fail.
I only ran to one issue, did not investigated yet. I had this rom as above and I removed the VGA rom and system stopped finding the seabios.
Dont know maybe there is smth wrong with remove,
Most likely. I thought I fixed remove, but I probably missed a corner case.
If you send me the working ROM I'll look at it before and after VGA removal.
Have you tried a fresh build without the VGA ROM?
Thanks, Myles
On Tue, May 05, 2009 at 09:13:32AM -0600, Myles Watson wrote:
- PCI ROM optimization. There should be some optimization for ROMs. a. Possibilities: 1. Traverse ROM once and set a flag in devices which have ROMs 2. Traverse ROM and keep a list of ROMs 3. ....
Are you referring to the email I sent on the time SeaBIOS spent traversing CBFS to find option roms?
I did modify SeaBIOS so that option roms scans are much faster. (The cost is now negligible.) I accomplished this by ending the file scan if a signature of zeros is found.
I don't think caching the list of files in the flash will work well - even one brute force scan will take too much time on large flash chips. Instead, I think CBFS needs to have an orderly layout of files which doesn't require users to do a brute force scan.
I think there's a simple way to accomplish this - have cbfstool create "null" files that contain all the empty space. Then the users of cbfs (eg, coreboot, seabios) wouldn't need any special code - they would skip them just like they skip over any other file.
-Kevin
-----Original Message----- From: Kevin O'Connor [mailto:kevin@koconnor.net] Sent: Tuesday, May 05, 2009 6:45 PM To: Myles Watson Cc: coreboot Subject: Re: [coreboot] CBFS transition plan
On Tue, May 05, 2009 at 09:13:32AM -0600, Myles Watson wrote:
- PCI ROM optimization. There should be some optimization for ROMs. a. Possibilities: 1. Traverse ROM once and set a flag in devices which have ROMs 2. Traverse ROM and keep a list of ROMs 3. ....
Are you referring to the email I sent on the time SeaBIOS spent traversing CBFS to find option roms?
I'm referring to the cost if CBFS can be sparse. I'm also referring to the number of scans required for some hardware. My Opteron boxes have many onboard devices, so there are 20+ CBFS scans.
I did modify SeaBIOS so that option roms scans are much faster. (The cost is now negligible.) I accomplished this by ending the file scan if a signature of zeros is found.
That only works if CBFS files are contiguous.
I don't think caching the list of files in the flash will work well - even one brute force scan will take too much time on large flash chips. Instead, I think CBFS needs to have an orderly layout of files which doesn't require users to do a brute force scan.
I think there's a simple way to accomplish this - have cbfstool create "null" files that contain all the empty space. Then the users of cbfs (eg, coreboot, seabios) wouldn't need any special code - they would skip them just like they skip over any other file.
I could see that working. It was zero-fill in v3 lar.
Thanks, Myles
On Tue, May 5, 2009 at 5:44 PM, Kevin O'Connor kevin@koconnor.net wrote:
I think there's a simple way to accomplish this - have cbfstool create "null" files that contain all the empty space. Then the users of cbfs (eg, coreboot, seabios) wouldn't need any special code - they would skip them just like they skip over any other file.
yes, this is easy. A simple post-pass like I did with LAR, take all empty segments and turn them into "null" files.
LZMA decoding -- cbfs will let us have several types of decoders available, one being lzma. cbfs is not done, it's work in progress but I do see real progress. Things are going to be pretty good in the next few weeks.
ron
-
Carl-Daniel Hailfinger -
Kevin O'Connor -
Myles Watson -
Patrick Georgi -
ron minnich -
Rudolf Marek