Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
There are a few nasty details to work out like how to handle services that are expected to work in vm86 mode. But I'm not certain I care.
Other thoughts?
After I come back from my christmas vacation I am going to have to try it and see how will it will actually work.
Eric
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thursday 18 December 2003 7:36 pm, Eric W. Biederman wrote:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
The way I implement my bootloader on ARM is like this:
1) First stage assembly loader sets up serial and DRAM. 2) First stage loader probes RAM, and sets up tagged list. 3) First stage jumps into zImage of special LAB (Linux As Bootldr) kernel. (currently just a 2.6.0 kernel from handhelds.org that has CONFIG_LAB defined.) 4) LAB kernel boots up until it gets ready to jump into init. 5) #ifdef'ed code takes over and calls a LAB main function which does all sorts of cool stuff including giving the user a CLI if requested (usually by holding the iPAQ's joypad down), or autobooting (running a predefined mkdir/mount/armboot sequence.)
Conceivably you could write a subapp for the CLI that does what you want, and put it in the autoboot script.
LAB for ARM's zImage is currently ~509kbytes for those who care. (We must keep it below 512KB.)
There are a few nasty details to work out like how to handle services that are expected to work in vm86 mode. But I'm not certain I care.
I've tinkered with writing an OS, but I still don't know too much about the x86 architecture, so I couldn't help there. Sorry.
Other thoughts?
Check out my LAB code, see what you think. It's in handhelds.org anoncvs, module linux/kernel26. Relevant crap is in bootldr/, drivers/bootldr/, and arch/arm/boot/.
After I come back from my christmas vacation I am going to have to try it and see how will it will actually work.
Take care, have a nice vacation.
Eric
/joshua
- -- Joshua Wise | www.joshuawise.com GPG Key | 0xEA80E0B3 Quote | <lilo> I akilled *@* by mistake In memoriam | Whiskers the hamster, 2001 - Dec 15, 2003
Joshua Wise joshua@joshuawise.com writes:
On Thursday 18 December 2003 7:36 pm, Eric W. Biederman wrote:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
The way I implement my bootloader on ARM is like this:
- First stage assembly loader sets up serial and DRAM.
- First stage loader probes RAM, and sets up tagged list.
Roughly what LinuxBIOS does.
- First stage jumps into zImage of special LAB (Linux As Bootldr) kernel.
(currently just a 2.6.0 kernel from handhelds.org that has CONFIG_LAB defined.) 4) LAB kernel boots up until it gets ready to jump into init. 5) #ifdef'ed code takes over and calls a LAB main function which does all sorts of cool stuff including giving the user a CLI if requested (usually by holding the iPAQ's joypad down), or autobooting (running a predefined mkdir/mount/armboot sequence.)
Conceivably you could write a subapp for the CLI that does what you want, and put it in the autoboot script.
Somehow I could write a subapp that would make linux look like a normal pcbios, but I can be surprised.
LAB for ARM's zImage is currently ~509kbytes for those who care. (We must keep it below 512KB.)
Ouch! My x86 images are below that, at least before decompression.
There are a few nasty details to work out like how to handle services that are expected to work in vm86 mode. But I'm not certain I care.
I've tinkered with writing an OS, but I still don't know too much about the x86 architecture, so I couldn't help there. Sorry.
Other thoughts?
Check out my LAB code, see what you think. It's in handhelds.org anoncvs, module linux/kernel26. Relevant crap is in bootldr/, drivers/bootldr/, and arch/arm/boot/.
I probably will. Doing that stuff inside the kernel does not really feel proper to me. I already have an x86 kernel that can load another kernel from user space. I'm just trying to find a good long term architecture for using the kernel as a bootloader.
After I come back from my christmas vacation I am going to have to try it and see how will it will actually work.
Take care, have a nice vacation.
Thanks.
Eric
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
- First stage assembly loader sets up serial and DRAM.
- First stage loader probes RAM, and sets up tagged list.
Roughly what LinuxBIOS does.
Right, ok. You can find my code in arch/arm/boot/bootldr-xscale.S.
Somehow I could write a subapp that would make linux look like a normal pcbios, but I can be surprised.
The subapp does run in kernelland, so you can do pretty much anything you want. (That's how we can boot another kernel from within LAB land.)
LAB for ARM's zImage is currently ~509kbytes for those who care. (We must keep it below 512KB.)
Ouch! My x86 images are below that, at least before decompression.
Whine whine whine. :) This is including lots of cruft.
I probably will. Doing that stuff inside the kernel does not really feel proper to me. I already have an x86 kernel that can load another kernel from user space. I'm just trying to find a good long term architecture for using the kernel as a bootloader.
The design decision was made to do this all from kernelland when LAB started off in 2.4. 2.4 did not have initramfs, so that was not even an option. The decision stuck as I upported to 2.6.
Yes, you have TKM for x86, but that's not very portable or readable. armboot is very readable (and portable) code, despite its name.
Regarding longevity, LAB will be supported for the next forever by handhelds.org.
Eric
/joshua
- -- Joshua Wise | www.joshuawise.com GPG Key | 0xEA80E0B3 Quote | <lilo> I akilled *@* by mistake In memoriam | Whiskers the hamster, 2001 - Dec 15, 2003
well I sure like the idea of using linux for the bios, as always ...
ron
ron minnich rminnich@lanl.gov writes:
well I sure like the idea of using linux for the bios, as always ...
Regardless of the actual details what I like is the added requirement that we attempt to support the legacy pcbios interface. It's ugly it's nasty but if we can do that with a Linux kernel driving the hardware we have a single solution which can work for everyone. Some people can strip it down and not include all of the features but...
Eric
On Fri, 2003-12-19 at 05:56, Eric W. Biederman wrote:
Regardless of the actual details what I like is the added requirement that we attempt to support the legacy pcbios interface. It's ugly it's nasty but if we can do that with a Linux kernel driving the hardware we have a single solution which can work for everyone. Some people can strip it down and not include all of the features but...
Or - cf the PXE discussions on etherboot-developers - have the stripped version as the "normal" code and load in the pc bios services as required. Indeed, why not add a "legacy layer" to OSs which need it, so that rather than adding the code to the loader, add it to the *loadee*; the common services can be lean, mean, open source, virtualised or whatever we please.
Has anyone tried this to see how small it compiles down to? Probally won't ever get as small as uclinux but might be useable for somebody.
http://lwn.net/Articles/62858/
Richard Smith rsmith@bitworks.com writes:
Has anyone tried this to see how small it compiles down to? Probally won't ever
get as small as uclinux but might be useable for somebody.
uclinux is in the same source tree. It looks like they have patches to compile out all of the recent bloat.
If the maintenance continues it looks very promising.
But I'm on vacataion at the moment so someone else gets to be the ginny pig.
Eric
Richard Smith rsmith@bitworks.com writes:
Has anyone tried this to see how small it compiles down to?
I just looked. With everything turned off I have built a 220K kernel. 371K uncompressed but that doesn't count.
It looks like there is light at the end of the tunnel.
Eric
On 4 Jan 2004, Eric W. Biederman wrote:
I just looked. With everything turned off I have built a 220K kernel. 371K uncompressed but that doesn't count.
this is 2.6 or ...
ron
ron minnich rminnich@lanl.gov writes:
On 4 Jan 2004, Eric W. Biederman wrote:
I just looked. With everything turned off I have built a 220K kernel. 371K uncompressed but that doesn't count.
this is 2.6 or ...
2.6.1-rc1-tiny1
Adding in - printk - serial console support - networking support ( Not IP networking) - an eepro100 driver
takes it up to about 306K compressed. With IPv4 support compiled it goes to about 396K compressed.
So the patches for 2.6 exist to make it usable. And there is still room for improvement.
Eric
ebiederman@lnxi.com (Eric W. Biederman) writes:
ron minnich rminnich@lanl.gov writes:
On 4 Jan 2004, Eric W. Biederman wrote:
I just looked. With everything turned off I have built a 220K kernel. 371K uncompressed but that doesn't count.
this is 2.6 or ...
2.6.1-rc1-tiny1
Adding in
- printk
- serial console support
- networking support ( Not IP networking)
- an eepro100 driver
takes it up to about 306K compressed. With IPv4 support compiled it goes to about 396K compressed.
So the patches for 2.6 exist to make it usable. And there is still room for improvement.
Playing a little more in the nothing selected case I get the kernel to 191K if I compile out support for block devices. After writing the patch that allows that.
Eric
you mean just like dosemu runs under linux ???
On 18 Dec 2003, Eric W. Biederman wrote:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
There are a few nasty details to work out like how to handle services that are expected to work in vm86 mode. But I'm not certain I care.
Other thoughts?
After I come back from my christmas vacation I am going to have to try it and see how will it will actually work.
Eric
Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
Adam Sulmicki adam@cfar.umd.edu writes:
you mean just like dosemu runs under linux ???
Right but for BSD and early versions of windows the dependencies were worse. I don't know which services matter though.
Eric
On Fri, Dec 19, 2003 at 12:49:42AM -0700, Eric W. Biederman wrote:
Adam Sulmicki adam@cfar.umd.edu writes:
you mean just like dosemu runs under linux ???
Right but for BSD and early versions of windows the dependencies were worse. I don't know which services matter though.
Last time I saw the source code of FreeBSD, its _bootloader_ has its own vm86 monitor (in assembly), works in protected mode, and calls all the BIOS calls (video, keyboard, disk,...) in vm86 mode. Also the kernel calls E820, APM, VESA, etc in vm86. Also I saw a recent version of Windows calls BIOS services in vm86.
Maybe we can modify FreeBSD, and have work-around for Windows, but at least I don't think it's the way to go.
My conclusion then was that PCBIOS had to work in real mode.
Maybe we can code it with GCC and .code16gcc hack. If 64KB is not enough, maybe 0xE000 segment can be used so that we have 64KB code and 64KB data segments. And all the "POST" code can be outside the real mode space. So it's not impossible or hard at all.
Takeshi Sone ts1@tsn.or.jp writes:
On Fri, Dec 19, 2003 at 12:49:42AM -0700, Eric W. Biederman wrote:
Adam Sulmicki adam@cfar.umd.edu writes:
you mean just like dosemu runs under linux ???
Right but for BSD and early versions of windows the dependencies were worse. I don't know which services matter though.
Last time I saw the source code of FreeBSD, its _bootloader_ has its own vm86 monitor (in assembly), works in protected mode, and calls all the BIOS calls (video, keyboard, disk,...) in vm86 mode. Also the kernel calls E820, APM, VESA, etc in vm86. Also I saw a recent version of Windows calls BIOS services in vm86.
Maybe we can modify FreeBSD, and have work-around for Windows, but at least I don't think it's the way to go.
Thanks. I knew this was the biggest danger, and from the description it looks like no half measures will do.
My conclusion then was that PCBIOS had to work in real mode.
If that is the case the PCBIOS has hit an evolutionary dead, as the code size cannot increase, there is very little room for change remaining. This explains ACPI. And it makes EFI look much more attractive.
I think I see one way out of this pickle, implement the firmware in System Management mode. System calls will be slow and we will need an interrupt reflector but this does allow us to escape the bounds of vm86 mode, and the legacy limitations. And our OS will even have some protection from ``user mode''.
System Management mode is essentially big real mode. So code would need to be compiled with .code16gcc, but there is a full 4G available. I know with amd's processors I can switch to 32bit or 64bit protected mode inside of smm mode, Intel's documentation is not clear on that point, so I don't know what we can do there. Depending on what works it may make sense simply to build an smm mode trampoline to get out of vm86 mode.
Maybe we can code it with GCC and .code16gcc hack. If 64KB is not enough, maybe 0xE000 segment can be used so that we have 64KB code and 64KB data segments. And all the "POST" code can be outside the real mode space. So it's not impossible or hard at all.
For just PCBIOS compatibility I agree. Primarily I want that layer to exist as an add-on and an after thought. Something that gives backwards compatibility but allows us to do something better.
With only 256KB of room, there simply is not enough room to implement drivers of interesting hardware. Nor is there enough room to implement interesting protocol stacks. And it limits what we can do in the future.
Eric
Takeshi Sone ts1@tsn.or.jp writes:
On Fri, Dec 19, 2003 at 12:49:42AM -0700, Eric W. Biederman wrote:
Adam Sulmicki adam@cfar.umd.edu writes:
you mean just like dosemu runs under linux ???
Right but for BSD and early versions of windows the dependencies were worse. I don't know which services matter though.
Last time I saw the source code of FreeBSD, its _bootloader_ has its own vm86 monitor (in assembly), works in protected mode, and calls all the BIOS calls (video, keyboard, disk,...) in vm86 mode. Also the kernel calls E820, APM, VESA, etc in vm86. Also I saw a recent version of Windows calls BIOS services in vm86.
Maybe we can modify FreeBSD, and have work-around for Windows, but at least I don't think it's the way to go.
My conclusion then was that PCBIOS had to work in real mode.
Maybe we can code it with GCC and .code16gcc hack. If 64KB is not enough, maybe 0xE000 segment can be used so that we have 64KB code and 64KB data segments. And all the "POST" code can be outside the real mode space. So it's not impossible or hard at all.
Ok Then I am up to idea #2. For cooperation, just before I go catch my flight.
Implement a minimal but real PCBIOS.
Have a linux kernel.
Use the PCBIOS to initialize the option roms. Jump to the linux base boot loader.
When I have loaded something I go to real mode with kexec and had off control to the loaded kernel. Which can make BIOS calls. I have traditionally avoided that because the PCBIOS tends to get wedged when I start linux but with the source that should not be a problem.
Eric
On 20 Dec 2003, Eric W. Biederman wrote:
Ok Then I am up to idea #2. For cooperation, just before I go catch my flight.
here's what I would like: - we support all things bootable that don't need a pcbios linux, plan9, memtest, etc. - we encourage other projects to move away from needing a bios this is mainly xyzbsd - we use emulation for the option roms so that PPC and other non-x86 will work (this is almost done) - we provide (for those who want it) PCBIOS support as an option Making it optional allows vendors to build Windows-proof PCs, and I have had some interest expressed in that concept: if you are Windows-proof, you are audit-proof. A desriable property.
we should really make PCBIOS an option of last resort. We should not plan for it as anything but a hack for OSes that are doing the wrong thing -- using a PCBIOS.
ron
On Sat, 20 Dec 2003, ron minnich wrote:
here's what I would like:
- we support all things bootable that don't need a pcbios linux, plan9, memtest, etc.
- we encourage other projects to move away from needing a bios this is mainly xyzbsd
- we use emulation for the option roms so that PPC and other non-x86 will work (this is almost done)
a bonus: these are all architecture-independent.
- we provide (for those who want it) PCBIOS support as an option
And this is for legacy OSes and very architecture-dependent.
ron
Hi,
When I read the part about being Windows-proof, I *really* liked that idea. Also, about the audit-proof aspect.
Would built-in file encryption be a foolish thing to wish for some day? (Maybe when the chips are bigger?)
Jorgen ve5jor@rac.ca
ps This list is *very* informative, and I am grateful for having access to it. I wish my talents were greater, and especially more current, so that I could contribute. Thanks to all, in the list. Merry Christmas, & Happy Holidays to all. --------------------------------------------------------------------------- On Saturday 20 December 2003 11:21 am, ron minnich wrote: ...... - we provide (for those who want it) PCBIOS support as an option Making it optional allows vendors to build Windows-proof PCs, and I have had some interest expressed in that concept: if you are Windows-proof, you are audit-proof. A desriable property.
we should really make PCBIOS an option of last resort. We should not plan for it as anything but a hack for OSes that are doing the wrong thing -- using a PCBIOS.
ron
_______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
When I read the part about being Windows-proof, I *really* liked that idea. Also, about the audit-proof aspect.
I think it would also work well for "grandpa's box" - you know, boots from a read-only CF that starts X and Mozilla - he can't mess anything up, and it takes about 5 seconds to boot.
Would built-in file encryption be a foolish thing to wish for some day? (Maybe when the chips are bigger?)
Not really possible if hard drive access doesn't go through the BIOS. And our goal is to completely eradicate the BIOS. However it could be possible to have encryption for BIOS-bound OSen such as DOS, and require an "auth sector" be present on the boot media to allow access...
Jorgen ve5jor@rac.ca
/joshua
- -- Joshua Wise | www.joshuawise.com GPG Key | 0xEA80E0B3 Quote | <lilo> I akilled *@* by mistake In memoriam | Whiskers the hamster, 2001 - Dec 15, 2003
On Sat, 20 Dec 2003, Another Happy Linux User wrote:
Would built-in file encryption be a foolish thing to wish for some day? (Maybe when the chips are bigger?)
we'll leave that in the hands of the kernel.
ron kb2zcw
* Eric W. Biederman ebiederman@lnxi.com [031219 01:36]:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
What functionality is it exactly that we need the uclinux kernel for? We won't do any scheduling, and we won't need source code drivers if we implement a pcbios "emulation" What I liked most about the LinuxBIOS2 approach is that code was only introduced in the tree when it was specifically needed.
Since I had my hands at the Alpha bootloader Milo, everything that uses a Linux kernel for it's operations kind of scares me...
Stefan
On Fri, 19 Dec 2003, Stefan Reinauer wrote:
- Eric W. Biederman ebiederman@lnxi.com [031219 01:36]:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
What functionality is it exactly that we need the uclinux kernel for? We won't do any scheduling, and we won't need source code drivers if we implement a pcbios "emulation" What I liked most about the LinuxBIOS2 approach is that code was only introduced in the tree when it was specifically needed.
I've been assuming that the uclinux approach is for a payload for those who want it. I did not expect to see this in the tree.
ron
Stefan Reinauer stepan@suse.de writes:
- Eric W. Biederman ebiederman@lnxi.com [031219 01:36]:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
What functionality is it exactly that we need the uclinux kernel for? We won't do any scheduling, and we won't need source code drivers if we implement a pcbios "emulation" What I liked most about the LinuxBIOS2 approach is that code was only introduced in the tree when it was specifically needed.
Since I had my hands at the Alpha bootloader Milo, everything that uses a Linux kernel for it's operations kind of scares me...
So this is for the bootloader, the payload and it will be optional. This will not go into the linuxBIOS tree, this is an alternative to etherboot.
I might do a kernel port to a weird hardware configuration but I will not hack up the kernel in the way Milo does. I have worked on it and I have the scars as well. Anything that borrows the kernel drivers is asking for trouble. But look at etherboot. It also has a dependency on linux drivers. But because etherboot actually ports the drivers, and then ports the bug fixes it works there is not a nightmare of a maintenance issue there. So I think using the kernel whole will be ok.
The observation has been that any sufficiently general firmware bootloader tends to become an OS, so why not use a real OS.
So the things I actually care about are: booting over myrinet, booting over inifinband, booting over quadrics, booting over a pair of bonded nics. Having a good tg3 driver. Getting Lustre as my root filesystem. Unless size issues kill it again I am going to use the linux kernel as my bootloader to do this.
So except for the case where someone plugs in a card with an option rom all the pcbios emulation would do would be to make system calls to the linux kernel. The biggest modification I would have to make would be to change the load address of the kernel, and not the let the kernel use all of RAM, but instead live in a small fixed area.
I guess the other problem I see with doing a pcbios emulation layer is I don't know if there it is there is an API to get the kernel drivers to stop.
If I can't have my cake and eat it too, I won't have pcbios compatibility. Oh, well.
Eric
On 19 Dec 2003, Eric W. Biederman wrote:
The observation has been that any sufficiently general firmware bootloader tends to become an OS, so why not use a real OS.
ah ha! we've come full circle :-)
good to hear.
So the things I actually care about are: booting over myrinet, booting over inifinband, booting over quadrics, booting over a pair of bonded nics. Having a good tg3 driver. Getting Lustre as my root filesystem. Unless size issues kill it again I am going to use the linux kernel as my bootloader to do this.
yah, this is why we still put linux in flash here at LANL.
ron
ron minnich rminnich@lanl.gov writes:
On 19 Dec 2003, Eric W. Biederman wrote:
The observation has been that any sufficiently general firmware bootloader tends to become an OS, so why not use a real OS.
ah ha! we've come full circle :-)
good to hear.
I have never disagreed. Now that we have something that works for the low end limited ROM solutions I can revisit some of the more interesting ideas.
So the things I actually care about are: booting over myrinet, booting over inifinband, booting over quadrics, booting over a pair of bonded nics. Having a good tg3 driver. Getting Lustre as my root filesystem. Unless size issues kill it again I am going to use the linux kernel as my bootloader to do this.
yah, this is why we still put linux in flash here at LANL.
Now if you had more than a proof of concept implementation of most of the pieces I would not be happier, but anyway...
Eric
On 19 Dec 2003, Eric W. Biederman wrote:
Now if you had more than a proof of concept implementation of most of the pieces I would not be happier, but anyway...
I kind of miss the point. Pink's been booting with linux in ide-flash for a year, and if the flash on pink were large enough, we would have been using that instead.
ron
ron minnich rminnich@lanl.gov writes:
On 19 Dec 2003, Eric W. Biederman wrote:
Now if you had more than a proof of concept implementation of most of the pieces I would not be happier, but anyway...
I kind of miss the point. Pink's been booting with linux in ide-flash for a year, and if the flash on pink were large enough, we would have been using that instead.
Possibly proof of concept is the wrong term.
What Pink does works. I can't throw J. Random kernel at beoboot and have that work. I can't throw memtest86 at it and have that work. I can't use an SMP aware kernel with beoboot. And last I looked beoboot had a bulky user space that makes it hard to squeeze into flash as well. So beoboot with 2 kernel monte works for a certain interesting subset of cases but it does not handle the general case.
Except possibly for the exotic driver case 512KB is a large enough FLASH chip to put in both LinuxBIOS and a kernel. You have to work at it with a 512KB flash chip but it is doable today.
Eric
On 19 Dec 2003, Eric W. Biederman wrote:
What Pink does works. I can't throw J. Random kernel at beoboot and have that work. I can't throw memtest86 at it and have that work. I can't use an SMP aware kernel with beoboot. And last I looked beoboot had a bulky user space that makes it hard to squeeze into flash as well. So beoboot with 2 kernel monte works for a certain interesting subset of cases but it does not handle the general case.
ok, gotcha. I don't think I would use beoboot for the general flash case, I think what Andrew Ip set up with kernel+kexec+initrd with some utils is the right model.
Except possibly for the exotic driver case 512KB is a large enough FLASH chip to put in both LinuxBIOS and a kernel. You have to work at it with a 512KB flash chip but it is doable today.
gotcha.
thanks
ron
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Friday 19 December 2003 4:04 pm, Eric W. Biederman wrote:
ron minnich rminnich@lanl.gov writes:
On 19 Dec 2003, Eric W. Biederman wrote:
The observation has been that any sufficiently general firmware bootloader tends to become an OS, so why not use a real OS.
ah ha! we've come full circle :-)
good to hear.
I have never disagreed. Now that we have something that works for the low end limited ROM solutions I can revisit some of the more interesting ideas.
*jumps up and down like a hyperactive two-year-old* LAB can do that stuff!
I would definately like to see LAB used on more than just ARM.
So the things I actually care about are: booting over myrinet, booting over inifinband, booting over quadrics, booting over a pair of bonded nics. Having a good tg3 driver. Getting Lustre as my root filesystem. Unless size issues kill it again I am going to use the linux kernel as my bootloader to do this.
If you can have it in the kernel you can have it in LAB. LAB's command line API is really easy - I've ported userland apps (mtderase) to LAB in sub-10 minutes.
As to size issues, we're at a 512k zImage. We do not have bzImages on ARM, so I could not tell you the size with it. 512k is with a few ARM-specific drivers, and jffs2. It does not have networking. This is with kernel 2.6.
yah, this is why we still put linux in flash here at LANL.
Oh and did I say LAB had MTD support? :)
Now if you had more than a proof of concept implementation of most of the pieces I would not be happier, but anyway...
*jumps up and down* I do I do!!
Eric
/joshua
- -- Joshua Wise | www.joshuawise.com GPG Key | 0xEA80E0B3 Quote | <lilo> I akilled *@* by mistake In memoriam | Whiskers the hamster, 2001 - Dec 15, 2003
Joshua Wise joshua@joshuawise.com writes:
On Friday 19 December 2003 4:04 pm, Eric W. Biederman wrote:
ron minnich rminnich@lanl.gov writes:
On 19 Dec 2003, Eric W. Biederman wrote:
The observation has been that any sufficiently general firmware bootloader tends to become an OS, so why not use a real OS.
ah ha! we've come full circle :-)
good to hear.
I have never disagreed. Now that we have something that works for the low end limited ROM solutions I can revisit some of the more interesting ideas.
*jumps up and down like a hyperactive two-year-old* LAB can do that stuff!
I would definately like to see LAB used on more than just ARM.
Yes we are reaching the point where we can converge on some of these things. LAB might be the right framework. And if it is something good it will save me the trouble of starting my own project. But it takes more than a hyper active 2 year old to convince me. It might take a hyperactive 2 year old to remind me about interesting ideas though.
So the things I actually care about are: booting over myrinet, booting over inifinband, booting over quadrics, booting over a pair of bonded nics. Having a good tg3 driver. Getting Lustre as my root filesystem. Unless size issues kill it again I am going to use the linux kernel as my bootloader to do this.
If you can have it in the kernel you can have it in LAB. LAB's command line API is really easy - I've ported userland apps (mtderase) to LAB in sub-10 minutes.
By and large I don't want a command line interface. I don't want a hardware monitor. I want a configurable but non-interactive boot.
As to size issues, we're at a 512k zImage. We do not have bzImages on ARM, so I could not tell you the size with it.
Compression wise there is not a difference between bzImage and zImage. zImage on x86 has a 640K limit because it loads below 1MB. bzImage breaks that ancient barrier.
512k is with a few ARM-specific drivers, and jffs2. It does not have networking. This is with kernel 2.6.
Hmm. I am pretty certain I have gotten 2.6 down some smaller. Our practical limit with LinuxBIOS etc is in the neighborhood of 384KB.
yah, this is why we still put linux in flash here at LANL.
Oh and did I say LAB had MTD support? :)
Well I think I have run finally convinced to use the MTD drivers... Mostly I prefer to flash from a production kernel rather than a bootloader, there are more recover options but anyway.
Now if you had more than a proof of concept implementation of most of the pieces I would not be happier, but anyway...
*jumps up and down* I do I do!!
I will see. Does LAB restrict it's kernel to a very small subset of memory? Or do you use something like kexec?
Eric
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Friday 19 December 2003 7:15 pm, Eric W. Biederman wrote:
Yes we are reaching the point where we can converge on some of these things. LAB might be the right framework. And if it is something good it will save me the trouble of starting my own project. But it takes more than a hyper active 2 year old to convince me. It might take a hyperactive 2 year old to remind me about interesting ideas though.
Right, well then you should see it in action. If you're in the Boston area sometime soon I can give you a demo on an iPAQ, perhaps.
If you can have it in the kernel you can have it in LAB. LAB's command line API is really easy - I've ported userland apps (mtderase) to LAB in sub-10 minutes.
By and large I don't want a command line interface. I don't want a hardware monitor. I want a configurable but non-interactive boot.
Right, that's where our autoboot stuff comes in. You can have it mount a partition by default, do whatever you want by default if no key has been struck on the serial line within 3 seconds.
Compression wise there is not a difference between bzImage and zImage. zImage on x86 has a 640K limit because it loads below 1MB. bzImage breaks that ancient barrier.
Right, I just read about that. Mea culpa.
512k is with a few ARM-specific drivers, and jffs2. It does not have networking. This is with kernel 2.6.
Hmm. I am pretty certain I have gotten 2.6 down some smaller. Our practical limit with LinuxBIOS etc is in the neighborhood of 384KB.
I've done 2.4 in 256k, but it's rather useless like that. If you do not plan to load modules at runtime, you can shave a good bit more off of it. If you write bzip2 compression support (or upport the stuff from kernel 2.4), you can shave even more off of it. I've pulled off 50k with bzip2 (not actually written the code, just did a bzip2 -9 < piggy > piggy.bz2). If you don't plan to have a framebuffer, you can shave some off of it. If you don't plan to have jffs2 you can shave a lot more off. Little tidbits here and there make the world go 'round.
Well I think I have run finally convinced to use the MTD drivers... Mostly I prefer to flash from a production kernel rather than a bootloader, there are more recover options but anyway.
Ah yes, the ancient problem. Instead of read/modify/erase/write, it often turns into read/modify/erase/poweroff. That's Bad.
I will see. Does LAB restrict it's kernel to a very small subset of memory? Or do you use something like kexec?
To boot a secondary kernel I use some code I wrote called armboot, although it's not very arm specific. It does something like this:
1) Load the new kernel into a contiguous vmalloced block. 2) We allocate 64k for a list of things that need to be relocated. We call this a pointer of type "struct physlist", which is 32 bytes. It has four ints: the new address, the old address, the block size, and whether this is the last block. 2) In blocks of the maximum kmalloc size (these blocks have to be contiguous), we kmalloc space for the kernel, and memcpy the kernel into those blocks. We then fill in a struct physlist, and move on to the next struct physlist. We can do this because kmalloc is always contiguous, and we can always map it with virt_to_phys(). 3) We set up another kmalloced block for the tagged list of boot parameters that you need on ARM. 4) We set up one more kmalloced block and copy an assembler function into it, to make sure we don't wipe ourself out while relocating. 5) We flush our data caches. 6) We call the relocated assembler function, which turns off the MMU, jumps into the relocated assembler function's physical address, and does actual relocating. Then we jump into our newly moved zImage. Confused yet? 7) If at any point we failed, the system could be in an inconsistent state. You will want to panic() if you fail, because you're leaking memory like a sieve, and if you failed there's probably something bigger wrong.
This looks more difficult than it actually is. The C segment is only about 170 lines, and the assembler bit is 90 lines.
The reason that this works is that kmalloc should allocate from the top of memory down. You need a fair bit of ram - say, 8MB - to prevent the tail from running over other important structures, such as the list of addresses to relocate. But it seems to work well enough, and it looks like it should be fairly portable. The important code is in handhelds.org cvs, module linux/kernel26, files drivers/bootldr/armboot.c and drivers/bootldr/armboot-asm.S.
Eric
/joshua
- -- Joshua Wise | www.joshuawise.com GPG Key | 0xEA80E0B3 Quote | <lilo> I akilled *@* by mistake In memoriam | Whiskers the hamster, 2001 - Dec 15, 2003
Joshua Wise joshua@joshuawise.com writes:
On Friday 19 December 2003 7:15 pm, Eric W. Biederman wrote:
Yes we are reaching the point where we can converge on some of these things. LAB might be the right framework. And if it is something good it will save me the trouble of starting my own project. But it takes more than a hyper active 2 year old to convince me. It might take a hyperactive 2 year old to remind me about interesting ideas though.
Right, well then you should see it in action. If you're in the Boston area sometime soon I can give you a demo on an iPAQ, perhaps.
Salt Lake City, and Illinois with my family for Christmas. Though a serial console logfile might be interesting.
512k is with a few ARM-specific drivers, and jffs2. It does not have networking. This is with kernel 2.6.
Hmm. I am pretty certain I have gotten 2.6 down some smaller. Our practical limit with LinuxBIOS etc is in the neighborhood of 384KB.
I've done 2.4 in 256k, but it's rather useless like that. If you do not plan to load modules at runtime, you can shave a good bit more off of it. If you write bzip2 compression support (or upport the stuff from kernel 2.4), you can shave even more off of it. I've pulled off 50k with bzip2 (not actually written the code, just did a bzip2 -9 < piggy > piggy.bz2).
The problem is that the bzip2 decompresser is huge, usually bzip2 is a net loss because of the decompresser. But it may be possible to write a tuned version. The cases I have typically worried about are much smaller and I have made huge gains by switching to nrv2b from upx because the decompresser is something like 100 bytes, and the compression is roughly as good as gzip.
If you don't plan to have a framebuffer, you can shave some off of it. If you don't plan to have jffs2 you can shave a lot more off. Little tidbits here and there make the world go 'round.
Quite true.
Well I think I have run finally convinced to use the MTD drivers... Mostly I prefer to flash from a production kernel rather than a bootloader, there are more recover options but anyway.
Ah yes, the ancient problem. Instead of read/modify/erase/write, it often turns into read/modify/erase/poweroff. That's Bad.
:)
I will see. Does LAB restrict it's kernel to a very small subset of memory? Or do you use something like kexec?
To boot a secondary kernel I use some code I wrote called armboot, although it's not very arm specific. It does something like this:
- Load the new kernel into a contiguous vmalloced block.
- We allocate 64k for a list of things that need to be relocated. We call
this a pointer of type "struct physlist", which is 32 bytes. It has four ints: the new address, the old address, the block size, and whether this is the last block. 2) In blocks of the maximum kmalloc size (these blocks have to be contiguous), we kmalloc space for the kernel, and memcpy the kernel into those blocks. We then fill in a struct physlist, and move on to the next struct physlist. We can do this because kmalloc is always contiguous, and we can always map it with virt_to_phys(). 3) We set up another kmalloced block for the tagged list of boot parameters that you need on ARM. 4) We set up one more kmalloced block and copy an assembler function into it, to make sure we don't wipe ourself out while relocating. 5) We flush our data caches. 6) We call the relocated assembler function, which turns off the MMU, jumps into the relocated assembler function's physical address, and does actual relocating. Then we jump into our newly moved zImage. Confused yet?
Nope. Having implemented something similar it sounds sane.
- If at any point we failed, the system could be in an inconsistent state.
You will want to panic() if you fail, because you're leaking memory like a sieve, and if you failed there's probably something bigger wrong.
Ouch.
This looks more difficult than it actually is. The C segment is only about 170 lines, and the assembler bit is 90 lines.
The reason that this works is that kmalloc should allocate from the top of memory down. You need a fair bit of ram - say, 8MB - to prevent the tail from running over other important structures, such as the list of addresses to relocate. But it seems to work well enough, and it looks like it should be fairly portable. The important code is in handhelds.org cvs, module linux/kernel26, files drivers/bootldr/armboot.c and drivers/bootldr/armboot-asm.S.
Ok. I need to get into that kernel tree and take a look. But it sounds similar to my kexec stuff. Which I discuss at least part of the time on fastboot@osdl.org. It sounds compatible enough that we could productively merge implementations, that plus my kexec stuff is still on Andrew Morton todo list ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/must-fix/should-fix-7.txt means it has a fair shot of getting into the stock kernel.
On a practical side I think I can boost it's priority high enough after I get back to actually do something.
A recent version of kexec patch is at: http://developer.osdl.org/rddunlap/kexec/
Kexec as it is currently structured is actually two system calls callable from user space.
sys_kexec_load() load the kernel into a linked list of pages, making certain that when those pages are copied to their final destination nothing will be stomped. And it allocates a chunk of memory with kmalloc for the bit of code that copies the kernel to it's final resting place. This can fail at any time and the system is in a consistent state.
sys_reboot(LINUX_REBOOT_CMD_KEXEC) initiates the transfer to the new kernel.
The new kernel is started in physical mode.
sys_kexec_load() is passed an entry point to jump to, and an array of physical destination address, virtual process space address, and virtual length regions to load. Which allows us to load arbitrary things.
The only requirement is that you have enough memory for both kernels simultaneously. For truly high end machines there are some other restrictions because physical mode does not allow access to all of their memory but anyway...
From the descriptions my kexec stuff is a little more general and a little
more robust than your armboot, so I'd like to merge your stuff into mine if possible. Now that 2.6.0 is out I can start sending patches again.
- I avoid deliberately avoid vmalloc, the vmalloc area is a limited resource, and I support loading large kernel ramdisk combinations. - I avoid greater than page size memory allocations because random memory fragmentation can make that fail. - I don't leak memory. - I have an implementation that works on SMP machines.
The worst part is getting all of the drivers to shut themselves down properly. But I have the appropriate hooks and it doesn't take an extension of the kernel api just lots of driver bug fixes.
If even the arm guys are up to using a kernel it should be easier to make progress in this array. On several projects the people I have talked to are two size constrained to even try using a kernel.
Eric
At 11:20 AM -0700 12/19/03, Eric W. Biederman wrote:
Stefan Reinauer stepan@suse.de writes:
- Eric W. Biederman ebiederman@lnxi.com [031219 01:36]:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
What functionality is it exactly that we need the uclinux kernel for? We won't do any scheduling, and we won't need source code drivers if we implement a pcbios "emulation" What I liked most about the LinuxBIOS2 approach is that code was only introduced in the tree when it was specifically needed.
Since I had my hands at the Alpha bootloader Milo, everything that uses a Linux kernel for it's operations kind of scares me...
So this is for the bootloader, the payload and it will be optional. This will not go into the linuxBIOS tree, this is an alternative to etherboot.
I would like to see this from the PPC perspective. Since neither filo nor etherboot work on PPC, this might be a good way of booting from, say, a disk. Which would be nice.
Greg
Greg Watson gwatson@lanl.gov writes:
I would like to see this from the PPC perspective. Since neither filo nor etherboot work on PPC, this might be a good way of booting from, say, a disk. Which would be nice.
I won't argue with that but this will take some time to come to fruition. etherboot should be fairly straight forward to port to the PPC. Currently etherboot supports 3 architectures 4 if you count x86_64. The PPC would not even be the first big endian port. So for a low end solution that works I still recommend etherboot.
But if we can avoid the bloat issues the Linux kernel is a good thing to have.
Any interest in porting kexec to the ppc? It is not strictly required if you can get your kernel to avoid the memory problem but it is quite useful.
Eric
At 2:02 PM -0700 12/19/03, Eric W. Biederman wrote:
Any interest in porting kexec to the ppc? It is not strictly required if you can get your kernel to avoid the memory problem but it is quite useful.
Yes. I want to do the two kernel monte thing on the PPC. Another thing on the todo list.
Greg
On Fri, 19 Dec 2003, Greg Watson wrote:
I would like to see this from the PPC perspective. Since neither filo nor etherboot work on PPC, this might be a good way of booting from, say, a disk. Which would be nice.
a linux kernel, with a small initrd with modules and kexec, is just about the ideal boot loader setup. cwlinux.com has been selling this configuration for some time. Setting up the same thing on PPC should be easy.
We have all the bits we need for a very capable bootloader, and have for 3 years. The only thing missing has been a big enough FLASH part. If the FLASH gets big enough soon enough, our problems will be solved for us.
ron
On Fri, 2003-12-19 at 00:36, Eric W. Biederman wrote:
Brainstorming earlier today I think I have found a way to use an linux kernel for the boot loader and to implement pcbios compatibility without too much cost. The idea is to use a uclinux kernel. And implement a ``user space'' aplication that is a user space shim that makes kernel calls.
There are a few nasty details to work out like how to handle services that are expected to work in vm86 mode. But I'm not certain I care.
Other thoughts?
Curse you, Eric, this was my idea. :)
But I approached it from a different angle. Getting Etherboot (the driver library and higher level code) to export PXE - effectively Intel's retrofit of networking code into the legacy BIOS - is similar to exporting LinuxBIOS's hardware initialisation library and a non-trivial OS via PC BIOS services. If VMWare, bochs or whatever can do it as a user space process, then it can be done all of a pece in firmware, right?
[For those of you not on etherboot-developers, I've waved my arms about recently at Eric, Ken Yap and others about how to resurrect the PXE code in Etherboot].
And yes, the "interesting" stuff is how to switch modes as appropriate and to sort out how to organise memory without frightening the horses. I'm on a very steep learning curve. And on the bottom of it, frankly.
The only difference between us is that I was thinking of building this on eCos, not uclinux. eCos is designed to minimally load only those bits of functionality needed. I don't think this affects the basic idea.
-
Adam Sulmicki -
Another Happy Linux User -
ebiederman＠lnxi.com -
Greg Watson -
Joshua Wise -
Peter Lister -
Richard Smith -
ron minnich -
Stefan Reinauer -
Takeshi Sone