Hi,
the "reference build" arima hdama seems to be too big when being compiled with gcc 3.3.1 or 3.3.3..
See attached log file.
Please, if anyone has time and test capabilities, have an eye on the non working ports and try to submit fixes.
Stefan
Stefan Reinauer stepan@openbios.org writes:
Hi,
the "reference build" arima hdama seems to be too big when being compiled with gcc 3.3.1 or 3.3.3..
So gcc 3.4 does produce smaller code. :)
I'm pushing things pretty close so I will take a look and see if I can get some more breathing room.
Eric
Stefan Reinauer stepan@openbios.org writes:
Hi,
the "reference build" arima hdama seems to be too big when being compiled with gcc 3.3.1 or 3.3.3..
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
Eric
On Thu, 2004-10-28 at 06:29, Eric W. Biederman wrote:
Stefan Reinauer stepan@openbios.org writes:
Hi,
the "reference build" arima hdama seems to be too big when being compiled with gcc 3.3.1 or 3.3.3..
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
Why don't we just enlarge the size to, say, 128kB ??
Ollie
Eric _______________________________________________ Linuxbios mailing list Linuxbios@clustermatic.org http://www.clustermatic.org/mailman/listinfo/linuxbios
On Thu, 28 Oct 2004, Li-Ta Lo wrote:
On Thu, 2004-10-28 at 06:29, Eric W. Biederman wrote:
Stefan Reinauer stepan@openbios.org writes:
Hi,
the "reference build" arima hdama seems to be too big when being compiled with gcc 3.3.1 or 3.3.3..
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
Why don't we just enlarge the size to, say, 128kB ??
YES!
Rather than remove debugging names, let's to ahead and move things around! I've done this once and it worked. We need to fix the .lds for things.
I have found the debugging names to be useful in the arium context, where i can browse memory and look for strings etc. We should be able to print these out at some points.
So move the c_payload to 0xfffe0000 and you'll have tons of room. So why not do that?
Please don't remove the debugging names, the gain is very small and I think that would be a mistake.
ron
On Thu, 2004-10-28 at 13:25, Ronald G. Minnich wrote:
Why don't we just enlarge the size to, say, 128kB ??
YES!
Rather than remove debugging names, let's to ahead and move things around! I've done this once and it worked. We need to fix the .lds for things.
Last time I tried to do that it was not that simple. The .lds is a mix of "#include" and "#define" in the ldscript and depends on the config too much.
Ollie
I have found the debugging names to be useful in the arium context, where i can browse memory and look for strings etc. We should be able to print these out at some points.
So move the c_payload to 0xfffe0000 and you'll have tons of room. So why not do that?
Please don't remove the debugging names, the gain is very small and I think that would be a mistake.
ron
On Thu, 28 Oct 2004, Li-Ta Lo wrote:
Last time I tried to do that it was not that simple. The .lds is a mix of "#include" and "#define" in the ldscript and depends on the config too much.
it was not simple but I got it to work ...
ron
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
hmm, this sounds like a lot of potential pain for a small gain. Can we talk about this a bit first.
ron
On Thu, 2004-10-28 at 08:14, Ronald G. Minnich wrote:
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
hmm, this sounds like a lot of potential pain for a small gain. Can we talk about this a bit first.
Some points I didn't make clear during the LNXI visit about the size problem are:
1. Virtually no one is putting the kernel in the FLASH ROM, people use CF instead. Every payload/bootloader we are using can support that. 2. The size of FLASH grows exponentially if there is no stupid limitation such as address lines in DIP32 package. Currently LinuxBIOS only grows linearly.
So why can not we relax the 64kB limit of LinuxBIOS ? I do think removing the name of the device is a wrong direction.
ollie
Li-Ta Lo wrote:
So why can not we relax the 64kB limit of LinuxBIOS ? I do think removing the name of the device is a wrong direction.
Why is this there in the first place?
On Thu, 2004-10-28 at 13:33, Richard Smith wrote:
Li-Ta Lo wrote:
So why can not we relax the 64kB limit of LinuxBIOS ? I do think removing the name of the device is a wrong direction.
Why is this there in the first place?
Are you asking the 64KB limit? One reason is to prevent "bloating" in LinuxBIOS. The other reason is some chip (AMD8111?) can only access the top 64kB of flash after boot.
Ollie
* Li-Ta Lo ollie@lanl.gov [041028 22:52]:
Are you asking the 64KB limit? One reason is to prevent "bloating" in LinuxBIOS. The other reason is some chip (AMD8111?) can only access the top 64kB of flash after boot.
We could declare some size limit per chipset if it has such a limitation?
OTOH, 64k should be enough for the setup task. I would have expected that long term this can be pushed even further down again...?!
Is there a rough estimation which part takes how much in a typical LinuxBIOS (AMD64) port? Maybe it makes sense to go look at the biggest chunks first.
Stefan
On Fri, 29 Oct 2004, Stefan Reinauer wrote:
OTOH, 64k should be enough for the setup task. I would have expected that long term this can be pushed even further down again...?!
Is there a rough estimation which part takes how much in a typical LinuxBIOS (AMD64) port? Maybe it makes sense to go look at the biggest chunks first.
last time I looked, romcc output (depending on board) was bumping in at 24-36K and c_payload made up the rest. So on some ports it is getting tight.
today, on arima, payload is 28K, and the romcc output is big enough that it won't fit in 64k. That was not the case yesterday :-)
ron
On Thu, 28 Oct 2004, Richard Smith wrote:
Why is this there in the first place?
most chipsets come up with only the top 64KB of flash enabled.
In the old days, we didn't turn that on until linuxbios was running. It's historical.
ron
"Ronald G. Minnich" rminnich@lanl.gov writes:
On Thu, 28 Oct 2004, Richard Smith wrote:
Why is this there in the first place?
most chipsets come up with only the top 64KB of flash enabled.
In the old days, we didn't turn that on until linuxbios was running. It's historical.
We have a hard limit that at least the romcc part of the codebase needs to run below that so we can enable more than 64KiB of flash.
It is my intention once I get the minor details firmed up to reexamine romcc and see what I can do size wise. I have come close to getting things smaller but there is some little glitch preventing things from working :(
Eric
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
We have a hard limit that at least the romcc part of the codebase needs to run below that so we can enable more than 64KiB of flash.
yah. But romcc does a good job. When I move c_payload to 0xfffe0000 there was a HUGE amount of room left for romcc ...
It is my intention once I get the minor details firmed up to reexamine romcc and see what I can do size wise. I have come close to getting things smaller but there is some little glitch preventing things from working :(
I think romcc is pretty darn good as is ... we just need to move c_payload (IMHO).
ron
Li-Ta Lo ollie@lanl.gov writes:
On Thu, 2004-10-28 at 08:14, Ronald G. Minnich wrote:
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
hmm, this sounds like a lot of potential pain for a small gain. Can we talk about this a bit first.
Some points I didn't make clear during the LNXI visit about the size problem are:
- Virtually no one is putting the kernel in the FLASH ROM, people use CF instead. Every payload/bootloader we are using can support that.
With kexec finally in the kernel and a knowledge of what is needed to keep kernel size under control and an availability of 1MiB flash parts, it is my intention to start soon. But it requires a moment to breath. What seems clear is that while we have hit a plateau with a working set of bootloaders we need to do something better.
As for CF. I guess I have only seen LANL doing that, and it seems a little unsatisfactory. There is no reason why beoboot needs that much space for example.
- The size of FLASH grows exponentially if there is no stupid limitation such as address lines in DIP32 package. Currently LinuxBIOS only grows linearly.
Hmm. I don't have enough data points to support that yet.
Currently romcc makes code that is 3x larger but more maintainable. 512KiB flash parts seem to be the norm for server boards. Although I think I have sighted 1MiB parts in the wild.
Still my fallback image size has already moved from 64KiB to 128KiB. And I really don't want to see it get much larger...
So why can not we relax the 64kB limit of LinuxBIOS ?
If we can still set an arbitrary limit when we rearrange the code I don't have a problem. For the current set of ports I don't need to and I would rather trim a little fat then to push things very far.
One of the challenges is that to keep the flashing sane the fallback image needs to fall nicely on flash block boundaries. Which are 64KiB in the worst case.
I do think removing the name of the device is a wrong direction.
It is wrong to remove the unused name of the chip?
Eric
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
With kexec finally in the kernel
good to hear :-)
Is this 2.6.9 or ...
thanks
ron
On Thu, Oct 28, 2004 at 05:34:58PM -0600, Ronald G. Minnich wrote:
Is this 2.6.9 or ...
-mm only right now, I believe (don't recall it showing up on bkbits). But I seem to recall that Andrew Morton is close to pushing it through to Linus's tree.
On Thu, 28 Oct 2004 linuxbios@mikebell.org wrote:
-mm only right now, I believe (don't recall it showing up on bkbits). But I seem to recall that Andrew Morton is close to pushing it through to Linus's tree.
once it's in we're going to stop worrying about kmonte any more :-)
but kmonte is not the reason we use CF. Driver size of stuff like myrinet and infiniband and quadrics is. Our big clusters only have ethernet on 1/16 of hte nodes.
ron
"Ronald G. Minnich" rminnich@lanl.gov writes:
On Thu, 28 Oct 2004 linuxbios@mikebell.org wrote:
-mm only right now, I believe (don't recall it showing up on bkbits). But I seem to recall that Andrew Morton is close to pushing it through to Linus's tree.
once it's in we're going to stop worrying about kmonte any more :-)
but kmonte is not the reason we use CF. Driver size of stuff like myrinet and infiniband and quadrics is. Our big clusters only have ethernet on 1/16 of hte nodes.
To be clear. The syscall number is reserved in the stable tree. The -mm tree is the current linux development tree. So the code is in. The ramifications of how to integrate kexec on panic for crash dumps are currently being worked through. Once the implications for implementation are understood kexec and the implementation is good the code will land in the stable kernel.
So if there is any feedback that needs to happen before the syscall interface becomes immutable now is the time to give it.
As for size, the beoboot userland is one of the reasons why you are still using CF. But that can be tuned.
Eric
On Fri, 29 Oct 2004, Eric W. Biederman wrote:
As for size, the beoboot userland is one of the reasons why you are still using CF. But that can be tuned.
No, beoboot really is not the reason at all!
We had a cluster at SC '03 with linuxbios/linux/beoboot in the flash part, NOT the cf. That booted over ethernet (faster than etherboot, which was interesting). As soon as you put IB drivers or myrinet drivers or quadrics drivers into the picture, you need the CF. That's just the way it is. Those drivers and their support code push the kernel size over the edge.
Plus, on Pink, the 1024 node cluster, we have a fallback/normal linuxbios, fallback/normal FILO in flash; the fallback/normal kernel and initrd are in CF. There's no room at all to do this in a 1 MB flash part.
Use of CF is not a beoboot/kexec issue at all.
ron
On Fri, 2004-10-29 at 17:38, Eric W. Biederman wrote:
As for size, the beoboot userland is one of the reasons why you are still using CF. But that can be tuned.
I see no relation between CF an beoboot. As I said, not everyone is happy with booting kernel over network. They don't use beoboot an they want CF !!!
Ollie
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
As for CF. I guess I have only seen LANL doing that, and it seems a little unsatisfactory. There is no reason why beoboot needs that much space for example.
the reason is that myrinet and other drivers need that much space.
CF is our future until the flash parts hit at least 4 MB.
ron
On Thu, 2004-10-28 at 16:58, Eric W. Biederman wrote:
As for CF. I guess I have only seen LANL doing that, and it seems a little unsatisfactory. There is no reason why beoboot needs that much space for example.
I bet there are a lot of people using CF in the embedded world, they can't use Etherboot to fetch the kernel from the net as you do. We should not only think about HPC applications. People in embedded market are in different enviroment and have different cost structure than you.
I do think removing the name of the device is a wrong direction.
It is wrong to remove the unused name of the chip?
It is not wrong to remove 'unused' name if it is actually 'unused'. It is wrong to ignore other people's need and refuse to consider alternatives. In my point of view, removing the .name is what those CS people called "premature optimization".
I think the most important issue is you tried to use ldscript magic to layout the rom image. But it is still somehow 'hardcoded'. If it is really that magical, people should have no problem changing the rom layout by editing their target/Config.lb. and put the c_payload wherever they want.
Ollie
On Fri, 29 Oct 2004, Li-Ta Lo wrote:
I bet there are a lot of people using CF in the embedded world, they can't use Etherboot to fetch the kernel from the net as you do. We should not only think about HPC applications. People in embedded market are in different enviroment and have different cost structure than you.
good point. The Sandia mini clusters use CF.
ron
Li-Ta Lo wrote:
On Thu, 2004-10-28 at 16:58, Eric W. Biederman wrote:
As for CF. I guess I have only seen LANL doing that, and it seems a little unsatisfactory. There is no reason why beoboot needs that much space for example.
I bet there are a lot of people using CF in the embedded world, they can't use Etherboot to fetch the kernel from the net as you do. We should not only think about HPC applications. People in embedded market are in different enviroment and have different cost structure than you.
I'm using a CF on our board.
I think the most important issue is you tried to use ldscript magic to layout the rom image. But it is still somehow 'hardcoded'. If it is really that magical, people should have no problem changing the rom layout by editing their target/Config.lb. and put the c_payload wherever they want.
I've got a concern/question about moving the payload location. Are we talking about the location in RAM?
My ADLO+BOCHS+vbios payload is 130kib. So moving the payload to 0x0e0000 in the ROM is going to cause me issues.
What exactly is the memory map of linuxBIOS? I've kinda looked at the ldscript but since I've never needed to chage it I must admit that its mostly magic to me.
On Fri, 29 Oct 2004, Richard Smith wrote:
I've got a concern/question about moving the payload location. Are we talking about the location in RAM?
not at all! We're only talking about how the linuxbios code is set up in flash. This has nothing to do with RAM locations.
The issue is this: right now, linubios compressed payload AND romcc object fit in the top 64k. There is no need to do this. They're not fitting anyway at this point.
romcc object startup can turn on full flash access, so the linuxbios compressed payload can be placed, not in the top 64k, but in the next-to-top 64k. The linuxbios image will then have 128k available to it, not just 64k. Then we can put all the debug prints we want into romcc code, which would be nice.
My ADLO+BOCHS+vbios payload is 130kib. So moving the payload to 0x0e0000 in the ROM is going to cause me issues.
How big is your flash?
ron
On Fri, 29 Oct 2004, Ronald G. Minnich wrote:
romcc object startup can turn on full flash access, so the linuxbios compressed payload can be placed, not in the top 64k, but in the next-to-top 64k. The linuxbios image will then have 128k available to it, not just 64k. Then we can put all the debug prints we want into romcc code, which would be nice.
I forgot to add: it could also be at 0xfffe8000, i.e. on a 32k boundary, and ... I've tested this and it works.
ron
Ronald G. Minnich wrote:
The issue is this: right now, linubios compressed payload AND romcc object fit in the top 64k. There is no need to do this. They're not fitting anyway at this point.
romcc object startup can turn on full flash access, so the linuxbios compressed payload can be placed, not in the top 64k, but in the next-to-top 64k. The linuxbios image will then have 128k available to it, not just 64k. Then we can put all the debug prints we want into romcc code, which would be nice.
So how is my stuff working then? Its way over 64k. Guess I'm still confused. Can you draw me a memory map?
My ADLO+BOCHS+vbios payload is 130kib. So moving the payload to 0x0e0000 in the ROM is going to cause me issues.
How big is your flash?
512kib actually I have 2 of them for 1 meg but the second has to be muxed and currently unused.
On Fri, 29 Oct 2004, Richard Smith wrote:
Ronald G. Minnich wrote:
The issue is this: right now, linubios compressed payload AND romcc object fit in the top 64k. There is no need to do this. They're not fitting anyway at this point. romcc object startup can turn on full flash access, so the linuxbios compressed payload can be placed, not in the top 64k, but in the next-to-top 64k. The linuxbios image will then have 128k available to it, not just 64k. Then we can put all the debug prints we want into romcc code, which would be nice.
old-fashioned linuxbios
----------- (e.g.) 0xfff80000 | payload | | | | | ----------- 0xffff0000 | linux | | bios | ----------|
V2:
--------------0xfff80000 | smith | | payload | --------------0xffff0000 | c_payload | | uncompress | | to ram | | ---- | | auto.C | | compiled | | code | --------------
So there are really two "payloads"
There is startup code -- now compiled by romcc -- that turns on ram and uncompresses the c_payload to ram. The RAM code then does final setup and uncompresses the "smith payload" or whatever (Etherboot, filo, linux, ADLO) to ram.
So I'm just saying do this.
--------------0xfff80000 | smith | | payload | --------------0xfffe0000 | c_payload | | uncompress | | to ram | |------------|0xffff0000 | auto.C | | compiled | | code | --------------
That's all you have to do to fix the issues with romcc getting too big.
ron
On Fri, 2004-10-29 at 11:36, Richard Smith wrote:
I'm using a CF on our board.
I think the most important issue is you tried to use ldscript magic to layout the rom image. But it is still somehow 'hardcoded'. If it is really that magical, people should have no problem changing the rom layout by editing their target/Config.lb. and put the c_payload wherever they want.
I've got a concern/question about moving the payload location. Are we talking about the location in RAM?
No, we are talking about the ROM image layout no how they are copied to the RAM. Currently, if you don't put the kernel in the ROM, most of the ROM space is 'unused'.
My ADLO+BOCHS+vbios payload is 130kib. So moving the payload to 0x0e0000 in the ROM is going to cause me issues.
What exactly is the memory map of linuxBIOS? I've kinda looked at the ldscript but since I've never needed to chage it I must admit that its mostly magic to me.
That is exactly my point. The way ldscript is done is too diffcult to understand, the different xxx_ROM_yyy options do not help neither.
Ollie
Li-Ta Lo wrote:
On Thu, 2004-10-28 at 16:58, Eric W. Biederman wrote:
As for CF. I guess I have only seen LANL doing that, and it seems a little unsatisfactory. There is no reason why beoboot needs that much space for example.
I bet there are a lot of people using CF in the embedded world, they can't use Etherboot to fetch the kernel from the net as you do. We should not only think about HPC applications. People in embedded market are in different enviroment and have different cost structure than you.
I haven't been following many linuxbios threads but this one caught my eye.
Though we are still using a very old version of Linuxbios and thus not currently doing any development here :-( we do boot from a CF and the ability to do so is very important.
Ollie is right, embedded usage patterns are VERY different from HPC.
Cheers! Ty
"Ronald G. Minnich" rminnich@lanl.gov writes:
On Thu, 28 Oct 2004, Eric W. Biederman wrote:
I will (hopefully) commit later today, but I have found one general technique that will cut down the code size but a small amount. In the chip_operations struct we have a name for debugging. Since we are no longer printing that name there is no point in keeping it.
hmm, this sounds like a lot of potential pain for a small gain. Can we talk about this a bit first.
Sure. First here are two examples. struct chip_opertations cpu_intel_socket_mPGA479M_control = { .name = "socket mPGA479M", };
Killing the name for cpu sockets allows us to kill the whole structure.
struct chip_operations northbridge_intel_e7501_ops = { .name = "intel E7501 Northbridge", .enable_dev = enable_dev, };
First note that all we did with the names was to print them when we converted from the static to the dynamic tree. Nothing uses them anymore.
Killing them is as simple as removing a structure member. And deleting a few lines of code. Bloatwise strings appear to be up towards the top of the list.
This is really a continuation of killing the static device tree.
If you like I will trade you gdb stubs for the currently useless debugging string names. ;) I just finished the code for that. The conversations of the last couple of days inspired me. :)
Eric
* Eric W. Biederman ebiederman@lnxi.com [041029 00:44]:
hmm, this sounds like a lot of potential pain for a small gain. Can we talk about this a bit first.
Sure. First here are two examples. struct chip_opertations cpu_intel_socket_mPGA479M_control = { .name = "socket mPGA479M", };
Killing the name for cpu sockets allows us to kill the whole structure.
What was the original intention of the structure? It might be nice to know how many sockets there are and what CPUs are in those sockets.. (ie some dmi kind of information)
If you like I will trade you gdb stubs for the currently useless debugging string names. ;) I just finished the code for that. The conversations of the last couple of days inspired me. :)
Hey.. this sounds cool!
Stefan
Stefan Reinauer stepan@openbios.org writes:
- Eric W. Biederman ebiederman@lnxi.com [041029 00:44]:
hmm, this sounds like a lot of potential pain for a small gain. Can we talk about this a bit first.
Sure. First here are two examples. struct chip_opertations cpu_intel_socket_mPGA479M_control = { .name = "socket mPGA479M", };
Killing the name for cpu sockets allows us to kill the whole structure.
What was the original intention of the structure? It might be nice to know how many sockets there are and what CPUs are in those sockets.. (ie some dmi kind of information)
We still have the cpu information via the device tree. As far as distinguishing cpus from sockets that is mostly a matter of refinement.
The code is currently structured so you pull in support for which cpus your socket supports. Which is the intention of having a socket directory.
When we started generating device structures directly almost all of the chip specific logic when away. But just a tiny bit was left because there are something things that you do on a chip basis rather than on a device basis.
struct chip is gone. struct chip_control was replaced by struct chip_operations. Which has now been pruned down to just one method, at least for now.
If you like I will trade you gdb stubs for the currently useless debugging string names. ;) I just finished the code for that. The conversations of the last couple of days inspired me. :)
Hey.. this sounds cool!
Done. Check it out :)
Eric
On Sat, 30 Oct 2004, Stefan Reinauer wrote:
What was the original intention of the structure? It might be nice to know how many sockets there are and what CPUs are in those sockets.. (ie some dmi kind of information)
it would have allowed for more debug information, but it appear that requests from a number of people to leave it in there were ignored.
ron
"Ronald G. Minnich" rminnich@lanl.gov writes:
On Sat, 30 Oct 2004, Stefan Reinauer wrote:
What was the original intention of the structure? It might be nice to know how many sockets there are and what CPUs are in those sockets.. (ie some dmi kind of information)
it would have allowed for more debug information, but it appear that requests from a number of people to leave it in there were ignored.
Nothing was ignored but no case was presented for retention of the .name member.
No one has even given me the details of how they expect the structure element to be used much less a working code example.
Eric
-
ebiederman＠lnxi.com -
Li-Ta Lo -
linuxbios＠mikebell.org -
Richard Smith -
Ronald G. Minnich -
Stefan Reinauer -
tyson＠irobot.com