[LinuxBIOS] printk()/post_code(): Compile failure for all hardware targets in v3

[Carl-Daniel Hailfinger]

Hi,

I can't compile any of the hardware targets in v3. The qemu target
builds just fine.

Error for the AMD Norwich target:
In file included from LinuxBIOSv3/mainboard/amd/norwich/initram.c:24:
include/post_code.h:23: error: ‘post_code’ redeclared as different kind
of symbol
include/console.h:38: error: previous declaration of ‘post_code’ was here

Error for the Advanced Digital Logic MSM800SEV:
In file included from LinuxBIOSv3/mainboard/adl/msm800sev/initram.c:24:
include/post_code.h:23: error: ‘post_code’ redeclared as different kind
of symbol
include/console.h:38: error: previous declaration of ‘post_code’ was here

Error for the Artec Group DBE61:
In file included from LinuxBIOSv3/mainboard/artecgroup/dbe61/initram.c:24:
include/post_code.h:23: error: ‘post_code’ redeclared as different kind
of symbol
include/console.h:38: error: previous declaration of ‘post_code’ was here

The reason is similar for all of them:
include/console.h specifies
> void post_code(u8 value);

include/post_code.h specifies (because _SHARED is defined)
> void stage0_post_code(u8 value) ;
> void (*post_code)(u8 value) = stage0_post_code;

Of  course these definitions do conflict.

While I appreciate the idea to use one printk()/post_code()
implementation for all stages, it seems the current realization has a
few aspects which give us various interesting problems to solve:
* The conflicting post_code() definitions above are not alone, the same
happens for printk.
* If you remove post_code() from include/console.h everywhere
post_code() is shared, you encounter the next problem: If two files with
_SHARED are linked together, each of them will contain the assignment
(*post_code)=stage0_post_code, resulting in linker errors because a
symbol appears twice.
* Even if you manage to avoid all the problems above, a new problem
arises: We have to build a LAR archive in one continuous flow because we
can't extract the location of the stage0 symbols from an existing
bootblock in a LAR archive and thus can't link initram and stage2
against an existing bootblock. Because of that, we never can do partial
BIOS updates, which defeats the whole purpose of LAR.

Suggestions for solving the problems mentioned above:
* Always wrap shared function definitions in SHARED macros.
* Make sure the assignment "ret (*func)(args) attr= stage0_##func"
happens only once per final linked object.
* Include a .map file of shared stage0 functions in the LAR.

Regards,
Carl-Daniel

[LinuxBIOS] printk()/post_code(): Compile failure for all hardware targets in v3

[Stefan Reinauer]

* Carl-Daniel Hailfinger  [071120 11:46]:
> The reason is similar for all of them:
> include/console.h specifies
> > void post_code(u8 value);
>
> include/post_code.h specifies (because _SHARED is defined)
> > void stage0_post_code(u8 value) ;
> > void (*post_code)(u8 value) = stage0_post_code;
>
> Of  course these definitions do conflict.

This issue seems trivial. include/console.h must not define a prototype
for post_code. I wonder how that double definition sneaked.
> The conflicting post_code() definitions above are not alone, the same
>
> happens for printk.

How so? I could not find a printk prototype in any other file. Where's
the bad part?
> If you remove post_code() from include/console.h everywhere
>
> post_code() is shared, you encounter the next problem: If two files with
> _SHARED are linked together, each of them will contain the assignment
> (*post_code)=stage0_post_code, resulting in linker errors because a
> symbol appears twice.

This is interesting. I never had any of these problems and I have been
testing that code quite a bit. What distribution and what toolchain are
you using?
> Even if you manage to avoid all the problems above, a new problem
>
> arises: We have to build a LAR archive in one continuous flow because we
> can't extract the location of the stage0 symbols from an existing
> bootblock in a LAR archive and thus can't link initram and stage2
> against an existing bootblock. Because of that, we never can do partial
> BIOS updates, which defeats the whole purpose of LAR.

Wait: You can not change the toolchain nor the version of the bootblock
in between. This is only a rather small limitation. But its not very
elegant, I agree.

Which is why I suggested a function pointer array with defined function
pointers at fixed, defined offsets. Yes. This means we have to define an
interface, something the Linux guys really hate.

The Amiga did a very similar thing. All functions in a shared library
would be available through a function pointer array. The callable
functions would be defined through the library version. So a program
could always react on finding a too old or not-existing library sanely
instead of just spitting out a linker error like our unix/elf based systems
do these days.
> Suggestions for solving the problems mentioned above:
> Always wrap shared function definitions in SHARED macros.

yes. this is how it should be done.
> Make sure the assignment "ret (*func)(args) attr= stage0_##func"
>
> happens only once per final linked object.

agreed.
> Include a .map file of shared stage0 functions in the LAR.

My first thought. but the map file is not sufficient for linking with
those symbols. You need the object file for that. Which is bad.

[LinuxBIOS] printk()/post_code(): Compile failure for all hardware targets in v3

[Carl-Daniel Hailfinger]

On 20.11.2007 14:14, Stefan Reinauer wrote:
> Carl-Daniel Hailfinger  [071120 11:46]:
> > The reason is similar for all of them:
> > include/console.h specifies
> > > void post_code(u8 value);
> >
> > include/post_code.h specifies (because _SHARED is defined)
> > > void stage0_post_code(u8 value) ;
> > > void (*post_code)(u8 value) = stage0_post_code;
> >
> > Of  course these definitions do conflict.
>
> This issue seems trivial. include/console.h must not define a prototype
> for post_code. I wonder how that double definition sneaked.

Because simply deleting it causes compilation problems somewhere else?
> > The conflicting post_code() definitions above are not alone, the same
> >
> > happens for printk.
>
> How so? I could not find a printk prototype in any other file. Where's
> the bad part?

Maybe I mixed that up with the multiple printk definitions below.
> > If you remove post_code() from include/console.h everywhere
> >
> > post_code() is shared, you encounter the next problem: If two files with
> > _SHARED are linked together, each of them will contain the assignment
> > (*post_code)=stage0_post_code, resulting in linker errors because a
> > symbol appears twice.
>
> This is interesting. I never had any of these problems and I have been
> testing that code quite a bit. What distribution and what toolchain are
> you using?

openSUSE 10.3 (i386)
gcc (GCC) 4.2.1 (SUSE Linux)
GNU ld (GNU Binutils) 2.17.50.20070726-14 (SUSE Linux)
> > Even if you manage to avoid all the problems above, a new problem
> >
> > arises: We have to build a LAR archive in one continuous flow because we
> > can't extract the location of the stage0 symbols from an existing
> > bootblock in a LAR archive and thus can't link initram and stage2
> > against an existing bootblock. Because of that, we never can do partial
> > BIOS updates, which defeats the whole purpose of LAR.
>
> Wait: You can not change the toolchain nor the version of the bootblock
> in between. This is only a rather small limitation. But its not very
> elegant, I agree.

My point was: For a given bootblock, we can't build the rest of the ROM
based on the bootblock alone because it has been stripped of symbols.
> Which is why I suggested a function pointer array with defined function
> pointers at fixed, defined offsets. Yes. This means we have to define an
> interface, something the Linux guys really hate.

I see no way around a function pointer array. However, we could avoid
the fixed offsets if we use the same technology the Linux kernel uses
for its symbol tables.
> The Amiga did a very similar thing. All functions in a shared library
> would be available through a function pointer array. The callable
> functions would be defined through the library version. So a program
> could always react on finding a too old or not-existing library sanely
> instead of just spitting out a linker error like our unix/elf based systems
> do these days.

That's probably overkill.
> > Suggestions for solving the problems mentioned above:
> > Always wrap shared function definitions in SHARED macros.
>
> yes. this is how it should be done.

Can you prepare a patch?
> > Make sure the assignment "ret (*func)(args) attr= stage0_##func"
> >
> > happens only once per final linked object.
>
> agreed.

That will get messy. Maybe move the assignment to a separate object
which is linked to the final object?
> > Include a .map file of shared stage0 functions in the LAR.
>
> My first thought. but the map file is not sufficient for linking with
> those symbols. You need the object file for that. Which is bad.

Can we recreate an object file from the map file? Or can we avoid
stripping all symbols from stage0/1?

Regards,
Carl-Daniel

[LinuxBIOS] printk()/post_code(): Compile failure for all hardware targets in v3

[Stefan Reinauer]

* Carl-Daniel Hailfinger  [071120 17:44]:
> On 20.11.2007 14:14, Stefan Reinauer wrote:
> > > include/post_code.h specifies (because _SHARED is defined)
> > > > void stage0_post_code(u8 value) ;
> > > > void (*post_code)(u8 value) = stage0_post_code;
> > >
> > > Of  course these definitions do conflict.
> >
> > This issue seems trivial. include/console.h must not define a prototype
> > for post_code. I wonder how that double definition sneaked.
>
> Because simply deleting it causes compilation problems somewhere else?

So including post_code.h instead does not work? (Except the multiple
definitions problem)
> > This is interesting. I never had any of these problems and I have been
> > testing that code quite a bit. What distribution and what toolchain are
> > you using?
>
> openSUSE 10.3 (i386)
> gcc (GCC) 4.2.1 (SUSE Linux)
> GNU ld (GNU Binutils) 2.17.50.20070726-14 (SUSE Linux)

I think I used the 10.2 toolchain back then.

I tried to compile v3 with

gcc version 4.3.0 20071016 (experimental) [trunk revision 129378] (SUSE Linux) 
GNU ld (GNU Binutils) 2.18.50.20071002-5 (SUSE Linux)

and I get lot and lots of errors in other pieces of the code. It's a
pity that gcc's understanding of what code is valid and what is not is
so volatile between the minor versions.
> My point was: For a given bootblock, we can't build the rest of the ROM
> based on the bootblock alone because it has been stripped of symbols.
> > Which is why I suggested a function pointer array with defined function
> > pointers at fixed, defined offsets. Yes. This means we have to define an
> > interface, something the Linux guys really hate.
>
> I see no way around a function pointer array. However, we could avoid
> the fixed offsets if we use the same technology the Linux kernel uses
> for its symbol tables.

What's the benefit? We don't have all the space available that Linux
freely consumes these days. Especially not if people want to use LAP
> > The Amiga did a very similar thing. All functions in a shared library
> > would be available through a function pointer array. The callable
> > functions would be defined through the library version. So a program
> > could always react on finding a too old or not-existing library sanely
> > instead of just spitting out a linker error like our unix/elf based systems
> > do these days.
>
> That's probably overkill.

A lot simpler than what Linux does.
> > > Suggestions for solving the problems mentioned above:
> > > Always wrap shared function definitions in SHARED macros.
> >
> > yes. this is how it should be done.
>
> Can you prepare a patch?

I'll try, but it will be a couple of days/weeks.
> > > Make sure the assignment "ret (*func)(args) attr= stage0_##func"
> > >
> > > happens only once per final linked object.
> That will get messy. Maybe move the assignment to a separate object
> which is linked to the final object?

Yes. the macro nature of that code should make that simple.

Isn't there something like extern inline?
> Can we recreate an object file from the map file? Or can we avoid
> stripping all symbols from stage0/1?

Nope, and nope, unfortunately. The unstripped bootblock is an ELF file,
but we need a binary blob there, so we have the reset vector and such in
the right place...

[LinuxBIOS] printk()/post_code(): Compile failure for all hardware targets in v3

[Carl-Daniel Hailfinger]

On 20.11.2007 18:12, Stefan Reinauer wrote:
> Carl-Daniel Hailfinger  [071120 17:44]:
> > Can we recreate an object file from the map file? Or can we avoid
> > stripping all symbols from stage0/1?
>
> Nope, and nope, unfortunately. The unstripped bootblock is an ELF file,
> but we need a binary blob there, so we have the reset vector and such in
> the right place...

We can recreate an object file from a small symbol file (similar to a
map file). Ph33r my m4d sk1llz! (I have no idea whether that is correct
l33t-speak.)

Do not try to understand the command sequence below. It will rot your
brain. It was written by me with the help of sleep deprivation.
Do not try to be clever with the command sequence below. objcopy in one
step works, but the output file is 8 times bigger, plus you will trigger
an assertion in BFD.
Do not try to use the command sequence below for any architecture
besides x86(32bit).

objcopy --extract-symbol --strip-all --keep-symbols=keepsymbols.txt -O symbolsrec build/stage0.o build/stage0-onlysymbols.sym
objcopy -I symbolsrec -O elf32-i386 build/stage0-onlysymbols.sym build/stage0-onlysymbols.o
echo -en "\003\000"|dd of=build/stage0-onlysymbols.o bs=1 count=2 seek=18 conv=notrunc

keepsymbols.txt must contain all shared symbol names from stage 0, one
symbol name per line.
stage0-onlysymbols.o contains all shared symbols and has to be shipped
in the LAR. We can link initram and stage2 against it without any problems.

If we use LZMA compression for stage0-onlysymbols.o, it needs about 200
bytes. The qemu bootblock currently has about 4600 bytes free, so there
is plenty of space for such a symbol table.

Comments?

Regards,
Carl-Daniel