RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

      Thinking about ADLO and the shadow enable/disable got some wheels turning.

I've been spending lots of time in V2 and I was wondering if the same
type of methodology can't work for ADLO.

In V2 there are specifc .c files that do thing in a chipset specific
way and the auto.c includes them as necesary.  Those get compiled and
then stuck in the startup assembly.

What if we created a shadow.c file that was in the northbridge
directory with a simple API type setup that enabled and disabled the
various shadow ranges.

Then the ADLO build system could just romcc these files and the loader
would include them just like crt0.S does.  That way there are no
callbacks across the payload boundry yet you could setup ADLO or other
payloads so they would work in a chipset agnostic way.

I suppose that that FILO and the emulator render most of ADLO
un-needed but if you are going to try and use FreeDOS or Windoze as an
OS then you will need ADLO.

-- 
Richard A. Smith

[Ronald G. Minnich]

On Wed, 26 Jan 2005, Richard Smith wrote:

  Then the ADLO build system could just romcc these
files and the loader
 would include them just like crt0.S does.
it makes sense except ... why romcc? You've got memory at this point, and 
Eric was pretty careful to make sure the same code will compile and work 
on gcc and romcc. 

ron

[BULK] RFC: Generic shadow mechanism useable from a payload

      Richard Smith <smithbone(a)gmail.com&gt; writes:

  Thinking about ADLO and the shadow enable/disable got
some wheels turning.
 
 I've been spending lots of time in V2 and I was wondering if the same
 type of methodology can't work for ADLO.
 
 In V2 there are specifc .c files that do thing in a chipset specific
 way and the auto.c includes them as necesary.  Those get compiled and
 then stuck in the startup assembly.
 
 What if we created a shadow.c file that was in the northbridge
 directory with a simple API type setup that enabled and disabled the
 various shadow ranges.
That is a fairly sane way to get code sharing, if we need it.  The code
is small enough and simple enough cut-and-paste code sharing is probably
just as good.  All you need to do is look at which chipset you are running
on.  We only need to drop out support if it becomes too much code having
support for all of the chipsets and cpus.

But I actually think that is overkill. v2 by default and design
enables all of the shadow ranges as memory.  So we just need to
use those ranges.  That should be much easier having to patch
ADLO each time.  It is fairly unlikely a writable ROM segment is
going to break anything. 

Once we find the application a writable rom segment breaks I will
be happy to reconsider, solutions.

  I suppose that that FILO and the emulator render most
of ADLO
 un-needed but if you are going to try and use FreeDOS or Windoze as an
 OS then you will need ADLO.
ADLO has a use case and for those 


Eric

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

  But I actually think that is overkill. v2 by default
and design
 enables all of the shadow ranges as memory.  So we just need to
 use those ranges.  That should be much easier having to patch
 ADLO each time.  It is fairly unlikely a writable ROM segment is
Ok.. Well when you put it that way yeah dosen't sound like its needed
at all. It should probally just work.

I't will be awhile but I guess I will find out when I get there.  I'm
planning on still trying to use ADLO when I get V2 up and going on our
board.

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

      On Wed, 26 Jan 2005 13:12:27 -0600, Richard Smith <smithbone(a)gmail.com&gt; wrote:
  But I actually
think that is overkill. v2 by default and design
 enables all of the shadow ranges as memory.  So we just need to
 use those ranges.  That should be much easier having to patch
 ADLO each time.  It is fairly unlikely a writable ROM segment is
 
 Ok.. Well when you put it that way yeah dosen't sound like its needed
 at all. It should probally just work.
Wait a minute.  I don't see how this will work.  To do the shadows you
need to read from the bios chip, yet write to the RAM.  Then re-enable
both read and writes to go to RAM.  This requires you to toggle some
bits in the norhtbridge.

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

      On Wed, 26 Jan 2005, Richard Smith wrote:

  On Wed, 26 Jan 2005 13:12:27 -0600, Richard Smith
<smithbone(a)gmail.com&gt; wrote:
  But I
actually think that is overkill. v2 by default and design
 enables all of the shadow ranges as memory.  So we just need to
 use those ranges.  That should be much easier having to patch
 ADLO each time.  It is fairly unlikely a writable ROM segment is
 Ok.. Well when you put it that way yeah dosen't sound like its needed
 at all. It should probally just work.
 Wait a minute.  I don't see how this will work.  To do the shadows you
 need to read from the bios chip, yet write to the RAM.  Then re-enable
 both read and writes to go to RAM.  This requires you to toggle some
 bits in the norhtbridge.
it is been a while but looking at loader.s


	http://cvs.sourceforge.net/viewcvs.py/freebios/freebios/util/ADLO/loader....

it mentions sources as 0x8000 and 0x18000, so I guess they have been 
aready copied out of rom into the the low memory by linuxbios ???

(i think that's part of the ELF header specs to tell LB where to put the 
image).

wget
http://cvs.sourceforge.net/viewcvs.py/*checkout*/freebios/freebios/util/A...

[adam@mtdew tmp]$ readelf -l elf-header-065kb.payload\?rev\=1.1

Elf file type is EXEC (Executable file)
Entry point 0x7c00
There are 1 program headers, starting at offset 64

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   LOAD           0x000100 0x00007c00 0x00007c00 0x10400 0x10400 R E 0x100

  I't will be awhile but I guess I will find out
when I get there.  I'm 
 planning on still trying to use ADLO when I get V2 up and going on our 
 board.
I'm looking forward hearing about results.

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

http://cvs.sourceforge.net/viewcvs.py/freebios/freebios/util/ADLO/loader....
 
 it mentions sources as 0x8000 and 0x18000, so I guess they have been
 aready copied out of rom into the the low memory by linuxbios ???
 
 (i think that's part of the ELF header specs to tell LB where to put the
 image).
So you think the ELF loader loads into ram and then loader moves it
into the right position?
I wonder if we could not just set things up where the ELF loader just
loas it into the right spot in the first place.  Perhaps that
overwrites where the ELF code lives?

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

      On Wed, 26 Jan 2005, Richard Smith wrote:

         
http://cvs.sourceforge.net/viewcvs.py/freebios/freebios/util/ADLO/loader....

 it mentions sources as 0x8000 and 0x18000, so I guess they have been
 aready copied out of rom into the the low memory by linuxbios ???

 (i think that's part of the ELF header specs to tell LB where to put the
 image).
 So you think the ELF loader loads into ram and then loader moves it
 into the right position?
I think so.

  I wonder if we could not just set things up where the
ELF loader just
 loas it into the right spot in the first place.  Perhaps that
 overwrites where the ELF code lives?
if the ram/shadowing was setup correctly _before_ payload is started, then 
perhaps.

but you still need to execute the switch to real mode code.

perhaps you could do that by the means of more elaborate elf header which 
tells

 	* load loader.s here
 	* load pc bios here
 	* load video bios here
 	* jump to loader.s

as far as I belive elf should spport loading multiple segments into 
arbirary places in ram.

[BULK] RFC: Generic shadow mechanism useable from a payload

      Adam Sulmicki <adam(a)cfar.umd.edu&gt; writes:

  On Wed, 26 Jan 2005, Richard Smith wrote:
  I wonder if we could not just set things up where
the ELF loader just
 loas it into the right spot in the first place.  Perhaps that
 overwrites where the ELF code lives?
 
 if the ram/shadowing was setup correctly _before_ payload is started, then
 perhaps.
Which it should be fairly universally doable with freebios2.
 
  but you still need to execute the switch to real mode
code.
 
 perhaps you could do that by the means of more elaborate elf header which tells
 
  	* load loader.s here
  	* load pc bios here
  	* load video bios here
  	* jump to loader.s
 
 as far as I belive elf should spport loading multiple segments into arbirary
 places in ram.
Yes you can do that with ELF.

The big gotcha is going to be that there are pirq tables current
stored at 0xf0000 that you are not going to want to stomp.

The fun question.  Can I make the bochs bios useable on a system
after the linux kernel has run.  Load it with kexec and then boot windows?

If LinuxBIOS is the native bios this should be no extra challenge.  If the
native BIOS is something else life gets a little more interesting.

Eric

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

  Yes you can do that with ELF.
 
 The big gotcha is going to be that there are pirq tables current
 stored at 0xf0000 that you are not going to want to stomp.
Actually thats not that much of an issue. It currently stomps it already

ADLO has that table broken out into a single binary file.  All I do is
copy the table binary table generated with linubios into the adlo
build file and it gets stuck in the adlo image.   Loader.s copies it
to a specific location.
So really if you loaded loader and ADLO code after the linuxbios PIRQ
table location and then tweaked loader to not make a PIRQ it should be
fine.

Not haveing to do the PIRQ table copy would be nice but its pretty
trival.  A script to do it automatically would be easy.
 
  The fun question.  Can I make the bochs bios useable
on a system
 after the linux kernel has run.  Load it with kexec and then boot windows?
 
 If LinuxBIOS is the native bios this should be no extra challenge.  If the
 native BIOS is something else life gets a little more interesting.
That _is_ a fun quesition..  I don't think you would have too many
problems though.  The ADLO bios is used to running on systems that
already have everything setup.  It does minimal hardware setup.  If
you put everything in the right spot and modified the loader so that
it didn't copy anything  I think it will work.

kexec loads an elf right?  So this would be pretty easy to test.

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

      Richard Smith <smithbone(a)gmail.com&gt; writes:

  Yes you can do
that with ELF.
 
 The big gotcha is going to be that there are pirq tables current
 stored at 0xf0000 that you are not going to want to stomp.
 
 Actually thats not that much of an issue. It currently stomps it already
 
 ADLO has that table broken out into a single binary file.  All I do is
 copy the table binary table generated with linubios into the adlo
 build file and it gets stuck in the adlo image.   Loader.s copies it
 to a specific location.
 So really if you loaded loader and ADLO code after the linuxbios PIRQ
 table location and then tweaked loader to not make a PIRQ it should be
 fine.
 
 Not haveing to do the PIRQ table copy would be nice but its pretty
 trival.  A script to do it automatically would be easy.
The fact that ADLO has to manually customized per board is what
keeps it from being a lot more generally usable.
  
  The fun
question.  Can I make the bochs bios useable on a system
 after the linux kernel has run.  Load it with kexec and then boot windows?
 
 If LinuxBIOS is the native bios this should be no extra challenge.  If the
 native BIOS is something else life gets a little more interesting.
 
 That _is_ a fun quesition..  I don't think you would have too many
 problems though.  The ADLO bios is used to running on systems that
 already have everything setup.  It does minimal hardware setup.  If
 you put everything in the right spot and modified the loader so that
 it didn't copy anything  I think it will work.
 
 kexec loads an elf right?  So this would be pretty easy to test.
Yes /sbin/kkexec loads an elf among other things.

Eric

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

  Yes you can do that with ELF.

 The big gotcha is going to be that there are pirq tables current
 stored at 0xf0000 that you are not going to want to stomp.
 Actually thats not that much of an issue. It currently stomps it already

 ADLO has that table broken out into a single binary file.  All I do is
 copy the table binary table generated with linubios into the adlo
 build file and it gets stuck in the adlo image.   Loader.s copies it
 to a specific location.
 So really if you loaded loader and ADLO code after the linuxbios PIRQ
 table location and then tweaked loader to not make a PIRQ it should be
 fine.

 Not haveing to do the PIRQ table copy would be nice but its pretty
 trival.  A script to do it automatically would be easy.
 The fact that ADLO has to manually customized per board is what
 keeps it from being a lot more generally usable.
something I have been wondering. Suppose someone starts working on that 
IDE driver to fix it for once in bochs bios. Won't we find out that 
getting it work right is chip-set specific thing?

curently bochs bios is based on intel 440fx chipset or some such.

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

  something I have been wondering. Suppose someone
starts working on that
 IDE driver to fix it for once in bochs bios. Won't we find out that
 getting it work right is chip-set specific thing?
 
 curently bochs bios is based on intel 440fx chipset or some such.
I don't think so.  FILO does it right.  Is it chipset specific?

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Stefan Reinauer]

      * Richard Smith <smithbone(a)gmail.com&gt; [050131 17:20]:
  something I
have been wondering. Suppose someone starts working on that
 IDE driver to fix it for once in bochs bios. Won't we find out that
 getting it work right is chip-set specific thing?
 
 curently bochs bios is based on intel 440fx chipset or some such.
 
 I don't think so.  FILO does it right.  Is it chipset specific?
 
It just uses the generic IDE io interface. No chipset specifics are
needed for IDE. Booting USB or PCMCIA is a bit trickier though.

Stefan

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

  I don't
think so.  FILO does it right.  Is it chipset specific?
  
 It just uses the generic IDE io interface. No chipset specifics are
 needed for IDE. Booting USB or PCMCIA is a bit trickier though.
So there you go.  Bochs shoud be able to be made generic.  I think it
is generic right now but I seem to remember that it did some stuff
with delays rather than checking the busy flag and that was the
difference between the ADLO that worked and the latest stuff.

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

  So there you go.  Bochs shoud be able to be made
generic.  I think it
 is generic right now but I seem to remember that it did some stuff
 with delays rather than checking the busy flag and that was the
 difference between the ADLO that worked and the latest stuff.
I still remmber an IDE code that would work with Samsung HDD's but not IBM 
hdd's. So it is possible to have disk/chipset specific code.

now all we need someone to fix the code :-)

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

 I still remmber an IDE code that would work with Samsung HDD's but not IBM
 hdd's. So it is possible to have disk/chipset specific code.
Thats buggy IDE hardware either the host chipset or the drive.  I've
written lowlevel IDE routines for another project that do direct LBA
sector reads and the interface to that is outlined in a spec and not
in anyway chipset specific.

  now all we need someone to fix the code :-)
Pehaps we could get the bochs people to do it.  Some e-mail to them
with our modified routines from ADLO would at least generate some
feeback on what they thought the most "correct" setup is.

But is it currently even a problem?  What does the latest bochs bios
do that the ADLO dosen't?

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

  now all we
need someone to fix the code :-)
 Pehaps we could get the bochs people to do it.  Some e-mail to them
 with our modified routines from ADLO would at least generate some
 feeback on what they thought the most "correct" setup is.
better communication between developers of LB and BOCHS would be needed. I 
don't think anyone has spoken with new bochs bios maintainer yet.

they also get kind of sensitive when the fixes for real hardware does not 
quite work right on their virtual hardware...... after all windows xp 
works for them so for them it is "it ain't broke, don't fix" issue.

  But is it currently even a problem?  What does the
latest bochs bios
 do that the ADLO dosen't?
we would prefer to avoid forking off the code.

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

 they also get kind of sensitive when the fixes for real hardware does not
 quite work right on their virtual hardware...... after all windows xp
 works for them so for them it is "it ain't broke, don't fix" issue.
Well  seems to me that if the movement toward "according to the spec"
is causes a problem then the project as a whole has an issue that
ought to be worked out.

  But is it
currently even a problem?  What does the latest bochs bios
 do that the ADLO dosen't?
 
 we would prefer to avoid forking off the code.
Who's we?  I appear to be the only person other than perhaps you
currently using ADLO.

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Richard Smith]

 Well  seems to me that if the movement toward "according to the spec"
 is causes a problem then the project as a whole has an issue that
 ought to be worked out.
Actually thinking about this more... I can probally see thier point. 
They don't have hardware so implementation of all the flags and
register correctness is just overhead.

However to even worry about going through this you gotta have a need
first and currently I don't think I need it so someone else will
probally have to do it or show me why I need it.

-- 
Richard A. Smith

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

      On Wed, 26 Jan 2005, Eric W. Biederman wrote:

  But I actually think that is overkill. v2 by default
and design
 enables all of the shadow ranges as memory.  So we just need to
 use those ranges.  That should be much easier having to patch
 ADLO each time.  It is fairly unlikely a writable ROM segment is
 going to break anything.
FWIW, I actually do remember someone on the list saying that they had to 
disable rom write since the things DID break otherwise... anyone else 
remembers more?

[BULK] RFC: Generic shadow mechanism useable from a payload

[Adam Sulmicki]

  But I actually think that is overkill. v2 by
default and design
 enables all of the shadow ranges as memory.  So we just need to
 use those ranges.  That should be much easier having to patch
 ADLO each time.  It is fairly unlikely a writable ROM segment is
 going to break anything.
 FWIW, I actually do remember someone on the list saying that they had to 
 disable rom write since the things DID break otherwise... anyone else 
 remembers more?
it was in context of ollie's work, not adlo:

Date: Thu, 13 Jan 2005 16:38:47 -0600
From: Richard Smith <smithbone(a)gmail.com&gt;
To: LinuxBIOS <linuxbios(a)clustermatic.org&gt;, YhLu <YhLu(a)tyan.com&gt;
Subject: Re: Running with VGA

  it from normal
BIOS, it will skip some operation. In this case,
 it is probably init font buffer.
 I have word from an ATI tech who says they don't do that.
I should clarify this.  I thought you guys were talking about zeroing
the entry pointer so that if you copied it from ram you would not get
a working copy.

Buy yes the ATI bios looks for a bit already set and if so then will
skip RAM init and various other things.

My copy checks to see if it can write the into the shadow space before
it self-modifies and if not I believe it tries to use some bits in the
CMOS.  So if you disable writes to your c000 shadow you might trick it
to not skip things.

[Stefan Reinauer]

      * Richard Smith <smithbone(a)gmail.com&gt; [050126 18:40]:
  What if we created a shadow.c file that was in the
northbridge
 directory with a simple API type setup that enabled and disabled the
 various shadow ranges.
What about going the LinuxBIOS table way, providing an array of writes

typedef struct { 
        device_t dev;
        unsigned pos;
        unsigned and_val;
        unsigned or_val;
} modifier_t;

and then provide two arrays of modifier_t for enable_shadow and
disable_shadow.

or whatever the according s-record representation will be in future ;-)
Then we're on the safe side without hurting philosophy.

Stefan