[OpenBIOS] forth kernel and dictionary dumping

[Stefan Reinauer]

* Segher Boessenkool <segher at koffie.nl> [021019 00:27]:
> > basically one of the problems we see before we can ever push
> > OpenBIOS to flash is that we need a way of getting Forth code
> > executed that is not part of paflof's unreadable preprocessor
> > magic.
> 
> Unreadable?!  It's almost straight Forth! ;)
 
Still prefer the real thing ;)

> > I've started writing code to dump a dictionary from within paflof
> > and read it back in. This gives several advantages:
> > 
> > * paflof can be used to bootstrap openbios forth code
> 
> It can already do this, or I don't understand what you mean?
 
I am not going to put forth source to flash and let it be
compiled to dictionaries at boot time. paflof can execute some forth
stuff we wrote, but there's quite some work to do before we can
consider this a bootstrap.

> > * the c core can be developed seperately from the forth code running
> >   on top of it.
> 
> You have to link in the Forth dictionaries anyway (the base dict + what
> you need to read from disk, at least), so having to compile them too isn't
> a huge burden, /me thinks.  Also, the kernel is already finished, it just
> needs some minor cleanups.

No, you don't need to link it. Just load it during system startup and
fix up the relocations and you're done. The stuff I wrote is like a
dynamic linker. This feature could later on be used for having multiple
dictionaries around and choose the one that is appropriate or wanted.
This is what SUN does with their tokenizer for example.

> 
> > * the base engine can be changed easily as it will be rewritten
> >   in forth instead of generating a binary dictionary with the C
> >   preprocessor as it is done now.
> 
> You need the base to be able to compile the base.  Chicken egg.  You
> need a crosscompiler to be able to generate your first base dictionary;
> it just so happens that our crosscompiler is ref.pl + gcc.

That is why I split paflof up in two parts: One is the userland
bootstrapper that allows us to create a dictionary from forth source.
Having the base in this dictionary is only one small part of them all.
Other things that will be compiled to a dictionary that goes to flash
are i.e. the evaluator, the package handling and all the other things
that are needed. No need to write these in a way that ref.pl can eat
them.

> > This is needed because we don't want to feed forth source to the C core
> > while running from flash, but rather compiled forth and FCode.
> > Before we can run fcode, we need a way of making the evaluator I wrote
> > accessible from paflof during early bootup.
> 
> You only need to run FCode from plugin cards, so that's not exactly
> "early bootup".
 
Or from system flash in case of onboard components. But that is not what
I am talking about. We need the evaluator before we are able to access
anything but the paflof kernel and a precompiled dictionary residing
somewhere in flash.

> > One of the nice side effects is that the C kernel can be reduced in size
> > (it is now 17k, most of it being debugging information)
> 
> It's less than 1kB.

The amount of native code is more than 1kb with the version you have in
CVS. Feel free to provide an update ;)

> > Currently there's a word DUMP-DICTIONARY which will dump the current
> > dictionary to a relocatable ELF file dict.dump, that will be loaded
> > automatically when paflof is started the next time.
> 
> ELF won't work cross-platform.  I'd rather have the only distinction
> be 32/64/BE/LE, so we only have 4 different binaries per dictionary,
> and we even can generate them all from one build only.  There is _no_
> binary relocatable format that's supported by most people's binutils,
> sorry :(
 
Wrong. ELF does - if done right. The way to go is to define an own
architecture type OpenBIOS. The information on 32/64bit and endianess is
available from these binaries as well as the fact that it's an openbios
dictionary. You cannot link it. with binutils because binutils does
not know it yet. I could have chosen any format for the dump, but ELF
has the features I need. If anyone needs more features out of ELF
than I implemented, feel free to implement them yourselfes. 

> > There is one minor glitch left to fix, but it can be worked around
> > manually: LAST/LATEST is not set correctly after bootup (well, it is,
> > but it's overwritten again.)
> 
> This makes me think your dumper/loader doesn't implement vocabularies...

No. And it would not belong there. I wrote the dumper/loader to have a
good base for developing further code. Implementing vocabularies in 
preprocessor forth is not what I want. My dumper will not flat out
dictionaries either - why should it, it's a dumper. It will NEVER
be needed to flatten out a dictionary while sitting in flash, so if that
should ever be needed, it's something that can be done seperately. 

> > * Internal relocs are just references within the dictionary.
> >   To fix these up, the old start address of the dictionary is subtracted
> >   and on load, the new address is added.
> > * external relocs. These reference an external word in the dictionary.
> >   if you use DUP, for example, it will relocate against that word, no
> >   matter whether it's moved around in the dictionary.
> > * CFA relocations are used to relocate against native prim words (code)
> >   paflof keeps a list of prim words and relocates them by a list index.
> >   This means a dictionary has to be redumped when the number of prim
> >   words changes.
> 
> Better relocate by name; that solves that problem, and reduces the number
> of relocation types to just one.
 
This is not possible for CFA pointers when doing runtime linking. All
the prim words have no real name but are just labels in a huge function 
(run_engine) jumped to by goto. Basically in a clean dictionary 
relocations to these prime words will be the only ones that are needed. 
But with my code it's also possible to load dictionaries that do not 
implement their own prim/base words. In this case I relocate by word
names.

> > I also managed to write a new prim word to the dictionary, using
> > HERE
> > LAST @ ,   \ backlink
> > 0 C,       \ flags
> > 3 C, 44 C, 55 C, 50 C, 0 C, 0 C, 0 C,
> >            \ 0 padded name (only on 32bit machines)
> 
> This happens to be right on 64-bit formats, too ;)
> 
> > ' DUP @ ,  \ fetch old prim CFA
> > DUP        \ update LAST/LATEST to the beginning of the new word DUP
> > LAST !
> > LATEST !
> 
> Also note that the internal dictionary format can change at any time, so
> please don't have anything rely on it.

A lot of the base dictionary already relies on the dictionary format.
Sure, if the format changes, code has to be changed. That's the same
when we're doing preprocessor forth, just a layer above.

Stefan
 
  
-- 
The x86 isn't all that complex - it just doesn't make a lot of
sense.          -- Mike Johnson, Leader of 80x86 Design at AMD
	                          Microprocessor Report (1994)


-
To unsubscribe: send mail to majordomo at freiburg.linux.de
with 'unsubscribe openbios' in the body of the message
http://www.freiburg.linux.de/OpenBIOS/ - free your system..