Hi,..
basically one of the problems we see before we can ever push OpenBIOS to flash is that we need a way of getting Forth code executed that is not part of paflof's unreadable preprocessor magic. I've started writing code to dump a dictionary from within paflof and read it back in. This gives several advantages:
* paflof can be used to bootstrap openbios forth code * the c core can be developed seperately from the forth code running on top of it. * the base engine can be changed easily as it will be rewritten in forth instead of generating a binary dictionary with the C preprocessor as it is done now. * it will be easy to change the amount of packages/features that will be flashed.
This is needed because we don't want to feed forth source to the C core while running from flash, but rather compiled forth and FCode. Before we can run fcode, we need a way of making the evaluator I wrote accessible from paflof during early bootup.
One of the nice side effects is that the C kernel can be reduced in size (it is now 17k, most of it being debugging information)
How does the thing work?
Currently there's a word DUMP-DICTIONARY which will dump the current dictionary to a relocatable ELF file dict.dump, that will be loaded automatically when paflof is started the next time. There is one minor glitch left to fix, but it can be worked around manually: LAST/LATEST is not set correctly after bootup (well, it is, but it's overwritten again.) When loading, it will tell you "LAST/LATEST=0xDEADBEEF" shortly before the ok prompt. then do a "DEADBEEF DUP LAST ! LATEST !" to make the loaded words actually visible. after that all should work as usual, just with a loaded dictionary.
Maybe this glitch can be worked around when moving the engine.in to engine.fs as there will be a new LAST/LATEST. (which will not be used until :, VARIABLE, CONSTANT etc are redefines. It might be enough to just set this to a valid value just before : etc are defined (which would require the dict.dump to be the only dictionary, with no other dictionary left)
There's currently three kinds of relocations in the ELF file: Internal, external and CFA relocations.
* Internal relocs are just references within the dictionary. To fix these up, the old start address of the dictionary is subtracted and on load, the new address is added. * external relocs. These reference an external word in the dictionary. if you use DUP, for example, it will relocate against that word, no matter whether it's moved around in the dictionary. * CFA relocations are used to relocate against native prim words (code) paflof keeps a list of prim words and relocates them by a list index. This means a dictionary has to be redumped when the number of prim words changes.
I also managed to write a new prim word to the dictionary, using HERE LAST @ , \ backlink 0 C, \ flags 3 C, 44 C, 55 C, 50 C, 0 C, 0 C, 0 C, \ 0 padded name (only on 32bit machines) ' DUP @ , \ fetch old prim CFA DUP \ update LAST/LATEST to the beginning of the new word DUP LAST ! LATEST !
What will be the next step in development?
We need to finish the engine.fs (a rewrite of engine.in in pure forth) that I already started. this is going to be the next bigger goal in development. After that dictionary scopes will be a lot easier to implement as it does not have to be done in preprocessor macros.
Last but not least, get the code at
http://www.freiburg.linux.de/OpenBIOS/bin/paflof-dump-20021016.tar.gz
Any comments, flames, tests, etc are heavily welcome.
Best regards, Stefan Reinauer
Stefan Reinauer wrote:
Hi,..
Howdy,
basically one of the problems we see before we can ever push OpenBIOS to flash is that we need a way of getting Forth code executed that is not part of paflof's unreadable preprocessor magic.
Unreadable?! It's almost straight Forth! ;)
I've started writing code to dump a dictionary from within paflof and read it back in. This gives several advantages:
- paflof can be used to bootstrap openbios forth code
It can already do this, or I don't understand what you mean?
- the c core can be developed seperately from the forth code running on top of it.
You have to link in the Forth dictionaries anyway (the base dict + what you need to read from disk, at least), so having to compile them too isn't a huge burden, /me thinks. Also, the kernel is already finished, it just needs some minor cleanups.
- the base engine can be changed easily as it will be rewritten in forth instead of generating a binary dictionary with the C preprocessor as it is done now.
You need the base to be able to compile the base. Chicken egg. You need a crosscompiler to be able to generate your first base dictionary; it just so happens that our crosscompiler is ref.pl + gcc.
- it will be easy to change the amount of packages/features that will be flashed.
That is the case for any setup.
This is needed because we don't want to feed forth source to the C core while running from flash, but rather compiled forth and FCode. Before we can run fcode, we need a way of making the evaluator I wrote accessible from paflof during early bootup.
You only need to run FCode from plugin cards, so that's not exactly "early bootup".
One of the nice side effects is that the C kernel can be reduced in size (it is now 17k, most of it being debugging information)
It's less than 1kB.
How does the thing work?
Currently there's a word DUMP-DICTIONARY which will dump the current dictionary to a relocatable ELF file dict.dump, that will be loaded automatically when paflof is started the next time.
ELF won't work cross-platform. I'd rather have the only distinction be 32/64/BE/LE, so we only have 4 different binaries per dictionary, and we even can generate them all from one build only. There is _no_ binary relocatable format that's supported by most people's binutils, sorry :(
There is one minor glitch left to fix, but it can be worked around manually: LAST/LATEST is not set correctly after bootup (well, it is, but it's overwritten again.)
This makes me think your dumper/loader doesn't implement vocabularies...
When loading, it will tell you "LAST/LATEST=0xDEADBEEF" shortly before the ok prompt. then do a "DEADBEEF DUP LAST ! LATEST !" to make the loaded words actually visible. after that all should work as usual, just with a loaded dictionary.
Maybe this glitch can be worked around when moving the engine.in to engine.fs as there will be a new LAST/LATEST. (which will not be used until :, VARIABLE, CONSTANT etc are redefines. It might be enough to just set this to a valid value just before : etc are defined (which would require the dict.dump to be the only dictionary, with no other dictionary left)
There's currently three kinds of relocations in the ELF file: Internal, external and CFA relocations.
- Internal relocs are just references within the dictionary. To fix these up, the old start address of the dictionary is subtracted and on load, the new address is added.
- external relocs. These reference an external word in the dictionary. if you use DUP, for example, it will relocate against that word, no matter whether it's moved around in the dictionary.
- CFA relocations are used to relocate against native prim words (code) paflof keeps a list of prim words and relocates them by a list index. This means a dictionary has to be redumped when the number of prim words changes.
Better relocate by name; that solves that problem, and reduces the number of relocation types to just one.
I also managed to write a new prim word to the dictionary, using HERE LAST @ , \ backlink 0 C, \ flags 3 C, 44 C, 55 C, 50 C, 0 C, 0 C, 0 C, \ 0 padded name (only on 32bit machines)
This happens to be right on 64-bit formats, too ;)
' DUP @ , \ fetch old prim CFA DUP \ update LAST/LATEST to the beginning of the new word DUP LAST ! LATEST !
Also note that the internal dictionary format can change at any time, so please don't have anything rely on it.
What will be the next step in development?
We need to finish the engine.fs (a rewrite of engine.in in pure forth) that I already started. this is going to be the next bigger goal in development. After that dictionary scopes will be a lot easier to implement as it does not have to be done in preprocessor macros.
Last but not least, get the code at
http://www.freiburg.linux.de/OpenBIOS/bin/paflof-dump-20021016.tar.gz
I'll take a look.
Cheers,
Segher
- To unsubscribe: send mail to majordomo@freiburg.linux.de with 'unsubscribe openbios' in the body of the message http://www.freiburg.linux.de/OpenBIOS/ - free your system..
* Segher Boessenkool segher@koffie.nl [021019 00:27]:
basically one of the problems we see before we can ever push OpenBIOS to flash is that we need a way of getting Forth code executed that is not part of paflof's unreadable preprocessor magic.
Unreadable?! It's almost straight Forth! ;)
Still prefer the real thing ;)
I've started writing code to dump a dictionary from within paflof and read it back in. This gives several advantages:
- paflof can be used to bootstrap openbios forth code
It can already do this, or I don't understand what you mean?
I am not going to put forth source to flash and let it be compiled to dictionaries at boot time. paflof can execute some forth stuff we wrote, but there's quite some work to do before we can consider this a bootstrap.
- the c core can be developed seperately from the forth code running on top of it.
You have to link in the Forth dictionaries anyway (the base dict + what you need to read from disk, at least), so having to compile them too isn't a huge burden, /me thinks. Also, the kernel is already finished, it just needs some minor cleanups.
No, you don't need to link it. Just load it during system startup and fix up the relocations and you're done. The stuff I wrote is like a dynamic linker. This feature could later on be used for having multiple dictionaries around and choose the one that is appropriate or wanted. This is what SUN does with their tokenizer for example.
- the base engine can be changed easily as it will be rewritten in forth instead of generating a binary dictionary with the C preprocessor as it is done now.
You need the base to be able to compile the base. Chicken egg. You need a crosscompiler to be able to generate your first base dictionary; it just so happens that our crosscompiler is ref.pl + gcc.
That is why I split paflof up in two parts: One is the userland bootstrapper that allows us to create a dictionary from forth source. Having the base in this dictionary is only one small part of them all. Other things that will be compiled to a dictionary that goes to flash are i.e. the evaluator, the package handling and all the other things that are needed. No need to write these in a way that ref.pl can eat them.
This is needed because we don't want to feed forth source to the C core while running from flash, but rather compiled forth and FCode. Before we can run fcode, we need a way of making the evaluator I wrote accessible from paflof during early bootup.
You only need to run FCode from plugin cards, so that's not exactly "early bootup".
Or from system flash in case of onboard components. But that is not what I am talking about. We need the evaluator before we are able to access anything but the paflof kernel and a precompiled dictionary residing somewhere in flash.
One of the nice side effects is that the C kernel can be reduced in size (it is now 17k, most of it being debugging information)
It's less than 1kB.
The amount of native code is more than 1kb with the version you have in CVS. Feel free to provide an update ;)
Currently there's a word DUMP-DICTIONARY which will dump the current dictionary to a relocatable ELF file dict.dump, that will be loaded automatically when paflof is started the next time.
ELF won't work cross-platform. I'd rather have the only distinction be 32/64/BE/LE, so we only have 4 different binaries per dictionary, and we even can generate them all from one build only. There is _no_ binary relocatable format that's supported by most people's binutils, sorry :(
Wrong. ELF does - if done right. The way to go is to define an own architecture type OpenBIOS. The information on 32/64bit and endianess is available from these binaries as well as the fact that it's an openbios dictionary. You cannot link it. with binutils because binutils does not know it yet. I could have chosen any format for the dump, but ELF has the features I need. If anyone needs more features out of ELF than I implemented, feel free to implement them yourselfes.
There is one minor glitch left to fix, but it can be worked around manually: LAST/LATEST is not set correctly after bootup (well, it is, but it's overwritten again.)
This makes me think your dumper/loader doesn't implement vocabularies...
No. And it would not belong there. I wrote the dumper/loader to have a good base for developing further code. Implementing vocabularies in preprocessor forth is not what I want. My dumper will not flat out dictionaries either - why should it, it's a dumper. It will NEVER be needed to flatten out a dictionary while sitting in flash, so if that should ever be needed, it's something that can be done seperately.
- Internal relocs are just references within the dictionary. To fix these up, the old start address of the dictionary is subtracted and on load, the new address is added.
- external relocs. These reference an external word in the dictionary. if you use DUP, for example, it will relocate against that word, no matter whether it's moved around in the dictionary.
- CFA relocations are used to relocate against native prim words (code) paflof keeps a list of prim words and relocates them by a list index. This means a dictionary has to be redumped when the number of prim words changes.
Better relocate by name; that solves that problem, and reduces the number of relocation types to just one.
This is not possible for CFA pointers when doing runtime linking. All the prim words have no real name but are just labels in a huge function (run_engine) jumped to by goto. Basically in a clean dictionary relocations to these prime words will be the only ones that are needed. But with my code it's also possible to load dictionaries that do not implement their own prim/base words. In this case I relocate by word names.
I also managed to write a new prim word to the dictionary, using HERE LAST @ , \ backlink 0 C, \ flags 3 C, 44 C, 55 C, 50 C, 0 C, 0 C, 0 C, \ 0 padded name (only on 32bit machines)
This happens to be right on 64-bit formats, too ;)
' DUP @ , \ fetch old prim CFA DUP \ update LAST/LATEST to the beginning of the new word DUP LAST ! LATEST !
Also note that the internal dictionary format can change at any time, so please don't have anything rely on it.
A lot of the base dictionary already relies on the dictionary format. Sure, if the format changes, code has to be changed. That's the same when we're doing preprocessor forth, just a layer above.
Stefan
-
Segher Boessenkool -
Stefan Reinauer