[OpenBIOS] bug in toke?
stepan at openbios.org
Thu Jan 6 11:49:01 CET 2005
* Mark Wenning <wenning at us.ibm.com> [050105 23:34]:
> We're looking at using toke, detok in our development process.
Nice to hear. Let me know how I can help make this happen.
> Our current setup to tokenize device drivers uses a "script" similar to
> ----- start kng_sr_script -----
> \ define these words for the tokenizer
> 0 constant ?SYSTEM-ROM
> 0 constant COMMENT-OUT
It's "not really a bug" but a lacking feature. ;-) The tokenizer[
directive should directly execute the forth words before ]tokenizer.
Instead toke only pushes the specified values to the stack. Simply
because it has no forth engine to call inside.
This results in your two problems:
> The version of toke I downloaded doesn't like this for 2 reasons:
> 1. Constant declarations in TOKENIZER[ ... ]TOKENIZER blocks emit
> bytes/tokens to the output,
> so we get extra stuff before the PCI header. I don't think it should
> do this; code inside tokenizer blocks
> should not be in the output FCODE. Is this correct?
> 2. Previous constant declarations aren't being looked up/interpreted
> when in Tokenizer mode. For
> tokenizer[ hex ibm-Code-Revision-Level decimal ]tokenizer SET-REV-LEVEL
> causes a "empty stack: error. I have to change it to
> tokenizer[ 0110 ]tokenizer SET-REV-LEVEL
> to make it work. Is this a bug?
It is something that needs to be fixed, no doubt.
> I'm a little new to this; basically what I'm asking is if this behaviour
> is what was intended, or should we attempt to fix it and submit a fix to
There are basically two possible solutions to the problem:
1) Implement parsing for the words you use in the existing framework of
toke. ie. implement a "constant" and a "hex" word. This is pretty
straight forward, but will break the next time someone is using a
different forthish feature.
2) the clean solution: Implement future versions of toke on top of the
OpenBIOS forth engine "BeginAgain". This is a pretty minimal forth
implementation that is well suited.
Clearly, 2. is the long term solution.
More information about the OpenBIOS