tokenizer bug

[Mark Wenning]

Greetings,

Our "OpenBIOS guy" thinks he found a bug in the tokenizer. 
Can you comment?
What's the procedure for submitting fixes?

Looks like my previous post attempt was screwed up by Lotus notes, 
trying again...

-----
Mark Wenning     [e-mail]     1-512-838-6263
Global FirmWare:  Networking and I/O, FC lackey
-----
Experience is what you get when you were expecting something else.     - 
Anon
-----
----- Forwarded by Mark W Wenning/Austin/IBM on 03/07/2005 10:02 AM -----
David L Paktor/Almaden/IBM
03/04/2005 05:33 PM   
To
[e-mail]
cc

Subject
Re: Found a serious bug in OpenBios Tokenizer

Hi, Mark.

As we spoke, I'm giving you some additional detail to describe the bug I 
found.

Here's a recap of what I wrote before:

It has to do with  fload ;  When your source  fload s a file, that file 
is supposed to be a continuation of your source-stream.

New definitions, as they are processed in the Tokenizer, are supposed to 
get successive FCode tokens assigned to them, starting at 0x0800.

But when the OpenBios Tokenizer  fload s a new file, it resets the FCode 
token-number, and starts assigning tokens from  0x0800 again.  So if you 
have:

definition A
definition B
definition C
definition D

\   and then

fload File1                       \  which has:

definition E
definition F
definition G
definition H

and then you have something that intends to invoke  A B C  or D   you 
will instead get  E F G  or  H , respectively....

Here's the additional material:

The source of the problem can be traced to the routines:  init_stream   
and  init_output   in the file stream.c  .  They are both called at the 
beginning of tokenization, one to prepare the source from which input 
will be scanned, and the other to prepare the area to which output will 
be delivered (broadly speaking).  When the  fload  directive is 
encountered, the input source is suspended and a new one is activated; 
when the new one is exhausted, the previous one is re-activated at the 
point from which it was suspended.  So far so good.  The  init_stream  
routine is called to effectuate that activation.  Unfortunately, the 
line that initializes the FCode token-numbers being assigning to new 
definitions -- the variable called nextfcode -- occurs inside the  
init_stream  routine.

I believe that moving that line to the  init_output  routine is 
sufficient to fix this bug.

[Stefan Reinauer]

* Mark Wenning  [050309 21:07]:
> Our "OpenBIOS guy" thinks he found a bug in the tokenizer. 
> Can you comment?
> What's the procedure for submitting fixes?
>
> Looks like my previous post attempt was screwed up by Lotus notes, 
> trying again...

Looks like the list server delayed the answer mails. The problem is
fixed.

I'll investigate the mail delays and fix those, too.

Stefan

[Stefan Reinauer]

* Stefan Reinauer  [050317 15:28]:
> Looks like the list server delayed the answer mails. 
> The problem is fixed.

^^^^^^^^^^^^^^^^^^^^

I was referring to the problem in the tokenizer :)
Please try out the latest version from http://www.openbios.org/

Stefan