Re: [SeaBIOS] [Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Kevin O'Connor]

On Mon, Mar 08, 2010 at 10:04:57AM +0800, Roy Tam wrote:
  Hi all,
 
 I found some regression bugs that seems relate to SeaBIOS and I hope
 we can add back booting with Bochs BIOS in git so that we can further
 testing that the bug is in SeaBIOS or in QEMU 0.12.
 Following list is the regression that I encountered and which works in 0.11.1:
Thanks for the report.

It would help quite a bit if you could gather some additional
debugging information from SeaBIOS.  The simplest way is to grab:

http://linuxtogo.org/~kevin/SeaBIOS/test/bios.bin-0.5.1-debug-20100228

and replace the standard bios.bin with this file.  Then launch qemu
with the additional parameter "-serial file:mylog".  Then forward the
"mylog" file back.

SeaBIOS has a wealth of debugging information that could help solve
these issues.

 
http://www.drdosprojects.de/cgi-bin/download.cgi/d090723b.zip
[...]
  http://homepage1.nifty.com/bible/dos/fdos0138.lzh
I'll take a look.

-Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/10 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Mon, Mar 08, 2010 at 10:04:57AM +0800, Roy Tam
wrote:
  Hi all,

 I found some regression bugs that seems relate to SeaBIOS and I hope
 we can add back booting with Bochs BIOS in git so that we can further
 testing that the bug is in SeaBIOS or in QEMU 0.12.
 Following list is the regression that I encountered and which works in 0.11.1:
 Thanks for the report.

 It would help quite a bit if you could gather some additional
 debugging information from SeaBIOS.  The simplest way is to grab:

 http://linuxtogo.org/~kevin/SeaBIOS/test/bios.bin-0.5.1-debug-20100228

 and replace the standard bios.bin with this file.  Then launch qemu
 with the additional parameter "-serial file:mylog".  Then forward the
 "mylog" file back.

 SeaBIOS has a wealth of debugging information that could help solve
 these issues.
Tried this this BIOS and 2 debug logs generated.
seabios-debug.log is booting fdos0138.img and closes QEMU after boots to prompt.
seabios-debug-2.log is booting fdos0138.img and type something and
Illegal Instruction occurred.

 
 
http://www.drdosprojects.de/cgi-bin/download.cgi/d090723b.zip
 [...]
  http://homepage1.nifty.com/bible/dos/fdos0138.lzh
 I'll take a look.

 -Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Kevin O'Connor]

On Wed, Mar 10, 2010 at 11:49:48AM +0800, Roy Tam wrote:
  2010/3/10 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  SeaBIOS has a wealth of debugging information
that could help solve
 these issues.
 
 Tried this this BIOS and 2 debug logs generated.
 seabios-debug.log is booting fdos0138.img and closes QEMU after boots to prompt.
 seabios-debug-2.log is booting fdos0138.img and type something and
 Illegal Instruction occurred.
I don't see an "Illegal Instruction" message.  Instead, I see the
keyboard just not working.  What qemu version and what command line
did you use?

I've tracked down why keys are lost on SeaBIOS and not lost on Bochs
BIOS.  The fdos0138.img code is taking over the irq handler for the
ps2 port hardware irq.  When the irq fires, it reads the ps2 port for
the key data and then calls the bios irq handler assuming it will
reread the key and process it.  However, SeaBIOS reads the ps2 port
status indicator and finds that there is no data pending (because the
key was already read by the fdos0138.img irq handler).

Bochs BIOS just reads the data from the ps2 port and assumes it is for
the keyboard.  SeaBIOS verifies the data being read and wont process
random data from the port.

What the fdos0138.img image is doing is broken - once it reads the key
from the ps2 port, nothing stops a new key from being read the next
time something reads from the port.  Indeed, although the keyboard
works in qemu-0.11 for fdos0138.img, if one types fast they'll see
duplicate and lost keys.

  >>
http://www.drdosprojects.de/cgi-bin/download.cgi/d090723b.zip
This image fails in the same way on both qemu-0.11 and qemu-0.12, so I
don't think it's a seabios issue.

-Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/10 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Wed, Mar 10, 2010 at 11:49:48AM +0800, Roy Tam
wrote:
  2010/3/10 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  SeaBIOS has a wealth of debugging information
that could help solve
 these issues.
 Tried this this BIOS and 2 debug logs generated.
 seabios-debug.log is booting fdos0138.img and closes QEMU after boots to prompt.
 seabios-debug-2.log is booting fdos0138.img and type something and
 Illegal Instruction occurred.
 I don't see an "Illegal Instruction" message.  Instead, I see the
 keyboard just not working.  What qemu version and what command line
 did you use?

 I've tracked down why keys are lost on SeaBIOS and not lost on Bochs
 BIOS.  The fdos0138.img code is taking over the irq handler for the
 ps2 port hardware irq.  When the irq fires, it reads the ps2 port for
 the key data and then calls the bios irq handler assuming it will
 reread the key and process it.  However, SeaBIOS reads the ps2 port
 status indicator and finds that there is no data pending (because the
 key was already read by the fdos0138.img irq handler).

 Bochs BIOS just reads the data from the ps2 port and assumes it is for
 the keyboard.  SeaBIOS verifies the data being read and wont process
 random data from the port.

 What the fdos0138.img image is doing is broken - once it reads the key
 from the ps2 port, nothing stops a new key from being read the next
 time something reads from the port.  Indeed, although the keyboard
 works in qemu-0.11 for fdos0138.img, if one types fast they'll see
 duplicate and lost keys.

  >>
http://www.drdosprojects.de/cgi-bin/download.cgi/d090723b.zip
 This image fails in the same way on both qemu-0.11 and qemu-0.12, so I
 don't think it's a seabios issue.
You have to use MAKEBOOT.BAT to create a boot floppy (you can do it
inside QEMU to write it into an image too.)

 
 -Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Kevin O'Connor]

On Wed, Mar 10, 2010 at 01:05:38PM +0800, Roy Tam wrote:
  2010/3/10 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  On Wed, Mar 10, 2010 at 11:49:48AM +0800, Roy Tam
wrote:
  >>
http://www.drdosprojects.de/cgi-bin/download.cgi/d090723b.zip
 This image fails in the same way on both qemu-0.11 and qemu-0.12, so I
 don't think it's a seabios issue.
 
 You have to use MAKEBOOT.BAT to create a boot floppy (you can do it
 inside QEMU to write it into an image too.)
Okay - this one is interesting.  The code in that image is calling
int_1601 (check key) and int_1a00 (check timer) in a tight loop with
irqs disabled.  Bochs BIOS explicitly turns on irqs during int_16 and
int_1a processing, while SeaBIOS does not.  Since SeaBIOS doesn't
enable irqs and the caller doesn't enable irqs, key events and timer
irqs don't trigger and the code never exits the tight loop.

I'll have to investigate this one further - I'd like to look for
documentation on calls where the bios is obligated to temporarily
enable interrupts.

-Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/10 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Wed, Mar 10, 2010 at 11:49:48AM +0800, Roy Tam
wrote:
  2010/3/10 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  SeaBIOS has a wealth of debugging information
that could help solve
 these issues.
 Tried this this BIOS and 2 debug logs generated.
 seabios-debug.log is booting fdos0138.img and closes QEMU after boots to prompt.
 seabios-debug-2.log is booting fdos0138.img and type something and
 Illegal Instruction occurred.
 I don't see an "Illegal Instruction" message.  Instead, I see the
 keyboard just not working.  What qemu version and what command line
 did you use?
latest git today. when you type fast.

 
 I've tracked down why keys are lost on SeaBIOS and not lost on Bochs
 BIOS.  The fdos0138.img code is taking over the irq handler for the
 ps2 port hardware irq.  When the irq fires, it reads the ps2 port for
 the key data and then calls the bios irq handler assuming it will
 reread the key and process it.  However, SeaBIOS reads the ps2 port
 status indicator and finds that there is no data pending (because the
 key was already read by the fdos0138.img irq handler).

 Bochs BIOS just reads the data from the ps2 port and assumes it is for
 the keyboard.  SeaBIOS verifies the data being read and wont process
 random data from the port.

 What the fdos0138.img image is doing is broken - once it reads the key
 from the ps2 port, nothing stops a new key from being read the next
 time something reads from the port.  Indeed, although the keyboard
 works in qemu-0.11 for fdos0138.img, if one types fast they'll see
 duplicate and lost keys.
But it is how programs(Chinese/Japanese/Korean Display Systems,
GW-BASIC, etc.) in the past get input from keyboard.
"Consider" legacy as "broken" is wrong IMHO.

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Kevin O'Connor]

On Wed, Mar 10, 2010 at 01:24:27PM +0800, Roy Tam wrote:
  2010/3/10 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  I don't see an "Illegal
Instruction" message.  Instead, I see the
 keyboard just not working.  What qemu version and what command line
 did you use?
 
 latest git today. when you type fast.
Bleh.  That image defines an int15 wrapper which corrupts %edi.
SeaBIOS calls int154f from the keyboard irq, and it is not prepared to
handle a corrupted edi.  In contrast, Bochs BIOS doesn't care if the
registers change.

  What the
fdos0138.img image is doing is broken - once it reads the key
 from the ps2 port, nothing stops a new key from being read the next
 time something reads from the port.  Indeed, although the keyboard
 works in qemu-0.11 for fdos0138.img, if one types fast they'll see
 duplicate and lost keys.
 
 But it is how programs(Chinese/Japanese/Korean Display Systems,
 GW-BASIC, etc.) in the past get input from keyboard.
Can you point me to documentation on this usage?

  "Consider" legacy as "broken" is
wrong IMHO.
It's broken because it causes key presses to be lost and corrupted.
The ps2 port hardware just doesn't work the way that software is
trying to use it.

-Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/11 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Wed, Mar 10, 2010 at 01:24:27PM +0800, Roy Tam
wrote:
  2010/3/10 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  I don't see an "Illegal
Instruction" message.  Instead, I see the
 keyboard just not working.  What qemu version and what command line
 did you use?
 latest git today. when you type fast.
 Bleh.  That image defines an int15 wrapper which corrupts %edi.
 SeaBIOS calls int154f from the keyboard irq, and it is not prepared to
 handle a corrupted edi.  In contrast, Bochs BIOS doesn't care if the
 registers change.

  What the
fdos0138.img image is doing is broken - once it reads the key
 from the ps2 port, nothing stops a new key from being read the next
 time something reads from the port.  Indeed, although the keyboard
 works in qemu-0.11 for fdos0138.img, if one types fast they'll see
 duplicate and lost keys.
 But it is how programs(Chinese/Japanese/Korean Display Systems,
 GW-BASIC, etc.) in the past get input from keyboard.
 Can you point me to documentation on this usage?
Sorry I can't find documentation on this usage. But instead I have
lots of old programs written with this usage.
Using undocumented features from BIOS/DOS is very usual in that time.

 
"Consider" legacy as "broken" is wrong IMHO.
 It's broken because it causes key presses to be lost and corrupted.
 The ps2 port hardware just doesn't work the way that software is
 trying to use it.
You said that "it causes key presses to be lost and corrupted" but I
haven't heard any complain about this.
Real BIOSes (Award BIOS, AMI BIOS, Phoenix BIOS) handle this usage
very well and no key press are lost or corrupted.
Any key press should generate 4 IRQs, for example when I press [Tab]
key, it should have IRQs like this:
ps2: data f (status=1d)
ps2: data f (status=1c)
ps2: data 8f (status=1d)
ps2: data 8f (status=1c)

Even if status&1==1 is handled by the program itself, status==0x1c is
still handled by BIOS.
Programs in the past will try to get data from BIOS in this moment to
ensure two scancodes are the same.

On the contrary *NOT* sending keycode in this moment cause bigger problem.

The following patch help people to see irq status and data change:
---
diff --git a/src/ps2port.c b/src/ps2port.c
index 49bf551..48d940f 100644
--- a/src/ps2port.c
+++ b/src/ps2port.c
@@ -154,11 +154,19 @@ process_ps2byte(u8 status, u8 data)
 static void
 process_ps2bytes(void)
 {
+    u8 status = 0;
+    u8 data = 0;
+    u8 old_status = 0;
+    u8 old_data = 0;
     for (;;) {
-        u8 status = inb(PORT_PS2_STATUS);
+        old_status = status;
+        old_data = data;
+        status = inb(PORT_PS2_STATUS);
+        data = inb(PORT_PS2_DATA);
+        if ((data != old_data)||(status!=old_status))
+            dprintf(1, "ps2: data %x (status=%x)\n", data, status);
         if (!(status & I8042_STR_OBF))
             return;
-        u8 data = inb(PORT_PS2_DATA);
         process_ps2byte(status, data);
     }
 }

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Kevin O'Connor]

On Thu, Mar 11, 2010 at 03:42:28PM +0800, Roy Tam wrote:
  Sorry I can't find documentation on this usage.
But instead I have
 lots of old programs written with this usage.
 Using undocumented features from BIOS/DOS is very usual in that time.
Can you confirm these other programs fail in the same way (no keyboard
input, and "ps2 irq but no data." messages in log when using
bios.bin-0.5.1-debug-20100228)?

  It's
broken because it causes key presses to be lost and corrupted.
 The ps2 port hardware just doesn't work the way that software is
 trying to use it.
 
 You said that "it causes key presses to be lost and corrupted" but I
 haven't heard any complain about this.
 Real BIOSes (Award BIOS, AMI BIOS, Phoenix BIOS) handle this usage
 very well and no key press are lost or corrupted.
Under qemu-0.11 normal typing lead to lots of keyboard errors for me.
It's possible real hardware would be less susceptible to this error,
but there is nothing that a BIOS inside qemu can do to stop the
corruption.

  Any key press should generate 4 IRQs, for example when
I press [Tab]
 key, it should have IRQs like this:
 ps2: data f (status=1d)
 ps2: data f (status=1c)
 ps2: data 8f (status=1d)
 ps2: data 8f (status=1c)
There is one irq on key press and one irq on key release.  Your
debugging output is in a loop and you're reporting the same event
twice.

  The following patch help people to see irq status and
data change:
The current SeaBIOS flow is effectively:

        // Read ps2 port status
        u8 status = inb(PORT_PS2_STATUS);
        if (!(status & I8042_STR_OBF))
            // No event in queue - nothing can be dequeued.
            return;
        // Event in queue - dequeue it
        u8 data = inb(PORT_PS2_DATA);
        // Process the event.
        process_ps2byte(status, data);

Your debugging patch is not correct becuase it performs the event
dequeue before the check to see if there is an event in the queue.

-Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/12 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Thu, Mar 11, 2010 at 03:42:28PM +0800, Roy Tam
wrote:
  Sorry I can't find documentation on this
usage. But instead I have
 lots of old programs written with this usage.
 Using undocumented features from BIOS/DOS is very usual in that time.
 Can you confirm these other programs fail in the same way (no keyboard
 input, and "ps2 irq but no data." messages in log when using
 bios.bin-0.5.1-debug-20100228)?

  It's
broken because it causes key presses to be lost and corrupted.
 The ps2 port hardware just doesn't work the way that software is
 trying to use it.
 You said that "it causes key presses to be lost and corrupted" but I
 haven't heard any complain about this.
 Real BIOSes (Award BIOS, AMI BIOS, Phoenix BIOS) handle this usage
 very well and no key press are lost or corrupted.
 Under qemu-0.11 normal typing lead to lots of keyboard errors for me.
 It's possible real hardware would be less susceptible to this error,
 but there is nothing that a BIOS inside qemu can do to stop the
 corruption.

  Any key press should generate 4 IRQs, for example
when I press [Tab]
 key, it should have IRQs like this:
 ps2: data f (status=1d)
 ps2: data f (status=1c)
 ps2: data 8f (status=1d)
 ps2: data 8f (status=1c)
 There is one irq on key press and one irq on key release.  Your
 debugging output is in a loop and you're reporting the same event
 twice.

  The following patch help people to see irq status
and data change:
 The current SeaBIOS flow is effectively:

        // Read ps2 port status
        u8 status = inb(PORT_PS2_STATUS);
        if (!(status & I8042_STR_OBF))
            // No event in queue - nothing can be dequeued.
            return;
        // Event in queue - dequeue it
        u8 data = inb(PORT_PS2_DATA);
        // Process the event.
        process_ps2byte(status, data);

 Your debugging patch is not correct becuase it performs the event
 dequeue before the check to see if there is an event in the queue.
I don't care about it. As mentioned in OSDEV wiki (
http://wiki.osdev.org/PS2_Keyboard ):
Note that if you repeatedly read the port 0x60 without waiting for
another IRQ, you'll read the same byte again. That's the 'normal'
behaviour of keyboard controller, [...]

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/12 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Thu, Mar 11, 2010 at 03:42:28PM +0800, Roy Tam
wrote:
  Sorry I can't find documentation on this
usage. But instead I have
 lots of old programs written with this usage.
 Using undocumented features from BIOS/DOS is very usual in that time.
 Can you confirm these other programs fail in the same way (no keyboard
 input, and "ps2 irq but no data." messages in log when using
 bios.bin-0.5.1-debug-20100228)?

  It's
broken because it causes key presses to be lost and corrupted.
 The ps2 port hardware just doesn't work the way that software is
 trying to use it.
 You said that "it causes key presses to be lost and corrupted" but I
 haven't heard any complain about this.
 Real BIOSes (Award BIOS, AMI BIOS, Phoenix BIOS) handle this usage
 very well and no key press are lost or corrupted.
 Under qemu-0.11 normal typing lead to lots of keyboard errors for me.
 It's possible real hardware would be less susceptible to this error,
 but there is nothing that a BIOS inside qemu can do to stop the
 corruption.
and Norton GHOST for DOS (all versions, 8.0/8.2/8.3/11.5 tested) are
affected too.
for me having corrupted/incomplete scancodes is far better than having null.

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Kevin O'Connor]

On Sat, Mar 13, 2010 at 12:13:04AM +0800, Roy Tam wrote:
  2010/3/12 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  Under qemu-0.11 normal typing lead to lots of
keyboard errors for me.
 It's possible real hardware would be less susceptible to this error,
 but there is nothing that a BIOS inside qemu can do to stop the
 corruption.
 
 and Norton GHOST for DOS (all versions, 8.0/8.2/8.3/11.5 tested) are
 affected too.
 for me having corrupted/incomplete scancodes is far better than having null.
Can you confirm GHOST keyboard fails in the same way - keyboard does
not work at all, and log fills with "ps2 irq but no data." when using
bios.bin-0.5.1-debug-20100311?

-Kevin

[Qemu-devel] Regression: more 0.12 regression (SeaBIOS related?)

[Roy Tam]

2010/3/13 Kevin O'Connor <kevin(a)koconnor.net>et>:
  On Sat, Mar 13, 2010 at 12:13:04AM +0800, Roy Tam
wrote:
  2010/3/12 Kevin O'Connor
<kevin(a)koconnor.net>et>:
  Under qemu-0.11 normal typing lead to lots of
keyboard errors for me.
 It's possible real hardware would be less susceptible to this error,
 but there is nothing that a BIOS inside qemu can do to stop the
 corruption.
 and Norton GHOST for DOS (all versions, 8.0/8.2/8.3/11.5 tested) are
 affected too.
 for me having corrupted/incomplete scancodes is far better than having null.
 Can you confirm GHOST keyboard fails in the same way - keyboard does
 not work at all, and log fills with "ps2 irq but no data." when using
 bios.bin-0.5.1-debug-20100311?
with debug 0228: yes
with debug 0311: no, but same issue occurs.

PC keyboard emulation (was: Regression: more 0.12 regression (SeaBIOS related?))

[Jamie Lokier]

Kevin O'Connor wrote:
  On Thu, Mar 11, 2010 at 03:42:28PM +0800, Roy Tam
wrote:
  Sorry I can't find documentation on this
usage. But instead I have
 lots of old programs written with this usage.
 Using undocumented features from BIOS/DOS is very usual in that time.
 
 Can you confirm these other programs fail in the same way (no keyboard
 input, and "ps2 irq but no data." messages in log when using
 bios.bin-0.5.1-debug-20100228)?
 
  It's
broken because it causes key presses to be lost and corrupted.
 The ps2 port hardware just doesn't work the way that software is
 trying to use it.
 
 You said that "it causes key presses to be lost and corrupted" but I
 haven't heard any complain about this.
 Real BIOSes (Award BIOS, AMI BIOS, Phoenix BIOS) handle this usage
 very well and no key press are lost or corrupted.
 
 Under qemu-0.11 normal typing lead to lots of keyboard errors for me.
 It's possible real hardware would be less susceptible to this error,
 but there is nothing that a BIOS inside qemu can do to stop the
 corruption.
The DOS coding method brought up in this thread, resulting
in two reads of port 0x60, is quite common.  It works on all real PCs,
and correct emulation must handle it.

I'm not sure if reading port 0x60 is supposed to clear the status bit
immediately or not.  The problem might be QEMU's hardware emulation
clearing the 8042 status bit too quickly, or it might be SeaBIOS
should not check the status bit - in which case it probably fails on
_real_ hardware too, when running these old DOS TSRs and similar
programs.

It would be good if someone can check the behaviour of real hardware.
Roy, are you able to run one of the DOS drivers on a real PC with a
modified keyboard TSR and read port 0x64 in between the two reads of
port 0x60?

The same keyboard problem was brought up on qemu-devel 6 months ago,
as an emulation problem (with hacky patch that didn't fix all uses),
and I thought it had been addressed in QEMU, but maybe not:

 > To: Jamie Lokier <jamie(a)shareable.org>
> Cc: qemu-devel(a)nongnu.org
> Date: Sun, 23 Aug 2009 19:20:31 +0200
> Subject: Re: [Qemu-devel] [PATCH 0 of 1] Fix for DOS keyboard problems
> From: Stefan Ring <stefanrin(a)gmail.com>
> 
> On Sun, Aug 23, 2009 at 2:44 PM, Jamie Lokier<jamie(a)shareable.org> wrote:
> > Stefan Ring wrote:
> >> The keyboard is still very unreliable when a keyboard driver is
> >> loaded inside DOS (I assume that the keyboard driver completely
> >> disables the BIOS handler).  This behavior is also present without
> >> my patch, and I personally don't care about it, so this should not
> >> be an obstacle.
> >
> > Can you say a bit more about what your patch actually fixes?
> >
> > I've been using QEMU with several versions of MS-DOS and haven't
> > noticed any keyboard problems.
> >
> > I even use various "halt-on-idle" idle programs, and haven't
noticed
> > any keyboard problems.
> >
> > So what behaviour does your patch fix?
> 
> The particular problem that I noticed and that is 100% reproducible
> and also very understandable when you look at what the DOS programs
> do, is that in Borland's Text-Mode DOS IDEs (most likely BC++ 3.1, 4,
> Turbo Pascal 6 but definitely Turbo Pascal 7) and IIRC also in Turbo
> Vision programs generated by them, every cursor key press is
> interpreted twice. So when you open up the leftmost menu and press the
> down cursor key once, the third entry gets selected instead of the
> second one.
> 
> The reason for this is that said programs install an IRQ 1 handler
> which does little more than read from port 60h and pass control to the
> underlying BIOS (or keyboard driver) handler. The BIOS handler reads
> port 60h again, and should apparently see the same value as the first
> handler. The problem with the cursor keys is that they generate two
> scan codes in succession on port 60h that should be read by two
> separate IRQ handler activations but because QEMU's original behavior
> is to consume one data byte per read from 60h, the data gets used up
> too fast.
There was a patch accompanying that thread.  However, this is what I
wrote about the patch:

[Jamie Lokier wrote:]
 >> Let's see if I understand your
explanation.
>> 
>>    1. Cursor key is pressed.  The key press is represented as two scan codes.
>>    2. IRQ 1 is entered.
>>    3. Borland's code reads port 60h - gets the first scan code.
>>    4. BIOS's code reads port 60h - gets the second scan code.
>>    5. Return from IRQ 1.
>> 
>>    6. Cursor key is released.  The key release is represented as two
>>       scan codes.
>>    7. IRQ 1 is entered.
>>    8. Borland's code reads port 60h - gets the first scan code.
>>    9. BIOS's code reads port 60h - gets the second scan code.
>>   10. Return from IRQ 1.
>> 
>> So both Borland's code and the BIOS are *missing* scan codes.
>> 
>> How does that result in Borland seeing *multiple* cursor key
>> press/release sequences?
>> 
>> 
>> Apparently when port 0x60 is read, that de-asserts IRQ1, resets the
>> IBF flag ("input buffer full"), and another byte could be received
>> from the keyboard.  However, reading port 0x60 quickly, before another
>> byte can be received over the keyboard cable, should return the same byte.
>> 
>> So I agree that the emulated port 0x60 should return the same value if
>> there has not been enough time for the (emulated) keyboard cable to
>> transmit another scan code.
>> 
>> But detecting the particular sequence used by Borland code and the
>> BIOS together is a hack.  I'm not surprised that, as you say, other
>> DOS keyboard drivers remain broken after the patch.
>> 
>> If it goes in, the patch should include a very clear comment that the
>> "held" value and detecting the disable/read/enable sequence is only a
>> workaround for what Borland does and also depends on the BIOS
>> sequence, and is not a correct emulation in general.
All the DOS code which transfers control to the BIOS after reading
port 0x60 assumes that the whole sequence is quite fast - faster than
the 8042 can change the byte.

That should always be true on a real PC.

Unfortunately that means "correct" hardware emulation must behave
differently if a short guest time elapses between two port 0x60 reads
versus a longer one.

Here are some excerpts from DOSEMU technical documentation:

    http://www.dosemu.org/docs/README-tech/0.99/README-tech-8.html

#

      if a dos application or TSR has redirected the keyboard
      interrupt, its handler might read from port 60h to get raw
      scancodes. Port 60h is of course virtualized, and the read
      returns the value from raw_buffer.

      Note that a mix between the two cases is also possible, e.g. a
      TSR's int9 handler first reads port 60h to check if a particular
      key was pressed, then gives over to the default int9
      handler. Even these cases should be (and are, I think) handled
      properly.

      Note also that in any case, int9 is called once for each raw
      scancode byte. Eg., suppose the user pressed the PgDn key, whose
      raw scancode is E0 51:

      - first call to int9:

          read port 60h        = 0xe0
          read port 60h        = 0xe0   (**)
          call get_bios_key()  = 0
          iret

        do_irq1() reschedules IRQ1 because further scancodes are in the queue

      - second call to int9

          read port 60h        = 0x51
          call get_bios_key()  = 0x5100    (bios scancode of PgDn)
          iret

      (** multiple port 60h reads during the same interrupt yield the
      same result.)

Note that (**) ^^ - DOSEMU authors knew about this behaviour.  And
they specifically note that DOS applications and TSRs redirect the
BIOS keyboard handling in this way, and require multiple reads to work.

It is even possible to have multiple TSRs installed, so that port 0x60
is read more than 2 times.

Something else is revealed from another DOSEMU document:

    http://www.dosemu.org/docs/README-tech/1.4/x1173.html

      5.3. Known bugs & incompatibilites

      If the interrupt is not acknowledged and the keyboard port is
      read we don't eventually give up like a real keyboard and
      deliver the next byte in the keyboard buffer.

In other words, 8042 emulation _should_ permit port 0x60 to be read
multiple times, but if sufficient time elapses without the irq being
acknowledged, it should drop the byte and deliver the next one.

Unfortunately, in QEMU _guaranteeing_ that the guest has had some
guest time to process the irq before the timeout is a bit tricky, if
QEMU is delayed by host scheduling.

I think that timeout is necessary for some code which polls the ports
with irqs disabled (it sounds familiar), and for just generally
unwedging :-) but I don't have an example in mind.

As I said earlier, I'm not sure if reading port 0x60 is supposed to
clear the port 0x64 status bit immediately, in which cause SeaBIOS
must be fixed, or if SeaBIOS is fine and the emulation must be fixed.

It would be good if someone can check the behaviour of real hardware
by reading port 0x64 between the two port 0x60 reads in rapid
succession, and/or check if SeaBIOS works fine on real hardware with
these DOS apps/TSRs.

-- Jamie

PC keyboard emulation (was: Regression: more 0.12 regression (SeaBIOS related?))

[Kevin O'Connor]

On Fri, Mar 12, 2010 at 11:24:27PM +0000, Jamie Lokier wrote:
  The DOS coding method brought up in this thread,
resulting
 in two reads of port 0x60, is quite common.  It works on all real PCs,
 and correct emulation must handle it.
 
 I'm not sure if reading port 0x60 is supposed to clear the status bit
 immediately or not.  The problem might be QEMU's hardware emulation
 clearing the 8042 status bit too quickly, or it might be SeaBIOS
 should not check the status bit - in which case it probably fails on
 _real_ hardware too, when running these old DOS TSRs and similar
 programs.
 
 It would be good if someone can check the behaviour of real hardware.
On my epia-cn, a read of port 0x60 immiediately clears the OBF flag of
the status register.

The GHOST program which drops/corrupts keys on qemu seems to work fine
on my epia-cn.

[...]
  In other words, 8042 emulation _should_ permit port
0x60 to be read
 multiple times, but if sufficient time elapses without the irq being
 acknowledged, it should drop the byte and deliver the next one.
The ps2 port can use up to a 16.6Khz clock, and it takes 11 clock
cycles to read a byte.  The next keyboard byte only starts being
transmitted after the first byte is dequeued, so a second read of the
data port shouldn't return a new byte in less than 660us.

However, I don't see anything that would prevent a mouse data byte
being read on the second read.  So, this still seems very sloppy to
me.

  Unfortunately, in QEMU _guaranteeing_ that the guest
has had some
 guest time to process the irq before the timeout is a bit tricky, if
 QEMU is delayed by host scheduling.
As a hack, I suppose qemu could check the time each inb(0x60) and
return old data if a new irq hasn't been observed and 660us hasn't
elapsed since the first read.

-Kevin

PC keyboard emulation (was: Regression: more 0.12 regression (SeaBIOS related?))

[Jamie Lokier]

Kevin O'Connor wrote:
  On Fri, Mar 12, 2010 at 11:24:27PM +0000, Jamie Lokier
wrote:
  The DOS coding method brought up in this thread,
resulting
 in two reads of port 0x60, is quite common.  It works on all real PCs,
 and correct emulation must handle it.
 
 I'm not sure if reading port 0x60 is supposed to clear the status bit
 immediately or not.  The problem might be QEMU's hardware emulation
 clearing the 8042 status bit too quickly, or it might be SeaBIOS
 should not check the status bit - in which case it probably fails on
 _real_ hardware too, when running these old DOS TSRs and similar
 programs.
 
 It would be good if someone can check the behaviour of real hardware.
 
 On my epia-cn, a read of port 0x60 immiediately clears the OBF flag of
 the status register.
Ok.  As in like this, in rapid succession?

   inb(0x64)  -> OBF is set
   inb(0x60)  -> keyboard byte
   inb(0x64)  -> OBF is clear
   inb(0x60)  -> same keyboard byte

That would mean SeaBIOS is broken - and should accept the byte on port
0x60 even if OBF is clear - even on real hardware.

  The GHOST program which drops/corrupts keys on qemu
seems to work fine
 on my epia-cn.
We'd expect that with any real PC running it's own BIOS.  The
interesting question is whether it works when running SeaBIOS on real
hardware, which is hard to test without suitable hardware :-)

  [...]
  In other words, 8042 emulation _should_ permit
port 0x60 to be read
 multiple times, but if sufficient time elapses without the irq being
 acknowledged, it should drop the byte and deliver the next one.
 
 The ps2 port can use up to a 16.6Khz clock, and it takes 11 clock
 cycles to read a byte.  The next keyboard byte only starts being
 transmitted after the first byte is dequeued, so a second read of the
 data port shouldn't return a new byte in less than 660us.
That's really helpful, thanks.

  However, I don't see anything that would prevent a
mouse data byte
 being read on the second read.  So, this still seems very sloppy to
 me.
DOS - those were the days :-)

I'm guessing those programs don't expect the controller to be in mux
mode.  After all they typically just read port 0x60 and don't look at
the status to determine which external port the byte came from.  Many
of them don't even check port 0x64 at all.

  Unfortunately,
in QEMU _guaranteeing_ that the guest has had some
 guest time to process the irq before the timeout is a bit tricky, if
 QEMU is delayed by host scheduling.
 
 As a hack, I suppose qemu could check the time each inb(0x60) and
 return old data if a new irq hasn't been observed and 660us hasn't
 elapsed since the first read.
That's no mere hack: it's a good idea.  But the 660us would have to be
"at least 660us guest time", not qemu clock time.  I'm not sure if
qemu has the ability to guarantee holding something for a minimum
guest CPU time.

The exact time almost certainly does not matter at all.  I doubt very
much if any DOS program times it.

All that matters, I expect, is that when these inb(0x60) sequences
happen and they are within a few thousand instructions / emulated
clock cycles of each other, the byte is not yet replaced.

 From what you've said, SeaBIOS should be changed
because it would fail
on real hardware too, and from others' reports, that's
enough to make
DOS int-intercepting programs work properly.  The timeout is then
needed to make DOS polling-in-a-loop-without-irqs programs work properly.

-- Jamie

PC keyboard emulation (was: Regression: more 0.12 regression (SeaBIOS related?))

[Kevin O'Connor]

On Mon, Mar 15, 2010 at 02:47:16AM +0000, Jamie Lokier wrote:
  Kevin O'Connor wrote:
  On my epia-cn, a read of port 0x60 immiediately
clears the OBF flag of
 the status register.
 
 Ok.  As in like this, in rapid succession?
 
    inb(0x64)  -> OBF is set
    inb(0x60)  -> keyboard byte
    inb(0x64)  -> OBF is clear
    inb(0x60)  -> same keyboard byte
Yes.

  That would mean SeaBIOS is broken - and should accept
the byte on port
 0x60 even if OBF is clear - even on real hardware.
I agree.  The latest SeaBIOS git has the needed changes.

  The GHOST
program which drops/corrupts keys on qemu seems to work fine
 on my epia-cn.
 
 We'd expect that with any real PC running it's own BIOS.  The
 interesting question is whether it works when running SeaBIOS on real
 hardware, which is hard to test without suitable hardware :-)
I tested it with coreboot+SeaBIOS (modified to not inspect 0x64) on my
epia-cn.  I didn't test with the factory BIOS.

  As a hack, I
suppose qemu could check the time each inb(0x60) and
 return old data if a new irq hasn't been observed and 660us hasn't
 elapsed since the first read.
 
 That's no mere hack: it's a good idea.  But the 660us would have to be
 "at least 660us guest time", not qemu clock time.
I don't think it would matter that much in practice.  The chance of
qemu being preemptively scheduled between two port 0x60 reads is
probably very small.  These old programs are dropping keys like crazy
now - if a fix made them only drop keys sporadically it would still be
a big improvement.

That said, I'm not sure there's that much desire to make these old
programs work well.

-Kevin