problems on X220

[JPT]

Hi,

I got several problems with coreboot on X220.
I report them for you to improve coreboot.
I's not important for me.
If you need more details on a specific problem, tell me please.

This is a build from today 4.11 branch with seabios and memtest payload

I've got several boot media ready to test how coreboot behaves.
that is an original DOS 5.0 installation, like 30 years old.
a fresh Kubuntu 20.4 installation
Memtest86 4.3.7 (latest BIOS version)

- DOS 5.0 does not boot, it stops just before the C:> prompt
- Kubuntu 20.4: systemd-udevd runs amok all the time. this might be
because it tries modprobe nvidia-uvm.
- Kubuntu 20.4: Shutdown does not work AE_NOT_FOUND
- Kubuntu 20.4: reboot just switches off AE_NOT_FOUND
- Kubuntu 20.4: Going to standby does not work. Switches off the screen,
but leaves on the CPU and cannot wake up again.
- Ctrl-Alt-Del does not work?
- ESC for reboot from payload memtest or USB memtest just stops (crashes?)
- access to CMOS using nvramtool does not work. complains about missing
HAVE_OPTION_TABLE. but both USE_ and HAVE_OPTION_TABLE are set. this
makes it impossible enabling the debug boot log for the board status
database
- I somehow cannot enter the grub boot menu. might this be related to
missing graphics bios?

so, is this normal? I expected coreboot to be more production ready to
be honest... :(

regards

Jan

[Paul Menzel]

Dear JPT,

Welcome to coreboot and congratulations for successfully running 
coreboot on a device. Thank you for your report.

Am 07.05.20 um 17:38 schrieb JPT:
> I got several problems with coreboot on X220.
> I report them for you to improve coreboot.
> I's not important for me.
> If you need more details on a specific problem, tell me please.

[…]
> so, is this normal?

I am unaware of most of these problems, and some of them sound like 
errors in the new Kubuntu 20.04 version. (Though I guess you will say 
everything works with the vendor firmware.)

A lot of fixes went into the code base since the 4.11 release. It’d be 
great if you could test the current master branch (also you might want 
to apply the change-set *mb/lenovo/x220: Add support for ThinkLight* [1] 
– go to DOWNLOAD and cherry-pick the commit on top the master branch).

If you will still experience these problems, please report them to the 
list with one message per problem. Please always attach the coreboot and 
Linux messages. `cbmem -1`, from `util/cbmem`, will retrieve the 
coreboot messages for you. Also supply your configuration by attaching 
`defconfig` created by the command below.

make savedefconfig
> I expected coreboot to be more production ready to be honest... :(

In my opinion, you shouldn’t have any expectations for something you get 
without a support contract. ;-) I am always surprised how many things 
are still improved and fixed after the vendor has stopped supporting 
these devices for a long time.

Kind regards,

Paul

[1]: https://review.coreboot.org/c/coreboot/+/40668

[JPT]

Thanks Paul ;)

Sorry, I assumed the 11 branch was the branch for the soon to come
release. I built with master again, but there isn't much difference.

I uploaded logs to
https://mega.nz/folder/fR521QIZ#ypfaHUIibD_cjWktYuyrNw

since I got so many problems, before I create reports of every single
one I would like somebody to check if everything is alright with my
build. eg.
- did the build install correctly into flash?
- did I really build from the correct version?
- is something odd about the .config?

btw, paul, the commit regarding the X220 you mentioned. is it already
applied to master?

thanks

JPT
> Dear JPT,
>
> Welcome to coreboot and congratulations for successfully running
> coreboot on a device. Thank you for your report.
>
> Am 07.05.20 um 17:38 schrieb JPT:
> > I got several problems with coreboot on X220.
> > I report them for you to improve coreboot.
> > I's not important for me.
> > If you need more details on a specific problem, tell me please.
>
> […]
> > so, is this normal?
>
> I am unaware of most of these problems, and some of them sound like
> errors in the new Kubuntu 20.04 version. (Though I guess you will say
> everything works with the vendor firmware.)
>
> A lot of fixes went into the code base since the 4.11 release. It’d be
> great if you could test the current master branch (also you might want
> to apply the change-set *mb/lenovo/x220: Add support for ThinkLight* [1]
> – go to DOWNLOAD and cherry-pick the commit on top the master branch).
>
> If you will still experience these problems, please report them to the
> list with one message per problem. Please always attach the coreboot and
> Linux messages. `cbmem -1`, from `util/cbmem`, will retrieve the
> coreboot messages for you. Also supply your configuration by attaching
> `defconfig` created by the command below.
>
> make savedefconfig
> > I expected coreboot to be more production ready to be honest... :(
>
> In my opinion, you shouldn’t have any expectations for something you get
> without a support contract. ;-) I am always surprised how many things
> are still improved and fixed after the vendor has stopped supporting
> these devices for a long time.
>
> Kind regards,
>
> Paul
>
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Nico Huber]

Hi Jan,

On 08.05.20 11:25, JPT wrote:
> Sorry, I assumed the 11 branch was the branch for the soon to come
> release. I built with master again, but there isn't much difference.

but, was there a difference? I've looked at your list of problems again,
at least one is expected. So maybe they are individual problems after
all.
> I uploaded logs to
> https://mega.nz/folder/fR521QIZ#ypfaHUIibD_cjWktYuyrNw
>
> since I got so many problems, before I create reports of every single
> one I would like somebody to check if everything is alright with my
> build. eg.
> did the build install correctly into flash?

You should flash with `--ifd -i BIOS`, otherwise the ME firmware gets
overwritten. But from the log it seems the ME is ok. If you use these
options for flashing, it's also unnecessary to extract and add flash
regions to coreboot.
> did I really build from the correct version?

4.11-2619-gd8bd3ff197-dirty

It's a recent upstream revision. The -dirty, though, means you have
changes in your local repository that are not upstream.
> is something odd about the .config?

It looks all good, AFAICS. Only odd thing is that you have

# CONFIG_USE_BLOBS is not set

in your `.config`. It built nevertheless, probably because you had
it enabled before (once the blobs are downloaded, they can be used
by later builds without this option).

So let's have a closer look at your problem list:
> DOS 5.0 does not boot, it stops just before the C:> prompt

I can't say unexpected, barely anybody tests DOS.
> Kubuntu 20.4: systemd-udevd runs amok all the time. this might be
>
> because it tries modprobe nvidia-uvm.

I really have no idea what this means. Probably just a user-space issue.
> Kubuntu 20.4: Shutdown does not work AE_NOT_FOUND
> Kubuntu 20.4: reboot just switches off AE_NOT_FOUND
> Kubuntu 20.4: Going to standby does not work. Switches off the screen,
>
> but leaves on the CPU and cannot wake up again.

These three are definitely unexpected and most likely related. Can you
get us a copy of these AE_NOT_FOUND messages?
> Ctrl-Alt-Del does not work?

Not sure what you mean. During boot? i.e. in the payload or GRUB? it's
up to each program (payload/bootloader/OS) running what it does when
this is pressed.
> ESC for reboot from payload memtest or USB memtest just stops (crashes?)

Probably in the not-much-tested category.
> access to CMOS using nvramtool does not work. complains about missing
>
> HAVE_OPTION_TABLE. but both USE_ and HAVE_OPTION_TABLE are set. this

Unexpected, but there were similar reports before. Worth investigating.
> makes it impossible enabling the debug boot log for the board status
> database

Doesn't sound like this is necessary. Maybe some outdated documentation
told you to do so?
> I somehow cannot enter the grub boot menu. might this be related to
>
> missing graphics bios?

If your on-disk GRUB is configured to switch to a graphics mode, no
matter what, this is expected. Your configuration coreboot display init
+ SeaVGABIOS does not allow mode switching.

Nico

[Paul Menzel]

Dear Jan, dear Nico,

Just some additions.

Am 08.05.20 um 21:38 schrieb Nico Huber:
> On 08.05.20 11:25, JPT wrote:
> > Sorry, I assumed the 11 branch was the branch for the soon to come
> > release. I built with master again, but there isn't much difference.
>
> but, was there a difference? I've looked at your list of problems again,
> at least one is expected. So maybe they are individual problems after
> all.
> > I uploaded logs to
> > https://mega.nz/folder/fR521QIZ#ypfaHUIibD_cjWktYuyrNw
> >
> > since I got so many problems, before I create reports of every single
> > one I would like somebody to check if everything is alright with my
> > build. eg.
> > did the build install correctly into flash?
>
> You should flash with `--ifd -i BIOS`, otherwise the ME firmware gets
> overwritten. But from the log it seems the ME is ok. If you use these
> options for flashing, it's also unnecessary to extract and add flash
> regions to coreboot.
> > did I really build from the correct version?
>  4.11-2619-gd8bd3ff197-dirty
>
> It's a recent upstream revision. The -dirty, though, means you have
> changes in your local repository that are not upstream.

You can check with `git status` what the state is.
> > is something odd about the .config?
>
> It looks all good, AFAICS. Only odd thing is that you have
>
> # CONFIG_USE_BLOBS is not set
>
> in your `.config`. It built nevertheless, probably because you had
> it enabled before (once the blobs are downloaded, they can be used
> by later builds without this option).
>
> So let's have a closer look at your problem list:
> > DOS 5.0 does not boot, it stops just before the C:> prompt
>
> I can't say unexpected, barely anybody tests DOS.

Does that work with QEMU, coreboot, and your SeaBIOS payload?
> > Kubuntu 20.4: systemd-udevd runs amok all the time. this might be
> >
> > because it tries modprobe nvidia-uvm.
>
> I really have no idea what this means. Probably just a user-space issue.
> > Kubuntu 20.4: Shutdown does not work AE_NOT_FOUND
> > Kubuntu 20.4: reboot just switches off AE_NOT_FOUND
> > Kubuntu 20.4: Going to standby does not work. Switches off the screen,
> >
> > but leaves on the CPU and cannot wake up again.
>
> These three are definitely unexpected and most likely related. Can you
> get us a copy of these AE_NOT_FOUND messages?
> > Ctrl-Alt-Del does not work?
>
> Not sure what you mean. During boot? i.e. in the payload or GRUB? it's
> up to each program (payload/bootloader/OS) running what it does when
> this is pressed.
> > ESC for reboot from payload memtest or USB memtest just stops (crashes?)
>
> Probably in the not-much-tested category.
> > access to CMOS using nvramtool does not work. complains about missing
> >
> > HAVE_OPTION_TABLE. but both USE_ and HAVE_OPTION_TABLE are set. this
>
> Unexpected, but there were similar reports before. Worth investigating.
> > makes it impossible enabling the debug boot log for the board status
> > database
>
> Doesn't sound like this is necessary. Maybe some outdated documentation
> told you to do so?

Commit b863468533 (util/board_status: Add support of CMOS values dump) 
[1] added the `nvramtool` dependency.
> > I somehow cannot enter the grub boot menu. might this be related to
> >
> > missing graphics bios?
>
> If your on-disk GRUB is configured to switch to a graphics mode, no
> matter what, this is expected. Your configuration coreboot display init
> SeaVGABIOS does not allow mode switching.

Please provide `/etc/default/grub` and `/boot/grub/grub.cfg`.

Kind regards,

Paul

[JPT]

Hi Nico, Hi Paul

I tried to apply your directions.
I think I did everything except trying qemu.

A lot points to a SeaBIOS problem.
Is there any guide on how to integrate a custom SeaBIOS?
> You should flash with `--ifd -i BIOS`, otherwise the ME firmware gets
> overwritten.

IFD this collides with the --layout parameter. which one should I use?
> > 4.11-2619-gd8bd3ff197-dirty
> > It's a recent upstream revision. The -dirty, though, means you have
> > changes in your local repository that are not upstream.

3rd-party/intel-microcode wasn't up to date.
enabling USE_BLOBS updated this subrepo.
But still the build id includes "-dirty". git status shows nothing.
> > > Kubuntu 20.4: systemd-udevd runs amok all the time. this might be
> > >
> > > because it tries modprobe nvidia-uvm.

Well, uninstalling all nvidia drivers solved the problem.
let's just ignore it for now.

[ACPI shutdown, reboot and standby]
> > These three are definitely unexpected and most likely related. Can you
> > get us a copy of these AE_NOT_FOUND messages?

Reached Target Power-Off
thinkpad_acpi: acpi_evalf(\BLTH, vd, ...) failed: AE_NOT_FOUND
thinkpad_acpi: acpi_evalf(\WGSV, vd, ...) failed: AE_NOT_FOUND
reboot: Power down
- then nothing happens

Reached Target Reboot
thinkpad_acpi: acpi_evalf(\BLTH, vd, ...) failed: AE_NOT_FOUND
thinkpad_acpi: acpi_evalf(\WGSV, vd, ...) failed: AE_NOT_FOUND
reboot: Restarting system
- after about 30 seconds it switches off.

[Grub menu not visible]
solved by setting text mode menu:
GRUB_TERMINAL=console
so it's definitely something about VESA. (SeaBIOS?)
> > Ctrl-Alt-Del does not work?
>
> Not sure what you mean. During boot? i.e. in the payload or GRUB? it's
> up to each program (payload/bootloader/OS) running what it does when
> this is pressed.

It doesn't work from grub menu, from memtest, from DOS, from SeaBIOS
boot menu. Then it should be SeaBIOS's fault.
from passmark Memtest it doesn't do anything. Just continues to run.
from everything else it just halts (eg no cursor blinking any more), no
reaction to any input.
> > > access to CMOS using nvramtool does not work. complains about missing
> > >
> > > HAVE_OPTION_TABLE. but both USE_ and HAVE_OPTION_TABLE are set.
> >
> > Unexpected, but there were similar reports before. Worth investigating.
> > > this makes it impossible enabling the debug boot log for the board status
> > > database
> >
> > Doesn't sound like this is necessary. Maybe some outdated documentation
> > told you to do so?

https://review.coreboot.org/cgit/coreboot.git/tree/util/board_status/README

that's it from me.
I updated the logs
https://mega.nz/folder/fR521QIZ#ypfaHUIibD_cjWktYuyrNw
but there is not much changed.

regards

Jan

[Nico Huber]

Hello Jan,

On 07.05.20 17:38, JPT wrote:
> I got several problems with coreboot on X220.

your issues seem very random and unexpected. Three possible explanations
come to mind: a) coreboot fails to properly initialize your RAM and this
causes general instability. b) your coreboot build is flawed, e.g. some-
thing odd in your configuration. c) you flashed the Intel ME Firmware
along with coreboot and it's in a bad state.

If you want to further analyze things, please attach your coreboot
`.config`.

Nico

[Alexander 'lynxis' Couzens]

Hi Jan,

i've a x220 around. If you send me your image, I can take a look, how
it behaves on my system.
> If you want to further analyze things, please attach your coreboot
> `.config`.

it's also included in your bios image, if you don't have it anymore.

Best,
lynxis