[coreboot] Thinkpad X230, Coreboot 4.7 + SeaBIOS 1.11.0 flashing, memtest86+ 5.01, fancontrol and WoL

[Duncan]

Hi Jan,

Jan Malakhovski:
> Hi.
> 
> # Primary TOC
> 
> * Thinkpad X230 seems to be a fairly common target for coreboot, but I
>   though that another recent success report wouldn't hurt.
> 
> * Flashing with "WoL on AC" trick (missing on the wiki).
> 
> * Failure report on using Debian's memtest86+ and success report on
>   using coreboot's memtest86+.
> 
> * The laptop also overheats while running memtest86+ with coreboot (but
>   not vendor BIOS) which, I think, is a bug/missing feature in coreboot.
> 
> * There's also a Wake on LAN issue/question.
> 
> # The Success Report
> 
> I flashed Thinkpad X230 with Coreboot tagged 4.7
> (fd470f7163709c1022ee6185134a2387812774ec) with a config that has the
> following diff from the default 4.7 X230 config (made by `cat
> configs/builder/config.lenovo_x230 > .config ; make menuconfig` followed
> by <ESC><ESC><RET>):
> 
> --- config.x230.v4.7.def
> +++ config.x230.v4.7.my
> @@ -11,7 +11,7 @@
>  CONFIG_CBFS_PREFIX="fallback"
>  CONFIG_COMPILER_GCC=y
>  # CONFIG_COMPILER_LLVM_CLANG is not set
> -# CONFIG_ANY_TOOLCHAIN is not set
> +CONFIG_ANY_TOOLCHAIN=y
>  # CONFIG_CCACHE is not set
>  # CONFIG_FMD_GENPARSER is not set
>  # CONFIG_UTIL_GENPARSER is not set
> @@ -112,6 +112,7 @@
>  CONFIG_MAX_CPUS=8
>  CONFIG_CACHE_ROM_SIZE_OVERRIDE=0x0
>  CONFIG_CBFS_SIZE=0x100000
> +CONFIG_PAYLOAD_CONFIGFILE=""
>  CONFIG_VGA_BIOS_ID="8086,0166"
>  CONFIG_ONBOARD_VGA_IS_PRIMARY=y
>  CONFIG_DIMM_SPD_SIZE=256
> @@ -136,8 +137,8 @@
>  CONFIG_FMDFILE=""
>  CONFIG_PRERAM_CBMEM_CONSOLE_SIZE=0xc00
>  # CONFIG_DRIVERS_UART_8250IO is not set
> -CONFIG_IFD_BIN_PATH="site-local/descriptor.bin"
> -CONFIG_ME_BIN_PATH="site-local/me.bin"
> +CONFIG_IFD_BIN_PATH="site-local/flashregion_0_flashdescriptor.bin"
> +CONFIG_ME_BIN_PATH="site-local/flashregion_2_intel_me.bin"
>  # CONFIG_BOARD_LENOVO_G505S is not set
>  # CONFIG_BOARD_LENOVO_L520 is not set
>  # CONFIG_BOARD_LENOVO_R400 is not set
> @@ -338,6 +339,7 @@
>  #
>  CONFIG_EC_ACPI=y
>  CONFIG_EC_LENOVO_H8=y
> +CONFIG_SEABIOS_PS2_TIMEOUT=3000
>  CONFIG_H8_BEEP_ON_DEATH=y
>  CONFIG_H8_FLASH_LEDS_ON_DEATH=y
>  # CONFIG_H8_SUPPORT_BT_ON_WIFI is not set
> @@ -348,9 +350,9 @@
>  # Intel Firmware
>  #
>  # CONFIG_EM100 is not set
> -# CONFIG_CHECK_ME is not set
> +CONFIG_CHECK_ME=y
>  # CONFIG_USE_ME_CLEANER is not set
> -CONFIG_GBE_BIN_PATH="site-local/gbe.bin"
> +CONFIG_GBE_BIN_PATH="site-local/flashregion_3_gbe.bin"
>  # CONFIG_HAVE_EC_BIN is not set
>  # CONFIG_MAINBOARD_HAS_CHROMEOS is not set
>  # CONFIG_GOOGLE_SMBIOS_MAINBOARD_VERSION is not set
> @@ -621,17 +623,31 @@
>  #
>  # Payload
>  #
> -CONFIG_PAYLOAD_NONE=y
> +# CONFIG_PAYLOAD_NONE is not set
>  # CONFIG_PAYLOAD_ELF is not set
>  # CONFIG_PAYLOAD_BAYOU is not set
>  # CONFIG_PAYLOAD_FILO is not set
>  # CONFIG_PAYLOAD_GRUB2 is not set
> -# CONFIG_PAYLOAD_SEABIOS is not set
> +CONFIG_PAYLOAD_SEABIOS=y
>  # CONFIG_PAYLOAD_UBOOT is not set
>  # CONFIG_PAYLOAD_LINUX is not set
>  # CONFIG_PAYLOAD_TIANOCORE is not set
> +CONFIG_PAYLOAD_FILE="payloads/external/SeaBIOS/seabios/out/bios.bin.elf"
> +CONFIG_SEABIOS_STABLE=y
> +# CONFIG_SEABIOS_MASTER is not set
> +# CONFIG_SEABIOS_REVISION is not set
> +# CONFIG_SEABIOS_THREAD_OPTIONROMS is not set
> +CONFIG_SEABIOS_VGA_COREBOOT=y
> +CONFIG_SEABIOS_BOOTORDER_FILE=""
> +CONFIG_PAYLOAD_VGABIOS_FILE="payloads/external/SeaBIOS/seabios/out/vgabios.bin"
> +CONFIG_SEABIOS_DEBUG_LEVEL=-1
> +
> +#
> +# Using default SeaBIOS log level
> +#
>  CONFIG_PAYLOAD_OPTIONS=""
>  # CONFIG_PXE is not set
> +CONFIG_COMPRESSED_PAYLOAD_LZMA=y
>  # CONFIG_PAYLOAD_IS_FLAT_BINARY is not set
> 
> You should probably ignore CONFIG_ANY_TOOLCHAIN=y since I simply built
> gcc 6.3 with coreboot patches via nix package manager and then built
> coreboot using that, so I'm using exactly the same compiler, only
> packaged via nix (because it was simpler for me this way).
> 
> # Flashing with "WoL on AC" trick
> 
> A couple of words on reading/flashing X230's SPI
> 
> Neither Raspberry Pi 3 B+, nor Beagle Bone Black can supply enough power
> to feed the MX chips to the board, so the wiki
> https://www.coreboot.org/Board:lenovo/x230 both recommends an external
> power supply and warns against using it at the same time (as it can
> brick the board).

The wiki page says many things. I've edited the page for clarity about
that and removed the odd Buspirate-specific instructions (I am not sure
why BusPirate users can't just look at Flashrom wiki etc).

In general don't use external power for in-system-programming (ISP) on
the X230 (or any other recent Intel laptop) - the boards in question
really weren't designed to have flash powered that way.
> 
> But there's a much simpler and safer method described in
> https://www.bios-mods.com/forum/Thread-REQUEST-Lenovo-Thinkpad-X230-Whitelist-removal?pid=91134#pid91134
> and
> https://www.bios-mods.com/forum/Thread-REQUEST-Lenovo-Thinkpad-X230-Whitelist-removal?pid=91787#pid91787
> that doesn't require an external power supply.
> 
> In short: enable Wake On Lan on AC power in Lenovo BIOS, disconnect AC
> and battery, insert live Ethernet cable, connect AC, Ethernet port
> should go live (LEDs). Connect your external programmer _but leave
> VCC/3.3V line disconnected_ (i.e. connect all the data lines and the
> ground, but not the VCC). The Ethernet port is on the same power line as
> MX chips, hence flashing with RPi/BBB with VCC disconnected will work as
> with older models.

Do NOT power system flash externally as well as with the WoL feature -
this can only do bad things to the board.
> 
> The only difference with older models is that you have to cut
> coreboot.rom into 8M and 4M pieces and flash them separately. (You can
> flash the top 4M only, but then, apparently, you wouldn't be able to
> reflash the ME region internally).

There isn't actually a difference to older models, except that they've
split system flash over two flash chips. The BIOS region (the area we
may install Coreboot on) on the X230 is actually 7MiB, not 4MiB. It's
just that the 4MiB can be used on its own for Coreboot, without having
to touch the 8MiB flash chip. If you look at the flash descriptor,
you'll see that the BIOS region is described as 7MiB, that is, leading
from the last 3MiB of the 8MiB flash chip to the end of the 4MiB flash
chip.

If you want to make use of the whole 7MiB (without changing the size of
other regions per se), either:
	1. Flash internally
	2. Build a Coreboot image using the flash descriptor and other Intel
blobs dumped from your system, and then split the image e.g. using dd so
that you can change flash contents of both the 8MiB and 4MiB flash chips.

Alternatively, you could replace the two flash chips with a single 16MiB
flash chip (as I understand it, this is the largest size supported by
the SPI controller), which is easier to use, and you'll have more space
for payloads (some payloads like a Linux payload can be very large, and
Linux payloads are very flexible and may be fun to play with).
> 
> The system boots ok into Linux, VGA seems to work, suspend-resume works.
> 
> # The whole process
> 
> Set WoL on AC in Lenovo BIOS. Exit saving changes.
> Disconnect battery and AC (charger).
> Insert live Ethernet cable into laptop's Ethernet port.
> Connect AC.
> Confirm Ethernet LEDs blink but the screen stays blank, the fan is off
> and the laptop does not start booting (if it does you have "power-on on
> AC connect" or whatever enabled in vendor BIOS, disable, save and repeat).
> 
> Disconnect everything.
> Remove keyboard, palm rest. I removed the hard disk too, just in case.
> Short flasher's (RPi/BBB/etc) and laptop's grounds (any metal part of
> the laptop, e.g. extension card bracket, will do).
> Connect the flasher's clip without the VCC line to the top SPI chip.
> Insert live Ethernet cable into laptop's Ethernet port.
> Connect AC.
> Confirm Ethernet LEDs blink.
> 
> So you attached the flasher without VCC to the top SPI chip.
> Confirm flashrom sees the chip.
> $ flashrom -p linux_spi:dev=/dev/spidev0.0 
> 
> Read the flash (chip model may vary, but the chip should be 4M).
> $ flashrom -c 'MX25L3206E/MX25L3208E' -p linux_spi:dev=/dev/spidev0.0 -r top.rom 
> Repeat, diff/checksum.
> 
> Reattach to the bottom chip.
> $ flashrom -c "MX25L6406E/MX25L6408E" -p linux_spi:dev=/dev/spidev0.0 -r bottom.rom
> Repeat, diff/checksum.
> 
> $ cat bottom.rom top.rom > full.rom
> $ mv full.rom /path/to/site-local
> $ cd /path/to/coreboot/site-local
> $ ifdtool -x full.rom # to extract flashregion_* blobs
> $ cd ..
> $ make distclean
> $ cat config.x230.v4.7.my > .config
> $ make # build
> $ mv build/coreboot.rom site-local
> $ cd site-local
> $ diff -Nua <(hexdump -C full.rom) <(hexdump -C coreboot.rom) | less # zsh syntax
> To confirm that only about 4 or so bytes change in the IFD header
> (`make` does `ifdtool -u` automagically) in the first 8M and everything
> changes in the last 4M.
> $ dd if=coreboot.rom of=to_flash.bottom.rom bs=1M count=8
> $ dd if=coreboot.rom of=to_flash.top.rom bs=1M skip=8
> $ mv to_flash* /path/to/flasher
> 
> Still attached to the bottom chip.
> $ flashrom -c "MX25L6406E/MX25L6408E" -p linux_spi:dev=/dev/spidev0.0 -w to_flash.bottom.rom
> 
> Reattach to the top chip.
> $ flashrom -c 'MX25L3206E/MX25L3208E' -p linux_spi:dev=/dev/spidev0.0 -r to_flash.top.rom 
> 
> Done.
> Disconnect the flasher.
> Disconnect the AC.
> Attach the keyboard.
> Attach some bootable drive (HDD/SDD/USB).
> Connect AC.
> Press power button.
> It should boot.
> 
> Worked like a charm for me.
> 
> # Memtest86+
> 
> The problem was with running distro-supplied (equivalent to Debian's)
> memtest86+.
> 
> The laptop has a single 4G stick of RAM. When running memtest86+ under
> vendor BIOS memtest86+ reports no errors after 24h of testing. Under
> coreboot and seabios memtest86+ starts as normal and draws its blue
> testing screen to the LVDS as expected, but
> 
> 1) it reports a lot more memory (~300M or so) compared to running under
>    vendor BIOS,
> 2) it immediately fails on test #2 with a bunch of errors at 000bfe8eXXX
>    (which is around 3070.8M, XXX changes between runs), with bits in
>    error mask 000000XX (XX changes too),
> 3) some tests make the screen flicker with black lines on some runs.
> 
> PaulePanter on IRC suggested applying patches from
> https://review.coreboot.org/memtest86plus and that version then works as
> expected. But I looked through the changes and I don't understand why
> exactly the Debian's memtest86+ fails with such strange errors. Does
> coreboot/seabios report VGA memory region in some non-standard way?
> 
> # Memtest86+ overheating issue
> 
> The machine overheats while running memtest86+. People on IRC suggest
> that this is because coreboot has no fan control in SMM. Placing the
> laptop into a cold room "fixes" it, but SMM fan control would be nice to
> have.
> 
> # Wake on LAN issue/question
> 
> After flashing coreboot I expected "Wake on Lan on AC connect" to stop
> working since it was configured in vendor BIOS, but it still works. Why?
> What code enables it? Can I disable it? How?
> 
> Also, does WoL need the GBE/ME blobs to work? It is kinda useful that it
> still works with coreboot since and I can now reflash with impunity
> using the same WoL method if I brick the ROM. But I want to know if I
> would be able to clean the ME region and wipe the GBE region (and use
> USB Ethernet instead) and still use the WoL trick for flashing.
> 
One thing to note is that the GbE "blob" isn't really a blob as one
would think. It is configuration data (which is arguably not possible to
copyright) and thus should be possible to generate. It's not executable
as far as I understand.

I'd honestly not recommend using USB ethernet - the onboard NIC performs
well and is reliable.

There are no issues with using the WoL feature and "cleaning" the ME
region. Some users have reported "cleaning" the ME region makes suspend
cease to work, but I've not experienced this. It may be kernel/OS
version or they may have installed their dump that is corrupted in some
way. It seems like something fairly difficult to debug.

It's possible that they are encountering issues with the ME not working
with their dumped firmware image. In general, it is best to flash back a
"factory fresh" firmware image (perhaps, in this case, "cleaned" with
the ME Cleaner program). It is not clear to me why this is, but this is
what some users have reported - as far as I understand, the ME itself
may hold some sort of persistent state. Spooky.

In any case, I'm not sure if it's a good idea to really change the ME
region, if you think it's risky.

Best,
Duncan