Hi,
on my Thinkpad X230 coreboot master does not boot when using the option CONFIG_USE_OPTION_TABLE=y. The screen just stays black. I have no log. This may also affect other platforms(?).
I git-bisected the issue and it seems the following commit has introduced it:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
Attached is a working an non-working defconfig. Note that CONFIG_NVRAMCUI_SECONDARY_PAYLOAD=y pulls in CONFIG_USE_OPTION_TABLE=y as a dependency.
Regards,
Confirmed on qemu-i440fx. It's strange that it already has different behavior in romstage between setting and not setting USE_OPTION_TABLE. I still don't know what is broken in this commit.
On Sun, Dec 06, 2020 at 11:24:11PM +0100, Merlin Büge wrote:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
By using gdb, I can debug on QEMU. I can see in bootblock, romstage and postcar, when USE_OPTION_TABLE is set, the debug_level option is always read, so there is a cbfs_map_ro() in each stage. The buggy thing is in postcar stage, the cbfs mcache cannot be found, so its size becomes zero, then all the files in the cbfs cannot be loaded because of the mcache overflow, which results in failing to load the ramstage.
On Mon, Dec 7, 2020 at 12:18 PM Iru Cai mytbk920423@gmail.com wrote:
Confirmed on qemu-i440fx. It's strange that it already has different behavior in romstage between setting and not setting USE_OPTION_TABLE. I still don't know what is broken in this commit.
On Sun, Dec 06, 2020 at 11:24:11PM +0100, Merlin Büge wrote:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
Sorry for the breakage and thanks for narrowing down the issue. I think Arthur had also just figured out the same problem and uploaded a quick fix here: https://review.coreboot.org/48407
On Sun, Dec 6, 2020 at 11:09 PM Iru Cai mytbk920423@gmail.com wrote:
By using gdb, I can debug on QEMU. I can see in bootblock, romstage and postcar, when USE_OPTION_TABLE is set, the debug_level option is always read, so there is a cbfs_map_ro() in each stage. The buggy thing is in postcar stage, the cbfs mcache cannot be found, so its size becomes zero, then all the files in the cbfs cannot be loaded because of the mcache overflow, which results in failing to load the ramstage.
On Mon, Dec 7, 2020 at 12:18 PM Iru Cai mytbk920423@gmail.com wrote:
Confirmed on qemu-i440fx. It's strange that it already has different behavior in romstage between setting and not setting USE_OPTION_TABLE. I still don't know what is broken in this commit.
On Sun, Dec 06, 2020 at 11:24:11PM +0100, Merlin Büge wrote:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
-- My website: https://vimacs.lcpu.club
Please do not send me Microsoft Office/Apple iWork documents. Send OpenDocument instead! http://fsf.org/campaigns/opendocument/
This is still broken for my vboot setup on Thinkpad T530. I have pulled in the most recent fixes (i.e., https://review.coreboot.org/48482). I flashed to RW_A, which does not boot. My old build on RO still works fine. The RO build is a few months old. On IRC, one suggestion was to try with NO_CBFS_MCACHE, which does work. That is, building with NO_CBFS_MCACHE and flashing to RW_A works. Any ideas ? It's quite difficult to open this laptop. So I would like to not touch RO if possible.
My defconfig: # CONFIG_USE_BLOBS is not set CONFIG_VENDOR_LENOVO=y CONFIG_FMDFILE="src/mainboard/lenovo/t530/vboot-rwab.fmd" CONFIG_VBOOT=y CONFIG_HAVE_IFD_BIN=y CONFIG_BOARD_LENOVO_T530=y CONFIG_PCIEXP_L1_SUB_STATE=y CONFIG_PCIEXP_CLK_PM=y CONFIG_H8_SUPPORT_BT_ON_WIFI=y CONFIG_H8_FN_KEY_AS_VBOOT_RECOVERY_SW=y CONFIG_HAVE_ME_BIN=y CONFIG_CHECK_ME=y CONFIG_USE_ME_CLEANER=y CONFIG_HAVE_GBE_BIN=y CONFIG_DRIVERS_PS2_KEYBOARD=y CONFIG_COREINFO_SECONDARY_PAYLOAD=y CONFIG_MEMTEST_SECONDARY_PAYLOAD=y CONFIG_NVRAMCUI_SECONDARY_PAYLOAD=y CONFIG_TINT_SECONDARY_PAYLOAD=y CONFIG_MEMTEST_MASTER=y
On Mon, Dec 7, 2020 at 3:48 PM Julius Werner jwerner@chromium.org wrote:
Sorry for the breakage and thanks for narrowing down the issue. I think Arthur had also just figured out the same problem and uploaded a quick fix here: https://review.coreboot.org/48407
On Sun, Dec 6, 2020 at 11:09 PM Iru Cai mytbk920423@gmail.com wrote:
By using gdb, I can debug on QEMU. I can see in bootblock, romstage and
postcar,
when USE_OPTION_TABLE is set, the debug_level option is always read, so
there
is a cbfs_map_ro() in each stage. The buggy thing is in postcar stage,
the cbfs mcache
cannot be found, so its size becomes zero, then all the files in the
cbfs cannot be loaded
because of the mcache overflow, which results in failing to load the
ramstage.
On Mon, Dec 7, 2020 at 12:18 PM Iru Cai mytbk920423@gmail.com wrote:
Confirmed on qemu-i440fx. It's strange that it already has different behavior in romstage between setting and not setting USE_OPTION_TABLE. I still don't know what is broken in this commit.
On Sun, Dec 06, 2020 at 11:24:11PM +0100, Merlin Büge wrote:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
-- My website: https://vimacs.lcpu.club
Please do not send me Microsoft Office/Apple iWork documents. Send
OpenDocument instead! http://fsf.org/campaigns/opendocument/ _______________________________________________ coreboot mailing list -- coreboot@coreboot.org To unsubscribe send an email to coreboot-leave@coreboot.org
Hi Prasun,
So just to understand your setup correctly, you're saying that you are using vboot in an RW-A/B configuration -- but your RO image is some old coreboot version, and you just updated the RW-A to the newest one? Unfortunately, that's not how vboot can work -- when building coreboot master, you always need to use RO and RW-A/B from the same commit. There are numerous interaction points between the RO and RW stages (e.g. the memory layout in CAR) that need to fit together perfectly to make it all work. We make changes to coreboot that are incompatible with older RO versions all the time (my patch series added a CBFS mcache area to the CAR layout, that's probably what's causing your issue), if you have been running this configuration for a while it was just pure luck that you didn't see it fail yet.
vboot was designed so you would create a branch when you build your RO version, and then the RW-A/B should only be updated with bug fixes built from that branch where you only cherry-pick in patches to solve specific small issues, and you need to carefully make sure that none of those patches touch the RO/RW interface. It's not designed to just keep updating the RW part to the newest master. We want to keep developing across all stages for master so unfortunately that's really the only way it can work. (I guess we could start having official RW-update branches for each coreboot release tag to pick in RW-safe fixes and small features... I think there just hasn't been enough interest in upstream vboot to do that for now.)
On Thu, Dec 10, 2020 at 2:24 AM Prasun Gera prasun.gera@gmail.com wrote:
This is still broken for my vboot setup on Thinkpad T530. I have pulled in the most recent fixes (i.e., https://review.coreboot.org/48482). I flashed to RW_A, which does not boot. My old build on RO still works fine. The RO build is a few months old. On IRC, one suggestion was to try with NO_CBFS_MCACHE, which does work. That is, building with NO_CBFS_MCACHE and flashing to RW_A works. Any ideas ? It's quite difficult to open this laptop. So I would like to not touch RO if possible.
My defconfig: # CONFIG_USE_BLOBS is not set CONFIG_VENDOR_LENOVO=y CONFIG_FMDFILE="src/mainboard/lenovo/t530/vboot-rwab.fmd" CONFIG_VBOOT=y CONFIG_HAVE_IFD_BIN=y CONFIG_BOARD_LENOVO_T530=y CONFIG_PCIEXP_L1_SUB_STATE=y CONFIG_PCIEXP_CLK_PM=y CONFIG_H8_SUPPORT_BT_ON_WIFI=y CONFIG_H8_FN_KEY_AS_VBOOT_RECOVERY_SW=y CONFIG_HAVE_ME_BIN=y CONFIG_CHECK_ME=y CONFIG_USE_ME_CLEANER=y CONFIG_HAVE_GBE_BIN=y CONFIG_DRIVERS_PS2_KEYBOARD=y CONFIG_COREINFO_SECONDARY_PAYLOAD=y CONFIG_MEMTEST_SECONDARY_PAYLOAD=y CONFIG_NVRAMCUI_SECONDARY_PAYLOAD=y CONFIG_TINT_SECONDARY_PAYLOAD=y CONFIG_MEMTEST_MASTER=y
On Mon, Dec 7, 2020 at 3:48 PM Julius Werner jwerner@chromium.org wrote:
Sorry for the breakage and thanks for narrowing down the issue. I think Arthur had also just figured out the same problem and uploaded a quick fix here: https://review.coreboot.org/48407
On Sun, Dec 6, 2020 at 11:09 PM Iru Cai mytbk920423@gmail.com wrote:
By using gdb, I can debug on QEMU. I can see in bootblock, romstage and postcar, when USE_OPTION_TABLE is set, the debug_level option is always read, so there is a cbfs_map_ro() in each stage. The buggy thing is in postcar stage, the cbfs mcache cannot be found, so its size becomes zero, then all the files in the cbfs cannot be loaded because of the mcache overflow, which results in failing to load the ramstage.
On Mon, Dec 7, 2020 at 12:18 PM Iru Cai mytbk920423@gmail.com wrote:
Confirmed on qemu-i440fx. It's strange that it already has different behavior in romstage between setting and not setting USE_OPTION_TABLE. I still don't know what is broken in this commit.
On Sun, Dec 06, 2020 at 11:24:11PM +0100, Merlin Büge wrote:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
-- My website: https://vimacs.lcpu.club
Please do not send me Microsoft Office/Apple iWork documents. Send OpenDocument instead! http://fsf.org/campaigns/opendocument/
coreboot mailing list -- coreboot@coreboot.org To unsubscribe send an email to coreboot-leave@coreboot.org
Hi Julius, Yes, I have been updating only the RW-A region which has worked quite well for over a year. Thanks for the explanation. My mental model of vboot was not accurate. So it seems like I will have to flash the entire image. This does, however, limit the utility of the RO region in its current form. My main motivation for using vboot was the safety from botched updates since it takes an entire afternoon to dismantle and put this laptop back together. The RW-safe branch tags would be great if that happens in the future.
Thanks, Prasun
On Mon, Dec 14, 2020 at 6:37 PM Julius Werner jwerner@chromium.org wrote:
Hi Prasun,
So just to understand your setup correctly, you're saying that you are using vboot in an RW-A/B configuration -- but your RO image is some old coreboot version, and you just updated the RW-A to the newest one? Unfortunately, that's not how vboot can work -- when building coreboot master, you always need to use RO and RW-A/B from the same commit. There are numerous interaction points between the RO and RW stages (e.g. the memory layout in CAR) that need to fit together perfectly to make it all work. We make changes to coreboot that are incompatible with older RO versions all the time (my patch series added a CBFS mcache area to the CAR layout, that's probably what's causing your issue), if you have been running this configuration for a while it was just pure luck that you didn't see it fail yet.
vboot was designed so you would create a branch when you build your RO version, and then the RW-A/B should only be updated with bug fixes built from that branch where you only cherry-pick in patches to solve specific small issues, and you need to carefully make sure that none of those patches touch the RO/RW interface. It's not designed to just keep updating the RW part to the newest master. We want to keep developing across all stages for master so unfortunately that's really the only way it can work. (I guess we could start having official RW-update branches for each coreboot release tag to pick in RW-safe fixes and small features... I think there just hasn't been enough interest in upstream vboot to do that for now.)
On Thu, Dec 10, 2020 at 2:24 AM Prasun Gera prasun.gera@gmail.com wrote:
This is still broken for my vboot setup on Thinkpad T530. I have pulled
in the most recent fixes (i.e., https://review.coreboot.org/48482). I flashed to RW_A, which does not boot. My old build on RO still works fine. The RO build is a few months old. On IRC, one suggestion was to try with NO_CBFS_MCACHE, which does work. That is, building with NO_CBFS_MCACHE and flashing to RW_A works. Any ideas ? It's quite difficult to open this laptop. So I would like to not touch RO if possible.
My defconfig: # CONFIG_USE_BLOBS is not set CONFIG_VENDOR_LENOVO=y CONFIG_FMDFILE="src/mainboard/lenovo/t530/vboot-rwab.fmd" CONFIG_VBOOT=y CONFIG_HAVE_IFD_BIN=y CONFIG_BOARD_LENOVO_T530=y CONFIG_PCIEXP_L1_SUB_STATE=y CONFIG_PCIEXP_CLK_PM=y CONFIG_H8_SUPPORT_BT_ON_WIFI=y CONFIG_H8_FN_KEY_AS_VBOOT_RECOVERY_SW=y CONFIG_HAVE_ME_BIN=y CONFIG_CHECK_ME=y CONFIG_USE_ME_CLEANER=y CONFIG_HAVE_GBE_BIN=y CONFIG_DRIVERS_PS2_KEYBOARD=y CONFIG_COREINFO_SECONDARY_PAYLOAD=y CONFIG_MEMTEST_SECONDARY_PAYLOAD=y CONFIG_NVRAMCUI_SECONDARY_PAYLOAD=y CONFIG_TINT_SECONDARY_PAYLOAD=y CONFIG_MEMTEST_MASTER=y
On Mon, Dec 7, 2020 at 3:48 PM Julius Werner jwerner@chromium.org
wrote:
Sorry for the breakage and thanks for narrowing down the issue. I think Arthur had also just figured out the same problem and uploaded a quick fix here: https://review.coreboot.org/48407
On Sun, Dec 6, 2020 at 11:09 PM Iru Cai mytbk920423@gmail.com wrote:
By using gdb, I can debug on QEMU. I can see in bootblock, romstage
and postcar,
when USE_OPTION_TABLE is set, the debug_level option is always read,
so there
is a cbfs_map_ro() in each stage. The buggy thing is in postcar
stage, the cbfs mcache
cannot be found, so its size becomes zero, then all the files in the
cbfs cannot be loaded
because of the mcache overflow, which results in failing to load the
ramstage.
On Mon, Dec 7, 2020 at 12:18 PM Iru Cai mytbk920423@gmail.com
wrote:
Confirmed on qemu-i440fx. It's strange that it already has different behavior in romstage between setting and not setting USE_OPTION_TABLE. I still don't know what is broken in this commit.
On Sun, Dec 06, 2020 at 11:24:11PM +0100, Merlin Büge wrote:
9d0cc2aea9 cbfs: Introduce cbfs_ro_map() and cbfs_ro_load() https://review.coreboot.org/c/coreboot/+/39306
-- My website: https://vimacs.lcpu.club
Please do not send me Microsoft Office/Apple iWork documents. Send
OpenDocument instead! http://fsf.org/campaigns/opendocument/
coreboot mailing list -- coreboot@coreboot.org To unsubscribe send an email to coreboot-leave@coreboot.org
-
Iru Cai -
Julius Werner -
Merlin Büge -
Prasun Gera