coreboot-gerrit November 2021

coreboot-gerrit@coreboot.org

1 participants
6394 discussions

Change in SerialTest[master]: Add rough approximation of SerialTest scaffold
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/SerialTest/+/39083 ) Change subject: Add rough approximation of SerialTest scaffold ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/SerialTest/+/39083 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: SerialTest Gerrit-Branch: master Gerrit-Change-Id: Iab8d507dee6087bf32280ad6845fdd2d7715bf5c Gerrit-Change-Number: 39083 Gerrit-PatchSet: 7 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Angel Pons <th3fanbus(a)gmail.com> Gerrit-Reviewer: Dossym Nurmukhanov <dossym(a)google.com> Gerrit-Reviewer: Martin Roth <martinroth(a)google.com> Gerrit-Reviewer: Ron Minnich <rminnich(a)gmail.com> Gerrit-Reviewer: Stefan Reinauer <stefan.reinauer(a)coreboot.org> Gerrit-CC: Greg Edelston <gredelston(a)google.com> Gerrit-CC: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-MessageType: abandon

1 0

Change in coreboot[master]: util/testing: Give meaningful error if intel-sec-tools aren't around
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/coreboot/+/57527 ) Change subject: util/testing: Give meaningful error if intel-sec-tools aren't around ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/coreboot/+/57527 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: I035747746ca5fd54841bd67352044dde12a28185 Gerrit-Change-Number: 57527 Gerrit-PatchSet: 3 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Christian Walter <christian.walter(a)9elements.com> Gerrit-Reviewer: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-Reviewer: Rizwan Qureshi <rizwan.qureshi(a)intel.com> Gerrit-Reviewer: build bot (Jenkins) <no-reply(a)coreboot.org> Gerrit-MessageType: abandon

1 0

Change in coreboot[master]: Documentation: Define which languages are permitted.
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/coreboot/+/58382 ) Change subject: Documentation: Define which languages are permitted. ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/coreboot/+/58382 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: Idd1c5850be6c4ce62f2c676e1ce3a4a733269122 Gerrit-Change-Number: 58382 Gerrit-PatchSet: 3 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Felix Singer <felixsinger(a)posteo.net> Gerrit-Reviewer: Idwer Vollering <vidwer(a)gmail.com> Gerrit-Reviewer: Michael Niewöhner <foss(a)mniewoehner.de> Gerrit-Reviewer: Ron Minnich <rminnich(a)gmail.com> Gerrit-Reviewer: Stefan Reinauer <stefan.reinauer(a)coreboot.org> Gerrit-Reviewer: build bot (Jenkins) <no-reply(a)coreboot.org> Gerrit-CC: Nico Huber <nico.h(a)gmx.de> Gerrit-CC: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-MessageType: abandon

1 0

Change in coreboot[master]: Makefile: Defer normalizing configuration for reproducible builds
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/coreboot/+/52292 ) Change subject: Makefile: Defer normalizing configuration for reproducible builds ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/coreboot/+/52292 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: Ia7afd32bd120e5405e65825144b0c30d69931a22 Gerrit-Change-Number: 52292 Gerrit-PatchSet: 8 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Alexander Couzens <lynxis(a)fe80.eu> Gerrit-Reviewer: Angel Pons <th3fanbus(a)gmail.com> Gerrit-Reviewer: Martin Roth <martinroth(a)google.com> Gerrit-Reviewer: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-Reviewer: build bot (Jenkins) <no-reply(a)coreboot.org> Gerrit-MessageType: abandon

1 0

Change in blobs[master]: README: Clarify that we need to be able to download
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/blobs/+/42476 ) Change subject: README: Clarify that we need to be able to download ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/blobs/+/42476 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: blobs Gerrit-Branch: master Gerrit-Change-Id: I45f218748de02e4587802aa147436112f5e0d0dd Gerrit-Change-Number: 42476 Gerrit-PatchSet: 2 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Angel Pons <th3fanbus(a)gmail.com> Gerrit-Reviewer: David Hendricks <david.hendricks(a)gmail.com> Gerrit-Reviewer: Nico Huber <nico.h(a)gmx.de> Gerrit-Reviewer: Stefan Reinauer <stefan.reinauer(a)coreboot.org> Gerrit-CC: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-MessageType: abandon

1 0

Change in coreboot[master]: util/testing: Build test all our tools
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has uploaded this change for review. ( https://review.coreboot.org/c/coreboot/+/49126 ) Change subject: util/testing: Build test all our tools ...................................................................... util/testing: Build test all our tools https://qa.coreboot.org/job/untested-coreboot-files reports a bunch of untouched Makefiles, so we never even attempt to build those tools. This commit currently doesn't build and serves as a reminder that there are tools to fix. Change-Id: Ib6378dbaef9787b3cd3e047068d01c73927fa73f Signed-off-by: Patrick Georgi <pgeorgi(a)google.com> --- M util/testing/Makefile.inc 1 file changed, 3 insertions(+), 1 deletion(-) git pull ssh://review.coreboot.org:29418/coreboot refs/changes/26/49126/1 diff --git a/util/testing/Makefile.inc b/util/testing/Makefile.inc index da6db60..ad30514 100644 --- a/util/testing/Makefile.inc +++ b/util/testing/Makefile.inc @@ -61,7 +61,9 @@ intelvbttool \ nvramtool \ post \ -superiotool +superiotool \ +uio_usbdebug \ +vgabios TEST_PAYLOADLIST_INTERNAL= \ coreinfo \ -- To view, visit https://review.coreboot.org/c/coreboot/+/49126 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: Ib6378dbaef9787b3cd3e047068d01c73927fa73f Gerrit-Change-Number: 49126 Gerrit-PatchSet: 1 Gerrit-Owner: Patrick Georgi <pgeorgi(a)google.com> Gerrit-Reviewer: Martin Roth <martinroth(a)google.com> Gerrit-MessageType: newchange

2 2

Change in coreboot[master]: Documentation: Add suggestion to document flag days
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/coreboot/+/52576 ) Change subject: Documentation: Add suggestion to document flag days ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/coreboot/+/52576 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: Ic7c01a0f1008821ebbda43a426ba1e3d6410e861 Gerrit-Change-Number: 52576 Gerrit-PatchSet: 2 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Felix Held <felix-coreboot(a)felixheld.de> Gerrit-Reviewer: Martin Roth <martinroth(a)google.com> Gerrit-Reviewer: Piotr Król <piotr.krol(a)3mdeb.com> Gerrit-Reviewer: build bot (Jenkins) <no-reply(a)coreboot.org> Gerrit-CC: Julius Werner <jwerner(a)chromium.org> Gerrit-CC: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-MessageType: abandon

1 0

Change in coreboot[master]: Documentation: Ask that contributors ensure complete descriptions
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has abandoned this change. ( https://review.coreboot.org/c/coreboot/+/56448 ) Change subject: Documentation: Ask that contributors ensure complete descriptions ...................................................................... Abandoned -- To view, visit https://review.coreboot.org/c/coreboot/+/56448 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: I66edafd3b5fdbbafde5a67809fcd838619be62f6 Gerrit-Change-Number: 56448 Gerrit-PatchSet: 3 Gerrit-Owner: Patrick Georgi <patrick(a)coreboot.org> Gerrit-Reviewer: Angel Pons <th3fanbus(a)gmail.com> Gerrit-Reviewer: Nico Huber <nico.h(a)gmx.de> Gerrit-Reviewer: Paul Menzel <paulepanter(a)mailbox.org> Gerrit-Reviewer: Tim Wawrzynczak <twawrzynczak(a)chromium.org> Gerrit-Reviewer: build bot (Jenkins) <no-reply(a)coreboot.org> Gerrit-MessageType: abandon

1 0

Change in coreboot[master]: Documentation: Add proposal for firmware testing
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has uploaded this change for review. ( https://review.coreboot.org/c/coreboot/+/37643 ) Change subject: Documentation: Add proposal for firmware testing ...................................................................... Documentation: Add proposal for firmware testing Change-Id: Icb9380050f8ff1aa13ecbb501079e2556e43ca06 Signed-off-by: Patrick Georgi <pgeorgi(a)google.com> --- A Documentation/technotes/2019-12-firmware-testing.md M Documentation/technotes/index.md 2 files changed, 176 insertions(+), 0 deletions(-) git pull ssh://review.coreboot.org:29418/coreboot refs/changes/43/37643/1 diff --git a/Documentation/technotes/2019-12-firmware-testing.md b/Documentation/technotes/2019-12-firmware-testing.md new file mode 100644 index 0000000..64838a7 --- /dev/null +++ b/Documentation/technotes/2019-12-firmware-testing.md @@ -0,0 +1,175 @@ +# Draft design of a firmware-aware integration test system + +## Problem statement + +Most full system test systems have assumptions baked in that make them +unsuitable for firmware level testing. For example, both Chrome OS test +runners assume that the DUT (Device Under Test) has an operational +kernel + network + sshd. That's a useful assumption when covering +userland testing, but not for firmware. Lava has its downsides as well +(TODO: as reported by Philipp, need details). + +This document outlines a system that, when implemented, helps +automatically test properties of firmware level code. It's _likely_ +also suitable for later stages (kernel and userland), but they're +not the main concern: there's enough tooling out there for them. + +## Collecting requirements + +This project doesn't consider a test system running only on the DUT +so it will require a test host that remains in control. Apart from +that, the system should be useful at all ends of the problem space: +hobbyists with little money to spare just the same as test labs +running automated continuous testing. + +Beyond DUT and test host, the system requires some way for them to +communicate, so we'll expect a bidirectional serial console connection +between them (although we don't care much about how this connection +looks like in practice). + +Such a bare-bone operation might require manual intervention directed +by the test host (e.g. the host asking the user to reboot the DUT when +it got stuck). It may also be augmented by additional circuitry that +allows automating these steps. + +These constraints ensure that a test suite can be executed "one-off" +by a developer with minimal hardware investment while also allowing +automated systems to execute the tests without user intervention. + +So these are + +*Requirement 1*: The system must be usable with minimal investment for +manual operation: a host, a DUT and a bidirectional serial connection +between them. + +and + +*Requirement 2*: The system must be able to drive additional controls +to support fully automated operation. + +There will be lots of different types of DUTs, and not all tests +apply to all of them: There's no battery on a desktop board, so no +need to test charging. A test for charging should be automatically +skipped on such a DUT without counting as failure. + +*Requirement 3*: The system must allow to specify DUTs and their +properties, as well as the properties that a test requires to be +meaningful. + +Tests should be self-contained. This means that they should start +from a flash cycle (that a tool like flashrom would optimize away to +a no-op if flash is still in the expected state) and power-on. + +*Requirement 4*: Tests always start with the required firmware image +to write and explicit power-on. + +Tests must be able to state that other tests must have succeeded before +them: A test that exercises boot loader options doesn't fail when the +boot process never gets to the options screen, it just shouldn't run. + +*Requirement 5*: Tests must be able to declare tests they depend on. + +State transitions should be well-defined: The DUT can move from the +power-off state to the power-on state. It can also fail to do so and +remain powered-off. These things need to be spelled out, together +with the consequences that arise from each transition: it's expected +that most transitions would be failures, but that should be noted down. + +And yet, this verbosity shouldn't mean that the tests become +unreadable. Ideally they could be used by a human operator as "regular +text" (with odd characters interspersed) outlining steps to execute +manually, without a test runner. + +*Requirement 6*: It must be simple to state "Initiate action, success +is W, X while failure is Y, Z". + +During the test, the host monitors the serial console and extracts +events from it. Such events could be, for example, "coreboot enters +ramstage". A test can expect events to appear on the stream and fail if +unexpected events appear or no events appear within a given deadline. + +The observed events are reported to the user. + +*Requirement 7*: There's an event parser looking for things that +happen on the DUT. + +The host can send characters to the DUT, for example in a boot loader +or on a terminal provided by an OS that eventually boots. + +*Requirement 8*: Test must be able to run an expect-like language to +drive complex interactions on the serial console. + +Some tests may be better run on the target system (e.g. to parse +out tables in memory, or otherwise inspect boot states). They should +have a minimal set of dependencies so they don't fail because the base +system changed slightly, be portable across architectures and operating +systems. The language should be popular and the implementation robust. + +*Requirement 9*: Implementation language must be popular, robust, +portable and operate with minimal dependencies. + +## Implementation proposal + +The test system will be implemented in Go: it's reasonably popular +(unlike Forth), robust (unlike shell scripts), its statically linked +binaries have minimal dependencies on the host environment (unlike +Python) and it supports multiple architectures and operating systems. + +The build creates a host side test runner (native OS/arch) and a +variable number of DUT test runners (for all supported DUT OS/arch +pairs) that are provided to the DUT on some storage medium (e.g. USB +stick). In a fully automated environment this might require switching +USB ports between host and DUT. + +There's a [liberally licensed implementation of expect][goexpect] that +could be integrated for handling interactive consoles. It supports +parallel execution which is useful to have a listener on the serial +console (when not driven by expect). + +[goexpect]: https://github.com/google/goexpect + +The test runner is called with the names of a set of tests to execute +(individually or as suites that tests can be assigned to) and the +type of DUT to work with. + +From this, it knows how to drive the DUT. The "manual operation" mode +would be just a special type of button driver that asks the user to +conduct some operation and potentially to press Enter. + +Tests are defined in go packages, with per-package set up and tear +down routines. The runner optimizes operations by telling the tear down +routine if the next test will be from the same package (so the amount +of tear down could be reduced). Setup isn't told about the previous +state but should assume a random state. It's only allowed to assume +that the tests it requires to execute earlier have passed. It can +still optimize based on things it measures such as flashrom not writing +unchanged blocks. + +### Flow + +``` + DUT name + Tests to execute +--------------+ + + +---->+Console driver| + | | +--------------+ + | +---->+Button driver | + | | +--------------+ + v v + +-----------+ +----------+ + |Test runner+--------+DUT object+<+ + +-----------+ +----------+ | + | executes |generates | + | v | + | sends+------------+ | control + +<-----+event stream| +----+ + | +------------+ | | + | | | + | +----------+ | + +------------->+Host tests| | + | +----------+ | + | | + | +--------------+ call +------------------+ + +------------->+Target tests +----------+Target test runner| + +--------------+ +------------------+ +``` + diff --git a/Documentation/technotes/index.md b/Documentation/technotes/index.md index 7c231fc..0df2a1b 100644 --- a/Documentation/technotes/index.md +++ b/Documentation/technotes/index.md @@ -2,3 +2,4 @@ * [Dealing with Untrusted Input in SMM](2017-02-dealing-with-untrusted-input-in-smm.md) * [Rebuilding coreboot image generation](2015-11-rebuilding-coreboot-image-generation.md) +* [firmware test runner](2019-12-firmware-testing.md) -- To view, visit https://review.coreboot.org/c/coreboot/+/37643 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: Icb9380050f8ff1aa13ecbb501079e2556e43ca06 Gerrit-Change-Number: 37643 Gerrit-PatchSet: 1 Gerrit-Owner: Patrick Georgi <pgeorgi(a)google.com> Gerrit-MessageType: newchange

6 23

Change in ...coreboot[master]: libpayload: Enable -Wconversion
by Patrick Georgi (Code Review) 19 Nov '21

19 Nov '21

Patrick Georgi has uploaded this change for review. ( https://review.coreboot.org/c/coreboot/+/32185 Change subject: libpayload: Enable -Wconversion ...................................................................... libpayload: Enable -Wconversion Together with -Werror this ensures that no new conversion related issues can appear. BUG=b:111443775 BRANCH=none TEST=make junit.xml has no failed builds Change-Id: Idc37c50cae90d283bd6ec240dd7de3546485a9a9 Signed-off-by: Patrick Georgi <pgeorgi(a)google.com> --- M payloads/libpayload/Makefile.inc 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://review.coreboot.org:29418/coreboot refs/changes/85/32185/1 diff --git a/payloads/libpayload/Makefile.inc b/payloads/libpayload/Makefile.inc index 7787762..ceac4ca 100644 --- a/payloads/libpayload/Makefile.inc +++ b/payloads/libpayload/Makefile.inc @@ -64,7 +64,7 @@ CFLAGS += -ffunction-sections -fdata-sections CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wmissing-prototypes CFLAGS += -Wwrite-strings -Wredundant-decls -Wno-trigraphs -CFLAGS += -Wstrict-aliasing -Wshadow -Werror +CFLAGS += -Wstrict-aliasing -Wshadow -Werror -Wconversion $(obj)/libpayload-config.h: $(KCONFIG_AUTOHEADER) cmp $@ $< 2>/dev/null || cp $< $@ -- To view, visit https://review.coreboot.org/c/coreboot/+/32185 To unsubscribe, or for help writing mail filters, visit https://review.coreboot.org/settings Gerrit-Project: coreboot Gerrit-Branch: master Gerrit-Change-Id: Idc37c50cae90d283bd6ec240dd7de3546485a9a9 Gerrit-Change-Number: 32185 Gerrit-PatchSet: 1 Gerrit-Owner: Patrick Georgi <pgeorgi(a)google.com> Gerrit-MessageType: newchange

2 4

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

coreboot-gerrit November 2021