Julius Werner has posted comments on this change. ( https://review.coreboot.org/c/coreboot/+/39893 )
Change subject: Documentation: Add proposal for firmware unit testing ......................................................................
Patch Set 1:
(2 comments)
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... File Documentation/technotes/2020-03-unit-testing-coreboot.md:
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 67: Requirements
Are any of the frameworks small enough to build into a linuxboot payload so we can run unit tests as […]
I feel like that would sort of defeat the purpose of unit tests, which is to be something that anyone can quickly and easily run from their local development machine. It would also preclude us from running them in the CI (which I think we definitely want).
I think we should use the crossgcc x86_64-pc-linux-gnu toolchain to build these (at least as an option, and that should be what the CI uses), then we hopefully cover most of the compiler-specific details that an actual firmware build would also have. Most unit tests (at least at first) could probably be architecture-independent (even for src/arch/ code, unless it specifically uses inline assembly or stuff). If we ever really want to test all architectures (probably a low priority thing compared to getting test coverage across the code base first), we'd probably want to do something with QEMU (like the Chrome OS SDK does).
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 190: Cmocka also include support : for mocks. One thing to investigate would be how we can mock out headers and inline functions. Cmocka has nice mocking support using that __wrap_ linker feature, but unfortunately it only works for actual link-level functions. Many places in coreboot use inlines (and for important reasons they have to be inlined, and would maybe not even work right if you forced them to be not inline).
For example, say I wanted to unit test src/arch/arm64/armv8/mmu.c -- it would be great to have a test that builds some page tables and then compares the output with hardcoded values to ensure they were built correctly. Unfortunately the code calls things like raw_write_ttbr0_el3() or tlbiall_el3() all over the place, which are inline functions inserting a specific inline assembly instruction. Those instructions won't work when building this for an x86 host, and I also want to confirm that they were called with the expected parameters, so I need to be able to mock them out somehow.
We would probably want to have an '-I test/include' before the normal coreboot include path when building for tests so we can mock those headers out. Since the same inline functions are used across many different files we'd want to test, we'll probably want to have a common library of header mocks that can be used by multiple tests somehow.