Name of user not set #1002873 has posted comments on this change. ( https://review.coreboot.org/c/coreboot/+/39893 )
Change subject: Documentation: Add proposal for firmware unit testing ......................................................................
Patch Set 1:
(9 comments)
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... File Documentation/technotes/2020-03-unit-testing-coreboot.md:
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 67: Requirements
Are any of the frameworks small enough to build into a linuxboot payload so we can run unit tests as […]
In general, I don't like the idea of running unit tests on the actual device. I expect unit test to be _very_ quick, so that I can run it a lot of times during development cycle - with building and flashing on the target, this cannot be met easily. For running on actual devices, integration testing makes more sense than unit testing. So answering your second question - I don't see this as a reasonable option, even if we can use the same compiler. That being said, Cmocka is a lightweight one, SputUnit also. Could you give me more info what is the "acceptable" size for linuxboot payload - how big unit test library can be at max?
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 79: JUnit-like XML output format for Jenkins
Right, Test Anything Protocol would work as well (worst case we'd need to put in some converter). […]
Yes, I think that any machine parsable output should work. Actually there are some examples of open-source converting scripts for some frameworks. Probably I should add a comment to this requirement explaining this. BTW, out of 4 candidates below, Cmocka & GoogleTest has support for generating TAP.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 84: however this is very unlikely
Maybe expand on this statement? […]
Sounds good.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 100: GPL
Considering that cmocka (Apache) seems to be the candidate, how do we feel about the compatibility i […]
Patrick, Correct.
Julius, My understanding of the license is that unless you are not distributing binaries, you should be OK. I'm looking at this more as a "private use" category. That being said, I'm not a lawyer - do we have a contact to someone with expertise in FOSS? Talking about example - OpenVPN (https://github.com/OpenVPN/openvpn) is GPLv2, while it is using CMocka for unit testing.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 105: for obvious reasons
"because that would take an excessive amount of time"? Not sure we should talk about "obvious" in do […]
I agree.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 176: Pros
please take care of trailing whitespace
Sure. Actually checkpatch found some whitespaces, when I was creating a commit (and fixed them), wonder how this slipped away.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 195: In the same time
At the same time (also a couple of times elsewhere in the doc), see e.g. https://www.italki. […]
Sure.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 206: form
from
Sure.
https://review.coreboot.org/c/coreboot/+/39893/1/Documentation/technotes/202... PS1, Line 232: prefix
suffix
Tim, Will fix.
Julius, I don't agree. I would like to avoid confusion between the two files - source.c and test.c - considering that they both have very similar path (src/* -> tests/*). Usually these files are being modified at the same time, I see a possibility of mistake here. Furthermore - if we want to create a dependency and object files for both test and source modules under build/tests/, it is necessary to have a different name. Lastly, other open source projects incorporating unit testing seem to follow that approach (however they sometimes use prefix instead of suffix as in my proposal).