Hi Carl-Daniel,
On Tue, Nov 23, 2010 at 11:50 AM, Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Agreed. Here is an interesting question: Given that the initial backup and the final restore are done with the system flashrom binary, shouldn't we try to restore the backup directly after making the backup? If that fails, we can't restore the image at the end, and we're better off detecting this problem before we mess with the chip.
Good point! I think there's no big point in restoring it right away, because if that fails we have an inconsistent state anyways, so we won't mess much up when running the tests. But we could try to restore it again with the flashrom under test binary if a restore with the system binary fails. It might be that the system binary is old enough and the tested binary is at least able to get the flash part back into a non-bricked state (even if some of the tests might fail)
In this case we should warn the user that each of the tests has the potential to brick his device.
Absolutely! Maybe the do_test script should ask for manual confirmation if not run with a --non-interactive parameter.
AFAICS the partial write torture test can/will result in a hanging EC if the EC code lives in the bottom 128 kB of the flash chip.
Is there some way we can detect this? Or have a black list with devices that have this issue?
arguments. The invocation: FLASHROM="../flashrom" FLASHROM_PARAM="-p internal:bus=spi" sh do_tests.sh partial_writes_x86_bios.sh chip_size.sh wp-toggle.sh wp-range.sh
Why the "sh" after FLASHROM_PARAM="..."?
Assuming this is because util/ or . is not in the users path, I guess ./do_tests.sh would do the job, too.
The big question for the size and protection tests is what you actually intend to test and whether the test will work with any chips besides the one you wrote the tests for. This is not a complaint directed at the RFC, it is rather a hint that writing good tests is hard (but you already know that).
The test should _work_ with any flash chip unless there is a bug in flashrom. That does not necessarily imply that it tests something useful though.
Overall, I think that more tests are good.
Do you have suggestions what kinds of tests would be needed?
Does it make sense to add instrumentation (code coverage) support so we can figure out the test depth? gcc in user space makes this kind of trivial.