[RFC] v3 dtc testsuite

[Carl-Daniel Hailfinger]

Hi,

the v3 dtc code has been an area where we had bugs in the past, and I
believe we might hit some old bugs and/or introduce new bugs in the
future. I'd like to spot bugs as early as possible during compile time,
not as random corruption/crash/misbehaviour during runtime.

I have a simple testcase dts on my disk which caused the generated
statictree.c to explode during compilation. That dtc bug was fixed by
the struct naming patch in r860.

There also is a more complicated torture dts for which statictree.c will
cause runtime NULL dereferences. Checking this at compile time would
require the build system to compile statictree.c and a verification
walker with HOSTCC before including it in the build.

Should we introduce compile-time checks for any possible bugs,
especially those we already hit in the past? What do you think?

Regards,
Carl-Daniel

[ron minnich]

would be interesting to have a dts regression test. Build it up bit by
bit. Compile dts into a .c, then compile all dtc into a big program,
then look for errors.

ron

[Stefan Reinauer]

Carl-Daniel Hailfinger wrote:
> Hi,
>
> the v3 dtc code has been an area where we had bugs in the past, and I
> believe we might hit some old bugs and/or introduce new bugs in the
> future. I'd like to spot bugs as early as possible during compile time,
> not as random corruption/crash/misbehaviour during runtime.
>
> I have a simple testcase dts on my disk which caused the generated
> statictree.c to explode during compilation. That dtc bug was fixed by
> the struct naming patch in r860.
>
> There also is a more complicated torture dts for which statictree.c will
> cause runtime NULL dereferences. Checking this at compile time would
> require the build system to compile statictree.c and a verification
> walker with HOSTCC before including it in the build.
>
> Should we introduce compile-time checks for any possible bugs,
> especially those we already hit in the past? What do you think?

I suggest not doing this on every compile but instead having a test
suite that people should run when they make changes to dtc.

This test suite should also be verified against code coverage to make
sure the test suite tests the complete program. We did this for the
OpenBIOS FCode Suite:

http://www.openfirmware.info/data/coverage-fcode-suite-1.0.2/