Patrick Georgi has uploaded this change for review. ( https://review.coreboot.org/c/coreboot/+/62666 )
Change subject: Documentation: Describe our Coverity Scan integration ......................................................................
Documentation: Describe our Coverity Scan integration
Change-Id: I0a2b68a4b4b54c7345280b252d624799316641b1 Signed-off-by: Patrick Georgi patrick@coreboot.org --- M Documentation/infrastructure/builders.md A Documentation/infrastructure/coverity.md M Documentation/infrastructure/index.md 3 files changed, 86 insertions(+), 1 deletion(-)
git pull ssh://review.coreboot.org:29418/coreboot refs/changes/66/62666/1
diff --git a/Documentation/infrastructure/builders.md b/Documentation/infrastructure/builders.md index 15c3ab9..80bcd23 100644 --- a/Documentation/infrastructure/builders.md +++ b/Documentation/infrastructure/builders.md @@ -61,7 +61,7 @@ There are a number of builds handled by the coreboot jenkins builders, for a number of different projects - coreboot, flashrom, memtest86+, em100, etc. Many of these have builders for their current master branch -as well as gerrit and coverity builds. +as well as gerrit and [coverity](coverity.md) builds.
You can see all the builds here: [https://qa.coreboot.org/%5D(https://qa.coreboot.org/) diff --git a/Documentation/infrastructure/coverity.md b/Documentation/infrastructure/coverity.md new file mode 100644 index 0000000..16e3a82 --- /dev/null +++ b/Documentation/infrastructure/coverity.md @@ -0,0 +1,84 @@ +# Coverity Scan for open source firmware + +## What’s Coverity and Coverity Scan? + +Coverity is a static analysis tool. It hooks into the build process +and in addition to the compiler creating object files, Coverity collects +information about the code. That data is then processed in a separate pass +to identify common programming errors, like out of bounds accesses in C. + +Coverity Scan is an online service for Open Source projects +providing this analysis for free. The analysis pass is done +on their servers and issues can be handled in their web UI at +[https://scan.coverity.com/%5D(https://scan.coverity.com/). + +The Scan service has some quotas based on code size to avoid overloading +the system, but even at one build per week, that’s usually good enough +because the identified issues still need to be triaged and fixed or they +will simply be re-identified next week. + +### Triage? + +The Web UI looks a bit like an issue tracker, even if it’s not a very +good one. It’s possible to mark identified issues as valid or invalid, +and annotate them with metadata which CLs fix them. The latter isn’t +strictly necessary because Coverity Scan simply marks issues it can’t +find anymore as fixed, but at times it helped identify issues that made +a comeback. + +### Alternatives + +There’s also clang’s scan-build, which is fully open-source, and +finds different issues. As such it’s less of an alternative and more +of a complement. + +There’s a regular run of that for coreboot but not for the other projects. + +One downside is that it emits a bunch of html to report on issues, +but there’s no interactivity (e.g. marking issues solved), no way +to merge multiple builds (e.g. multiple board builds of a single tree) +or a simple way to extract burndown charts and the like from that. + +#### Looking for a project? + +On the upside, it can emit the data in a machine readable format, so if +anybody needs a project, a scan-build web-frontend like Coverity Scan +would be feasible without having to go through scan-build’s guts, +just by parsing text files (plus all the stateful and web rest) + +## Logging into Coverity Scan + +Coverity Scan needs an account. It supports its own accounts and GitHub +OAuth. + +Access to the dashboards needs approval: Request and you shall receive. + +## coreboot & friends and Coverity Scan + +coreboot, flashrom, Chromium EC and other projects of that family have +been made Coverity aware, that is, their build systems support building +with a custom compiler configuration passed in “just right” to enable +Coverity to add its hooks. + +The public coreboot CI system at +[https://qa.coreboot.org/%5D(https://qa.coreboot.org/) regularly does +builds with Coverity and sends them off to Coverity Scan. + +Specifically, it covers: + +* Chromium EC: [Coverity Scan site](https://scan.coverity.com/projects/chromium-ec?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/ChromeEC-Coverity/)) +* coreboot: [Coverity Scan site](https://scan.coverity.com/projects/coreboot?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/coreboot-coverity/)), [scan-build output](https://www.coreboot.org/scan-build/) ([build job](https://qa.coreboot.org/job/coreboot_scanbuild/)) +* em100: [Coverity Scan site](https://scan.coverity.com/projects/em100?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/em100-coverity/)) +* fcode-utils: [Coverity Scan site](https://scan.coverity.com/projects/fcode-utils?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/fcode-utils-coverity/)) +* flashrom: [Coverity Scan site](https://scan.coverity.com/projects/flashrom?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/flashrom-coverity/)) +* memtest86+: [Coverity Scan site](https://scan.coverity.com/projects/memtest86?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/memtest86plus-coverity/)) +* vboot: [Coverity Scan site](https://scan.coverity.com/projects/vboot?tab=overview) ([build job](https://qa.coreboot.org/view/coverity/job/vboot-coverity/)) + +Some projects (e.g. Chromium EC) build a different subset of boards on +each run, ensuring that everything is analyzed eventually. The downside +is that coverity issues pop up and disappear somewhat randomly as they +are discovered and go unnoticed in a later build. + +More projects that are hosted on review.coreboot.org (potentially as a +mirror, like vboot and EC) could be served through that pipeline. Reach +out to {stepan,patrick,martin}@coreboot.org. diff --git a/Documentation/infrastructure/index.md b/Documentation/infrastructure/index.md index a452013..3007bbe 100644 --- a/Documentation/infrastructure/index.md +++ b/Documentation/infrastructure/index.md @@ -4,3 +4,4 @@
## Jenkins builders and builds * [Setting up Jenkins build machines](builders.md) +* [Coverity Scan integration](coverity.md)