Martin Roth has uploaded this change for review. ( https://review.coreboot.org/c/coreboot/+/43984 )
Change subject: Documentation: Add documentation on jenkins builders ......................................................................
Documentation: Add documentation on jenkins builders
Signed-off-by: Martin Roth martin@coreboot.org Change-Id: I54a0204e7525a25f2fd717a73007b304aac67396 --- A Documentation/community/builders.md 1 file changed, 391 insertions(+), 0 deletions(-)
git pull ssh://review.coreboot.org:29418/coreboot refs/changes/84/43984/1
diff --git a/Documentation/community/builders.md b/Documentation/community/builders.md new file mode 100644 index 0000000..bc35bcc --- /dev/null +++ b/Documentation/community/builders.md @@ -0,0 +1,391 @@ +# Jenkins builder setup and configuration + +## How to set up a new jenkins builder + +### Contact a jenkins admin + +Let a jenkins admin know that you’re interested in setting up a jenkins +build system. + +For a permanent build system, this should generally be a dedicated +machine that is not generally being used for other purposes. The +coreboot builds are very intensive. + +It's also best to be aware that although we don't know of any security +issues, the jenkins-node image is run with the privileged flag which +gives the container root access to the build machine. See +[this article](https://blog.trendmicro.com/trendlabs-security-intelligence/why-running-a-pr...) +about why this is discouraged. + +It's recommended that you give an admin root access on your machine so +that they can reset it in case of a failure. This is not a requirement, +as the system can just be disabled until someone is available to fix any +issues. + +Currently active Jenkins admins: +* Patrick Georgi: + * Email: [patrick@georgi-clan.de](mailto:patrick@georgi-clan.de) + * IRC: pgeorgi + + +### Build Machine requirements + +For a builder, we need a fast system with lots of threads and plenty of +RAM. The builder builds and stores the git repos and output in tmpfs +along with the ccache save area, so if there isn't enough memory, the +builds will slow down because of smaller ccache areas and can run into +"out of storage space" errors. + +#### Current Build Machines + +To give an idea of what a suitable build machine might be, currently the +coreboot project has 3 active jenkins build machines. + +* Congenialbuilder - 128 threads, 256GiB RAM + * Fastest Passing coreboot gerrit build: 4 min, 30 sec + * Slowest Passing coreboot gerrit build: 9 min, 56 sec + + +* Gleeful builder - 64 thread, 64GiB RAM + * Fastest Passing coreboot gerrit build: 6 min, 6 sec + * Slowest Passing coreboot gerrit build, 34 min + + +* Ultron (9elements) - 48 threads, 128GiB RAM + * Fastest Passing coreboot gerrit build: 6 min, 32 sec + * Slowest Passing coreboot gerrit build: 44 min + + +### Jenkins Builds + +There are a number of builds handled by the coreboot jenkins builders, +for a number of different projects - coreboot, flashrom, memtest86+, +em100, etc. Many of these have builders for their current master branch +as well as gerrit and coverity builds. + +You can see all the builds here: +[https://qa.coreboot.org/%5D(https://qa.coreboot.org/) + +Most of the time on the builders is taken up by the coreboot master and +gerrit builds. + +* [coreboot gerrit build](https://qa.coreboot.org/job/coreboot-gerrit/) +([Time trend](https://qa.coreboot.org/job/coreboot-gerrit/buildTimeTrend)) + + +* [coreboot master build](https://qa.coreboot.org/job/coreboot/) + ([Time trend](https://qa.coreboot.org/job/coreboot/buildTimeTrend)) + + +### Stress test the machine + +Test the machine to make sure that building won't stress the hardware +too much. Install stress-ng, then run the stress test for at least an +hour. + +On a system with 32 cores, it was tested with this command: + +``` +$ stress-ng --cpu 20 --io 6 --vm 6 --vm-bytes 1G --verify --metrics-brief -t 60m +``` + +You can watch the temperature with the sensors package or with ‘acpi -t’ +if your machine supports that. + +You can check for thermal throttling by running this command and seeing +if the values go down on any of the cores after it's been running for a +while. + +``` +$ while [ true ]; do clear; cat /proc/cpuinfo | grep 'cpu MHz' ; sleep 1; done +``` + +If the machine throttles or resets, you probably need to upgrade the +cooling system. + + +## jenkins-server docker installation + + +### Manual Installation + +If you’ve met all the above requirements, and an admin has agreed to set +up the builder in jenkins, you’re ready to go on to the next steps. + + +### Set up your network so jenkins can talk to the container + +Expose a local port through any firewalls you might have on your router. +This would generally be in the port forwarding section, and you'd just +forward a port (typically 49151) from the internet directly to the +builder’s IP address. + +You might also want to set up a port to forward to port 22 on your +machine and set up openssh so you can manage the machine remotely. + + +### Install and set up docker + +Install docker by following the directions on the docker site. These +instructions keep changing, so just check the latest information. + +[https://docs.docker.com/engine/install/%5D(https://docs.docker.com/engine/in...) + + +#### Set up environment variables + +To make configuration and the later commands easier, these should go in +your shell's .rc file. Note that you only need to set them if you're +using something other than the default. + +``` +# Set the port used on your machine to connect to jenkins. +export COREBOOT_JENKINS_PORT=49151 + +# Set the revision of the container from docker hub +export DOCKER_COMMIT=65718760fa + +# Set the location of where the jenkins cache directory will be. +export COREBOOT_JENKINS_CACHE_DIR="/srv/docker/coreboot-builder/cache" + +# Set the name of the container +export COREBOOT_JENKINS_CONTAINER="coreboot_jenkins" +``` + +Make sure any variables needed are set in your environment before +continuing to the next step. + + +### Using the Makefile for docker installation + +From the coreboot directory, run + +``` +make -C util/docker help +``` + +This will show you the available targets and variables needed: + +``` +Commands for working with docker images: + coreboot-sdk - Build coreboot-sdk container + upload-coreboot-sdk - Upload coreboot-sdk to hub.docker.com + coreboot-jenkins-node - Build coreboot-jenkins-node container + upload-coreboot-jenkins-node - Upload coreboot-jenkins-node to hub.docker.com + doc.coreboot.org - Build doc.coreboot.org container + clean-coreboot-containers - Remove all docker coreboot containers + clean-coreboot-images - Remove all docker coreboot images + docker-clean - Remove docker coreboot containers & images + +Commands for using docker images + docker-build-coreboot - Build coreboot under coreboot-sdk + <BUILD_CMD=target> + docker-abuild - Run abuild under coreboot-sdk + <ABUILD_ARGS='-a -B'> + docker-what-jenkins-does - Run 'what-jenkins-does' target + docker-shell - Bash prompt in coreboot-jenkins-node + <USER=root or USER=coreboot> + docker-jenkins-server - Run coreboot-jenkins-node image (for server) + docker-jenkins-attach - Open shell in running jenkins server + docker-build-docs - Build the documentation + docker-livehtml-docs - Run sphinx-autobuild + +Variables: + COREBOOT_JENKINS_PORT=49151 + COREBOOT_JENKINS_CACHE_DIR=/srv/docker/coreboot-builder/cache + COREBOOT_JENKINS_CONTAINER=coreboot_jenkins + COREBOOT_IMAGE_TAG=f2741aa632f + DOCKER_COMMIT=65718760fa +``` + +### Set up the system for the jenkins builder + +As a regular user - *Not root*, run: + +``` +sudo mkdir -p ${COREBOOT_JENKINS_CACHE_DIR} +sudo mkdir -p ${COREBOOT_JENKINS_CCACHE_DIR} +sudo chown $(whoami):$(whoami) ${COREBOOT_JENKINS_CCACHE_DIR} +sudo chown $(whoami):$(whoami) ${COREBOOT_JENKINS_CACHE_DIR} +wget http://www.dediprog.com/save/78.rar/to/EM100Pro.rar +mv EM100Pro.rar ${COREBOOT_JENKINS_CACHE_DIR} +``` + +### Install the coreboot jenkins builder + +``` +make -C util/docker docker-jenkins-server +``` + +Your installation is complete on your side. + +### Tell the Admins that the machine is set up +Let the admins know that the builder is set up so they can set up the +machine profile on qa.coreboot.org. + +They need to know: +* Your external IP address or domain name. If you don’t have a static +IP, make sure you have a dynamic dns hostname configured. +* The port on your machine and firewall that’s exposed for jenkins: +`$COREBOOT_JENKINS_PORT` +* The core count of the machine. +* How much memory is available on the machine. This helps determine +the amount of memory used for ccache. + + +### First build +On the first build after a machine is reset, it will frequently take +20-25 minutes to do the entire what-jenkins-does build while the ccache +is getting filled up and the entire coreboot repo gets downloaded. As +the ccache gets populated, the build time will drop. + + +## Additional Information + + +### How to log in to the docker instance for debugging +``` + $ make -C util/docker docker-jenkins-attach + $ su coreboot + $ cd ~/slave-root/workspace + $ bash +``` + + +WARNING: This should not be used to make changes to the build system, +but just to debug issues. Changes to the build system are highly +discouraged as it leads to situations where patches can pass the build +testing on one builder and fail on another builder. Any changes that are +made in the image will be lost on the next update, so if you +accidentally change something, you can remove the containers and images +and update to get a fresh installation. + + +### How to download containers/images for a fresh installation and remove old containers + +To delete the old containers & images: + +``` +$ docker stop $COREBOOT_JENKINS_CONTAINER +$ docker rm $COREBOOT_JENKINS_CONTAINER +$ docker images # lists all existing images +$ docker rmi XXXX # Use the image ID found in the above command. +``` + +To get and run the new coreboot-jenkins image, change the value in the +`DOCKER_COMMIT` variable to the new image value. + +``` +$ make -C util/docker docker-jenkins-server +``` + +#### Getting ready to push the docker images + +Set up an account on hub.docker.com + +Get an admin to add the account to the coreboot team on hub.docker.com + +[https://hub.docker.com/u/coreboot/dashboard/teams/?team=owners%5D(https://hu...) + +Make sure your credentials are configured on your host machine by +running + +``` +$ docker login +``` + +This will prompt you for your docker username, password, and your email +address, and write out to ~/.docker/config.json. Without this file, you +won’t be able to push the images. + +#### Updating the Dockerfiles: + +The coreboot-sdk Dockerfile will need to be updated when any additional +dependencies are added. Both the coreboot-sdk and the +coreboot-jenkins-node Dockerfiles will need to be updated to the new +version number and git commit id anytime the toolchain is updated. Both +files are stored in the coreboot repo under coreboot/util/docker. + +Read the [dockerfile best practices](https://docs.docker.com/v1.8/articles/dockerfile_best-practices/) +page before updating the files. + +#### Rebuilding the coreboot-sdk docker image to update the toolchain: + +``` +$ make -C util/docker coreboot-sdk +``` + +This takes a relatively long time. + +#### Test the coreboot-sdk docker image: + +There are two methods of running the docker image - interactively as a +shell, or doing the build directly. Running interactively as a shell is +useful for early testing, because it allows you to update the image +(without any changes getting saved) and re-test builds. This saves the +time of having to rebuild the image for every issue you find. + +#### Running the docker image interactively: + +Run: + +``` +$ make -C util/docker docker-jenkins-server +$ make -C util/docker docker-jenkins-attach +``` + +#### Running the build directly: + +From the coreboot directory: + +``` +$ make -C util/docker docker-build-coreboot +``` + +You’ll also want to test building the other projects and payloads: +ChromeEC, flashrom, memtest86+, em100, Grub2, SeaBIOS, iPXE, coreinfo, +nvramcui, tint... + +#### Pushing the coreboot-sdk image to hub.docker.com for use: + +When you’re satisfied with the testing, push the coreboot-sdk image to +the hub.docker.com + +``` +$ make -C util/docker upload-coreboot-sdk +``` + +#### Building and pushing the coreboot-jenkins-node docker image: + +This docker image is pretty simple, so there’s not really any testing +that needs to be done. + +``` +$ make -C util/docker coreboot-jenkins-node +$ make -C util/docker upload-coreboot-jenkins-node +``` + +### Coverity Setup + +To run coverity jobs, the builder needs to have the tools available, and +to be marked as a coverity builder. + + +#### Set up the Coverity tools + +Download the Linux-64 coverity build tool and decompress it into your +cache directory as defined by the `$COREBOOT_JENKINS_CACHE_DIR` variable + +[https://scan.coverity.com/download%5D(https://scan.coverity.com/download) + +Rename the directory from its original name +(cov-analysis-linux64-7.7.0.4) to ‘coverity’, or better, create a +symlink: + +``` +ln -s cov-analysis-linux64-7.7.0.4 coverity +``` + + +Let the admins know that the ‘coverity’ label can be added to the +builder.
Patrick Georgi has posted comments on this change. ( https://review.coreboot.org/c/coreboot/+/43984 )
Change subject: Documentation: Add documentation on jenkins builders ......................................................................
Patch Set 1: Code-Review+1
(3 comments)
pretty good doc, thanks. have some nits, and if there's anything to improve we can do that in a follow-up.
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... File Documentation/community/builders.md:
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... PS1, Line 121: whitespace
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... PS1, Line 124: you you (and the jenkins admins if you allow them)?
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... PS1, Line 129: directions put the link here?
Martin Roth has posted comments on this change. ( https://review.coreboot.org/c/coreboot/+/43984 )
Change subject: Documentation: Add documentation on jenkins builders ......................................................................
Patch Set 1:
(3 comments)
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... File Documentation/community/builders.md:
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... PS1, Line 121:
whitespace
Done
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... PS1, Line 124: you
you (and the jenkins admins if you allow them)?
Done
https://review.coreboot.org/c/coreboot/+/43984/1/Documentation/community/bui... PS1, Line 129: directions
put the link here?
Done
Hello build bot (Jenkins), Patrick Georgi,
I'd like you to reexamine a change. Please visit
https://review.coreboot.org/c/coreboot/+/43984
to look at the new patch set (#2).
Change subject: Documentation: Add documentation on jenkins builders ......................................................................
Documentation: Add documentation on jenkins builders
Put this in a new directory called 'infrastructure' and make a link and an index.md file for the directory.
Signed-off-by: Martin Roth martin@coreboot.org Change-Id: I54a0204e7525a25f2fd717a73007b304aac67396 --- M Documentation/index.md A Documentation/infrastructure/builders.md A Documentation/infrastructure/index.md 3 files changed, 399 insertions(+), 0 deletions(-)
git pull ssh://review.coreboot.org:29418/coreboot refs/changes/84/43984/2
Attention is currently required from: Martin Roth. Angel Pons has posted comments on this change. ( https://review.coreboot.org/c/coreboot/+/43984 )
Change subject: Documentation: Add documentation on jenkins builders ......................................................................
Patch Set 2: Code-Review+2
Patrick Georgi has submitted this change. ( https://review.coreboot.org/c/coreboot/+/43984 )
Change subject: Documentation: Add documentation on jenkins builders ......................................................................
Documentation: Add documentation on jenkins builders
Put this in a new directory called 'infrastructure' and make a link and an index.md file for the directory.
Signed-off-by: Martin Roth martin@coreboot.org Change-Id: I54a0204e7525a25f2fd717a73007b304aac67396 Reviewed-on: https://review.coreboot.org/c/coreboot/+/43984 Tested-by: build bot (Jenkins) no-reply@coreboot.org Reviewed-by: Angel Pons th3fanbus@gmail.com --- M Documentation/index.md A Documentation/infrastructure/builders.md A Documentation/infrastructure/index.md 3 files changed, 399 insertions(+), 0 deletions(-)
Approvals: build bot (Jenkins): Verified Angel Pons: Looks good to me, approved
diff --git a/Documentation/index.md b/Documentation/index.md index fd1ecb1..82ff12d 100644 --- a/Documentation/index.md +++ b/Documentation/index.md @@ -187,5 +187,6 @@ * [SuperIO](superio/index.md) * [Vendorcode](vendorcode/index.md) * [Utilities](util.md) +* [coreboot infrastructure](infrastructure/index.md) * [Release notes for past releases](releases/index.md) * [Flashing firmware tutorial](flash_tutorial/index.md) diff --git a/Documentation/infrastructure/builders.md b/Documentation/infrastructure/builders.md new file mode 100644 index 0000000..15c3ab9 --- /dev/null +++ b/Documentation/infrastructure/builders.md @@ -0,0 +1,392 @@ +# Jenkins builder setup and configuration + +## How to set up a new jenkins builder + +### Contact a jenkins admin + +Let a jenkins admin know that you’re interested in setting up a jenkins +build system. + +For a permanent build system, this should generally be a dedicated +machine that is not generally being used for other purposes. The +coreboot builds are very intensive. + +It's also best to be aware that although we don't know of any security +issues, the jenkins-node image is run with the privileged flag which +gives the container root access to the build machine. See +[this article](https://blog.trendmicro.com/trendlabs-security-intelligence/why-running-a-pr...) +about why this is discouraged. + +It's recommended that you give an admin root access on your machine so +that they can reset it in case of a failure. This is not a requirement, +as the system can just be disabled until someone is available to fix any +issues. + +Currently active Jenkins admins: +* Patrick Georgi: + * Email: [patrick@georgi-clan.de](mailto:patrick@georgi-clan.de) + * IRC: pgeorgi + + +### Build Machine requirements + +For a builder, we need a fast system with lots of threads and plenty of +RAM. The builder builds and stores the git repos and output in tmpfs +along with the ccache save area, so if there isn't enough memory, the +builds will slow down because of smaller ccache areas and can run into +"out of storage space" errors. + +#### Current Build Machines + +To give an idea of what a suitable build machine might be, currently the +coreboot project has 3 active jenkins build machines. + +* Congenialbuilder - 128 threads, 256GiB RAM + * Fastest Passing coreboot gerrit build: 4 min, 30 sec + * Slowest Passing coreboot gerrit build: 9 min, 56 sec + + +* Gleeful builder - 64 thread, 64GiB RAM + * Fastest Passing coreboot gerrit build: 6 min, 6 sec + * Slowest Passing coreboot gerrit build, 34 min + + +* Ultron (9elements) - 48 threads, 128GiB RAM + * Fastest Passing coreboot gerrit build: 6 min, 32 sec + * Slowest Passing coreboot gerrit build: 44 min + + +### Jenkins Builds + +There are a number of builds handled by the coreboot jenkins builders, +for a number of different projects - coreboot, flashrom, memtest86+, +em100, etc. Many of these have builders for their current master branch +as well as gerrit and coverity builds. + +You can see all the builds here: +[https://qa.coreboot.org/%5D(https://qa.coreboot.org/) + +Most of the time on the builders is taken up by the coreboot master and +gerrit builds. + +* [coreboot gerrit build](https://qa.coreboot.org/job/coreboot-gerrit/) +([Time trend](https://qa.coreboot.org/job/coreboot-gerrit/buildTimeTrend)) + + +* [coreboot master build](https://qa.coreboot.org/job/coreboot/) + ([Time trend](https://qa.coreboot.org/job/coreboot/buildTimeTrend)) + + +### Stress test the machine + +Test the machine to make sure that building won't stress the hardware +too much. Install stress-ng, then run the stress test for at least an +hour. + +On a system with 32 cores, it was tested with this command: + +``` +$ stress-ng --cpu 20 --io 6 --vm 6 --vm-bytes 1G --verify --metrics-brief -t 60m +``` + +You can watch the temperature with the sensors package or with ‘acpi -t’ +if your machine supports that. + +You can check for thermal throttling by running this command and seeing +if the values go down on any of the cores after it's been running for a +while. + +``` +$ while [ true ]; do clear; cat /proc/cpuinfo | grep 'cpu MHz' ; sleep 1; done +``` + +If the machine throttles or resets, you probably need to upgrade the +cooling system. + + +## jenkins-server docker installation + + +### Manual Installation + +If you’ve met all the above requirements, and an admin has agreed to set +up the builder in jenkins, you’re ready to go on to the next steps. + + +### Set up your network so jenkins can talk to the container + +Expose a local port through any firewalls you might have on your router. +This would generally be in the port forwarding section, and you'd just +forward a port (typically 49151) from the internet directly to the +builder’s IP address. + +You might also want to set up a port to forward to port 22 on your +machine and set up openssh so you or the jenkins admins can manage +the machine remotely (if you allow them). + + +### Install and set up docker + +Install docker by following the +[directions](https://docs.docker.com/engine/install/) on the docker +site. These instructions keep changing, so just check the latest +information. + + +#### Set up environment variables + +To make configuration and the later commands easier, these should go in +your shell's .rc file. Note that you only need to set them if you're +using something other than the default. + +``` +# Set the port used on your machine to connect to jenkins. +export COREBOOT_JENKINS_PORT=49151 + +# Set the revision of the container from docker hub +export DOCKER_COMMIT=65718760fa + +# Set the location of where the jenkins cache directory will be. +export COREBOOT_JENKINS_CACHE_DIR="/srv/docker/coreboot-builder/cache" + +# Set the name of the container +export COREBOOT_JENKINS_CONTAINER="coreboot_jenkins" +``` + +Make sure any variables needed are set in your environment before +continuing to the next step. + + +### Using the Makefile for docker installation + +From the coreboot directory, run + +``` +make -C util/docker help +``` + +This will show you the available targets and variables needed: + +``` +Commands for working with docker images: + coreboot-sdk - Build coreboot-sdk container + upload-coreboot-sdk - Upload coreboot-sdk to hub.docker.com + coreboot-jenkins-node - Build coreboot-jenkins-node container + upload-coreboot-jenkins-node - Upload coreboot-jenkins-node to hub.docker.com + doc.coreboot.org - Build doc.coreboot.org container + clean-coreboot-containers - Remove all docker coreboot containers + clean-coreboot-images - Remove all docker coreboot images + docker-clean - Remove docker coreboot containers & images + +Commands for using docker images + docker-build-coreboot - Build coreboot under coreboot-sdk + <BUILD_CMD=target> + docker-abuild - Run abuild under coreboot-sdk + <ABUILD_ARGS='-a -B'> + docker-what-jenkins-does - Run 'what-jenkins-does' target + docker-shell - Bash prompt in coreboot-jenkins-node + <USER=root or USER=coreboot> + docker-jenkins-server - Run coreboot-jenkins-node image (for server) + docker-jenkins-attach - Open shell in running jenkins server + docker-build-docs - Build the documentation + docker-livehtml-docs - Run sphinx-autobuild + +Variables: + COREBOOT_JENKINS_PORT=49151 + COREBOOT_JENKINS_CACHE_DIR=/srv/docker/coreboot-builder/cache + COREBOOT_JENKINS_CONTAINER=coreboot_jenkins + COREBOOT_IMAGE_TAG=f2741aa632f + DOCKER_COMMIT=65718760fa +``` + +### Set up the system for the jenkins builder + +As a regular user - *Not root*, run: + +``` +sudo mkdir -p ${COREBOOT_JENKINS_CACHE_DIR} +sudo mkdir -p ${COREBOOT_JENKINS_CCACHE_DIR} +sudo chown $(whoami):$(whoami) ${COREBOOT_JENKINS_CCACHE_DIR} +sudo chown $(whoami):$(whoami) ${COREBOOT_JENKINS_CACHE_DIR} +wget http://www.dediprog.com/save/78.rar/to/EM100Pro.rar +mv EM100Pro.rar ${COREBOOT_JENKINS_CACHE_DIR} +``` + +### Install the coreboot jenkins builder + +``` +make -C util/docker docker-jenkins-server +``` + +Your installation is complete on your side. + +### Tell the Admins that the machine is set up +Let the admins know that the builder is set up so they can set up the +machine profile on qa.coreboot.org. + +They need to know: +* Your external IP address or domain name. If you don’t have a static +IP, make sure you have a dynamic dns hostname configured. +* The port on your machine and firewall that’s exposed for jenkins: +`$COREBOOT_JENKINS_PORT` +* The core count of the machine. +* How much memory is available on the machine. This helps determine +the amount of memory used for ccache. + + +### First build +On the first build after a machine is reset, it will frequently take +20-25 minutes to do the entire what-jenkins-does build while the ccache +is getting filled up and the entire coreboot repo gets downloaded. As +the ccache gets populated, the build time will drop. + + +## Additional Information + + +### How to log in to the docker instance for debugging +``` + $ make -C util/docker docker-jenkins-attach + $ su coreboot + $ cd ~/slave-root/workspace + $ bash +``` + + +WARNING: This should not be used to make changes to the build system, +but just to debug issues. Changes to the build system are highly +discouraged as it leads to situations where patches can pass the build +testing on one builder and fail on another builder. Any changes that are +made in the image will be lost on the next update, so if you +accidentally change something, you can remove the containers and images +and update to get a fresh installation. + + +### How to download containers/images for a fresh installation and remove old containers + +To delete the old containers & images: + +``` +$ docker stop $COREBOOT_JENKINS_CONTAINER +$ docker rm $COREBOOT_JENKINS_CONTAINER +$ docker images # lists all existing images +$ docker rmi XXXX # Use the image ID found in the above command. +``` + +To get and run the new coreboot-jenkins image, change the value in the +`DOCKER_COMMIT` variable to the new image value. + +``` +$ make -C util/docker docker-jenkins-server +``` + +#### Getting ready to push the docker images + +Set up an account on hub.docker.com + +Get an admin to add the account to the coreboot team on hub.docker.com + +[https://hub.docker.com/u/coreboot/dashboard/teams/?team=owners%5D(https://hu...) + +Make sure your credentials are configured on your host machine by +running + +``` +$ docker login +``` + +This will prompt you for your docker username, password, and your email +address, and write out to ~/.docker/config.json. Without this file, you +won’t be able to push the images. + +#### Updating the Dockerfiles: + +The coreboot-sdk Dockerfile will need to be updated when any additional +dependencies are added. Both the coreboot-sdk and the +coreboot-jenkins-node Dockerfiles will need to be updated to the new +version number and git commit id anytime the toolchain is updated. Both +files are stored in the coreboot repo under coreboot/util/docker. + +Read the [dockerfile best practices](https://docs.docker.com/v1.8/articles/dockerfile_best-practices/) +page before updating the files. + +#### Rebuilding the coreboot-sdk docker image to update the toolchain: + +``` +$ make -C util/docker coreboot-sdk +``` + +This takes a relatively long time. + +#### Test the coreboot-sdk docker image: + +There are two methods of running the docker image - interactively as a +shell, or doing the build directly. Running interactively as a shell is +useful for early testing, because it allows you to update the image +(without any changes getting saved) and re-test builds. This saves the +time of having to rebuild the image for every issue you find. + +#### Running the docker image interactively: + +Run: + +``` +$ make -C util/docker docker-jenkins-server +$ make -C util/docker docker-jenkins-attach +``` + +#### Running the build directly: + +From the coreboot directory: + +``` +$ make -C util/docker docker-build-coreboot +``` + +You’ll also want to test building the other projects and payloads: +ChromeEC, flashrom, memtest86+, em100, Grub2, SeaBIOS, iPXE, coreinfo, +nvramcui, tint... + +#### Pushing the coreboot-sdk image to hub.docker.com for use: + +When you’re satisfied with the testing, push the coreboot-sdk image to +the hub.docker.com + +``` +$ make -C util/docker upload-coreboot-sdk +``` + +#### Building and pushing the coreboot-jenkins-node docker image: + +This docker image is pretty simple, so there’s not really any testing +that needs to be done. + +``` +$ make -C util/docker coreboot-jenkins-node +$ make -C util/docker upload-coreboot-jenkins-node +``` + +### Coverity Setup + +To run coverity jobs, the builder needs to have the tools available, and +to be marked as a coverity builder. + + +#### Set up the Coverity tools + +Download the Linux-64 coverity build tool and decompress it into your +cache directory as defined by the `$COREBOOT_JENKINS_CACHE_DIR` variable + +[https://scan.coverity.com/download%5D(https://scan.coverity.com/download) + +Rename the directory from its original name +(cov-analysis-linux64-7.7.0.4) to ‘coverity’, or better, create a +symlink: + +``` +ln -s cov-analysis-linux64-7.7.0.4 coverity +``` + + +Let the admins know that the ‘coverity’ label can be added to the +builder. diff --git a/Documentation/infrastructure/index.md b/Documentation/infrastructure/index.md new file mode 100644 index 0000000..42bcbf6 --- /dev/null +++ b/Documentation/infrastructure/index.md @@ -0,0 +1,6 @@ +# coreboot infrastructure + +This section contains documentation about coreboot infrastructure + +## Jenkins builders and builds +[Setting up Jenkins build machines](builders.md)
-
Angel Pons (Code Review) -
Martin Roth (Code Review)
-
Patrick Georgi (Code Review)