Hi folks, As part of an effort underway to get chromium.org's flashrom sync'd with upstream (for real), I have written an entirely new test script to help get thru regression testing. It's still rather fugly and will likely go thru several more revisions, but is currently usable for all major use cases I could think of and would benefit from some early feedback.
I wrote up a document with usage examples here: https://goo.gl/3jNoL7 . Long-term the plan is to migrate this to the flashrom wiki.
There were a few "must haves" for this script in the near-term: - Region awareness. The script understands how to use layout files and flashmaps (for Chromebooks). Descriptor awareness is on the TODO list for Intel platforms. - Can control flashrom on a remote host. - Can utilize an external programmer in addition to the native programmer on the target machine. This allows us to verify changes made to programmer code since one programmer can be considered to be trusted while the other can have code that was modified.
Longer-term goals include performance testing and flash endurance testing. We might also add write-protect testing, though there's an open question of whether that would be better in another script or as a whitebox test (more on that later...).
Here's the patch as of revision #14 on chromium.org's gerrit server: https://chromium-review.googlesource.com/#/c/353912/
As in the gerrit message, Signed-off-by: David Hendricks < dhendrix@chromium.org>
From 353aa26a174d9beb1ec886f7f752473f94023ed8 Mon Sep 17 00:00:00 2001
From: David Hendricks dhendrix@chromium.org Date: Sun, 19 Jun 2016 12:53:22 -0700 Subject: [PATCH] WIP: New test script for flashrom
** work in progress **
Shiny new testing capabilities for Flashrom: - Region awareness - Remote testing - Primary and secondary programmer support - Autotest-friendly
Long-term goals: - Performance measurement - Endurance testing
BUG=chromium:621715 BRANCH=none TEST=this *is* the test
Change-Id: Ic643cf2edaf7d8772c63ce8119363d882d1b116d Signed-off-by: David Hendricks dhendrix@chromium.org --- tests/tests_v2/cmd.sh | 109 ++++++ tests/tests_v2/test_v2.sh | 909 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1018 insertions(+) create mode 100644 tests/tests_v2/cmd.sh create mode 100644 tests/tests_v2/test_v2.sh
diff --git a/tests/tests_v2/cmd.sh b/tests/tests_v2/cmd.sh new file mode 100644 index 0000000..443ee2c --- /dev/null +++ b/tests/tests_v2/cmd.sh @@ -0,0 +1,109 @@ +#!/bin/sh +# +# This file is part of the flashrom project. It is derived from +# board_status.sh in coreboot. +# +# Copyright (C) 2016 Google Inc. +# Copyright (C) 2014 Sage Electronic Engineering, LLC. + +# test a command +# +# $1: 0 ($LOCAL) to run command locally, +# 1 ($REMOTE) to run remotely if remote host defined +# $2: command to test +# $3: 0 ($FATAL) Exit with an error if the command fails +# 1 ($NONFATAL) Don't exit on command test failure +test_cmd() +{ + local rc + local cmd__="$(echo $2 | cut -d ' ' -f -1)" + local args="$(echo $2 | cut -d ' ' -f 2-)" + + if [ -e "$cmd__" ]; then + return + fi + + if [ "$1" -eq "$REMOTE" ] && [ -n "$REMOTE_HOST" ]; then + ssh $REMOTE_PORT_OPTION root@${REMOTE_HOST} command -v "$cmd__" $args > /dev/null 2>&1 + rc=$? + else + command -v "$cmd__" $args >/dev/null 2>&1 + rc=$? + fi + + if [ $rc -eq 0 ]; then + return 0 + fi + + if [ "$3" = "1" ]; then + return 1 + fi + + echo "$2 not found" + exit $EXIT_FAILURE +} + +# Same args as cmd() but with the addition of $4 which determines if the +# command should be totally silenced or not. +_cmd() +{ + local silent=$4 + + if [ -n "$3" ]; then + pipe_location="${3}" + else + pipe_location="/dev/null" + fi + + if [ $1 -eq $REMOTE ] && [ -n "$REMOTE_HOST" ]; then + if [ $silent -eq 0 ]; then + ssh $REMOTE_PORT_OPTION "root@${REMOTE_HOST}" "$cmd $args" > "$pipe_location" 2>/dev/null + else + ssh $REMOTE_PORT_OPTION "root@${REMOTE_HOST}" "$cmd $args" > /dev/null 2>&1 + fi + else + if [ $silent -eq 0 ]; then + $2 > "$pipe_location" 2>/dev/null + else + $2 > /dev/null 2>&1 + fi + fi + + return $? +} + +# run a command +# +# $1: 0 ($LOCAL) to run command locally, +# 1 ($REMOTE) to run remotely if remote host defined +# $2: command +# $3: filename to direct output of command into +cmd() +{ + _cmd $1 "$2" "$3" 0 + + if [ $? -eq 0 ]; then + return + fi + + echo "Failed to run "$2", aborting" + rm -f "$3" # don't leave an empty file + exit $EXIT_FAILURE +} + +# run a command silently +# +# $1: 0 ($LOCAL) to run command locally, +# 1 ($REMOTE) to run remotely if remote host defined +# $2: command +scmd() +{ + _cmd $1 "$2" "" 1 + + if [ $? -eq 0 ]; then + return + fi + + echo "Failed to run "$2", aborting" + exit $EXIT_FAILURE +} diff --git a/tests/tests_v2/test_v2.sh b/tests/tests_v2/test_v2.sh new file mode 100644 index 0000000..5a8cb79 --- /dev/null +++ b/tests/tests_v2/test_v2.sh @@ -0,0 +1,909 @@ +#!/bin/sh +# +# Copyright (C) 2016 Google Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +EXIT_SUCCESS=0 +EXIT_FAILURE=1 +RC=$EXIT_SUCCESS +FATAL=0 +NONFATAL=1 + +# For old flashrom version, if -o is not specified +DEFAULT_FLASHROM_PATH="/usr/local/sbin/flashrom" + +# +# Stuff obtained from command-line +# + +# Generic options +BACKUP_IMAGE="" +OLD_FLASHROM="" +NEW_FLASHROM="" +NO_CLEAN=0 +SKIP_CONSISTENCY_CHECK=0 +UPLOAD_RESULTS=0 + +# Primary/Secondary programmer options +PRIMARY_OPTS="" +SECONDARY_OPTS="" + +# Region modes +REGION_MODE_UNKNOWN=0 +REGION_MODE_CLOBBER=1 +REGION_MODE_DESCRIPTOR=2 +REGION_MODE_FLASHMAP=3 +REGION_MODE_LAYOUT=4 +REGION_MODE=$REGION_MODE_UNKNOWN +DESCRIPTOR_REGION="bios" +FLASHMAP_REGION="RW_SECTION_A" +LAYOUT_FILE="" +LAYOUT_REGION="RW" +SMALL_REGION=0 + +# Remote testing options +SSH_PORT="" +REMOTE_HOST="" +REMOTE_PORT_OPTION="" +REMOTE_ONLY=0 +REMOTE_PROGRAMMER_PARAMS="" +SSH_CMD="ssh $REMOTE_PORT_OPTION root@${REMOTE_HOST} command -v" + +LOCAL=0 +REMOTE=1 +DO_REMOTE=$LOCAL # boolean to use for cmd() and tests + +# For suppressing flashrom output +SILENT=">/dev/null 2>&1" + +# 1KB +K="1024" + +show_help() { + echo "Usage: + ${0} <options> + +General options: + -b, --backup-image + Backup image to write unconditionally at end of testing. + -h, --help + Show this message. + -l, --layout-file + Layout file (required if mode is "layout", resides locally). + -m, --mode <arg> + Region access mode (clobber, descriptor, flashmap, layout). + -n, --new <path> + Path to new version of flashrom. + -o, --old <path> + Path to old version of flashrom. + -p, --primary-programmer <parameters> + Primary programmer options. + -r, --remote-host <host> + Remote host to test primary programmer on. + -s, --secondary-programmer <parameters> + Secondary programmer options. + -u, --upload-results + Upload results to flashrom.org. + +Long options: + --descriptor-region + Specify region to use in descriptor mode (default: $DESCRIPTOR_REGION) + --fmap-region + Specify region to use in flashmap mode (default: $FLASHMAP_REGION) + --layout-region + Specify region to use in layout mode (default: $LAYOUT_REGION) + --no-clean + Do not remove temporary files. + --skip-consistency-check + Skip the consistency check (two consecutive reads) at beginning. + --small-region + Omit tests that require large amounts of space (>16KB). + +Remote connectivity options: + --ssh-port <port> + Use a specific SSH port. + +This script is intended to cover two possible test scenarios: +1. Test flashrom using primary programmer only (default). +2. Test flashrom using primary programmer and verify result using + a secondary programmer, for example, an external USB programmer. + +If a secondary programmer is provided, certain steps will be verified +using the secondary programmer. The secondary programmer must be +connected to the machine which is running this script. + +If a remote host is provided, this script will SSH into the remote +host and run tests using the primary programmer interface parameters +provided. + +See documentation for usage examples (TODO: write documentation) +" +} + +getopt -T +if [ $? -ne 4 ]; then + echo "GNU-compatible getopt(1) required." + exit $EXIT_FAILURE +fi + +LONGOPTS="backup-image:,help,,new:,old:,remote-host:,upload-results:" +LONGOPTS="${LONGOPTS},primary-programmer:,secondary-programmer:" +LONGOPTS="${LONGOPTS},skip-consistency-check" +LONGOPTS="${LONGOPTS},mode:,clobber,descriptor,flashmap,layout,small-region" +LONGOPTS="${LONGOPTS},layout-file:,desctriptor-region:,flashmap-region:,layout-region:" +LONGOPTS="${LONGOPTS},no-clean" +LONGOPTS="${LONGOPTS},ssh-port:" + +ARGS=$(getopt -o b:hl:m:n:o:p:r:s:u -l "$LONGOPTS" -n "$0" -- "$@"); +if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi +eval set -- "$ARGS" +while true ; do + case "$1" in + # Generic options + -b|--backup-image) + shift + BACKUP_IMAGE="$1" + ;; + -h|--help) + show_help + exit $EXIT_SUCCESS + ;; + -l|--layout-file) + shift + LAYOUT_FILE="$1" + ;; + -m|--mode) + shift + if [ "$1" == "clobber" ]; then + REGION_MODE=$REGION_MODE_CLOBBER + elif [ "$1" == "descriptor" ]; then + REGION_MODE=$REGION_MODE_DESCRIPTOR + elif [ "$1" == "flashmap" ]; then + REGION_MODE=$REGION_MODE_FLASHMAP + elif [ "$1" == "layout" ]; then + REGION_MODE=$REGION_MODE_LAYOUT + else + echo "Unknown mode: $1" + exit $EXIT_FAILURE + fi + ;; + -n|--new) + shift + NEW_FLASHROM="$1" + ;; + -o|--old) + shift + OLD_FLASHROM="$1" + ;; + -p|--primary_programmer) + shift + PRIMARY_OPTS="-p $1" + ;; + -s|--secondary_programmer) + shift + SECONDARY_OPTS="-p $1" + ;; + -r|--remote-host) + DO_REMOTE=1 + shift + REMOTE_HOST="$1" + ;; + -u|--upload-results) + UPLOAD_RESULTS=1 + ;; + + # Longopts only + --descriptor-region) + shift + DESCRIPTOR_REGION="$1" + ;; + --flashmap-region) + shift + FLASHMAP_REGION="$1" + ;; + --layout-region) + shift + LAYOUT_REGION="$1" + ;; + --no-clean) + NO_CLEAN=1 + ;; + --skip-consistency-check) + SKIP_CONSISTENCY_CHECK=1 + ;; + + # Remote testing options + --ssh-port) + shift + REMOTE_PORT_OPTION="-p $1" + ;; + + # error handling + --) + shift + if [ -n "$*" ]; then + echo "Non-option parameters detected: '$*'" + exit $EXIT_FAILURE + fi + break + ;; + *) + echo "error processing options at '$1'" + exit $EXIT_FAILURE + esac + shift +done + +# TODO: Implement this. +if [ $UPLOAD_RESULTS -eq 1 ]; then + echo "TODO: Implement ability to upload results." + exit $EXIT_FAILURE +fi + +# +# Test command-line validity. +# +if [ $REGION_MODE -eq $REGION_MODE_UNKNOWN ]; then + echo "Must specify a region access mode (-m/--mode)." + exit $EXIT_FAILURE +elif [ $REGION_MODE -eq $REGION_MODE_LAYOUT ]; then + if [ ! -e "$LAYOUT_FILE" ]; then + echo "Must specify a layout file when using layout mode." + exit $EXIT_FAILURE + fi +fi + +# +# Source helper scripts +# +export REMOTE_HOST REMOTE_PORT_OPTION +export LOCAL REMOTE FATAL NONFATAL EXIT_SUCCESS EXIT_FAILURE +. "$(pwd)/tests/tests_v2/cmd.sh" + +if [ $DO_REMOTE -eq 1 ]; then + # Test connection to remote host + test_cmd $DO_REMOTE "ls /" $NONFATAL + if [ $? -ne 0 ]; then + echo "Could not connect to remote host $REMOTE_HOST" + exit $EXIT_FAILURE + fi +fi + +# +# Dependencies +# + +# cmp is used to compare files +test_cmd $DO_REMOTE "cmp" $FATAL + +if [ ! -e "/dev/urandom" ]; then + echo "This script uses /dev/urandom" + exit $EXIT_FAILURE +fi + +if [ ! -e "/dev/zero" ]; then + echo "This script uses /dev/zero" + exit $EXIT_FAILURE +fi + +# +# Setup. +# +grep -rH 'projectname = .*flashrom' .git/config >/dev/null 2>&1 +if [ $? -ne 0 ]; then + echo "Script must be run from root of flashrom directory" + exit $EXIT_FAILURE +fi + +if [ -z "$OLD_FLASHROM" ]; then + OLD_FLASHROM="$DEFAULT_FLASHROM_PATH" +fi +test_cmd $DO_REMOTE "$OLD_FLASHROM --help" $NONFATAL +if [ $? -ne 0 ]; then + echo "Old flashrom binary is not usable." + exit $EXIT_FAILURE +fi + +# +# Setup temporary working directories: +# LOCAL_TMPDIR: Working directory on local host. +# REMOTE_TMPDIR: Working directory on remote host. +# TMPDIR: The temporary directy in which we do most of the work. This is +# convenient for commands that depend on $DO_REMOTE. +LOCAL_TMPDIR=$(mktemp -d --tmpdir flashrom_test.XXXXXXXX) +if [ $? -ne 0 ] ; then + echo "Could not create temporary directory" + exit $EXIT_FAILURE +fi + +if [ $DO_REMOTE -eq 1 ]; then + REMOTE_TMPDIR=$(ssh root@${REMOTE_HOST} mktemp -d --tmpdir flashrom_test.XXXXXXXX) + if [ $? -ne 0 ] ; then + echo "Could not create temporary directory" + exit $EXIT_FAILURE + fi +fi + +if [ $DO_REMOTE -eq 0 ]; then + TMPDIR="$LOCAL_TMPDIR" +else + TMPDIR="$REMOTE_TMPDIR" +fi + +# Copy files from local tmpdir to remote host tmpdir +copy_to_remote() +{ + for F in $@; do + # FIXME: quoting for REMOTE_TMPDIR? + scp "${LOCAL_TMPDIR}/$F" root@${REMOTE_HOST}:${REMOTE_TMPDIR} 2>&1
/dev/null
+ done +} + +# Copy files from remote host tmpdir to local tmpdir +copy_from_remote() +{ + for F in $@; do + # FIXME: quoting for REMOTE_TMPDIR? + scp root@${REMOTE_HOST}:${REMOTE_TMPDIR}/$F "${LOCAL_TMPDIR}/" 2>/dev/null + done +} + +# Read current image as backup in case one hasn't already been specified. +if [ -z "$BACKUP_IMAGE" ]; then + local backup_file="backup.bin" + if [ $DO_REMOTE -eq 1 ]; then + BACKUP_IMAGE="${REMOTE_TMPDIR}/${backup_file}" + else + BACKUP_IMAGE="${LOCAL_TMPDIR}/${backup_file}" + fi + + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS -r $BACKUP_IMAGE" + if [ $? -ne 0 ]; then + echo "Failed to read backup image, aborting." + exit $EXIT_FAILURE + fi + + if [ $DO_REMOTE -eq 1 ]; then + copy_from_remote "${backup_file}" + fi +fi + +# The copy of flashrom to test. If unset, we'll assume the user wants to test +# a newly built flashrom binary in the current directory. +if [ -z "$NEW_FLASHROM" ] ; then + if [ -x "flashrom" ]; then + NEW_FLASHROM="flashrom" + else + echo "Must supply new flashrom version to test" + exit $EXIT_FAILURE + fi +fi + +echo "Stable flashrom binary: $OLD_FLASHROM" +echo "New flashrom binary to test: $NEW_FLASHROM" +echo "Local temporary files will be stored in $LOCAL_TMPDIR" +if [ $DO_REMOTE -eq 1 ]; then + echo "Remote temporary files will be stored in ${REMOTE_HOST}:${REMOTE_TMPDIR}" +fi +echo "Backup image: $BACKUP_IMAGE" + +# +# Now the fun begins. +# +cmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS --get-size" "${LOCAL_TMPDIR}/chip_size.txt" +tmp=$(cat ${LOCAL_TMPDIR}/chip_size.txt) +cmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS --get-size" "${LOCAL_TMPDIR}/chip_size.txt" +CHIP_SIZE=$(cat ${LOCAL_TMPDIR}/chip_size.txt) +CHIP_SIZE_KB=$(($CHIP_SIZE / 1024)) +if [ $CHIP_SIZE -ne $tmp ]; then + echo "New flashrom and old flashrom disagree on chip size. Aborting." + exit $EXIT_FAILURE +else + echo "Chip size: ${CHIP_SIZE_KB} KiB" +fi + +# Upload results +#do_upload() +#{ +# # TODO: implement this +#} + +# Remove temporary files +do_cleanup() +{ + if [ $NO_CLEAN -eq 1 ]; then + echo "Skipping cleanup." + return $EXIT_SUCCESS + fi + + rm -rf "$LOCAL_TMPDIR" + if [ -n "$REMOTE_HOST" ]; then + ssh root@${REMOTE_HOST} rm -rf "${REMOTE_TMPDIR}" + fi + + return $EXIT_SUCCESS +} + +# $1: Message to display to user. +test_fail() +{ + echo "$1" + do_cleanup + exit $EXIT_FAILURE +} + +write_backup_image() +{ + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS -w $BACKUP_IMAGE" +} + +# Read a region twice and compare results +# $1: address of region (in bytes) +# $2: length of region (in bytes) +double_read_test() +{ + local cmp1="${TMPDIR}/cmp1.bin" + local cmp2="${TMPDIR}/cmp2.bin" + local layout="double_read_test_layout.txt" + + printf "Doing double read test, size: %u KiB\n" $(($2 / $K)) + # FIXME: Figure out how to do printf remotely... + printf "%06x:%06x region\n" $1 $(($1 + $2 - 1)) > "${LOCAL_TMPDIR}/${layout}" + if [ $DO_REMOTE -eq 1 ]; then copy_to_remote "$layout" ; fi + + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS -r -l ${TMPDIR}/${layout} --ignore-fmap -i region:$cmp1" + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS -r -l ${TMPDIR}/${layout} --ignore-fmap -i region:$cmp2" + scmd $DO_REMOTE "cmp $cmp1 $cmp2" + if [ $? -ne 0 ]; then + test_fail "Double-read test failed, aborting." + fi +} + +# Simple partial write test. Given a region: +# - Write a sub-region within a single eraseable block. +# - Write a sub-region that crosses over into the next eraseable block. +# +# We assume that eraseable block size can be either 4KB or 64KB and +# must test for both. This means the given region must be at least 128KB. +# +# $1: region addr +# $2: region size (in bytes) +simple_partial_write_test() +{ + local addr="$1" + local size="$2" + local layout="simple_partial_write_test_layout.txt" + local random_2k_subregion="simple_partial_write_test_random_2k_subregion.bin" + local random_2k_crossover="simple_partial_write_test_random_2k_crossover.bin" + local random_32k_subregion="simple_partial_write_test_random_32k_subregion.bin" + local random_32k_crossover="simple_partial_write_test_random_32k_crossover.bin" + local whole_region="simple_partial_write_test_whole_region.bin" + local opts="--ignore-fmap -l ${TMPDIR}/${layout}" + +# FIXME: integer/string conversion +# if [ $size -lt 131072 ]; then +# test_fail "Region is not big enough for partial write test." +# fi +# +# if [ $(($addr + $size)) -gt $CHIP_SIZE ]; then +# test_fail "Region exceeds chip size." +# fi + + # FIXME: For now, assume the given region is aligned to 64KB. + + printf "Doing partial write test\n" + + # Create a layout file for regions we're interested in. + # FIXME: figure out how to do this using cmd + printf "%06x:%06x 4k_region\n" $addr $(($addr + 4*$K - 1)) >> "${LOCAL_TMPDIR}/${layout}" + printf "%06x:%06x 2k_subregion\n" $(($addr + $K)) $(($addr + 4*$K - $K - 1)) >> "${LOCAL_TMPDIR}/${layout}" + printf "%06x:%06x 2k_crossover\n" $(($addr + 4*$K - $K)) $(($addr + 4*$K + $K - 1)) >> "${LOCAL_TMPDIR}/${layout}" + + printf "%06x:%06x 64k_region\n" $addr $(($addr + 64*$K - 1)) >> "${LOCAL_TMPDIR}/${layout}" + printf "%06x:%06x 32k_subregion\n" $(($addr + 16*$K)) $(($addr + 64*$K - 16*$K - 1)) >> "${LOCAL_TMPDIR}/${layout}" + printf "%06x:%06x 32k_crossover\n" $(($addr + 64*$K - 16*$K)) $(($addr + 64*$K + 16*$K - 1)) >> "${LOCAL_TMPDIR}/${layout}" + + printf "%06x:%06x whole" $addr $(($addr + $size - 1)) >> "${LOCAL_TMPDIR}/${layout}" + + dd if=/dev/urandom of=${LOCAL_TMPDIR}/${random_2k_subregion} bs=1k count=2 2>/dev/null + dd if=/dev/urandom of=${LOCAL_TMPDIR}/${random_2k_crossover} bs=1k count=2 2>/dev/null + dd if=/dev/urandom of=${LOCAL_TMPDIR}/${random_32k_subregion} bs=1k count=32 2>/dev/null + dd if=/dev/urandom of=${LOCAL_TMPDIR}/${random_32k_crossover} bs=1k count=32 2>/dev/null + + # Example for creating a file filled with 0xaa bytes: + # dd if=/dev/zero bs=1k count=2 | tr "\000" "\252" > "aa_2K.bin" + # Example to create file with 0xff bytes: + # dd if=/dev/zero bs=1k count=2 | tr "\000" "\377" > "ff_2k.bin" + + if [ $DO_REMOTE -eq 1 ]; then + copy_to_remote "${layout}" + copy_to_remote "${random_2k_subregion}" "${random_2k_crossover}" + copy_to_remote "${random_32k_subregion}" "${random_32k_crossover}" + fi + + # Create backup of region. This will be modified along with the ROM content, + # read back, and compared. + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -r -i whole:${TMPDIR}/${whole_region}" + + scmd $DO_REMOTE "dd if=${TMPDIR}/${random_2k_subregion} of=${TMPDIR}/${whole_region} bs=1 seek=$K conv=notrunc" + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -w --fast-verify -i 2k_subregion:${TMPDIR}/${random_2k_subregion}" + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -r -i 2k_subregion:${TMPDIR}/cmp.bin" + scmd $DO_REMOTE "cmp ${TMPDIR}/${random_2k_subregion} ${TMPDIR}/cmp.bin" + if [ $? -ne 0 ]; then + test_fail "Failed to verify partial write to 2k subregion." + else + printf "\t2k subregion test passed\n" + fi + + scmd $DO_REMOTE "dd if=${TMPDIR}/${random_2k_crossover} of=${TMPDIR}/${whole_region} bs=1 seek=$((4*$K - $K)) conv=notrunc" + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -w --fast-verify -i 2k_crossover:${TMPDIR}/${random_2k_crossover}" + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -r -i 2k_crossover:${TMPDIR}/cmp.bin" + scmd $DO_REMOTE "cmp ${TMPDIR}/${random_2k_crossover} ${TMPDIR}/cmp.bin" + if [ $? -ne 0 ]; then + test_fail "Failed to verify partial write to 2k crossover region." + else + printf "\t2k crossover region test passed\n" + fi + + if [ $SMALL_REGION -eq 0 ]; then + scmd $DO_REMOTE "dd if=${TMPDIR}/${random_32k_subregion} of=${TMPDIR}/${whole_region} bs=1 seek=$((16*$K)) conv=notrunc" + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -w --fast-verify -i 32k_subregion:${TMPDIR}/${random_32k_subregion}" + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -r -i 32k_subregion:${TMPDIR}/cmp.bin" + scmd $DO_REMOTE "cmp ${TMPDIR}/${random_32k_subregion} ${TMPDIR}/cmp.bin" + if [ $? -ne 0 ]; then + test_fail "Failed to verify partial write to 32k subregion." + else + printf "\t32k subregion test passed\n" + fi + + scmd $DO_REMOTE "dd if=${TMPDIR}/${random_32k_crossover} of=${TMPDIR}/${whole_region} bs=1 seek=$((64*$K - 16*$K)) conv=notrunc" + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -w --fast-verify -i 32k_crossover:${TMPDIR}/${random_32k_crossover}" + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -r -i 32k_crossover:${TMPDIR}/cmp.bin" + scmd $DO_REMOTE "cmp ${TMPDIR}/${random_32k_crossover} ${TMPDIR}/cmp.bin" + if [ $? -ne 0 ]; then + test_fail "Failed to verify partial write to 32k crossover region." + else + printf "\t32k crossover region test passed\n" + fi + fi + + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -r -i whole:${TMPDIR}/cmp.bin" + scmd $DO_REMOTE "cmp ${TMPDIR}/${whole_region} ${TMPDIR}/cmp.bin" + if [ $? -ne 0 ]; then + test_fail "Failed to verify correctness of final image." + fi + + if [ -n "$SECONDARY_ARGS" ]; then + if [ $DO_REMOTE -eq 1 ]; then + copy_from_remote "${whole_region}" + fi + + scmd $LOCAL "$OLD_FLASHROM $SECONDARY_OPTS $opts -r -i whole:${LOCAL_TMPDIR}/cmp.bin" + scmd $LOCAL "cmp ${LOCAL_TMPDIR}/${whole_region} ${LOCAL_TMPDIR}/cmp.bin" + if [ $? -ne 0 ]; then + test_fail "Failed to verify correctness of final image." + fi + fi + + return $EXIT_SUCCESS +} + +# Regional partial write test. Given a region name, this will write patterns +# of bytes designed to test corner cases. +# +# We assume that eraseable block size can be either 4KB or 64KB and +# must test for both. For simplicity, we'll assume the region size is +# at least 256KB. +# +# $1: Region name +region_partial_write_test() +{ + local opts="--fast-verify" + local layout="region_partial_write_test_layout.txt" + local region_size=0 + local align_size_kb=0 + local region_name="$1" + local filename="" + local offset_kb=0 + local size_kb=0 + local test_num=0 + local prev_test_num=0 + + # FIXME: Sanity checking: + # - Region must be aligned to 64KB + # - Region must be at least 128KB in size. + + if [ $REGION_MODE -ne $REGION_MODE_FLASHMAP ]; then + opts="$opts --ignore-fmap" + fi + + if [ $SMALL_REGION -eq 1 ]; then + align_size_kb=16 + else + align_size_kb=256 + fi + + echo "Doing region-based partial write test on region "$region_name"" + filename="${TMPDIR}/${region_name}.bin" + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -r -i ${region_name}:${filename}" + copy_from_remote "${region_name}.bin" + region_size=$(stat --format=%s ${LOCAL_TMPDIR}/${region_name}.bin) + + if [ $region_size -lt $(($align_size_kb * $K)) ]; then + echo "Region $region_name is too small" + return $EXIT_FAILURE + fi + + if [ $(($region_size % $(($align_size_kb * $K)))) -ne 0 ]; then + echo "Region $region_name is not aligned to ${align_size}KB" + return $EXIT_FAILURE + fi + + # Test procedure: + # Clobber region with random content first. Then do writes using the + # following sequences for each 128KB: + # 0-2K : 0x00 (\000) Partial 4KB sector, lower half + # 2K-6K : 0x11 (\021) Crossover 4KB sector boundary + # 6K-8K : 0x22 (\042) Partial 4KB sector, upper half + # 8K-16K : 0x33 (\063) Full 4KB sectors + # + # Repeat the above sequence for 64KB-aligned sizes + # 0-32K : 0x44 (\104) Partial 64KB block, lower half + # 32K-96K : 0x55 (\125) Crossover 64KB block boundary + # 96K-128K : 0x66 (\146) Partial 64KB block, upper half + # 128K-256K : 0x77 (\167) Full 64KB blocks + + test_num=0 + dd if=/dev/urandom of="${LOCAL_TMPDIR}/random_4k_test.bin" bs=4k count=$(($region_size / $((4 * $K)))) 2>/dev/null + + # 0-2K : 0x00 (\000) Partial 4KB sector, lower half + offset_kb=0 + size_kb=2 + hex="0x00" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/random_4k_test.bin" "${LOCAL_TMPDIR}/4k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/4k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + prev_test_num=$test_num + test_num=$(($test_num + 1)) + + # 2K-6K : 0x11 (\021) Crossover 4KB sector boundary + offset_kb=2 + size_kb=4 + hex="0x11" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/4k_test_${prev_test_num}.bin" "${LOCAL_TMPDIR}/4k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/4k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + test_num=$(($test_num + 1)) + + # 6K-8K : 0x22 (\042) Partial 4KB sector, upper half + offset_kb=2 + size_kb=4 + hex="0x11" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/4k_test_${prev_test_num}.bin" "${LOCAL_TMPDIR}/4k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/4k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + test_num=$(($test_num + 1)) + + # 8K-16K : 0x33 (\063) Full 4KB sectors + offset_kb=8 + size_kb=8 + hex="0x22" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/4k_test_${prev_test_num}.bin" "${LOCAL_TMPDIR}/4k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/4k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + test_num=$(($test_num + 1)) + + for F in ${LOCAL_TMPDIR}/random_4k_test.bin ${LOCAL_TMPDIR}/4k_test_*.bin ; do + filename=$(basename $F) + if [ $DO_REMOTE -eq 1 ]; then + copy_to_remote $filename + fi + + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -w -i ${region_name}:${TMPDIR}/${filename}" + if [ $? -ne 0 ]; then + test_fail "Failed to write $filename to $region_name" + fi + + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -v -i ${region_name}:${TMPDIR}/${filename}" + if [ $? -ne 0 ]; then + test_fail "Failed to verify write of $filename to $region_name" + fi + + if [ -n "$SECONDARY_OPTS" ]; then + scmd $LOCAL "$OLD_FLASHROM $SECONDARY_OPTS $opts -v -i ${region_name}:${TMPDIR}/${filename}" + if [ $? -ne 0 ]; then + test_fail "Failed to verify write of $filename to $region_name using secondary programmer" + fi + fi + + printf "\tWrote $filename to $region_name region successfully.\n" + done + + if [ $SMALL_REGION -eq 1 ]; then + return $EXIT_SUCCESS + fi + + # + # Second half: Tests for 64KB chunks + # + test_num=0 + dd if=/dev/urandom of="${LOCAL_TMPDIR}/random_64k_test.bin" bs=128k count=$(($region_size / $((128*$K)))) 2>/dev/null + + # 0-32K : 0x44 (\104) Partial 64KB block, lower half + offset_kb=0 + size_kb=32 + hex="0x44" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/random_64k_test.bin" "${LOCAL_TMPDIR}/64k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/64k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + prev_test_num=$test_num + test_num=$(($test_num + 1)) + + # 32K-96K : 0x55 (\125) Crossover 64KB block boundary + offset_kb=32 + size_kb=64 + hex="0x55" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/64k_test_${prev_test_num}.bin" "${LOCAL_TMPDIR}/64k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/64k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + test_num=$(($test_num + 1)) + + # 96K-128K : 0x66 (\146) Partial 64KB block, upper half + offset_kb=96 + size_kb=32 + hex="0x66" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/64k_test_${prev_test_num}.bin" "${LOCAL_TMPDIR}/64k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/64k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + test_num=$(($test_num + 1)) + + # 128K-256K : 0x77 (\167) Full 64KB blocks + offset_kb=128 + size_kb=128 + hex="0x77" + oct="\$(printf "%03o" $hex)" + cp "${LOCAL_TMPDIR}/64k_test_${prev_test_num}.bin" "${LOCAL_TMPDIR}/64k_test_${test_num}.bin" + while [ $(($offset_kb * $K)) -lt $region_size ]; do + dd if=/dev/zero bs=1k count=$size_kb 2>/dev/null | tr "\000" "$oct" > "${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" + dd if="${LOCAL_TMPDIR}/${hex}_${size_kb}k.bin" of="${LOCAL_TMPDIR}/64k_test_${test_num}.bin" bs=1k count=${size_kb} seek=${offset_kb} conv=notrunc 2>/dev/null + offset_kb=$(($offset_kb + $align_size_kb)) + done + + for F in ${LOCAL_TMPDIR}/random_64k_test.bin ${LOCAL_TMPDIR}/64k_test_*.bin ; do + filename=$(basename $F) + if [ $DO_REMOTE -eq 1 ]; then + copy_to_remote $filename + fi + + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS $opts -w -i ${region_name}:${TMPDIR}/${filename}" + if [ $? -ne 0 ]; then + test_fail "Failed to write $filename to $region_name" + fi + + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS $opts -v -i ${region_name}:${TMPDIR}/${filename}" + if [ $? -ne 0 ]; then + test_fail "Failed to verify write of $filename to $region_name" + fi + + if [ -n "$SECONDARY_OPTS" ]; then + scmd $LOCAL "$OLD_FLASHROM $SECONDARY_OPTS $opts -v -i ${region_name}:${TMPDIR}/${filename}" + if [ $? -ne 0 ]; then + test_fail "Failed to verify write of $filename to $region_name using secondary programmer" + fi + fi + + printf "\tWrote $filename to $region_name region successfully.\n" + done + + return $EXIT_SUCCESS +} + +# Do a consistency check for sanity before any other test. +if [ $SKIP_CONSISTENCY_CHECK -eq 0 ]; then + double_read_test 0 $CHIP_SIZE +fi + +if [ $REGION_MODE -eq $REGION_MODE_CLOBBER ]; then + random_file="${TMPDIR}/random_${CHIP_SIZE_KB}K.bin" + cmp_file="${TMPDIR}/cmp.bin" + + scmd $DO_REMOTE "dd if=/dev/urandom of=$random_file bs=1k count=${CHIP_SIZE_KB}" + scmd $DO_REMOTE "$NEW_FLASHROM $PRIMARY_OPTS -w $random_file" + scmd $DO_REMOTE "$OLD_FLASHROM $PRIMARY_OPTS -r $cmp_file" + scmd $DO_REMOTE "cmp $random_file $cmp_file" + if [ $? -ne 0 ]; then + write_backup_image + test_fail "Failed to clobber entire ROM." + fi + scmd $DO_REMOTE "rm -f $cmp_file $random_file" + + simple_partial_write_test "0" "$CHIP_SIZE" +elif [ $REGION_MODE -eq $REGION_MODE_DESCRIPTOR ]; then + # This depends on descriptor regions being translated into internal + # layout representation automatically so we can target them using -i. + echo "TODO: Descriptor mode" + exit $EXIT_FAILURE +elif [ $REGION_MODE -eq $REGION_MODE_FLASHMAP ]; then + region_partial_write_test "$FLASHMAP_REGION" 0 + if [ $? -ne 0 ]; then + echo "Flashmap mode test failed" + RC=$EXIT_FAILURE + fi +elif [ $REGION_MODE -eq $REGION_MODE_LAYOUT ]; then + rw_layout="" + addr="" + end="" + size="" + + # Look for a region named "RW" with any amount of leading whitespace + # and no trailing whitespace or characters. + rw_layout=$(grep "\s${LAYOUT_REGION}$" $LAYOUT_FILE | head -n 1) + if [ -z "$rw_layout" ]; then + printf "No region matching "${LAYOUT_REGION}" found layout file "%s"\n" "$LAYOUT_FILE" + exit $EXIT_FAILURE + fi + + addr="0x$(echo "$rw_layout" | cut -d ' ' -f -1 | awk -F ':' '{ print $1 }')" + end="0x$(echo "$rw_layout" | cut -d ' ' -f -1 | awk -F ':' '{ print $2 }')" + size="$(($end - $addr + 1))" + + printf "RW region address: ${addr}, size: %u KiB\n" $(($size / $K)) + simple_partial_write_test "$addr" "$size" + if [ $? -ne 0 ]; then + echo "Flashmap mode test failed" + RC=$EXIT_FAILURE + fi +fi + +# restore and cleanup +write_backup_image +do_cleanup + +if [ $RC -eq 0 ]; then + echo "Test status: PASS" +else + echo "Test status: FAIL" +fi + +exit $RC