Re: [flashrom] [PATCH] Real partial writes

[Idwer Vollering]

2010/10/22 Carl-Daniel Hailfinger
> Given that the version which addresses the review comments apparently is
> buggy, I hereby propose to merge the earlier version which worked. It is
> here again for your reference with some slight cosmetic fixes which
> should not impact functionality.
> Error recovery still needs work (see the FIXME comment). It will work in
> most cases, but if a given erase command for a block only erased parts
> of the block (partial write lock), the code will make incorrect
> assumptions.
>
> If anyone feels adventurous, I would love to see logs on Intel/VIA
> chipsets with SPI (preferably locked down chipsets or with r1193 reverted).
>
> If you're going to review this, make sure you keep a stack of bananas
> (quickly mobilized carbohydrates for your brain), a bucket of ice (to
> prevent brain overheating) and a bottle of aspirin handy. If any code is
> unclear, please tell me and I'll try to add comments to improve
> readability.
>
> This code has been tested. Testing erase (and checking with a separate
> readback that erase actually worked) and write (same test with separate
> readback) would be highly appreciated. Verbose logs are even more
> appreciated.
>
> I think the code is ready for merge if you trust write/erase to never
> fail. The error cases still need to be tested. Should we reread the
> whole chip if write/erase failed to make sure our current view of the
> chip contents is not stale?
>
> This patch makes flashrom use real partial writes. If you write an image
> full of 0xff, flashrom will erase and detect that no write is needed. If
> you write an image which differs only in some parts from the current
> flash contents, flashrom will detect that and not touch unchanged areas.
>
> Fix a long-standing bug in need_erase() for 256 byte granularity as well.
>
> Nice side benefit: Detailed progress printing.
> S means skipped
> E means erased
> W means written
>
> Thanks to Andrew Morgan for testing countless iterations of this patch.
> Thanks to Richard A. Smith for testing on Dediprog.
> Thanks to David Hendricks for the review (will be addressed later).
>
> Signed-off-by: Carl-Daniel Hailfinger

With your latest patch (http://patchwork.coreboot.org/patch/2160/) applied:

$ sudo ./flashrom -p nicintel_spi -V -w nicintel_spi.rom -c M25P10.RES
flashrom v0.9.3-r1215 on FreeBSD 8.1-RELEASE (i386), built with libpci
3.1.7, GCC 4.2.1 20070719  [FreeBSD], little endian
flashrom is free software, get the source code at http://www.flashrom.org

Calibrating delay loop... OS timer resolution is 4 usecs, 942M loops per
second, delay more than 10% too short (got 80% of expected delay),
recalculating... 1923M loops per second, 10 myus = 11 us, 100 myus = 138 us,
1000 myus = 981 us, 10000 myus = 10947 us, 16 myus = 33 us, OK.
Initializing nicintel_spi programmer
Found "Intel 82541PI Gigabit Ethernet Controller" (8086:107c, BDF 01:03.0).
Requested BAR is MEM, 32bit, not prefetchable
Probing for ST M25P10.RES, 128 KB: probe_spi_res1: id 0x10
Chip status register is 00
Found chip "ST M25P10.RES" (128 KB, SPI) at physical address 0xfffe0000.
===
This flash part has status UNTESTED for operations: PROBE READ ERASE WRITE
The test status of this chip may have been updated in the latest development
version of flashrom. If you are running the latest development version,
please email a report to [e-mail] if any of the above
operations
work correctly for you with this flash part. Please include the flashrom
output with the additional -V option for all operations you tested (-V, -Vr,
-Vw, -VE), and mention which mainboard or programmer you tested.
Please mention your board in the subject line. Thanks for your help!
Reading old flash chip contents...
Erasing and writing flash chip... Looking at blockwise erase function 0...
trying... 0x000000-0x007fff:W, 0x008000-0x00ffff:W, 0x010000-0x017fff:W,
0x018000-0x01ffff:S

Done.
Verifying flash... VERIFY FAILED at 0x00008000! Expected=0xc6, Read=0x15,
failed byte count from 0x00000000-0x0001ffff: 0x8d75
Your flash chip is in an unknown state.
Get help on IRC at irc.freenode.net (channel #flashrom) or
mail [e-mail] with FAILED: your board name in the subject line!
-------------------------------------------------------------------------------
DO NOT REBOOT OR POWEROFF!
> Index: flashrom-partial_write_rolling_erase_write/flashrom.c
>
> --- flashrom-partial_write_rolling_erase_write/flashrom.c       (Revision
> 1215)
> +++ flashrom-partial_write_rolling_erase_write/flashrom.c
> (Arbeitskopie)
> @@ -793,6 +793,7 @@
> Check if the buffer @have can be programmed to the content of @want
>
> without
> erasing. This is only possible if all chunks of size @gran are either
>
> kept
> as-is or changed from an all-ones state to any other state.
> The following write granularities (enum @gran) are known:
> 1 bit. Each bit can be cleared individually.
> 1 byte. A byte can be written once. Further writes to an already
>
> written
> @@ -803,10 +804,12 @@
> this function.
> 256 bytes. If less than 256 bytes are written, the contents of the
> unwritten bytes are undefined.
> Warning: This function assumes that @have and @want point to naturally
> aligned regions.
> @have        buffer with current content
> @want        buffer with desired content
> @len         length of the verified area
> @len                length of the checked area
> @gran       write granularity (enum, not count)
> @return      0 if no erase is needed, 1 otherwise
>
> */
> @@ -838,7 +841,7 @@
>                                continue;
>                        /* have needs to be in erased state. */
>                        for (i = 0; i < limit; i++)
>                           if (have[i] != 0xff) {
>                           if (have[j * 256 + i] != 0xff) {
>                                  result = 1;
>                                  break;
>                          }
>
> @@ -846,10 +849,102 @@
>                                 break;
>                }
>                break;
>   default:
>           msg_cerr("%s: Unsupported granularity! Please report a bug
>
> at "
>                    "[e-mail]\n", __func__);
>  }
>  return result;
>
> }
>
> +/**
> Check if the buffer @have needs to be programmed to get the content of
>
> @want.
> If yes, return 1 and fill in first_start with the start address of the
> write operation and first_len with the length of the first
>
> to-be-written
> chunk. If not, return 0 and leave first_start and first_len undefined.
> Warning: This function assumes that @have and @want point to naturally
> aligned regions.
> @have       buffer with current content
> @want       buffer with desired content
> @len                length of the checked area
> @gran       write granularity (enum, not count)
> @return     0 if no write is needed, 1 otherwise
> @first_start        offset of the first byte which needs to be written
> @first_len  length of the first contiguous area which needs to be
>
> written
> FIXME: This function needs a parameter which tells it about coalescing
> in relation to the max write length of the programmer and the max write
> length of the chip.
> */
>
> +static int get_next_write(uint8_t *have, uint8_t *want, int len,
>                     int *first_start, int *first_len,
>                     enum write_granularity gran)
>
> +{
>   int result = 0;
>   int i, limit;
>   /* The passed in variable might be uninitialized. */
>   *first_len = 0;
>   switch (gran) {
>   case write_gran_1bit:
>   case write_gran_1byte:
>           for (i = 0; i < len; i++) {
>                   if (have[i] != want[i]) {
>                           if (!result) {
>                                   /* First location where have and
>
> want
>                                    * differ.
>                                    */
>                                   result = 1;
>                                   *first_start = i;
>                           }
>                   } else {
>                           if (result) {
>                                   /* First location where have and
>
> want
>                                    * do not differ anymore.
>                                    */
>                                   *first_len = i - *first_start;
>                                   break;
>                           }
>                   }
>           }
>           /* Did the loop terminate without setting first_len? */
>           if (result && ! *first_len)
>                   *first_len = i - *first_start;
>           break;
>   case write_gran_256bytes:
>           for (i = 0; i < len / 256; i++) {
>                   limit = min(256, len - i * 256);
>                   /* Are 'have' and 'want' identical? */
>                   if (memcmp(have + i * 256, want + i * 256, limit))
>
> {
>                           if (!result) {
>                                   /* First location where have and
>
> want
>                                    * differ.
>                                    */
>                                   result = 1;
>                                   *first_start = i * 256;
>                           }
>                   } else {
>                           if (result) {
>                                   /* First location where have and
>
> want
>                                    * do not differ anymore.
>                                    */
>                                   *first_len = i * 256 -
>
> *first_start;
>                                   break;
>                           }
>                   }
>           }
>           /* Did the loop terminate without setting first_len? */
>           if (result && ! *first_len)
>                   *first_len = min(i * 256 - *first_start, len);
>           break;
>   default:
>           msg_cerr("%s: Unsupported granularity! Please report a bug
>
> at "
>                    "[e-mail]\n", __func__);
>   }
>   return result;
>
> +}
>
> /* This function generates various test patterns useful for testing
> controller
> and chip communication as well as chip behaviour.
>
> @@ -1203,7 +1298,8 @@
>         return ret;
>  }
>
> -/* This function shares a lot of its structure with erase_flash().
> +/* This function shares a lot of its structure with
> erase_and_write_flash() and
> walk_eraseregions().
> Even if an error is found, the function will keep going and check the
>
> rest.
>  */
>  static int selfcheck_eraseblocks(struct flashchip *flash)
> @@ -1271,10 +1367,67 @@
>         return ret;
>  }
>
> +static int erase_and_write_block_helper(struct flashchip *flash,
>                                   unsigned int start, unsigned int
>
> len,
>                                   uint8_t *oldcontents,
>                                   uint8_t *newcontents,
>                                   int (*erasefn) (struct flashchip
>
> *flash,
>                                                   unsigned int addr,
>                                                   unsigned int len))
>
> +{
>   int starthere = 0;
>   int lenhere = 0;
>   int ret = 0;
>   int skip = 1;
>   int writecount = 0;
>   enum write_granularity gran = write_gran_256bytes; /* FIXME */
>   /* oldcontents and newcontents are opaque to walk_eraseregions, and
>    * need to be adjusted here to keep the impression of proper
>
> abstraction
>    */
>   oldcontents += start;
>   newcontents += start;
>   msg_cdbg(":");
>   /* FIXME: Assume 256 byte granularity for now to play it safe. */
>   if (need_erase(oldcontents, newcontents, len, gran)) {
>           msg_cdbg("E");
>           ret = erasefn(flash, start, len);
>           if (ret)
>                   return ret;
>           /* Erase was successful. Adjust oldcontents. */
>           memset(oldcontents, 0xff, len);
>           skip = 0;
>   }
>   while (get_next_write(oldcontents + starthere,
>                         newcontents + starthere,
>                         len - starthere, &starthere,
>                         &lenhere, gran)) {
>           if (!writecount++)
>                   msg_cdbg("W");
>           /* Needs the partial write function signature. */
>           ret = flash->write(flash, newcontents + starthere,
>                              start + starthere, lenhere);
>           if (ret)
>                   return ret;
>           starthere += lenhere;
>           skip = 0;
>   }
>   if (skip)
>           msg_cdbg("S");
>   return ret;
>
> +}
>
> static int walk_eraseregions(struct flashchip *flash, int erasefunction,
>                             int (*do_something) (struct flashchip *flash,
>                                                  unsigned int addr,
>                                             unsigned int len))
>                                             unsigned int len,
>                                             uint8_t *param1,
>                                             uint8_t *param2,
>                                             int (*erasefn) (
>                                                   struct flashchip
>
> *flash,
>                                                   unsigned int addr,
>                                                   unsigned int len)),
>                        void *param1, void *param2)
>
> {
>        int i, j;
>        unsigned int start = 0;
> @@ -1286,21 +1439,34 @@
>                  */
>                len = eraser.eraseblocks[i].size;
>                for (j = 0; j < eraser.eraseblocks[i].count; j++) {
>                   msg_cdbg("0x%06x-0x%06x, ", start,
>                   /* Print this for every block except the first one.
>
> */
>                   if (i || j)
>                           msg_cdbg(", ");
>                   msg_cdbg("0x%06x-0x%06x", start,
>                               start + len - 1);
>                   if (do_something(flash, start, len))
>                   if (do_something(flash, start, len, param1, param2,
>                                    eraser.block_erase)) {
>                           msg_cdbg("\n");
>                          return 1;
>                   }
>                  start += len;
>          }
>  }
>   msg_cdbg("\n");
>  return 0;
>
> }
>
> -int erase_flash(struct flashchip *flash)
> +int erase_and_write_flash(struct flashchip *flash, uint8_t *oldcontents,
> uint8_t *newcontents)
>  {
>        int k, ret = 0, found = 0;
>   uint8_t *curcontents;
>   unsigned long size = flash->total_size * 1024;
>
>   msg_cinfo("Erasing flash chip... ");
>   curcontents = (uint8_t *) malloc(size);
>   /* Copy oldcontents to curcontents to avoid clobbering oldcontents.
>
> */
>   memcpy(curcontents, oldcontents, size);
>   msg_cinfo("Erasing and writing flash chip... ");
>  for (k = 0; k < NUM_ERASEFUNCTIONS; k++) {
>          struct block_eraser eraser = flash->block_erasers[k];
>
> @@ -1324,12 +1490,19 @@
>                 }
>                found = 1;
>                msg_cdbg("trying... ");
>           ret = walk_eraseregions(flash, k, eraser.block_erase);
>           ret = walk_eraseregions(flash, k,
>
> &erase_and_write_block_helper, curcontents, newcontents);
>                msg_cdbg("\n");
>                /* If everything is OK, don't try another erase function. */
>                if (!ret)
>                        break;
>           /* FIXME: Reread the whole chip here so we know the current
>            * chip contents? curcontents might be up to date, but this
>            * code is only reached if something failed, and then we
>
> don't
>            * know exactly what failed, and how.
>            */
>   }
>   /* Free the scratchpad. */
>   free(curcontents);
>  if (!found) {
>          msg_cerr("ERROR: flashrom has no erase function for this
>
> flash chip.\n");
>                return 1;
> @@ -1338,7 +1511,7 @@
>         if (ret) {
>                msg_cerr("FAILED!\n");
>        } else {
>           msg_cinfo("SUCCESS.\n");
>           msg_cinfo("Done.\n");
>  }
>  return ret;
>
> }
> @@ -1637,6 +1810,19 @@
>                 programmer_shutdown();
>                return ret;
>        }
>   oldcontents = (uint8_t *) malloc(size);
>   /* Assume worst case: All bits are 0. */
>   memset(oldcontents, 0x00, size);
>   newcontents = (uint8_t *) malloc(size);
>   /* Assume best case: All bits should be 1. */
>   memset(newcontents, 0xff, size);
>   /* Side effect of the assumptions above: Default write action is
>
> erase
>    * because newcontents looks like a completely erased chip, and
>    * oldcontents being completely 0x00 means we have to erase
>
> everything
>    * before we can write.
>    */
>  if (erase_it) {
>          /* FIXME: Do we really want the scary warning if erase
>
> failed?
>                 * After all, after erase the chip is either blank or
> partially
> @@ -1644,15 +1830,14 @@
>                  * so if the user wanted erase and reboots afterwards, the
> user
>                 * knows very well that booting won't work.
>                 */
>           if (erase_flash(flash)) {
>           if (erase_and_write_flash(flash, oldcontents, newcontents))
>
> {
>                        emergency_help_message();
>                   programmer_shutdown();
>                   return 1;
>                   ret = 1;
>          }
>           programmer_shutdown();
>           return ret;
>  }
>
>   newcontents = (uint8_t *) calloc(size, sizeof(char));
>  if (write_it || verify_it) {
>          if (read_buf_from_file(newcontents, size, filename)) {
>                  programmer_shutdown();
>
> @@ -1665,8 +1850,6 @@
>  #endif
>        }
>   oldcontents = (uint8_t *) calloc(size, sizeof(char));
>  /* Read the whole chip to be able to check whether regions need to
>
> be
>         * erased and to give better diagnostics in case write fails.
>         * The alternative would be to read only the regions which are to be
> @@ -1686,9 +1869,9 @@
>         // ////////////////////////////////////////////////////////////
>    if (write_it) {
>           if (erase_flash(flash)) {
>                   msg_cerr("Uh oh. Erase failed. Checking if anything
>
> "
>                            "changed.\n");
>           if (erase_and_write_flash(flash, oldcontents, newcontents))
>
> {
>                   msg_cerr("Uh oh. Erase/write failed. Checking if "
>                            "anything changed.\n");
>                  if (!flash->read(flash, newcontents, 0, size)) {
>                          if (!memcmp(oldcontents, newcontents, size))
>
> {
>                                        msg_cinfo("Good. It seems nothing
> was "
> @@ -1702,26 +1885,6 @@
>                         programmer_shutdown();
>                        return 1;
>                }
>           msg_cinfo("Writing flash chip... ");
>           ret = flash->write(flash, newcontents, 0, size);
>           if (ret) {
>                   msg_cerr("Uh oh. Write failed. Checking if anything
>
> "
>                            "changed.\n");
>                   if (!flash->read(flash, newcontents, 0, size)) {
>                           if (!memcmp(oldcontents, newcontents,
>
> size)) {
>                                   msg_cinfo("Good. It seems nothing
>
> was "
>                                             "changed.\n");
>                                   nonfatal_help_message();
>                                   programmer_shutdown();
>                                   return 1;
>                           }
>                   }
>                   emergency_help_message();
>                   programmer_shutdown();
>                   return 1;
>           } else {
>                   msg_cinfo("COMPLETE.\n");
>           }
>  }
>
>  if (verify_it) {
>
>
> flashrom mailing list
> [e-mail]
> http://www.flashrom.org/mailman/listinfo/flashrom