Sol Boucher (solb@chromium.org) just uploaded a new patch set to gerrit, which you can find at http://review.coreboot.org/10134
-gerrit
commit 59851815fd6160242e4a20b4dbe3e91293a4a434 Author: Sol Boucher solb@chromium.org Date: Wed Mar 25 13:40:08 2015 -0700
cbfstool: Restructure around support for reading/writing portions of files
The buffer API that cbfstool uses to read and write files only directly supports one-shot operations on whole files. This adds an intermediate partitioned_file module that sits on top of the buffer system and has an awareness of FMAP entries. It provides an easy way to get a buffer for an individual region of a larger image file based on FMAP section name, as well as incrementally write those smaller buffers back to the backing file at the appropriate offset. The module has two distinct modes of operation: - For new images whose layout is described exclusively by an FMAP section, all the aforementioned functionality will be available. - For images in the current format, where the CBFS master header serves as the root of knowledge of the image's size and layout, the module falls back to a legacy operation mode, where it only allows manipulation of the entire image as one unit, but exposes this support through the same interface by mapping the region named SECTION_NAME_PRIMARY_CBFS ("COREBOOT") to the whole file.
The tool is presently only ported onto the new module running in legacy mode: higher-level support for true "partitioned" images will be forthcoming. However, as part of this change, the crusty cbfs_image_from_file() and cbfs_image_write_file() abstractions are removed and replaced with a single cbfs_image function, cbfs_image_from_buffer(), as well as centralized image reading/writing directly in cbfstool's main() function. This reduces the boilerplate required to implement each new action, makes the create action much more similar to the others, and will make implementing additional actions and adding in support for the new format much easier.
BUG=chromium:470407 TEST=Build panther and nyan_big coreboot.rom images with and without this patch and diff their hexdumps. Ensure that no differences occur at different locations from the diffs between subsequent builds of an identical source tree. Then flash a full new build onto nyan_big and watch it boot normally. BRANCH=None
Change-Id: I25578c7b223bc8434c3074cb0dd8894534f8c500 Signed-off-by: Sol Boucher solb@chromium.org Original-Commit-Id: 7e1c96a48e7a27fc6b90289d35e6e169d5e7ad20 Original-Change-Id: Ia4a1a4c48df42b9ec2d6b9471b3a10eb7b24bb39 Original-Signed-off-by: Sol Boucher solb@chromium.org Original-Reviewed-on: https://chromium-review.googlesource.com/265581 Original-Reviewed-by: Aaron Durbin adurbin@chromium.org --- util/cbfstool/Makefile | 5 +- util/cbfstool/Makefile.inc | 5 + util/cbfstool/cbfs_image.c | 75 ++++---- util/cbfstool/cbfs_image.h | 15 +- util/cbfstool/cbfstool.c | 255 ++++++++++++++------------- util/cbfstool/common.c | 1 + util/cbfstool/common.h | 11 ++ util/cbfstool/flashmap/fmap.c | 5 +- util/cbfstool/flashmap/fmap.h | 3 +- util/cbfstool/partitioned_file.c | 367 +++++++++++++++++++++++++++++++++++++++ util/cbfstool/partitioned_file.h | 174 +++++++++++++++++++ 11 files changed, 742 insertions(+), 174 deletions(-)
diff --git a/util/cbfstool/Makefile b/util/cbfstool/Makefile index 1edb092..4238445 100644 --- a/util/cbfstool/Makefile +++ b/util/cbfstool/Makefile @@ -14,10 +14,13 @@ LDFLAGS += -g3 CBFSTOOL_BINARY:=$(obj)/cbfstool CBFSTOOL_COMMON:=common.o cbfs_image.o compress.o fit.o CBFSTOOL_COMMON+=elfheaders.o cbfs-mkstage.o cbfs-mkpayload.o xdr.o -CBFSTOOL_COMMON+=linux_trampoline.o cbfs-payload-linux.o +CBFSTOOL_COMMON+=partitioned_file.o linux_trampoline.o cbfs-payload-linux.o # LZMA CBFSTOOL_COMMON+=lzma/lzma.o CBFSTOOL_COMMON+=lzma/C/LzFind.o lzma/C/LzmaDec.o lzma/C/LzmaEnc.o +# FMAP +CBFSTOOL_COMMON+=flashmap/fmap.o +CBFSTOOL_COMMON+=flashmap/kv_pair.o flashmap/valstr.o
CBFSTOOL_COMMON:=$(addprefix $(obj)/,$(CBFSTOOL_COMMON))
diff --git a/util/cbfstool/Makefile.inc b/util/cbfstool/Makefile.inc index bf7e1b8..1aa9d76 100644 --- a/util/cbfstool/Makefile.inc +++ b/util/cbfstool/Makefile.inc @@ -8,11 +8,16 @@ cbfsobj += cbfs-mkpayload.o cbfsobj += elfheaders.o cbfsobj += xdr.o cbfsobj += fit.o +cbfsobj += partitioned_file.o # LZMA cbfsobj += lzma.o cbfsobj += LzFind.o cbfsobj += LzmaDec.o cbfsobj += LzmaEnc.o +# FMAP +cbfsobj += fmap.o +cbfsobj += kv_pair.o +cbfsobj += valstr.o # linux as payload cbfsobj += linux_trampoline.o cbfsobj += cbfs-payload-linux.o diff --git a/util/cbfstool/cbfs_image.c b/util/cbfstool/cbfs_image.c index 16dc3dc..d07f806 100644 --- a/util/cbfstool/cbfs_image.c +++ b/util/cbfstool/cbfs_image.c @@ -184,18 +184,22 @@ void cbfs_get_header(struct cbfs_header *header, void *src)
int cbfs_image_create(struct cbfs_image *image, uint32_t architecture, - size_t size, uint32_t align, struct buffer *bootblock, uint32_t bootblock_offset, uint32_t header_offset, uint32_t entries_offset) { + assert(image); + assert(image->buffer.data); + assert(bootblock); + struct cbfs_file *entry; int32_t *rel_offset; uint32_t cbfs_len; - size_t entry_header_len; void *header_loc; + size_t empty_header_len; + size_t size = image->buffer.size;
DEBUG("cbfs_image_create: bootblock=0x%x+0x%zx, " "header=0x%x+0x%zx, entries_offset=0x%x\n", @@ -205,17 +209,13 @@ int cbfs_image_create(struct cbfs_image *image, // This attribute must be given in order to prove that this module // correctly preserves certain CBFS properties. See the block comment // near the top of this file (and the associated commit message). - size_t empty_header_len = cbfs_calculate_file_header_size(""); + empty_header_len = cbfs_calculate_file_header_size(""); if (align < empty_header_len) { ERROR("CBFS must be aligned to at least %zu bytes\n", empty_header_len); return -1; }
- if (buffer_create(&image->buffer, size, "(new)") != 0) - return -1; - memset(image->buffer.data, CBFS_CONTENT_DEFAULT_VALUE, size); - // Adjust legcay top-aligned address to ROM offset. if (IS_TOP_ALIGNED_ADDRESS(entries_offset)) entries_offset = size + (int32_t)entries_offset; @@ -274,10 +274,9 @@ int cbfs_image_create(struct cbfs_image *image, entries_offset, align); return -1; } - entry_header_len = cbfs_calculate_file_header_size(""); - if (entries_offset + entry_header_len > size) { + if (entries_offset + empty_header_len > size) { ERROR("Offset (0x%x+0x%zx) exceed ROM size(0x%zx)\n", - entries_offset, entry_header_len, size); + entries_offset, empty_header_len, size); return -1; } entry = (struct cbfs_file *)(image->buffer.data + entries_offset); @@ -292,34 +291,26 @@ int cbfs_image_create(struct cbfs_image *image, // correctly preserves certain CBFS properties. See the block comment // near the top of this file (and the associated commit message). cbfs_len -= cbfs_len % align; - cbfs_len -= entries_offset + entry_header_len; + cbfs_len -= entries_offset + empty_header_len; cbfs_create_empty_entry(entry, cbfs_len, ""); LOG("Created CBFS image (capacity = %d bytes)\n", cbfs_len); return 0; }
-int cbfs_image_from_file(struct cbfs_image *image, - const char *filename, uint32_t offset) +int cbfs_image_from_buffer(struct cbfs_image *out, struct buffer *in, + uint32_t offset) { - void *header_loc; - - if (buffer_from_file(&image->buffer, filename) != 0) - return -1; - DEBUG("read_cbfs_image: %s (%zd bytes)\n", image->buffer.name, - image->buffer.size); - header_loc = cbfs_find_header(image->buffer.data, - image->buffer.size, - offset); - if (!header_loc) { - ERROR("%s does not have CBFS master header.\n", filename); - cbfs_image_delete(image); - return -1; + assert(out); + assert(in); + assert(in->data); + + buffer_clone(&out->buffer, in); + void *header_loc = cbfs_find_header(in->data, in->size, offset); + if (header_loc) { + cbfs_get_header(&out->header, header_loc); + cbfs_fix_legacy_size(out, header_loc); } - - cbfs_get_header(&image->header, header_loc); - cbfs_fix_legacy_size(image, header_loc); - - return 0; + return !header_loc; }
int cbfs_copy_instance(struct cbfs_image *image, size_t copy_offset, @@ -396,12 +387,6 @@ int cbfs_copy_instance(struct cbfs_image *image, size_t copy_offset, return 0; }
-int cbfs_image_write_file(struct cbfs_image *image, const char *filename) -{ - assert(image && image->buffer.data); - return buffer_write_file(&image->buffer, filename); -} - int cbfs_image_delete(struct cbfs_image *image) { if (image == NULL) @@ -961,12 +946,16 @@ uint32_t cbfs_get_entry_addr(struct cbfs_image *image, struct cbfs_file *entry)
int cbfs_is_valid_entry(struct cbfs_image *image, struct cbfs_file *entry) { - return (entry && - (char *)entry >= image->buffer.data && - (char *)entry + sizeof(entry->magic) < - image->buffer.data + image->buffer.size && - memcmp(entry->magic, CBFS_FILE_MAGIC, - sizeof(entry->magic)) == 0); + uint32_t offset = cbfs_get_entry_addr(image, entry); + + if (offset >= image->buffer.size) + return 0; + + struct buffer entry_data; + buffer_clone(&entry_data, &image->buffer); + buffer_seek(&entry_data, offset); + return buffer_check_magic(&entry_data, CBFS_FILE_MAGIC, + strlen(CBFS_FILE_MAGIC)); }
int cbfs_create_empty_entry(struct cbfs_file *entry, diff --git a/util/cbfstool/cbfs_image.h b/util/cbfstool/cbfs_image.h index 51d06dc..ddee3a7 100644 --- a/util/cbfstool/cbfs_image.h +++ b/util/cbfstool/cbfs_image.h @@ -38,30 +38,29 @@ void cbfs_put_header(void *dest, const struct cbfs_header *header); void cbfs_get_header(struct cbfs_header *header, void *src);
/* Creates an empty CBFS image by given size, and description to its content - * (bootblock, align, header location, starting offset of CBFS entries. + * (bootblock, align, header location, starting offset of CBFS entries). * The output image will contain a valid cbfs_header, with one cbfs_file * entry with type CBFS_COMPONENT_NULL, with max available size. * Returns 0 on success, otherwise none-zero. */ int cbfs_image_create(struct cbfs_image *image, uint32_t arch, - size_t size, uint32_t align, struct buffer *bootblock, uint32_t bootblock_offset, uint32_t header_offset, uint32_t entries_offset);
-/* Loads a CBFS image from file. Returns 0 on success, otherwise non-zero. */ -int cbfs_image_from_file(struct cbfs_image *image, - const char *filename, uint32_t offset); +/* Constructs a cbfs_image from a buffer. The resulting image contains a shallow + * copy of the buffer; releasing either one is the legal way to clean up after + * both of them at once. Always produces a cbfs_image, but... + * Returns 0 if it contains a valid CBFS, non-zero if it's unrecognized data. */ +int cbfs_image_from_buffer(struct cbfs_image *out, struct buffer *in, + uint32_t offset);
/* Create a duplicate CBFS image. Returns 0 on success, otherwise non-zero. */ int cbfs_copy_instance(struct cbfs_image *image, size_t copy_offset, size_t copy_size);
-/* Writes a CBFS image into file. Returns 0 on success, otherwise non-zero. */ -int cbfs_image_write_file(struct cbfs_image *image, const char *filename); - /* Releases the CBFS image. Returns 0 on success, otherwise non-zero. */ int cbfs_image_delete(struct cbfs_image *image);
diff --git a/util/cbfstool/cbfstool.c b/util/cbfstool/cbfstool.c index 024c9cf..a242a94 100644 --- a/util/cbfstool/cbfstool.c +++ b/util/cbfstool/cbfstool.c @@ -29,16 +29,23 @@ #include "common.h" #include "cbfs.h" #include "cbfs_image.h" +#include "cbfs_sections.h" #include "fit.h" +#include "partitioned_file.h"
struct command { const char *name; const char *optstring; int (*function) (void); + // Whether to populate param.image_region before invoking function + bool accesses_region; + // Whether to write that region's contents back to image_file at the end + bool modifies_region; };
static struct param { - char *cbfs_name; + partitioned_file_t *image_file; + struct buffer *image_region; char *name; char *filename; char *bootblock; @@ -74,8 +81,7 @@ static struct param {
typedef int (*convert_buffer_t)(struct buffer *buffer, uint32_t *offset);
-static int cbfs_add_integer_component(const char *cbfs_name, - const char *name, +static int cbfs_add_integer_component(const char *name, uint64_t u64val, uint32_t offset, uint32_t headeroffset) { @@ -94,10 +100,9 @@ static int cbfs_add_integer_component(const char *cbfs_name, for (i = 0; i < 8; i++) buffer.data[i] = (u64val >> i*8) & 0xff;
- if (cbfs_image_from_file(&image, cbfs_name, headeroffset) != 0) { - ERROR("Could not load ROM image '%s'.\n", cbfs_name); - buffer_delete(&buffer); - return 1; + if (cbfs_image_from_buffer(&image, param.image_region, headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); + goto done; }
if (cbfs_get_entry(&image, name)) { @@ -112,26 +117,20 @@ static int cbfs_add_integer_component(const char *cbfs_name, goto done; }
- if (cbfs_image_write_file(&image, cbfs_name) == 0) - ret = 0; + ret = 0;
done: buffer_delete(&buffer); - cbfs_image_delete(&image); return ret; }
-static int cbfs_add_component(const char *cbfs_name, - const char *filename, +static int cbfs_add_component(const char *filename, const char *name, uint32_t type, uint32_t offset, uint32_t headeroffset, convert_buffer_t convert) { - struct cbfs_image image; - struct buffer buffer; - if (!filename) { ERROR("You need to specify -f/--filename.\n"); return 1; @@ -147,44 +146,37 @@ static int cbfs_add_component(const char *cbfs_name, return 1; }
- if (cbfs_image_from_file(&image, cbfs_name, headeroffset)) + struct cbfs_image image; + if (cbfs_image_from_buffer(&image, param.image_region, headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); return 1; + }
+ struct buffer buffer; if (buffer_from_file(&buffer, filename) != 0) { ERROR("Could not load file '%s'.\n", filename); - cbfs_image_delete(&image); return 1; }
if (convert && convert(&buffer, &offset) != 0) { ERROR("Failed to parse file '%s'.\n", filename); buffer_delete(&buffer); - cbfs_image_delete(&image); return 1; }
if (cbfs_get_entry(&image, name)) { ERROR("'%s' already in ROM image.\n", name); buffer_delete(&buffer); - cbfs_image_delete(&image); return 1; }
if (cbfs_add_entry(&image, &buffer, name, type, offset) != 0) { ERROR("Failed to add '%s' into ROM image.\n", filename); buffer_delete(&buffer); - cbfs_image_delete(&image); - return 1; - } - - if (cbfs_image_write_file(&image, cbfs_name) != 0) { - buffer_delete(&buffer); - cbfs_image_delete(&image); return 1; }
buffer_delete(&buffer); - cbfs_image_delete(&image); return 0; }
@@ -222,6 +214,7 @@ static int cbfstool_convert_mkpayload(struct buffer *buffer, /* Not a supported payload type */ if (ret != 0) { ERROR("Not a supported payload type (ELF / FV).\n"); + buffer_delete(buffer); return -1; }
@@ -247,11 +240,9 @@ static int cbfstool_convert_mkflatpayload(struct buffer *buffer, return 0; }
- static int cbfs_add(void) { - return cbfs_add_component(param.cbfs_name, - param.filename, + return cbfs_add_component(param.filename, param.name, param.type, param.baseaddress, @@ -261,8 +252,7 @@ static int cbfs_add(void)
static int cbfs_add_stage(void) { - return cbfs_add_component(param.cbfs_name, - param.filename, + return cbfs_add_component(param.filename, param.name, CBFS_COMPONENT_STAGE, param.baseaddress, @@ -272,8 +262,7 @@ static int cbfs_add_stage(void)
static int cbfs_add_payload(void) { - return cbfs_add_component(param.cbfs_name, - param.filename, + return cbfs_add_component(param.filename, param.name, CBFS_COMPONENT_PAYLOAD, param.baseaddress, @@ -293,8 +282,7 @@ static int cbfs_add_flat_binary(void) "-e/--entry-point.\n"); return 1; } - return cbfs_add_component(param.cbfs_name, - param.filename, + return cbfs_add_component(param.filename, param.name, CBFS_COMPONENT_PAYLOAD, param.baseaddress, @@ -304,8 +292,7 @@ static int cbfs_add_flat_binary(void)
static int cbfs_add_integer(void) { - return cbfs_add_integer_component(param.cbfs_name, - param.name, + return cbfs_add_integer_component(param.name, param.u64val, param.baseaddress, param.headeroffset); @@ -313,46 +300,35 @@ static int cbfs_add_integer(void)
static int cbfs_remove(void) { - struct cbfs_image image; - if (!param.name) { ERROR("You need to specify -n/--name.\n"); return 1; }
- if (cbfs_image_from_file(&image, param.cbfs_name, param.headeroffset)) + struct cbfs_image image; + if (cbfs_image_from_buffer(&image, param.image_region, + param.headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); return 1; + }
if (cbfs_remove_entry(&image, param.name) != 0) { ERROR("Removing file '%s' failed.\n", param.name); - cbfs_image_delete(&image); - return 1; - } - if (cbfs_image_write_file(&image, param.cbfs_name) != 0) { - cbfs_image_delete(&image); return 1; }
- cbfs_image_delete(&image); return 0; }
static int cbfs_create(void) { - struct cbfs_image image; - struct buffer bootblock; - - if (param.size == 0) { - ERROR("You need to specify a valid -s/--size.\n"); - return 1; - } - if (param.arch == CBFS_ARCHITECTURE_UNKNOWN) { ERROR("You need to specify -m/--machine arch.\n"); return 1; }
+ struct buffer bootblock; if (!param.bootblock) { DEBUG("-B not given, creating image without bootblock.\n"); buffer_create(&bootblock, 0, "(dummy)"); @@ -403,34 +379,29 @@ static int cbfs_create(void) } }
+ struct cbfs_image image; + if (!cbfs_image_from_buffer(&image, param.image_region, -1)) + // It *already* contains a CBFS?! This should be a blank file. + assert(false); + if (cbfs_image_create(&image, param.arch, - param.size, param.alignment, &bootblock, param.baseaddress, param.headeroffset, param.cbfsoffset) != 0) { - ERROR("Failed to create %s.\n", param.cbfs_name); + ERROR("Failed to initialize CBFS structure.\n"); + buffer_delete(&bootblock); return 1; } - buffer_delete(&bootblock);
- if (cbfs_image_write_file(&image, param.cbfs_name) != 0) { - ERROR("Failed to write %s.\n", param.cbfs_name); - cbfs_image_delete(&image); - return 1; - } - cbfs_image_delete(&image); + buffer_delete(&bootblock); return 0; }
static int cbfs_locate(void) { - struct cbfs_image image; - struct buffer buffer; - int32_t address; - if (!param.filename) { ERROR("You need to specify -f/--filename.\n"); return 1; @@ -441,33 +412,35 @@ static int cbfs_locate(void) return 1; }
- if (cbfs_image_from_file(&image, param.cbfs_name, param.headeroffset)) + struct cbfs_image image; + if (cbfs_image_from_buffer(&image, param.image_region, + param.headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); return 1; + }
if (cbfs_get_entry(&image, param.name)) WARN("'%s' already in CBFS.\n", param.name);
+ struct buffer buffer; if (buffer_from_file(&buffer, param.filename) != 0) { ERROR("Cannot load %s.\n", param.filename); - cbfs_image_delete(&image); return 1; }
- address = cbfs_locate_entry(&image, param.name, buffer.size, + int32_t address = cbfs_locate_entry(&image, param.name, buffer.size, param.pagesize, param.alignment); buffer_delete(&buffer);
if (address == -1) { ERROR("'%s' can't fit in CBFS for page-size %#x, align %#x.\n", param.name, param.pagesize, param.alignment); - cbfs_image_delete(&image); return 1; }
if (param.top_aligned) address = address - image.header.romsize;
- cbfs_image_delete(&image); printf("0x%x\n", address); return 0; } @@ -475,19 +448,18 @@ static int cbfs_locate(void) static int cbfs_print(void) { struct cbfs_image image; - if (cbfs_image_from_file(&image, param.cbfs_name, param.headeroffset)) + if (cbfs_image_from_buffer(&image, param.image_region, + param.headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); return 1; + }
cbfs_print_directory(&image); - cbfs_image_delete(&image); return 0; }
static int cbfs_extract(void) { - int result = 0; - struct cbfs_image image; - if (!param.filename) { ERROR("You need to specify -f/--filename.\n"); return 1; @@ -498,21 +470,18 @@ static int cbfs_extract(void) return 1; }
- if (cbfs_image_from_file(&image, param.cbfs_name, param.headeroffset)) - result = 1; - else if (cbfs_export_entry(&image, param.name, - param.filename)) - result = 1; + struct cbfs_image image; + if (cbfs_image_from_buffer(&image, param.image_region, + param.headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); + return 1; + }
- cbfs_image_delete(&image); - return result; + return cbfs_export_entry(&image, param.name, param.filename); }
static int cbfs_update_fit(void) { - int ret = 0; - struct cbfs_image image; - if (!param.name) { ERROR("You need to specify -n/--name.\n"); return 1; @@ -524,21 +493,18 @@ static int cbfs_update_fit(void) return 1; }
- if (cbfs_image_from_file(&image, param.cbfs_name, param.headeroffset)) + struct cbfs_image image; + if (cbfs_image_from_buffer(&image, param.image_region, + param.headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); return 1; + }
- ret = fit_update_table(&image, param.fit_empty_entries, param.name); - if (!ret) - ret = cbfs_image_write_file(&image, param.cbfs_name); - - cbfs_image_delete(&image); - return ret; + return fit_update_table(&image, param.fit_empty_entries, param.name); }
static int cbfs_copy(void) { - struct cbfs_image image; - if (!param.copyoffset_assigned) { ERROR("You need to specify -D/--copy_offset.\n"); return 1; @@ -549,31 +515,30 @@ static int cbfs_copy(void) return 1; }
- if (cbfs_image_from_file(&image, param.cbfs_name, - param.headeroffset) != 0) - return 1; - - if (cbfs_copy_instance(&image, param.copyoffset, param.size)) + struct cbfs_image image; + if (cbfs_image_from_buffer(&image, param.image_region, + param.headeroffset)) { + ERROR("Selected image region is not a CBFS.\n"); return 1; + }
- /* Save the new image. */ - return buffer_write_file(&image.buffer, param.cbfs_name); - + return cbfs_copy_instance(&image, param.copyoffset, param.size); }
static const struct command commands[] = { - {"add", "H;f:n:t:b:vh?", cbfs_add}, - {"add-flat-binary", "H:f:n:l:e:c:b:vh?", cbfs_add_flat_binary}, - {"add-payload", "H:f:n:t:c:b:vh?C:I:", cbfs_add_payload}, - {"add-stage", "H:f:n:t:c:b:S:vh?", cbfs_add_stage}, - {"add-int", "H:i:n:b:vh?", cbfs_add_integer}, - {"create", "s:B:b:H:a:o:m:vh?", cbfs_create}, - {"copy", "H:D:s:", cbfs_copy}, - {"extract", "H:n:f:vh?", cbfs_extract}, - {"locate", "H:f:n:P:a:Tvh?", cbfs_locate}, - {"print", "H:vh?", cbfs_print}, - {"remove", "H:n:vh?", cbfs_remove}, - {"update-fit", "H:n:x:vh?", cbfs_update_fit}, + {"add", "H:f:n:t:b:vh?", cbfs_add, true, true}, + {"add-flat-binary", "H:f:n:l:e:c:b:vh?", cbfs_add_flat_binary, true, + true}, + {"add-payload", "H:f:n:t:c:b:vh?C:I:", cbfs_add_payload, true, true}, + {"add-stage", "H:f:n:t:c:b:S:vh?", cbfs_add_stage, true, true}, + {"add-int", "H:i:n:b:vh?", cbfs_add_integer, true, true}, + {"copy", "H:D:s:", cbfs_copy, true, true}, + {"create", "s:B:b:H:a:o:m:vh?", cbfs_create, true, true}, + {"extract", "H:n:f:vh?", cbfs_extract, true, false}, + {"locate", "H:f:n:P:a:Tvh?", cbfs_locate, true, false}, + {"print", "H:vh?", cbfs_print, true, false}, + {"remove", "H:n:vh?", cbfs_remove, true, true}, + {"update-fit", "H:n:x:vh?", cbfs_update_fit, true, true}, };
static struct option long_options[] = { @@ -666,7 +631,7 @@ int main(int argc, char **argv) return 1; }
- param.cbfs_name = argv[1]; + char *image_name = argv[1]; char *cmd = argv[2]; optind += 2;
@@ -793,7 +758,59 @@ int main(int argc, char **argv) } }
- return commands[i].function(); + if (commands[i].function == cbfs_create) { + if (param.size == 0) { + ERROR("You need to specify a valid -s/--size.\n"); + return 1; + } + param.image_file = partitioned_file_create_flat( + image_name, param.size); + } else { + param.image_file = + partitioned_file_reopen(image_name, + partitioned_file_open_as_flat); + } + if (!param.image_file) + return 1; + + // If the action needs to read an image region, as indicated by + // having accesses_region set in its command struct, that + // region's buffer struct will be stored here and the client + // will receive a pointer to it via param.image_region. It + // need not write the buffer back to the image file itself, + // since this behavior can be requested via its modifies_region + // field. Additionally, it should never free the region buffer, + // as that is performed automatically once it completes. + struct buffer image_region; + memset(&image_region, 0, sizeof(image_region)); + + if (commands[i].accesses_region) { + assert(param.image_file); + + if (!partitioned_file_read_region(&image_region, + param.image_file, SECTION_NAME_PRIMARY_CBFS)) { + partitioned_file_close(param.image_file); + return 1; + } + param.image_region = &image_region; + } + + int error = commands[i].function(); + + if (!error && commands[i].modifies_region) { + assert(param.image_file); + assert(commands[i].accesses_region); + + if (!partitioned_file_write_region(param.image_file, + &image_region)) { + partitioned_file_close(param.image_file); + return 1; + } + } + + partitioned_file_close(param.image_file); + + return error; }
ERROR("Unknown command '%s'.\n", cmd); diff --git a/util/cbfstool/common.c b/util/cbfstool/common.c index cdc04f3..8773fe4 100644 --- a/util/cbfstool/common.c +++ b/util/cbfstool/common.c @@ -87,6 +87,7 @@ int buffer_from_file(struct buffer *buffer, const char *filename) if (fread(buffer->data, 1, buffer->size, fp) != buffer->size) { fprintf(stderr, "incomplete read: %s\n", filename); fclose(fp); + buffer_delete(buffer); return -1; } fclose(fp); diff --git a/util/cbfstool/common.h b/util/cbfstool/common.h index 0cf6b6e..416d0a4 100644 --- a/util/cbfstool/common.h +++ b/util/cbfstool/common.h @@ -20,8 +20,10 @@ #ifndef __CBFSTOOL_COMMON_H #define __CBFSTOOL_COMMON_H
+#include <stdbool.h> #include <stddef.h> #include <stdint.h> +#include <string.h> #include <assert.h>
/* Endianess */ @@ -121,6 +123,15 @@ static inline void buffer_seek(struct buffer *b, size_t size) b->data += size; }
+/* Returns whether the buffer begins with the specified magic bytes. */ +static inline bool buffer_check_magic(const struct buffer *b, const char *magic, + size_t magic_len) +{ + assert(magic); + return b && b->size >= magic_len && + memcmp(b->data, magic, magic_len) == 0; +} + /* Creates an empty memory buffer with given size. * Returns 0 on success, otherwise non-zero. */ int buffer_create(struct buffer *buffer, size_t size, const char *name); diff --git a/util/cbfstool/flashmap/fmap.c b/util/cbfstool/flashmap/fmap.c index 4a8e534..5438d49 100644 --- a/util/cbfstool/flashmap/fmap.c +++ b/util/cbfstool/flashmap/fmap.c @@ -304,10 +304,11 @@ int fmap_append_area(struct fmap **fmap, return new_size; }
-struct fmap_area *fmap_find_area(struct fmap *fmap, const char *name) +const struct fmap_area *fmap_find_area(const struct fmap *fmap, + const char *name) { int i; - struct fmap_area *area = NULL; + const struct fmap_area *area = NULL;
if (!fmap || !name) return NULL; diff --git a/util/cbfstool/flashmap/fmap.h b/util/cbfstool/flashmap/fmap.h index 22accf3..30ceb29 100644 --- a/util/cbfstool/flashmap/fmap.h +++ b/util/cbfstool/flashmap/fmap.h @@ -166,7 +166,8 @@ extern int fmap_append_area(struct fmap **fmap, * returns a pointer to the entry in the fmap structure if successful * returns NULL to indicate failure or if no matching area entry is found */ -extern struct fmap_area *fmap_find_area(struct fmap *fmap, const char *name); +extern const struct fmap_area *fmap_find_area(const struct fmap *fmap, + const char *name);
/* unit testing stuff */ extern int fmap_test(void); diff --git a/util/cbfstool/partitioned_file.c b/util/cbfstool/partitioned_file.c new file mode 100644 index 0000000..6473963 --- /dev/null +++ b/util/cbfstool/partitioned_file.c @@ -0,0 +1,367 @@ +/* + * partitioned_file.c, read and write binary file "partitions" described by FMAP + * + * Copyright (C) 2015 Google, Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA, 02110-1301 USA + */ + +#include "partitioned_file.h" + +#include "cbfs_sections.h" + +#include <assert.h> +#include <stdlib.h> +#include <string.h> + +struct partitioned_file { + struct fmap *fmap; + struct buffer buffer; + FILE *stream; +}; + +static bool fill_ones_through(struct partitioned_file *file) +{ + assert(file); + + memset(file->buffer.data, 0xff, file->buffer.size); + return partitioned_file_write_region(file, &file->buffer); +} + +static unsigned count_selected_fmap_entries(const struct fmap *fmap, + partitioned_file_fmap_selector_t callback, const void *arg) +{ + assert(fmap); + assert(callback); + + unsigned count = 0; + for (unsigned index = 0; index < fmap->nareas; ++index) { + if (callback(fmap->areas + index, arg)) + ++count; + } + return count; +} + +static partitioned_file_t *reopen_flat_file(const char *filename) +{ + assert(filename); + + struct partitioned_file *file = calloc(1, sizeof(*file)); + if (!file) { + ERROR("Failed to allocate partitioned file structure\n"); + return NULL; + } + + if (buffer_from_file(&file->buffer, filename)) { + free(file); + return NULL; + } + + file->stream = fopen(filename, "rb+"); + if (!file->stream) { + perror(filename); + partitioned_file_close(file); + return NULL; + } + + return file; +} + +partitioned_file_t *partitioned_file_create_flat(const char *filename, + size_t image_size) +{ + assert(filename); + + struct partitioned_file *file = calloc(1, sizeof(*file)); + if (!file) { + ERROR("Failed to allocate partitioned file structure\n"); + return NULL; + } + + file->stream = fopen(filename, "wb"); + if (!file->stream) { + perror(filename); + free(file); + return NULL; + } + + if (buffer_create(&file->buffer, image_size, filename)) { + partitioned_file_close(file); + return NULL; + } + + if (!fill_ones_through(file)) { + partitioned_file_close(file); + return NULL; + } + + return file; +} + +partitioned_file_t *partitioned_file_create(const char *filename, + struct buffer *flashmap) +{ + assert(filename); + assert(flashmap); + assert(flashmap->data); + + if (fmap_find((const uint8_t *)flashmap->data, flashmap->size) != 0) { + ERROR("Attempted to create a partitioned image out of something that isn't an FMAP\n"); + return NULL; + } + struct fmap *bootstrap_fmap = (struct fmap *)flashmap->data; + + const struct fmap_area *fmap_area = + fmap_find_area(bootstrap_fmap, SECTION_NAME_FMAP); + if (!fmap_area) { + ERROR("Provided FMAP missing '%s' region\n", SECTION_NAME_FMAP); + return NULL; + } + + if (count_selected_fmap_entries(bootstrap_fmap, + partitioned_file_fmap_select_children_of, fmap_area)) { + ERROR("Provided FMAP's '%s' region contains other regions\n", + SECTION_NAME_FMAP); + return NULL; + } + + int fmap_len = fmap_size(bootstrap_fmap); + if (fmap_len < 0) { + ERROR("Unable to determine size of provided FMAP\n"); + return NULL; + } + assert((size_t)fmap_len <= flashmap->size); + if ((uint32_t)fmap_len > fmap_area->size) { + ERROR("Provided FMAP's '%s' region needs to be at least %d bytes\n", + SECTION_NAME_FMAP, fmap_len); + return NULL; + } + + partitioned_file_t *file = partitioned_file_create_flat(filename, + bootstrap_fmap->size); + if (!file) + return NULL; + + struct buffer fmap_region; + buffer_splice(&fmap_region, &file->buffer, fmap_area->offset, fmap_area->size); + memcpy(fmap_region.data, bootstrap_fmap, fmap_len); + if (!partitioned_file_write_region(file, &fmap_region)) { + partitioned_file_close(file); + return NULL; + } + file->fmap = (struct fmap *)(file->buffer.data + fmap_area->offset); + + return file; +} + +partitioned_file_t *partitioned_file_reopen(const char *filename, + partitioned_file_flat_decider_t flat_override) +{ + assert(filename); + + partitioned_file_t *file = reopen_flat_file(filename); + if (!file) + return NULL; + + if (flat_override && flat_override(&file->buffer)) { + INFO("Opening image as a flat file in response to explicit request\n"); + return file; + } + + long fmap_region_offset = fmap_find((const uint8_t *)file->buffer.data, + file->buffer.size); + if (fmap_region_offset < 0) { + INFO("Opening image as a flat file because it doesn't contain any FMAP\n"); + return file; + } + file->fmap = (struct fmap *)(file->buffer.data + fmap_region_offset); + + if (file->fmap->size > file->buffer.size) { + int fmap_region_size = fmap_size(file->fmap); + ERROR("FMAP records image size as %u, but file is only %zu bytes%s\n", + file->fmap->size, file->buffer.size, + fmap_region_offset == 0 && + (signed)file->buffer.size == fmap_region_size ? + " (is it really an image, or *just* an FMAP?)" : + " (did something truncate this file?)"); + partitioned_file_close(file); + return NULL; + } + + const struct fmap_area *fmap_fmap_entry = + fmap_find_area(file->fmap, SECTION_NAME_FMAP); + if (fmap_fmap_entry->offset != fmap_region_offset) { + ERROR("FMAP's '%s' section doesn't point back to FMAP start (did something corrupt this file?)\n", + SECTION_NAME_FMAP); + partitioned_file_close(file); + return NULL; + } + + return file; +} + +bool partitioned_file_write_region(partitioned_file_t *file, + const struct buffer *buffer) +{ + assert(file); + assert(file->stream); + assert(buffer); + assert(buffer->data); + + if (buffer->data - buffer->offset != file->buffer.data) { + ERROR("Attempted to write a partition buffer back to a different file than it came from\n"); + return false; + } + if (buffer->offset + buffer->size > file->buffer.size) { + ERROR("Attempted to write data off the end of image file\n"); + return false; + } + + if (fseek(file->stream, buffer->offset, SEEK_SET)) { + ERROR("Failed to seek within image file\n"); + return false; + } + if (!fwrite(buffer->data, buffer->size, 1, file->stream)) { + ERROR("Failed to write to image file\n"); + return false; + } + return true; +} + +bool partitioned_file_read_region(struct buffer *dest, + const partitioned_file_t *file, const char *region) +{ + assert(dest); + assert(file); + assert(file->buffer.data); + assert(region); + + if (file->fmap) { + const struct fmap_area *area = fmap_find_area(file->fmap, + region); + if (!area) { + ERROR("Image is missing '%s' region\n", region); + return false; + } + if (area->offset + area->size > file->buffer.size) { + ERROR("Region '%s' runs off the end of the image file\n", + region); + return false; + } + buffer_splice(dest, &file->buffer, area->offset, area->size); + } else { + if (strcmp(region, SECTION_NAME_PRIMARY_CBFS) != 0) { + ERROR("This is a legacy image that contains only a CBFS\n"); + return false; + } + buffer_clone(dest, &file->buffer); + } + + return true; +} + +void partitioned_file_close(partitioned_file_t *file) +{ + if (!file) + return; + + file->fmap = NULL; + buffer_delete(&file->buffer); + if (file->stream) { + fclose(file->stream); + file->stream = NULL; + } + free(file); +} + +bool partitioned_file_is_partitioned(const partitioned_file_t *file) +{ + return partitioned_file_get_fmap(file) != NULL; +} + +bool partitioned_file_region_check_magic(const partitioned_file_t *file, + const char *region, const char *magic, size_t magic_len) +{ + struct buffer area; + return partitioned_file_read_region(&area, file, region) && + buffer_check_magic(&area, magic, magic_len); +} + +bool partitioned_file_region_contains_nested(const partitioned_file_t *file, + const char *region) +{ + assert(file); + assert(region); + + if (!file->fmap) + return false; + const struct fmap_area *area = fmap_find_area(file->fmap, region); + return area && partitioned_file_fmap_count(file, + partitioned_file_fmap_select_children_of, area); +} + +const struct fmap *partitioned_file_get_fmap(const partitioned_file_t *file) +{ + assert(file); + + return file->fmap; +} + +unsigned partitioned_file_fmap_count(const partitioned_file_t *file, + partitioned_file_fmap_selector_t callback, const void *arg) +{ + assert(file); + assert(callback); + + if (!file->fmap) + return 0; + return count_selected_fmap_entries(file->fmap, callback, arg); +} + +static bool select_all(unused const struct fmap_area *area, + unused const void *arg) +{ + return true; +} +const partitioned_file_fmap_selector_t partitioned_file_fmap_select_all = + select_all; + +static bool select_children_of(const struct fmap_area *child, const void *arg) +{ + assert(child); + assert(arg); + + const struct fmap_area *parent = (const struct fmap_area *)arg; + if (child == arg || (child->offset == parent->offset && + child->size == parent->size)) + return false; + return child->offset >= parent->offset && + child->offset + child->size <= parent->offset + parent->size; +} +const partitioned_file_fmap_selector_t + partitioned_file_fmap_select_children_of = select_children_of; + +static bool select_parents_of(const struct fmap_area *parent, const void *arg) +{ + return select_children_of((const struct fmap_area *)arg, parent); +} +const partitioned_file_fmap_selector_t partitioned_file_fmap_select_parents_of = + select_parents_of; + +static bool open_as_flat(unused struct buffer *buffer) +{ + return true; +} +const partitioned_file_flat_decider_t partitioned_file_open_as_flat = + open_as_flat; diff --git a/util/cbfstool/partitioned_file.h b/util/cbfstool/partitioned_file.h new file mode 100644 index 0000000..97d1b57 --- /dev/null +++ b/util/cbfstool/partitioned_file.h @@ -0,0 +1,174 @@ +/* + * partitioned_file.h, read and write binary file "partitions" described by FMAP + * + * Copyright (C) 2015 Google, Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA, 02110-1301 USA + */ + +#ifndef PARITITONED_FILE_H_ +#define PARITITONED_FILE_H_ + +#include "common.h" +#include "flashmap/fmap.h" + +#include <stdbool.h> +#include <stddef.h> + +typedef struct partitioned_file partitioned_file_t; + +/** @return Whether the specific existing file should be opened in flat mode. */ +typedef bool (*partitioned_file_flat_decider_t)(struct buffer *buffer); + +/** Pass to partitioned_file_reopen() to force opening as a partitioned file. */ +#define partitioned_file_open_as_partitioned NULL + +/** Pass to partitioned_file_reopen() to force opening as a flat file. */ +extern const partitioned_file_flat_decider_t partitioned_file_open_as_flat; + +/** + * Create a new filesystem-backed flat buffer. + * This backwards-compatibility function creates a new in-memory buffer and + * backing binary file of the specified size. Although the file won't actually + * have multiple regions, it'll still be possible to access and manipulate it + * using this module; this is accomplished by requesting the special region + * whose name matches SECTION_NAME_PRIMARY_CBFS, which maps to the whole file. + * Note that the caller will be responsible for calling partitioned_file_close() + * on the returned object, and that this function will overwrite any existing + * file with the given name without warning. + * + * @param filename Name of the backing file + * @param image_size Size of the image + * @return Caller-owned partitioned file, or NULL on error + */ +partitioned_file_t *partitioned_file_create_flat(const char *filename, + size_t image_size); + +/** + * Create a new filesystem-backed partitioned buffer. + * This creates a new in-memory buffer and backing binary file. Both are + * segmented into regions according to the provided flashmap's sections, and the + * flashmap itself is automatically copied into the region named + * SECTION_NAME_FMAP: a section with this name must already exist in the FMAP. + * After calling this function, it is safe for the caller to clean up flashmap + * at any time. The partitioned_file_t returned from this function is separately + * owned by the caller, and must later be passed to partitioned_file_close(). + * Note that this function will overwrite any existing file with the given name + * without warning. + * + * @param filename Name of the backing file + * @param flashmap Buffer containing an FMAP file layout + * @return Caller-owned partitioned file, or NULL on error + */ +partitioned_file_t *partitioned_file_create(const char *filename, + struct buffer *flashmap); + +/** + * Read a file back in from the disk. + * An in-memory buffer is created and populated with the file's contents. If + * flat_override is NULL and the image contains an FMAP, it will be opened as a + * full partitioned file; otherwise, it will be opened as a flat file as if it + * had been created by partitioned_file_create_flat(). This selection behavior + * is extensible: if a flat_override function is provided, it is invoked before + * searching for an FMAP, and has the option of explicitly instructing the + * module to open the image as a flat file based on its contents. + * The partitioned_file_t returned from this function is separately owned by the + * caller, and must later be passed to partitioned_file_close(); + * + * @param filename Name of the file to read in + * @param flat_override Callback that can decide to open it as flat, or NULL + * @return Caller-owned partitioned file, or NULL on error + */ +partitioned_file_t *partitioned_file_reopen(const char *filename, + partitioned_file_flat_decider_t flat_override); + +/** + * Write a buffer's contents to its original region within a segmented file. + * This function should only be called on buffers originally retrieved by a call + * to partitioned_file_read_region() on the same partitioned file object. The + * contents of this buffer are copied back to the same region of the buffer and + * backing file that the region occupied before. + * + * @param file Partitioned file to which to write the data + * @param buffer Modified buffer obtained from partitioned_file_read_region() + * @return Whether the operation was successful + */ +bool partitioned_file_write_region(partitioned_file_t *file, + const struct buffer *buffer); + +/** + * Obtain one particular region of a segmented file. + * The result is owned by the partitioned_file_t and shared among every caller + * of this function. Thus, it is an error to buffer_delete() it; instead, clean + * up the entire partitioned_file_t once it's no longer needed with a single + * call to partitioned_file_close(). + * Note that, if the buffer obtained from this function is modified, the changes + * will be reflected in any buffers handed out---whether earlier or later---for + * any region inclusive of the altered location(s). However, the backing file + * will not be updated until someone calls partitioned_file_write_region() on a + * buffer that includes the alterations. + * + * @param dest Empty destination buffer for the data + * @param file Partitioned file from which to read the data + * @param region Name of the desired FMAP region + * @return Whether the copy was performed successfully + */ +bool partitioned_file_read_region(struct buffer *dest, + const partitioned_file_t *file, const char *region); + +/** @param file Partitioned file to flush and cleanup */ +void partitioned_file_close(partitioned_file_t *file); + +/** @return Whether the file is partitioned (i.e. not flat). */ +bool partitioned_file_is_partitioned(const partitioned_file_t *file); + +/** @return Whether the specified region begins with the magic bytes. */ +bool partitioned_file_region_check_magic(const partitioned_file_t *file, + const char *region, const char *magic, size_t magic_len); + +/** @return Whether the specified region exists and contains nested regions. */ +bool partitioned_file_region_contains_nested(const partitioned_file_t *file, + const char *region); + +/** @return An immutable reference to the FMAP, or NULL for flat images. */ +const struct fmap *partitioned_file_get_fmap(const partitioned_file_t *file); + +/** @return Whether to include area in the running count. */ +typedef bool (*partitioned_file_fmap_selector_t) + (const struct fmap_area *area, const void *arg); + +/** + * Count the number of FMAP entries fulfilling a certain criterion. + * The result is always 0 if run on a flat (non-partitioned) image. + * + * @param file File on whose FMAP entries the operation should be run + * @param callback Decider answering whether each individual region should count + * @param arg Additional information to furnish to the decider on each call + * @return The number of FMAP sections with that property + */ +unsigned partitioned_file_fmap_count(const partitioned_file_t *file, + partitioned_file_fmap_selector_t callback, const void *arg); + +/** Selector that counts every single FMAP section. */ +extern const partitioned_file_fmap_selector_t partitioned_file_fmap_select_all; + +/** Selector that counts FMAP sections that are descendants of fmap_area arg. */ +extern const partitioned_file_fmap_selector_t + partitioned_file_fmap_select_children_of; + +/** Selector that counts FMAP sections that contain the fmap_area arg. */ +extern const partitioned_file_fmap_selector_t + partitioned_file_fmap_select_parents_of; + +#endif