Ronald G. Minnich (rminnich@gmail.com) just uploaded a new patch set to gerrit, which you can find at http://review.coreboot.org/4262
-gerrit
commit 486fd50028d41c2415a7e9ebf05c447689d0080a Author: Ronald G. Minnich rminnich@google.com Date: Fri Nov 22 09:28:48 2013 -0800
cbfs: fix issues with word size and endianness.
Add xdr functions and use them to convert the ELF headers to native headers, using the Elf64 structs to ensure we accomodate all word sizes.
This may seem overly complex but it turned out to be much the easiest way to do this. Note that the basic elf parsing function in cbfs-mkstage.c now works over all ELF files.
Change-Id: I3457b1782f27cb930a2101d1c53f58e2488ed8a1 Signed-off-by: Ronald G. Minnich rminnich@google.com --- util/cbfstool/Makefile | 2 +- util/cbfstool/Makefile.inc | 1 + util/cbfstool/cbfs-mkstage.c | 293 +++++++++++++++++++++++++++++++++++-------- util/cbfstool/common.h | 11 ++ util/cbfstool/xdr.c | 141 +++++++++++++++++++++ 5 files changed, 393 insertions(+), 55 deletions(-)
diff --git a/util/cbfstool/Makefile b/util/cbfstool/Makefile index 3aa5edc..281d75c 100644 --- a/util/cbfstool/Makefile +++ b/util/cbfstool/Makefile @@ -7,7 +7,7 @@ CFLAGS += -D_7ZIP_ST BINARY:=$(obj)/cbfstool
COMMON:=cbfstool.o common.o cbfs_image.o compress.o fit.o -COMMON+=cbfs-mkstage.o cbfs-mkpayload.o +COMMON+=cbfs-mkstage.o cbfs-mkpayload.o xdr.o # LZMA COMMON+=lzma/lzma.o COMMON+=lzma/C/LzFind.o lzma/C/LzmaDec.o lzma/C/LzmaEnc.o diff --git a/util/cbfstool/Makefile.inc b/util/cbfstool/Makefile.inc index fc120f3..4270d75 100644 --- a/util/cbfstool/Makefile.inc +++ b/util/cbfstool/Makefile.inc @@ -5,6 +5,7 @@ cbfsobj += compress.o cbfsobj += cbfs_image.o cbfsobj += cbfs-mkstage.o cbfsobj += cbfs-mkpayload.o +cbfsobj += xdr.o cbfsobj += fit.o # LZMA cbfsobj += lzma.o diff --git a/util/cbfstool/cbfs-mkstage.c b/util/cbfstool/cbfs-mkstage.c index dfc93c2..4d74522 100644 --- a/util/cbfstool/cbfs-mkstage.c +++ b/util/cbfstool/cbfs-mkstage.c @@ -28,90 +28,270 @@ #include "cbfs.h" #include "elf.h"
-static unsigned int idemp(unsigned int x) +/* + * Short form: this is complicated, but we've tried making it simple + * and we keep hitting problems with our ELF parsing. + * + * The ELF parsing situation has always been a bit tricky. In fact, + * we (and most others) have been getting it wrong in small ways for + * years. Recently this has caused real trouble for the ARM V8 build. + * In this file we attempt to finally get it right for all variations + * of endian-ness and word size and target architectures and + * architectures we might get run on. Phew!. To do this we borrow a + * page from the FreeBSD NFS xdr model (see elf_ehdr and elf_phdr), + * the Plan 9 endianness functions (see xdr.c), and Go interfaces (see + * how we use buffer structs in this file). This ends up being a bit + * wordy at the lowest level, but greatly simpplifies the elf parsing + * code aad removes a common source of bugs, namely, forgetting to + * flip type endianness when referencing a struct member. + * + * ELF files can have four combinations of data layout: 32/64, and + * big/little endian. Further, to add to the fun, depending on the + * word size, the size of the ELF structs varies. The coreboot SELF + * format is simpler in theory: it's supposed to be always BE, and the + * various struct members allow room for growth: the entry point is + * always 64 bits, for example, so the size of a SELF struct is + * constant, regardless of target architecture word size. Hence, we + * need to do some transformation of the ELF files. + * + * A given architecture, realistically, only supports one of the four + * combinations at a time as the 'native' formst. Hence, our code has + * been sprinkled with every variation of [nh]to[hn][sll] over the + * years. We've never quite gotten it all right, however, and a quick + * pass over this code revealed another bug. It's all worked because, + * until now, all the working platforms that had CBFS were 32 LE. Even then, + * however, bugs crept in: we recently realized that we're not + * transforming the entry point to big format when we store into the + * SELF image. + * + * The problem is essentially an XDR operation: + * we have something in a foreign format and need to transform it. + * It's most like XDR because: + * 1) the byte order can be wrong + * 2) the word size can be wrong + * 3) the size of elements in the stream depends on the value + * of other elements in the stream + * it's not like XDR because: + * 1) the byte order can be right + * 2) the word size can be right + * 3) the struct members are all on a natural alignment + * + * Hence, this new approach. + * To cover word size issues, we *always* transform the two structs we care about, + * the file header and program header, into a native struct in the 64 bit format: + * [32,little] -> [Elf64_Ehdr, Elf64_Phdr] + * [64,little] -> [Elf64_Ehdr, Elf64_Phdr] + * [32,big] -> [Elf64_Ehdr, Elf64_Phdr] + * [64,big] -> [Elf64_Ehdr, Elf64_Phdr] + * Then we just use those structs, and all the need for inline ntoh* goes away, + * as well as all the chances for error. + * This works because all the SELF structs have fields large enough for + * the largest ELF 64 struct members, and all the Elf64 struct members + * are at least large enough for all ELF 32 struct members. + * We end up with one function to do all our ELF parsing, and two functions + * to transform the headers. For the put case, we also have + * XDR functions, and hopefully we'll never again spend 5 years with the + * wrong endian-ness on an output value :-) + * This should work for all word sizes and endianness we hope to target. + * I *really* don't want to be here for 128 bit addresses. + * + * The parse functions are called with a pointer to an input buffer + * struct. One might ask: are there enough bytes in the input buffer? + * We know there need to be at *least* sizeof(Elf32_Ehdr) + + * sizeof(Elf32_Phdr) bytes. Realistically, there has to be some data + * too. If we start to worry, though we have not in the past, we + * might apply the simple test: the input buffer needs to be at least + * sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) bytes because, even if it's + * ELF 32, there's got to be *some* data! This is not theoretically + * accurate but it is actually good enough in practice. It allows the + * header transformation code to ignore the possibility of underrun. + * + * We also must accomodate different ELF files, and hence formats, + * in the same cbfs invocation. We might load a 64-bit payload + * on a 32-bit machine; we might even have a mixed armv7/armv8 + * SOC or even a system with an x86/ARM! + * + * A possibly problematic (though unlikely to be so) assumption + * is that we expect the BIOS to remain in the lowest 32 bits + * of the physical address space. Since ARMV8 has standardized + * on that, and x86_64 also has, this seems a safe assumption. + * + * To repeat, ELF structs are different sizes because ELF struct + * members are different sizes, depending on values in the ELF file + * header. For this we use the functions defined in xdr.c, which + * consume bytes, convert the endianness, and advance the data pointer + * in the buffer struct. + */ + +/* Get the ident array, so we can figure out + * endian-ness, word size, and in future other useful + * parameters + */ +static void +elf_eident(struct buffer *input, Elf64_Ehdr *ehdr) { - return x; + memmove(ehdr->e_ident, input->data, sizeof(ehdr->e_ident)); + input->data += sizeof(ehdr->e_ident); + input->size -= sizeof(ehdr->e_ident); }
-/* This is a wrapper around the swab32() macro to make it - * usable for the current implementation of parse_elf_to_stage() - */ -static unsigned int swap32(unsigned int x) +static void +elf_ehdr(struct buffer *input, Elf64_Ehdr *ehdr, struct xdr *xdr, int bit64) +{ + ehdr->e_type = xdr->get16(input); + ehdr->e_machine = xdr->get16(input); + ehdr->e_version = xdr->get32(input); + if (bit64){ + ehdr->e_entry = xdr->get64(input); + ehdr->e_phoff = xdr->get64(input); + ehdr->e_shoff = xdr->get64(input); + } else { + ehdr->e_entry = xdr->get32(input); + ehdr->e_phoff = xdr->get32(input); + ehdr->e_shoff = xdr->get32(input); + } + ehdr->e_flags = xdr->get32(input); + ehdr->e_ehsize = xdr->get16(input); + ehdr->e_phentsize = xdr->get16(input); + ehdr->e_phnum = xdr->get16(input); + ehdr->e_shentsize = xdr->get16(input); + ehdr->e_shnum = xdr->get16(input); + ehdr->e_shstrndx = xdr->get16(input); +} + +void +elf_phdr(struct buffer *pinput, Elf64_Phdr *phdr, int entsize, struct xdr *xdr, int bit64) { - return swab32(x); + /* + * The entsize need not be sizeof(*phdr). + * Hence, it is easier to keep a copy of the input, + * as the xdr functions may not advance the input + * pointer the full entsize; rather than get tricky + * we just advance it below. + */ + struct buffer input = *pinput; + if (bit64){ + phdr->p_type = xdr->get32(&input); + phdr->p_flags = xdr->get32(&input); + phdr->p_offset = xdr->get64(&input); + phdr->p_vaddr = xdr->get64(&input); + phdr->p_paddr = xdr->get64(&input); + phdr->p_filesz = xdr->get64(&input); + phdr->p_memsz = xdr->get64(&input); + phdr->p_align = xdr->get64(&input); + } else { + phdr->p_type = xdr->get32(&input); + phdr->p_offset = xdr->get32(&input); + phdr->p_vaddr = xdr->get32(&input); + phdr->p_paddr = xdr->get32(&input); + phdr->p_filesz = xdr->get32(&input); + phdr->p_memsz = xdr->get32(&input); + phdr->p_flags = xdr->get32(&input); + phdr->p_align = xdr->get32(&input); + } + pinput->size -= entsize; + pinput->data += entsize; }
-unsigned int (*elf32_to_native) (unsigned int) = idemp; +/* Get the headers from the buffer. + * Return -1 in the event of an error. + */ +int elf_headers(const struct buffer *pinput, Elf64_Ehdr *ehdr, Elf64_Phdr **pphdr) +{ + int i; + struct xdr *xdr = &xdr_le; + int bit64 = 0; + struct buffer input = *(struct buffer *)pinput; + struct buffer phdr_buf; + Elf64_Phdr *phdr;
-/* returns size of result, or -1 if error */ + if (!iself((unsigned char *)pinput->data)) { + ERROR("The stage file is not in ELF format!\n"); + return -1; + } + + elf_eident(&input, ehdr); + bit64 = ehdr->e_ident[EI_CLASS] == ELFCLASS64; + /* Assume LE unless we are sure otherwise. + * We're not going to take on the task of + * fully validating the ELF file. That way + * lies madness. + */ + if (ehdr->e_ident[EI_DATA] == ELFDATA2MSB) + xdr = &xdr_be; + + elf_ehdr(&input, ehdr, xdr, bit64); + + /* cons up an input buffer for the headers. + * Note that the program headers can be anywhere, + * per the ELF spec, You'd be surprised how many ELF + * readers miss this little detail. + */ + phdr_buf.data = &pinput->data[ehdr->e_phoff]; + phdr_buf.size = ehdr->e_phentsize * ehdr->e_phnum; + if (phdr_buf.size > (pinput->size - ehdr->e_phoff)){ + ERROR("The file is not large enough for the program headers." + "%d bytes left, %d bytes of headers\n", + pinput->size - ehdr->e_phoff, phdr_buf.size); + return -1; + } + /* gather up all the phdrs. + * We do them all at once because there is more + * than one loop over all the phdrs. + */ + phdr = calloc(sizeof(*phdr), ehdr->e_phnum); + for (i = 0; i < ehdr->e_phnum; i++) + elf_phdr(&phdr_buf, &phdr[i], ehdr->e_phentsize, xdr, bit64); + *pphdr = phdr; + return 0; +} +/* returns size of result, or -1 if error. + * Note that, with the new code, this function + * works for all elf files, not just the restricted set. + */ int parse_elf_to_stage(const struct buffer *input, struct buffer *output, comp_algo algo, uint32_t *location) { - Elf32_Phdr *phdr; - Elf32_Ehdr *ehdr = (Elf32_Ehdr *)input->data; - char *header, *buffer; + Elf64_Phdr *phdr; + Elf64_Ehdr ehdr; + char *buffer;
int headers; int i; struct cbfs_stage *stage; - unsigned int data_start, data_end, mem_end; - - int elf_bigendian = 0; + uint32_t data_start, data_end, mem_end;
comp_func_ptr compress = compression_function(algo); if (!compress) return -1;
DEBUG("start: parse_elf_to_stage(location=0x%x)\n", *location); - if (!iself((unsigned char *)input->data)) { - ERROR("The stage file is not in ELF format!\n"); - return -1; - }
- // The tool may work in architecture-independent way. - if (arch != CBFS_ARCHITECTURE_UNKNOWN && - !((ehdr->e_machine == EM_ARM) && (arch == CBFS_ARCHITECTURE_ARMV7)) && - !((ehdr->e_machine == EM_386) && (arch == CBFS_ARCHITECTURE_X86))) { - ERROR("The stage file has the wrong architecture\n"); + if (elf_headers(input, &ehdr, &phdr) < 0) return -1; - } - - if (ehdr->e_ident[EI_DATA] == ELFDATA2MSB) { - elf_bigendian = 1; - } - if (elf_bigendian != is_big_endian()) { - elf32_to_native = swap32; - } - - headers = ehdr->e_phnum; - header = (char *)ehdr;
- phdr = (Elf32_Phdr *) & header[elf32_to_native(ehdr->e_phoff)]; + headers = ehdr.e_phnum;
- /* Now, regular headers - we only care about PT_LOAD headers, - * because thats what we're actually going to load - */ - - data_start = 0xFFFFFFFF; + data_start = ~0; data_end = 0; mem_end = 0;
for (i = 0; i < headers; i++) { unsigned int start, mend, rend;
- if (elf32_to_native(phdr[i].p_type) != PT_LOAD) + if (phdr[i].p_type != PT_LOAD) continue;
/* Empty segments are never interesting */ - if (elf32_to_native(phdr[i].p_memsz) == 0) + if (phdr[i].p_memsz == 0) continue;
/* BSS */
- start = elf32_to_native(phdr[i].p_paddr); + start = phdr[i].p_paddr;
- mend = start + elf32_to_native(phdr[i].p_memsz); - rend = start + elf32_to_native(phdr[i].p_filesz); + mend = start + phdr[i].p_memsz; + rend = start + phdr[i].p_filesz;
if (start < data_start) data_start = start; @@ -128,8 +308,9 @@ int parse_elf_to_stage(const struct buffer *input, struct buffer *output, }
if (data_end <= data_start) { - ERROR("data ends before it starts. Make sure the " - "ELF file is correct and resides in ROM space.\n"); + ERROR("data ends (%08lx) before it starts(%08lx). Make sure the " + "ELF file is correct and resides in ROM space.\n", + data_end, data_start); exit(1); }
@@ -146,21 +327,21 @@ int parse_elf_to_stage(const struct buffer *input, struct buffer *output, for (i = 0; i < headers; i++) { unsigned int l_start, l_offset = 0;
- if (elf32_to_native(phdr[i].p_type) != PT_LOAD) + if (phdr[i].p_type != PT_LOAD) continue;
- if (elf32_to_native(phdr[i].p_memsz) == 0) + if (phdr[i].p_memsz == 0) continue;
- l_start = elf32_to_native(phdr[i].p_paddr); + l_start = phdr[i].p_paddr; if (l_start < *location) { l_offset = *location - l_start; l_start = *location; }
memcpy(buffer + (l_start - data_start), - &header[elf32_to_native(phdr[i].p_offset)+l_offset], - elf32_to_native(phdr[i].p_filesz)-l_offset); + &input->data[phdr[i].p_offset + l_offset], + phdr[i].p_filesz - l_offset); }
/* Now make the output buffer */ @@ -172,12 +353,16 @@ int parse_elf_to_stage(const struct buffer *input, struct buffer *output, } memset(output->data, 0, output->size);
+ /* N.B. The original plan was that SELF data was B.E. + * but: this is all L.E. + * Maybe we should just change the spec. + */ stage = (struct cbfs_stage *)output->data;
stage->load = data_start; /* FIXME: htonll */ stage->memlen = mem_end - data_start; stage->compression = algo; - stage->entry = ehdr->e_entry; /* FIXME: htonll */ + stage->entry = ehdr.e_entry;
compress(buffer, data_end - data_start, (output->data + sizeof(*stage)), (int *)&stage->len); diff --git a/util/cbfstool/common.h b/util/cbfstool/common.h index 6e12fcb..c68f0ff 100644 --- a/util/cbfstool/common.h +++ b/util/cbfstool/common.h @@ -131,6 +131,17 @@ uint32_t cbfs_find_location(const char *romfile, uint32_t filesize,
void print_supported_filetypes(void);
+/* xdr.c */ +struct xdr { + uint16_t (*get16)(struct buffer *input); + uint32_t (*get32)(struct buffer *input); + uint64_t (*get64)(struct buffer *input); + void (*put16)(struct buffer *input, uint16_t val); + void (*put32)(struct buffer *input, uint32_t val); + void (*put64)(struct buffer *input, uint64_t val); +}; + +extern struct xdr xdr_le, xdr_be; #define ARRAY_SIZE(a) (int)(sizeof(a) / sizeof((a)[0]))
#endif diff --git a/util/cbfstool/xdr.c b/util/cbfstool/xdr.c new file mode 100644 index 0000000..ab4ed70 --- /dev/null +++ b/util/cbfstool/xdr.c @@ -0,0 +1,141 @@ + /* + * cbfstool, CLI utility for CBFS file manipulation + * + * Copyright 2013 Google Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA, 02110-1301 USA + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <ctype.h> +#include <unistd.h> +#include <stdint.h> +#include "common.h" + +/* The assumption in all this code is that we're given a pointer to enough data. + * Hence, we do not check for underflow. + */ +uint8_t get8(struct buffer *input) +{ + uint8_t ret = *input->data++; + input->size--; + return ret; +} + +uint16_t get16be(struct buffer *input) +{ + uint16_t ret; + ret = get8(input)<<8; + ret |= get8(input); + return ret; +} + +uint32_t get32be(struct buffer *input) +{ + uint32_t ret; + ret = get16be(input)<<16; + ret |= get16be(input); + return ret; +} + +uint64_t get64be(struct buffer *input) +{ + uint64_t ret; + ret = get32be(input); + ret <<= 32; + ret |= get32be(input); + return ret; +} + +void put8(struct buffer *input, uint8_t val) +{ + input->data[size] = val; + input->size++; +} + +void put16be(struct buffer *input, uint16_t val) +{ + put8(input, val>>8); + put8(input, val); +} + +void put32be(struct buffer *input, uint32_t val) +{ + put16be(input, val>>16); + put16be(input, val); +} + +void put64be(struct buffer *input, uint64_t val) +{ + put32be(input, val>>32); + put32be(input, val); +} + +uint16_t get16le(struct buffer *input) +{ + uint16_t ret; + ret = get8(input); + ret |= get8(input) << 8; + return ret; +} + +uint32_t get32le(struct buffer *input) +{ + uint32_t ret; + ret = get16le(input); + ret |= get16le(input) << 16; + return ret; +} + +uint64_t get64le(struct buffer *input) +{ + uint64_t ret; + uint32_t low; + low = get32le(input); + ret = get32le(input); + ret <<= 32; + ret |= low; + return ret; +} + +void put16le(struct buffer *input, uint16_t val) +{ + put8(input, val); + put8(input, val>>8); +} + +void put32le(struct buffer *input, uint32_t val) +{ + put16le(input, val); + put16le(input, val>>16); +} + +void put64le(struct buffer *input, uint64_t val) +{ + put32le(input, val); + put32le(input, val>>32); +} + +struct xdr xdr_be = { + get16le, get32le, get64le, + put16be, put32be, put64be +}; + +struct xdr xdr_le = { + get16le, get32le, get64le, + put16le, put32le, put64le +}; +