[LinuxBIOS] patch: extending LAR, and removing elf from linuxbios (it is not needed)

Carl-Daniel Hailfinger c-d.hailfinger.devel.2006 at gmx.net
Tue Aug 28 21:32:41 CEST 2007


On 28.08.2007 20:41, ron minnich wrote:
> Round 2.
> 
> I had not realized that lar had changed. So all my patches were
> against the wrong files.
> 
> This code is against the right files in lar. I have also done a fair
> amount of refactoring as I needed it.
> 
> don't get upset about the close of the fd right after mmap; it's legal
> and even recommended.
> 
> Comments welcome as usual.
> 
> ron
> p.s. I can not get the LARCHIVE problem to NOT happen now. So, until
> we fix it right, let's take my wrong fix. Three of us have tried and
> failed to fix this, so, let's assume it's not trivial.

I have added LARCHIVE debugging code to your patch below to find out why
and where the problem happens. Could you post the log of your patch with
my added debugging? I hope we can see an obvious fix through that.

Other than that, this patch is Acked by me, but please wait for an
additional ack from somebody else.

Carl-Daniel

> ------------------------------------------------------------------------
> 
> This patch is revised based on comments. 
> 
> It also includes a demo of how to do a PIC initram, and:
> 
> EMERGENCY PATCH! Please see patch to lib/lar.c and include/lar.h for the 
> MAGIC constant. This fixes a bug I hit just now.
> 
> This patch also includes an EXPERT option for enabling no-ELF mode. 
> The system will default to old behaviour. See Kconfig in the root. 
> 
> I still wish to kill ELF mode very soon, however. 
> 
> LAR is a very capable format. With two simple extensions, we can use
> LAR to replace all that we are using ELF for now. This change can
> really make life better:
> 1. we can use streaming decompress instead of the current "uncompress
> elf to memory and then copy segments" approach. So we can get rid of
> THIS hardcode:
> #define UNCOMPRESS_AREA (0x400000)
> 2. A simple lar l can show ALL segments, including payload segments
> 3. It's really easy to see where things will go in memory, and catch problems
> 4. We can figure out an ELF input file is bogus BEFORE we flash, not
> AFTER we flash and try to boot it
> 5. did I mention streaming decompress?
> 6. We no longer have to worry about where we decompress the elf in
> memory (this problem was causing trouble when the payload was a linux
> kernel -- it was so big)
> 7. Since we have a load address, we can create this lar entry:
> normal/cmdline
> and specify that it be loaded at a place where linux will find it as
> the cmdline.
> 8. The decision on whether to XIP can be made in the LAR entry, not in
> hardcode. For example, if initram needs to be XIP, set the load
> address to 0xffffffff. Done.
> 
> The change is simple. Add a load address and entry point to the lar
> header. Extend the lar tool to parse the elf file and create multiple
> lar segments. It looks like this:
>  normal/payload0 (33192 bytes, lzma compressed to 18088 bytes @0x38
> load @0x100000, entry 0x105258)
>  normal/payload1 (72 bytes, lzma compressed to 47 bytes @0x4718 load
> @0x1225a0, entry 0x105258)
>  normal/option_table (932 bytes @0x4798 load @0, entry 0)
>  normal/stage2 (33308 bytes, lzma compressed to 15474 bytes @0x4b78
> load @0, entry 0)
>  normal/initram (4208 bytes @0x8828 load @0, entry 0)
>  linuxbios.bootblock (16384 bytes @0xfc000 load @0, entry 0)
> 
> note that the payload is now payload/segment0, payload/segment1, etc. I've extended
> linuxbios to look for these. Note that you can now see all the things
> that get loaded ;they're no longer hidden in an ELF header somewhere.
> Elf failures are gone!
> 
> Note that I've left legacy elf support in, for now, but recommend we
> get rid of it as soon as possible.
> 
> patch attached. This is a first pass. lar.c needs some refactoring but
> I want to get the cmdline going. You can now have a linux payload and
> it will uncompress with no problems.
> 
> This has been tested with filo and BOCHS.
> 
> Signed-off-by: Ronald G. Minnich <rminnich at gmail.com>
> Index: Kconfig
> ===================================================================
> --- Kconfig	(revision 480)
> +++ Kconfig	(working copy)
> @@ -53,6 +53,25 @@
>  	help
>  	  Append an extra string to the end of the LinuxBIOS version.
>  
> +config NOELF
> +	bool "Don't use ELF for payloads"
> +	depends EXPERT
> +	default n
> +	help
> +	  Until now, LinuxBIOS has used elf for the payload. There are many problems 
> +	  this, not least being the inefficiency -- the ELF has to be decompressed to
> +	  memory and then the segments have to be copied. Plus, lar can't see the segments 
> +	  in the elf -- to see all segments, you have to extract the elf and run readelf on it.
> +	  There are problems with collisions of the decompressed ELF location in memory
> +	  and the segment locations in memory. 
> +	  Finally, validation of the ELF is done at run time, once you have flashed the 
> +	  FLASH and rebooted the machine. Boot time is really not the time you want to find 
> +	  out your ELF payload is broken. 
> +	  With this option, LinuxBIOS will direct lar to break each elf segment into a LAR 
> +	  entry. ELF will not be used at all. Note that (for now) LinuxBIOS is backward
> +	  compatible -- if you put an ELF payload in, LinuxBIOS can still parse it. We hope
> +	  to remove ELF entirely in the future. 
> +
>  config BEEPS
>  	bool "Enable beeps upon certain LinuxBIOS events"
>  	depends EXPERT
> Index: include/lar.h
> ===================================================================
> --- include/lar.h	(revision 480)
> +++ include/lar.h	(working copy)
> @@ -52,9 +52,10 @@
>  
>  #include <types.h>
>  
> -#define MAGIC "LARCHIVE"
> +/* see note in lib/lar.c as to why this is ARCHIVE and not LARCHIVE */
> +#define MAGIC "ARCHIVE"
>  #define MAX_PATHLEN 1024
> -
> +/* NOTE -- This and the user-mode lar.h are NOT IN SYNC. Be careful. */
>  struct lar_header {
>  	char magic[8];
>  	u32 len;
> @@ -62,7 +63,14 @@
>  	u32 checksum;
>  	u32 compchecksum;
>  	u32 offset;
> +	/* Compression:
> +	 * 0 = no compression
> +	 * 1 = lzma
> +	 * 2 = nrv2b
> +	 */
>  	u32 compression;
> +	u32 entry;		/* we might need to make this u64 */
> +	u32 loadaddress; /* ditto */
>  };
>  
>  struct mem_file {
> @@ -70,6 +78,8 @@
>  	int len;
>  	u32 reallen;
>  	u32 compression;
> +	void *entry;
> +	void *loadaddress;
>  };
>  
>  /* Prototypes. */
> @@ -77,5 +87,6 @@
>  int copy_file(struct mem_file *archive, char *filename, void *where);
>  int run_file(struct mem_file *archive, char *filename, void *where);
>  int execute_in_place(struct mem_file *archive, char *filename);
> -
> +int run_address(void *f);
> +void *load_file(struct mem_file *archive, char *filename);
>  #endif /* LAR_H */
> Index: mainboard/emulation/qemu-x86/initram.c
> ===================================================================
> --- mainboard/emulation/qemu-x86/initram.c	(revision 480)
> +++ mainboard/emulation/qemu-x86/initram.c	(working copy)
> @@ -19,10 +19,17 @@
>  
>  #include <console.h>
>  
> +
> +/* This is how we force an absolute call when we are in PIC code (which this file is)
> + * If there is a nicer way to do this, I'd like to hear it. Ideally, we could tell 
> + * gcc to force abs jumps on certain symbols.
> + */
> +int (*pk)(int msg_level, const char *fmt, ...) = printk;
> +
>  int main(void)
>  {
> -	printk(BIOS_INFO, "RAM init code started.\n");
> -	printk(BIOS_INFO, "Nothing to do.\n");
> +	pk(BIOS_INFO, "RAM init code started.\n");
> +	pk(BIOS_INFO, "Nothing to do.\n");
>  
>  	return 0;
>  }
> Index: mainboard/emulation/qemu-x86/Makefile
> ===================================================================
> --- mainboard/emulation/qemu-x86/Makefile	(revision 480)
> +++ mainboard/emulation/qemu-x86/Makefile	(working copy)
> @@ -41,14 +41,17 @@
>  #
>  
>  INITRAM_OBJ = $(obj)/mainboard/$(MAINBOARDDIR)/initram.o
> +$(obj)/mainboard/$(MAINBOARDDIR)/initram.o: $(src)/mainboard/$(MAINBOARDDIR)/initram.c
> +	cc -c $(INITCFLAGS)  -fPIE $(src)/mainboard/$(MAINBOARDDIR)/initram.c -o $(obj)/mainboard/$(MAINBOARDDIR)/initram.o
> +
>  # These are possibly not permanent
> -INITRAM_OBJ += $(obj)/lib/console.o $(obj)/lib/vtxprintf.o $(obj)/lib/uart8250.o $(obj)/arch/x86/post_code.o
> +#INITRAM_OBJ += $(obj)/lib/console.o $(obj)/lib/vtxprintf.o $(obj)/lib/uart8250.o $(obj)/arch/x86/post_code.o
>  
>  $(obj)/linuxbios.initram: $(obj)/stage0.init $(obj)/stage0.o $(INITRAM_OBJ)
>  	$(Q)# initram links against stage0
>  	$(Q)printf "  LD      $(subst $(shell pwd)/,,$(@))\n"
> -	$(Q)$(LD)  -Ttext 0x80000 $(INITRAM_OBJ) \
> -		--entry=main -o $(obj)/linuxbios.initram.o
> +	$(Q)$(LD)  $(INITRAM_OBJ) \
> +		--entry=main -R $(obj)/stage0.o -o $(obj)/linuxbios.initram.o
>  	$(Q)printf "  OBJCOPY $(subst $(shell pwd)/,,$(@))\n"
>  	$(Q)$(OBJCOPY) -O binary $(obj)/linuxbios.initram.o \
>  		$(obj)/linuxbios.initram
> Index: lib/lar.c
> ===================================================================
> --- lib/lar.c	(revision 480)
> +++ lib/lar.c	(working copy)
> @@ -31,6 +31,13 @@
>  #define ntohl(x) (x)
>  #endif
>  
> +int run_address(void *f)
> +{
> +	int (*v) (void);
> +	v = f;
> +	return v();
> +}
> +
>  int find_file(struct mem_file *archive, char *filename, struct mem_file *result)
>  {
>  	char *walk, *fullname;
> @@ -42,29 +49,103 @@
>  
>  	for (walk = archive->start;
>  	     (walk - 1) < (char *)(archive->start + archive->len - 1 ); walk += 16) {
> -		if (strcmp(walk, MAGIC) != 0)
> +		/* I am leaving this code in here because it is so dangerous. MAGIC is
> + 		 * #define'd to a string. That string lives in data space. All of the 1M linuxbios 
> + 		 * image is a LAR file. Therefore, this search can walk ALL of linuxbios. 
> + 		 * IF the MAGIC string (in code space) just happens to be 16-byte aligned, 
> + 		 * Then the strcmp will succeed, and you will match a non-LAR entry, 
> + 		 * and you are screwed. can this happen? YES!
> + 		 * LAR: Attempting to open 'fallback/initram'.
> + 		 * LAR: Start 0xfff00000 len 0x100000
> + 		 * LAR: current filename is normal/payload
> + 		 * LAR: current filename is normal/option_table
> + 		 * LAR: current filename is normal/stage2
> + 		 * LAR: current filename is normal/initram
> + 		 * LAR: current filename is R: it matches %s @ %p
> + 		 * That garbage is there because the pointer is in the middle of a bunch 
> + 		 * of non-null-terminated junk. The fix is easy, as you can see. 
> +		 * if (strcmp(walk, MAGIC) != 0)
> +		 *	continue;
> +		 * And, yes, this did indeed fix the problem!
> +		 */

> +		if (walk[0] != 'L')
>  			continue;
>  
> +		if (strcmp(&walk[1], MAGIC) != 0)
> +			continue;
> +

Maybe change the code above to add add extra debugging code:
> +		printk(BIOS_SPEW, "LAR: looking for archive header at %p", walk);
> +		if (walk[0] != 'L') {
> +			printk(BIOS_SPEW, "LAR: no L magic at %p", walk);
>  			continue;
> +		}
>  
> +		if (strcmp(&walk[1], MAGIC) != 0) {
> +			printk(BIOS_SPEW, "LAR: no ARCHIVE magic at %p", walk);
> +			continue;
> +		}
> +
> +		printk(BIOS_SPEW, "LAR: full magic at %p", walk);


>  		header = (struct lar_header *)walk;
>  		fullname = walk + sizeof(struct lar_header);
>  
> -		printk(BIOS_SPEW, "LAR: current filename is %s\n", fullname);
> +		printk(BIOS_SPEW, "LAR: search for %s\n", fullname);
>  		// FIXME: check checksum
>  
>  		if (strcmp(fullname, filename) == 0) {
> +			printk(BIOS_SPEW, "LAR: FOUND %s @ %p\n", fullname, header);
>  			result->start = walk + ntohl(header->offset);
>  			result->len = ntohl(header->len);
>  			result->reallen = ntohl(header->reallen);
>  			result->compression = ntohl(header->compression);
> +			result->entry = (void *)ntohl(header->entry);
> +			result->loadaddress = (void *)ntohl(header->loadaddress);
> +			printk(BIOS_SPEW, 
> +			"start %p len %d reallen %d compression %x entry %p loadaddress %p\n", 
> +				result->start, result->len, result->reallen, 
> +				result->compression, result->entry, result->loadaddress);
>  			return 0;
>  		}
>  		// skip file
>  		walk += (ntohl(header->len) + ntohl(header->offset) -
>  			1) & 0xfffffff0;
>  	}
> +	printk(BIOS_SPEW, "NO FILE FOUND\n");
>  	return 1;
>  }
>  
> +
> +void *load_file(struct mem_file *archive, char *filename)
> +{
> +	int ret;
> +	struct mem_file result;
> +	void *where;
> +	void *entry;
> +
> +	ret = find_file(archive, filename, &result);
> +	if (ret) {
> +		printk(BIOS_INFO, "LAR: load_file: No such file '%s'\n",
> +		       filename);
> +		return (void *)-1;
> +	}
> +	entry = result.entry;
> +	where = result.loadaddress;
> +	printk(BIOS_SPEW, "LAR: Compression algorithm #%i used\n", result.compression);
> +	/* no compression */
> +	if (result.compression == 0) {
> +		memcpy(where, result.start, result.len);
> +		return entry;
> +	}
> +#ifdef CONFIG_COMPRESSION_LZMA
> +	/* lzma */
> +	unsigned long ulzma(unsigned char *src, unsigned char *dst);
> +	if (result.compression == 1) {
> +		ulzma(result.start, where);
> +		return entry;
> +	}
> +#endif
> +#ifdef CONFIG_COMPRESSION_NRV2B
> +	/* nrv2b */
> +	unsigned long unrv2b(u8 *src, u8 *dst, unsigned long *ilen_p);
> +	if (result.compression == 2) {
> +		int tmp;
> +		unrv2b(result.start, where, &tmp);
> +		return entry;
> +	}
> +#endif
> +	printk(BIOS_INFO, "LAR: Compression algorithm #%i not supported!\n", result.compression);
> +	return (void *)-1;
> +}
> +
> +/* FIXME -- most of copy_file should be replaced by load_file */
>  int copy_file(struct mem_file *archive, char *filename, void *where)
>  {
>  	int ret;
> @@ -85,7 +166,7 @@
>  	}
>  #ifdef CONFIG_COMPRESSION_LZMA
>  	/* lzma */
> -	unsigned long ulzma(unsigned char * src, unsigned char * dst);
> +	unsigned long ulzma(unsigned char *src, unsigned char *dst);
>  	if (result.compression == 1) {
>  		ulzma(result.start, where);
>  		return 0;
> @@ -93,7 +174,7 @@
>  #endif
>  #ifdef CONFIG_COMPRESSION_NRV2B
>  	/* nrv2b */
> -	unsigned long unrv2b(u8 * src, u8 * dst, unsigned long *ilen_p);
> +	unsigned long unrv2b(u8 *src, u8 *dst, unsigned long *ilen_p);
>  	if (result.compression == 2) {
>  		int tmp;
>  		unrv2b(result.start, where, &tmp);
> @@ -113,6 +194,7 @@
>  {
>  	int (*v) (void);
>  	struct mem_file result;
> +	int ret;
>  
>  	if ((u32) where != 0xFFFFFFFF) {
>  		if (copy_file(archive, filename, where)) {
> @@ -130,9 +212,11 @@
>  		}
>  		where = result.start;
>  	}
> -
> +	printk(BIOS_SPEW, "where is %p\n", where);
>  	v = where;
> -	return v();
> +	ret =  v();
> +	printk(BIOS_SPEW, "run_file returns with %d\n", ret);
> +	return ret;
>  }
>  
>  /**
> Index: lib/stage2.c
> ===================================================================
> --- lib/stage2.c	(revision 480)
> +++ lib/stage2.c	(working copy)
> @@ -99,5 +99,6 @@
>  	write_tables();
>  	show_all_devs();
>  
> +	printk(BIOS_SPEW, "STAGE2 NOW RETURNING\n");
>  	return 0;
>  }
> Index: arch/x86/stage1.c
> ===================================================================
> --- arch/x86/stage1.c	(revision 480)
> +++ arch/x86/stage1.c	(working copy)
> @@ -22,12 +22,16 @@
>  #include <io.h>
>  #include <console.h>
>  #include <lar.h>
> +#include <string.h>
>  #include <tables.h>
>  #include <lib.h>
>  #include <mc146818rtc.h>
>  #include <post_code.h>
>  
> -#define UNCOMPRESS_AREA 0x60000
> +/* ah, well, what a mess! This is a hard code. FIX ME but how? 
> + * By getting rid of ELF ...
> + */
> +#define UNCOMPRESS_AREA (0x400000)
>  
>  /* these prototypes should go into headers */
>  void uart_init(void);
> @@ -48,6 +52,24 @@
>  	post_code(0xf2);
>  }
>  
> +
> +/* until we get rid of elf */
> +int legacy(struct mem_file *archive, char *name, void *where, struct lb_memory *mem)
> +{
> +	int ret;
> +	struct mem_file result;
> +	int elfboot_mem(struct lb_memory *mem, void *where, int size);
> +	ret = copy_file(archive, name, where);
> +	if (ret) {
> +		printk(BIOS_ERR, "'%s' found, but could not load it.\n", name);
> +	}
> +
> +	ret =  elfboot_mem(mem, where, result.reallen);
> +
> +	printk(BIOS_ERR, "elfboot_mem returns %d\n", ret);
> +	return -1;
> +}
> +
>  /*
>   * This function is called from assembler code whith its argument on the
>   * stack. Force the compiler to generate always correct code for this case.
> @@ -57,6 +79,8 @@
>  	int ret;
>  	struct mem_file archive, result;
>  	int elfboot_mem(struct lb_memory *mem, void *where, int size);
> +	void *entry;
> +	int i;
>  
>  	/* we can't statically init this hack. */
>  	unsigned char faker[64];
> @@ -144,20 +168,32 @@
>  	printk(BIOS_DEBUG, "Stage2 code done.\n");
>  
>  	ret = find_file(&archive, "normal/payload", &result);
> -	if (ret) {
> -		printk(BIOS_ERR, "No such file '%s'.\n", "normal/payload");
> -		die("FATAL: No payload found.\n");
> +	if (! ret)
> +		legacy(&archive, "normal/payload", (void *)UNCOMPRESS_AREA, mem);
> +
> +
> +	/* new style lar boot. Install all the files in memory. 
> +	  * By convention we take the entry point from the first 
> +	  * one. Look for a cmdline as well. 
> +	  */
> +	for(i = 0, entry = (void *)0; ;i++) {
> +		char filename[64];
> +		void *newentry;
> +		sprintf(filename, "normal/payload/segment%d", i);
> +		archive.len = *(u32 *)0xfffffff4;
> +		archive.start =(void *)(0UL-archive.len);
> +		newentry = load_file(&archive, filename);
> +		printk("newentry is %p\n", newentry);
> +		if (newentry == (void *)-1)
> +			break;
> +		if (! entry)
> +			entry = newentry;
>  	}
> -	ret = copy_file(&archive, "normal/payload", (void *)UNCOMPRESS_AREA);
> -	if (ret) {
> -		printk(BIOS_ERR, "'%s' found, but could not load it.\n", "normal/payload");
> -		die("FATAL: No usable payload found.\n");
> -	}
> +	printk(BIOS_SPEW, "all loaded, entry %p\n", entry);
> +	run_address(entry);
>  
> -	ret =  elfboot_mem(mem, (void *)UNCOMPRESS_AREA, result.reallen);
> +	die("FATAL: No usable payload found.\n");
>  
> -	printk(BIOS_ERR, "elfboot_mem returns %d\n", ret);
> -
>  	die ("FATAL: Last stage returned to LinuxBIOS.\n");
>  }
>  
> Index: arch/x86/Makefile
> ===================================================================
> --- arch/x86/Makefile	(revision 480)
> +++ arch/x86/Makefile	(working copy)
> @@ -22,7 +22,7 @@
>  ifeq ($(CONFIG_ARCH_X86),y)
>  
>  INITCFLAGS := $(CFLAGS) -I$(src)/include/arch/x86 -I$(src)/include \
> -		-I$(obj) -fno-builtin
> +	-I$(obj) -fno-builtin  
>  
>  SILENT := >/dev/null 2>&1
>  
> @@ -78,7 +78,7 @@
>  endif
>  	$(Q)printf "  LAR     $(subst $(shell pwd)/,,$(@))\n"
>  	$(Q)rm -f $(obj)/linuxbios.rom
> -	$(Q)cd $(obj)/lar.tmp && ../util/lar/lar $(COMPRESSFLAG) -c \
> +	$(Q)cd $(obj)/lar.tmp && ../util/lar/lar $(PARSEELF) $(COMPRESSFLAG) -c \
>  		../linuxbios.rom \
>  		$(LARFILES) \
>  		-s $(ROM_SIZE) -b $(obj)/linuxbios.bootblock
> @@ -122,6 +122,11 @@
>  endif
>  endif
>  
> +ifeq ($(CONFIG_NOELF), y)
> +	PARSEELF = -e
> +else
> +	PARSEELF = 
> +endif
>  
>  STAGE0_OBJ := $(patsubst %,$(obj)/lib/%,$(STAGE0_LIB_OBJ)) \
>  	      $(patsubst %,$(obj)/arch/x86/%,$(STAGE0_ARCH_X86_OBJ)) \
> @@ -143,6 +148,9 @@
>  	$(Q)printf "  OBJCOPY $(subst $(shell pwd)/,,$(@))\n"
>  	$(Q)$(OBJCOPY) -O binary $(obj)/stage0.o $(obj)/stage0.init
>  
> +	$(Q)printf " OBJCOPY (stage0 link) $(subst $(shell pwd)/,,$(@))\n"
> +	$(Q)$(OBJCOPY) --prefix-symbols=stage0 $(obj)/stage0.o $(obj)/stage0_link.o
> +
>  	$(Q)printf "  TEST    $(subst $(shell pwd)/,,$(@))\n"
>  	$(Q)test `wc -c < $(obj)/stage0.init` -gt 16128 && \
>  		printf "Error. Bootblock got too big.\n" || true
> Index: util/lar/lar.c
> ===================================================================
> --- util/lar/lar.c	(revision 480)
> +++ util/lar/lar.c	(working copy)
> @@ -30,13 +30,14 @@
>  #include <sys/stat.h>
>  #include <sys/mman.h>
>  
> +#include "lar.h"
>  #include "lib.h"
> -#include "lar.h"
>  
>  #define VERSION "v0.9.1"
>  #define COPYRIGHT "Copyright (C) 2006-2007 coresystems GmbH"
>  
>  static int isverbose = 0;
> +static int iselfparse = 0;
>  static long larsize = 0;
>  static char *bootblock = NULL;
>  enum compalgo algo = none;
> @@ -44,7 +45,7 @@
>  static void usage(char *name)
>  {
>  	printf("\nLAR - the LinuxBIOS Archiver " VERSION "\n" COPYRIGHT "\n\n"
> -	       "Usage: %s [-cxal] archive.lar [[[file1] file2] ...]\n\n", name);
> +	       "Usage: %s [-ecxal] archive.lar [[[file1] file2] ...]\n\n", name);
>  	printf("Examples:\n");
>  	printf("  lar -c -s 32k -b bootblock myrom.lar foo nocompress:bar\n");
>  	printf("  lar -a myrom.lar foo blob:baz\n");
> @@ -66,13 +67,18 @@
>  	printf("                \ta 'm' suffix to multiple the size by 1024*1024.\n");
>  	printf("  -b [bootblock]\tSpecify the bootblock blob\n");
>  	printf("  -C [lzma|nrv2b]\tSpecify the compression method to use\n\n");
> +	printf("  -e pre-parse the payload ELF into LAR segments. Recommended\n\n");
>  
>  	printf("General options\n");
>  	printf("  -v\tEnable verbose mode\n");
>  	printf("  -V\tShow the version\n");
>  	printf("  -h\tShow this help\n");
>  	printf("\n");
> +}
>  
> +int elfparse(void)
> +{
> +	return iselfparse;
>  }
>  
>  /* Add a human touch to the LAR size by allowing suffixes:
> @@ -209,6 +215,7 @@
>  		{"list", 0, 0, 'l'},
>  		{"size", 1, 0, 's'},
>  		{"bootblock", 1, 0, 'b'},
> +		{"elfparse", 1, 0, 'e'},
>  		{"verbose", 0, 0, 'v'},
>  		{"version", 0, 0, 'V'},
>  		{"help", 0, 0, 'h'},
> @@ -220,7 +227,7 @@
>  		exit(1);
>  	}
>  
> -	while ((opt = getopt_long(argc, argv, "acC:xls:b:vVh?",
> +	while ((opt = getopt_long(argc, argv, "acC:xels:b:vVh?",
>  				  long_options, &option_index)) != EOF) {
>  		switch (opt) {
>  		case 'a':
> @@ -240,6 +247,9 @@
>  		case 'l':
>  			larmode = LIST;
>  			break;
> +		case 'e':
> +			iselfparse = 1;
> +			break;
>  		case 'x':
>  			larmode = EXTRACT;
>  			break;
> Index: util/lar/lar.h
> ===================================================================
> --- util/lar/lar.h	(revision 480)
> +++ util/lar/lar.h	(working copy)
> @@ -60,6 +60,7 @@
>  typedef uint32_t u32;
>  typedef uint8_t  u8;
>  
> +/* NOTE -- This and the linuxbios lar.h are NOT IN SYNC. Be careful. */
>  struct lar_header {
>  	char magic[8];
>  	u32 len;
> @@ -73,6 +74,8 @@
>  	 * 2 = nrv2b
>  	 */
>  	u32 compression;
> +	u32 entry;		/* we might need to make this u64 */
> +	u32 loadaddress; /* ditto */
>  };
>  
>  /**\struct
> Index: util/lar/lib.h
> ===================================================================
> --- util/lar/lib.h	(revision 480)
> +++ util/lar/lib.h	(working copy)
> @@ -38,6 +38,7 @@
>  
>  /* prototypes for lar.c functions */
>  int verbose(void);
> +int elfparse(void);
>  long get_larsize(void);
>  char *get_bootblock(void);
>  
> @@ -50,13 +51,24 @@
>  struct file *get_files(void);
>  void free_files(void);
>  
> +/* Prototypes for ELF functions */
> +int iself(char *filebuf);
> +
> +/* Prototypes for in-memory LAR operations */
> +int lar_process_name(char *name, char **pfilename, char **ppathname, 
> +		enum compalgo *thisalgo);
> +u32 lar_compress(char *ptr, ssize_t size, char *temp, enum compalgo *thisalgo);
> +int lar_add_entry(struct lar *lar, char *pathname, void *data, 
> +	u32 complen, u32 reallen, u32 loadaddress, u32 entry, 
> +	enum compalgo thisalgo);
>  /* Prototypes for the LAR I/O functions */
> +char *mapfile(char *filename, u32 *size);
>  struct lar * lar_new_archive(const char *archive, unsigned int size);
>  struct lar * lar_open_archive(const char *archive);
>  void lar_close_archive(struct lar *lar);
>  
>  void lar_list_files(struct lar *lar, struct file *files);
> -int lar_add_file(struct lar *lar, const char *name);
> +int lar_add_file(struct lar *lar, char *name);
>  int lar_add_bootblock(struct lar *lar, const char *bootblock);
>  int lar_extract_files(struct lar *lar, struct file *files);
>  
> Index: util/lar/stream.c
> ===================================================================
> --- util/lar/stream.c	(revision 480)
> +++ util/lar/stream.c	(working copy)
> @@ -32,9 +32,10 @@
>  #include <sys/mman.h>
>  #include <netinet/in.h>
>  #include <libgen.h>
> +#include <elf.h>
>  
> +#include "lar.h"
>  #include "lib.h"
> -#include "lar.h"
>  
>  /**
>   * \def err(fmt,args...)
> @@ -44,7 +45,126 @@
>  
>  extern enum compalgo algo;
>  
> +/* ELF processing */
>  /**
> + * Given a ptr to data, determine if the data is an ELF image. 
> + * @param filebuf pointer to the data
> + * @return 1 if ELF, 0 if not
> + */
> +int iself(char *filebuf)
> +{
> +	Elf32_Ehdr *ehdr;
> +	/* validate elf header */
> +	ehdr = (Elf32_Ehdr *)filebuf;
> +	if (memcmp(ehdr->e_ident, ELFMAG, 4))
> +		return 0;
> +	return 1;
> +}
> +
> +/**
> + * Output all the ELF segments for a given file
> + * @param lar The LAR Archoe
> + * @param name The LAR name
> + * @param filebuf The ELF file
> + * @param filelen Size of the ELF file
> + * @param algo The recommend compression algorithm
> + * Return 0 on success, -1 on failure
> + */
> +int output_elf_segments(struct lar *lar, char *name, char *filebuf,
> +			int filelen, enum compalgo algo)
> +{
> +	int ret;
> +        Elf32_Phdr *phdr;
> +	Elf32_Ehdr *ehdr;
> +	u32 entry;
> +	int i;
> +	int size;
> +	unsigned char *header;
> +	char ename[64];
> +	int headers;
> +	char *temp;
> +	enum compalgo thisalgo;
> +	u32 complen;
> +
> +	/* Allocate a temporary buffer to compress into - this is unavoidable,
> +	   because we need to make sure that the compressed data will fit in
> +	   the LAR, and we won't know the size of the compressed data until
> +	   we actually compress it */
> +
> +	temp = calloc(filelen, 1);
> +
> +	if (temp == NULL) {
> +		err("Out of memory.\n");
> +		return -1;
> +	}
> +
> +	/* validate elf header */
> +	ehdr = (Elf32_Ehdr *)filebuf;
> +	headers = ehdr->e_phnum;
> +	header  = (unsigned char *)ehdr;
> +	if (verbose())
> +		fprintf(stderr, "Type %d machine %d version %d entry %p phoff %d shoff %d flags %#x hsize %d phentsize %d phnum %d s_hentsize %d s_shnum %d \n", 
> +		ehdr->e_type,
> +		ehdr->e_machine,
> +		ehdr->e_version,
> +		(void *)ehdr->e_entry,
> +		ehdr->e_phoff,
> +		ehdr->e_shoff,
> +		ehdr->e_flags,
> +		ehdr->e_ehsize,
> +		ehdr->e_phentsize,
> +		ehdr->e_phnum,
> +		ehdr->e_shentsize,
> +		ehdr->e_shnum);
> +        phdr = (Elf32_Phdr *)&(header[ehdr->e_phoff]);
> +
> +	if (verbose())
> +		fprintf(stderr, "%s: header %p #headers %d\n", __FUNCTION__, ehdr, headers);
> +	entry = ehdr->e_entry;
> +	for(i = 0; i < headers; i++) {
> +		/* Ignore data that I don't need to handle */
> +		if (phdr[i].p_type != PT_LOAD) {
> +			if (verbose())
> +				fprintf(stderr, "Dropping non PT_LOAD segment\n");
> +			continue;
> +		}
> +		if (phdr[i].p_memsz == 0) {
> +			if (verbose())
> +				fprintf(stderr, "Dropping empty segment\n");
> +			continue;
> +		}
> +		thisalgo = algo;
> +		if (verbose())
> +			fprintf(stderr,  "New segment addr 0x%ulx size 0x%ulx offset 0x%ulx filesize 0x%ulx\n",
> +			(u32)phdr[i].p_paddr, (u32)phdr[i].p_memsz, 
> +			(u32)phdr[i].p_offset, (u32)phdr[i].p_filesz);
> +		/* Clean up the values */
> +		size = phdr[i].p_filesz;
> +		if (phdr[i].p_filesz > phdr[i].p_memsz)  {
> +			size = phdr[i].p_memsz;
> +		}
> +		if (verbose()) {
> +			fprintf(stderr, "(cleaned up) New segment addr %p size 0x%#x offset 0x%x\n",
> +				(void *)phdr[i].p_paddr, size, phdr[i].p_offset);
> +			fprintf(stderr, "Copy to %p from %p for %d bytes\n", 
> +				(unsigned char *)phdr[i].p_paddr, 
> +				&header[phdr[i].p_offset], size);
> +			fprintf(stderr, "entry %ux loadaddr %ux\n", 
> +				entry,  phdr[i].p_paddr);
> +		}
> +			/* ok, copy it out */
> +		sprintf(ename, "%s/segment%d", name, i);
> +		complen = lar_compress(filebuf, size, temp, &thisalgo);
> +		ret = lar_add_entry(lar, ename, &header[phdr[i].p_offset], 
> +				    complen, size, 
> +				    phdr[i].p_paddr, entry, thisalgo);
> +	}
> +	return 0;
> +out:
> +	return -1;
> +}
> +
> +/**
>   * Given a size, return the offset of the bootblock (including the
>   * header)
>   * @param size Size of the LAR archive
> @@ -259,7 +379,7 @@
>  	}
>  
>  	return lar;
> - err:
> +err:
>  	lar_close_archive(lar);
>  
>  	/* Don't leave a halfbaked LAR laying around */
> @@ -315,7 +435,7 @@
>  
>  	return lar;
>  
> - err:
> +err:
>  	lar_close_archive(lar);
>  	return NULL;
>  }
> @@ -535,110 +655,139 @@
>  }
>  
>  /**
> - * Add a new file to the LAR archive
> - * @param lar The LAR archive to write into
> - * @param name The name of the file to add
> - * @return  0 on success, or -1 on failure
> + * Given a pathname in the form [option;]path, determine the file name,
> + * LAR path name, and compression algorithm. 
> + * @param name name in the [option:][./]path form
> + * @param pfilename reference pointer to file name -- this is modified
> + * @param ppathname reference pointer to LAR path name -- this is modified
> + * @param thisalgo pointer to algorithm, which can be modified by path name
> + * @return 0 success, or -1 on failure (i.e. a bad name)
>   */
> -int lar_add_file(struct lar *lar, const char *name)
> +int lar_process_name(char *name, char **pfilename, char **ppathname, 
> +		     enum compalgo *thisalgo)
>  {
> -	char *filename, *ptr, *temp;
> -	char *pathname;
> +	char *filename = name, *pathname = name;
>  
> -	enum compalgo thisalgo;
> -	struct lar_header *header;
> -	u32 offset;
> -	int ret, fd, hlen;
> -	u32 complen;
> -	int pathlen;
> -	struct stat s;
> -	u32 *walk,  csum;
> -
> -	/* Find the beginning of the available space in the LAR */
> -	offset = lar_empty_offset(lar);
> -
> -	thisalgo = algo;
> -
> -	filename = (char *) name;
> -
>  	if (!strncmp(name, "nocompress:",11)) {
>  		filename += 11;
> -		thisalgo = none;
> +		*thisalgo = none;
>  	}
>  
> +	/* this is dangerous */
>  	if (filename[0] == '.' && filename[1] == '/')
>  		filename += 2;
>  
>  	pathname = strchr(filename, ':');
>  
>  	if (pathname != NULL) {
> -	  *pathname = '\0';
> -	  pathname++;
> +		*pathname = '\0';
> +		pathname++;
>  
> -	  if (!strlen(pathname)) {
> -	    err("Invalid pathname specified.\n");
> -	    return -1;
> -	  }
> +		if (!strlen(pathname)) {
> +			err("Invalid pathname specified.\n");
> +			return -1;
> +		}
>  	}
>  	else
>  		pathname = filename;
> +	*pfilename = filename;
> +	*ppathname = pathname;
> +	return 0;
> +}
>  
> +
> +/**
> + * Given a pathname, open and mmap the file. 
> + * @param filename
> + * @param size pointer to returned size
> + * @return ptr to mmap'ed area on success, or MAP_FAILED on failure
> + */
> +char *mapfile(char *filename, u32 *size)
> +{
> +	int fd;
> +	struct stat s;
> +	char *ptr;
> +
>  	/* Open the file */
>  	fd = open(filename, O_RDONLY);
>  
>  	if (fd == -1) {
>  		err("Unable to open %s\n", filename);
> -		return -1;
> +		return MAP_FAILED;
>  	}
>  
>  	if (fstat(fd, &s)) {
>  		err("Unable to stat the file %s\n", filename);
>  		close(fd);
> -		return -1;
> +		return MAP_FAILED;
>  	}
>  
> -	/* Allocate a temporary buffer to compress into - this is unavoidable,
> -	   because we need to make sure that the compressed data will fit in
> -	   the LAR, and we won't know the size of the compressed data until
> -	   we actually compress it */
> -
> -	temp = calloc(s.st_size, 1);
> -
> -	if (temp == NULL) {
> -		err("Out of memory.\n");
> -		return -1;
> -	}
> -
>  	ptr = mmap(0, s.st_size, PROT_READ, MAP_SHARED, fd, 0);
> +	close(fd);
>  
>  	if (ptr == MAP_FAILED) {
>  		err("Unable to map %s\n", filename);
> -		close(fd);
> -		free(temp);
> -		return -1;
> +		return MAP_FAILED;
>  	}
> +	*size = s.st_size;
> +	return ptr;
> +}
>  
> +/**
> + * Compress an area according to an algorithm. If the area grows, 
> + * use no compression. 
> + * @param ptr data to be compressed
> + * @param size size of the data 
> + * @param temp destination of compressed data
> + * @param thisalgo pointer to algorithm -- this can be modified
> + * @return  size of compressed data
> + */
> +u32 lar_compress(char *ptr, ssize_t size, char *temp, enum compalgo *thisalgo)
> +{
> +	u32 complen;
> +	compress_functions[*thisalgo](ptr, size, temp, &complen);
>  
> -	/* Do the compression step */
> -	compress_functions[thisalgo](ptr, s.st_size, temp, &complen);
> -
> -	if (complen >= s.st_size && (thisalgo != none)) {
> -		thisalgo = none;
> -		compress_functions[thisalgo](ptr, s.st_size, temp, &complen);
> +	if (complen >= size && (*thisalgo != none)) {
> +		*thisalgo = none;
> +		compress_functions[*thisalgo](ptr, size, temp, &complen);
>  	}
> +	return complen;
> +}
>  
> -	munmap(ptr, s.st_size);
> -	close(fd);
> +/**
> + * Add a new entry to the LAR archive
> + * @param lar The LAR archive to write into
> + * @param pathname The name of the segment
> + * @param data The data for the segment
> + * @param complen The compressed length of the segment
> + * @param reallen The real (uncompressed) length of the segment
> + * @param loadaddress The load address of the segment
> + * @param entry The entry point of the segment
> + * @param thisalgo The compression algorithm
> + * @return  0 on success, or -1 on failure
> + */
> +int lar_add_entry(struct lar *lar, char *pathname, void *data, 
> +		  u32 complen, u32 reallen, u32 loadaddress, u32 entry, 
> +		  enum compalgo thisalgo)
> +{
> +	struct lar_header *header;
> +	int ret, hlen;
> +	int pathlen;
> +	u32 *walk,  csum;
> +	u32 offset;
>  
> -	pathlen = strlen(pathname) + 1 > MAX_PATHLEN ? MAX_PATHLEN : strlen(pathname) + 1;
> +	/* Find the beginning of the available space in the LAR */
> +	offset = lar_empty_offset(lar);
>  
> +	pathlen = strlen(pathname) + 1 > MAX_PATHLEN ? 
> +		MAX_PATHLEN : strlen(pathname) + 1;
> +
>  	/* Figure out how big our header will be */
>  	hlen = sizeof(struct lar_header) + pathlen;
>  	hlen = (hlen + 15) & 0xFFFFFFF0;
>  
>  	if (offset + hlen + complen >= get_bootblock_offset(lar->size)) {
>  		err("Not enough room in the LAR to add the file.\n");
> -		free(temp);
>  		return -1;
>  	}
>  
> @@ -651,16 +800,17 @@
>  
>  	memcpy(header, MAGIC, 8);
>  	header->compression = htonl(thisalgo);
> -	header->reallen = htonl(s.st_size);
> +	header->reallen = htonl(reallen);
>  	header->len = htonl(complen);
>  	header->offset = htonl(hlen);
> -
> +	header->loadaddress = htonl(loadaddress);
> +	header->entry = htonl(entry);
>  	/* Copy the path name */
>  	strncpy((char *) (lar->map + offset + sizeof(struct lar_header)),
>  		pathname, pathlen - 1);
>  
>  	/* Copy in the data */
> -	memcpy(lar->map + (offset + hlen), temp, complen);
> +	memcpy(lar->map + (offset + hlen), data, complen);
>  
>  	/* Figure out the checksum */
>  
> @@ -671,7 +821,58 @@
>  		csum += ntohl(*walk);
>  	}
>  	header->checksum = htonl(csum);
> +	return 0;
> +}
>  
> +/**
> + * Add a new file to the LAR archive
> + * @param lar The LAR archive to write into
> + * @param name The name of the file to add
> + * @return  0 on success, or -1 on failure
> + */
> +int lar_add_file(struct lar *lar, char *name)
> +{
> +	char *filename, *ptr, *temp;
> +	char *pathname;
> +
> +	enum compalgo thisalgo;
> +	struct lar_header *header;
> +	int ret, hlen;
> +	u32 complen;
> +	int pathlen;
> +	u32 size;
> +
> +	thisalgo = algo;
> +	lar_process_name(name, &filename, &pathname, &thisalgo);
> +
> +	ptr = mapfile(filename, &size);
> +
> +	if (elfparse() && iself(ptr)) {
> +		output_elf_segments(lar, pathname, ptr, size, thisalgo);
> +		return 0;
> +	}
> +
> +	/* This is legacy stuff. */
> +
> +	/* Allocate a temporary buffer to compress into - this is unavoidable,
> +	   because we need to make sure that the compressed data will fit in
> +	   the LAR, and we won't know the size of the compressed data until
> +	   we actually compress it */
> +
> +	temp = calloc(size, 1);
> +
> +	if (temp == NULL) {
> +		err("Out of memory.\n");
> +		munmap(ptr, size);
> +		return -1;
> +	}
> +
> +	complen = lar_compress(ptr, size, temp, &thisalgo);
> +
> +	munmap(ptr, size);
> +
> +	ret = lar_add_entry(lar, pathname, temp, complen, size, 0, 0, thisalgo);
> +
>  	free(temp);
> -	return 0;
> +	return ret;
>  }
> Index: util/lar/Makefile
> ===================================================================
> --- util/lar/Makefile	(revision 480)
> +++ util/lar/Makefile	(working copy)
> @@ -17,7 +17,6 @@
>  ## along with this program; if not, write to the Free Software
>  ## Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA, 02110-1301 USA
>  ##
> -
>  LAROBJ := lar.o stream.o lib.o
>  
>  LARDIR := lardir
> @@ -29,7 +28,6 @@
>  LARDIR     += nrv2bdir
>  
>  LAROBJ_ABS := $(patsubst %,$(obj)/util/lar/%,$(LAROBJ))
> -
>  lardir:
>  	$(Q)printf "  BUILD   LAR\n"
>  	$(Q)mkdir -p $(obj)/util/lar
> 


-- 
http://www.hailfinger.org/




More information about the coreboot mailing list