On 28.08.2007 17:26, ron minnich wrote:
On 8/28/07, Stefan Reinauer stepan@coresystems.de wrote:
- Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net [070828 01:54]:
What about creating directories for each payload so we can differentiate between multiple payloads (in the original sense)?
Yes, I would prefer something like that, too
done. normal/payload/segment0 normal/payload/segment1 etc.
OK, so I was unclear. The idea was to number the payloads as well, not only the segments.
Maybe we should introduce a major/minor version in the LAR header? Or a different magic string for each revision?
I generally hate versions. Think features, not numbers. But since this is probably not doable without significant overhead, I'd just go with a normal version header that gets increased every time we change the format. No need for major, minor, subminor, ...
but then you have to handle the versioning, and figure out what to do if it does not match, and .... Let's not do this.
We simply could reserve some space at the end of the header and use that for future extensions without having to change a version number. Only if the extension needs changes in earlier fields of the struct we would have to change the MAGIC.
What was that linker auto rename trick that Marc mentioned recently?
The naming is not the issue. The issue is getting gcc to do abs calls, not relative calls, for some symbols.
OK, but please include your explanation as code comment.
This patch is revised based on comments.
It also includes a demo of how to do a PIC initram, and:
EMERGENCY PATCH! Please see patch to lib/lar.c and include/lar.h for the MAGIC constant. This fixes a bug I hit just now.
This patch also includes an EXPERT option for enabling no-ELF mode. The system will default to old behaviour. See Kconfig in the root.
I still wish to kill ELF mode very soon, however.
LAR is a very capable format. With two simple extensions, we can use LAR to replace all that we are using ELF for now. This change can really make life better:
- we can use streaming decompress instead of the current "uncompress
elf to memory and then copy segments" approach. So we can get rid of THIS hardcode: #define UNCOMPRESS_AREA (0x400000) 2. A simple lar l can show ALL segments, including payload segments 3. It's really easy to see where things will go in memory, and catch problems 4. We can figure out an ELF input file is bogus BEFORE we flash, not AFTER we flash and try to boot it 5. did I mention streaming decompress? 6. We no longer have to worry about where we decompress the elf in memory (this problem was causing trouble when the payload was a linux kernel -- it was so big) 7. Since we have a load address, we can create this lar entry: normal/cmdline and specify that it be loaded at a place where linux will find it as the cmdline. 8. The decision on whether to XIP can be made in the LAR entry, not in hardcode. For example, if initram needs to be XIP, set the load address to 0xffffffff. Done.
The change is simple. Add a load address and entry point to the lar header. Extend the lar tool to parse the elf file and create multiple lar segments. It looks like this: normal/payload/segment0 (33192 bytes, lzma compressed to 18088 bytes @0x38 load @0x100000, entry 0x105258) normal/payload/segment1 (72 bytes, lzma compressed to 47 bytes @0x4718 load @0x1225a0, entry 0x105258) normal/option_table (932 bytes @0x4798 load @0, entry 0) normal/stage2 (33308 bytes, lzma compressed to 15474 bytes @0x4b78 load @0, entry 0) normal/initram (4208 bytes @0x8828 load @0, entry 0) linuxbios.bootblock (16384 bytes @0xfc000 load @0, entry 0)
note that the payload is now payload0, payload1, etc. I've extended linuxbios to look for these. Note that you can now see all the things that get loaded ;they're no longer hidden in an ELF header somewhere. Elf failures are gone!
Please update the comment above.
Note that I've left legacy elf support in, for now, but recommend we get rid of it as soon as possible.
patch attached. This is a first pass. lar.c needs some refactoring but I want to get the cmdline going. You can now have a linux payload and it will uncompress with no problems.
This has been tested with filo and BOCHS.
Signed-off-by: Ronald G. Minnich rminnich@gmail.com Index: Kconfig =================================================================== --- Kconfig (revision 480) +++ Kconfig (working copy) @@ -53,6 +53,25 @@ help Append an extra string to the end of the LinuxBIOS version.
+config NOELF
- bool "Don't use ELF for payloads"
- depends EXPERT
- default n
- help
Until now, LinuxBIOS has used elf for the payload. There are many problems
this, not least being the inefficiency -- the ELF has to be decompressed to
memory and then the segments have to be copied. Plus, lar can't see the segments
in the elf -- to see all segments, you have to extract the elf and run readelf on it.
There are problems with collisions of the decompressed ELF location in memory
and the segment locations in memory.
Finally, validation of the ELF is done at run time, once you have flashed the
FLASH and rebooted the machine. Boot time is really not the time you want to find
out your ELF payload is broken.
With this option, LinuxBIOS will direct lar to break each elf segment into a LAR
entry. ELF will not be used at all. Note that (for now) LinuxBIOS is backward
compatible -- if you put an ELF payload in, LinuxBIOS can still parse it. We hope
to remove ELF entirely in the future.
config BEEPS bool "Enable beeps upon certain LinuxBIOS events" depends EXPERT Index: include/lar.h =================================================================== --- include/lar.h (revision 480) +++ include/lar.h (working copy) @@ -52,9 +52,10 @@
#include <types.h>
-#define MAGIC "LARCHIVE" +/* see note in lib/lar.c as to why this is ARCHIVE and not LARCHIVE */ +#define MAGIC "ARCHIVE" #define MAX_PATHLEN 1024
+/* NOTE -- This and the user-mode lar.h are NOT IN SYNC. Be careful. */ struct lar_header { char magic[8]; u32 len; @@ -62,7 +63,14 @@ u32 checksum; u32 compchecksum; u32 offset;
- /* Compression:
* 0 = no compression
* 1 = lzma
* 2 = nrv2b
u32 compression;*/
- u32 entry; /* we might need to make this u64 */
- u32 loadaddress; /* ditto */
};
struct mem_file { @@ -70,6 +78,8 @@ int len; u32 reallen; u32 compression;
- void * entry;
- void * loadaddress;
Cosmetic (no space between * and name): void *entry; void *loadaddress;
};
/* Prototypes. */ @@ -77,5 +87,6 @@ int copy_file(struct mem_file *archive, char *filename, void *where); int run_file(struct mem_file *archive, char *filename, void *where); int execute_in_place(struct mem_file *archive, char *filename);
+int run_address(void *f); +void * load_file(struct mem_file *archive, char *filename);
Same cosmetic.
#endif /* LAR_H */ Index: mainboard/emulation/qemu-x86/initram.c =================================================================== --- mainboard/emulation/qemu-x86/initram.c (revision 480) +++ mainboard/emulation/qemu-x86/initram.c (working copy) @@ -19,10 +19,12 @@
#include <console.h>
+int (*pk)(int msg_level, const char *fmt, ...) = printk;
Can you add your explanation from above?
/* The issue is getting gcc to do abs calls, not relative calls, for some symbols. */
int main(void) {
- printk(BIOS_INFO, "RAM init code started.\n");
- printk(BIOS_INFO, "Nothing to do.\n");
pk(BIOS_INFO, "RAM init code started.\n");
pk(BIOS_INFO, "Nothing to do.\n");
return 0;
} Index: mainboard/emulation/qemu-x86/Makefile =================================================================== --- mainboard/emulation/qemu-x86/Makefile (revision 480) +++ mainboard/emulation/qemu-x86/Makefile (working copy) @@ -41,14 +41,17 @@ #
INITRAM_OBJ = $(obj)/mainboard/$(MAINBOARDDIR)/initram.o +$(obj)/mainboard/$(MAINBOARDDIR)/initram.o: $(src)/mainboard/$(MAINBOARDDIR)/initram.c
- cc -c $(INITCFLAGS) -fPIE $(src)/mainboard/$(MAINBOARDDIR)/initram.c -o $(obj)/mainboard/$(MAINBOARDDIR)/initram.o
# These are possibly not permanent -INITRAM_OBJ += $(obj)/lib/console.o $(obj)/lib/vtxprintf.o $(obj)/lib/uart8250.o $(obj)/arch/x86/post_code.o +#INITRAM_OBJ += $(obj)/lib/console.o $(obj)/lib/vtxprintf.o $(obj)/lib/uart8250.o $(obj)/arch/x86/post_code.o
$(obj)/linuxbios.initram: $(obj)/stage0.init $(obj)/stage0.o $(INITRAM_OBJ) $(Q)# initram links against stage0 $(Q)printf " LD $(subst $(shell pwd)/,,$(@))\n"
- $(Q)$(LD) -Ttext 0x80000 $(INITRAM_OBJ) \
--entry=main -o $(obj)/linuxbios.initram.o
- $(Q)$(LD) $(INITRAM_OBJ) \
$(Q)printf " OBJCOPY $(subst $(shell pwd)/,,$(@))\n" $(Q)$(OBJCOPY) -O binary $(obj)/linuxbios.initram.o \ $(obj)/linuxbios.initram--entry=main -R $(obj)/stage0.o -o $(obj)/linuxbios.initram.o
Index: lib/lzmadecode.c
--- lib/lzmadecode.c (revision 480) +++ lib/lzmadecode.c (working copy) @@ -206,7 +206,6 @@ RC_GET_BIT(probLit, symbol) } previousByte = (Byte)symbol;
Empty line removal. Was this intentional?
outStream[nowPos++] = previousByte; if (state < 4) state = 0; else if (state < 10) state -= 3;
Index: lib/lar.c
--- lib/lar.c (revision 480) +++ lib/lar.c (working copy) @@ -31,6 +31,13 @@ #define ntohl(x) (x) #endif
+int run_address(void *f) +{
- int (*v) (void);
- v = f;
- return v();
+}
int find_file(struct mem_file *archive, char *filename, struct mem_file *result) { char *walk, *fullname; @@ -42,9 +49,31 @@
for (walk = archive->start; (walk - 1) < (char *)(archive->start + archive->len - 1 ); walk += 16) {
if (strcmp(walk, MAGIC) != 0)
/* I am leaving this code in here because it is so dangerous. MAGIC is
* #define'd to a string. That string lives in data space. All of the 1M linuxbios
* image is a LAR file. Therefore, this search can walk ALL of linuxbios.
* IF the MAGIC string (in code space) just happens to be 16-byte aligned,
* Then the strcmp will succeed, and you will match a non-LAR entry,
* and you are screwed. can this happen? YES!
* LAR: Attempting to open 'fallback/initram'.
* LAR: Start 0xfff00000 len 0x100000
* LAR: current filename is normal/payload
* LAR: current filename is normal/option_table
* LAR: current filename is normal/stage2
* LAR: current filename is normal/initram
* LAR: current filename is R: it matches %s @ %p
* That garbage is there because the pointer is in the middle of a bunch
* of non-null-terminated junk. The fix is easy, as you can see.
* if (strcmp(walk, MAGIC) != 0)
* continue;
* And, yes, this did indeed fix the problem!
*/
if (walk[0] != 'L') continue;
if (strcmp(&walk[1], MAGIC) != 0)
continue;
There has to be a cleaner trick. Just in case somebody has the expected MAGIC string in an uncompressed lar file somewhere (other sources of that are possible!), we still fail, but less often.
header = (struct lar_header *)walk; fullname = walk + sizeof(struct lar_header);
@@ -52,19 +81,69 @@ // FIXME: check checksum
if (strcmp(fullname, filename) == 0) {
printk(BIOS_SPEW, "LAR: it matches %s @ %p\n", fullname, header);
replace "it" with "filename"?
result->start = walk + ntohl(header->offset); result->len = ntohl(header->len); result->reallen = ntohl(header->reallen); result->compression = ntohl(header->compression);
result->entry = (void *)ntohl(header->entry);
result->loadaddress = (void *)ntohl(header->loadaddress);
printk(BIOS_SPEW, "start %p len %d reallen %d compression %x entry %p loadaddress %p\n",
result->start, result->len, result->reallen, result->compression, result->entry, result->loadaddress);
Any chance to break that into multiple lines?
return 0; } // skip file walk += (ntohl(header->len) + ntohl(header->offset) - 1) & 0xfffffff0;
}
- printk(BIOS_SPEW, "NO FILE FOUND\n"); return 1;
}
+void * load_file(struct mem_file *archive, char *filename)
cosmetic: void *load_file(..)
+{
- int ret;
- struct mem_file result;
- void *where;
- void *entry;
- ret = find_file(archive, filename, &result);
- if (ret) {
printk(BIOS_INFO, "LAR: load_file: No such file '%s'\n",
filename);
return (void *)-1;
- }
- entry = result.entry;
- where = result.loadaddress;
- printk(BIOS_SPEW, "LAR: Compression algorithm #%i used\n", result.compression);
- /* no compression */
- if (result.compression == 0) {
memcpy(where, result.start, result.len);
return entry;
- }
+#ifdef CONFIG_COMPRESSION_LZMA
- /* lzma */
- unsigned long ulzma(unsigned char * src, unsigned char * dst);
cosmetic: + unsigned long ulzma(unsigned char *src, unsigned char *dst);
- if (result.compression == 1) {
ulzma(result.start, where);
return entry;
- }
+#endif +#ifdef CONFIG_COMPRESSION_NRV2B
- /* nrv2b */
- unsigned long unrv2b(u8 * src, u8 * dst, unsigned long *ilen_p);
Cosmetic: + unsigned long unrv2b(u8 *src, u8 *dst, unsigned long *ilen_p);
- if (result.compression == 2) {
int tmp;
unrv2b(result.start, where, &tmp);
return entry;
- }
+#endif
- printk(BIOS_INFO, "LAR: Compression algorithm #%i not supported!\n", result.compression);
- return (void *)-1;
+}
+/* FIXME -- most of copy_file should be replaced by load_file */ int copy_file(struct mem_file *archive, char *filename, void *where) { int ret; @@ -113,6 +192,7 @@ { int (*v) (void); struct mem_file result;
int ret;
if ((u32) where != 0xFFFFFFFF) { if (copy_file(archive, filename, where)) {
@@ -130,9 +210,11 @@ } where = result.start; }
- printk(BIOS_SPEW, "where is %p\n", where); v = where;
- return v();
- ret = v();
- printk(BIOS_SPEW, "run_file returns with %d\n", ret);
- return ret;
}
/** Index: lib/stage2.c =================================================================== --- lib/stage2.c (revision 480) +++ lib/stage2.c (working copy) @@ -99,5 +99,6 @@ write_tables(); show_all_devs();
- printk(BIOS_SPEW, "STAGE2 NOW RETURNING\n"); return 0;
} Index: arch/x86/stage1.c =================================================================== --- arch/x86/stage1.c (revision 480) +++ arch/x86/stage1.c (working copy) @@ -22,12 +22,16 @@ #include <io.h> #include <console.h> #include <lar.h> +#include <string.h> #include <tables.h> #include <lib.h> #include <mc146818rtc.h> #include <post_code.h>
-#define UNCOMPRESS_AREA 0x60000 +/* ah, well, what a mess! This is a hard code. FIX ME but how?
- By getting rid of ELF ...
- */
+#define UNCOMPRESS_AREA (0x400000)
/* these prototypes should go into headers */ void uart_init(void); @@ -48,6 +52,24 @@ post_code(0xf2); }
+/* until we get rid of elf */ +int legacy(struct mem_file *archive, char *name, void *where, struct lb_memory *mem)
cosmetic whitespace: +int legacy(struct mem_file *archive, char *name, void *where, struct lb_memory *mem)
+{
- int ret;
- struct mem_file result;
- int elfboot_mem(struct lb_memory *mem, void *where, int size);
- ret = copy_file(archive, name, where);
- if (ret) {
printk(BIOS_ERR, "'%s' found, but could not load it.\n", name);
- }
- ret = elfboot_mem(mem, where, result.reallen);
- printk(BIOS_ERR, "elfboot_mem returns %d\n", ret);
- return -1;
+}
/*
- This function is called from assembler code whith its argument on the
- stack. Force the compiler to generate always correct code for this case.
@@ -57,6 +79,8 @@ int ret; struct mem_file archive, result; int elfboot_mem(struct lb_memory *mem, void *where, int size);
void *entry;
int i;
/* we can't statically init this hack. */ unsigned char faker[64];
@@ -144,20 +168,32 @@ printk(BIOS_DEBUG, "Stage2 code done.\n");
ret = find_file(&archive, "normal/payload", &result);
- if (ret) {
printk(BIOS_ERR, "No such file '%s'.\n", "normal/payload");
die("FATAL: No payload found.\n");
- if (! ret)
legacy(&archive, "normal/payload", (void *)UNCOMPRESS_AREA, mem);
- /* new style lar boot. Install all the files in memory.
* By convention we take the entry point from the first
* one. Look for a cmdline as well.
*/
- for(i = 0, entry = (void *)0; ;i++) {
char filename[64];
void *newentry;
sprintf(filename, "normal/payload/segment%d", i);
archive.len = *(u32 *)0xfffffff4;
archive.start =(void *)(0UL-archive.len);
newentry = load_file(&archive, filename);
printk("newentry is %p\n", newentry);
if (newentry == (void *)-1)
break;
if (! entry)
}entry = newentry;
- ret = copy_file(&archive, "normal/payload", (void *)UNCOMPRESS_AREA);
- if (ret) {
printk(BIOS_ERR, "'%s' found, but could not load it.\n", "normal/payload");
die("FATAL: No usable payload found.\n");
- }
- printk(BIOS_SPEW, "all loaded, entry %p\n", entry);
- run_address(entry);
- ret = elfboot_mem(mem, (void *)UNCOMPRESS_AREA, result.reallen);
- die("FATAL: No usable payload found.\n");
- printk(BIOS_ERR, "elfboot_mem returns %d\n", ret);
- die ("FATAL: Last stage returned to LinuxBIOS.\n");
}
Index: arch/x86/Makefile
--- arch/x86/Makefile (revision 480) +++ arch/x86/Makefile (working copy) @@ -22,7 +22,7 @@ ifeq ($(CONFIG_ARCH_X86),y)
INITCFLAGS := $(CFLAGS) -I$(src)/include/arch/x86 -I$(src)/include \
-I$(obj) -fno-builtin
- -I$(obj) -fno-builtin
SILENT := >/dev/null 2>&1
@@ -78,7 +78,7 @@ endif $(Q)printf " LAR $(subst $(shell pwd)/,,$(@))\n" $(Q)rm -f $(obj)/linuxbios.rom
- $(Q)cd $(obj)/lar.tmp && ../util/lar/lar $(COMPRESSFLAG) -c \
- $(Q)cd $(obj)/lar.tmp && ../util/lar/lar $(PARSEELF) $(COMPRESSFLAG) -c \ ../linuxbios.rom \ $(LARFILES) \ -s $(ROM_SIZE) -b $(obj)/linuxbios.bootblock
@@ -122,6 +122,11 @@ endif endif
+ifeq ($(CONFIG_NOELF), y)
- PARSEELF = -e
+else
- PARSEELF =
+endif
STAGE0_OBJ := $(patsubst %,$(obj)/lib/%,$(STAGE0_LIB_OBJ)) \ $(patsubst %,$(obj)/arch/x86/%,$(STAGE0_ARCH_X86_OBJ)) \ @@ -143,6 +148,9 @@ $(Q)printf " OBJCOPY $(subst $(shell pwd)/,,$(@))\n" $(Q)$(OBJCOPY) -O binary $(obj)/stage0.o $(obj)/stage0.init
- $(Q)printf " OBJCOPY (stage0 link) $(subst $(shell pwd)/,,$(@))\n"
- $(Q)$(OBJCOPY) --prefix-symbols=stage0 $(obj)/stage0.o $(obj)/stage0_link.o
- $(Q)printf " TEST $(subst $(shell pwd)/,,$(@))\n" $(Q)test `wc -c < $(obj)/stage0.init` -gt 16128 && \ printf "Error. Bootblock got too big.\n" || true
Index: util/lar/lar.c
--- util/lar/lar.c (revision 480) +++ util/lar/lar.c (working copy) @@ -37,6 +37,7 @@ #define COPYRIGHT "Copyright (C) 2006-2007 coresystems GmbH"
static int isverbose = 0; +static int iselfparse = 0; static long larsize = 0; static char *bootblock = NULL; enum compalgo algo = none; @@ -44,7 +45,7 @@ static void usage(char *name) { printf("\nLAR - the LinuxBIOS Archiver " VERSION "\n" COPYRIGHT "\n\n"
"Usage: %s [-cxal] archive.lar [[[file1] file2] ...]\n\n", name);
printf("Examples:\n"); printf(" lar -c -s 32k -b bootblock myrom.lar foo nocompress:bar\n"); printf(" lar -a myrom.lar foo blob:baz\n");"Usage: %s [-ecxal] archive.lar [[[file1] file2] ...]\n\n", name);
@@ -66,13 +67,18 @@ printf(" \ta 'm' suffix to multiple the size by 1024*1024.\n"); printf(" -b [bootblock]\tSpecify the bootblock blob\n"); printf(" -C [lzma|nrv2b]\tSpecify the compression method to use\n\n");
printf(" -e pre-parse the payload ELF into LAR segments. Recommended\n\n");
printf("General options\n"); printf(" -v\tEnable verbose mode\n"); printf(" -V\tShow the version\n"); printf(" -h\tShow this help\n"); printf("\n");
+}
+int elfparse(void) +{
- return iselfparse;
}
/* Add a human touch to the LAR size by allowing suffixes: @@ -209,6 +215,7 @@ {"list", 0, 0, 'l'}, {"size", 1, 0, 's'}, {"bootblock", 1, 0, 'b'},
{"verbose", 0, 0, 'v'}, {"version", 0, 0, 'V'}, {"help", 0, 0, 'h'},{"elfparse", 1, 0, 'e'},
@@ -220,7 +227,7 @@ exit(1); }
- while ((opt = getopt_long(argc, argv, "acC:xls:b:vVh?",
- while ((opt = getopt_long(argc, argv, "acC:xels:b:vVh?", long_options, &option_index)) != EOF) { switch (opt) { case 'a':
@@ -240,6 +247,9 @@ case 'l': larmode = LIST; break;
case 'e':
iselfparse = 1;
case 'x': larmode = EXTRACT; break;break;
Index: util/lar/lar.h
--- util/lar/lar.h (revision 480) +++ util/lar/lar.h (working copy) @@ -60,6 +60,7 @@ typedef uint32_t u32; typedef uint8_t u8;
+/* NOTE -- This and the linuxbios lar.h are NOT IN SYNC. Be careful. */ struct lar_header { char magic[8]; u32 len; @@ -73,6 +74,8 @@ * 2 = nrv2b */ u32 compression;
- u32 entry; /* we might need to make this u64 */
- u32 loadaddress; /* ditto */
};
/**\struct Index: util/lar/lib.h =================================================================== --- util/lar/lib.h (revision 480) +++ util/lar/lib.h (working copy) @@ -38,6 +38,7 @@
/* prototypes for lar.c functions */ int verbose(void); +int parseelf(void); long get_larsize(void); char *get_bootblock(void);
Index: util/lar/Makefile
--- util/lar/Makefile (revision 480) +++ util/lar/Makefile (working copy) @@ -17,7 +17,6 @@ ## along with this program; if not, write to the Free Software ## Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA, 02110-1301 USA ##
LAROBJ := lar.o stream.o lib.o
LARDIR := lardir @@ -29,7 +28,6 @@ LARDIR += nrv2bdir
LAROBJ_ABS := $(patsubst %,$(obj)/util/lar/%,$(LAROBJ))
lardir: $(Q)printf " BUILD LAR\n" $(Q)mkdir -p $(obj)/util/lar