On Wed, 8 Jul 2015 09:10:14 +0100 Stefan Hajnoczi stefanha@gmail.com wrote:
On Sat, Jul 4, 2015 at 6:57 PM, Kevin O'Connor kevin@koconnor.net wrote:
On Fri, Jul 03, 2015 at 03:12:14PM +0100, Stefan Hajnoczi wrote:
On Fri, Jul 3, 2015 at 2:13 PM, Kevin O'Connor kevin@koconnor.net wrote:
I took a look a month or so ago:
http://article.gmane.org/gmane.comp.emulators.kvm.devel/136207
Last test I ran I had SeaBIOS down to 16ms (not including the time to deploy or jump to the linux kernel). More reductions are also possible - the above was just with Kconfig settings.
I've been measuring the time for different configurations and setups.
The conclusion is that the main bottleneck to boot QEMU is fw_cfg.
Running a default SeaBIOS configuration gives these times (in seconds):
(QEMU startup time is the time between the execution and the first kvm_entry, BIOS startup time is the time between the first kvm_entry and the ROM booting, and fw_cfg is the time between the ROM booting and the jump to the Linux kernel).
QEMU startup time: .033 BIOS startup time: .154 fw_cfg setup time: .375
And these results are more or less the same in QBoot, when using fw_cfg:
QEMU startup time: .026 BIOS startup time: .013 fw_cfg setup time: .373
The difference between SeaBIOS and QBoot is big, but, as I said, this SeaBIOS is not stripped-down. Using the .config that Kevin sent, I can get .01 seconds, but I cannot boot a ROM (I still have to play with the configuration options until it boots). Probably it can do it in less than 20 msec, which not far from QBoot.
On the other side, QBoot with cbfs can boot really fast:
QEMU startup time: .027 Kernel setup time: .017 Total time: .045
And I'll check CoreBoot once more, to try to figure out why it takes so long.
Thanks Marc
On Wed, Jul 08, 2015 at 03:44:33PM +0200, Marc Marí wrote:
On Wed, 8 Jul 2015 09:10:14 +0100 Stefan Hajnoczi stefanha@gmail.com wrote:
On Sat, Jul 4, 2015 at 6:57 PM, Kevin O'Connor kevin@koconnor.net wrote:
On Fri, Jul 03, 2015 at 03:12:14PM +0100, Stefan Hajnoczi wrote:
On Fri, Jul 3, 2015 at 2:13 PM, Kevin O'Connor kevin@koconnor.net wrote:
I took a look a month or so ago:
http://article.gmane.org/gmane.comp.emulators.kvm.devel/136207
Last test I ran I had SeaBIOS down to 16ms (not including the time to deploy or jump to the linux kernel). More reductions are also possible - the above was just with Kconfig settings.
I've been measuring the time for different configurations and setups.
The conclusion is that the main bottleneck to boot QEMU is fw_cfg.
Running a default SeaBIOS configuration gives these times (in seconds):
(QEMU startup time is the time between the execution and the first kvm_entry, BIOS startup time is the time between the first kvm_entry and the ROM booting, and fw_cfg is the time between the ROM booting and the jump to the Linux kernel).
QEMU startup time: .033 BIOS startup time: .154 fw_cfg setup time: .375
And these results are more or less the same in QBoot, when using fw_cfg:
QEMU startup time: .026 BIOS startup time: .013 fw_cfg setup time: .373
The difference between SeaBIOS and QBoot is big, but, as I said, this SeaBIOS is not stripped-down. Using the .config that Kevin sent, I can get .01 seconds, but I cannot boot a ROM (I still have to play with the configuration options until it boots). Probably it can do it in less than 20 msec, which not far from QBoot.
On the other side, QBoot with cbfs can boot really fast:
QEMU startup time: .027 Kernel setup time: .017 Total time: .045
Thanks. Can you try the latest seabios with the patch below and the attached config? The patch is a complete hack, but with it you should be able to launch seabios with the same cbfs.rom file generated for qboot. I see SeaBIOS take about 20ms on my old AMD system (again, further optimizations could be made with some simple code changes). I used the following qemu command line:
qemu-system-x86_64 -drive if=pflash,file=../seabios/out/bios.bin,readonly=on -drive if=pflash,file=cbfs.rom,readonly=on -enable-kvm -serial mon:stdio
As a side note, if there is real interest in placing a kernel in a "flash" like memory mapped device, it would be nice to do something like the CBFS simple-elf (SELF) format so that the firmware doesn't have to parse the bzimage.
-Kevin
diff --git a/src/Kconfig b/src/Kconfig index 14c38fb..7484782 100644 --- a/src/Kconfig +++ b/src/Kconfig @@ -74,7 +74,7 @@ endchoice "bootorder" file.
config COREBOOT_FLASH - depends on COREBOOT + #depends on COREBOOT bool "coreboot CBFS support" default y help diff --git a/src/fw/coreboot.c b/src/fw/coreboot.c index 8fd8449..3af67a8 100644 --- a/src/fw/coreboot.c +++ b/src/fw/coreboot.c @@ -421,6 +421,9 @@ coreboot_cbfs_init(void) return;
struct cbfs_header *hdr = *(void **)(CONFIG_CBFS_LOCATION - 4); + if (CBFS_HEADER_MAGIC && (u32)hdr > CONFIG_CBFS_LOCATION) + // Looks like the pointer is relative to CONFIG_CBFS_LOCATION + hdr = (void*)hdr + CONFIG_CBFS_LOCATION; if (hdr->magic != cpu_to_be32(CBFS_HEADER_MAGIC)) { dprintf(1, "Unable to find CBFS (ptr=%p; got %x not %x)\n" , hdr, hdr->magic, cpu_to_be32(CBFS_HEADER_MAGIC)); @@ -464,6 +467,73 @@ coreboot_cbfs_init(void) process_links_file(); }
+#include "bregs.h" +struct cbfs_file *File_cmdline, *File_initrd; + +void +cbfs_run_vmlinuz(struct cbfs_file *fhdr) +{ + dprintf(1, "Loading kernel\n"); + void *kernel_src = (void*)fhdr + be32_to_cpu(fhdr->offset); + u32 kernel_size = be32_to_cpu(fhdr->len); + if (readl(kernel_src+0x202) != 0x53726448) { + dprintf(1, "Not valid kernel\n"); + return; + } + u16 protocol = readw(kernel_src+0x206); + if (protocol < 0x203) { + dprintf(1, "Old kernel (v %x) not supported\n", protocol); + return; + } + u32 cmdline_size = 0, initrd_size = 0; + void *initrd_src, *cmdline_src; + if (File_cmdline) { + cmdline_size = be32_to_cpu(File_cmdline->len); + cmdline_src = (void*)File_cmdline + be32_to_cpu(File_cmdline->offset); + } + if (File_initrd) { + initrd_size = be32_to_cpu(File_initrd->len); + initrd_src = (void*)File_initrd + be32_to_cpu(File_initrd->offset); + } + u32 real_addr=0x10000, cmdline_addr=0x20000; + u32 prot_addr=0x100000, initrd_addr=0; + if (initrd_size) { + u32 initrd_max = readl(kernel_src+0x22c); + if (initrd_max > LegacyRamSize - 1) + initrd_max = LegacyRamSize - 1; + initrd_addr = ALIGN_DOWN(initrd_max - initrd_size, 4096); + } + u32 setup_size = (readb(kernel_src+0x1f1) + 1) * 512; + if (setup_size <= 512) + setup_size = 5 * 512; + kernel_size -= setup_size; + + // Copy data + memcpy((void*)real_addr, (void*)kernel_src, setup_size); + memcpy((void*)prot_addr, (void*)kernel_src + setup_size, kernel_size); + if (initrd_size) + memcpy((void*)initrd_addr, (void*)initrd_src, initrd_size); + if (cmdline_size) + memcpy((void*)cmdline_addr, (void*)cmdline_src, cmdline_size); + writel((void*)real_addr+0x228, cmdline_addr); + writeb((void*)real_addr+0x210, 0xB0); + writeb((void*)real_addr+0x211, readb((void*)real_addr+0x211) | 0x80); + writew((void*)real_addr+0x224, cmdline_addr-real_addr-0x200); + writel((void*)real_addr+0x218, initrd_addr); + writel((void*)real_addr+0x21c, initrd_size); + + dprintf(1, "Jumping to kernel %d@%x %d@%x %d@%x %d@%x\n" + , setup_size, real_addr, cmdline_size, cmdline_addr + , kernel_size, prot_addr, initrd_size, initrd_addr); + struct bregs br; + memset(&br, 0, sizeof(br)); + extern void kernel_stub(void); + br.ebx = real_addr >> 4; + br.edx = cmdline_addr - real_addr - 16; + br.code = SEGOFF(SEG_BIOS, (u32)kernel_stub - BUILD_BIOS_ADDR); + farcall16big(&br); +} + struct cbfs_payload_segment { u32 type; u32 compression; @@ -489,6 +559,10 @@ cbfs_run_payload(struct cbfs_file *fhdr) if (!CONFIG_COREBOOT_FLASH || !fhdr) return; dprintf(1, "Run %s\n", fhdr->filename); + if (strcmp(fhdr->filename, "vmlinuz") == 0) { + cbfs_run_vmlinuz(fhdr); + return; + } struct cbfs_payload *pay = (void*)fhdr + be32_to_cpu(fhdr->offset); struct cbfs_payload_segment *seg = pay->segments; for (;;) { @@ -550,4 +624,19 @@ cbfs_payload_setup(void) char *desc = znprintf(MAXDESCSIZE, "Payload [%s]", &filename[4]); boot_add_cbfs(cfile->fhdr, desc, bootprio_find_named_rom(filename, 0)); } + + file = romfile_find("vmlinuz"); + if (file) { + struct cbfs_romfile_s *cfile; + cfile = container_of(file, struct cbfs_romfile_s, file); + char *desc = znprintf(MAXDESCSIZE, "vmlinuz"); + boot_add_cbfs(cfile->fhdr, desc, bootprio_find_named_rom("vmlinuz", 0)); + + file = romfile_find("cmdline"); + if (file) + File_cmdline = container_of(file, struct cbfs_romfile_s, file)->fhdr; + file = romfile_find("initrd"); + if (file) + File_initrd = container_of(file, struct cbfs_romfile_s, file)->fhdr; + } } diff --git a/src/romlayout.S b/src/romlayout.S index 7938e22..f98b820 100644 --- a/src/romlayout.S +++ b/src/romlayout.S @@ -197,6 +197,27 @@ __farcall16: IRQ_TRAMPOLINE 4a
+ DECLFUNC kernel_stub +kernel_stub: + movw %bx, %ds + movw %bx, %es + movw %bx, %fs + movw %bx, %gs + movw %bx, %ss + movl %edx, %esp + addw $0x20, %bx + pushw %bx // push CS + pushw $0 // push IP + xorl %eax, %eax + xorl %ebx, %ebx + xorl %ecx, %ecx + xorl %edx, %edx + xorl %edi, %edi + xorl %esi, %esi + xorl %ebp, %ebp + lretw + + /**************************************************************** * Misc. entry points. ****************************************************************/
On Wed, 8 Jul 2015 18:57:26 -0400 Kevin O'Connor kevin@koconnor.net wrote:
On Wed, Jul 08, 2015 at 03:44:33PM +0200, Marc Marí wrote:
On Wed, 8 Jul 2015 09:10:14 +0100 Stefan Hajnoczi stefanha@gmail.com wrote:
On Sat, Jul 4, 2015 at 6:57 PM, Kevin O'Connor kevin@koconnor.net wrote:
On Fri, Jul 03, 2015 at 03:12:14PM +0100, Stefan Hajnoczi wrote:
On Fri, Jul 3, 2015 at 2:13 PM, Kevin O'Connor kevin@koconnor.net wrote:
I took a look a month or so ago:
http://article.gmane.org/gmane.comp.emulators.kvm.devel/136207
Last test I ran I had SeaBIOS down to 16ms (not including the time to deploy or jump to the linux kernel). More reductions are also possible - the above was just with Kconfig settings.
I've been measuring the time for different configurations and setups.
The conclusion is that the main bottleneck to boot QEMU is fw_cfg.
Running a default SeaBIOS configuration gives these times (in seconds):
(QEMU startup time is the time between the execution and the first kvm_entry, BIOS startup time is the time between the first kvm_entry and the ROM booting, and fw_cfg is the time between the ROM booting and the jump to the Linux kernel).
QEMU startup time: .033 BIOS startup time: .154 fw_cfg setup time: .375
And these results are more or less the same in QBoot, when using fw_cfg:
QEMU startup time: .026 BIOS startup time: .013 fw_cfg setup time: .373
The difference between SeaBIOS and QBoot is big, but, as I said, this SeaBIOS is not stripped-down. Using the .config that Kevin sent, I can get .01 seconds, but I cannot boot a ROM (I still have to play with the configuration options until it boots). Probably it can do it in less than 20 msec, which not far from QBoot.
On the other side, QBoot with cbfs can boot really fast:
QEMU startup time: .027 Kernel setup time: .017 Total time: .045
Thanks. Can you try the latest seabios with the patch below and the attached config? The patch is a complete hack, but with it you should be able to launch seabios with the same cbfs.rom file generated for qboot. I see SeaBIOS take about 20ms on my old AMD system (again, further optimizations could be made with some simple code changes). I used the following qemu command line:
qemu-system-x86_64 -drive if=pflash,file=../seabios/out/bios.bin,readonly=on -drive if=pflash,file=cbfs.rom,readonly=on -enable-kvm -serial mon:stdio
As a side note, if there is real interest in placing a kernel in a "flash" like memory mapped device, it would be nice to do something like the CBFS simple-elf (SELF) format so that the firmware doesn't have to parse the bzimage.
-Kevin
I tested with the given configuration and it boots. And it is faster than QBoot.
SeaBIOS: - QEMU startup time: .030 - BIOS startup time: .008 - cbfs setup time: .0021
QBoot: - QEMU startup time: .030 - BIOS startup time: .013 - cbfs setup time: .0045
The profiling points are the same as before. "cbfs setup time" is the time between the call to the cbfs-processing function and the jump to the Linux kernel.
This is a great improvement on the original times.
Marc
Hi,
The conclusion is that the main bottleneck to boot QEMU is fw_cfg.
https://www.kraxel.org/cgit/qemu/log/?h=rebase/fw-cfg-dma-wip
Some experimental (and untested) bits implementing a dma interface for fw_cfg (also some unrelated fw_cfg stuff).
You might want try wire that up for x86 and see how it speeds up things ...
cheers, Gerd
On Thu, Jul 09, 2015 at 10:45:23AM +0200, Gerd Hoffmann wrote:
Hi,
The conclusion is that the main bottleneck to boot QEMU is fw_cfg.
https://www.kraxel.org/cgit/qemu/log/?h=rebase/fw-cfg-dma-wip
Some experimental (and untested) bits implementing a dma interface for fw_cfg (also some unrelated fw_cfg stuff).
You might want try wire that up for x86 and see how it speeds up things ...
Interesting. This probably isn't the right place to discuss the implementation, but I have a couple of comments on the dma interface.
The interface doesn't have a "skip" field and that's quite helpful in the firmware to avoid having to memmove stuff around. (So, instead of select/addr/len it would be preferable to have select/addr/offset/len.)
Have you considered using a transfer descriptor struct and dma'ing that? That is, use just two 32bit IO registers (descriptor addr high / descriptor addr low, active on write to low register) and then define a descriptor struct with the select, target addr, offset, len, and status fields - fw_cfg could then read the descriptor struct, perform the requested action, and then update the descriptor struct upon completion.
Cheers, -Kevin
-
Gerd Hoffmann -
Kevin O'Connor -
Marc Marí