Re: [coreboot] Disassembly of coreboot binaries

[Chauhan, Himanshu]

On Mon, Dec 19, 2016 at 10:03 PM, Aaron Durbin  wrote:
> On Mon, Dec 19, 2016 at 9:55 AM, Chauhan, Himanshu
>  wrote:
> > On Mon, Dec 19, 2016 at 9:09 PM, Aaron Durbin  wrote:
> > > On Sun, Dec 18, 2016 at 11:04 PM, Chauhan, Himanshu
> > >  wrote:
> > > > On Mon, Dec 19, 2016 at 12:40 AM, Aaron Durbin  wrote:
> > > > > On Sun, Dec 18, 2016 at 9:37 AM, Chauhan, Himanshu
> > > > >  wrote:
> > > > > > Hi Aaron,
> > > > > >
> > > > > > I figured out the crash. It wan't because wrong load of the ROM image
> > > > > > (thanks to the nifty post_code which I could trap on IO). I see that
> > > > > > the page fault I am getting is in following code:
> > > > > > (gdb) list *(((0xfff81e41 - 0xfff80000)-200)+0x2000000)
> > > > >
> > > > > I'm curious about the 200 and 16MiB offset being applied.
> > > >
> > > > 0x2000000 is the new address where romstage is linked. Earlier
> > > > (atleast in 2014) the linked address used to be 0xfff80000. This is
> > > > the same address (guest physical) where I map the ROM code. In the
> > > > above calculation I am taking the offset from 0xfff80000 and adding to
> > > > the link address of romstage (0x2000000). The 0x200 is the difference
> > > > I see to map the addresses correctly. This calculation seems fine to
> > > > me because with this I am able to pin point all the earlier faults and
> > > > the post_code trap rIP.
> > >
> > > If you provide 'cbfstool print -k' output, I could most likely provide
> > > the exact offset mapping. Alternatively you could extract the
> > > romstage.elf from the image using 'cbfstool extract -m x86', but it
> > > won't have debug info. But it'd provide the information to compare
> > > against the pre-relocated image for the correct mapping.
> >
> > How exactly to run it? It says unknown option -k (cbfstool in build directory).
>
> ./coreboot-builds/sharedutils/cbfstool/cbfstool
> coreboot-builds/GOOGLE_REEF/coreboot.rom print -k

hchauhan@panini:build$ ./cbfstool coreboot.rom print -k

Performing operation on 'COREBOOT' region...

Name    Offset  Type    Metadata Size   Data Size       Total Size

cbfs master header      0x0     cbfs header     0x38    0x20    0x58

fallback/romstage       0x80    stage   0x64    0x320c  0x3270

fallback/ramstage       0x3300  stage   0x38    0x99d7  0x9a0f

config  0xcd40  raw     0x38    0x238   0x270

revision        0xcfc0  raw     0x38    0x239   0x271

cmos_layout.bin 0xd240  cmos_layout     0x38    0x304   0x33c

fallback/dsdt.aml       0xd580  raw     0x48    0xfb5   0xffd

fallback/payload        0xe580  payload 0x38    0x6b85  0x6bbd

(empty) 0x15140 null    0x28    0x6a998 0x6a9c0

bootblock       0x7fb00 bootblock       0x40    0x3c0   0x400
> That's an example after me building reef with abuild. How old is your
> coreboot checkout?

Pulled just a few days back.
> > > > > > 0x2001d79 is in imd_recover (src/lib/imd.c:139).
> > > > > > 134
> > > > > > 135     static void imdr_init(struct imdr *ir, void *upper_limit)
> > > > > > 136     {
> > > > > > 137             uintptr_t limit = (uintptr_t)upper_limit;
> > > > > > 138             /* Upper limit is aligned down to 4KiB */
> > > > > > 139             ir->limit = ALIGN_DOWN(limit, LIMIT_ALIGN);
> > > > > > 140             ir->r = NULL;
> > > > > > 141     }
> > > > > > 142
> > > > > > 143     static int imdr_create_empty(struct imdr *imdr, size_t root_size,
> > > > > >
> > > > > > I see that this function is being called multiple times (I added some
> > > > > > more post_code and see them being trapped). I get a series of page
> > > > > > faults which I am able to honour all but last.
> > > > >
> > > > > I don't see how imdr_init would be faulting. That's just assigning
> > > > > fields of a struct sitting on the stack. What's your stack pointer
> > > > > value at the time of the faults?
> > > >
> > > > "ir" should be on stack or on top of the RAM. Right now it looks like
> > > > its on top of the RAM. That area is not mapped initially. On a page
> > > > fault, I map a 4K page. For the reference, the following is the
> > > > register dump of coreboot. RSP is 0x9fe54.
> > >
> > > The values should not be striding. That object is always on the stack.
> > > Where the stack is located could be in low or high memory. I still
> > > need to know what platform you are targeting for the image to provide
> > > details. However, it would not be striding.
> >
> > I am building this for qemu i440-fx.
>
> OK. What is your cmos emulation returning at addresses 0x34, 0x35,
> 0x5d, 0x5c and 0x5b?
>
> I also don't understand why we're adding 16MiB to
> qemu_get_memory_size() unconditionally.
> > > > GUEST guest0/vcpu0 dump state:
> > > >
> > > > RAX: 0x9fe80 RBX: 0xfffff8 RCX: 0x1b RDX: 0x53a11439
> > > > R08: 0x0 R09: 0x0 R10: 0x0 R11: 0x0
> > > > R12: 0x0 R13: 0x0 R14: 0x0 R15: 0x0
> > > > RSP: 0x9fe54 RBP: 0xa0000 RDI: 0xfff801e4 RSI: 0x9fe80
> > > > RIP: 0xfff81e41
> > > >
> > > > CR0: 0xe0000011 CR2: 0x0 CR3: 0xa23000 CR4: 0x0
> > > > CS    : Sel: 0x00000008 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
> > > > 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type: 11)
> > > > DS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
> > > > 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > ES    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
> > > > 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > SS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
> > > > 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > FS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
> > > > 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > GS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
> > > > 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > GDT   : Sel: 0x00000000 Limit: 0x0000001f Base: 0xfff80200 (G:  0 DB:
> > > > 0 L:  0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
> > > > LDT   : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  0 DB:
> > > > 0 L:  0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
> > > > IDT   : Sel: 0x00000000 Limit: 0x00000000 Base: 0x00000000 (G:  0 DB:
> > > > 0 L:  0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
> > > > TR    : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  1 DB:
> > > > 0 L:  1 AVL:  1 P:  0 DPL:  0 S:  0 Type:  0)
> > > > RFLAGS: 0xa    [ ]
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f7fffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f7effc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f7dffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f7cffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f7bffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f7affc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f79ffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f78ffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f77ffc (rIP: 00000000FFF81E41)
> > > > > > (__handle_vm_exception:543) Guest fault: 0x7f76ffc (rIP: 00000000FFF81E41)
> > > > > >
> > > > > > <snip>
> > > > >
> > > > > Are those non-rIP addresses the page fault address?
> > > >
> > > > Guest fault: 0x7f7fffc is the address which I think is pointing to
> > > > "ir". If you look all the faulting addresses are 4K apart which is my
> > > > default page size for mapping all the guest pages. It also means that
> > > > multiple times "imdr_init" is being called it faults for different
> > > > addresses hence the same rIP.
> > >
> > > I just don't see how we're using that much stack. That doesn't seem
> > > right at all.
> >
> > Yes. Something is terribly wrong. I had this working back in 2014.
> > Please take a look at this video that I created at that time.
> > https://www.youtube.com/watch?v=jPAzzLQ0NgU
>
> i see you do have serial port. It'd be interesting to get full logs
> when the thing is booting to see where it goes off the rails.
> > I couldn't work on it for quite some time and meantime core boot
> > changed a lot. I have one question. In earlier core boot images,
> > romstage was linked to 0xfff80000 and now its 0x2000000. Any reason?
>
> It's just linked at CONFIG_ROMSTAGE_ADDR to avoid a double link step.
> It's linked once and cbfstool relocates the image when placing it into
> CBFS. It previously was linked at a specific address then the xip
> address was calculated by performing a pseudo CBFS add operation. Then
> romstage was re-linked and added to CBFS.
>
> The offset for address translation is the entry point differences
> between the 2 elf files. You can extract the one in coreboot.rom to
> get a the entry point of the romstage being ran.
> > > > > > handle_guest_realmode_page_fault: offset: 0x3ffc fault: 0x1003ffc reg: 0x1000000
> > > > > > handle_guest_realmode_page_fault: offset: 0x2ffc fault: 0x1002ffc reg: 0x1000000
> > > > > > handle_guest_realmode_page_fault: offset: 0x1ffc fault: 0x1001ffc reg: 0x1000000
> > > > > > handle_guest_realmode_page_fault: offset: 0xffc fault: 0x1000ffc reg: 0x1000000
> > > > >
> > > > > What is the above detailing? I'm not sure what the 'fault' value means.
> > > >
> > > > These are same as Guest fault above. You can disregard them.
> > > > > > (__handle_vm_exception:561) ERROR: No region mapped to guest physical: 0xfffffc
> > > > > >
> > > > > > I want to understand why imd_recover gets called multiple times
> > > > > > starting from top of memory (128MB is what I have assigned to the
> > > > > > guest) to 16MB last (after which I can't honour). There is something
> > > > > > amiss in my understanding of core boot memory map.
> > > > > >
> > > > > > Could you please help?
> > > > >
> > > > > The imd library contains the implementation of cbmem. See
> > > > > include/cbmem.h for more details, but how it works is that the
> > > > > platform needs to supply the implementation of cbmem_top() which
> > > > > defines the exclusive upper boundary to start growing entries downward
> > > > > from. There is a large and small object size with large blocks being
> > > > > 4KiB in size and small blocks being 32 byes. I don't understand why
> > > > > the faulting addresses are offset from 128MiB by 512KiB with a 4KiB
> > > > > stride.
> > > > >
> > > > > What platform are you targeting for your coreboot build? Are you
> > > > > restarting the instruction that faults? I'm really curious about the
> > > > > current fault patterns.  It looks like things are faulting around
> > > > > accessing the imd_root_pointer root_offset field. Are these faults
> > > > > reads or writes? However, that's assuming cbmem_top() is returning
> > > > > 128MiB-512KiB. However, it doesn't explain the successive strides. Do
> > > > > you have serial port emulation to get the console messages out?
> > > > >
> > > > > So in your platform code ensure 2 things are happening:
> > > > > cbmem_top() returns a highest address in 'ram' of the guest once
> > > > >
> > > > > it's online. 128MiB if that's your expectation. The value cbmem_top()
> > > > > returns should never change from successive calls aside from NULL
> > > > > being returned when ram is not yet available.

It will always return 0x6f8. This is decided when the guest is created.
> > > > > cbmem_initialize_empty() is called one time once the 'ram' is
> > > > >
> > > > > online for use in the non-S3 resume path and cbmem_initialize() in the
> > > > > S3 resume path. If S3 isn't supported in your guest then just use
> > > > > cbmem_initialize_empty().
> > > >
> > > > I will look in it. I see that RAM top is being provided by the CMOS
> > > > emulator. I will look at cbmem_initialize_empty().
> > >
> > > If you could provide me the info on the platform you are targeting
> > > coreboot builds with it'd be easier to analyze. Where is this 'CMOS
> > > emulator' and why is it needed?
> >
> > Coreboot calls on port 0x34/0x35 to get the amount of memory. The cmos
> > emulator traps these (just like qemu) and provides that information to
> > core boot.
>
> cbmem_recovery(0) is effectively cbmem_initialize_empty(). That's
> being called in src/mainboard/emulation/qemu-i440fx/romstage.c.
>
> Your RSP value of 0x9fe54 aligns with
> src/mainboard/emulation/qemu-i440fx/cache_as_ram.inc using 0xa0000 as
> the initial stack.
>
> So I don't think imd_recover() is your culprit. Something is changing
> the value of cbmem_top() it feels like.
> > > > > > Regards
> > > > > > Himanshu
> > > > > >
> > > > > > On Wed, Dec 14, 2016 at 9:27 PM, Chauhan, Himanshu
> > > > > >  wrote:
> > > > > > > Hi Aaron,
> > > > > > >
> > > > > > > Yes, I am mapping the memory where coreboot.rom is loaded to upper 4GiB. I
> > > > > > > create a fixed shadow page table entry for reset vector.
> > > > > > >
> > > > > > > Coreboot doesn't have a linked address of RIP that I shared. I think with
> > > > > > > the increase in size of coreboot (from the previous tag I was using) the
> > > > > > > load address (guest physical) has changed. I used to calculate the load
> > > > > > > address manually. I will check this and get back.
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > On Wed, Dec 14, 2016 at 8:17 PM, Aaron Durbin  wrote:
> > > > > > >>
> > > > > > >> On Wed, Dec 14, 2016 at 3:11 AM, Chauhan, Himanshu
> > > > > > >>  wrote:
> > > > > > >> > Hi,
> > > > > > >> >
> > > > > > >> > I am working on a hypvervisor and am using coreboot + FILO as guest
> > > > > > >> > BIOS.
> > > > > > >> > While things were fine a while back, it has stopped working. I see that
> > > > > > >> > my
> > > > > > >> > hypervisor can't handle address 0xFFFFFC while coreboot's RIP is at
> > > > > > >> > 0xfff81e41.
> > > > > > >>
> > > > > > >>
> > > > > > >> How are you loading up coreboot.rom in the VM? Are you just memory
> > > > > > >> mapping it at the top of 4GiB address space? If so, what does
> > > > > > >> 'cbfstool coreboot.rom print' show?
> > > > > > >>
> > > > > > >> >
> > > > > > >> > The exact register dump of guest is as follow:
> > > > > > >> >
> > > > > > >> > [guest0/uart0] (__handle_vm_exception:558) ERROR: No region mapped to
> > > > > > >> > guest
> > > > > > >> > physical: 0xfffffc
> > > > > > >> >
> > > > > > >> > GUEST guest0/vcpu0 dump state:
> > > > > > >> >
> > > > > > >> > RAX: 0x9fe80 RBX: 0xfffff8 RCX: 0x1b RDX: 0x53a11439
> > > > > > >> > R08: 0x0 R09: 0x0 R10: 0x0 R11: 0x0
> > > > > > >> > R12: 0x0 R13: 0x0 R14: 0x0 R15: 0x0
> > > > > > >> > RSP: 0x9fe54 RBP: 0xa0000 RDI: 0xfff801e4 RSI: 0x9fe80
> > > > > > >> > RIP: 0xfff81e41
> > > > > > >> >
> > > > > > >> > CR0: 0xe0000011 CR2: 0x0 CR3: 0xa23000 CR4: 0x0
> > > > > > >> > CS    : Sel: 0x00000008 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type: 11)
> > > > > > >> > DS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > > > >> > ES    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > > > >> > SS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > > > >> > FS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > > > >> > GS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
> > > > > > >> > GDT   : Sel: 0x00000000 Limit: 0x0000001f Base: 0xfff80200 (G:  0 DB:  0
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
> > > > > > >> > LDT   : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  0 DB:  0
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
> > > > > > >> > IDT   : Sel: 0x00000000 Limit: 0x00000000 Base: 0x00000000 (G:  0 DB:  0
> > > > > > >> > L:
> > > > > > >> > 0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
> > > > > > >> > TR    : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  1 DB:  0
> > > > > > >> > L:
> > > > > > >> > 1 AVL:  1 P:  0 DPL:  0 S:  0 Type:  0)
> > > > > > >> > RFLAGS: 0xa    [ ]
> > > > > > >> >
> > > > > > >> > I want to know which binary file (.o) should I disassemble to look at
> > > > > > >> > the
> > > > > > >> > RIP?
> > > > > > >> >
> > > > > > >> > I was looking at
> > > > > > >> > objdump -D  -mi386 -Maddr16,data16 generated/ramstage.o
> > > > > > >> >
> > > > > > >> > but this is prior to linking and thus only has offsets.
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> >
> > > > > > >> > Regards
> > > > > > >> > [Himanshu Chauhan]
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > coreboot mailing list: [e-mail]
> > > > > > >> > https://www.coreboot.org/mailman/listinfo/coreboot
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Regards
> > > > > > > [Himanshu Chauhan]
> > > > > > >
> > > > > >
> > > > > > Regards
> > > > > > [Himanshu Chauhan]
> > > >
> > > > Regards
> > > > [Himanshu Chauhan]
> >
> > Regards
> > [Himanshu Chauhan]