[coreboot] Disassembly of coreboot binaries

Chauhan, Himanshu hschauhan at nulltrace.org
Mon Dec 19 16:55:53 CET 2016 

On Mon, Dec 19, 2016 at 9:09 PM, Aaron Durbin <adurbin at google.com> wrote:
> On Sun, Dec 18, 2016 at 11:04 PM, Chauhan, Himanshu
> <hschauhan at nulltrace.org> wrote:
>> On Mon, Dec 19, 2016 at 12:40 AM, Aaron Durbin <adurbin at google.com> wrote:
>>> On Sun, Dec 18, 2016 at 9:37 AM, Chauhan, Himanshu
>>> <hschauhan at nulltrace.org> wrote:
>>>> Hi Aaron,
>>>>
>>>> I figured out the crash. It wan't because wrong load of the ROM image
>>>> (thanks to the nifty post_code which I could trap on IO). I see that
>>>> the page fault I am getting is in following code:
>>>> (gdb) list *(((0xfff81e41 - 0xfff80000)-200)+0x2000000)
>>>
>>> I'm curious about the 200 and 16MiB offset being applied.
>>
>> 0x2000000 is the new address where romstage is linked. Earlier
>> (atleast in 2014) the linked address used to be 0xfff80000. This is
>> the same address (guest physical) where I map the ROM code. In the
>> above calculation I am taking the offset from 0xfff80000 and adding to
>> the link address of romstage (0x2000000). The 0x200 is the difference
>> I see to map the addresses correctly. This calculation seems fine to
>> me because with this I am able to pin point all the earlier faults and
>> the post_code trap rIP.
>>
>
> If you provide 'cbfstool print -k' output, I could most likely provide
> the exact offset mapping. Alternatively you could extract the
> romstage.elf from the image using 'cbfstool extract -m x86', but it
> won't have debug info. But it'd provide the information to compare
> against the pre-relocated image for the correct mapping.
>
How exactly to run it? It says unknown option -k (cbfstool in build directory).

>>>
>>>> 0x2001d79 is in imd_recover (src/lib/imd.c:139).
>>>> 134
>>>> 135     static void imdr_init(struct imdr *ir, void *upper_limit)
>>>> 136     {
>>>> 137             uintptr_t limit = (uintptr_t)upper_limit;
>>>> 138             /* Upper limit is aligned down to 4KiB */
>>>> 139             ir->limit = ALIGN_DOWN(limit, LIMIT_ALIGN);
>>>> 140             ir->r = NULL;
>>>> 141     }
>>>> 142
>>>> 143     static int imdr_create_empty(struct imdr *imdr, size_t root_size,
>>>>
>>>> I see that this function is being called multiple times (I added some
>>>> more post_code and see them being trapped). I get a series of page
>>>> faults which I am able to honour all but last.
>>>
>>> I don't see how imdr_init would be faulting. That's just assigning
>>> fields of a struct sitting on the stack. What's your stack pointer
>>> value at the time of the faults?
>>
>> "ir" should be on stack or on top of the RAM. Right now it looks like
>> its on top of the RAM. That area is not mapped initially. On a page
>> fault, I map a 4K page. For the reference, the following is the
>> register dump of coreboot. RSP is 0x9fe54.
>>
>
> The values should not be striding. That object is always on the stack.
> Where the stack is located could be in low or high memory. I still
> need to know what platform you are targeting for the image to provide
> details. However, it would not be striding.

I am building this for qemu i440-fx.

>
>> GUEST guest0/vcpu0 dump state:
>>
>> RAX: 0x9fe80 RBX: 0xfffff8 RCX: 0x1b RDX: 0x53a11439
>> R08: 0x0 R09: 0x0 R10: 0x0 R11: 0x0
>> R12: 0x0 R13: 0x0 R14: 0x0 R15: 0x0
>> RSP: 0x9fe54 RBP: 0xa0000 RDI: 0xfff801e4 RSI: 0x9fe80
>> RIP: 0xfff81e41
>>
>> CR0: 0xe0000011 CR2: 0x0 CR3: 0xa23000 CR4: 0x0
>> CS    : Sel: 0x00000008 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
>> 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type: 11)
>> DS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
>> 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>> ES    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
>> 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>> SS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
>> 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>> FS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
>> 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>> GS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:
>> 1 L:  0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>> GDT   : Sel: 0x00000000 Limit: 0x0000001f Base: 0xfff80200 (G:  0 DB:
>> 0 L:  0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
>> LDT   : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  0 DB:
>> 0 L:  0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
>> IDT   : Sel: 0x00000000 Limit: 0x00000000 Base: 0x00000000 (G:  0 DB:
>> 0 L:  0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
>> TR    : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  1 DB:
>> 0 L:  1 AVL:  1 P:  0 DPL:  0 S:  0 Type:  0)
>> RFLAGS: 0xa    [ ]
>>
>>
>>>>
>>>> (__handle_vm_exception:543) Guest fault: 0x7f7fffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f7effc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f7dffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f7cffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f7bffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f7affc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f79ffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f78ffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f77ffc (rIP: 00000000FFF81E41)
>>>> (__handle_vm_exception:543) Guest fault: 0x7f76ffc (rIP: 00000000FFF81E41)
>>>> <snip>
>>>
>>> Are those non-rIP addresses the page fault address?
>>
>> Guest fault: 0x7f7fffc is the address which I think is pointing to
>> "ir". If you look all the faulting addresses are 4K apart which is my
>> default page size for mapping all the guest pages. It also means that
>> multiple times "imdr_init" is being called it faults for different
>> addresses hence the same rIP.
>
> I just don't see how we're using that much stack. That doesn't seem
> right at all.
>

Yes. Something is terribly wrong. I had this working back in 2014.
Please take a look at this video that I created at that time.
https://www.youtube.com/watch?v=jPAzzLQ0NgU

I couldn't work on it for quite some time and meantime core boot
changed a lot. I have one question. In earlier core boot images,
romstage was linked to 0xfff80000 and now its 0x2000000. Any reason?

>>
>>>
>>>>
>>>> handle_guest_realmode_page_fault: offset: 0x3ffc fault: 0x1003ffc reg: 0x1000000
>>>> handle_guest_realmode_page_fault: offset: 0x2ffc fault: 0x1002ffc reg: 0x1000000
>>>> handle_guest_realmode_page_fault: offset: 0x1ffc fault: 0x1001ffc reg: 0x1000000
>>>> handle_guest_realmode_page_fault: offset: 0xffc fault: 0x1000ffc reg: 0x1000000
>>>
>>> What is the above detailing? I'm not sure what the 'fault' value means.
>>
>> These are same as Guest fault above. You can disregard them.
>>
>>>
>>>>
>>>> (__handle_vm_exception:561) ERROR: No region mapped to guest physical: 0xfffffc
>>>>
>>>>
>>>> I want to understand why imd_recover gets called multiple times
>>>> starting from top of memory (128MB is what I have assigned to the
>>>> guest) to 16MB last (after which I can't honour). There is something
>>>> amiss in my understanding of core boot memory map.
>>>>
>>>> Could you please help?
>>>
>>> The imd library contains the implementation of cbmem. See
>>> include/cbmem.h for more details, but how it works is that the
>>> platform needs to supply the implementation of cbmem_top() which
>>> defines the exclusive upper boundary to start growing entries downward
>>> from. There is a large and small object size with large blocks being
>>> 4KiB in size and small blocks being 32 byes. I don't understand why
>>> the faulting addresses are offset from 128MiB by 512KiB with a 4KiB
>>> stride.
>>>
>>> What platform are you targeting for your coreboot build? Are you
>>> restarting the instruction that faults? I'm really curious about the
>>> current fault patterns.  It looks like things are faulting around
>>> accessing the imd_root_pointer root_offset field. Are these faults
>>> reads or writes? However, that's assuming cbmem_top() is returning
>>> 128MiB-512KiB. However, it doesn't explain the successive strides. Do
>>> you have serial port emulation to get the console messages out?
>>>
>>> So in your platform code ensure 2 things are happening:
>>>
>>> 1. cbmem_top() returns a highest address in 'ram' of the guest once
>>> it's online. 128MiB if that's your expectation. The value cbmem_top()
>>> returns should never change from successive calls aside from NULL
>>> being returned when ram is not yet available.
>>> 2. cbmem_initialize_empty() is called one time once the 'ram' is
>>> online for use in the non-S3 resume path and cbmem_initialize() in the
>>> S3 resume path. If S3 isn't supported in your guest then just use
>>> cbmem_initialize_empty().
>>>
>>
>> I will look in it. I see that RAM top is being provided by the CMOS
>> emulator. I will look at cbmem_initialize_empty().
>
> If you could provide me the info on the platform you are targeting
> coreboot builds with it'd be easier to analyze. Where is this 'CMOS
> emulator' and why is it needed?

Coreboot calls on port 0x34/0x35 to get the amount of memory. The cmos
emulator traps these (just like qemu) and provides that information to
core boot.

>
>>
>>>>
>>>> Regards
>>>> Himanshu
>>>>
>>>> On Wed, Dec 14, 2016 at 9:27 PM, Chauhan, Himanshu
>>>> <hschauhan at nulltrace.org> wrote:
>>>>> Hi Aaron,
>>>>>
>>>>> Yes, I am mapping the memory where coreboot.rom is loaded to upper 4GiB. I
>>>>> create a fixed shadow page table entry for reset vector.
>>>>>
>>>>> Coreboot doesn't have a linked address of RIP that I shared. I think with
>>>>> the increase in size of coreboot (from the previous tag I was using) the
>>>>> load address (guest physical) has changed. I used to calculate the load
>>>>> address manually. I will check this and get back.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Wed, Dec 14, 2016 at 8:17 PM, Aaron Durbin <adurbin at google.com> wrote:
>>>>>>
>>>>>> On Wed, Dec 14, 2016 at 3:11 AM, Chauhan, Himanshu
>>>>>> <hschauhan at nulltrace.org> wrote:
>>>>>> > Hi,
>>>>>> >
>>>>>> > I am working on a hypvervisor and am using coreboot + FILO as guest
>>>>>> > BIOS.
>>>>>> > While things were fine a while back, it has stopped working. I see that
>>>>>> > my
>>>>>> > hypervisor can't handle address 0xFFFFFC while coreboot's RIP is at
>>>>>> > 0xfff81e41.
>>>>>>
>>>>>>
>>>>>> How are you loading up coreboot.rom in the VM? Are you just memory
>>>>>> mapping it at the top of 4GiB address space? If so, what does
>>>>>> 'cbfstool coreboot.rom print' show?
>>>>>>
>>>>>> >
>>>>>> > The exact register dump of guest is as follow:
>>>>>> >
>>>>>> > [guest0/uart0] (__handle_vm_exception:558) ERROR: No region mapped to
>>>>>> > guest
>>>>>> > physical: 0xfffffc
>>>>>> >
>>>>>> > GUEST guest0/vcpu0 dump state:
>>>>>> >
>>>>>> > RAX: 0x9fe80 RBX: 0xfffff8 RCX: 0x1b RDX: 0x53a11439
>>>>>> > R08: 0x0 R09: 0x0 R10: 0x0 R11: 0x0
>>>>>> > R12: 0x0 R13: 0x0 R14: 0x0 R15: 0x0
>>>>>> > RSP: 0x9fe54 RBP: 0xa0000 RDI: 0xfff801e4 RSI: 0x9fe80
>>>>>> > RIP: 0xfff81e41
>>>>>> >
>>>>>> > CR0: 0xe0000011 CR2: 0x0 CR3: 0xa23000 CR4: 0x0
>>>>>> > CS    : Sel: 0x00000008 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type: 11)
>>>>>> > DS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>>>>>> > ES    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>>>>>> > SS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>>>>>> > FS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>>>>>> > GS    : Sel: 0x00000010 Limit: 0xffffffff Base: 0x00000000 (G:  1 DB:  1
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  1 DPL:  0 S:  1 Type:  3)
>>>>>> > GDT   : Sel: 0x00000000 Limit: 0x0000001f Base: 0xfff80200 (G:  0 DB:  0
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
>>>>>> > LDT   : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  0 DB:  0
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
>>>>>> > IDT   : Sel: 0x00000000 Limit: 0x00000000 Base: 0x00000000 (G:  0 DB:  0
>>>>>> > L:
>>>>>> > 0 AVL:  0 P:  0 DPL:  0 S:  0 Type:  0)
>>>>>> > TR    : Sel: 0x00000000 Limit: 0x0000ffff Base: 0x00000000 (G:  1 DB:  0
>>>>>> > L:
>>>>>> > 1 AVL:  1 P:  0 DPL:  0 S:  0 Type:  0)
>>>>>> > RFLAGS: 0xa    [ ]
>>>>>> >
>>>>>> > I want to know which binary file (.o) should I disassemble to look at
>>>>>> > the
>>>>>> > RIP?
>>>>>> >
>>>>>> > I was looking at
>>>>>> > objdump -D  -mi386 -Maddr16,data16 generated/ramstage.o
>>>>>> >
>>>>>> > but this is prior to linking and thus only has offsets.
>>>>>> >
>>>>>> > --
>>>>>> >
>>>>>> > Regards
>>>>>> > [Himanshu Chauhan]
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > coreboot mailing list: coreboot at coreboot.org
>>>>>> > https://www.coreboot.org/mailman/listinfo/coreboot
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Regards
>>>>> [Himanshu Chauhan]
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards
>>>> [Himanshu Chauhan]
>>
>>
>>
>> --
>>
>> Regards
>> [Himanshu Chauhan]



-- 

Regards
[Himanshu Chauhan]

More information about the coreboot mailing list