Matt DeVillier has submitted this change. ( https://review.coreboot.org/c/coreboot/+/79160?usp=email )
Change subject: Documentation: Improve x86_64 ......................................................................
Documentation: Improve x86_64
* Move x86_64 documentation to dedicated page * Update with better description of current implementation * Update TODOs
Change-Id: Ia5ba51be629a8c878aad64d3297176457cf8e855 Signed-off-by: Patrick Rudolph siro@das-labor.org Reviewed-on: https://review.coreboot.org/c/coreboot/+/79160 Tested-by: build bot (Jenkins) no-reply@coreboot.org Reviewed-by: David Hendricks david.hendricks@gmail.com --- M Documentation/arch/x86/index.md A Documentation/arch/x86/x86_64.md 2 files changed, 110 insertions(+), 87 deletions(-)
Approvals: build bot (Jenkins): Verified David Hendricks: Looks good to me, approved
diff --git a/Documentation/arch/x86/index.md b/Documentation/arch/x86/index.md index f2597ac..1c5edb3 100644 --- a/Documentation/arch/x86/index.md +++ b/Documentation/arch/x86/index.md @@ -6,91 +6,5 @@ :maxdepth: 1
x86 PAE support <pae.md> +x86_64 support <x86_64.md> ``` - -## State of x86_64 support -Some SOCs now support 64bit mode. Search for HAVE_X86_64_SUPPORT in Kconfig. - -In order to add support for x86_64 the following assumptions were made: -* The CPU supports long mode -* All memory returned by malloc must be below 4GiB in physical memory -* All code that is to be run must be below 4GiB in physical memory -* The high dword of pointers is always zero -* The reference implementation is qemu -* x86 payloads are loaded below 4GiB in physical memory and are jumped - to in *protected mode* - -## Assumptions for all stages using the reference implementation -* 0-4GiB are identity mapped using 2MiB-pages as WB -* Memory above 4GiB isn't accessible -* page tables reside in memory mapped ROM -* A stage can install new page tables in RAM - -## Page tables -A `pagetables` cbfs file is generated based on an assembly file. - -To generate the static page tables it must know the physical address where to -place the file. - -The page tables contains the following structure: -* PML4E pointing to PDPE -* PDPE with *$n* entries each pointing to PDE -* *$n* PDEs with 512 entries each - -At the moment *$n* is 4, which results in identity mapping the lower 4 GiB. - -## Basic x86_64 support -Basic support for x86_64 has been implemented for QEMU mainboard target. - -## Reference implementation -The reference implementation is -```{toctree} -:maxdepth: 1 - -QEMU i440fx <../../mainboard/emulation/qemu-i440fx.md> -QEMU Q35 <../../mainboard/emulation/qemu-q35.md> -``` - -## TODO -* Identity map memory above 4GiB in ramstage - -## Future work - -1. Fine grained page tables for SMM: - * Must not have execute and write permissions for the same page. - * Must allow only that TSEG pages can be marked executable -2. Support 64bit PCI BARs above 4GiB -3. Place and run code above 4GiB - -## Porting other boards -* Fix compilation errors -* Test how well CAR works with x86_64 and paging -* Improve mode switches - -## Known problems on real hardware - -Running VGA rom directly fails. Yabel works fine though. - -## Known bugs on KVM enabled qemu - -The `x86_64` reference code runs fine in qemu soft-cpu, but has serious issues -when using KVM mode on some machines. The workaround is to *not* place -page-tables in ROM, as done in -[CB:49228](https://review.coreboot.org/c/coreboot/+/49228). - -Here's a list of known issues: - -* After entering long mode, the FPU doesn't work anymore, including accessing - MMX registers. It works fine before entering long mode. It works fine when - switching back to protected mode. Other registers, like SSE registers, are - working fine. -* Reading from virtual memory, when the page tables are stored in ROM, causes - the MMU to abort the "page table walking" mechanism when the lower address - bits of the virtual address to be translated have a specific pattern. - Instead of loading the correct physical page, the one containing the - page tables in ROM will be loaded and used, which breaks code and data as - the page table doesn't contain the expected data. This in turn leads to - undefined behaviour whenever the 'wrong' address is being read. -* Disabling paging in compatibility mode crashes the CPU. -* Returning from long mode to compatibility mode crashes the CPU. -* Entering long mode crashes on AMD host platforms. diff --git a/Documentation/arch/x86/x86_64.md b/Documentation/arch/x86/x86_64.md new file mode 100644 index 0000000..b9aebd0 --- /dev/null +++ b/Documentation/arch/x86/x86_64.md @@ -0,0 +1,109 @@ +# x86_64 architecture documentation + +This section documents coreboot's x86_64 support. When enabled, +every coreboot stage is built for x86_64, contrary to UEFI's implementation +that only runs some stages in x86_64. +On UEFI the PEI phase, which is x86_32, brings up DRAM and installs +page tables for the x86_64 DXE and BDS phases. + +## Toolchain requirements for x86_64 support +* The compiler must support generating code for the *large memory model* + (-mcmodel=large). It's supported since GCC 4.4. + + Page tables can be used to provide security benefits, such as by marking + memory as non-executable or removing it entirely. This could be useful + for SMM to mark regular DRAM as NX. + + The large memory model causes the compiler to emit 64bit addressing + instructions, which increases code size. In theory, this is roughly + made up for by the faster execution of the x86_64 code. + +* All x86 coreboot stages and payloads must be loaded below 4GiB in + physical memory. When jumping to the payload coreboot will drop from + long mode back to protected mode to keep compatibility with these payloads. + +## Comparison to UEFI +On UEFI the SEC and PEI phases (similar to coreboot's bootblock and romstage) +are run in x86_32 mode. The following (guessed) reasons are likely: +* There's no need for x86_64 as memory hasn't been trained yet. The whole 4GiB + address space, including CAR, memory mapped SPI flash and PCI BARs, are + accessible in x86_32. +* When the EFI specification was written compilers did not support + *large memory model*, required in CAR when using a 1:1 page mapping +* Code is 20% bigger in x86_64 due to *large memory model* where pointers and + function calls always use 8 byte addressing. However flash size was very + limited, compared to today's flash chips, when the EFI spec was written. + +## Current software constraints for x86_64 support +The following constraints are coreboot limitations as it was intended to run in +protected mode only. The code still assumes 32bit pointers in some places and thus: +* The high dword of pointers must always be zero. +* All memory returned by malloc must be below 4GiB in physical memory. +* All code that is to be run must be below 4GiB in physical memory. +* CBMEM must reside below 4GiB in physical memory. + +Any software within coreboot must not access memory resources above 4GiB until +end of BS_DEV_RESOURCES in ramstage. Only at that point the full memory map is +known and identity mapped. + +## Supported boards +On the supported boards you can enable x86_64 compilation by setting the +Kconfig `USE_X86_64_SUPPORT`. This config option is enabled if the SOC/CPU +selects `HAVE_X86_64_SUPPORT`. + +## Protected mode wrappers +On some platforms binary blobs are run to initialize parts of the hardware. +When these binary blobs have been compiled for x86_32, then coreboot must +switch to protected mode in order to call and run the blobs. Once the invoked +blobs finish running, coreboot needs to switch back to long mode. + +Since every BLOB is different a SoC must be enabled to support x86_64 mode +by providing the correct wrapper around the x86_32 BLOBs. + +## TODO +* Support more platforms +* Fix running VGA Option ROMs +* Fix running MRC.bin (Sandy Bridge / Haswell boards) +* Identity map memory above 4GiB in ramstage +* Fine grained page tables for SMM: + * Must not have execute and write permissions for the same page. + * Must only allow TSEG pages to be marked as executable. + * Must reside in SMRAM. + * Must be placed together with SMM rmodule. +* Support 64bit PCI BARs above 4GiB +* Jump to compatible payloads in long mode + +## Porting other boards +* Fix compilation errors +* Test how well CAR works with x86_64 and paging +* Improve mode switches +* Test libgfxinit / VGA Option ROMs / FSP + +## Known bugs on real hardware + +According to Intel x86_64 mode hasn't been validated in CAR environments. +However, coreboot developers working on x86_64 support have tried this on +various Intel platforms, and so far haven't found any issues with CAR when +running in x86_64 mode. + +## Known bugs on KVM enabled QEMU + +The `x86_64` reference code runs fine in QEMU's soft-cpu, but has serious issues +when using KVM mode on some machines. This is due to various mechanisms trying +to accelerate the code execution. + +Known issues in QEMU: +* After entering long mode, the FPU doesn't work anymore, including accessing + MMX registers. It works fine before entering long mode. It works fine when + switching back to protected mode. Other registers, like SSE registers, are + working fine. +* Reading from virtual memory, when the page tables are stored in ROM, causes + the MMU to abort the "page table walking" mechanism when the lower address + bits of the virtual address to be translated have a specific pattern. + Instead of loading the correct physical page, the one containing the + page tables in ROM will be loaded and used, which breaks code and data as + the page table doesn't contain the expected data. This in turn leads to + undefined behaviour whenever the 'wrong' address is being read. +* Disabling paging in compatibility mode crashes the CPU. +* Returning from long mode to compatibility mode crashes the CPU. +* Entering long mode crashes on AMD host platforms.
-
Matt DeVillier (Code Review)