About Paging, Realmode and what is going on
Dear folks and techpriests, the more I want to contribute and learn about low-level-code the less I understand, it seems. 1. cb switches the CPU immediately to Protected Mode, yet Payloads like seaBIOS work in Real Mode. Does coreboot switch the CPU always back to RM before jumping to the payload? 2. When CB switches to PM - who generates and administrates the Page Tables and where? 3. Gustavo Duarte writes <http://duartes.org/gustavo/blog/post/how-computers-boot-up/> that GRUB switches from protected mode to real mode and vice versa all the time to address >1MiB of RAM and also use the BIOS-calls. If this is true using GRUB as payload would not work, as GRUB needs to call the non-existent BIOS, right? 4. Once CB is in PM it can't access physical addresses anymore? It doesn't need to, too? 5. PM means RAM-access is only possible through virtual addresses which are translated by the MMU using the Page Tables. This question is similar to [2.]: If coreboot generates the Page Tables and the payload would start in PM as well (is this even possible? At least the Linux-Kernel has entry points for RM and PM) this would mean the payload needs to use the Page Tables generated by CB. That wouldn't be a problem as they're linked in the register CR3 anyways? And an unimportant bonus question: * Why does every modern CPU still start in RM? I do get the compatibility problem, but on the other hand: Do you need it for anything beside booting MS-DOS on your Ryzen? Is it really impossible for AMD and Intel to create a new CPU-generation with the x86-instruction set without RM, 16-bit-registers and 20-bit-mode registers like CS, SS etc. No modern OS uses bios calls. No CPU is ever switched to RM again after booting up. They should get rid of this old stuff. Would be cool if someone could put this in its true light. Thanks, Philipp
2017-07-31 10:52 GMT+02:00 Philipp Stanner <stanner@posteo.de>:
1. cb switches the CPU immediately to Protected Mode, yet Payloads like seaBIOS work in Real Mode. Does coreboot switch the CPU always back to RM before jumping to the payload? No, payloads are started in pmode.
2. When CB switches to PM - who generates and administrates the Page Tables and where? We use flat mode: 4gb segments starting at 0, for code and data. Virtual address == logical address == physical address
3. Gustavo Duarte writes that GRUB switches from protected mode to real mode and vice versa all the time to address >1MiB of RAM and also use the BIOS-calls. If this is true using GRUB as payload would not work, as GRUB needs to call the non-existent BIOS, right? It uses BIOS calls, except when built for coreboot.
4. Once CB is in PM it can't access physical addresses anymore? It doesn't need to, too? We use flat mode, see above.
5. PM means RAM-access is only possible through virtual addresses which are translated by the MMU using the Page Tables. This question is similar to [2.]: If coreboot generates the Page Tables and the payload would start in PM as well (is this even possible? At least the Linux-Kernel has entry points for RM and PM) this would mean the payload needs to use the Page Tables generated by CB. That wouldn't be a problem as they're linked in the register CR3 anyways? As stated above, payloads start in pmode. As stated above, we use a flat representation which comes with no surprises. The payload can then reconfigure the system to setup its own configuration.
Why does every modern CPU still start in RM? I do get the compatibility problem, but on the other hand: Do you need it for anything beside booting MS-DOS on your Ryzen? Is it really impossible for AMD and Intel to create a new CPU-generation with the x86-instruction set without RM, 16-bit-registers and 20-bit-mode registers like CS, SS etc. No modern OS uses bios calls. No CPU is ever switched to RM again after booting up. They should get rid of this old stuff. "Every modern x86 CPU". In the end, that's something to ask the CPU vendors (but don't expect any answers). Some guesses:
1. Windows 7 uses BIOS calls (although they stopped switching back to real mode for that, they use x86 16bit emulation. still, BIOS services need to be there) 2. CPUs might not be switched back to real mode today, but from 32bit modes it's a pretty short route to vm86 modes, which are effectively identical to real mode and still in use. 3. Why change a working system and risk compatibility issues? x86's biggest selling point is compatibility, and if you forfeit that, users may move off your architecture entirely. Regards, Patrick -- Google Germany GmbH, ABC-Str. 19, 20354 Hamburg Registergericht und -nummer: Hamburg, HRB 86891, Sitz der Gesellschaft: Hamburg Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Philipp Stanner wrote:
the more I want to contribute and learn about low-level-code the less I understand, it seems.
The x86 is a true rabbit hole. :)
2. When CB switches to PM - who generates and administrates the Page Tables and where?
Note that PM != paging. Neither coreboot nor proprietary BIOS products used paging traditionally. Ron pushed for paging, there was a bit of support. I don't know the current situation though. Also note that PM != "flat real mode" or "32-bit real mode", it's not really documented as a feature, so I don't know if it has an official name. What you do is prepare a GDT with most or all entries having base 0 size 4GB, then enable PM in CR0, ldgdt, set at least cs and ds (usually more) to the 4GB GDT selectors, then disable PM in CR0 - *and do not reload cs, ds and other selectors/segment registers* once back in RM. This then allows 32-bit memory access in RM on 386 up.
4. Once CB is in PM it can't access physical addresses anymore? It doesn't need to, too?
That's a policy decision made in the GDT and the page tables. If base=0 and paging disabled and/or page tables are set the right way, then virtual == physical.
5. PM means RAM-access is only possible through virtual addresses which are translated by the MMU using the Page Tables.
If you stay in PM and enable paging, yes.
If coreboot generates the Page Tables and the payload would start in PM as well (is this even possible? At least the Linux-Kernel has entry points for RM and PM) this would mean the payload needs to use the Page Tables generated by CB. That wouldn't be a problem as they're linked in the register CR3 anyways?
A few different bootloader binary interfaces exist on x86. There are at least BIOS, UEFI, multiboot and coreboot. The general rule has always been to not make very many assumptions, so expect every bootloader and/or libpayload to set up their own flat real mode GDT.
* Why does every modern CPU still start in RM?
Many industries run on DOS. Many system developers have created in-house BIOS extensions. x86 will never fully lose its 16-bit legacy. //Peter
On Tue, Aug 01, 2017 at 02:49:27PM +0000, Peter Stuge wrote:
Philipp Stanner wrote: [...]
* Why does every modern CPU still start in RM?
Many industries run on DOS. Many system developers have created in-house BIOS extensions. x86 will never fully lose its 16-bit legacy.
And, just to throw a wrench into things, the Bootguard startup ACM seems to start in a 32-bit mode of some sort and then shift back to real mode when it jumps into the legacy reset vector. I'm not sure what the memory map looks like during the ACM. -- Trammell
On 01.08.2017 16:49, Peter Stuge wrote:
Note that PM != paging. Neither coreboot nor proprietary BIOS products used paging traditionally. Ron pushed for paging, there was a bit of support. I don't know the current situation though.
Also note that PM != "flat real mode" or "32-bit real mode", it's not really documented as a feature, so I don't know if it has an official name.
What you do is prepare a GDT with most or all entries having base 0 size 4GB, then enable PM in CR0, ldgdt, set at least cs and ds (usually more) to the 4GB GDT selectors, then disable PM in CR0 - *and do not reload cs, ds and other selectors/segment registers* once back in RM. This then allows 32-bit memory access in RM on 386 up.
Why would I want to address memory in RM with 32 Bits? I don't see any difference to using PM without Paging enabled.
Philipp Stanner wrote:
Why would I want to address memory in RM with 32 Bits? I don't see any difference to using PM without Paging enabled.
In a bootloader (after coreboot) you often want to call BIOS interrupt services which assume real mode, because that was the only mode, when the interrupt services were created. For coreboot itself, I think there is little if any advantage to big real mode. //Peter
Am Donnerstag, den 03.08.2017, 12:48 +0000 schrieb Peter Stuge:
Philipp Stanner wrote:
Why would I want to address memory in RM with 32 Bits? I don't see any difference to using PM without Paging enabled.
In a bootloader (after coreboot) you often want to call BIOS interrupt services which assume real mode, because that was the only mode, when the interrupt services were created.
For coreboot itself, I think there is little if any advantage to big real mode.
//Peter
Wait. Coreboot doesn't implement BIOS-Interrupt-Services. So no payload started by cb should ever try to call BIOS, should it? And you can't use the 32-Bit-Instructionset anymore, can you? P.
Philipp Stanner wrote:
Why would I want to address memory in RM with 32 Bits? I don't see any difference to using PM without Paging enabled.
In a bootloader (after coreboot) you often want to call BIOS interrupt services which assume real mode, because that was the only mode, when the interrupt services were created.
For coreboot itself, I think there is little if any advantage to big
Wait. Coreboot doesn't implement BIOS-Interrupt-Services.
Sure, but a payload can. SeaBIOS aims to provide a complete BIOS with all neccessary interrupt services for legacy compatibility.
So no payload started by cb should ever try to call BIOS, should it?
That depends. A *payload* can not assume interrupt services, but a bootloader (e.g. in MBR) can very well assume interrupt services, especially since MBR is a BIOS paradigm.
And you can't use the 32-Bit-Instructionset anymore, can you?
That's orthogonal. You can use 32-bit instructions in 16-bit mode and vice versa. The 0x66 and 0x67 opcode prefixes can be used to set an instruction mode for a single instruction. //Peter
Am Samstag, den 02.09.2017, 12:36 +0000 schrieb Peter Stuge:
Sure, but a payload can. SeaBIOS aims to provide a complete BIOS with all neccessary interrupt services for legacy compatibility.
Once coreboot jumped into SeaBIOS-code the latter is responsible for providing the right interface for interrupt services. I guess it doesn't matter very much in the end who switches the CPU to Real Mode. My point is that coreboot's bios calls can't be called because they don't exist ^^
That depends. A *payload* can not assume interrupt services, but a bootloader (e.g. in MBR) can very well assume interrupt services, especially since MBR is a BIOS paradigm.
Every payload built for and with coreboot won't try to call interrupts.
And you can't use the 32-Bit-Instructionset anymore, can you?
That's orthogonal. You can use 32-bit instructions in 16-bit mode and vice versa. The 0x66 and 0x67 opcode prefixes can be used to set an instruction mode for a single instruction.
But in the beginning of this thread on 02 Aug Ron Minnich wrote that coreboot switches to PM, because: "And we want the 32-bit instruction set." P.
Philipp Stanner wrote:
Once coreboot jumped into SeaBIOS-code the latter is responsible for providing the right interface for interrupt services.
Sure, coreboot does not provide interrupt services. I did not mean to claim that it does. Sorry if there was confusion.
I guess it doesn't matter very much in the end who switches the CPU to Real Mode.
One point I tried to make is that the CPU can very well be switched back and forth several times before the actual OS kernel is running.
My point is that coreboot's bios calls can't be called because they don't exist ^^
Sure, that's accurate, but having coreboot doesn't mean that there are never BIOS interrupt services.
That depends. A *payload* can not assume interrupt services, but a bootloader (e.g. in MBR) can very well assume interrupt services, especially since MBR is a BIOS paradigm.
Every payload built for and with coreboot won't try to call interrupts.
No payload is built with coreboot. Programs built as payloads will indeed not call interrupt services, because they expect none. But a bootloader built as a payload could also be built to use BIOS interfaces. GRUB is one example of this. And it is technically possible to put multiple payloads into CBFS. coreboot itself can only start one payload, but SeaBIOS allows the user to choose which of those payloads to start, in which case a payload *does* have interrupt services available. Just to let you know that this is possible.
And you can't use the 32-Bit-Instructionset anymore, can you?
That's orthogonal. You can use 32-bit instructions in 16-bit mode and vice versa. The 0x66 and 0x67 opcode prefixes can be used to set an instruction mode for a single instruction.
But in the beginning of this thread on 02 Aug Ron Minnich wrote that coreboot switches to PM, because:
"And we want the 32-bit instruction set."
Maybe he didn't consider the prefix, or maybe he was thinking about how GCC+binutils are pretty poor at dealing with the different modes and swapping between them. On the instruction level it's not a big deal to explicitly set the "other" mode, but the toolchain is not very strong when it comes to 16-bit, making it more difficult to do so when writing C code. It was created for the operating system environment and it shows, but that's perfectly fine. There is no "32-bit instruction set". The instruction set is the same, there is no difference in what opcodes could be used (well except maybe gdt and idt stuff) but *addressing* is obviously different in 32-bit mode, and using 32-bit addressing makes everything a lot easier. //Peter
Thanks so far. Very interesting. Am Montag, den 04.09.2017, 07:28 +0000 schrieb Peter Stuge:
coreboot itself can only start one payload, but SeaBIOS allows the user to choose which of those payloads to start, in which case a payload *does* have interrupt services available.
Just to let you know that this is possible.
There is no "32-bit instruction set". The instruction set is the same, there is no difference in what opcodes could be used (well except maybe gdt and idt stuff) but *addressing* is obviously different in 32-bit mode, and using 32-bit addressing makes everything a lot easier.
If there is no difference according the instruction set and as coreboot itself doesn't provide BIOS-services: Why don't we just switch the CPU to flat protected mode, do our duty and jump into the payload (SeaBIOS mainly) which then switches back to real mode if necessary, change the GDT or do whatever it needs to do? Switching back to Real Mode, using 32-Bit-addresses (what sounds like a hack to me. I can't imagine Intel intended this) is much more complicated than staying in PM, isn't it? x86 is pain. Sometimes I wonder why it even works at all. Let's hope there will be RISC-V-Boards one day. P.
But a bootloader built as a payload could also be built to use BIOS interfaces. GRUB is one example of this.
Let us assume the following configuration: FSP -> Coreboot -> Payload: GRUB2 -> Linux No legacy interrupts, correct? So, what is this for the architecture? CSM? UEFI look alike? I assume, the Linux image on the HDD/SSD was installed far before as CSM, so there is already existing MBR, and some CSM implemented legacy services (INTs) already exist, am I correct? I assume UEFI installed Linux would NOT run, since GRUB2 will be not able to find MBR... I assume GRUB2 as payload to Coreboot assumes MBR/Legacy (by default) implementation. Any comments on what I wrote here? ;-) Zoran On Mon, Sep 4, 2017 at 12:01 PM, Philipp Stanner <stanner@posteo.de> wrote:
Thanks so far. Very interesting.
Am Montag, den 04.09.2017, 07:28 +0000 schrieb Peter Stuge:
coreboot itself can only start one payload, but SeaBIOS allows the user to choose which of those payloads to start, in which case a payload *does* have interrupt services available.
Just to let you know that this is possible.
There is no "32-bit instruction set". The instruction set is the same, there is no difference in what opcodes could be used (well except maybe gdt and idt stuff) but *addressing* is obviously different in 32-bit mode, and using 32-bit addressing makes everything a lot easier.
If there is no difference according the instruction set and as coreboot itself doesn't provide BIOS-services: Why don't we just switch the CPU to flat protected mode, do our duty and jump into the payload (SeaBIOS mainly) which then switches back to real mode if necessary, change the GDT or do whatever it needs to do? Switching back to Real Mode, using 32-Bit-addresses (what sounds like a hack to me. I can't imagine Intel intended this) is much more complicated than staying in PM, isn't it?
x86 is pain. Sometimes I wonder why it even works at all. Let's hope there will be RISC-V-Boards one day.
P.
-- coreboot mailing list: coreboot@coreboot.org https://mail.coreboot.org/mailman/listinfo/coreboot
Hello Philipp, There are lot of confusion you have created with your naming convention... So we need here to "Divide and Concur", since this thread really remains me of Spaghetti Bologneze. Here is the partial picture how x86 operates: [image: Inline image 1] So you see here three modes, you are talking about. Every INTEL CPU starts in Real Mode, and then switches to the protected mode. So, you have some legacy you MUST maintain for the backward HW compatibility. As Patrick Georgi very correctly mentioned, let me recap what he said, with my additions, to be much clearer with the picture I have provided. [1] Every x86 CPU starts in Real Mode, for the legacy reasons. the Lecacy mode must be maintained (Real Mode replicates with the VERY first PC XT far back to 1980, with some first 1MB memory mapping improvements). The BackWard HW compatibility was maintained for years, and it went too far that it can be changed over the nite. I am not going to explain why: too long it takes. [2] CB operates in Protected Mode, WITHOUT Paging/Virtual Mode switched on, since this mode is ONLY used with MMU (which is HUGE HW extention to create Virtual use of the Memory), which is managed by true OS (like Linux, WIN, QNX, MAC OS, VMS, you name it). CoreBoot, U-Boot, BIOS boot-loaders: they all operate from the flat/Protected Mode, and Patrick explained this very well/excellent. In addition, I must say that this first 4GB MUST be assigned to the benefit of the underlying HW (PCIe, GFX, SMM mode, Flash region, ACPI tables, and so on). This assignment is done strictly Physically, with the CPU HW helping for some regions for remapping them (since CPU does see what CB does for it as Physical memory assignment, but MMU [when OS runs] sees it quite differently)! [3] As for payloads (also working in Protected Mode), there are some which will make their own definitions of the memory, some/most which will work deeper (as SeaBIOS, for example) on the CB Physical memory definition. As well as these payloads (depending upon what the usage is) will add more drivers for the certain devices, CB did not add in the first place. For example, SeaBIOS adds IDE (maybe even AHCI, not sure) driver to this set. Some people can define their own payloads with the set of specific drivers, depending upon what the HW platform is intended for. [4] Once the OS boot loader takes over (Like GRUB), it'll start OS, which will switch from the Protected to the Virtual Mode (using MMU), Then paging will take places, and each process will have its own set of tables, and its own initial value for CR3 (when process context switches). I warmly suggest to you to inspect (bit by bit) CR1 register, since this one is crucial/essential for introducing these modes. I really hope this helps (I've tried to develop some systematic approach to the topic). The booting process will go to stages, and once CPU abandons Real Mode, it'll (generally) not return to it (only in some special cases such as SMM Mode). Once It abandons Protected Mode, it'll stay in Virtual Mode, and so on. But backward transitions are possible, as you see from the picture. Zoran On Mon, Jul 31, 2017 at 10:52 AM, Philipp Stanner <stanner@posteo.de> wrote:
Dear folks and techpriests,
the more I want to contribute and learn about low-level-code the less I understand, it seems.
1. cb switches the CPU immediately to Protected Mode, yet Payloads like seaBIOS work in Real Mode. Does coreboot switch the CPU always back to RM before jumping to the payload? 2. When CB switches to PM - who generates and administrates the Page Tables and where? 3. Gustavo Duarte writes <http://duartes.org/gustavo/blog/post/how-computers-boot-up/> that GRUB switches from protected mode to real mode and vice versa all the time to address >1MiB of RAM and also use the BIOS-calls. If this is true using GRUB as payload would not work, as GRUB needs to call the non-existent BIOS, right? 4. Once CB is in PM it can't access physical addresses anymore? It doesn't need to, too? 5. PM means RAM-access is only possible through virtual addresses which are translated by the MMU using the Page Tables. This question is similar to [2.]: If coreboot generates the Page Tables and the payload would start in PM as well (is this even possible? At least the Linux-Kernel has entry points for RM and PM) this would mean the payload needs to use the Page Tables generated by CB. That wouldn't be a problem as they're linked in the register CR3 anyways?
And an unimportant bonus question:
- Why does every modern CPU still start in RM? I do get the compatibility problem, but on the other hand: Do you need it for anything beside booting MS-DOS on your Ryzen? Is it really impossible for AMD and Intel to create a new CPU-generation with the x86-instruction set without RM, 16-bit-registers and 20-bit-mode registers like CS, SS etc. No modern OS uses bios calls. No CPU is ever switched to RM again after booting up. They should get rid of this old stuff.
Would be cool if someone could put this in its true light.
Thanks,
Philipp
-- coreboot mailing list: coreboot@coreboot.org https://mail.coreboot.org/mailman/listinfo/coreboot
participants (5)
-
Patrick Georgi -
Peter Stuge -
Philipp Stanner -
Trammell Hudson -
Zoran Stojsavljevic