Hi,
I'm wondering if there's a specification describing the state of the computer at the hand-off between LinuxBIOS and a payload.
So far I've got LinxBIOS booting a dummy "for(;;)" payload in Bochs, and I've found out there's a "LinuxBIOS Table" and that it can be found by searching for "LBIO" in the first 4 KB of RAM and in the area between 0xF000 and 0xFFFFF. I don't know if there's a checksum or any other validation, and have been unable to find anything describing what this table contains.
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
There's also other issues, for e.g: - will an 80x86 system always be in 32-bit protected mode for 32-bit ELF payloads, and is long mode possible for ELF64 payloads (and if so, how is paging setup)? - are any parameters passed on the stack to the payload's entry function? - are AP CPUs started (or waiting for a SIPI sequence)? - can the payload assume anything about the state of the PIC, I/O APIC/s and local APIC/s, PIT, RTC, ISA DMA controllers, serial ports, etc? - what state are PCI buses and devices left in (would a payload need to do full PCI bus enumeration, or..)? - is there anything in the LinuxBIOS table that could be used to determine if the computer is "headless" and which serial port or other device to use if it is? - can anything be assumed about AGP and the first video card (if the computer isn't headless)? - is there any form of "LinuxBIOS API" that can be used by the payload for generic low level disk access (floppy/ATA/ATAPI/SCSI)?
Thanks,
Brendan
* Brendan Trotter btrotter@gmail.com [070612 06:23]:
Hi,
I'm wondering if there's a specification describing the state of the computer at the hand-off between LinuxBIOS and a payload.
So far I've got LinxBIOS booting a dummy "for(;;)" payload in Bochs, and I've found out there's a "LinuxBIOS Table" and that it can be found by searching for "LBIO" in the first 4 KB of RAM and in the area between 0xF000 and 0xFFFFF. I don't know if there's a checksum or any other validation, and have been unable to find anything describing what this table contains.
check LinuxBIOSv2/util/lbtdump
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
They're not used by payloads (except the Linux kernel)
- will an 80x86 system always be in 32-bit protected mode for 32-bit
ELF payloads, and is long mode possible for ELF64 payloads (and if so, how is paging setup)?
long mode is not yet possible. If you know how to do it and have time, we'd appreciate any patches though!
- are any parameters passed on the stack to the payload's entry function?
no
- are AP CPUs started (or waiting for a SIPI sequence)?
yes, hardware is completely up when linuxbios is done.
- can the payload assume anything about the state of the PIC, I/O
APIC/s and local APIC/s, PIT, RTC, ISA DMA controllers, serial ports, etc?
yes. they're initialized.
- what state are PCI buses and devices left in (would a payload need
to do full PCI bus enumeration, or..)?
They have their resources.
What do you mean by full pci bus enumeration? What would be the opposite?
- is there anything in the LinuxBIOS table that could be used to
determine if the computer is "headless" and which serial port or other device to use if it is?
Not at the moment. But in General this is easy. If no VGA class PCI device is there, the machine is 99.99% headless.
- can anything be assumed about AGP and the first video card (if the
computer isn't headless)?
What would you like to assume?
- is there any form of "LinuxBIOS API" that can be used by the payload
for generic low level disk access (floppy/ATA/ATAPI/SCSI)?
No. No Callbacks. That's our intended design.
We don't have functions to access floppy or SCSI in LinuxBIOS anyways (and in v3 we also dropped ATA)
Hi,
On 6/12/07, Stefan Reinauer stepan@coresystems.de wrote:
- Brendan Trotter btrotter@gmail.com [070612 06:23]:
I'm wondering if there's a specification describing the state of the computer at the hand-off between LinuxBIOS and a payload.
So far I've got LinxBIOS booting a dummy "for(;;)" payload in Bochs, and I've found out there's a "LinuxBIOS Table" and that it can be found by searching for "LBIO" in the first 4 KB of RAM and in the area between 0xF000 and 0xFFFFF. I don't know if there's a checksum or any other validation, and have been unable to find anything describing what this table contains.
check LinuxBIOSv2/util/lbtdump
I've found "src/include/boot/linuxbios_tables.h" and have also looked at the source for LinuxBIOSv2/util/lbtdump.
However, I fail to see how any source code can be considered a viable alternative to documentation for something that is (hopefully) a non-proprietory common standard. Reverse engineering (including relying on assumptions derived from the source code of any specific implementation) is what people do when they have no documentation and are willing to stumble around in the dark hoping to implement something that works until the wind changes.
For a simple example, which piece of source code would tell me if "lb_mainboard.vendor_idx" is guaranteed to be unique both now and in the future?
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
They're not used by payloads (except the Linux kernel)
Surely LinuxBIOS isn't intended as a "Linux only" BIOS though....
- are AP CPUs started (or waiting for a SIPI sequence)?
yes, hardware is completely up when linuxbios is done.
I tried it on a dual CPU Bochs emulation (second CPU was left in "wait for SIPI" state) - I'm guessing I need to enable multi-CPU support via. some compile time option for the "emulation/qemu-i386" target?
- can the payload assume anything about the state of the PIC, I/O
APIC/s and local APIC/s, PIT, RTC, ISA DMA controllers, serial ports, etc?
yes. they're initialized.
:-)
Initialised to which states? For e.g. would the PIT be guaranteed to be initialised to generate IRQ0 at a certain frequency, and would the PIC (and/or I/O APIC?) be guaranteed to be initialised to generate specific interrupt when IRQ0 occurs; or (from the payload's perspective) are these things left in an undefined state (i.e. the state they happened to be left in when/if LinuxBIOS used them)?
- what state are PCI buses and devices left in (would a payload need
to do full PCI bus enumeration, or..)?
They have their resources.
What do you mean by full pci bus enumeration? What would be the opposite?
Either: a) PCI buses and bridges aren't guaranteed to be enumerated, or b) PCI buses and bridges are guaranteed to be enumerated but PCI devices aren't guaranteed to be assigned resources, or c) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (including ROMs) d) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (except ROMs)
"They have their resources." would mean either c) or d), where each device may or may not be in a power saving state (if supported by the device), and may or may not be still configured to use MSI (if supported by the device)....
- is there anything in the LinuxBIOS table that could be used to
determine if the computer is "headless" and which serial port or other device to use if it is?
Not at the moment. But in General this is easy. If no VGA class PCI device is there, the machine is 99.99% headless.
And if a VGA class PCI device is present, you've got no idea if no monitor is attached...
- can anything be assumed about AGP and the first video card (if the
computer isn't headless)?
What would you like to assume?
Video card set to default "80 * 25" text mode accessible at 0xB8000; or some video mode set with display memory directly accessible at some physical address (where video mode details and physical address included in LinuxBIOS Table); or AGP aperture configured, AGP transfer speed set, MTRRs set to "write combining" and some video mode set with display memory directly accessible at some physical address; or anything else, as long as a payload writer knows how much or how little they can assume about it (and hopefully doesn't need to implement a wide range of video drivers just to do a "Hello World" that works on all video cards)... ;)
Cheers,
Brendan
* Brendan Trotter btrotter@gmail.com [070612 12:43]:
check LinuxBIOSv2/util/lbtdump
I've found "src/include/boot/linuxbios_tables.h" and have also looked at the source for LinuxBIOSv2/util/lbtdump.
However, I fail to see how any source code can be considered a viable alternative to documentation for something that is (hopefully) a non-proprietory common standard. Reverse engineering (including relying on assumptions derived from the source code of any specific implementation) is what people do when they have no documentation and are willing to stumble around in the dark hoping to implement something that works until the wind changes.
Yes, no doubt. You are perfectly right, and we are looking forward to your patches to fix this problem. Of course you do not have to reverse engineer, but just read the code, as we are non-proprietary. And of course we are always here to be asked.
So please, submit any documentation to this list for inclusion as soon as you wrote it.
For a simple example, which piece of source code would tell me if "lb_mainboard.vendor_idx" is guaranteed to be unique both now and in the future?
It is of course not unique. It is the index of the vendor name within the strings array of struct lb_mainboard. Since we're not trying to close things down, we don't have to work with magic numbers. So nothing needs to stay unique.
But I suggest you have a look at the code and document it as you start understanding it.
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
They're not used by payloads (except the Linux kernel)
Surely LinuxBIOS isn't intended as a "Linux only" BIOS though....
Of course not. ACPI is not really useful for payloads though, unless the payload wants to carry a full blown ACPI interpreter. Which rules out basically anything but an OS.
- are AP CPUs started (or waiting for a SIPI sequence)?
yes, hardware is completely up when linuxbios is done.
I tried it on a dual CPU Bochs emulation (second CPU was left in "wait for SIPI" state) - I'm guessing I need to enable multi-CPU support via. some compile time option for the "emulation/qemu-i386" target?
Yes, the emulation/qemu-i386 is per default not supporting SMP.
Either: a) PCI buses and bridges aren't guaranteed to be enumerated, or b) PCI buses and bridges are guaranteed to be enumerated but PCI devices aren't guaranteed to be assigned resources, or c) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (including ROMs) d) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (except ROMs)
"They have their resources." would mean either c) or d), where each device may or may not be in a power saving state (if supported by the device), and may or may not be still configured to use MSI (if supported by the device)....
yes, either c or d. You choose.
Not at the moment. But in General this is easy. If no VGA class PCI device is there, the machine is 99.99% headless.
And if a VGA class PCI device is present, you've got no idea if no monitor is attached...
Obviously not. and if the device does not do EDD you have no reliable way of finding out either.
- can anything be assumed about AGP and the first video card (if the
computer isn't headless)?
What would you like to assume?
Video card set to default "80 * 25" text mode accessible at 0xB8000; or some video mode set with display memory directly accessible at some physical address (where video mode details and physical address included in LinuxBIOS Table); or AGP aperture configured, AGP transfer speed set, MTRRs set to "write combining" and some video mode set with display memory directly accessible at some physical address; or anything else, as long as a payload writer knows how much or how little they can assume about it (and hopefully doesn't need to implement a wide range of video drivers just to do a "Hello World" that works on all video cards)... ;)
Ok, the video hardware is in the state that the VGA Option ROM sets it to. Since that is closed source, we can not guarantee anything, obviously. A rule of thumb tells you that 80x25 at B8000 is a reasonable assumption. The AGP setup is chipset specific, so it might vary from chipset to chipset.
Hi,
On 6/12/07, Stefan Reinauer stepan@coresystems.de wrote:
- Brendan Trotter btrotter@gmail.com [070612 12:43]:
check LinuxBIOSv2/util/lbtdump
I've found "src/include/boot/linuxbios_tables.h" and have also looked at the source for LinuxBIOSv2/util/lbtdump.
However, I fail to see how any source code can be considered a viable alternative to documentation for something that is (hopefully) a non-proprietory common standard. Reverse engineering (including relying on assumptions derived from the source code of any specific implementation) is what people do when they have no documentation and are willing to stumble around in the dark hoping to implement something that works until the wind changes.
Yes, no doubt. You are perfectly right, and we are looking forward to your patches to fix this problem. Of course you do not have to reverse engineer, but just read the code, as we are non-proprietary. And of course we are always here to be asked.
Hehe - by the time you've finished answering all my questions you'd wish you'd written the documentation beforehand... ;-)
Is "uint32_t" always little-endian or is it architecture specific (reverse engineering ruled out "always big-endian"), and would the "lb_header.signature" field always have an ASCII 'L' at the lowest address (or is that architecture specific too)?
The source file "util/lbtdump/lbtdump.c" does this:
if (head->header_bytes != sizeof(*head)) { fprintf(stderr, "Header bytes of %d are incorrect\n", head->header_bytes); continue; }
From this can I assume that the size of the LinuxBIOS Table header
will always be 24 bytes, and the LinuxBIOS Table header will never be extended in the future (or is this just a bug that makes "lbtdump" unable to handle forward/backward compatability)?
The file "src/include/boot/linuxbios_tables.h" defines 3 types of memory ranges:
#define LB_MEM_RAM 1 /* Memory anyone can use */ #define LB_MEM_RESERVED 2 /* Don't use this memory region */ #define LB_MEM_TABLE 16 /* Ram configuration tables are kept in */
The file "util/lbtdump/lbtdump.c" doesn't agree:
switch(rec->map[i].type) { case 1: mem_type = "ram"; break; case 3: mem_type = "acpi"; break; case 4: mem_type = "nvs"; break; default: case 2: mem_type = "reserved"; break; }
Can I assume that the defines in "linuxbios_tables.h" are out-of-date, and that the memory type field is the same as specified by version 2 of the ACPI specification (but perhaps not version 3, as that defines a new "5 = faulty RAM" type)? If "linuxbios_tables.h" is out-of-date, is "16 = RAM configuration tables are stored in" still possible on older versions of LinuxBIOS (and how would I detect which version of LinuxBIOS is present, considering there's no version number in the LinuxBIOS Table header)?
What is the LB_TAG_HWRPB structure. Is it something to do with a "Hardware Restart Parameter Block", and what does it contain? For e.g. would a value of zero indicate that the computer was shutdown correctly before boot, and a non-zero value indicate that the computer crashed or had hardware problems?
Would I also be correct to assume that a payload doesn't need to care about any of the "cmos_entries" and "cmos_enums", unless a BIOS setup/configuration utility is being built into the payload? Alternatively, could/would a payload search for strings to find CMOS values (e.g. search for the cmos_entry with the name string "power_on_after_fail" to see if additional hardware testing can be skipped, search for "baud_rate" to determine the serial port configuration, etc)?
It is of course not unique. It is the index of the vendor name within the strings array of struct lb_mainboard. Since we're not trying to close things down, we don't have to work with magic numbers. So nothing needs to stay unique.
Of course - it's similar to the way SMI/DMI does strings, yet are the strings themselves guaranteed to be unique, or is it possible for one manufacturer to use an identifying string in the future that another (possibly out of business) manufacturer had used in the past?
Motherboard manufacturer identification is one of the things that (IMHO) should be unique. The motherboard part number string should also be unique for that manufacturer ID. Otherwise it'd be difficult for payloads/utilities/OSs to use these IDs to avoid any hardware problems in specific motherboards.
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
They're not used by payloads (except the Linux kernel)
Surely LinuxBIOS isn't intended as a "Linux only" BIOS though....
Of course not. ACPI is not really useful for payloads though, unless the payload wants to carry a full blown ACPI interpreter. Which rules out basically anything but an OS.
I can think of a variety of uses, ranging from utilities that do nothing more than display hardware information, to diagnostics, to benchmarking, to hyper-visors, to full OSs...
- are AP CPUs started (or waiting for a SIPI sequence)?
yes, hardware is completely up when linuxbios is done.
I tried it on a dual CPU Bochs emulation (second CPU was left in "wait for SIPI" state) - I'm guessing I need to enable multi-CPU support via. some compile time option for the "emulation/qemu-i386" target?
Yes, the emulation/qemu-i386 is per default not supporting SMP.
I'm becoming skeptical here. I can't find anything in LinuxBIOS source code that starts AP CPUs, but in "src/arch/i386/smp/mpspec.c" I did find:
/* If we assume a symmetric processor configuration we can * get all of the information we need to write the processor * entry from the bootstrap processor. * Plus I don't think linux really even cares. * Having the proper apicid's in the table so the non-bootstrap * processors can be woken up should be enough. */
It might be possible that no AP CPUs are ever started, and are instead left in a "wait for SIPI" default state, assumed to be identical to the BSP, and even assumed to be working (no BIST checks).
AFAIK the normal approach is to send a broadcast INIT-SIPI sequence and wait to see which AP CPUs start (if any), and then either record details somewhere or have each AP CPU add itself to the MP specification and ACPI tables, then setup their MTTRs, thermal monitoring, HT links, SMBASE, etc before sending the APs back to the "wait for SIPI" state - i.e. in theory it should be mostly "configuration-less".
In any case, I'm guessing I need to get LinuxBIOS to generate MP specification and ACPI tables to suit the number of CPUs being emulated by Bochs/Qemu. Is this as simple as adding something like "option CPUs = ?" to the "targets/emulation/qemu-i386/Config.lb" file? I'm guessing it's more involved than this, as I couldn't find an example in the "Config.lb" of other targets.
Either: a) PCI buses and bridges aren't guaranteed to be enumerated, or b) PCI buses and bridges are guaranteed to be enumerated but PCI devices aren't guaranteed to be assigned resources, or c) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (including ROMs) d) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (except ROMs)
"They have their resources." would mean either c) or d), where each device may or may not be in a power saving state (if supported by the device), and may or may not be still configured to use MSI (if supported by the device)....
yes, either c or d. You choose.
You mean all LinuxBIOS developers are willing to make sure all PCI device ROMs either have physical addresses assigned or don't have physical addresses assigned based on my advice? Perhaps I've misunderstood... :-)
Not at the moment. But in General this is easy. If no VGA class PCI device is there, the machine is 99.99% headless.
And if a VGA class PCI device is present, you've got no idea if no monitor is attached...
Obviously not. and if the device does not do EDD you have no reliable way of finding out either.
There's a simple and easy way to find out if a monitor is attached or not, and it doesn't involve messing about with EDD - simply give the end-user the option of setting (or not setting) a value in the CMOS.
My (limited) reverse engineering has already shown that there's CMOS entries for a serial port (possibly intended for setting up a serial cable to communicate with a terminal emulator?).
- can anything be assumed about AGP and the first video card (if the
computer isn't headless)?
What would you like to assume?
Video card set to default "80 * 25" text mode accessible at 0xB8000; or some video mode set with display memory directly accessible at some physical address (where video mode details and physical address included in LinuxBIOS Table); or AGP aperture configured, AGP transfer speed set, MTRRs set to "write combining" and some video mode set with display memory directly accessible at some physical address; or anything else, as long as a payload writer knows how much or how little they can assume about it (and hopefully doesn't need to implement a wide range of video drivers just to do a "Hello World" that works on all video cards)... ;)
Ok, the video hardware is in the state that the VGA Option ROM sets it to. Since that is closed source, we can not guarantee anything, obviously. A rule of thumb tells you that 80x25 at B8000 is a reasonable assumption. The AGP setup is chipset specific, so it might vary from chipset to chipset.
Cool :-)
Next question would be how to get that to work under Qemu and/or Bochs. I realise this may not be easy (I've had problems with Bochs before, as the VGA ROM seems to behave like an ISA card ROM and doesn't behave like a PCI card ROM - i.e. the ROM ignores settings in PCI configuration space)...
Thanks,
Brendan
Brendan Trotter wrote:
Hi,
On 6/12/07, Stefan Reinauer stepan@coresystems.de wrote:
- Brendan Trotter btrotter@gmail.com [070612 12:43]:
check LinuxBIOSv2/util/lbtdump
I've found "src/include/boot/linuxbios_tables.h" and have also looked at the source for LinuxBIOSv2/util/lbtdump.
However, I fail to see how any source code can be considered a viable alternative to documentation for something that is (hopefully) a non-proprietory common standard. Reverse engineering (including relying on assumptions derived from the source code of any specific implementation) is what people do when they have no documentation and are willing to stumble around in the dark hoping to implement something that works until the wind changes.
Yes, no doubt. You are perfectly right, and we are looking forward to your patches to fix this problem. Of course you do not have to reverse engineer, but just read the code, as we are non-proprietary. And of course we are always here to be asked.
Hehe - by the time you've finished answering all my questions you'd wish you'd written the documentation beforehand... ;-)
Probably, but personally I'd rather see Stefan devote his time to the great work they're doing on v3 ;) I'll try to help a little, although my knowledge of the inner workings of LB is somewhat limited.
Is "uint32_t" always little-endian or is it architecture specific (reverse engineering ruled out "always big-endian"),
Yes, it should be dependent on the architecture
and would the "lb_header.signature" field always have an ASCII 'L' at the lowest address (or is that architecture specific too)?
no idea...
The source file "util/lbtdump/lbtdump.c" does this:
if (head->header_bytes != sizeof(*head)) { fprintf(stderr, "Header bytes of %d are incorrect\n", head->header_bytes); continue; }From this can I assume that the size of the LinuxBIOS Table header
will always be 24 bytes, and the LinuxBIOS Table header will never be extended in the future (or is this just a bug that makes "lbtdump" unable to handle forward/backward compatability)?
The file "src/include/boot/linuxbios_tables.h" defines 3 types of memory ranges:
#define LB_MEM_RAM 1 /* Memory anyone can use */ #define LB_MEM_RESERVED 2 /* Don't use this memory region */ #define LB_MEM_TABLE 16 /* Ram configuration tables are kept in */
The file "util/lbtdump/lbtdump.c" doesn't agree:
switch(rec->map[i].type) { case 1: mem_type = "ram"; break; case 3: mem_type = "acpi"; break; case 4: mem_type = "nvs"; break; default: case 2: mem_type = "reserved"; break; }Can I assume that the defines in "linuxbios_tables.h" are out-of-date, and that the memory type field is the same as specified by version 2 of the ACPI specification (but perhaps not version 3, as that defines a new "5 = faulty RAM" type)? If "linuxbios_tables.h" is out-of-date, is "16 = RAM configuration tables are stored in" still possible on older versions of LinuxBIOS (and how would I detect which version of LinuxBIOS is present, considering there's no version number in the LinuxBIOS Table header)?
What is the LB_TAG_HWRPB structure. Is it something to do with a "Hardware Restart Parameter Block", and what does it contain? For e.g. would a value of zero indicate that the computer was shutdown correctly before boot, and a non-zero value indicate that the computer crashed or had hardware problems?
Again, no clue really..
Would I also be correct to assume that a payload doesn't need to care about any of the "cmos_entries" and "cmos_enums", unless a BIOS setup/configuration utility is being built into the payload? Alternatively, could/would a payload search for strings to find CMOS values (e.g. search for the cmos_entry with the name string "power_on_after_fail" to see if additional hardware testing can be skipped, search for "baud_rate" to determine the serial port configuration, etc)?
My *understanding* is that different vendors do different things with the cmos, so payloads couldn't count on them for the vendor bios, so they don't for linuxbios either. I suppose a payload *could* do this with linuxbios, if the project wanted to, but I can't think of any that does, although openbios may. Anything cmos-related, such as power on after fail, should currently be handled between linuxbios and a userspace utility, like lxbios.
It is of course not unique. It is the index of the vendor name within the strings array of struct lb_mainboard. Since we're not trying to close things down, we don't have to work with magic numbers. So nothing needs to stay unique.
Of course - it's similar to the way SMI/DMI does strings, yet are the strings themselves guaranteed to be unique, or is it possible for one manufacturer to use an identifying string in the future that another (possibly out of business) manufacturer had used in the past?
This is something google would be good for, I can't come up with a good answer. Yes, they SHOULD always be unique, given that there are 65536 possible combinations, but whether there's some sort of registration or assignment system or it's at the manufacturers disposal, I don't know.
Motherboard manufacturer identification is one of the things that (IMHO) should be unique. The motherboard part number string should also be unique for that manufacturer ID. Otherwise it'd be difficult for payloads/utilities/OSs to use these IDs to avoid any hardware problems in specific motherboards.
Manufacturers are often very lax about this, in some cases motherboards have 5 or more different (subsystem) device IDs on the same board, or in others they don't bother setting most of them. This is one area where mainboard manufacturers need to clean up.
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
They're not used by payloads (except the Linux kernel)
Surely LinuxBIOS isn't intended as a "Linux only" BIOS though....
Of course not. ACPI is not really useful for payloads though, unless the payload wants to carry a full blown ACPI interpreter. Which rules out basically anything but an OS.
I can think of a variety of uses, ranging from utilities that do nothing more than display hardware information, to diagnostics, to benchmarking, to hyper-visors, to full OSs...
- are AP CPUs started (or waiting for a SIPI sequence)?
yes, hardware is completely up when linuxbios is done.
I tried it on a dual CPU Bochs emulation (second CPU was left in "wait for SIPI" state) - I'm guessing I need to enable multi-CPU support via. some compile time option for the "emulation/qemu-i386" target?
Yes, the emulation/qemu-i386 is per default not supporting SMP.
I'm becoming skeptical here. I can't find anything in LinuxBIOS source code that starts AP CPUs, but in "src/arch/i386/smp/mpspec.c" I did find:
/* If we assume a symmetric processor configuration we can
- get all of the information we need to write the processor
- entry from the bootstrap processor.
- Plus I don't think linux really even cares.
- Having the proper apicid's in the table so the non-bootstrap
- processors can be woken up should be enough.
*/
It might be possible that no AP CPUs are ever started, and are instead left in a "wait for SIPI" default state, assumed to be identical to the BSP, and even assumed to be working (no BIST checks).
AFAIK the normal approach is to send a broadcast INIT-SIPI sequence and wait to see which AP CPUs start (if any), and then either record details somewhere or have each AP CPU add itself to the MP specification and ACPI tables, then setup their MTTRs, thermal monitoring, HT links, SMBASE, etc before sending the APs back to the "wait for SIPI" state - i.e. in theory it should be mostly "configuration-less".
In any case, I'm guessing I need to get LinuxBIOS to generate MP specification and ACPI tables to suit the number of CPUs being emulated by Bochs/Qemu. Is this as simple as adding something like "option CPUs = ?" to the "targets/emulation/qemu-i386/Config.lb" file? I'm guessing it's more involved than this, as I couldn't find an example in the "Config.lb" of other targets.
Please check out the k8 code, especially init_cpus.c, I think you'll find what you're looking for there. Afaik, the k8 is the only smp-supported cpu for the moment.
Either: a) PCI buses and bridges aren't guaranteed to be enumerated, or b) PCI buses and bridges are guaranteed to be enumerated but PCI devices aren't guaranteed to be assigned resources, or c) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (including ROMs) d) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (except ROMs)
"They have their resources." would mean either c) or d), where each device may or may not be in a power saving state (if supported by the device), and may or may not be still configured to use MSI (if supported by the device)....
yes, either c or d. You choose.
You mean all LinuxBIOS developers are willing to make sure all PCI device ROMs either have physical addresses assigned or don't have physical addresses assigned based on my advice? Perhaps I've misunderstood... :-)
I think I have too...
Not at the moment. But in General this is easy. If no VGA class PCI device is there, the machine is 99.99% headless.
And if a VGA class PCI device is present, you've got no idea if no monitor is attached...
Obviously not. and if the device does not do EDD you have no reliable way of finding out either.
There's a simple and easy way to find out if a monitor is attached or not, and it doesn't involve messing about with EDD - simply give the end-user the option of setting (or not setting) a value in the CMOS.
This is already an option, to have vga you run the vga rom and send output to the monitor. Otherwise, you keep serial output and don't bother with the rom.
My (limited) reverse engineering has already shown that there's CMOS entries for a serial port (possibly intended for setting up a serial cable to communicate with a terminal emulator?).
Why reverse engineer? The source code is openly available ;) BTW, we use the serial port as our main debugging source, LinuxBIOS initially sets it up in the pre-ram init code, usually to 115200 8N1. Then the post-ram code reads from the cmos to determine if that speed should be changed or if serial output should be turned off entirely in favor of vga.
Hope I've been a little help
-Corey
Hi,
On 6/12/07, Corey Osgood corey.osgood@gmail.com wrote:
Brendan Trotter wrote:
On 6/12/07, Stefan Reinauer stepan@coresystems.de wrote:
- Brendan Trotter btrotter@gmail.com [070612 12:43]:
check LinuxBIOSv2/util/lbtdump
I've found "src/include/boot/linuxbios_tables.h" and have also looked at the source for LinuxBIOSv2/util/lbtdump.
However, I fail to see how any source code can be considered a viable alternative to documentation for something that is (hopefully) a non-proprietory common standard. Reverse engineering (including relying on assumptions derived from the source code of any specific implementation) is what people do when they have no documentation and are willing to stumble around in the dark hoping to implement something that works until the wind changes.
Yes, no doubt. You are perfectly right, and we are looking forward to your patches to fix this problem. Of course you do not have to reverse engineer, but just read the code, as we are non-proprietary. And of course we are always here to be asked.
Hehe - by the time you've finished answering all my questions you'd wish you'd written the documentation beforehand... ;-)
Probably, but personally I'd rather see Stefan devote his time to the great work they're doing on v3 ;) I'll try to help a little, although my knowledge of the inner workings of LB is somewhat limited.
Thanks :-)
Is "uint32_t" always little-endian or is it architecture specific (reverse engineering ruled out "always big-endian"),
Yes, it should be dependent on the architecture
and would the "lb_header.signature" field always have an ASCII 'L' at the lowest address (or is that architecture specific too)?
I'll assume that also architecture dependant.
no idea...
The source file "util/lbtdump/lbtdump.c" does this:
if (head->header_bytes != sizeof(*head)) { fprintf(stderr, "Header bytes of %d are incorrect\n", head->header_bytes); continue; }From this can I assume that the size of the LinuxBIOS Table header
will always be 24 bytes, and the LinuxBIOS Table header will never be extended in the future (or is this just a bug that makes "lbtdump" unable to handle forward/backward compatability)?
The file "src/include/boot/linuxbios_tables.h" defines 3 types of memory ranges:
#define LB_MEM_RAM 1 /* Memory anyone can use */ #define LB_MEM_RESERVED 2 /* Don't use this memory region */ #define LB_MEM_TABLE 16 /* Ram configuration tables are kept in */
The file "util/lbtdump/lbtdump.c" doesn't agree:
switch(rec->map[i].type) { case 1: mem_type = "ram"; break; case 3: mem_type = "acpi"; break; case 4: mem_type = "nvs"; break; default: case 2: mem_type = "reserved"; break; }Can I assume that the defines in "linuxbios_tables.h" are out-of-date, and that the memory type field is the same as specified by version 2 of the ACPI specification (but perhaps not version 3, as that defines a new "5 = faulty RAM" type)? If "linuxbios_tables.h" is out-of-date, is "16 = RAM configuration tables are stored in" still possible on older versions of LinuxBIOS (and how would I detect which version of LinuxBIOS is present, considering there's no version number in the LinuxBIOS Table header)?
What is the LB_TAG_HWRPB structure. Is it something to do with a "Hardware Restart Parameter Block", and what does it contain? For e.g. would a value of zero indicate that the computer was shutdown correctly before boot, and a non-zero value indicate that the computer crashed or had hardware problems?
Again, no clue really..
Would I also be correct to assume that a payload doesn't need to care about any of the "cmos_entries" and "cmos_enums", unless a BIOS setup/configuration utility is being built into the payload? Alternatively, could/would a payload search for strings to find CMOS values (e.g. search for the cmos_entry with the name string "power_on_after_fail" to see if additional hardware testing can be skipped, search for "baud_rate" to determine the serial port configuration, etc)?
My *understanding* is that different vendors do different things with the cmos, so payloads couldn't count on them for the vendor bios, so they don't for linuxbios either. I suppose a payload *could* do this with linuxbios, if the project wanted to, but I can't think of any that does, although openbios may. Anything cmos-related, such as power on after fail, should currently be handled between linuxbios and a userspace utility, like lxbios.
I just think it'd be annoying for end-users if (for e.g.) the payload decided to use the 28800 8N1 for serial communication when LinuxBIOS is using 57600 8N1 because there was no way for the payload to determine how LinuxBIOS is currently configured.
If LinuxBIOS told the payload "use the frambuffer, which is in ? * ? mode at ?" or "use the serial port at ? at ? baud and ? protocol", then the payload could seemlessly autodetect at run-time and continue using the same method for text output that LinuxBIOS used. Without this end-users would need to mess about with compile-time settings for the payload, and possibly recompile the payload if CMOS settings are changed after LinuxBIOS is installed.
It is of course not unique. It is the index of the vendor name within the strings array of struct lb_mainboard. Since we're not trying to close things down, we don't have to work with magic numbers. So nothing needs to stay unique.
Of course - it's similar to the way SMI/DMI does strings, yet are the strings themselves guaranteed to be unique, or is it possible for one manufacturer to use an identifying string in the future that another (possibly out of business) manufacturer had used in the past?
This is something google would be good for, I can't come up with a good answer. Yes, they SHOULD always be unique, given that there are 65536 possible combinations, but whether there's some sort of registration or assignment system or it's at the manufacturers disposal, I don't know.
I'm not sure googling for the information about the strings the LinuxBIOS project may or may not use in future for it's identifiers would help, unless (for e.g.) the "payload hand-off specification" explicitly states that the identifiers (if present) must be identical to those used by SMI/DMI, in which case I could google for SMI/DMI identifiers....
Motherboard manufacturer identification is one of the things that (IMHO) should be unique. The motherboard part number string should also be unique for that manufacturer ID. Otherwise it'd be difficult for payloads/utilities/OSs to use these IDs to avoid any hardware problems in specific motherboards.
Manufacturers are often very lax about this, in some cases motherboards have 5 or more different (subsystem) device IDs on the same board, or in others they don't bother setting most of them. This is one area where mainboard manufacturers need to clean up.
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
They're not used by payloads (except the Linux kernel)
Surely LinuxBIOS isn't intended as a "Linux only" BIOS though....
Of course not. ACPI is not really useful for payloads though, unless the payload wants to carry a full blown ACPI interpreter. Which rules out basically anything but an OS.
I can think of a variety of uses, ranging from utilities that do nothing more than display hardware information, to diagnostics, to benchmarking, to hyper-visors, to full OSs...
- are AP CPUs started (or waiting for a SIPI sequence)?
yes, hardware is completely up when linuxbios is done.
I tried it on a dual CPU Bochs emulation (second CPU was left in "wait for SIPI" state) - I'm guessing I need to enable multi-CPU support via. some compile time option for the "emulation/qemu-i386" target?
Yes, the emulation/qemu-i386 is per default not supporting SMP.
I'm becoming skeptical here. I can't find anything in LinuxBIOS source code that starts AP CPUs, but in "src/arch/i386/smp/mpspec.c" I did find:
/* If we assume a symmetric processor configuration we can
- get all of the information we need to write the processor
- entry from the bootstrap processor.
- Plus I don't think linux really even cares.
- Having the proper apicid's in the table so the non-bootstrap
- processors can be woken up should be enough.
*/
It might be possible that no AP CPUs are ever started, and are instead left in a "wait for SIPI" default state, assumed to be identical to the BSP, and even assumed to be working (no BIST checks).
AFAIK the normal approach is to send a broadcast INIT-SIPI sequence and wait to see which AP CPUs start (if any), and then either record details somewhere or have each AP CPU add itself to the MP specification and ACPI tables, then setup their MTTRs, thermal monitoring, HT links, SMBASE, etc before sending the APs back to the "wait for SIPI" state - i.e. in theory it should be mostly "configuration-less".
In any case, I'm guessing I need to get LinuxBIOS to generate MP specification and ACPI tables to suit the number of CPUs being emulated by Bochs/Qemu. Is this as simple as adding something like "option CPUs = ?" to the "targets/emulation/qemu-i386/Config.lb" file? I'm guessing it's more involved than this, as I couldn't find an example in the "Config.lb" of other targets.
Please check out the k8 code, especially init_cpus.c, I think you'll find what you're looking for there. Afaik, the k8 is the only smp-supported cpu for the moment.
Either: a) PCI buses and bridges aren't guaranteed to be enumerated, or b) PCI buses and bridges are guaranteed to be enumerated but PCI devices aren't guaranteed to be assigned resources, or c) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (including ROMs) d) PCI buses and bridges are guaranteed to be enumerated and PCI devices are guaranteed to have resources assigned (except ROMs)
"They have their resources." would mean either c) or d), where each device may or may not be in a power saving state (if supported by the device), and may or may not be still configured to use MSI (if supported by the device)....
yes, either c or d. You choose.
You mean all LinuxBIOS developers are willing to make sure all PCI device ROMs either have physical addresses assigned or don't have physical addresses assigned based on my advice? Perhaps I've misunderstood... :-)
I think I have too...
As far as I can tell, Stefan's "You choose." either implies that it's my choice what LinuxBIOS developers do or that it's my choice how end-users configure LinuxBIOS. I doubt I've got that much influence with either group.
I'll assume that whether or not ROMs are assigned physical addresses, whether or not device may be left in a power saving state, and whether or not MSI may still be setup is all "undefined" - not guaranteed either way.
Not at the moment. But in General this is easy. If no VGA class PCI device is there, the machine is 99.99% headless.
And if a VGA class PCI device is present, you've got no idea if no monitor is attached...
Obviously not. and if the device does not do EDD you have no reliable way of finding out either.
There's a simple and easy way to find out if a monitor is attached or not, and it doesn't involve messing about with EDD - simply give the end-user the option of setting (or not setting) a value in the CMOS.
This is already an option, to have vga you run the vga rom and send output to the monitor. Otherwise, you keep serial output and don't bother with the rom.
This is an option for end-users, but not something a payload can assume.
My (limited) reverse engineering has already shown that there's CMOS entries for a serial port (possibly intended for setting up a serial cable to communicate with a terminal emulator?).
Why reverse engineer? The source code is openly available ;)
There is little practical difference between reverse engineering in the traditional sense, and "reverse engineering" source code. In both cases it tells you nothing about what is guaranteed and what is coincedence (or what is incorrect behaviour), which values are valid for different fields, or what may or may not be implemented in future or past versions.
Thanks,
Brendan
On Wed, Jun 13, 2007 at 05:28:02AM +0000, Brendan Trotter wrote:
I just think it'd be annoying for end-users if (for e.g.) the payload decided to use the 28800 8N1 for serial communication when LinuxBIOS is using 57600 8N1 because there was no way for the payload to determine how LinuxBIOS is currently configured.
It can just not set up the serial port but use it immediately and it will keep the settings that LinuxBIOS used.
Granted, which one to choose must still be specified in the payload.
As far as I can tell, Stefan's "You choose." either implies that it's my choice what LinuxBIOS developers do or that it's my choice how end-users configure LinuxBIOS. I doubt I've got that much influence with either group.
You would be surprised how receptive and tolerant this group of people is. Ron even writes code the way I want if I nag him enough. (Hi, Ron! :)
I think what Stefan wanted to say was that LinuxBIOS behavior is rather configurable and the person building it gets to decide many of the things you're asking about.
I'll assume that whether or not ROMs are assigned physical addresses, whether or not device may be left in a power saving state, and whether or not MSI may still be setup is all "undefined"
- not guaranteed either way.
I think this is belittling the effort in this project to initialize hardware.
There's a simple and easy way to find out if a monitor is attached or not, and it doesn't involve messing about with EDD
- simply give the end-user the option of setting (or not
setting) a value in the CMOS.
It depends. This is not workable unless there actually is an end-user. It's also not workable unless said end-user is actually the system administrator. LB was initially designed to explicitly not have much user interaction and certainly not at boot time. A few settings are available, but the idea is to always boot the OS regardless.
This is already an option, to have vga you run the vga rom and send output to the monitor. Otherwise, you keep serial output and don't bother with the rom.
This is an option for end-users, but not something a payload can assume.
Watch out there. CMOS VGA/SERIAL is also an option for end-users, but not something a (robust) payload will want to assume.
My (limited) reverse engineering has already shown that there's CMOS entries for a serial port (possibly intended for setting up a serial cable to communicate with a terminal emulator?).
Why reverse engineer? The source code is openly available ;)
There is little practical difference between reverse engineering in the traditional sense, and "reverse engineering" source code.
This depends a lot on the source code IMHO.
In both cases it tells you nothing about what is guaranteed and what is coincedence (or what is incorrect behaviour), which values are valid for different fields, or what may or may not be implemented in future or past versions.
The code may not, but comments may, and discussion will.
I find that getting "into" open source projects is by far the best way to learn how they think. Documenting the train of thought is IMHO not worthwhile (although for new developers it would be nice) until the code itself actually becomes rather high quality. It's better to write code.
You are right that LB could have more and better documentation but there is already a good amount available. There has not yet been much demand for the kind of documentation you are asking for but perhaps you can help us improve.
Meanwhile just talking to us will get you far.
(Please don't forget to read COPYING if you haven't already, though.)
//Peter
Hi,
On 6/13/07, Peter Stuge stuge-linuxbios@cdy.org wrote:
On Wed, Jun 13, 2007 at 05:28:02AM +0000, Brendan Trotter wrote:
I just think it'd be annoying for end-users if (for e.g.) the payload decided to use the 28800 8N1 for serial communication when LinuxBIOS is using 57600 8N1 because there was no way for the payload to determine how LinuxBIOS is currently configured.
It can just not set up the serial port but use it immediately and it will keep the settings that LinuxBIOS used.
Granted, which one to choose must still be specified in the payload.
As far as I can tell, Stefan's "You choose." either implies that it's my choice what LinuxBIOS developers do or that it's my choice how end-users configure LinuxBIOS. I doubt I've got that much influence with either group.
You would be surprised how receptive and tolerant this group of people is. Ron even writes code the way I want if I nag him enough. (Hi, Ron! :)
If I nag enough, would LinuxBIOS developers be willing to define a few tags for the LinuxBIOS table that a payload can use to determine what LinuxBIOS was using for text output immediately before starting the payload?
For example, in "src/include/boot/linuxbios_tables.h" add something like:
----8<---
#define LB_TAG_OUTPUT_UNKNOWN 0x???? struct lb_output_unknown { uint32_t tag; uint32_t size; };
#define LB_TAG_OUTPUT_SERIAL 0x???? struct lb_output_serial { uint32_t tag; uint32_t size; uint32_t base_port; // Base I/O port for pre-configured serial port uint32_t IRQ // PIC input number for pre-configured serial port };
#define LB_TAG_OUTPUT_VIDEO 0x???? struct lb_output_video { uint32_t tag; uint32_t size; };
#define LB_TAG_OUTPUT_EXTENDED_VIDEO 0x???? struct lb_output_ext_video { uint32_t tag; uint32_t size; uint64_t phys_address; // Physical address of first byte of frame buffer uint32_t resolution_x; // Horizontal resolution of video mode uint32_t resolution_y; // Vertical resolution of video mode uint32_t bytes_per_line; // Number of bytes per screen line uint32_t bpp; // Bits per pixel: // 0 = text mode // 4 = standard VGA planar 16 colour mode // 8 = standard VGA 256 colour mode (with VGA pallete) // 15 = a 5:5:5 pixel format // 16 = a 5:6:5 pixel format // 24 = a 8:8:8 pixel format // 32 = a 8:8:8:8 pixel format uint32_t red_mask; // Mask of bits that control red colour component (unused if "bpp" <= 8) uint32_t green_mask; // Mask of bits that control green colour component (unused if "bpp" <= 8) uint32_t blue_mask; // Mask of bits that control blue colour component (unused if "bpp" <= 8) };
----8<---
Then in the "LinuxBIOS to payload hand-off specification", have something like this (with standard definitions for words like "should", "may", "must", etc):
----8<---
The firmware should use one (and only one) of LB_TAG_OUTPUT_UNKNOWN, LB_TAG_OUTPUT_SERIAL, LB_TAG_OUTPUT_VIDEO, or LB_TAG_OUTPUT_EXTENDED_VIDEO to inform the payload of details for the device it was using immediately before starting the payload. The LB_TAG_OUTPUT_VIDEO tag is provided as an abbreviated version of LB_TAG_OUTPUT_EXTENDED_VIDEO where 80 * 25 text mode (with 160 bytes per line) is implied rather than specified directly.
The firmware may also provide none of these tags in the LinuxBIOS Table, and payloads must handle this as a "no text output possible" condition.
If the LB_TAG_OUTPUT_UNKNOWN tag is specified by the firmware, the payload may either assume that no text output is possible or attempt to use alternative methods to determine if a suitable output device is present.
If the firmware is configured to send output to a serial port, it should include a LB_TAG_OUTPUT_SERIAL tag (and leave the baud rate and communication protocol setup) regardless of whether or not the firmware knows if anything (e.g. a terminal emulator) is connected to that serial port.
----8<---
The intention here is to allow payloads to auto-detect the method they should use for text output, such that no end-user configuration of the payload is necessary.
For a seperate issue, would it be possible to modify LinuxBIOS so that the size of the ROM and the amount used by LinuxBIOS is stored at a fixed locations within the ROM (e.g. so that these values can be read at physical address 0xFFFFFFE8 and 0xFFFFFFEC, just below the first instruction executed by the CPU at power-on)?
Also, would it be possible to have (or is there already) a firmware signature somewhere in the ROM, so that if/when GRUB is used as a payload then software booted by GRUB could check if the signature is present in the ROM (e.g. by checking an 8-byte signature at 0xFFFFFFE0) to determine if legacy BIOS services are still present (no signature) or if LinuxBIOS (and the LinuxBIOS Table) is still present?
The multi-boot specification states that the BIOS environment must be preserved by the boot loader (but doesn't specify *which* BIOS environment). I've written code in the past that relies on this (e.g. drop back to real mode, do a STI, and use BIOS interrupts). It'd be nice if this code could check if the BIOS environment is a legacy BIOS environment or LinuxBIOS... ;-)
I'll assume that whether or not ROMs are assigned physical addresses, whether or not device may be left in a power saving state, and whether or not MSI may still be setup is all "undefined"
- not guaranteed either way.
I think this is belittling the effort in this project to initialize hardware.
It's belittling the specification for the hand-off between LinuxBIOS and payloads, that both LinuxBIOS developers and payload developers would be attempting to comply with.
Consider the multi-boot specification. OS developers can write code that complies with the multi-boot specification without regard to any bootloader, and bootloader writers can write code with that complies with the multi-boot specification without caring about any OS. Despite this, all OSs and all boot loaders that comply with the multi-boot specification will work together without problems. The same applies to EFI and OpenFirmware.
Cheers,
Brendan
* Brendan Trotter btrotter@gmail.com [070614 19:31]:
If I nag enough, would LinuxBIOS developers be willing to define a few tags for the LinuxBIOS table that a payload can use to determine what LinuxBIOS was using for text output immediately before starting the payload?
We should maybe write this down in an architecture or requirements specification, something which i started a while ago but never continued on it yet http://coresystems.de/PDFs/LinuxBIOS-testing/RequirementsSpecification.pdf
Or we should get things porperly documented here instead: http://qa.linuxbios.org/docs/doxygen/
----8<--- #define LB_TAG_OUTPUT_UNKNOWN 0x???? struct lb_output_unknown { uint32_t tag; uint32_t size; };
#define LB_TAG_OUTPUT_SERIAL 0x???? struct lb_output_serial { uint32_t tag; uint32_t size; uint32_t base_port; // Base I/O port for pre-configured serial port uint32_t IRQ // PIC input number for pre-configured serial port };
#define LB_TAG_OUTPUT_VIDEO 0x???? struct lb_output_video { uint32_t tag; uint32_t size; };
#define LB_TAG_OUTPUT_EXTENDED_VIDEO 0x???? struct lb_output_ext_video { uint32_t tag; uint32_t size; uint64_t phys_address; // Physical address of first byte of frame buffer uint32_t resolution_x; // Horizontal resolution of video mode uint32_t resolution_y; // Vertical resolution of video mode uint32_t bytes_per_line; // Number of bytes per screen line uint32_t bpp; // Bits per pixel: // 0 = text mode // 4 = standard VGA planar 16 colour mode // 8 = standard VGA 256 colour mode (with VGA pallete) // 15 = a 5:5:5 pixel format // 16 = a 5:6:5 pixel format // 24 = a 8:8:8 pixel format // 32 = a 8:8:8:8 pixel format uint32_t red_mask; // Mask of bits that control red colour component (unused if "bpp" <= 8) uint32_t green_mask; // Mask of bits that control green colour component (unused if "bpp" <= 8) uint32_t blue_mask; // Mask of bits that control blue colour component (unused if "bpp" <= 8) };
----8<---
Any comments?
In v3 we are going to export the complete device tree in the linuxbios table (are we?)
Maybe we should go and define the above as properties of the devices they are provided by.
The intention here is to allow payloads to auto-detect the method they should use for text output, such that no end-user configuration of the payload is necessary.
This is a very good idea.
For a seperate issue, would it be possible to modify LinuxBIOS so that the size of the ROM and the amount used by LinuxBIOS is stored at a fixed locations within the ROM (e.g. so that these values can be read at physical address 0xFFFFFFE8 and 0xFFFFFFEC, just below the first instruction executed by the CPU at power-on)?
There are plans to do this in v3.
Also, would it be possible to have (or is there already) a firmware signature somewhere in the ROM, so that if/when GRUB is used as a payload then software booted by GRUB could check if the signature is present in the ROM (e.g. by checking an 8-byte signature at 0xFFFFFFE0) to determine if legacy BIOS services are still present (no signature) or if LinuxBIOS (and the LinuxBIOS Table) is still present?
It could look for the linuxbios table in ram. Then it does not need to assume where the rom is.
drop back to real mode, do a STI, and use BIOS interrupts). It'd be nice if this code could check if the BIOS environment is a legacy BIOS environment or LinuxBIOS... ;-)
LinuxBIOS has no such environment, except the linuxbios table. If you want callbacks, openfirmware might be a good payload for you.
Consider the multi-boot specification. OS developers can write code that complies with the multi-boot specification without regard to any bootloader, and bootloader writers can write code with that complies with the multi-boot specification without caring about any OS. Despite this, all OSs and all boot loaders that comply with the multi-boot specification will work together without problems. The same applies to EFI and OpenFirmware.
I think some of this is actually more in the domain of EFI and Open Firmware than in LinuxBIOS, which is more a low level firmware. So we should work on getting EFI / OFW easily usable as an interface for everyone instead of defining a third interface. to the OS and user. Our interface is to OFW / EFI though (plus other options of course)
Stefan
On 14/06/07 21:33 +0200, Stefan Reinauer wrote:
The intention here is to allow payloads to auto-detect the method they should use for text output, such that no end-user configuration of the payload is necessary.
This is a very good idea.
I disagree. Letting payloads rely on LinuxBIOS to set up devices is the start of a slippery slope that we should try like mad to avoid.
There is nothing about LinuxBIOS today that guarantees that any output method is available (even serial). If we encourage payloads to use structures like these and make assumptions about the state of the hardware, then when a developer chooses to omit initializing the serial port or other output method, his e-mail will quickly catch on fire, thereby implicitly forcing everybody to support serial and video and heaven knows what else. It is not LinuxBIOS's reponsiblity to make life easier for the payloads.
Consider the multi-boot specification. OS developers can write code that complies with the multi-boot specification without regard to any bootloader, and bootloader writers can write code with that complies with the multi-boot specification without caring about any OS. Despite this, all OSs and all boot loaders that comply with the multi-boot specification will work together without problems. The same applies to EFI and OpenFirmware.
I think some of this is actually more in the domain of EFI and Open Firmware than in LinuxBIOS, which is more a low level firmware. So we should work on getting EFI / OFW easily usable as an interface for everyone instead of defining a third interface. to the OS and user. Our interface is to OFW / EFI though (plus other options of course)
Absolutely. LinuxBIOS is not a OS loader, it shouldn't be asked to act as one.
Jordan
On Thu, Jun 14, 2007 at 04:36:01PM -0600, Jordan Crouse wrote:
On 14/06/07 21:33 +0200, Stefan Reinauer wrote:
The intention here is to allow payloads to auto-detect the method they should use for text output, such that no end-user configuration of the payload is necessary.
This is a very good idea.
I disagree. Letting payloads rely on LinuxBIOS to set up devices is the start of a slippery slope that we should try like mad to avoid.
I understand your concern but I agree with Bretton and Stefan.
There is nothing about LinuxBIOS today that guarantees that any output method is available (even serial).
We haven't thought enough about it yet, but there has been discussion about a LinuxBIOS panic() that can still actually be used to do meaningful things with the system. (Reboot, download code, execute code, reflash, peek and poke maybe.) This needs to communicate somehow. Serial isn't always available but we do want _some_ console. For making the port we NEED a console. In theory we may not but in practice ICE debuggers cost mega-$ and code written dry is not complete after the first try running it.
If we encourage payloads to use structures like these and make assumptions about the state of the hardware,
The idea is that structures should be used to _not_ make assumptions.
They should have enough info about the hw init that has been done so that anyone interested can have a well-defined workflow.
It is not LinuxBIOS's reponsiblity to make life easier for the payloads.
Actually that is it's sole responsibility. :)
LinuxBIOS is not a OS loader, it shouldn't be asked to act as one.
Completely true!
//Peter
On Fri, Jun 15, 2007 at 12:46:42AM +0200, Peter Stuge wrote:
Bretton
I am so sorry Brendan.
I have made a lot of these name booboos lately. :(
//Peter
* Jordan Crouse jordan.crouse@amd.com [070615 00:36]:
I disagree. Letting payloads rely on LinuxBIOS to set up devices is the start of a slippery slope that we should try like mad to avoid.
We do know there are devices that won't work if LinuxBIOS does not do the job. RAM, PCI resources, SCSI, VGA, ...
There are plenty of cases where the payload relies on LinuxBIOS setting up the hardware.
This is the whole point of LinuxBIOS: setting up the hardware.
There is nothing about LinuxBIOS today that guarantees that any output method is available (even serial). If we encourage payloads to use structures like these and make assumptions about the state of the hardware, then when a developer chooses to omit initializing the serial port or other output method, his e-mail will quickly catch on fire, thereby implicitly forcing everybody to support serial and video and heaven knows what else.
Why would LinuxBIOS not initialize serial console, but a payload would? If LinuxBIOS does not initialize it, it's not there.
The opposite: Payload writers' email catches fire because you set a baud rate with "lxbios" but the payload just hardcoded its own serial speed. So change baud rate once and you end up with a messed up terminal every time you boot until you change it back. Bad. Solution: Rely on LinuxBIOS doing its job.
It is not LinuxBIOS's reponsiblity to make life easier for the payloads.
Ok, so what would you say is LinuxBIOS' job then?
Stefan
On 15/06/07 01:02 +0200, Stefan Reinauer wrote:
- Jordan Crouse jordan.crouse@amd.com [070615 00:36]:
I disagree. Letting payloads rely on LinuxBIOS to set up devices is the start of a slippery slope that we should try like mad to avoid.
We do know there are devices that won't work if LinuxBIOS does not do the job. RAM, PCI resources, SCSI, VGA, ...
There are plenty of cases where the payload relies on LinuxBIOS setting up the hardware.
This is the whole point of LinuxBIOS: setting up the hardware.
I guess it sort of depends on your definition of "hardware". :) We do the steps to enable the system to run, and to enumerate certain parts of the hardware so that the payload can see them, but we don't often make policy decisions for devices hanging off the end of the busses. We don't set up IDE drives. We don't configure sound cards, or initialize NICs, or any of those things. The point is, that LinuxBIOS shouldn't really be required to leave individual devices in any sort of sane state.
There is nothing about LinuxBIOS today that guarantees that any output method is available (even serial). If we encourage payloads to use structures like these and make assumptions about the state of the hardware, then when a developer chooses to omit initializing the serial port or other output method, his e-mail will quickly catch on fire, thereby implicitly forcing everybody to support serial and video and heaven knows what else.
Why would LinuxBIOS not initialize serial console, but a payload would? If LinuxBIOS does not initialize it, it's not there.
Again, words get in the way. LinuxBIOS does initalize and enumerate it, but I don't really think that the intent is to require LinuxBIOS to tromp out and touch the 16550 and set baud rates, and other such configuration settings so that the payload doesn't have to. Am I wrong?
It is not LinuxBIOS's responsibility to make life easier for the payloads.
Ok, so what would you say is LinuxBIOS' job then?
I think LinuxBIOS is there to initalize and enumerate the hardware at a very low level. I don't think it is there to be a surrogate serial / smbus / IDE / VGA / whatever driver for payloads that don't wish to implement those things on their own. Of course, I say that because I'm pretty much only interested in loading payloads that do support those services on their own - but I personally think one of LinuxBIOS's greatest assets is that it does only what it needs to, and puts the responsibility on me to do the rest. The power and flexibility of that single trait is worth it.
Jordan
On Thu, Jun 14, 2007 at 05:28:26PM -0600, Jordan Crouse wrote:
We don't configure sound cards, or initialize NICs, or any of those things.
Well, we sort-of do.
The point is, that LinuxBIOS shouldn't really be required to leave individual devices in any sort of sane state.
It should leave devices in a "reachable" state. Depending on the bus they are connected to this may or may not imply some level of sanity.
Why would LinuxBIOS not initialize serial console, but a payload would? If LinuxBIOS does not initialize it, it's not there.
Again, words get in the way. LinuxBIOS does initalize and enumerate it, but I don't really think that the intent is to require LinuxBIOS to tromp out and touch the 16550 and set baud rates, and other such configuration settings so that the payload doesn't have to. Am I wrong?
I think so, for the specific case of the console serial port (or a replacement).
The serial port (or replacement) is crucial for developers and initially it was also crucial for users. Those users are still around but now we also want to help users that consider other devices to be crucial.
I think LinuxBIOS is there to initalize and enumerate the hardware at a very low level. I don't think it is there to be a surrogate serial / smbus / IDE / VGA / whatever driver for payloads that don't wish to implement those things on their own.
This is a very important point and I agree completely.
LinuxBIOS is not a set of drivers and does not provide services.
Of course, I say that because I'm pretty much only interested in loading payloads that do support those services on their own
Software needs system buses to be in a "working" state where devices are "reachable" and some of us considers this to imply that devices are "initialized" and set to some state because they have in fact been touched while others consider only the bus itself to be initialized.
Both are completely accurate, they actually mean the same thing.
Devices are never _driven_ by LinuxBIOS after the payload has been started however, and most devices aren't driven before the payload starts either.
However - LinuxBIOS will need to "hand over" the system to the payload and I guess that the formalization of this handover is what this thread is all about. It should include providing the payload with information about how the system is set up as of now so that good payloads can be written which will work on any setup.
//Peter
On 15/06/07 02:08 +0200, Peter Stuge wrote:
However - LinuxBIOS will need to "hand over" the system to the payload and I guess that the formalization of this handover is what this thread is all about. It should include providing the payload with information about how the system is set up as of now so that good payloads can be written which will work on any setup.
Ah - perhaps thats the problem. I read the request as saying "We don't want to set up the serial, so set it up and tell us how you did it", but perhaps the request was really "if you set up the serial, then tell me, and I'll do it the same way".
So let me ask you this - under the proposal, what if I choose to tell the payload that the baud rate is 0. Does the payload then pick a sane baud rate of its own, and program the 16550 itself, or does it collapse in a heap and claim that LinuxBIOS didn't work because the serial settings are not sane?
If its the first scenario - then I'm on board. Its the second that I am worried about.
Jordan
On Thu, Jun 14, 2007 at 06:24:47PM -0600, Jordan Crouse wrote:
On 15/06/07 02:08 +0200, Peter Stuge wrote:
However - LinuxBIOS will need to "hand over" the system to the payload and I guess that the formalization of this handover is what this thread is all about. It should include providing the payload with information about how the system is set up as of now so that good payloads can be written which will work on any setup.
Ah - perhaps thats the problem. I read the request as saying "We don't want to set up the serial, so set it up and tell us how you did it", but perhaps the request was really "if you set up the serial, then tell me, and I'll do it the same way".
No need to set it up again - but with the payload knowing how the system _is_ set up it can just adapt and start using the system without doing any setup.
So let me ask you this - under the proposal, what if I choose to tell the payload that the baud rate is 0. Does the payload then pick a sane baud rate of its own, and program the 16550 itself, or does it collapse in a heap
This depends completely on the payload.
If you tell the payload (via LB or directly) that the baud rate is 0 it can't do much for you.
If you tell LB that the rate is 0 but the payload that the rate is 115200 I expect the payload to set up the UART so you will get serial from the payload.
and claim that LinuxBIOS didn't work because the serial settings are not sane?
There is no spec so there can't really be any expectations yet.
In practice, it has turned out that a serial port is usually there and working.
If its the first scenario - then I'm on board. Its the second that I am worried about.
Payloads should be smart and handle all configs that make sense but LB needs to provide data points to support those smart decisions.
IMHO if a payload does not work because of something it can not control (the mainboard has no serial, but payload requires serial) then it is broken or applied where it does not fit. Broken because it could have looked at the (future) system spec (which could be the device tree) and figured out how to fail gracefully when no serial port exists all on it's own.
//Peter
This should all be so simple.
The original idea was that LB would turn on dram, call OS, done. OS would do all config space.
Then, turned out linux can't configure unconfigured PCI BARs. And, we needed to do some PCI config for SPD.
So, LB starts doing ALL PCI config -- no way out.
Then, turns out some devices put things in IO space that affect PCI config space. YUCK! But we have to start doing things in IO space for PCI config space.
Then, turns out that many IRQ tables are wrong. So, we start setting IRQs.
And so on and so on ....
Really, I think LB should only do those things the OS can not possibly do. The "OS can't do" is a boundary we hoped would be very solid and immovable. The problem is, buggy hardware, and buggy kernels, make that boundary very, very hard to find. And, both hardware and kernels keep changing --> the boundary is flexible. In 1999 I thought there would be simple, hard rules; I was quite wrong.
Every new piece of hardware seems to push the boundary around in some way. It is difficult sometimes to know what to do. But, in the end, LB should really try to do almost nothing, and let its payload(s) do almost everything.
ron
This should all be so simple.
Famous last words? ;-)
The original idea was that LB would turn on dram, call OS, done. OS would do all config space.
Really, I think LB should only do those things the OS can not possibly do. The "OS can't do" is a boundary we hoped would be very solid and immovable.
Well there's not only "OS can't do", but also "OS won't do." In theory the OS could do _everything_.
A more realistic boundary is for LB to do only the bare minimum that any system firmware should reasonably do; i.e. initialise system resources (CPUs, system busses, system controllers (north/south bridge config, superio, board-specific gpios, that kind of stuff)), and of course do "some stuff" with peripheral device the firmware itself want to use (serial port, for example).
So basically all the "core logic" and the busses "inside the box", so when you call the OS it will have a good stable well-defined starting point to work from.
In theory the OS could configure most of the system busses, but as you point out, in many cases people got way too creative in their system design so you really need to set up those busses quite a bit earlier.
Segher
"ron minnich" rminnich@gmail.com writes:
This should all be so simple.
The original idea was that LB would turn on dram, call OS, done. OS would do all config space.
Then, turned out linux can't configure unconfigured PCI BARs. And, we needed to do some PCI config for SPD.
So, LB starts doing ALL PCI config -- no way out.
Then, turns out some devices put things in IO space that affect PCI config space. YUCK! But we have to start doing things in IO space for PCI config space.
Then, turns out that many IRQ tables are wrong. So, we start setting IRQs.
And so on and so on ....
Really, I think LB should only do those things the OS can not possibly do. The "OS can't do" is a boundary we hoped would be very solid and immovable. The problem is, buggy hardware, and buggy kernels, make that boundary very, very hard to find. And, both hardware and kernels keep changing --> the boundary is flexible. In 1999 I thought there would be simple, hard rules; I was quite wrong.
I think we can have a definition that is slightly more stringent. LinuxBIOS should take care of the motherboard specific details, and ensure architectural hardware shows up at the architectural defined location.
None of this suggests we will setup any of this hardware for use other than setting it's bars.
It makes the responsibility of the OS/payload to figure out how to use everything that is not hardware specific.
Every new piece of hardware seems to push the boundary around in some way. It is difficult sometimes to know what to do. But, in the end, LB should really try to do almost nothing, and let its payload(s) do almost everything.
Yep.
Eric
* Jordan Crouse jordan.crouse@amd.com [070615 02:24]:
So let me ask you this - under the proposal, what if I choose to tell the payload that the baud rate is 0. Does the payload then pick a sane baud rate of its own, and program the 16550 itself, or does it collapse in a heap and claim that LinuxBIOS didn't work because the serial settings are not sane?
If its the first scenario - then I'm on board. Its the second that I am worried about.
It should not collapse of course. Maybe LB should not report 0 but just dont report any baud rate at all in this case.
* Peter Stuge stuge-linuxbios@cdy.org [070615 02:08]:
I think so, for the specific case of the console serial port (or a replacement).
The serial port (or replacement) is crucial for developers and initially it was also crucial for users. Those users are still around but now we also want to help users that consider other devices to be crucial.
Yes, serial port is at least crucial until we get linuxbios shipped on a given board as default boot firmware
I think LinuxBIOS is there to initalize and enumerate the hardware at a very low level. I don't think it is there to be a surrogate serial / smbus / IDE / VGA / whatever driver for payloads that don't wish to implement those things on their own.
This is a very important point and I agree completely.
LinuxBIOS is not a set of drivers and does not provide services.
So are you saying we should drop VGA init from v3 completely and make that a seperate payload? While rather working on making payloads executable serially than to push this code into LinuxBIOS?
But then the LB table needs to be enhancable by any payload any time later.
LinuxBIOS should provide all information it knows about the hw to the code that runs afterwards. This "we leave everything undefined" game does not work out. Because no one else feels responsible to change the "undefined" state later on either. The bootloader does not care except for the boot device, the OS does not do the job. Houston we have a problem. Bringing the hardware into a defined state is what we want. Power on default can very well be a defined state for many/most devices.
Software needs system buses to be in a "working" state where devices are "reachable" and some of us considers this to imply that devices are "initialized" and set to some state because they have in fact been touched while others consider only the bus itself to be initialized.
One example is VGA. It's not a bus. But if you are saying it is not the BIOS' task to enable VGA. Whose is it. The bootloaders? Come on, that would not be an improvement of design. Also we have more bootloaders than we have low level firmware. It would be a desaster for the workload and thus stability of bootloader writers. Calling a VGA option rom in a bootloader is about as wrong in my opinion as initializing DRAM in an OS kernel is.
Initializing VGA in the kernel is too late for 90% of real world applications. Imagine someone wants to change the boot device. And no, we do not have 16MBit flashes to pack Linux into on most systems. Not this year and not next year. And the opportunities in 2009 should be discussed, but not be a problem for our use today.
Devices are never _driven_ by LinuxBIOS after the payload has been started however, and most devices aren't driven before the payload starts either.
Most devices, like video output do not need to be driven. Drivers in LinuxBIOS context means initialization. Thats all LinuxBIOS ever does: Initialize HW. No, no sound cards. But: Basic I/O subsystems. The machine needs to talk to us.
The first computer I was working on was an ITT3030. It's firmware would have you start the OS loader from disk by typing a blind sequence of debugger commands into the firmware prompt (that I never saw, I had no idea about a console at the time. I just knew the sequence). I feel we should not go back to 1984 or earlier with our standards.
However - LinuxBIOS will need to "hand over" the system to the payload and I guess that the formalization of this handover is what this thread is all about.
honestly, I dont think so. The handover is loading an ELF file. How unformalized can that be.
It should include providing the payload with information about how the system is set up as of now so that good payloads can be written which will work on any setup.
yes!
One example is VGA. It's not a bus. But if you are saying it is not the BIOS' task to enable VGA. Whose is it.
The kernel (or some userland, preferably/partially) needs to be able to initialise graphics controllers from the ground up, to support sleep modes (and also to support more-than-one-card setups, in the legacy BIOS case). This is a well-understood problem in the Linux graphics world, but quite hard to solve it seems -- it is taking forever. Some progress is being made though.
If LB wants to use the graphics hardware by itself, i.e. for the boot menu or something similar, it of course has to init the graphics and contain a minimal driver, too.
Calling a VGA option rom in a bootloader is about as wrong in my opinion as initializing DRAM in an OS kernel is.
It is philosophically wrong, and hard to make it work reliably too, esp. in a generic cross-vendor way. It is more of a band-aid than an actual solution.
Devices are never _driven_ by LinuxBIOS after the payload has been started however,
It can be useful to have a single callback into LB to display debug output on whatever the debug output device is. Anything more than that is quite not in line with the LB philosophy I'd say; you'd be better of using a full blown Open Firmware implementation if you want a feature-rich boot environment.
However - LinuxBIOS will need to "hand over" the system to the payload and I guess that the formalization of this handover is what this thread is all about.
honestly, I dont think so. The handover is loading an ELF file. How unformalized can that be.
The handover is the jumping to the ELF entry point, instead ;-)
Segher
On Thu, Jun 14, 2007 at 05:28:26PM -0600, Jordan Crouse wrote:
On 15/06/07 01:02 +0200, Stefan Reinauer wrote:
- Jordan Crouse jordan.crouse@amd.com [070615 00:36]:
I disagree. Letting payloads rely on LinuxBIOS to set up devices is the start of a slippery slope that we should try like mad to avoid.
We do know there are devices that won't work if LinuxBIOS does not do the job. RAM, PCI resources, SCSI, VGA, ...
There are plenty of cases where the payload relies on LinuxBIOS setting up the hardware.
This is the whole point of LinuxBIOS: setting up the hardware.
I guess it sort of depends on your definition of "hardware". :) We do the steps to enable the system to run, and to enumerate certain parts of the hardware so that the payload can see them, but we don't often make policy decisions for devices hanging off the end of the busses. We don't set up IDE drives. We don't configure sound cards, or initialize NICs, or any of those things.
Well, depends on the definition of "set up" or "configure" :)
LinuxBIOS _can_ enable the secondary IDE interface, for example. Or it may not do it (or disable it explicitly), so that the OS (or payload) won't see or be able to access that IDE interface. This is configurable.
The point is, that LinuxBIOS shouldn't really be required to leave individual devices in any sort of sane state.
I agree, and the important word here is "required"!
A payload (bootloader, OS, application, whatever) should _not_ depend on device so-and-so being enabled or setup. In general, it can't do that anyway. Say you're an OS. You have no idea whether there's a floppy drive; it may not be present, it may have been disabled by the BIOS, you just don't know; thus, you cannot rely on it _anyway_. The same is true for almost all other devices in any modern computer.
Now to the important point (IMHO):
There are _some_ things which a payload _must_ be able to rely upon, and that's what "we" (LinuxBIOS) must set up correctly. And that's exactly the stuff we should formally define somewhere as minimum requirements a payload _can_ rely upon when it's invoked.
This is not much, though. Short brainstorming:
- RAM must work. It it's not configured, no reasonable payload will be able to work. This is the very minimum we _must_ do.
- ???
That's about it? Everything else is probably optional (in my eyes). Please make other suggestions, maybe I'm overlooking something, but RAM is just about everything I can come up with right now.
IDE, SATA, floppy, sound, USB, NIC, serial console, USB console, etc. etc. etc. All of that is completely optional. It might be enabled or it might not (it's user-configurable). Thus, a payload can (in general) not rely on this stuff.
Again, words get in the way. LinuxBIOS does initalize and enumerate it, but I don't really think that the intent is to require LinuxBIOS to tromp out and touch the 16550 and set baud rates, and other such configuration settings so that the payload doesn't have to. Am I wrong?
Nope, I don't think so.
We _can_ do that (and do it for most boards), but it's an optional thing. The payload cannot rely on an already setup serial port. And rightly so.
Some payloads (FILO, GRUB, memtest, Linux) have their own options to enable serial consoles and that already works great. No need for us to intervene here. We set up a serial console if _we_ (LinuxBIOS) want to print stuff to a serial console. We don't care whether the payload which comes after us wants or needs a serial console. That's none of our business, IMO.
Ok, so what would you say is LinuxBIOS' job then?
I think LinuxBIOS is there to initalize and enumerate the hardware at a very low level.
Full ack.
Maybe even "at _that_ low level where any other payload _cannot_ do the job even if it tried to".
There is some hardware which _has_ to be enabled by the BIOS / low level firmware, otherwise no OS or payload can use it:
- HPET (which is only half true, there are patches for Linux to force-enable it even if the BIOS didn't enable it)
- TPM chip (I guess if the BIOS disableѕ it explicitly no OS can re-enable it again!?)
- Hardware virtualization features (Pacifica / Vanderpool). My understanding is that these -- if disabled by the BIOS -- cannot be enabled by any OS(?)
- ...
This is the sort of stuff which is the "job" of LinuxBIOS, IMO. But still, this is _configurable_ so a payload _cannot_ rely on it anyway.
I don't think it is there to be a surrogate serial / smbus / IDE / VGA / whatever driver for payloads that don't wish to implement those things on their own.
I agree.
However, what would be useful is sort of a small "payload library" (which is _not_ part of LinuxBIOS) which payloads can use to speed up development. That library could be something libc-like or something smallish like a 'Serial+VGA+Keyboard-driver'-lib...
There are some potential projects out there where we could borrow code to implement something like that (with some hacking maybe), e.g. dietlibc, uclibc, klibc, newlib, FILO, GRUB2, memtest86, busybox, ...
All of them have some sort of keyboard driver, some sort of serial driver etc. etc.
Of course, I say that because I'm pretty much only interested in loading payloads that do support those services on their own - but I personally think one of LinuxBIOS's greatest assets is that it does only what it needs to, and puts the responsibility on me to do the rest. The power and flexibility of that single trait is worth it.
Full Ack.
Uwe.
* Uwe Hermann uwe@hermann-uwe.de [070615 02:33]:
IDE, SATA, floppy, sound, USB, NIC, serial console, USB console, etc. etc. etc. All of that is completely optional. It might be enabled or it might not (it's user-configurable). Thus, a payload can (in general) not rely on this stuff.
For SATA a payload can often just use a normal IDE driver, if LinuxBIOS set up some parts of the device, i.e. enable the IDE interface of the controller or pump some microcode into the controller.
Hey, and then think VSA. We want the VGA device to be VGA compatible, thus VSA is used to "enable" the hardware. We want to have VSA drivers, or each payload has to know how to initialize board specific(?) code for the given system. Bad idea.
There is some hardware which _has_ to be enabled by the BIOS / low level firmware, otherwise no OS or payload can use it:
- HPET (which is only half true, there are patches for Linux to force-enable it even if the BIOS didn't enable it)
How do these patches know where the HPET sits?
- TPM chip (I guess if the BIOS disableѕ it explicitly no OS can re-enable it again!?)
There are locking mechanisms, but the background here is that you want a safe boot chain for example, so the LLFW needs to enable TPM to make sure the payload is trusted, which needs it to make sure the OS is trusted, etc.
- Hardware virtualization features (Pacifica / Vanderpool). My understanding is that these -- if disabled by the BIOS -- cannot be enabled by any OS(?)
Again, some BIOSes lock these, then the OS can't do anything. We don't enable them, but we don't lock them either. So the OS can do what it wants.
However, what would be useful is sort of a small "payload library" (which is _not_ part of LinuxBIOS) which payloads can use to speed up development. That library could be something libc-like or something smallish like a 'Serial+VGA+Keyboard-driver'-lib...
Yes, definitely a good idea. Another attempt besides the great unification is to make sure all of the main payloads work really well. None of them really does so far - they all have drawbacks.
I think a payload should not need to be compiled for the system it is going to run on but should be able to ask LinuxBIOS what system it is running on and go generic from there on.
Stefan
Hi,
On 6/14/07, Stefan Reinauer stepan@coresystems.de wrote:
- Brendan Trotter btrotter@gmail.com [070612 16:51]:
The file "src/include/boot/linuxbios_tables.h" defines 3 types of memory ranges:
#define LB_MEM_RAM 1 /* Memory anyone can use */ #define LB_MEM_RESERVED 2 /* Don't use this memory region */ #define LB_MEM_TABLE 16 /* Ram configuration tables are kept in */
first two are types, third is the magic for the linuxbios table ("table type")
The file "util/lbtdump/lbtdump.c" doesn't agree:
switch(rec->map[i].type) { case 1: mem_type = "ram"; break;case 3: mem_type = "acpi"; break; case 4: mem_type = "nvs"; break;so these two are missing above. Maybe they were never used..
For legacy BIOSs, the ACPI memory type defines an area that contains the ACPI tables, that the OS may free and use as normal RAM after it's finished parsing the ACPI tables. In this case (IMHO) for compatability purposes LinuxBIOS should also use the ACPI memory type to tell payloads (and OSs, via. the payload/s) that the area can be freed and used as normal RAM. BTW I'm assuming the LB_MEM_TABLE may be used by an OS in the same way (ie. the area can be used as RAM if the OS is finished with the tables).
The ACPI NVS memory type also has special meaning, although to be honest I've never fully understood it's purpose. As far as I can tell there's at least 2 reasons for it.
The first is that an OS may access CMOS/RTC, and the firmware (the BIOS's SMI handler and/or the ACPI AML interpretter) may also access CMOSs. For some (most?) motherboards the CMOS has a single index and data I/O port, which creates a problem when both try to access the CMOS at the same time (for e.g. OS sets the index I/O port before reading a CMOS location, firmware changes the index I/O port and reads from the data I/O port, then the OS reads from data I/O port without knowing the firmware messed with it and gets the wrong location). To avoid this the BIOS creates a copy of CMOS's contents and stores it in NVS, so that the AML interpretter and/or BIOS's SMI handler can access NVS instead of the CMOS.
The ACPI NVS area is also involved with system power management somehow - IIRC an OS must save the NVS area before entering some sleep states (e.g. "hybernate") and restore it when waking up from that sleep state.
To simplify documentation (and compatability problems for OSs not designed for LinuxBIOS, given that payloads like GRUB would probably just copy memory ranges from the LinuxBIOS Table to their own data structures), it may be reasonable for LinuxBIOS to use the same values for memory types as ACPI, plus define the additional LB_MEM_TABLE memory type.
Alternatively, could/would a payload search for strings to find CMOS values (e.g. search for the cmos_entry with the name string "power_on_after_fail" to see if additional hardware testing can be skipped, search for "baud_rate" to determine the serial port configuration, etc)?
Yes.
Hehe - is there a master list of standard strings defined by LinuxBIOS for the "cmos_entry" structures?
Yes, the emulation/qemu-i386 is per default not supporting SMP.
I'm becoming skeptical here. I can't find anything in LinuxBIOS source code that starts AP CPUs, but in "src/arch/i386/smp/mpspec.c" I did find:
check src/cpu/x86/lapic/lapic_cpu_init.c
My apologies - you are of course correct. I'll look into enabling Qemu SMP support (and LinuxBIOSv3 and VGA) soon :-)
On 6/14/07, Stefan Reinauer stepan@coresystems.de wrote:
- Brendan Trotter btrotter@gmail.com [070614 19:31]:
If I nag enough, would LinuxBIOS developers be willing to define a few tags for the LinuxBIOS table that a payload can use to determine what LinuxBIOS was using for text output immediately before starting the payload?
We should maybe write this down in an architecture or requirements specification, something which i started a while ago but never continued on it yet http://coresystems.de/PDFs/LinuxBIOS-testing/RequirementsSpecification.pdf
Or we should get things porperly documented here instead: http://qa.linuxbios.org/docs/doxygen/
A project like LinuxBIOS involves 4 seperate groups of people - LinuxBIOS developers, motherboard manufacturers, payload developers, and end-users. Each group of people has different requirements.
LinuxBIOS developers need the source code itself, plus (optionally) something to guide them on the future direction of the project and something to make becoming familiar with the inner workings easier. AFAIK this group is adequately addressed via. the mailing list, docbook and the wiki.
Motherboard manufacturers are completely different. They need something they can use while determining whether or not to support LinuxBIOS - e.g. a concise document that describes all the benefits of supporting LinuxBIOS, plus information on how to become a LinuxBIOS developer (in case they become willing to offer direct support) and a list of what information existing LinuxBIOS developers would need to support their product/s (in case they become willing to offer indirect support).
Payload developers are different again. The only thing that really matters to these people is a document describing the state of hardware when their code is started, and what format their code needs to be in. This could be compared to things like the multi-boot specification, the ELF specification, the OpenFirmware specification, and (to a much lesser extent, due to the ugly mess it's become) the many specifications and other documentation that makes up the legacy PC BIOS "standard".
End-users are different again. All they want is something to describe how to configure, install and use LinuxBIOS. They don't necessarily care about any of the documentation and other resources used by any of the other groups. Currently this group is addressed by the "Build Tutorials" section of the wiki, and possibly other parts of the wiki.
In general, the best documentation for one group isn't really the best documentation for any other group.
In v3 we are going to export the complete device tree in the linuxbios table (are we?)
If it was my descision, I'd use something similar to the approach ACPI used - ie. LinuxBIOS includes information that can't be obtained in other standard ways (like scanning PCI configurations space, the ISA Plug & Play specification, MP specification, ACPI specification, etc).
I'd have 3 exceptions to this (for 80x86): 1) basic text output, so that if a problem occurs while attempting to use those other standard ways, then the payload can still let the end-user know what the problem is. 2) motherboard resources, as the only standard way of detecting them is ACPI, but it requires a large and over-complicated AML interpretter to find out the simplest of things. 3) crappy old ISA cards, where no sane method of detection exists (for OSs, payloads, or LinuxBIOS itself).
drop back to real mode, do a STI, and use BIOS interrupts). It'd be nice if this code could check if the BIOS environment is a legacy BIOS environment or LinuxBIOS... ;-)
LinuxBIOS has no such environment, except the linuxbios table. If you want callbacks, openfirmware might be a good payload for you.
From my perspective, it's simply about knowing what can and can't be
used. For e.g. if my code knows it can't drop back to real mode and use legacy BIOS interrupts, then it can skip it or use an alternative (in this specific case, I only used legacy BIOS interrupts to allow the user to select and change the video mode).
Consider the multi-boot specification. OS developers can write code that complies with the multi-boot specification without regard to any bootloader, and bootloader writers can write code with that complies with the multi-boot specification without caring about any OS. Despite this, all OSs and all boot loaders that comply with the multi-boot specification will work together without problems. The same applies to EFI and OpenFirmware.
I think some of this is actually more in the domain of EFI and Open Firmware than in LinuxBIOS, which is more a low level firmware. So we should work on getting EFI / OFW easily usable as an interface for everyone instead of defining a third interface. to the OS and user. Our interface is to OFW / EFI though (plus other options of course)
Put more generally, all interfaces between components developed by different parties should be adequately documented. This includes the interface between hardware and LinuxBIOS, the interface between LinuxBIOS and it's payloads, the interface/s between payloads and anything that use the payload/s, the interface between boot loaders and OSs, the interface between an OS and software designed for that OS, etc.
BTW I'm more interested in "boot from ROM" systems, where the "OS" is stored in ROM, and doesn't require or rely on disk drives or networking (but may use disk drives and/or networking if present, after the system is running). GRUB, Openfirmware and EFI payloads would consume too much of the limited space in ROM.
Cheers,
Brendan
* Brendan Trotter btrotter@gmail.com [070613 07:28]:
I just think it'd be annoying for end-users if (for e.g.) the payload decided to use the 28800 8N1 for serial communication when LinuxBIOS is using 57600 8N1 because there was no way for the payload to determine how LinuxBIOS is currently configured.
Often, the payloads have the choice to expect the serial port being initialized, so they don't change the speed. But I agree your approach makes sense.
One thing we are lacking though is a library or small set of functions that allows the (os less) payload to access the cmos and parse those parts of the lb table. all a payload ever parsed that i know of is the "memory" part of the memory tables
If LinuxBIOS told the payload "use the frambuffer, which is in ? * ? mode at ?" or "use the serial port at ? at ? baud and ? protocol", then the payload could seemlessly autodetect at run-time and continue using the same method for text output that LinuxBIOS used. Without this end-users would need to mess about with compile-time settings for the payload, and possibly recompile the payload if CMOS settings are changed after LinuxBIOS is installed.
Yes. We must export a lot more information in the LB table. I like this idea. And I think v3 will be a lot more like that.
I'm not sure googling for the information about the strings the LinuxBIOS project may or may not use in future for it's identifiers would help, unless (for e.g.) the "payload hand-off specification" explicitly states that the identifiers (if present) must be identical to those used by SMI/DMI, in which case I could google for SMI/DMI identifiers....
Nope, any similarities are coincidential. And not intended? ;-)
As far as I can tell, Stefan's "You choose." either implies that it's my choice what LinuxBIOS developers do or that it's my choice how end-users configure LinuxBIOS. I doubt I've got that much influence with either group.
When you build a LinuxBIOS image, you can decide whether this image will call PCI option ROMs or not. Of course the developers will try to set reasonable defaults. Of course the user might remove all plugin cards from his machine.. :-)
I'll assume that whether or not ROMs are assigned physical addresses, whether or not device may be left in a power saving state, and whether or not MSI may still be setup is all "undefined" - not guaranteed either way.
No device will end up with ROMs assigned, as the PCI standard forbids that. No device will be left in a power saving state. No idea about MSI, I never did that on AMD hw
There is little practical difference between reverse engineering in the traditional sense, and "reverse engineering" source code.
You are no doubt coming from a "legacy grade environment" or "commercial grade source code quality". Since we are trying to get more people into the project, we try to make the source code as easily understandable as possible, and we work on improving it.
In both cases it tells you nothing about what is guaranteed and what is coincedence (or what is incorrect behaviour), which values are valid for different fields, or what may or may not be implemented in future or past versions.
"guaranteed" is a difficult word. You can always fork for a specific application and thus guarantee that the environment will not change.
We might break compatibility since hardware is living fast and we rather want to do a good job on supporting hardware well.
Since we are a heterogenous group, it is hard to forsee what may be implemented in the future (heck, no homogenous group can do that either, except the illuminati) It's mostly "requirement driven development".
* Brendan Trotter btrotter@gmail.com [070612 16:51]:
Hehe - by the time you've finished answering all my questions you'd wish you'd written the documentation beforehand... ;-)
:-) Well, unless someone asks it's often not clear what documentation is required, so I think this is a good way of finding out.
Is "uint32_t" always little-endian or is it architecture specific (reverse engineering ruled out "always big-endian"), and would the "lb_header.signature" field always have an ASCII 'L' at the lowest address (or is that architecture specific too)?
It's probably arch specific. But we might want to change this.
if (head->header_bytes != sizeof(*head)) { fprintf(stderr, "Header bytes of %d are incorrect\n", head->header_bytes); continue; }From this can I assume that the size of the LinuxBIOS Table header will always be 24 bytes, and the LinuxBIOS Table header will never be extended in the future (or is this just a bug that makes "lbtdump" unable to handle forward/backward compatability)?
The header will never change. Dont think it should
The file "src/include/boot/linuxbios_tables.h" defines 3 types of memory ranges:
#define LB_MEM_RAM 1 /* Memory anyone can use */ #define LB_MEM_RESERVED 2 /* Don't use this memory region */ #define LB_MEM_TABLE 16 /* Ram configuration tables are kept in */
first two are types, third is the magic for the linuxbios table ("table type")
The file "util/lbtdump/lbtdump.c" doesn't agree:
switch(rec->map[i].type) { case 1: mem_type = "ram"; break;
case 3: mem_type = "acpi"; break; case 4: mem_type = "nvs"; break;
so these two are missing above. Maybe they were never used..
default: case 2: mem_type = "reserved"; break; }
Can I assume that the defines in "linuxbios_tables.h" are out-of-date, and that the memory type field is the same as specified by version 2 of the ACPI specification (but perhaps not version 3, as that defines a new "5 = faulty RAM" type)? If "linuxbios_tables.h" is out-of-date, is "16 = RAM configuration tables are stored in" still possible on older versions of LinuxBIOS (and how would I detect which version of LinuxBIOS is present, considering there's no version number in the LinuxBIOS Table header)?
LinuxBIOS table format has not changed from v1 to v2. It will be enhanced in v3. But nothing will break.
Would I also be correct to assume that a payload doesn't need to care about any of the "cmos_entries" and "cmos_enums", unless a BIOS setup/configuration utility is being built into the payload?
yes. A payload only cares for that information it needs, or cares about. It can skip the other table entries in the lbtable. This way old payloads and tools will always work with new tables. (But not promote their benefits of course)
But of course the bios setup utility does not need to be a payload. The most common case is that it is a linux userspace program.
Alternatively, could/would a payload search for strings to find CMOS values (e.g. search for the cmos_entry with the name string "power_on_after_fail" to see if additional hardware testing can be skipped, search for "baud_rate" to determine the serial port configuration, etc)?
Yes.
Of course - it's similar to the way SMI/DMI does strings, yet are the strings themselves guaranteed to be unique, or is it possible for one manufacturer to use an identifying string in the future that another (possibly out of business) manufacturer had used in the past?
No, the name strings are unique.
Unless in cases where, hmm.., AMD buys ATI and we port LinuxBIOS to a piece of ATI hardware but still might call it AMD something if AMD changes the name as well.
Motherboard manufacturer identification is one of the things that (IMHO) should be unique. The motherboard part number string should also be unique for that manufacturer ID. Otherwise it'd be difficult for payloads/utilities/OSs to use these IDs to avoid any hardware problems in specific motherboards.
Yes, absolutely right!
Yes, the emulation/qemu-i386 is per default not supporting SMP.
I'm becoming skeptical here. I can't find anything in LinuxBIOS source code that starts AP CPUs, but in "src/arch/i386/smp/mpspec.c" I did find:
check src/cpu/x86/lapic/lapic_cpu_init.c
In any case, I'm guessing I need to get LinuxBIOS to generate MP specification and ACPI tables to suit the number of CPUs being emulated by Bochs/Qemu.
You need to enable CONFIG_SMP and possibly fix some of the Qemu cpu specific code, as it removed most of the init code of real CPUs as it is not required. Alternatively, you could try to use real code, ie that of a P6.
Is this as simple as adding something like "option CPUs = ?" to the "targets/emulation/qemu-i386/Config.lb" file?
I'm guessing it's more involved than this, as I couldn't find an example in the "Config.lb" of other targets.
About half of the files have this:
default CONFIG_SMP=1 default CONFIG_MAX_CPUS=2
yes, either c or d. You choose.
You mean all LinuxBIOS developers are willing to make sure all PCI device ROMs either have physical addresses assigned or don't have physical addresses assigned based on my advice? Perhaps I've misunderstood... :-)
No, both is possible. You choose what you want. I choose what I want. ;-)
You can disable option rom execution and drop a lot of cruft if you know you wont have option roms anyways.
My (limited) reverse engineering has already shown that there's CMOS entries for a serial port (possibly intended for setting up a serial cable to communicate with a terminal emulator?).
Yes, basically all LinuxBIOS debugging happens over serial console with a null modem cable
Next question would be how to get that to work under Qemu and/or Bochs. I realise this may not be easy (I've had problems with Bochs before, as the VGA ROM seems to behave like an ISA card ROM and doesn't behave like a PCI card ROM - i.e. the ROM ignores settings in PCI configuration space)...
I fixed the ROM a while ago so it behaves right. If you do Qemu, I suggest that you look at LinuxBIOSv3 as it is where we are trying to go now. svn://linuxbios.org/repository/LinuxBIOSv3
Lots of stuff is still missing, but it is tested and working with the (patched) vga rom.
I am also figuring out about the hand-off between LB and the payload these days. Till now, I haven't found any specification about this issue. But I think the source code files explain it clearly.
For example, the "struct lb_memory *write_tables(void)" in the file of /src/arch/i386/boot/tables.c can answer some issues you mentioned.
There's some doxygen documentation of LinuxBIOS here:
http://qa.linuxbios.org/docs/doxygen/. It is sometimes useful to follow the code flow more easily.
I also care much about this subject you submitted.
-----邮件原件----- 发件人: linuxbios-bounces@linuxbios.org [mailto:linuxbios-bounces@linuxbios. org] 代表 Brendan Trotter 发送时间: 2007年6月11日 21:24 收件人: linuxbios@linuxbios.org 主题: [LinuxBIOS] Specifications?
Hi,
I'm wondering if there's a specification describing the state of the computer at the hand-off between LinuxBIOS and a payload.
So far I've got LinxBIOS booting a dummy "for(;;)" payload in Bochs, and I've found out there's a "LinuxBIOS Table" and that it can be found by searching for "LBIO" in the first 4 KB of RAM and in the area between 0xF000 and 0xFFFFF. I don't know if there's a checksum or any other validation, and have been unable to find anything describing what this table contains.
I assume there may also be ACPI, MP specification, PIRQ routing and/or (possibly) SMI/DMI tables, and I'm guessing that (if present) these tables can be searched for using the same methods as described in their corresponding specifications.
There's also other issues, for e.g: - will an 80x86 system always be in 32-bit protected mode for 32-bit ELF payloads, and is long mode possible for ELF64 payloads (and if so, how is paging setup)? - are any parameters passed on the stack to the payload's entry function? - are AP CPUs started (or waiting for a SIPI sequence)? - can the payload assume anything about the state of the PIC, I/O APIC/s and local APIC/s, PIT, RTC, ISA DMA controllers, serial ports, etc? - what state are PCI buses and devices left in (would a payload need to do full PCI bus enumeration, or..)? - is there anything in the LinuxBIOS table that could be used to determine if the computer is "headless" and which serial port or other device to use if it is? - can anything be assumed about AGP and the first video card (if the computer isn't headless)? - is there any form of "LinuxBIOS API" that can be used by the payload for generic low level disk access (floppy/ATA/ATAPI/SCSI)?
Thanks,
Brendan
-
Brendan Trotter -
Corey Osgood -
ebiederman＠lnxi.com -
Jordan Crouse -
Peter Stuge -
ron minnich -
Segher Boessenkool -
Stefan Reinauer -
Uwe Hermann -
xiongyi