On Tue, Mar 18, 2014 at 07:23:17PM -0400, Gabriel L. Somlo wrote:
At this point, can anyone with access to a real, physical, NUMA system dump the smbios tables with dmidecode and post them here? I think that would be very informative.
So I thrashed around a bit trying to find a real NUMA box, and found a Dell R410 whose BIOS claims to support NUMA by disabling the "Node Interleaving" option (haven't actually configured it to run NUMA, but based on what else I found, I no longer think I need to -- keep reading :)
So, to my surprise, I noticed this machine did NOT have any Type 20 tables in SMBIOS at all !!!
Then, after a more careful reading of the SMBIOS manual, I noticed that Type 20 was made OPTIONAL as v2.5 of the spec, cca. 2006 !!!
In conclusion, we *could* simply scan e820 for E820_RAM type regions and generate Type 19 tables for each one we find, and we no longer have to care at all about generating Type 20 nodes to link 19s to 17s, which basically makes the problem go away, as far as I am concerned.
I tested this (omitting Type 20) on OS X (10.6 through 10.9), XPsp3 and Win7, successfully, without either of them noticing any changes from previous executions where smbios tables were "different" (i.e., when they used to include T20).
At this point, though, I'd like some feedback before I shoot out another version (v5) of this patch set:
- Should I pretend we're generating v2.5 tables ?
- this means Type 4 now has extra fields (i.e. the number of cores, etc), which should be relatively easy to add, so I'd be OK doing that, if we agree on everything else. Heck, how about going all out and implementing v2.8 ? That would get us past the 2T limit in Types 16, 17, and 19 !
- SeaBIOS is still in charge of providing the smbios_entry_point structure, and it's unlikely we can reasonably expect it to bump the version to 2.5 (not that it seems to matter, if my tests are to be believed)
- on that note, SeaBIOS will still cheerfully generate Type 20 tables if nothing is supplied to it via fw_cfg from QEMU, and, again, I'm not sure how likely it is we can get it to stop doing that :)
- Does anyone still care ? ;)
Let me know what you all think!
Thanks, --Gabriel
On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
- SeaBIOS is still in charge of providing the smbios_entry_point structure, and it's unlikely we can reasonably expect it to bump the version to 2.5 (not that it seems to matter, if my tests are to be believed)
This is why ultimately we want to use the romfile_loader mechanism for smbios - that interface will allow qemu to generate both the smbios table and the smbios entry point.
-Kevin
On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
On Tue, Mar 18, 2014 at 07:23:17PM -0400, Gabriel L. Somlo wrote:
At this point, can anyone with access to a real, physical, NUMA system dump the smbios tables with dmidecode and post them here? I think that would be very informative.
So I thrashed around a bit trying to find a real NUMA box, and found a Dell R410 whose BIOS claims to support NUMA by disabling the "Node Interleaving" option
So, flipping that option on and off does indeed switch the system between NUMA and UMA modes. Running "numactl -H" in UMA mode gets us this output:
available: 1 nodes (0) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 node 0 size: 131059 MB node 0 free: 127898 MB node distances: node 0 0: 10
In NUMA mode, we get this:
available: 2 nodes (0-1) node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 node 0 size: 65536 MB node 0 free: 64047 MB node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 node 1 size: 65523 MB node 1 free: 63841 MB node distances: node 0 1 0: 10 20 1: 20 10
The "dmidecode" output stays unchanged between UMA and NUMA modes (showing only memory tables, types 16-19, and there is NO type 20):
Handle 0x1000, DMI type 16, 15 bytes Physical Memory Array Location: System Board Or Motherboard Use: System Memory Error Correction Type: Multi-bit ECC Maximum Capacity: 128 GB Error Information Handle: Not Provided Number Of Devices: 8
Handle 0x1100, DMI type 17, 28 bytes Memory Device Array Handle: 0x1000 Error Information Handle: Not Provided Total Width: 72 bits Data Width: 64 bits Size: 16384 MB Form Factor: DIMM Set: 1 Locator: DIMM_A1 Bank Locator: Not Specified Type: DDR3 Type Detail: Synchronous Registered (Buffered) Speed: 1066 MHz Manufacturer: 00CE00B380CE Serial Number: 44056D5E Asset Tag: 01105061 Part Number: M393B2K70CM0-YF8 Rank: 4
Handle 0x1101, DMI type 17, 28 bytes Handle 0x1102, DMI type 17, 28 bytes Handle 0x1103, DMI type 17, 28 bytes Handle 0x1109, DMI type 17, 28 bytes Handle 0x110A, DMI type 17, 28 bytes Handle 0x110B, DMI type 17, 28 bytes Handle 0x110C, DMI type 17, 28 bytes /* all of them similar to 0x1100 above [GLS] */
Handle 0x1300, DMI type 19, 15 bytes Memory Array Mapped Address Starting Address: 0x00000000000 Ending Address: 0x000BFFFFFFF Range Size: 3 GB Physical Array Handle: 0x1000 Partition Width: 2
Handle 0x1301, DMI type 19, 15 bytes Memory Array Mapped Address Starting Address: 0x00100000000 Ending Address: 0x0203FFFFFFF Range Size: 125 GB Physical Array Handle: 0x1000 Partition Width: 2
The output of "dmesg | grep -i e820" remains unchanged between UMA and NUMA modes:
e820: BIOS-provided physical RAM map: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable BIOS-e820: [mem 0x0000000000100000-0x00000000bf378fff] usable BIOS-e820: [mem 0x00000000bf379000-0x00000000bf38efff] reserved BIOS-e820: [mem 0x00000000bf38f000-0x00000000bf3cdfff] ACPI data BIOS-e820: [mem 0x00000000bf3ce000-0x00000000bfffffff] reserved BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved BIOS-e820: [mem 0x00000000fe000000-0x00000000ffffffff] reserved BIOS-e820: [mem 0x0000000100000000-0x000000203fffffff] usable e820: update [mem 0x00000000-0x0000ffff] usable ==> reserved e820: remove [mem 0x000a0000-0x000fffff] usable e820: last_pfn = 0x2040000 max_arch_pfn = 0x400000000 e820: update [mem 0xc0000000-0xffffffff] usable ==> reserved e820: last_pfn = 0xbf379 max_arch_pfn = 0x400000000 e820: [mem 0xc0000000-0xdfffffff] available for PCI devices PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820 e820: reserve RAM buffer [mem 0xbf379000-0xbfffffff]
So, even in NUMA mode, there still appear to be only two contiguous E820_RAM type entries in the "sanitized" e820 table (the regions marked "usable" appear to be adjacent). And the E820_RAM contiguous regions are not per-node or anything like that. Not that this is to be taken as the "One True Way", but still, it's a data point.
Anyhow, my original question/worry ("Oonce you create Type17 DIMMs from all the RAM, and Type19 regions for each E820_RAM type entry, how do you tie them together with Type20s ?") has been answered ("You just don't" :) )
Just figured I'd share this, maybe it can be useful to someone else besides me...
Cheers, --Gabriel
On Wed, Mar 26, 2014 at 06:36:10PM -0400, Kevin O'Connor wrote:
On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
- SeaBIOS is still in charge of providing the smbios_entry_point structure, and it's unlikely we can reasonably expect it to bump the version to 2.5 (not that it seems to matter, if my tests are to be believed)
This is why ultimately we want to use the romfile_loader mechanism for smbios - that interface will allow qemu to generate both the smbios table and the smbios entry point.
Ah, so romfile_loader is the "whole-blob" fw_cfg transfer method (so far in smbios.c we've been dealing with individual fields, and individual table types).
The only sticking point remaining would be who gets to generate the Type 0 (BIOS Information) table and when, which is something QEMU should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
I guess everyone's busy with QEMU 2.0, so I'll keep playing with this stuff on my own and bring it up again afterwards.
Obviously, more comments and feedback are welcome at any time, should there be any interest :)
Thanks, --Gabriel
On 03/31/14 22:18, Gabriel L. Somlo wrote:
On Wed, Mar 26, 2014 at 06:36:10PM -0400, Kevin O'Connor wrote:
On Wed, Mar 26, 2014 at 03:58:50PM -0400, Gabriel L. Somlo wrote:
- SeaBIOS is still in charge of providing the smbios_entry_point structure, and it's unlikely we can reasonably expect it to bump the version to 2.5 (not that it seems to matter, if my tests are to be believed)
This is why ultimately we want to use the romfile_loader mechanism for smbios - that interface will allow qemu to generate both the smbios table and the smbios entry point.
Ah, so romfile_loader is the "whole-blob" fw_cfg transfer method (so far in smbios.c we've been dealing with individual fields, and individual table types).
The only sticking point remaining would be who gets to generate the Type 0 (BIOS Information) table and when, which is something QEMU should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
I guess everyone's busy with QEMU 2.0, so I'll keep playing with this stuff on my own and bring it up again afterwards.
Obviously, more comments and feedback are welcome at any time, should there be any interest :)
Sorry I can follow this discussion only intermittently.
- I think OVMF would be fine with the Type 0 table as-is (it has a default table and adheres to field-level patches). Full tables for other types would be welcome.
- In edk2 (and on platforms that conform to the platform init (PI) spec), platform drivers install their smbios tables through EFI_SMBIOS_PROTOCOL. (Documented in Vol5 of the PI spec.)
In edk2, the "central" driver that produces this protocol (== provides the service) is "MdeModulePkg/Universal/SmbiosDxe".
The OVMF platform driver that installs the smbios tables through the protocol (== consumes the service) is "OvmfPkg/SmbiosPlatformDxe".
The set and the contents of the smbios tables are the jurisdiction of the platform driver (the service consumer) -- the driver in OvmfPkg.
The smbios entry point (including the smbios std version) is the jurisdiction of the service producer -- the driver in MdeModulePkg.
The platform driver can query the smbios std version from EFI_SMBIOS_PROTOCOL. The platform driver should also filter the tables it intends to install through the protocol so that the table types conform to the smbios std version supported by the service provider.
This is to say, OVMF can fetch tables via fw_cfg, and install them via the EFI_SMBIOS_PROTOCOL instance that "MdeModulePkg/Universal/SmbiosDxe" provides. However, OVMF can't change the SMBIOS version or other aspects of the smbios entry point. At best OVMF could filter out tables (grabbed from fw_cfg) that are above the std version that EFI_SMBIOS_PROTOCOL supports/advertises.
Currently OVMF does no such filtering (it would require hard-coding which table type is defined in which smbios version). It does require support for version 2.3, or it gives up.
In any case, the SMBIOS version currently supported by "MdeModulePkg/Universal/SmbiosDxe" is 2.7.1 (SVN r11690), which is quite permissive.
Summary: - type 0 should be OK as-is, other types are most welcome as full blobs, - OvmfPkg can't influence the smbios version advertised, it belongs to qanother (central) driver in edk2 that is built into OVMF.fd, - OvmfPkg currently doesn't filter fw_cfg table types against the smbios version (determined by that other, central driver in edk2), - however that version is reasonably high (2.7.1).
(Note: most of "OvmfPkg/SmbiosPlatformDxe" that's relevant for the above is not in upstream edk2. You can find the patches in http://www.kraxel.org/repos/manual/edk2/ and http://copr-be.cloud.fedoraproject.org/results/bonzini/ovmf/.)
Thanks Laszlo
On Tue, Apr 01, 2014 at 10:40:00AM +0200, Laszlo Ersek wrote:
On 03/31/14 22:18, Gabriel L. Somlo wrote:
The only sticking point remaining would be who gets to generate the Type 0 (BIOS Information) table and when, which is something QEMU should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
I guess everyone's busy with QEMU 2.0, so I'll keep playing with this stuff on my own and bring it up again afterwards.
Obviously, more comments and feedback are welcome at any time, should there be any interest :)
Sorry I can follow this discussion only intermittently.
- I think OVMF would be fine with the Type 0 table as-is (it has a
default table and adheres to field-level patches). Full tables for other types would be welcome.
When SeaBIOS generates the smbios table, it creates a hardcoded type 0 sub-table. (See SeaBIOS code src/fw/smbios.c:smbios_init_type_0.) If OVMF is okay with the same hardcodes, then I'd suggest QEMU create the table the same way SeaBIOS currently does.
-Kevin
On 04/01/14 16:39, Kevin O'Connor wrote:
On Tue, Apr 01, 2014 at 10:40:00AM +0200, Laszlo Ersek wrote:
On 03/31/14 22:18, Gabriel L. Somlo wrote:
The only sticking point remaining would be who gets to generate the Type 0 (BIOS Information) table and when, which is something QEMU should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
I guess everyone's busy with QEMU 2.0, so I'll keep playing with this stuff on my own and bring it up again afterwards.
Obviously, more comments and feedback are welcome at any time, should there be any interest :)
Sorry I can follow this discussion only intermittently.
- I think OVMF would be fine with the Type 0 table as-is (it has a
default table and adheres to field-level patches). Full tables for other types would be welcome.
When SeaBIOS generates the smbios table, it creates a hardcoded type 0 sub-table. (See SeaBIOS code src/fw/smbios.c:smbios_init_type_0.) If OVMF is okay with the same hardcodes, then I'd suggest QEMU create the table the same way SeaBIOS currently does.
In SeaBIOS,
if (!get_field(0, offsetof(struct smbios_type_0, bios_characteristics_extension_bytes), &p->bios_characteristics_extension_bytes)) { p->bios_characteristics_extension_bytes[0] = 0; /* Enable targeted content distribution. Needed for SVVP */ p->bios_characteristics_extension_bytes[1] = 4; }
bit 2 of the BIOS Characteristics Extension Byte 2 (7.1.2.2) is set, for "Enable Targeted Content Distribution".
In OVMF, the same byte has the following bits set:
Bit 3 -- UEFI Specification is supported. Bit 4 -- The SMBIOS table describes a virtual machine. (If this bit is not set, no inference can be made about the virtuality of the system.)
I have nothing against bit 2 (I didn't include it because I had no clue what it meant, but we can certainly set that bit down the road). Bit 3 would be wrong for SeaBIOS, and it would be wrong to leave clear for OVMF. Bit 4 would be wrong for SeaBIOS (as a static default), but is correct (and very nice, although not necessary) for OVMF.
Gerd posted a textual comparison before (with dmidecode):
http://thread.gmane.org/gmane.comp.emulators.qemu/260804/focus=261248
But I'm including a textual diff too (RHEL-7 host, RHEL-7 guests):
--- seabios-rhel7.txt 2014-04-01 17:39:49.148601405 +0200 +++ ovmf-rhel7.txt 2014-04-01 17:40:01.075658204 +0200 @@ -1,128 +1,34 @@ # dmidecode 2.12 -SMBIOS 2.4 present. -11 structures occupying 369 bytes. -Table at 0x000FDD80. +# SMBIOS entry point at 0x3fed8000 +SMBIOS 2.7 present. +3 structures occupying 185 bytes. +Table at 0x3FED7000.
Handle 0x0000, DMI type 0, 24 bytes BIOS Information
- Vendor: Bochs
- Version: Bochs
- Release Date: 01/01/2011
- Vendor: EFI Development Kit II / OVMF
- Version: 0.1
- Release Date: 06/03/2013 Address: 0xE8000 Runtime Size: 96 kB ROM Size: 64 kB Characteristics: BIOS characteristics not supported
Targeted content distribution is supported- BIOS Revision: 1.0
UEFI is supportedSystem is a virtual machine- BIOS Revision: 0.1
Of course, this is not to say that SeaBIOS and OVMF should continue patching the Type 0 table field-wise. If we can control the Table 0 fields on this level of detail on the qemu command line, and the parent of qemu (libvirtd or the user's shell) takes care to set them in accordance with the guest firmware, then the full blob suffices for Table 0 too.
(I'm retaining the rest of the diff below.)
Thanks Laszlo
-Handle 0x0100, DMI type 1, 27 bytes +Handle 0x0001, DMI type 1, 27 bytes System Information Manufacturer: Red Hat Product Name: KVM Version: RHEL 7.0.0 PC (i440FX + PIIX, 1996)
- Serial Number: Not Specified
- UUID: C0F384E5-6AEA-46E9-8340-E5AF2F0DCC9B
- Serial Number: n/a
- UUID: 8A2A6D47-99A6-486D-AFAE-A53741464C37 Wake-up Type: Power Switch
- SKU Number: Not Specified
- SKU Number: n/a Family: Red Hat Enterprise Linux
-Handle 0x0300, DMI type 3, 20 bytes -Chassis Information
- Manufacturer: Bochs
- Type: Other
- Lock: Not Present
- Version: Not Specified
- Serial Number: Not Specified
- Asset Tag: Not Specified
- Boot-up State: Safe
- Power Supply State: Safe
- Thermal State: Safe
- Security Status: Unknown
- OEM Information: 0x00000000
- Height: Unspecified
- Number Of Power Cords: Unspecified
-Handle 0x0401, DMI type 4, 32 bytes -Processor Information
- Socket Designation: CPU 1
- Type: Central Processor
- Family: Other
- Manufacturer: Bochs
- ID: 63 06 00 00 FD FB 8B 07
- Version: Not Specified
- Voltage: Unknown
- External Clock: Unknown
- Max Speed: 2000 MHz
- Current Speed: 2000 MHz
- Status: Populated, Enabled
- Upgrade: Other
- L1 Cache Handle: Not Provided
- L2 Cache Handle: Not Provided
- L3 Cache Handle: Not Provided
-Handle 0x0402, DMI type 4, 32 bytes -Processor Information
- Socket Designation: CPU 2
- Type: Central Processor
- Family: Other
- Manufacturer: Bochs
- ID: 63 06 00 00 FD FB 8B 07
- Version: Not Specified
- Voltage: Unknown
- External Clock: Unknown
- Max Speed: 2000 MHz
- Current Speed: 2000 MHz
- Status: Populated, Enabled
- Upgrade: Other
- L1 Cache Handle: Not Provided
- L2 Cache Handle: Not Provided
- L3 Cache Handle: Not Provided
-Handle 0x1000, DMI type 16, 15 bytes -Physical Memory Array
- Location: Other
- Use: System Memory
- Error Correction Type: Multi-bit ECC
- Maximum Capacity: 1 GB
- Error Information Handle: Not Provided
- Number Of Devices: 1
-Handle 0x1100, DMI type 17, 21 bytes -Memory Device
- Array Handle: 0x1000
- Error Information Handle: 0x0000
- Total Width: 64 bits
- Data Width: 64 bits
- Size: 1024 MB
- Form Factor: DIMM
- Set: None
- Locator: DIMM 0
- Bank Locator: Not Specified
- Type: RAM
- Type Detail: None
-Handle 0x1300, DMI type 19, 15 bytes -Memory Array Mapped Address
- Starting Address: 0x00000000000
- Ending Address: 0x0003FFFFFFF
- Range Size: 1 GB
- Physical Array Handle: 0x1000
- Partition Width: 1
-Handle 0x1400, DMI type 20, 19 bytes -Memory Device Mapped Address
- Starting Address: 0x00000000000
- Ending Address: 0x0003FFFFFFF
- Range Size: 1 GB
- Physical Device Handle: 0x1100
- Memory Array Mapped Address Handle: 0x1300
- Partition Row Position: 1
-Handle 0x2000, DMI type 32, 11 bytes -System Boot Information
- Status: No errors detected
-Handle 0x7F00, DMI type 127, 4 bytes +Handle 0xFEFF, DMI type 127, 4 bytes End Of Table
On Tue, Apr 01, 2014 at 05:47:09PM +0200, Laszlo Ersek wrote:
On 04/01/14 16:39, Kevin O'Connor wrote:
On Tue, Apr 01, 2014 at 10:40:00AM +0200, Laszlo Ersek wrote:
On 03/31/14 22:18, Gabriel L. Somlo wrote:
The only sticking point remaining would be who gets to generate the Type 0 (BIOS Information) table and when, which is something QEMU should arguably NOT be doing on behalf of SeaBIOS (or OVMF, or ...).
Sorry I can follow this discussion only intermittently.
- I think OVMF would be fine with the Type 0 table as-is (it has a
default table and adheres to field-level patches). Full tables for other types would be welcome.
When SeaBIOS generates the smbios table, it creates a hardcoded type 0 sub-table. (See SeaBIOS code src/fw/smbios.c:smbios_init_type_0.) If OVMF is okay with the same hardcodes, then I'd suggest QEMU create the table the same way SeaBIOS currently does.
OK, smbios_init_type_0() is called by the add_struct() macro in smbios_setup(), but not if fw_cfg already holds a QEMU-supplied type 0 table.
When it is called, smbios_init_type_0() will look for individual fields supplied via fw_cfg, and only use its own defaults otherwise.
By default, QEMU will not set any type 0 fields, nor generate a type 0 table, which means that each specific BIOS gets to supply its own default type 0 table.
My patch set supplies various full tables via fw_cfg, but by default it still doesn't built a type 0. That only happens if either a binary table or default fields for type 0 are provided on the command line (same as the current behavior, except I'm always sending a full table rather than field-by-field).
bit 2 of the BIOS Characteristics Extension Byte 2 (7.1.2.2) is set, for "Enable Targeted Content Distribution".
In OVMF, the same byte has the following bits set:
Bit 3 -- UEFI Specification is supported. Bit 4 -- The SMBIOS table describes a virtual machine. (If this bit is not set, no inference can be made about the virtuality of the system.)
I have nothing against bit 2 (I didn't include it because I had no clue what it meant, but we can certainly set that bit down the road). Bit 3 would be wrong for SeaBIOS, and it would be wrong to leave clear for OVMF. Bit 4 would be wrong for SeaBIOS (as a static default), but is correct (and very nice, although not necessary) for OVMF.
I can add an extra command line option for type 0 defaults (e.g. "char_ext" but we can pick a better name):
"-smbios type=0,vendor='foo',version='x.y',char_ext=4"
... and make the user responsible for supplying the correct value if/when they wish to override the defaults. I'll do that in the v5 patch set I'm working on right now :)
As an aside, I think in the end it doesn't matter much if we supply individual field defaults or full tables for *optional* types such as type 0. I just like to generate full tables across the board to keep reusing the same code, rather than leave the individual-field logic in just for this one table type...
From the conversation so far, it seems to me that:
- type 0 is best left to the BIOS (user overrides via command line at their own risk)
- therefore, the maximum granularity of QEMU-generated elements should be full tables of a given type, and not the full SMBIOS blob at once (other mechanisms to allow the BIOS to insert its own type 0 welcome, but so far this seems the most straightforward one).
- this means the smbios structure header has to be left up to the BIOS
- the BIOS is then responsible for setting the smbios spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
On that last point, at least Linux seems to be OK with individual type tables having a higher version than the structure header; i.e., dmidecode works fine when e.g. the structure header says 2.4 but the type 4 cpu record is 2.6. I'll test on Windows and OS X as well, and post my results here.
My one remaining question is: how do we get the BIOS to *not* generate a certain table type that's being left out on purpose by QEMU ?
I'm talking here of type 20, which is no longer required as of spec v2.5, and which would unnecessarily complicate things if/when more than two E820_RAM memory areas are present...
Thanks much, --Gabriel
On Tue, Apr 01, 2014 at 02:47:27PM -0400, Gabriel L. Somlo wrote:
On Tue, Apr 01, 2014 at 05:47:09PM +0200, Laszlo Ersek wrote:
bit 2 of the BIOS Characteristics Extension Byte 2 (7.1.2.2) is set, for "Enable Targeted Content Distribution".
In OVMF, the same byte has the following bits set:
Bit 3 -- UEFI Specification is supported. Bit 4 -- The SMBIOS table describes a virtual machine. (If this bit is not set, no inference can be made about the virtuality of the system.)
I have nothing against bit 2 (I didn't include it because I had no clue what it meant, but we can certainly set that bit down the road). Bit 3 would be wrong for SeaBIOS, and it would be wrong to leave clear for OVMF. Bit 4 would be wrong for SeaBIOS (as a static default), but is correct (and very nice, although not necessary) for OVMF.
I can add an extra command line option for type 0 defaults (e.g. "char_ext" but we can pick a better name):
"-smbios type=0,vendor='foo',version='x.y',char_ext=4"... and make the user responsible for supplying the correct value if/when they wish to override the defaults. I'll do that in the v5 patch set I'm working on right now :)
As an aside, I think in the end it doesn't matter much if we supply individual field defaults or full tables for *optional* types such as type 0. I just like to generate full tables across the board to keep reusing the same code, rather than leave the individual-field logic in just for this one table type...
From the conversation so far, it seems to me that:
type 0 is best left to the BIOS (user overrides via command line at their own risk)
therefore, the maximum granularity of QEMU-generated elements should be full tables of a given type, and not the full SMBIOS blob at once (other mechanisms to allow the BIOS to insert its own type 0 welcome, but so far this seems the most straightforward one).
I don't agree - I think ultimately we want QEMU to generate the full SMBIOS table and pass it to the firmware via the romfile_loader mechanism. The only thing that has been raised as an issue with this is one bit in the smbios table (UEFI support). For this one bit, I think QEMU can just put together a sane default and the firmware can patch up the one bit (either manually or via a new romfile_loader command).
this means the smbios structure header has to be left up to the BIOS
the BIOS is then responsible for setting the smbios spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
On that last point, at least Linux seems to be OK with individual type tables having a higher version than the structure header; i.e., dmidecode works fine when e.g. the structure header says 2.4 but the type 4 cpu record is 2.6. I'll test on Windows and OS X as well, and post my results here.
My one remaining question is: how do we get the BIOS to *not* generate a certain table type that's being left out on purpose by QEMU ?
I'm talking here of type 20, which is no longer required as of spec v2.5, and which would unnecessarily complicate things if/when more than two E820_RAM memory areas are present...
The above are good examples why I think QEMU should be the sole owner of the SMBIOS.
-Kevin
On Tue, Apr 01, 2014 at 04:28:32PM -0400, Kevin O'Connor wrote:
From the conversation so far, it seems to me that:
type 0 is best left to the BIOS (user overrides via command line at their own risk)
therefore, the maximum granularity of QEMU-generated elements should be full tables of a given type, and not the full SMBIOS blob at once (other mechanisms to allow the BIOS to insert its own type 0 welcome, but so far this seems the most straightforward one).
I don't agree - I think ultimately we want QEMU to generate the full SMBIOS table and pass it to the firmware via the romfile_loader mechanism. The only thing that has been raised as an issue with this is one bit in the smbios table (UEFI support). For this one bit, I think QEMU can just put together a sane default and the firmware can patch up the one bit (either manually or via a new romfile_loader command).
this means the smbios structure header has to be left up to the BIOS
the BIOS is then responsible for setting the smbios spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
On that last point, at least Linux seems to be OK with individual type tables having a higher version than the structure header; i.e., dmidecode works fine when e.g. the structure header says 2.4 but the type 4 cpu record is 2.6. I'll test on Windows and OS X as well, and post my results here.
My one remaining question is: how do we get the BIOS to *not* generate a certain table type that's being left out on purpose by QEMU ?
I'm talking here of type 20, which is no longer required as of spec v2.5, and which would unnecessarily complicate things if/when more than two E820_RAM memory areas are present...
The above are good examples why I think QEMU should be the sole owner of the SMBIOS.
Assuming all relevant QEMU maintainers are OK with the idea of creating a full SMBIOS blob (with e.g. type 0 defaulting to the relevant SeaBIOS values, override-able to fit some different bios, e.g. OVMF), would you take a patch to check for this blob in smbios_setup() (in SeaBIOS src/fw/smbios.c) ? Right now, it's either individual fields or table-at-a-time blobs only, AFAICT.
Assuming "yes", would OVMF accept a similar patch (unless it's already set up to receive such a blob, I forget whether that came up earlier in the thread) ?
Thanks, Gabriel
-- In Soviet Russia, problem divide-and-conquer YOU ;)
On 04/01/14 23:28, Gabriel L. Somlo wrote:
On Tue, Apr 01, 2014 at 04:28:32PM -0400, Kevin O'Connor wrote:
From the conversation so far, it seems to me that:
type 0 is best left to the BIOS (user overrides via command line at their own risk)
therefore, the maximum granularity of QEMU-generated elements should be full tables of a given type, and not the full SMBIOS blob at once (other mechanisms to allow the BIOS to insert its own type 0 welcome, but so far this seems the most straightforward one).
I don't agree - I think ultimately we want QEMU to generate the full SMBIOS table and pass it to the firmware via the romfile_loader mechanism. The only thing that has been raised as an issue with this is one bit in the smbios table (UEFI support). For this one bit, I think QEMU can just put together a sane default and the firmware can patch up the one bit (either manually or via a new romfile_loader command).
this means the smbios structure header has to be left up to the BIOS
the BIOS is then responsible for setting the smbios spec version (2.4 for SeaBIOS, 2.7.1 for OVMF).
On that last point, at least Linux seems to be OK with individual type tables having a higher version than the structure header; i.e., dmidecode works fine when e.g. the structure header says 2.4 but the type 4 cpu record is 2.6. I'll test on Windows and OS X as well, and post my results here.
My one remaining question is: how do we get the BIOS to *not* generate a certain table type that's being left out on purpose by QEMU ?
I'm talking here of type 20, which is no longer required as of spec v2.5, and which would unnecessarily complicate things if/when more than two E820_RAM memory areas are present...
The above are good examples why I think QEMU should be the sole owner of the SMBIOS.
Assuming all relevant QEMU maintainers are OK with the idea of creating a full SMBIOS blob (with e.g. type 0 defaulting to the relevant SeaBIOS values, override-able to fit some different bios, e.g. OVMF), would you take a patch to check for this blob in smbios_setup() (in SeaBIOS src/fw/smbios.c) ? Right now, it's either individual fields or table-at-a-time blobs only, AFAICT.
Assuming "yes", would OVMF accept a similar patch (unless it's already set up to receive such a blob, I forget whether that came up earlier in the thread) ?
Right now, OVMF can accept individual fields, or table-at-a-time blobs, via fw_cfg.
The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time (for which table-at-a-time blobs are a perfect match).
If qemu gives OVMF a complete, concatenated dump of all tables, I'll have to split that up into individual tables, and install those one by one.
qemu --[fw_cfg]--> OVMF platform code --[EFI_SMBIOS_PROTOCOL]--> edk2 "some" format: strictly per-table - field patch - per-table blob - complete dump?
I think that concatenating table-at-a-time blobs in SeaBIOS is easier than parsing & splitting a complete dump into tables in OVMF.
Kevin might disagree of course :)
Thanks Laszlo
On Tue, Apr 01, 2014 at 05:28:10PM -0400, Gabriel L. Somlo wrote:
Assuming all relevant QEMU maintainers are OK with the idea of creating a full SMBIOS blob (with e.g. type 0 defaulting to the relevant SeaBIOS values, override-able to fit some different bios, e.g. OVMF), would you take a patch to check for this blob in smbios_setup() (in SeaBIOS src/fw/smbios.c) ? Right now, it's either individual fields or table-at-a-time blobs only, AFAICT.
Yes.
-Kevin
On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
Right now, OVMF can accept individual fields, or table-at-a-time blobs, via fw_cfg.
The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time (for which table-at-a-time blobs are a perfect match).
I wasn't aware of this. The SMBIOS spec calls for all the sub-tables to be concatenanted into a single linear area of memory. Is there something in EFI or OVMF that is dictating otherwise? Can you provide a link so I can further understand? (I briefly checked through the UEFI v2.3.1 spec and nothing popped out at me.)
I think that concatenating table-at-a-time blobs in SeaBIOS is easier than parsing & splitting a complete dump into tables in OVMF.
I don't think it's very difficult either way. It would be nice, though, if there was just one owner for the smbios. The current setup where some data comes from QEMU and some from the firmware, along with mechanisms for providing defaults and overrides is way too complex in my opinion.
Thanks, -Kevin
On 04/02/14 00:00, Kevin O'Connor wrote:
On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
Right now, OVMF can accept individual fields, or table-at-a-time blobs, via fw_cfg.
The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time (for which table-at-a-time blobs are a perfect match).
I wasn't aware of this. The SMBIOS spec calls for all the sub-tables to be concatenanted into a single linear area of memory. Is there something in EFI or OVMF that is dictating otherwise? Can you provide a link so I can further understand? (I briefly checked through the UEFI v2.3.1 spec and nothing popped out at me.)
The "UEFI Specification" is not relevant here, the "UEFI Platform Initialization (PI) Specification" is.
You can download the PI spec at http://www.uefi.org/specs/access. The relevant section is (I have version 1.2.1):
VOLUME 5: Platform Initialization Specification Standards 6 SMBIOS Protocol
The function to call is EFI_SMBIOS_PROTOCOL.Add().
I think that concatenating table-at-a-time blobs in SeaBIOS is easier than parsing & splitting a complete dump into tables in OVMF.
I don't think it's very difficult either way. It would be nice, though, if there was just one owner for the smbios. The current setup where some data comes from QEMU and some from the firmware, along with mechanisms for providing defaults and overrides is way too complex in my opinion.
I certainly agree with the direction. I'm OK with the current table-at-a-time blobs (which should leave only the SMBIOS entry point to the firmware). I'm also OK with any new, comprehensive format that allows me, with reasonable parsing, to identify the individual tables in the big concat (and to throw away the passed down entry point).
I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess I could repeat the exercise if it's unavoidable... :)
Thanks Laszlo
On Wed, Apr 02, 2014 at 12:35:26AM +0200, Laszlo Ersek wrote:
On 04/02/14 00:00, Kevin O'Connor wrote:
On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
Right now, OVMF can accept individual fields, or table-at-a-time blobs, via fw_cfg.
The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time (for which table-at-a-time blobs are a perfect match).
I wasn't aware of this. The SMBIOS spec calls for all the sub-tables to be concatenanted into a single linear area of memory. Is there something in EFI or OVMF that is dictating otherwise? Can you provide a link so I can further understand? (I briefly checked through the UEFI v2.3.1 spec and nothing popped out at me.)
The "UEFI Specification" is not relevant here, the "UEFI Platform Initialization (PI) Specification" is.
You can download the PI spec at http://www.uefi.org/specs/access. The relevant section is (I have version 1.2.1):
VOLUME 5: Platform Initialization Specification Standards 6 SMBIOS Protocol
The function to call is EFI_SMBIOS_PROTOCOL.Add().
I think that concatenating table-at-a-time blobs in SeaBIOS is easier than parsing & splitting a complete dump into tables in OVMF.
I don't think it's very difficult either way. It would be nice, though, if there was just one owner for the smbios. The current setup where some data comes from QEMU and some from the firmware, along with mechanisms for providing defaults and overrides is way too complex in my opinion.
I certainly agree with the direction. I'm OK with the current table-at-a-time blobs (which should leave only the SMBIOS entry point to the firmware). I'm also OK with any new, comprehensive format that allows me, with reasonable parsing, to identify the individual tables in the big concat (and to throw away the passed down entry point).
I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess I could repeat the exercise if it's unavoidable... :)
Kevin, Laszlo,
What if I found a way to send an entry point structure via fw_cfg, as a signal to ${BIOS} to simply assemble all the table-at-a-time blobs, but without generating any of its own ?
I'm still working my way through whether *I* like the idea or not, but figured I'd throw it out there as a potential compromise ? :)
Thx, --Gabriel
On 04/02/14 14:38, Gabriel L. Somlo wrote:
On Wed, Apr 02, 2014 at 12:35:26AM +0200, Laszlo Ersek wrote:
On 04/02/14 00:00, Kevin O'Connor wrote:
On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
Right now, OVMF can accept individual fields, or table-at-a-time blobs, via fw_cfg.
The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time (for which table-at-a-time blobs are a perfect match).
I wasn't aware of this. The SMBIOS spec calls for all the sub-tables to be concatenanted into a single linear area of memory. Is there something in EFI or OVMF that is dictating otherwise? Can you provide a link so I can further understand? (I briefly checked through the UEFI v2.3.1 spec and nothing popped out at me.)
The "UEFI Specification" is not relevant here, the "UEFI Platform Initialization (PI) Specification" is.
You can download the PI spec at http://www.uefi.org/specs/access. The relevant section is (I have version 1.2.1):
VOLUME 5: Platform Initialization Specification Standards 6 SMBIOS Protocol
The function to call is EFI_SMBIOS_PROTOCOL.Add().
I think that concatenating table-at-a-time blobs in SeaBIOS is easier than parsing & splitting a complete dump into tables in OVMF.
I don't think it's very difficult either way. It would be nice, though, if there was just one owner for the smbios. The current setup where some data comes from QEMU and some from the firmware, along with mechanisms for providing defaults and overrides is way too complex in my opinion.
I certainly agree with the direction. I'm OK with the current table-at-a-time blobs (which should leave only the SMBIOS entry point to the firmware). I'm also OK with any new, comprehensive format that allows me, with reasonable parsing, to identify the individual tables in the big concat (and to throw away the passed down entry point).
I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess I could repeat the exercise if it's unavoidable... :)
Kevin, Laszlo,
What if I found a way to send an entry point structure via fw_cfg, as a signal to ${BIOS} to simply assemble all the table-at-a-time blobs, but without generating any of its own ?
I'm still working my way through whether *I* like the idea or not, but figured I'd throw it out there as a potential compromise ? :)
If you send the entry point structure in a new fw_cfg file, then (I believe) I could ignore it easily, and just go ahead with the table-at-a-time blobs. OVMF has defaults (fallbacks) only for Type 0 and Type 1 tables now, which you are (almost) guaranteed to send down anyway, so the above approach might work for OVMF without even changing the OVMF code. (Regarding individual fields, you'd simply not send such, if I understand correctly.)
Thanks Laszlo
Hi,
From the conversation so far, it seems to me that:
- type 0 is best left to the BIOS (user overrides via command line at their own risk)
I think it was a bad idea to allow overriding type0 fields in the first place. It also isn't used in practice. I don't think it is a big problem to loose that capability.
- therefore, the maximum granularity of QEMU-generated elements should be full tables of a given type, and not the full SMBIOS blob at once (other mechanisms to allow the BIOS to insert its own type 0 welcome, but so far this seems the most straightforward one).
Just an idea: Is the table ordering important? I don't think so. If qemu supplies a blob with all tables except type0, can the firmware simply append a type0 record to the blob supplied by qemu?
I don't agree - I think ultimately we want QEMU to generate the full SMBIOS table and pass it to the firmware via the romfile_loader mechanism.
I still think the firmware (and *only* the firmware) should supply the type0 table. I also think seabios should fill in something more sensible in there, such as "Vendor: SeaBIOS" and "Version: rel-1.7.4-0-g96917a8-...".
The only thing that has been raised as an issue with this is one bit in the smbios table (UEFI support).
IMO 'dmidecode -t0' should show what firmware you are running (seabios/ovmf/coreboot/whatever), not something made up by qemu.
cheers, Gerd
Hi,
If qemu gives OVMF a complete, concatenated dump of all tables, I'll have to split that up into individual tables, and install those one by one.
I feel like I should have a look at how coreboot handles this for an additional data point ...
cheers, Gerd
On Wed, Apr 02, 2014 at 05:07:05PM +0200, Gerd Hoffmann wrote:
Hi,
If qemu gives OVMF a complete, concatenated dump of all tables, I'll have to split that up into individual tables, and install those one by one.
I feel like I should have a look at how coreboot handles this for an additional data point ...
Sounds like maybe I also need to take a step back and try this stuff out (at least with Linux and Windows), for some first hand experience with how smbios tables are handled outside SeaBIOS... :)
Speaking of, I *thought* I had a vague idea of how all this stuff fits together, but it turns out I don't... There's
- OVMF http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF
- TianoCore http://www.coreboot.org/TianoCore
- coreboot http://www.coreboot.org/Download_coreboot
Apparently, TianoCore is a "coreboot payload", which in my mind is somewhat analogous to bootloader "stages" chaining off each other, but then what's OVMF (the only thing I actually tried, which only works on piix) ? Is it a packaged bundle of coreboot+tianocore ? or something else entirely ?
What if I want to send a patch against this whole "thing" to facilitate integration with the new smbios table generator in qemu ?
Which git repos do I need to have around, and how to stitch them together to obtain "the thing you use as an argument to -bios in lieu of SeaBIOS", when it comes time to test ? :)
I'm guessing this is a FAQ, so if there's one place that explains it all, please point me at it. Otherwise I'd be happy to write it up once I get my head wrapped around it :)
Thanks much, --Gabriel
PS. Coreboot says it supports q35 as well (http://www.coreboot.org/Supported_Motherboards) so if my guess that "coreboot + tianocore == ovmf" is true, then it must be tianocore that doesn't yet support q35 ?
On Wed, Apr 02, 2014 at 01:01:28PM -0400, Gabriel L. Somlo wrote:
Speaking of, I *thought* I had a vague idea of how all this stuff fits together, but it turns out I don't... There's
OVMF http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF
TianoCore http://www.coreboot.org/TianoCore
Apparently, TianoCore is a "coreboot payload", which in my mind is somewhat analogous to bootloader "stages" chaining off each other, but then what's OVMF (the only thing I actually tried, which only works on piix) ? Is it a packaged bundle of coreboot+tianocore ? or something else entirely ?
What if I want to send a patch against this whole "thing" to facilitate integration with the new smbios table generator in qemu ?
Which git repos do I need to have around, and how to stitch them together to obtain "the thing you use as an argument to -bios in lieu of SeaBIOS", when it comes time to test ? :)
I'm guessing this is a FAQ, so if there's one place that explains it all, please point me at it. Otherwise I'd be happy to write it up once I get my head wrapped around it :)
Nevermind, it seems it's all under git://git.code.sf.net/p/tianocore/edk2 :)
Although the nomenclature is still a bit fuzzy to me, the "thing to build" within edk2 appears to be OvmfPkg (ACTIVE_PLATFORM = OvmfPkg/OvmfPkgX64.dsc, with TOOL_CHAIN_TAG = GCC48, in Conf/target.txt, at least on F20).
I now have the latest and greatest "upstream" OVMF.fd, and I can use it (piix only) to boot Fedora 20 live x86_64. Guess I'm on my way :)
I get "missing smbios entry point" when I do a dmidecode, BTW. QEMU is sending type 1, 3, 4, etc. blobs in fw_cfg, not sure yet what I need to do to get OVMF to add the entry point... Maybe I should try without my smbios-patched qemu ?
Thanks, --Gabriel
On 04/03/14 03:57, Gabriel L. Somlo wrote:
On Wed, Apr 02, 2014 at 01:01:28PM -0400, Gabriel L. Somlo wrote:
Speaking of, I *thought* I had a vague idea of how all this stuff fits together, but it turns out I don't... There's
OVMF http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF
TianoCore http://www.coreboot.org/TianoCore
Apparently, TianoCore is a "coreboot payload", which in my mind is somewhat analogous to bootloader "stages" chaining off each other, but then what's OVMF (the only thing I actually tried, which only works on piix) ? Is it a packaged bundle of coreboot+tianocore ? or something else entirely ?
What if I want to send a patch against this whole "thing" to facilitate integration with the new smbios table generator in qemu ?
Which git repos do I need to have around, and how to stitch them together to obtain "the thing you use as an argument to -bios in lieu of SeaBIOS", when it comes time to test ? :)
Unless you want to do OVMF development yourself (ie. as long as you'd like to test only), you're best off with
(a) Gerd's packages:
(b) If you use a Fedora host, you can also try a (recently refreshed) Copr build, thanks to Paolo:
http://copr-be.cloud.fedoraproject.org/results/bonzini/ovmf/
Under (a) you find some short instructions, and a set of RPMs that is automatically rebuilt twice a day (IIRC).
Both (a) and (b) include the downstream-only SMBIOS patches.
I'm guessing this is a FAQ, so if there's one place that explains it all, please point me at it. Otherwise I'd be happy to write it up once I get my head wrapped around it :)
Nevermind, it seems it's all under git://git.code.sf.net/p/tianocore/edk2 :)
Although the nomenclature is still a bit fuzzy to me, the "thing to build" within edk2 appears to be OvmfPkg (ACTIVE_PLATFORM = OvmfPkg/OvmfPkgX64.dsc, with TOOL_CHAIN_TAG = GCC48, in Conf/target.txt, at least on F20).
I now have the latest and greatest "upstream" OVMF.fd, and I can use it (piix only) to boot Fedora 20 live x86_64. Guess I'm on my way :)
You can most certainly build OVMF yourself, yes; the OvmfPkg/build.sh script is a convenience wrapper. See also OvmfPkg/README.
I get "missing smbios entry point" when I do a dmidecode, BTW. QEMU is sending type 1, 3, 4, etc. blobs in fw_cfg, not sure yet what I need to do to get OVMF to add the entry point... Maybe I should try without my smbios-patched qemu ?
You don't see SMBIOS tables in the guest because you've built upstream OVMF. As I said before, upstream OvmfPkg doesn't include my SMBIOS patches. Both (a) and (b) do however.
One further note (also mentioned in OvmfPkg/README): don't use OVMF.fd with -bios; use it with -pflash (you need a Linux 3.7+ host for this). This will give your guest real runtime variable services -- non-volatile variable data will be written back to the OVMF.fd file.
Thanks Laszlo
On Thu, Apr 03, 2014 at 11:42:31AM +0200, Laszlo Ersek wrote:
Unless you want to do OVMF development yourself (ie. as long as you'd like to test only), you're best off with
(a) Gerd's packages:
(b) If you use a Fedora host, you can also try a (recently refreshed) Copr build, thanks to Paolo:
http://copr-be.cloud.fedoraproject.org/results/bonzini/ovmf/
Under (a) you find some short instructions, and a set of RPMs that is automatically rebuilt twice a day (IIRC).
Both (a) and (b) include the downstream-only SMBIOS patches.
[...]
You don't see SMBIOS tables in the guest because you've built upstream OVMF. As I said before, upstream OvmfPkg doesn't include my SMBIOS patches. Both (a) and (b) do however.
Oh, OK. Basically, I'm interested in looking at the sources (specifically, the SMBIOS code :) ) to try and develop my own sense of what might be the most agreeable way forward for my QEMU smbios patches... That way, I'd actually have a (hopefully somewhat intelligent) position of my own to support :) :)
For that, ideally, I'd like to clone a git repo (and pull once every few days to stay on top of things). Looks like the top-level upstream one doesn't have your smbios code, so would there be a "mid-stream" (for the lack of a better term) git repo you have that I can clone and hack against for the smbios stuff ?
I ask because yesterday was the first time I started really paying attention to edk2 and ovmf, and I don't yet have a sense of how fast the "state of the universe" would run away from me while my back is turned, i.e. how fast a point-in-time snapshot of e.g. Gerd or Paolo's RPMs would become stale... Extracting and patching source code from RPMs is OK once or twice, but I imagine will be much less fun than "git pull" once it gets repetitive :D
One further note (also mentioned in OvmfPkg/README): don't use OVMF.fd with -bios; use it with -pflash (you need a Linux 3.7+ host for this). This will give your guest real runtime variable services -- non-volatile variable data will be written back to the OVMF.fd file.
Good to know (-bios == read-only, -pflash == writable) :)
I'm really only in it for testing smbios interactions with qemu though, at least at this stage... :)
Thanks a ton, --Gabriel
On 04/03/14 15:32, Gabriel L. Somlo wrote:
On Thu, Apr 03, 2014 at 11:42:31AM +0200, Laszlo Ersek wrote:
You don't see SMBIOS tables in the guest because you've built upstream OVMF. As I said before, upstream OvmfPkg doesn't include my SMBIOS patches. Both (a) and (b) do however.
Oh, OK. Basically, I'm interested in looking at the sources (specifically, the SMBIOS code :) ) to try and develop my own sense of what might be the most agreeable way forward for my QEMU smbios patches... That way, I'd actually have a (hopefully somewhat intelligent) position of my own to support :) :)
For that, ideally, I'd like to clone a git repo (and pull once every few days to stay on top of things). Looks like the top-level upstream one doesn't have your smbios code, so would there be a "mid-stream" (for the lack of a better term) git repo you have that I can clone and hack against for the smbios stuff ?
https://github.com/lersek/edk2/commits/smbios
(I don't push to this repo normally, so you should continue fetching from tianocore/edk2.)
I've thrown in two convenience patches for more detailed debug messages. (See also the debug console hints in the README file.)
I ask because yesterday was the first time I started really paying attention to edk2 and ovmf, and I don't yet have a sense of how fast the "state of the universe" would run away from me while my back is turned, i.e. how fast a point-in-time snapshot of e.g. Gerd or Paolo's RPMs would become stale... Extracting and patching source code from RPMs is OK once or twice, but I imagine will be much less fun than "git pull" once it gets repetitive :D
Taking the patches from the SRPM and applying them manually would bring you to the same spot as fetching the patches from the above link (except the debug patches). Afterwards you should continue fetching form tianocore/edk2, either merging those commits into your (now new) branch, or rebasing your (now new) branch.
(In general I don't push my WIP publicly.)
One further note (also mentioned in OvmfPkg/README): don't use OVMF.fd with -bios; use it with -pflash (you need a Linux 3.7+ host for this). This will give your guest real runtime variable services -- non-volatile variable data will be written back to the OVMF.fd file.
Good to know (-bios == read-only, -pflash == writable) :)
I'm really only in it for testing smbios interactions with qemu though, at least at this stage... :)
Thanks a ton,
No; thank you for doing this.
Laszlo
On Wed, Apr 02, 2014 at 05:04:57PM +0200, Gerd Hoffmann wrote:
- therefore, the maximum granularity of QEMU-generated elements should be full tables of a given type, and not the full SMBIOS blob at once (other mechanisms to allow the BIOS to insert its own type 0 welcome, but so far this seems the most straightforward one).
Just an idea: Is the table ordering important? I don't think so. If qemu supplies a blob with all tables except type0, can the firmware simply append a type0 record to the blob supplied by qemu?
I don't see anything in the spec that would prohibit it, but I don't think it's done in practice. Given how many different OS parsers there are and their dubious quality, I think we'd want to be as simple as possible and continue to put table 0 at the start.
I don't agree - I think ultimately we want QEMU to generate the full SMBIOS table and pass it to the firmware via the romfile_loader mechanism.
I still think the firmware (and *only* the firmware) should supply the type0 table. I also think seabios should fill in something more sensible in there, such as "Vendor: SeaBIOS" and "Version: rel-1.7.4-0-g96917a8-...".
The only thing that has been raised as an issue with this is one bit in the smbios table (UEFI support).
IMO 'dmidecode -t0' should show what firmware you are running (seabios/ovmf/coreboot/whatever), not something made up by qemu.
Ultimately my preference would be to make a clean break from the existing smbios fw_cfg system as it is too complex, too confusing, and too inflexible. We could implement the above as you suggest, but I fear it would require keeping much of the complexity of the current fw_cfg interface. (It's also a new feature as SeaBIOS doesn't currently put any useful info in type 0.)
The already existing romfile_loader interface seems to provide a nearly complete solution to replace the smbios fw_cfg system. If there is really demand for more firmware info in the type 0 table, why don't we use romfile_loader, have qemu put together a dummy type 0 table, and then have seabios update it in place? Sure, that's ugly, but I'm pretty sure it would be less ugly then keeping the existing smbios fw_cfg system around.
-Kevin
On Fri, Apr 04, 2014 at 08:34:11PM -0400, Kevin O'Connor wrote:
IMO 'dmidecode -t0' should show what firmware you are running (seabios/ovmf/coreboot/whatever), not something made up by qemu.
Ultimately my preference would be to make a clean break from the existing smbios fw_cfg system as it is too complex, too confusing, and too inflexible. We could implement the above as you suggest, but I fear it would require keeping much of the complexity of the current fw_cfg interface. (It's also a new feature as SeaBIOS doesn't currently put any useful info in type 0.)
I like this idea too: older/current versions of qemu just do what they do, and work with SeaBIOS the way they always have. Future versions just send a completely new fw_cfg message type, which, when encountered by SeaBIOS, causes it to only add in (or edit) type 0, and leave everything else as is.
I don't see anything in the spec that would prohibit it, but I don't think it's done in practice. Given how many different OS parsers there are and their dubious quality, I think we'd want to be as simple as possible and continue to put table 0 at the start. ... The already existing romfile_loader interface seems to provide a nearly complete solution to replace the smbios fw_cfg system. If there is really demand for more firmware info in the type 0 table, why don't we use romfile_loader, have qemu put together a dummy type 0 table, and then have seabios update it in place? Sure, that's ugly, but I'm pretty sure it would be less ugly then keeping the existing smbios fw_cfg system around.
The only fly in this ointment may be that type 0 doesn't have a fixed length that could be edited in place, if you consider the various strings that get tacked on to the end of it. So you'd still have to slide the rest of the smbios payload left or right to shrink or enlarge the type 0 blob, depending on how you modify the various strings it contains...
--Gabriel
On Fri, Apr 04, 2014 at 09:15:14PM -0400, Gabriel L. Somlo wrote:
On Fri, Apr 04, 2014 at 08:34:11PM -0400, Kevin O'Connor wrote:
IMO 'dmidecode -t0' should show what firmware you are running (seabios/ovmf/coreboot/whatever), not something made up by qemu.
Ultimately my preference would be to make a clean break from the existing smbios fw_cfg system as it is too complex, too confusing, and too inflexible. We could implement the above as you suggest, but I fear it would require keeping much of the complexity of the current fw_cfg interface. (It's also a new feature as SeaBIOS doesn't currently put any useful info in type 0.)
I like this idea too: older/current versions of qemu just do what they do, and work with SeaBIOS the way they always have. Future versions just send a completely new fw_cfg message type, which, when encountered by SeaBIOS, causes it to only add in (or edit) type 0, and leave everything else as is.
Right.
I don't see anything in the spec that would prohibit it, but I don't think it's done in practice. Given how many different OS parsers there are and their dubious quality, I think we'd want to be as simple as possible and continue to put table 0 at the start. ... The already existing romfile_loader interface seems to provide a nearly complete solution to replace the smbios fw_cfg system. If there is really demand for more firmware info in the type 0 table, why don't we use romfile_loader, have qemu put together a dummy type 0 table, and then have seabios update it in place? Sure, that's ugly, but I'm pretty sure it would be less ugly then keeping the existing smbios fw_cfg system around.
The only fly in this ointment may be that type 0 doesn't have a fixed length that could be edited in place, if you consider the various strings that get tacked on to the end of it. So you'd still have to slide the rest of the smbios payload left or right to shrink or enlarge the type 0 blob, depending on how you modify the various strings it contains...
The dummy type 0 subtable that QEMU generates can have dummy space padded strings that the firmware can overwrite. Until recently, the max size smbios string was 64 bytes, so that size could be used. (As above, I admit that this is ugly, but the alternatives also seem ugly.) Another option would be to just leave the strings at a QEMU default as that's no different from what SeaBIOS does today.
-Kevin
On Wed, Apr 02, 2014 at 12:35:26AM +0200, Laszlo Ersek wrote:
On 04/02/14 00:00, Kevin O'Connor wrote:
On Tue, Apr 01, 2014 at 11:44:12PM +0200, Laszlo Ersek wrote:
Right now, OVMF can accept individual fields, or table-at-a-time blobs, via fw_cfg.
The internal interface (EFI_SMBIOS_PROTOCOL) expects one table at a time (for which table-at-a-time blobs are a perfect match).
I wasn't aware of this. The SMBIOS spec calls for all the sub-tables to be concatenanted into a single linear area of memory. Is there something in EFI or OVMF that is dictating otherwise? Can you provide a link so I can further understand? (I briefly checked through the UEFI v2.3.1 spec and nothing popped out at me.)
The "UEFI Specification" is not relevant here, the "UEFI Platform Initialization (PI) Specification" is.
You can download the PI spec at http://www.uefi.org/specs/access. The relevant section is (I have version 1.2.1):
Oh, those crazy UEFI guys!
The internal edk2 code appears to use these individual calls to ultimately build the concatenated smbios table from them. So, nothing special here - just standard UEFI over-design.
I think that concatenating table-at-a-time blobs in SeaBIOS is easier than parsing & splitting a complete dump into tables in OVMF.
I don't think it's very difficult either way. It would be nice, though, if there was just one owner for the smbios. The current setup where some data comes from QEMU and some from the firmware, along with mechanisms for providing defaults and overrides is way too complex in my opinion.
I certainly agree with the direction. I'm OK with the current table-at-a-time blobs (which should leave only the SMBIOS entry point to the firmware). I'm also OK with any new, comprehensive format that allows me, with reasonable parsing, to identify the individual tables in the big concat (and to throw away the passed down entry point).
I already wrote display_uuid() [src/fw/smbios.c] for SeaBIOS, so I guess I could repeat the exercise if it's unavoidable... :)
Makes sense.
Thanks. -Kevin
On Do, 2014-04-03 at 11:42 +0200, Laszlo Ersek wrote:
(a) Gerd's packages:
Under (a) you find some short instructions, and a set of RPMs that is automatically rebuilt twice a day (IIRC).
It polls the git repos once per hour and kicks a build on new commits. So usually it should not take more than two hours from commit to updated packages.
cheers, Gerd
On Do, 2014-04-03 at 09:32 -0400, Gabriel L. Somlo wrote:
For that, ideally, I'd like to clone a git repo (and pull once every few days to stay on top of things). Looks like the top-level upstream one doesn't have your smbios code, so would there be a "mid-stream" (for the lack of a better term) git repo you have that I can clone and hack against for the smbios stuff ?
http://www.kraxel.org/cgit/jenkins/edk2/tree/
spec file and patches. specfile is patched by the jenkins build scripts, so a manual rpm build will need some fiddeling, but I guess you are more interested in the patches ;)
cheers, Gerd
Hi,
The only fly in this ointment may be that type 0 doesn't have a fixed length that could be edited in place, if you consider the various strings that get tacked on to the end of it. So you'd still have to slide the rest of the smbios payload left or right to shrink or enlarge the type 0 blob, depending on how you modify the various strings it contains...
The dummy type 0 subtable that QEMU generates can have dummy space padded strings that the firmware can overwrite. Until recently, the max size smbios string was 64 bytes, so that size could be used. (As above, I admit that this is ugly, but the alternatives also seem ugly.) Another option would be to just leave the strings at a QEMU default as that's no different from what SeaBIOS does today.
I don't think we need to make it that complicated. smbios tables don't have any references, right? I mean any references which would need a fixup (such as table pointers in RSDP in acpi) and therefore would need the romfile_loader. The string references within a table are relative don't need special care.
Gabriel has code to generate all tables needed in qemu meanwhile, so I think we can simply have a blob in fw_cfg with all tables (except type0). firmware generates type0 table like it does today, then simply appends the fw_cfg blob as-is, then appends a end-of-tables marker. Done.
OVMF probably would have to parse the blob, split it into tables, then install them one by one. But I suspect that will be less code than dealing with the complex smbios fw_cfg interface we have today ...
cheers, Gerd
On Mon, Apr 07, 2014 at 09:09:56AM +0200, Gerd Hoffmann wrote:
The only fly in this ointment may be that type 0 doesn't have a fixed length that could be edited in place, if you consider the various strings that get tacked on to the end of it. So you'd still have to slide the rest of the smbios payload left or right to shrink or enlarge the type 0 blob, depending on how you modify the various strings it contains...
The dummy type 0 subtable that QEMU generates can have dummy space padded strings that the firmware can overwrite. Until recently, the max size smbios string was 64 bytes, so that size could be used. (As above, I admit that this is ugly, but the alternatives also seem ugly.) Another option would be to just leave the strings at a QEMU default as that's no different from what SeaBIOS does today.
I don't think we need to make it that complicated. smbios tables don't have any references, right? I mean any references which would need a fixup (such as table pointers in RSDP in acpi) and therefore would need the romfile_loader. The string references within a table are relative don't need special care.
The smbios anchor table needs to have the address of the main smbios table. It would be preferable to get the anchor table from qemu as the anchor table has the smbios version info.
But, anchor table aside, you are correct.
Gabriel has code to generate all tables needed in qemu meanwhile, so I think we can simply have a blob in fw_cfg with all tables (except type0). firmware generates type0 table like it does today, then simply appends the fw_cfg blob as-is, then appends a end-of-tables marker. Done.
OVMF probably would have to parse the blob, split it into tables, then install them one by one. But I suspect that will be less code than dealing with the complex smbios fw_cfg interface we have today ...
How about having QEMU produce the smbios table with a dummy type0 table and then both seabios and ovmf can replace the type0 table if desired. After all, if OVMF is splitting the blob into tables, it can just as easily replace type0 as append it.
This way, the QEMU output is technically complete. And if someone wishes to code up SeaBIOS to do the type0 replace (I'm not convinced it's even necessary) then at least that SeaBIOS code could be used on coreboot as well.
-Kevin
On 04/07/14 16:14, Kevin O'Connor wrote:
On Mon, Apr 07, 2014 at 09:09:56AM +0200, Gerd Hoffmann wrote:
The only fly in this ointment may be that type 0 doesn't have a fixed length that could be edited in place, if you consider the various strings that get tacked on to the end of it. So you'd still have to slide the rest of the smbios payload left or right to shrink or enlarge the type 0 blob, depending on how you modify the various strings it contains...
The dummy type 0 subtable that QEMU generates can have dummy space padded strings that the firmware can overwrite. Until recently, the max size smbios string was 64 bytes, so that size could be used. (As above, I admit that this is ugly, but the alternatives also seem ugly.) Another option would be to just leave the strings at a QEMU default as that's no different from what SeaBIOS does today.
I don't think we need to make it that complicated. smbios tables don't have any references, right? I mean any references which would need a fixup (such as table pointers in RSDP in acpi) and therefore would need the romfile_loader. The string references within a table are relative don't need special care.
The smbios anchor table needs to have the address of the main smbios table. It would be preferable to get the anchor table from qemu as the anchor table has the smbios version info.
But, anchor table aside, you are correct.
Gabriel has code to generate all tables needed in qemu meanwhile, so I think we can simply have a blob in fw_cfg with all tables (except type0). firmware generates type0 table like it does today, then simply appends the fw_cfg blob as-is, then appends a end-of-tables marker. Done.
OVMF probably would have to parse the blob, split it into tables, then install them one by one. But I suspect that will be less code than dealing with the complex smbios fw_cfg interface we have today ...
How about having QEMU produce the smbios table with a dummy type0 table and then both seabios and ovmf can replace the type0 table if desired. After all, if OVMF is splitting the blob into tables, it can just as easily replace type0 as append it.
This way, the QEMU output is technically complete. And if someone wishes to code up SeaBIOS to do the type0 replace (I'm not convinced it's even necessary) then at least that SeaBIOS code could be used on coreboot as well.
Works for me.
Thanks Laszlo
On Mon, Apr 07, 2014 at 10:14:36AM -0400, Kevin O'Connor wrote:
How about having QEMU produce the smbios table with a dummy type0 table and then both seabios and ovmf can replace the type0 table if desired. After all, if OVMF is splitting the blob into tables, it can just as easily replace type0 as append it.
This way, the QEMU output is technically complete. And if someone wishes to code up SeaBIOS to do the type0 replace (I'm not convinced it's even necessary) then at least that SeaBIOS code could be used on coreboot as well.
OK, so as far as I'm concerned, it's down to two alternatives:
1. I create a full blob, starting with the anchor/entrypoint, and followed by a (dummy) type 0, type 1, etc, etc. all the way to the type 127 end marker. Pass that in via fw_cfg, and each BIOS is then responsible for editing type 0, overwriting it with appropriate values.
I like this very much :) except for a serious discomfort with the idea of imposing an additional "convention" on the size (and choice of) strings included with the type0 table, beyond the smbios spec.
2. I create a third fw_cfg smbios "type" for the entry point structure.
The BIOS (e.g. smbios_setup() in SeaBIOS) checks for the presence of this blob *first*. If present, it simply grabs all table blobs from fw_cfg and assembles them (e.g. via get_external()). Create its own type 0 if not already included in fw_cfg, and recompute the checksum for the entry point.
If the entry point blob is not found, smbios_setup() proceeds with its current logic (looking for table or field blobs in fw_cfg, and creating its own entry point structure).
Now, 2 would be uglier, hands down, but has the advantage of not adding arbitrary restrictions on what the type0 structure can look like.
If we go with 1 (all I need is you all to say "go for it, we're OK with arbitrary restrictions on type0" :) ), I'll add prominent comments in the qemu smbios source about what the restrictions are.
We have vendor, version, and date, currently. Date can be "YYYY-MM-DD" or somesuch (for a total strlen of 10, not including \0).
Any idea what the longest "vendor" and "version" string might look like ?
What do we do if we have *shorter* strings than that, fill with spaces to keep the hardcoded strlen intact across the qemu/bios demarc ?
Thanks, --Gabriel
On Mon, Apr 07, 2014 at 10:49:54AM -0400, Gabriel L. Somlo wrote:
On Mon, Apr 07, 2014 at 10:14:36AM -0400, Kevin O'Connor wrote:
How about having QEMU produce the smbios table with a dummy type0 table and then both seabios and ovmf can replace the type0 table if desired. After all, if OVMF is splitting the blob into tables, it can just as easily replace type0 as append it.
This way, the QEMU output is technically complete. And if someone wishes to code up SeaBIOS to do the type0 replace (I'm not convinced it's even necessary) then at least that SeaBIOS code could be used on coreboot as well.
OK, so as far as I'm concerned, it's down to two alternatives:
- I create a full blob, starting with the anchor/entrypoint, and
followed by a (dummy) type 0, type 1, etc, etc. all the way to the type 127 end marker. Pass that in via fw_cfg, and each BIOS is then responsible for editing type 0, overwriting it with appropriate values.
I like this very much :) except for a serious discomfort with the idea of imposing an additional "convention" on the size (and choice of) strings included with the type0 table, beyond the smbios spec.
I agree that was too ugly. What I'm proposing now is that QEMU produce a valid type0 table (with strings populated, but no special padding) along with all the other tables (up to and including type 127).
Then SeaBIOS and OVMF can either pass this on exactly as is, or they can modify the table to replace type0.
This is a few more lines of code for SeaBIOS (to replace type0 instead of just patching it), but it has the advantage that QEMU developers know they must produce a valid smbios table according to the specs and the firmware developers know they will get a valid smbios table (according to the specs) and know they must ultimately produce a valid smbios table.
So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file with the valid anchor table (the address pointer can be just set to zero), and an smbios table file with the complete set of smbios tables formatted according to the smbios spec. SeaBIOS can then use the existence of one of these new files to determine if it should deploy (and optionally modify) them or if it should use the old smbios generation code.
-Kevin
On Mon, Apr 07, 2014 at 11:23:44AM -0400, Kevin O'Connor wrote:
So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file with the valid anchor table (the address pointer can be just set to zero), and an smbios table file with the complete set of smbios tables formatted according to the smbios spec. SeaBIOS can then use the existence of one of these new files to determine if it should deploy (and optionally modify) them or if it should use the old smbios generation code.
Oh, OK. Right now we have (in qemu):
#define SMBIOS_FIELD_ENTRY 0 #define SMBIOS_TABLE_ENTRY 1
I will be adding (actually, migrating to):
#define SMBIOS_ANCHOR_ENTRY 2 /* for the smbios entry point table */ #define SMBIOS_FULLTABLE_ENTRY 3 /* for the blob containing all types */
The cool thing here is that, along with the payload for each type, I can create a wrapper structure, like there already exists for fields and individual table types:
struct smbios_field { struct smbios_header header; uint8_t type; uint16_t offset; uint8_t data[]; } QEMU_PACKED;
struct smbios_table { struct smbios_header header; uint8_t data[]; } QEMU_PACKED;
I can add such a structure for the anchor/entrypoint table and for the full blob-of-tables payload, in which I can tell you how big type 0 is, so the BIOS (SeaBIOS/TianoCore) side surgery can be made that much easier...
Thanks, --Gabriel
On Mon, Apr 07, 2014 at 02:05:21PM -0400, Gabriel L. Somlo wrote:
On Mon, Apr 07, 2014 at 11:23:44AM -0400, Kevin O'Connor wrote:
So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file with the valid anchor table (the address pointer can be just set to zero), and an smbios table file with the complete set of smbios tables formatted according to the smbios spec. SeaBIOS can then use the existence of one of these new files to determine if it should deploy (and optionally modify) them or if it should use the old smbios generation code.
Oh, OK. Right now we have (in qemu):
#define SMBIOS_FIELD_ENTRY 0 #define SMBIOS_TABLE_ENTRY 1
I will be adding (actually, migrating to):
#define SMBIOS_ANCHOR_ENTRY 2 /* for the smbios entry point table */ #define SMBIOS_FULLTABLE_ENTRY 3 /* for the blob containing all types */
No - don't do that. Lets leave the existing smbios fw_cfg entry (0x8001) unchanged. Instead, introduce two new fw_cfg files using the fw_cfg_add_file() interface (eg, "etc/smbios/smbios-anchor" and "etc/smbios/smbios-tables").
[...]
I can add such a structure for the anchor/entrypoint table and for the full blob-of-tables payload, in which I can tell you how big type 0 is, so the BIOS (SeaBIOS/TianoCore) side surgery can be made that much easier...
It's trivial for the firmware to calculate this on its own, so I recommend just putting the anchor table and main tables unmodified in their respective fw_cfg files.
-Kevin
On Mon, Apr 07, 2014 at 02:57:08PM -0400, Kevin O'Connor wrote:
On Mon, Apr 07, 2014 at 02:05:21PM -0400, Gabriel L. Somlo wrote:
On Mon, Apr 07, 2014 at 11:23:44AM -0400, Kevin O'Connor wrote:
So, I'm suggesting QEMU produce two new fw_cfg files: an anchor file with the valid anchor table (the address pointer can be just set to zero), and an smbios table file with the complete set of smbios tables formatted according to the smbios spec. SeaBIOS can then use the existence of one of these new files to determine if it should deploy (and optionally modify) them or if it should use the old smbios generation code.
Oh, OK. Right now we have (in qemu):
#define SMBIOS_FIELD_ENTRY 0 #define SMBIOS_TABLE_ENTRY 1
I will be adding (actually, migrating to):
#define SMBIOS_ANCHOR_ENTRY 2 /* for the smbios entry point table */ #define SMBIOS_FULLTABLE_ENTRY 3 /* for the blob containing all types */
No - don't do that. Lets leave the existing smbios fw_cfg entry (0x8001) unchanged. Instead, introduce two new fw_cfg files using the fw_cfg_add_file() interface (eg, "etc/smbios/smbios-anchor" and "etc/smbios/smbios-tables").
OK.
I can add such a structure for the anchor/entrypoint table and for the full blob-of-tables payload, in which I can tell you how big type 0 is, so the BIOS (SeaBIOS/TianoCore) side surgery can be made that much easier...
It's trivial for the firmware to calculate this on its own, so I recommend just putting the anchor table and main tables unmodified in their respective fw_cfg files.
Not sure why it never occurred to me before (lack of caffeine, or various day-job related distractions :) ) but if the QEMU default dummy type 0 table is just that -- a dummy table -- there's *nothing* preventing me from leaving all (three) of its strings *empty* :)
Then we'll know *exactly* what the size of type 0 is, when it's time to surgically transplant a new one within the BIOS...
I remember neither Windows, Linux (F20 live) or OS X objecting to a type 0 smbios table with all-undefined strings, during some previous tests.
I'll try to send out an updated patch set later in the week.
Thanks again, --Gabriel
-
Gabriel L. Somlo -
Gerd Hoffmann -
Kevin O'Connor -
Laszlo Ersek