[SeaBIOS] [Qemu-devel] SeaBIOS, FW_CFG_NUMA, and FW_CFG_MAX_CPUS

[Blue Swirl]

On Fri, Jul 20, 2012 at 8:00 PM, Eduardo Habkost <ehabkost at redhat.com> wrote:
> Hi,
>
> While working at the CPU index vs APIC ID changes, I stumbled upon
> another not-very-well-defined interface between SeaBIOS and QEMU, and I
> would like to clarify the semantics and constraints of some FW_CFG
> entries.
>
> First, the facts/assumptions:
>
> - There's no concept of "CPU index" or "CPU identifier" that SeaBIOS and
>   QEMU agree upon, except for the APIC ID. All SeaBIOS can really see
>   are the CPU APIC IDs, on boot or on CPU hotplug.
> - The APIC ID is already a perfectly good CPU identifier, that is
>   present on bare metal hardware too.
>   - Adding a new kind of "CPU identifier" in addition to the APIC ID
>     would just make things more complex.
>   - The only problem with APIC IDs is that they may not be contiguous.
>
> That said, I would like to clarify the meaning of:
>
> - FW_CFG_MAX_CPUS
>
> What are the basic semantics and expectations about FW_CFG_MAX_CPUS?

FYI: This originates from Sparc and PPC, it says how many SMP CPUs
there are in the system. There we don't have (at least now) any CPU
IDs and of course no APIC.

But I have no idea what x86 should use. As a general rule, what would
happen on a real machine should be emulated, but QEMU can also assist
BIOS (for example to skip some complex HW probes).

> Considering that the APIC IDs may not be contiguous, is it supposed to
> be:
>
> a) the maximum number of CPUs that will be ever online, doesn't matter
>    their APIC IDs, or
> b) a value so that every CPU has APIC ID < MAX_CPUS.
>
> A practical example: suppose we have a machine with 18 CPUs with the
> following APIC IDs: 0x00, 0x01, 0x02, 0x04, 0x05, 0x06, 0x08, 0x09,
> 0x0a, 0x10, 0x11, 0x12, 0x14, 0x15, 0x16, 0x18, 0x19, 0x1a.
>
> (That's the expected result for a machine with 2 sockets, 3 cores per
> socket, 3 threads per core.)
>
> In that case, should FW_CFG_MAX_CPUS be: a) 18, or b) 27 (0x1b)?
>
> If it should be 18, it will require additional work on SeaBIOS to make:
> - CPU hotplug work
> - SRAT/MADT/SSDT tables be built with Processor ID != APIC ID
> - SRAT/MADT/SSDT tables be kept stable if the system is hibernated and
>   resumed after a CPU is hot-plugged.
>
> (Probably in that case I would suggest introducing a FW_CFG_MAX_APIC_ID
> entry, so that SeaBIOS can still build the ACPI tables more easily).
>
>
> - FW_CFG_NUMA
>
> The first problem with FW_CFG_NUMA is that it depends on FW_CFG_MAX_CPUS
> (so it inherits the same questions above). The second is that
> FW_CFG_NUMA is a CPU-based table, but there's nothing SeaBIOS can use to
> know what CPUs FW_CFG_NUMA is refering to, except for the APIC IDs. So,
> should FW_CFG_NUMA be indexed by APIC IDs?
>
>
> - My proposal:
>
> My proposal is to try to keep things simple, and just use the following
> rule:
>
>  - Never have a CPU with APIC ID > FW_CFG_MAX_CPUS.
>
> This way:
> - The SeaBIOS ACPI code can be kept simple.
> - The current CPU hotplug interface can work as-is (up to 256 CPUs),
>   based on APIC IDs.
> - The current FW_CFG_NUMA interface can work as-is, indexed by APIC IDs.
> - The ACPI tables can be easily kept stable between hibernate and
>   resume, after CPU hotplug.
>
> This is the direction I am trying to go, and I am sending this just to
> make sure nobody is against it, and to not surprise anybody when I send
> a QEMU patch to make FW_CFG_MAX_CPUS be based on APIC IDs.
>
>
> My second proposal would be to introduce a FW_CFG_MAX_APIC_ID entry, so
> the SeaBIOS ACPI code can be kept simple.
>
> My third proposal would be to introduce a FW_CFG CPU Index => APIC ID
> table, but I really wouldn't like to introduce a new type of CPU
> identifier to be used between QEMU and SeaBIOS, when the APIC ID is a
> perfectly good unique CPU identifier that already exists in bare metal
> hardware.
>
> --
> Eduardo
>