When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
* 0 1 2 3 - first logical CPU on each core of socket 0 * 4 5 6 7 - first logical CPU on each core of socket 1 * 8 9 10 11 - second logical CPU on each core of socket 0 * 12 13 14 15 - second logical CPU on each core of socket 1
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de --- src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c @@ -158,13 +158,53 @@ build_madt(void) memset(madt, 0, madt_size); madt->local_apic_address = cpu_to_le32(BUILD_APIC_ADDR); madt->flags = cpu_to_le32(1); + u32 eax, ebx, ecx, edx; + cpuid(1, &eax, &ebx, &ecx, &edx); + int htt = 0; + if (!eax) { + dprintf(1, "No cpuid\n"); + } else { + htt = !!(edx & (1 << 28)); + /* assuming at most 2 logical CPUs per core */ + if (htt) + dprintf(4, "Found HTT feature\n"); + } struct madt_processor_apic *apic = (void*)&madt[1]; int i; for (i=0; i<MaxCountCPUs; i++) { apic->type = APIC_PROCESSOR; apic->length = sizeof(*apic); apic->processor_id = i; - apic->local_apic_id = i; + if (htt) { + /* + * In a system with 2 packages, 4 cores per package, and 2 CPUs per + * core we want to achieve the following coupling: + * CPU# APIC ID + * 0 0 [first socket, first logical CPUs] + * 1 2 + * 2 4 + * 3 6 + * 4 8 [second socket, first logical CPUs] + * 5 10 + * 6 12 + * 7 14 + * 8 1 [first socket, second logical CPUs] + * 9 3 + * 10 5 + * 11 7 + * 12 9 [second socket, second logical CPUs] + * 13 11 + * 14 13 + * 15 15 + * where the least-significant bit of the APIC ID identifies the + * first (0) or second (1) logical CPU in a core. + */ + apic->local_apic_id = i * 2; + if (apic->local_apic_id >= MaxCountCPUs) + apic->local_apic_id -= MaxCountCPUs - 1; + } else { + apic->local_apic_id = i; + } if (apic_id_is_present(apic->local_apic_id)) apic->flags = cpu_to_le32(1); else
A commodity BIOS does not punch a memory hole between 640 KiB and 1 MiB in SRAT. Make SeaBIOS behave in the same way.
Signed-off-by: Filippo Sironi sironi@amazon.de --- src/fw/acpi.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 864c247..32889df 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c @@ -520,7 +520,7 @@ build_srat(void) struct system_resource_affinity_table *srat; int srat_size = sizeof(*srat) + sizeof(struct srat_processor_affinity) * max_cpu + - sizeof(struct srat_memory_affinity) * (nb_numa_nodes + 2); + sizeof(struct srat_memory_affinity) * (nb_numa_nodes + 1);
srat = malloc_high(srat_size); if (!srat) { @@ -557,22 +557,16 @@ build_srat(void) int slots = 0; u64 mem_len, mem_base, next_base = 0;
- acpi_build_srat_memory(numamem, 0, 640*1024, 0, 1); - next_base = 1024 * 1024; - numamem++; - slots++; - for (i = 1; i < nb_numa_nodes + 1; ++i) { + for (i = 0; i < nb_numa_nodes; ++i) { mem_base = next_base; mem_len = *numadata++; - if (i == 1) - mem_len -= 1024 * 1024; next_base = mem_base + mem_len;
/* Cut out the PCI hole */ if (mem_base <= RamSize && next_base > RamSize) { mem_len -= next_base - RamSize; if (mem_len > 0) { - acpi_build_srat_memory(numamem, mem_base, mem_len, i-1, 1); + acpi_build_srat_memory(numamem, mem_base, mem_len, i, 1); numamem++; slots++; } @@ -580,11 +574,11 @@ build_srat(void) mem_len = next_base - RamSize; next_base += (1ULL << 32) - RamSize; } - acpi_build_srat_memory(numamem, mem_base, mem_len, i-1, 1); + acpi_build_srat_memory(numamem, mem_base, mem_len, i, 1); numamem++; slots++; } - for (; slots < nb_numa_nodes + 2; slots++) { + for (; slots < nb_numa_nodes + 1; slots++) { acpi_build_srat_memory(numamem, 0, 0, 0, 0); numamem++; }
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
-Kevin
On 01/27/17 15:27, Kevin O'Connor wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
QEMU constructs minimal / condensed APIC IDs so that they precisely correspond to the socket / core / thread hierarchy requested on the command line. If I remember correctly, the algorithm was implemented directly from Intel's MP spec. If the APIC IDs in the guest don't look as desired, I think the -smp option might have to be tweaked.
I also seem to recall that this spec-adherent behavior could be machine type dependent (i.e., unavailable before a specific machine type). The machtypes not conforming to it should be ancient IMO.
CC'ing Eduardo and Igor.
Thanks Laszlo
On 01/27/17 15:57, Laszlo Ersek wrote:
On 01/27/17 15:27, Kevin O'Connor wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
QEMU constructs minimal / condensed APIC IDs so that they precisely correspond to the socket / core / thread hierarchy requested on the command line. If I remember correctly, the algorithm was implemented directly from Intel's MP spec. If the APIC IDs in the guest don't look as desired, I think the -smp option might have to be tweaked.
... by that I attempted (and failed) to say that the error is likely on the concrete QEMU command line, not in the code.
I also seem to recall that this spec-adherent behavior could be machine type dependent (i.e., unavailable before a specific machine type). The machtypes not conforming to it should be ancient IMO.
CC'ing Eduardo and Igor.
Thanks Laszlo
On Fri, 27 Jan 2017 15:57:19 +0100 Laszlo Ersek lersek@redhat.com wrote:
On 01/27/17 15:27, Kevin O'Connor wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
ACPI spec says " platform firmware should list the first logical processor of each of the individual multi-threaded processors in the MADT before listing any of the second logical processors. "
socket order is not mandated there and for example I have a system where apic id's re listed as following: 0x0 - first logical CPU on each core 0 of socket 0 0x10 - first logical CPU on each core 0 of socket 1 0x2 - first logical CPU on each core 1 of socket 0 0x12 - first logical CPU on each core 1 of socket 1 ...
the main point is that first threads are listed first (which socket/core order is not important except of boot CPU) and only after that goes list of secondary threads.
QEMU currently doesn't conform to above guideline as MADT is 1.0 revision doesn't have such requirement. However patches are welcome to make it adhering to the later ACPI specs. See build_madt() function to see how it's generated now.
Also does patch fix any particular issue/problem or just making MADT conform to spec?
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
QEMU constructs minimal / condensed APIC IDs so that they precisely correspond to the socket / core / thread hierarchy requested on the command line. If I remember correctly, the algorithm was implemented directly from Intel's MP spec. If the APIC IDs in the guest don't look as desired, I think the -smp option might have to be tweaked.
I also seem to recall that this spec-adherent behavior could be machine type dependent (i.e., unavailable before a specific machine type). The machtypes not conforming to it should be ancient IMO.
CC'ing Eduardo and Igor.
Thanks Laszlo
On 01/30/17 13:39, Igor Mammedov wrote:
On Fri, 27 Jan 2017 15:57:19 +0100 Laszlo Ersek lersek@redhat.com wrote:
On 01/27/17 15:27, Kevin O'Connor wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
ACPI spec says " platform firmware should list the first logical processor of each of the individual multi-threaded processors in the MADT before listing any of the second logical processors. "
socket order is not mandated there and for example I have a system where apic id's re listed as following: 0x0 - first logical CPU on each core 0 of socket 0 0x10 - first logical CPU on each core 0 of socket 1 0x2 - first logical CPU on each core 1 of socket 0 0x12 - first logical CPU on each core 1 of socket 1 ...
the main point is that first threads are listed first (which socket/core order is not important except of boot CPU) and only after that goes list of secondary threads.
QEMU currently doesn't conform to above guideline as MADT is 1.0 revision doesn't have such requirement. However patches are welcome to make it adhering to the later ACPI specs.
Ah I didn't realize that. Thank you for the analysis!
See build_madt() function to see how it's generated now.
Hopefully Filippo can look into that.
Thanks! Laszlo
Also does patch fix any particular issue/problem or just making MADT conform to spec?
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
QEMU constructs minimal / condensed APIC IDs so that they precisely correspond to the socket / core / thread hierarchy requested on the command line. If I remember correctly, the algorithm was implemented directly from Intel's MP spec. If the APIC IDs in the guest don't look as desired, I think the -smp option might have to be tweaked.
I also seem to recall that this spec-adherent behavior could be machine type dependent (i.e., unavailable before a specific machine type). The machtypes not conforming to it should be ancient IMO.
CC'ing Eduardo and Igor.
Thanks Laszlo
On 30 Jan 2017, at 04:39, Igor Mammedov imammedo@redhat.com wrote:
On Fri, 27 Jan 2017 15:57:19 +0100 Laszlo Ersek lersek@redhat.com wrote:
On 01/27/17 15:27, Kevin O'Connor wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
ACPI spec says " platform firmware should list the first logical processor of each of the individual multi-threaded processors in the MADT before listing any of the second logical processors. "
socket order is not mandated there and for example I have a system where apic id's re listed as following: 0x0 - first logical CPU on each core 0 of socket 0 0x10 - first logical CPU on each core 0 of socket 1 0x2 - first logical CPU on each core 1 of socket 0 0x12 - first logical CPU on each core 1 of socket 1 ...
the main point is that first threads are listed first (which socket/core order is not important except of boot CPU) and only after that goes list of secondary threads.
QEMU currently doesn't conform to above guideline as MADT is 1.0 revision doesn't have such requirement. However patches are welcome to make it adhering to the later ACPI specs. See build_madt() function to see how it's generated now.
Also does patch fix any particular issue/problem or just making MADT conform to spec?
SeaBIOS changes aside, when running a Linux VM with QEMU/KVM with the following command line:
qemu-system-x86_64 -machine q35,accel=kvm,iommu=on -cpu host -smp threads=2,cores=2,sockets=1 -m 2048 -device e1000,netdev=local,romfile= -netdev user,id=local,hostfwd=tcp::7000-:22 -serial stdio -vnc :1 -k en-us -drive file=ubuntu-14.04.qcow2,index=0,media=disk
/proc/cpuinfo reports the following:
# cat /proc/cpuinfo processor : 0 ... physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 ...
processor : 1 ... physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 1 initial apicid : 1 ...
processor : 2 ... physical id : 0 siblings : 4 core id : 1 cpu cores : 2 apicid : 2 initial apicid : 2 ...
processor : 3 ... physical id : 0 siblings : 4 core id : 1 cpu cores : 2 apicid : 3 initial apicid : 3
where the HT siblings couples are CPU0 and CPU1, CPU2 and CPU3 while according to the ACPI specification they should be CPU0 and CPU2, CPU1 and CPU3. So, my understanding is that even QEMU needs to be fixed and I can work to provide a patch. However, as I was mentioning before, I'm also interesting in updating SeaBIOS since one can use it without using QEMU.
Filippo
Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
On Tue, 31 Jan 2017 17:32:55 +0000 "Sironi, Filippo" sironi@amazon.de wrote:
On 30 Jan 2017, at 04:39, Igor Mammedov imammedo@redhat.com wrote:
On Fri, 27 Jan 2017 15:57:19 +0100 Laszlo Ersek lersek@redhat.com wrote:
On 01/27/17 15:27, Kevin O'Connor wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
ACPI spec says " platform firmware should list the first logical processor of each of the individual multi-threaded processors in the MADT before listing any of the second logical processors. "
socket order is not mandated there and for example I have a system where apic id's re listed as following: 0x0 - first logical CPU on each core 0 of socket 0 0x10 - first logical CPU on each core 0 of socket 1 0x2 - first logical CPU on each core 1 of socket 0 0x12 - first logical CPU on each core 1 of socket 1 ...
the main point is that first threads are listed first (which socket/core order is not important except of boot CPU) and only after that goes list of secondary threads.
QEMU currently doesn't conform to above guideline as MADT is 1.0 revision doesn't have such requirement. However patches are welcome to make it adhering to the later ACPI specs. See build_madt() function to see how it's generated now.
Also does patch fix any particular issue/problem or just making MADT conform to spec?
SeaBIOS changes aside, when running a Linux VM with QEMU/KVM with the following command line:
qemu-system-x86_64 -machine q35,accel=kvm,iommu=on -cpu host -smp threads=2,cores=2,sockets=1 -m 2048 -device e1000,netdev=local,romfile= -netdev user,id=local,hostfwd=tcp::7000-:22 -serial stdio -vnc :1 -k en-us -drive file=ubuntu-14.04.qcow2,index=0,media=disk
/proc/cpuinfo reports the following:
# cat /proc/cpuinfo processor : 0 ... physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 ...
processor : 1 ... physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 1 initial apicid : 1 ...
processor : 2 ... physical id : 0 siblings : 4 core id : 1 cpu cores : 2 apicid : 2 initial apicid : 2 ...
processor : 3 ... physical id : 0 siblings : 4 core id : 1 cpu cores : 2 apicid : 3 initial apicid : 3
where the HT siblings couples are CPU0 and CPU1, CPU2 and CPU3 while according to the ACPI specification they should be CPU0 and CPU2, CPU1 and CPU3.
IMHO: CPU numbering in kernel is not ACPI spec regulated and I'd say it's arbitrary. For x86, cpus are numbered in order they are initialized, so if for example there is hotpluggable CPUs and they were hotplugged in reverse order (ex. the last socket first) then kernel would continue sequentially numbering them in the order they are hotplugged.
So, my understanding is that even QEMU needs to be fixed and I can work to provide a patch. However, as I was mentioning before, I'm also interesting in updating SeaBIOS since one can use it without using QEMU.
most of smp initialization in SeaBIOS is called in QEMU only mode so ordering in legacy MADT is the least of issues if one would like to run SeaBIOS on baremetal.
Filippo
Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
On Wed, Feb 01, 2017 at 12:19:31PM +0100, Igor Mammedov wrote:
On Tue, 31 Jan 2017 17:32:55 +0000 "Sironi, Filippo" sironi@amazon.de wrote:
So, my understanding is that even QEMU needs to be fixed and I can work to provide a patch. However, as I was mentioning before, I'm also interesting in updating SeaBIOS since one can use it without using QEMU.
most of smp initialization in SeaBIOS is called in QEMU only mode so ordering in legacy MADT is the least of issues if one would like to run SeaBIOS on baremetal.
FYI, SeaBIOS runs on real hardware with coreboot, but in that setup coreboot does all the SMP initialization and coreboot creates the ACPI tables.
-Kevin
On 27 Jan 2017, at 06:27, Kevin O'Connor kevin@koconnor.net wrote:
On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
-Kevin
What if someone is using SeaBIOS but isn't using QEMU?
Filippo
Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
On Tue, Jan 31, 2017 at 04:38:04PM +0000, Sironi, Filippo wrote:
On 27 Jan 2017, at 06:27, Kevin O'Connor kevin@koconnor.net wrote: On Fri, Jan 27, 2017 at 02:18:39AM +0100, Filippo Sironi wrote:
When running Linux on an Intel machine with a commodity BIOS, CPUs are numbered starting from 0 and the first half refers to the first logical CPU on each core while the second half refers to the second logical CPU on each core. As an example, on machine with 2 sockets, 4 cores per socket, and 2 logical CPUs per core, CPUs would be numbered:
- 0 1 2 3 - first logical CPU on each core of socket 0
- 4 5 6 7 - first logical CPU on each core of socket 1
- 8 9 10 11 - second logical CPU on each core of socket 0
- 12 13 14 15 - second logical CPU on each core of socket 1
With seabios (prior to this patch), CPU 0 would be the first logical CPU on core 0 of socket 0, CPU 1 would be the second logical CPU on core 0 of socket 0, and so on. This is due to the fact that processor_id and local_apic_id are assigned with the same value when building MADT.
Exhaust the most-significant 7 bits of the 8-bit APIC ID and then come back to the least-significant bit when assigning local_apic_id, since the least-significant bit identifies the second logical CPU on a core.
If HTT isn't available, keep the previous behavior.
Signed-off-by: Filippo Sironi sironi@amazon.de
src/fw/acpi.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
diff --git a/src/fw/acpi.c b/src/fw/acpi.c index 8bc2ca6..864c247 100644 --- a/src/fw/acpi.c +++ b/src/fw/acpi.c
The acpi.c code in seabios is deprecated. With all recent version of qemu/kvm the acpi tables are obtained from qemu. So, if there is a desired change to the tables then a patch should be made for qemu.
What if someone is using SeaBIOS but isn't using QEMU?
The fw/acpi.c code is only ever active on QEMU.
-Kevin
-
Filippo Sironi -
Igor Mammedov -
Kevin O'Connor -
Laszlo Ersek -
Sironi, Filippo