Re: MP tables do not report multiple CPUs in Qemu 6.2.0 on x86 when given -smp cpus=n flag
On Wed, 19 Jan 2022 15:48:20 +0000 Peter Maydell peter.maydell@linaro.org wrote:
On Wed, 19 Jan 2022 at 14:44, Godmar Back gback@cs.vt.edu wrote:
after upgrading to 6.2.0, I observed that code such as MIT's xv6 (see [1]) is no longer able to detect multiple CPUs. Their code works in 6.1.1, however.
Hi; this isn't a great place for reporting QEMU bugs, because it's more of a user-to-user discussion list. Not all QEMU developers read it. I've cc'd the ACPI maintainers, who hopefully may have an idea about what's happening here. You could also file a bug at https://gitlab.com/qemu-project/qemu/-/issues
I built 6.1.1 from source and 6.2.0 from source and I have also tested with CentOS stream's 6.1.1 qemu-kvm and was able to pinpoint this change to these 2 versions of qemu. I am using qemu-system-i386 specifically.
I tried to go through the ChangeLog to see if the `-smp` option was deprecated or changed. I found this note [2] about invalid topologies having been removed in 5.2. Here's what I found after long experimentation:
The legacy MP tables appear to work only if you specify the longform `-smp cpus=4,cores=1,threads=1,sockets=4` in 6.2.0. If you specify `-smp 4` or `-smp cpus=4` it will not work in 6.2.0 (it worked in 6.1.1). I am guessing that perhaps the MP tables add entries for each socket, but that perhaps the behavior of the shortcuts `-smp n` and `-smp cpus=n` was changed to influence the number of cores rather than sockets.
In other words, `-smp cpus=n` now means `-smp cpus=n,cores=n,threads=1,sockets=1` whereas in 6.1.1 and before it meant `-smp cpus=n,cores=1,threads=1,sockets=n`. I note that specifying `-smp cpus=4,cores=4,threads=1,sockets=1` in 6.1.1 also does not create 4 entries in the legacy MP tables.
Thanks for reporting issue in such a detailed way.
QEMU doesn't generate legacy MP tables and as reported the above issue is still present in earlier versions when cores are used. Well seabios has a comment: /* Only populate the MPS tables with the first logical CPU in each package */ So I'd guess it has never worked for anything but sockets. With QEMU starting to prefer cores over sockets by default I'd suggest to either * explicitly provide desired topology (i.e. sockets) * use older machine type which still preffers sockets by default (ex: up to 6.1 machine types)
If anybody cares about legacy tables + cores/threads usecase, I suggest to investigate what can be done on SeaBIOS side which generates MP tables (assuming if anything could be done at all). CCing SeaBIOS mail-list.
Can someone confirm/deny this? If so, it's a breaking change that perhaps could be mentioned in the Changelog to save others the time when they upgrade. Affected educational OS include MIT's xv6 and Stanford's pintos OS.
Legacy MP table is not actively maintained part of the code, hence it's configuration which is not tested. However if someone is interested in maintaining this, one should contribute at least a testcase that will warn developers early if usecase is broken. We can't promise not breaking it ever but at least we would be able to document any breaking changes in release notes.
Thanks for all the work you do on qemu!
- Godmar
[1] https://github.com/mit-pdos/xv6-public/blob/eeb7b415dbcb12cc362d0783e41c3d1f... [2] https://qemu-project.gitlab.io/qemu/about/removed-features.html#smp-invalid-...
(I'm typing this email in gmail using the plaintext setting, hopefully it'll preserve line breaks.)
thanks -- PMM
[this is a follow-up email; I took qemu-discuss and qemu-devel off the distribution list]
On Thu, Jan 20, 2022 at 4:04 AM Igor Mammedov imammedo@redhat.com wrote:
Legacy MP table is not actively maintained part of the code, hence it's configuration which is not tested. However if someone is interested in maintaining this, one should contribute at least a testcase that will warn developers early if usecase is broken. We can't promise not breaking it ever but at least we would be able to document any breaking changes in release notes.
May I ask a clarification question? Is it correct that the only option to not use the MP tables and learn the CPU configuration would be to implement support for ACPI? For a small educational OS like xv6 or Pintos, we're aiming at keeping the codebase small, which is why so far we haven't taken this step. Is there a way to learn the CPU configuration without using ACPI that involves relatively little code?
- Godmar
On Thu, 20 Jan 2022 09:46:09 -0500 Godmar Back godmar@gmail.com wrote:
[this is a follow-up email; I took qemu-discuss and qemu-devel off the distribution list]
On Thu, Jan 20, 2022 at 4:04 AM Igor Mammedov imammedo@redhat.com wrote:
Legacy MP table is not actively maintained part of the code, hence it's configuration which is not tested. However if someone is interested in maintaining this, one should contribute at least a testcase that will warn developers early if usecase is broken. We can't promise not breaking it ever but at least we would be able to document any breaking changes in release notes.
May I ask a clarification question? Is it correct that the only option to not use the MP tables and learn the CPU configuration would be to implement support for ACPI? For a small educational OS like xv6 or Pintos, we're aiming at keeping the codebase small, which is why so far we haven't taken this step. Is there a way to learn the CPU configuration without using ACPI that involves relatively little code?
I do see value from education point of view in legacy MP table, however I'd guess its support will continue to deteriorate over the time, as ACPI is what majority of mainstream operating systems currently use.
If you wish to use qemu-6.2 and later and MP tables you have 2 options to do this: 1) use 6.1 machine type (or any older one): qemu-system-i386 -M pc-i440fx-6.1 -smp 4 ... 2) use default (latest) machine type and define explicit cpu layout supported by SeaBIOS qemu-system-i386 -smp 4,sockets=4 ...
As for adding ACPI support to xv6, is not so difficult if you only use it for detecting CPUs. You only need to locate RSDP which points to a list of ACPI tables ((X)RSDT) and then jump and parse MADT table which is analog of MP table. It's all very well documented in ACPI spec and you can use SeaBIOS code for inspiration as well, look for RSDP_SIGNATURE as starting point (SeaBIOS does a lot more stuff with ACPI but you don't really need it (you can start with minimum of necessary code and then build up on top as needed)).
Alternative approach to discover present CPUs with ad-hock approach SeaBIOS use(s|d) to build its own MADT tables (note: since QEMU-1.7 it uses ACPI tables provided by QEMU). See smp_scan() + handle_smp() for an example.
- Godmar
On Fri, 21 Jan 2022, Igor Mammedov wrote:
As for adding ACPI support to xv6, is not so difficult if you only use it for detecting CPUs. You only need to locate RSDP which points to a list of ACPI tables ((X)RSDT) and then jump and parse MADT table which is analog of MP table. It's all very well documented in ACPI spec and you can use SeaBIOS code for inspiration as well, look for RSDP_SIGNATURE as starting point (SeaBIOS does a lot more stuff with ACPI but you don't really need it (you can start with minimum of necessary code and then build up on top as needed)).
bios-tables-test.c in tests/qtest also has basic code to parse the various tables. It can also be a good starting point. I also find this site useful: https://wiki.osdev.org/RSDP
It tells you the spec without going through the larger PDF.
-
Ani Sinha -
Godmar Back -
Igor Mammedov