I’ve been working a little to see if I can get multi-threaded TGC to work for the Mac OS/X in qemu, but Openbios doesn’t seem to support more than one CPU and doesn’t build any device for a second CPU.
I’ve looked at the openbios/arch/ppc/qemu/init.c, and that seems to be the place to apply a patch for multi cpu support, but I don’t really seem any graceful way of doing it.
Can anyone offer any help?
TGC-Work/bin/qemu-system-ppc -M mac99,via=pmu -serial stdio -smp 2,cores=2,threads=1 -accel tcg,thread=multi -cpu g4 -boot c -prom-env auto-boot?=true -prom-env boot-args=-v -hda /Users/jam/os9.2.1.img -m 256 -bios /Users/jam/aty7/openbios/obj-ppc/openbios-qemu.elf heathrow_class_init: init 2 core99_machine_class_init: init 2 qemu-system-ppc: warning: Guest not yet converted to MTTCG - you may get unexpected results s>> et_property: NULL phandle
============================================================= OpenBIOS 1.1 [Aug 6 2019 02:05] Configuration device id QEMU version 1 machine id 1 CPUs: 2 Memory: 256M UUID: 00000000-0000-0000-0000-000000000000 CPU type PowerPC,G4
milliseconds isn't unique.
switching to new context: call-method slw_update_keymap failed with error ffffffdf call-method slw_update_keymap failed with error ffffffdf
On Sun, 18 Aug 2019, Jd Lyons via OpenBIOS wrote:
I’ve been working a little to see if I can get multi-threaded TGC to work for the Mac OS/X in qemu, but Openbios doesn’t seem to support more than one CPU and doesn’t build any device for a second CPU.
I’ve looked at the openbios/arch/ppc/qemu/init.c, and that seems to be the place to apply a patch for multi cpu support, but I don’t really seem any graceful way of doing it.
Can anyone offer any help?
I think passing info about machine to OpenBIOS via a flattened device tree as was proposed before would solve this too becuase then QEMU (which knows how many and what kind of CPUs it emulates) could add the necessary info about CPUs in the FDT and OpenBIOS would not have to know or do anything about it just get it from QEMU. But I wasn't able to make this work in OpenBIOS yet. (I did manage to get FDT unflattening in OpenFirmware working but that's not complete yet so its experimental and OpenFirmware does not have Mac drivers so cannot replace OpenBIOS yet. I've also posted an attempted port of SLOF's fdt.fs before but that needs some work to be usable in OpenBIOS.) You can see a simple example of how to pass info about CPU and memory at the end of latest pegasos2 version here:
https://osdn.net/projects/qmiga/scm/git/qemu/blobs/pegasos2/hw/ppc/pegasos2....
(More complex examples can be found in spapr and pnv machines that likely handle multiple CPUs.) This creates an FDT with basic info that it puts in guest memory where the firmware parses it the same way as SLOF does to create the initial device tree which then it can amend as needed to add Forth words PCI buses and detected cards. But this also allows removing the hard coded CPU and board infos and to get rid of values passed over FW_CFG currently so I think would simplify OpenBIOS a lot.
TGC-Work/bin/qemu-system-ppc -M mac99,via=pmu -serial stdio -smp 2,cores=2,threads=1 -accel tcg,thread=multi -cpu g4 -boot c -prom-env auto-boot?=true -prom-env boot-args=-v -hda /Users/jam/os9.2.1.img -m 256 -bios /Users/jam/aty7/openbios/obj-ppc/openbios-qemu.elf heathrow_class_init: init 2 core99_machine_class_init: init 2 qemu-system-ppc: warning: Guest not yet converted to MTTCG - you may get unexpected results
This warning probably means that your main problem may not be missing info about CPUs in device tree yet but maybe you'd need to fix emulation first or it may not work at all with MTTCG yet. (I think MTTCG might be interesting but fixing hardfloat for PPC would provide a greater speedup for most programs currently so that could be more interesting to look at.)
Regards, BALATON Zoltan
On 18/08/2019 19:41, Jd Lyons via OpenBIOS wrote:
I’ve been working a little to see if I can get multi-threaded TGC to work for the Mac OS/X in qemu, but Openbios doesn’t seem to support more than one CPU and doesn’t build any device for a second CPU.
I’ve looked at the openbios/arch/ppc/qemu/init.c, and that seems to be the place to apply a patch for multi cpu support, but I don’t really seem any graceful way of doing it.
Can anyone offer any help?
TGC-Work/bin/qemu-system-ppc -M mac99,via=pmu -serial stdio -smp 2,cores=2,threads=1 -accel tcg,thread=multi -cpu g4 -boot c -prom-env auto-boot?=true -prom-env boot-args=-v -hda /Users/jam/os9.2.1.img -m 256 -bios /Users/jam/aty7/openbios/obj-ppc/openbios-qemu.elf heathrow_class_init: init 2 core99_machine_class_init: init 2 qemu-system-ppc: warning: Guest not yet converted to MTTCG - you may get unexpected results s>> et_property: NULL phandle
============================================================= OpenBIOS 1.1 [Aug 6 2019 02:05] Configuration device id QEMU version 1 machine id 1 CPUs: 2 Memory: 256M UUID: 00000000-0000-0000-0000-000000000000 CPU type PowerPC,G4
milliseconds isn't unique.
switching to new context: call-method slw_update_keymap failed with error ffffffdf call-method slw_update_keymap failed with error ffffffdf
You can grab the current number of CPUs from the fwcfg interface at register FW_CFG_NB_CPUS. As a starting point I'd try writing a for loop somewhere around https://github.com/openbios/openbios/blob/master/arch/ppc/qemu/init.c#L996 to create the relevant number of CPU nodes in the DT.
Once that is done I suspect the next part of the puzzle will be figuring out how to wire up the machine and its IRQs correctly...
ATB,
Mark.
On Sun, 18 Aug 2019, Mark Cave-Ayland wrote:
On 18/08/2019 19:41, Jd Lyons via OpenBIOS wrote:
I’ve been working a little to see if I can get multi-threaded TGC to work for the Mac OS/X in qemu, but Openbios doesn’t seem to support more than one CPU and doesn’t build any device for a second CPU.
I’ve looked at the openbios/arch/ppc/qemu/init.c, and that seems to be the place to apply a patch for multi cpu support, but I don’t really seem any graceful way of doing it.
Can anyone offer any help?
TGC-Work/bin/qemu-system-ppc -M mac99,via=pmu -serial stdio -smp 2,cores=2,threads=1 -accel tcg,thread=multi -cpu g4 -boot c -prom-env auto-boot?=true -prom-env boot-args=-v -hda /Users/jam/os9.2.1.img -m 256 -bios /Users/jam/aty7/openbios/obj-ppc/openbios-qemu.elf heathrow_class_init: init 2 core99_machine_class_init: init 2 qemu-system-ppc: warning: Guest not yet converted to MTTCG - you may get unexpected results s>> et_property: NULL phandle
============================================================= OpenBIOS 1.1 [Aug 6 2019 02:05] Configuration device id QEMU version 1 machine id 1 CPUs: 2 Memory: 256M UUID: 00000000-0000-0000-0000-000000000000 CPU type PowerPC,G4
milliseconds isn't unique.
switching to new context: call-method slw_update_keymap failed with error ffffffdf call-method slw_update_keymap failed with error ffffffdf
You can grab the current number of CPUs from the fwcfg interface at register FW_CFG_NB_CPUS. As a starting point I'd try writing a for loop somewhere around https://github.com/openbios/openbios/blob/master/arch/ppc/qemu/init.c#L996 to create the relevant number of CPU nodes in the DT.
(I probably should not be pushing this further but I can't resist, sorry. I promise to stop after this one. This is strictly technical discussion, nothing against anyone.)
Remember what I've said about relying on random hacks that you were offended by? The whole FW_CFG stuff were just added as needed and I think we already have more than should be there. It would be much cleaner to pass this whole thing in one FDT than bits and pieces that also need knowledge in OpenBIOS to understand and assemble into a device tree again. Then a lot of logic in OpenBIOS now only there for this could be cleaned up. Also the above only fixes it for PPC/QEMU, what about other archs? They would need to implement similar mechanisms separately while with FDT unflattening all QEMU archs and machines could use the same way without needing more knowledge in OpenBIOS. (I think we're at a point where not adding more but removing stuff would be an improvement as that would make OpenBIOS simpler and more generic. Firmwares are usually highly board specific with intricate knowledge about the hardware when they run on real hardware but for emulation like QEMU where most hardware init that is the source of knowledge about hardware is not needed a firmware can be generic as SLOF has demonstrated.)
Once that is done I suspect the next part of the puzzle will be figuring out how to wire up the machine and its IRQs correctly...
I haven't thought about this before but now you say it this is likely where more hacks may be needed if you rely on FW_CFG only or you end up hard coding knowledge into OpenBIOS and get something only working with current machines emulated by QEMU that will need to be changed when QEMU is changed.
On the other hand this info is already in QEMU which could put it in the FDT (on embedded systems this is what the dtb is used for so it can describe it) then after unflattening, the references should be replaced with the appropriate Forth handles which can be done generically without knowledge about the board. The SLOF fdt.fs has code for this but I could not get that work yet as I had nothing to test with so far.
So I think it would be better to go towards the FDT way rather than adding more knowledge that are just for one arch or may need to be refined later even if that's initially more work but could be better on the long run.
But it's your decision so if I still could not convince you this would be a good idea I'll leave it and won't bring this up again.
Regards, BALATON Zoltan
On Aug 18, 2019, at 4:49 PM, Mark Cave-Ayland mark.cave-ayland@ilande.co.uk wrote:
On 18/08/2019 19:41, Jd Lyons via OpenBIOS wrote:
I’ve been working a little to see if I can get multi-threaded TGC to work for the Mac OS/X in qemu, but Openbios doesn’t seem to support more than one CPU and doesn’t build any device for a second CPU.
I’ve looked at the openbios/arch/ppc/qemu/init.c, and that seems to be the place to apply a patch for multi cpu support, but I don’t really seem any graceful way of doing it.
Can anyone offer any help?
TGC-Work/bin/qemu-system-ppc -M mac99,via=pmu -serial stdio -smp 2,cores=2,threads=1 -accel tcg,thread=multi -cpu g4 -boot c -prom-env auto-boot?=true -prom-env boot-args=-v -hda /Users/jam/os9.2.1.img -m 256 -bios /Users/jam/aty7/openbios/obj-ppc/openbios-qemu.elf heathrow_class_init: init 2 core99_machine_class_init: init 2 qemu-system-ppc: warning: Guest not yet converted to MTTCG - you may get unexpected results s>> et_property: NULL phandle
============================================================= OpenBIOS 1.1 [Aug 6 2019 02:05] Configuration device id QEMU version 1 machine id 1 CPUs: 2 Memory: 256M UUID: 00000000-0000-0000-0000-000000000000 CPU type PowerPC,G4
milliseconds isn't unique.
switching to new context: call-method slw_update_keymap failed with error ffffffdf call-method slw_update_keymap failed with error ffffffdf
You can grab the current number of CPUs from the fwcfg interface at register FW_CFG_NB_CPUS. As a starting point I'd try writing a for loop somewhere around https://github.com/openbios/openbios/blob/master/arch/ppc/qemu/init.c#L996 to create the relevant number of CPU nodes in the DT.
Once that is done I suspect the next part of the puzzle will be figuring out how to wire up the machine and its IRQs correctly...
ATB,
Mark.
Thanks Mark,
Doing this gets me 2 /cpus but both are @0
/* This needs adjusting if #size-cells gets increased. Alternatively use multiple (address, size) tuples. */ PUSH(ram_size & 0xffffffff); fword("encode-int"); fword("encode+"); push_str("reg"); fword("property");
cpu = id_cpu(); cpu->initfn(cpu); printk("CPU type %s\n", cpu->name);
cpu = id_cpu(); cpu->initfn(cpu); printk("CPU type %s\n", cpu->name);
snprintf(buf, sizeof(buf), "/cpus/%s", cpu->name); ofmem_register(find_dev("/memory"), find_dev(buf)); node_methods_init(buf);
How would I go about making the second cpu be @1?
I can’t seem to figure where in the code the cpu is set to @0, is this a function of Openbios or Qemu?
On Mon, 19 Aug 2019, Jd Lyons via OpenBIOS wrote:
On Aug 18, 2019, at 4:49 PM, Mark Cave-Ayland mark.cave-ayland@ilande.co.uk wrote: You can grab the current number of CPUs from the fwcfg interface at register FW_CFG_NB_CPUS. As a starting point I'd try writing a for loop somewhere around https://github.com/openbios/openbios/blob/master/arch/ppc/qemu/init.c#L996 to create the relevant number of CPU nodes in the DT.
Once that is done I suspect the next part of the puzzle will be figuring out how to wire up the machine and its IRQs correctly...
Thanks Mark,
Doing this gets me 2 /cpus but both are @0
/* This needs adjusting if #size-cells gets increased. Alternatively use multiple (address, size) tuples. */ PUSH(ram_size & 0xffffffff); fword("encode-int"); fword("encode+"); push_str("reg"); fword("property");
cpu = id_cpu(); cpu->initfn(cpu); printk("CPU type %s\n", cpu->name);
cpu = id_cpu(); cpu->initfn(cpu); printk("CPU type %s\n", cpu->name);
snprintf(buf, sizeof(buf), "/cpus/%s", cpu->name); ofmem_register(find_dev("/memory"), find_dev(buf)); node_methods_init(buf);
How would I go about making the second cpu be @1?
I can’t seem to figure where in the code the cpu is set to @0, is this a function of Openbios or Qemu?
I think this comes from the reg property so you'd need to set that for additional CPUs to increasing numbers. The cpu->initfn() you call above is a function pointer defined in the static const struct cpudef ppc_defs[] array which directs it to different init functions for different cpu types (as identified by id_cpu() by looking at PVR). Some of these init functions actually set reg to 0 for some reason, I think that's wrong and the reg property should be set in arch_of_init instead when adding a cpu but if you're just hacking (as you haven't added a for loop above just another cpu) you can ignore that for now and just override reg property after adding second cpu and see if that works.
(I'm not mentioning that all of the above mess with CPU table and functions could be gone if this info came from QEMU via FDT. Knowledge about CPUs might be needed on real hardware but this is in qemu specific part which could be generic. OK, I promised not to bring it up again but I see a lot of these simplifications possible by going that way so I really think that would make OpenBIOS more managable.)
Regards, BALATON Zoltan
On Aug 19, 2019, at 6:04 AM, BALATON Zoltan balaton@eik.bme.hu wrote:
you can ignore that for now and just override reg property after adding second cpu and see if that works.
Thanks Zoltan,
How would I go about overriding the reg property?
On Mon, 19 Aug 2019, Jd Lyons wrote:
How would I go about overriding the reg property?
Not sure, look for "reg" (with quotes) an see how it's done elsewhere. Looks like there are two ways via C function:
set_property(phandle, "reg", (char *)props, ncells * sizeof(props[0]));
which needs a phandle from somewhere and an array to hold the property value. Don't know how to get those but there should be examples in the code.
Or using Forth:
PUSH(0); fword("encode-int"); push_str("reg"); fword("property");
but that needs to be done during the device is created, between new-device and finish-device. When after cpu init func that called finish-device and you want to modify existing node you'd need to call find-device instead of new-device like :
push_str("/cpus/PowerPC,whatever"); fword("find-device");
but if both of your CPUs are called the same how do you get a path to the second one? So it looks like it's already botched by the cpu init function and there may not be an easy way to fix up after that (or I don't know how) so you may need to add a parameter to the init func to tell it to set reg property from that, but then you'd need to modify all init functions as the function pointer definition will change so if we're back to hacking and just to try if it would work you may add a global cpunum variable (well, static to qemu/init.c, outside all functions) and pass cpu number to init func via that, then think about how to clean it up later (of course this won't be acceptable as a solution just for testing the idea).
Actually it seems all cpu init functions add a reg property but those that don't set it to 0 set it to the PIR of the CPU which is some kind of ID register:
https://qemu-devel.nongnu.narkive.com/sCLUgMqF/patch-ppc-add-pir-register-to...
I don't know how all this works or should work so can't tell what will break by modifying it.
Regards, BALATON Zoltan
-
BALATON Zoltan -
Jd Lyons -
Mark Cave-Ayland