Re: [RFC PATCH 0/2] QEMU/openbios: PPC Software TLB support in the G4 family
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
============================================================= OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message) So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well. Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
On Wed, 24 Nov 2021, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
============================================================= OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int
This CPU appears in PowerMac G4 so maybe better use -machine mac99,via=pmu with it as it's strange to put it in a g3beige but that may not matter for reproducing the problem. As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4). According to <https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7450> this CPU was used in <https://en.wikipedia.org/wiki/Power_Mac_G4#Digital_Audio/Quicksilver> and it runs up to Mac OS 10.4.11. (Although OpenBIOS sets the device tree according to a PowerMac3,1 so not sure it's entirely correct for the PowerMac3,5 that has a 7450 CPU and if it matters for Mac OS X.) I asked about this before but got no reply back then: https://lists.nongnu.org/archive/html/qemu-ppc/2020-03/msg00292.html This was because pegasos2 should have 7447 but it did not work so currently I've set it to 7400 which also works. The original board firmware had some problem detecting it but I think that only results in wrong CPU speed shown which is only a cosmetic problem, otherwise it seems to work. Since pegasos2 does not use OpenBIOS but either VOF or the board's original firmware it may be an alternative way to test at least 7447 which the firmware and guests running on that board should work with. At least Debian 8.11 powerpc version had support for pegasos2 and should boot, I'm not sure newer versions still work. More info on pegasos2 can be found at: http://zero.eik.bme.hu/~balaton/qemu/amiga/#morphos and https://osdn.net/projects/qmiga/wiki/SubprojectPegasos2 I don't remember what problem I had with 7447 but if it does not work with pegasos2 then maybe there's some other problem with it too. I think it was maybe related to TLBs but I don't know and had no time to try again so I could be entirely wrong about this. Regards, BALATON Zoltan
Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
On 11/25/21 01:45, BALATON Zoltan wrote:
On Wed, 24 Nov 2021, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
>> ============================================================= >> OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int
This CPU appears in PowerMac G4 so maybe better use -machine mac99,via=pmu with it as it's strange to put it in a g3beige but that may not matter for reproducing the problem.
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4). According to <https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7450> this CPU was used in <https://en.wikipedia.org/wiki/Power_Mac_G4#Digital_Audio/Quicksilver> and it runs up to Mac OS 10.4.11. (Although OpenBIOS sets the device tree according to a PowerMac3,1 so not sure it's entirely correct for the PowerMac3,5 that has a 7450 CPU and if it matters for Mac OS X.)
I asked about this before but got no reply back then: https://lists.nongnu.org/archive/html/qemu-ppc/2020-03/msg00292.html
This was because pegasos2 should have 7447 but it did not work so currently I've set it to 7400 which also works. The original board firmware had some problem detecting it but I think that only results in wrong CPU speed shown which is only a cosmetic problem, otherwise it seems to work. Since pegasos2 does not use OpenBIOS but either VOF or the board's original firmware it may be an alternative way to test at least 7447 which the firmware and guests running on that board should work with. At least Debian 8.11 powerpc version had support for pegasos2 and should boot, I'm not sure newer versions still work. More info on pegasos2 can be found at: http://zero.eik.bme.hu/~balaton/qemu/amiga/#morphos and https://osdn.net/projects/qmiga/wiki/SubprojectPegasos2
I would be nice to add some documentation for these machines under : https://qemu.readthedocs.io/en/latest/system/target-ppc.html Thanks C.
I don't remember what problem I had with 7447 but if it does not work with pegasos2 then maybe there's some other problem with it too. I think it was maybe related to TLBs but I don't know and had no time to try again so I could be entirely wrong about this.
Regards, BALATON Zoltan
Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
On Thu, 25 Nov 2021, Cédric Le Goater wrote:
On 11/25/21 01:45, BALATON Zoltan wrote:
On Wed, 24 Nov 2021, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
>> ============================================================= >> OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int
This CPU appears in PowerMac G4 so maybe better use -machine mac99,via=pmu with it as it's strange to put it in a g3beige but that may not matter for reproducing the problem.
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4). According to <https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7450> this CPU was used in <https://en.wikipedia.org/wiki/Power_Mac_G4#Digital_Audio/Quicksilver> and it runs up to Mac OS 10.4.11. (Although OpenBIOS sets the device tree according to a PowerMac3,1 so not sure it's entirely correct for the PowerMac3,5 that has a 7450 CPU and if it matters for Mac OS X.)
I asked about this before but got no reply back then: https://lists.nongnu.org/archive/html/qemu-ppc/2020-03/msg00292.html
This was because pegasos2 should have 7447 but it did not work so currently I've set it to 7400 which also works. The original board firmware had some problem detecting it but I think that only results in wrong CPU speed shown which is only a cosmetic problem, otherwise it seems to work. Since pegasos2 does not use OpenBIOS but either VOF or the board's original firmware it may be an alternative way to test at least 7447 which the firmware and guests running on that board should work with. At least Debian 8.11 powerpc version had support for pegasos2 and should boot, I'm not sure newer versions still work. More info on pegasos2 can be found at: http://zero.eik.bme.hu/~balaton/qemu/amiga/#morphos and https://osdn.net/projects/qmiga/wiki/SubprojectPegasos2
I would be nice to add some documentation for these machines under :
https://qemu.readthedocs.io/en/latest/system/target-ppc.html
Yes, I thought about that but haven't got to it yet. I'm also not sure what should I put in that doc so if you have time feel free to copy from the above URLs what you think is better to have in the docs. Otherwise I'll do it sometimes in the future. Regards, BALATON Zoltan
Thanks
C.
I don't remember what problem I had with 7447 but if it does not work with pegasos2 then maybe there's some other problem with it too. I think it was maybe related to TLBs but I don't know and had no time to try again so I could be entirely wrong about this.
Regards, BALATON Zoltan
Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
https://qemu.readthedocs.io/en/latest/system/target-ppc.html
Yes, I thought about that but haven't got to it yet. I'm also not sure what should I put in that doc so if you have time feel free to copy from the above URLs what you think is better to have in the docs. Otherwise I'll do it sometimes in the future.
ok. We don't have to copy all of the contents. We can reference external URLs for more complete information. Thanks, C.
Hi! On Thu, Nov 25, 2021 at 01:45:00AM +0100, BALATON Zoltan wrote:
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe
OSX uses hardware pagetables.
MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4).
I have no idea what MorphOS uses, but I bet HPT as well. That is because HPT is fastest in general. Software TLB reloads are good in special cases only; the most common is real-time OSes, which can use its lower guaranteed latency for some special address spaces (and can have a simpler address map in general). Segher
Hello, On 11/25/21 10:38, Segher Boessenkool wrote:
Hi!
On Thu, Nov 25, 2021 at 01:45:00AM +0100, BALATON Zoltan wrote:
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe
OSX uses hardware pagetables.
MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4).
I have no idea what MorphOS uses, but I bet HPT as well. That is because HPT is fastest in general. Software TLB reloads are good in special cases only; the most common is real-time OSes, which can use its lower guaranteed latency for some special address spaces (and can have a simpler address map in general).
The support was added to QEMU knowing that Linux didn't handle soft TLBs. And the commit says that it was kept disabled initially. I guess that was broken these last years. C. $ git show 7dbe11acd807 commit 7dbe11acd807 Author: Jocelyn Mayer <l_indien@magic.fr> Date: Mon Oct 1 05:16:57 2007 +0000 Handle all MMU models in switches, even if it's just to abort because of lack of supporting code. Implement 74xx software TLB model. Keep 74xx with software TLB disabled, as Linux is not able to handle TLB miss on those processors. C.
Hi! On Fri, Nov 26, 2021 at 09:34:44AM +0100, Cédric Le Goater wrote:
On 11/25/21 10:38, Segher Boessenkool wrote:
On Thu, Nov 25, 2021 at 01:45:00AM +0100, BALATON Zoltan wrote:
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe
OSX uses hardware pagetables.
MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4).
I have no idea what MorphOS uses, but I bet HPT as well. That is because HPT is fastest in general. Software TLB reloads are good in special cases only; the most common is real-time OSes, which can use its lower guaranteed latency for some special address spaces (and can have a simpler address map in general).
The support was added to QEMU knowing that Linux didn't handle soft TLBs. And the commit says that it was kept disabled initially. I guess that was broken these last years.
Ah :-) So when was it enabled, do you know?
$ git show 7dbe11acd807 commit 7dbe11acd807 Author: Jocelyn Mayer <l_indien@magic.fr> Date: Mon Oct 1 05:16:57 2007 +0000
Handle all MMU models in switches, even if it's just to abort because of lack of supporting code. Implement 74xx software TLB model. Keep 74xx with software TLB disabled, as Linux is not able to handle TLB miss on those processors.
This is very specifically for 7450, not 7400, fwiw. 7400 is a nice core, while 7450 is ugly and asymmetric and unbalanced as hell. It can be faster though ;-) Segher
Segher Boessenkool <segher@kernel.crashing.org> writes:
Hi!
On Fri, Nov 26, 2021 at 09:34:44AM +0100, Cédric Le Goater wrote:
On 11/25/21 10:38, Segher Boessenkool wrote:
On Thu, Nov 25, 2021 at 01:45:00AM +0100, BALATON Zoltan wrote:
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe
OSX uses hardware pagetables.
MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4).
I have no idea what MorphOS uses, but I bet HPT as well. That is because HPT is fastest in general. Software TLB reloads are good in special cases only; the most common is real-time OSes, which can use its lower guaranteed latency for some special address spaces (and can have a simpler address map in general).
The support was added to QEMU knowing that Linux didn't handle soft TLBs. And the commit says that it was kept disabled initially. I guess that was broken these last years.
Ah :-) So when was it enabled, do you know?
Hm.. That commit message does not match the code. They simply added the software TLB implementation to an already existing SOFT_74xx MMU model. I don't see anything that would keep it disabled at that time.
On 11/26/21 13:13, Fabiano Rosas wrote:
Segher Boessenkool <segher@kernel.crashing.org> writes:
Hi!
On Fri, Nov 26, 2021 at 09:34:44AM +0100, Cédric Le Goater wrote:
On 11/25/21 10:38, Segher Boessenkool wrote:
On Thu, Nov 25, 2021 at 01:45:00AM +0100, BALATON Zoltan wrote:
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe
OSX uses hardware pagetables.
MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4).
I have no idea what MorphOS uses, but I bet HPT as well. That is because HPT is fastest in general. Software TLB reloads are good in special cases only; the most common is real-time OSes, which can use its lower guaranteed latency for some special address spaces (and can have a simpler address map in general).
The support was added to QEMU knowing that Linux didn't handle soft TLBs. And the commit says that it was kept disabled initially. I guess that was broken these last years.
Ah :-) So when was it enabled, do you know?
Hm.. That commit message does not match the code. They simply added the software TLB implementation to an already existing SOFT_74xx MMU model. I don't see anything that would keep it disabled at that time.
because most of the cpu definitions in ppc_defs[] are protected by a : #if defined (TODO) See below. commit 8ca3f6c3824c ("Allow selection of all defined PowerPC 74xx (aka G4) CPUs.") removed the TODO without a reason :/ This is old, when SVN was in used. Thanks, C. .... #if defined (TODO) /* PowerPC 7450 (G4) */ POWERPC_DEF("7450", CPU_POWERPC_7450, 0xFFFFFFFF, 7450), /* Code name for PowerPC 7450 */ POWERPC_DEF("Vger", CPU_POWERPC_7450, 0xFFFFFFFF, 7450), #endif #if defined (TODO) /* PowerPC 7450 v1.0 (G4) */ POWERPC_DEF("7450v1.0", CPU_POWERPC_7450_v10, 0xFFFFFFFF, 7450), #endif #if defined (TODO) /* PowerPC 7450 v1.1 (G4) */ POWERPC_DEF("7450v1.1", CPU_POWERPC_7450_v11, 0xFFFFFFFF, 7450), #endif #if defined (TODO) /* PowerPC 7450 v1.2 (G4) */ POWERPC_DEF("7450v1.2", CPU_POWERPC_7450_v12, 0xFFFFFFFF, 7450), #endif #if defined (TODO) /* PowerPC 7450 v2.0 (G4) */ POWERPC_DEF("7450v2.0", CPU_POWERPC_7450_v20, 0xFFFFFFFF, 7450), #endif #if defined (TODO) /* PowerPC 7450 v2.1 (G4) */ POWERPC_DEF("7450v2.1", CPU_POWERPC_7450_v21, 0xFFFFFFFF, 7450), #endif #if defined (TODO) /* PowerPC 7441 (G4) */ POWERPC_DEF("7441", CPU_POWERPC_74x1, 0xFFFFFFFF, 7440), /* PowerPC 7451 (G4) */ ....
Cédric Le Goater <clg@kaod.org> writes:
On 11/26/21 13:13, Fabiano Rosas wrote:
Segher Boessenkool <segher@kernel.crashing.org> writes:
Hi!
On Fri, Nov 26, 2021 at 09:34:44AM +0100, Cédric Le Goater wrote:
On 11/25/21 10:38, Segher Boessenkool wrote:
On Thu, Nov 25, 2021 at 01:45:00AM +0100, BALATON Zoltan wrote:
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe
OSX uses hardware pagetables.
MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4).
I have no idea what MorphOS uses, but I bet HPT as well. That is because HPT is fastest in general. Software TLB reloads are good in special cases only; the most common is real-time OSes, which can use its lower guaranteed latency for some special address spaces (and can have a simpler address map in general).
The support was added to QEMU knowing that Linux didn't handle soft TLBs. And the commit says that it was kept disabled initially. I guess that was broken these last years.
Ah :-) So when was it enabled, do you know?
Hm.. That commit message does not match the code. They simply added the software TLB implementation to an already existing SOFT_74xx MMU model. I don't see anything that would keep it disabled at that time.
because most of the cpu definitions in ppc_defs[] are protected by a :
#if defined (TODO)
See below. commit 8ca3f6c3824c ("Allow selection of all defined PowerPC 74xx (aka G4) CPUs.") removed the TODO without a reason :/ This is old, when SVN was in used.
Ah nice catch!
BALATON Zoltan <balaton@eik.bme.hu> writes:
On Wed, 24 Nov 2021, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
============================================================= OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int
This CPU appears in PowerMac G4 so maybe better use -machine mac99,via=pmu with it as it's strange to put it in a g3beige but that may not matter for reproducing the problem.
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4). According to <https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7450> this CPU was used in <https://en.wikipedia.org/wiki/Power_Mac_G4#Digital_Audio/Quicksilver> and it runs up to Mac OS 10.4.11. (Although OpenBIOS sets the device tree according to a PowerMac3,1 so not sure it's entirely correct for the PowerMac3,5 that has a 7450 CPU and if it matters for Mac OS X.)
I asked about this before but got no reply back then: https://lists.nongnu.org/archive/html/qemu-ppc/2020-03/msg00292.html
This was because pegasos2 should have 7447 but it did not work so currently I've set it to 7400 which also works. The original board firmware had some problem detecting it but I think that only results in wrong CPU speed shown which is only a cosmetic problem, otherwise it seems to work. Since pegasos2 does not use OpenBIOS but either VOF or the board's original firmware it may be an alternative way to test at least 7447 which the firmware and guests running on that board should work with. At least Debian 8.11 powerpc version had support for pegasos2 and should boot, I'm not sure newer versions still work. More info on pegasos2 can be found at: http://zero.eik.bme.hu/~balaton/qemu/amiga/#morphos and https://osdn.net/projects/qmiga/wiki/SubprojectPegasos2
I don't remember what problem I had with 7447 but if it does not work with pegasos2 then maybe there's some other problem with it too. I think it was maybe related to TLBs but I don't know and had no time to try again so I could be entirely wrong about this.
So yesterday I tested these: * all with my patched OpenBIOS ** all of them work fine with the 7400 CPU - macos9 w/ -M mac99 and -cpu 7450 OS starts and then freezes. Upon further inspection I see that it has the 0x1000, 0x1100 and 0x1200 vectors implemented, but not the 7450 ones. It implements the 6xx SW TLB handler, i.e. it accesses SPR 976 instead of 980. - macosx10 w/ -M mac99 and -cpu 7450 Shows the apple logo and then spins. Looking at the asm I don't see anything resembling the 7450 software TLB code. I'm calling it unsupported. - debian 10 w/ -M mac99 and -cpu 7450 Boots linux and then spins. It has the vectors implemented, but it also uses different SPRs. The data misses come via 976, which is different from 7450, which uses only 980 (tlbmiss) for instruction and data. - morphos w/ -M pegasos2 and -cpu 7447|7450 Hangs. It also has a different software TLB model implemented: Trying to read invalid spr 978 (0x3d2) at 00001100 Trying to read invalid spr 977 (0x3d1) at 0000110c Trying to read invalid spr 979 (0x3d3) at 0000115c Trying to read invalid spr 976 (0x3d0) at 00001188 So my initial impression that no OS supports the 7450 software TLB seems to match these findings and what people have said elsewhere in the thread.
Regards, BALATON Zoltan
Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
On Fri, 26 Nov 2021, Fabiano Rosas wrote:
BALATON Zoltan <balaton@eik.bme.hu> writes:
On Wed, 24 Nov 2021, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
============================================================= OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int
This CPU appears in PowerMac G4 so maybe better use -machine mac99,via=pmu with it as it's strange to put it in a g3beige but that may not matter for reproducing the problem.
As for guests, those running on the said PowerMac G4 should have support for these CPUs so maybe you can try some Mac OS X versions (or maybe MorphOS but that is not the best for debugging as there's no source available nor any help from its owners but just to see if it boots it may be sufficient, it should work on real PowerMac G4). According to <https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7450> this CPU was used in <https://en.wikipedia.org/wiki/Power_Mac_G4#Digital_Audio/Quicksilver> and it runs up to Mac OS 10.4.11. (Although OpenBIOS sets the device tree according to a PowerMac3,1 so not sure it's entirely correct for the PowerMac3,5 that has a 7450 CPU and if it matters for Mac OS X.)
I asked about this before but got no reply back then: https://lists.nongnu.org/archive/html/qemu-ppc/2020-03/msg00292.html
This was because pegasos2 should have 7447 but it did not work so currently I've set it to 7400 which also works. The original board firmware had some problem detecting it but I think that only results in wrong CPU speed shown which is only a cosmetic problem, otherwise it seems to work. Since pegasos2 does not use OpenBIOS but either VOF or the board's original firmware it may be an alternative way to test at least 7447 which the firmware and guests running on that board should work with. At least Debian 8.11 powerpc version had support for pegasos2 and should boot, I'm not sure newer versions still work. More info on pegasos2 can be found at: http://zero.eik.bme.hu/~balaton/qemu/amiga/#morphos and https://osdn.net/projects/qmiga/wiki/SubprojectPegasos2
I don't remember what problem I had with 7447 but if it does not work with pegasos2 then maybe there's some other problem with it too. I think it was maybe related to TLBs but I don't know and had no time to try again so I could be entirely wrong about this.
So yesterday I tested these:
* all with my patched OpenBIOS ** all of them work fine with the 7400 CPU
- macos9 w/ -M mac99 and -cpu 7450
OS starts and then freezes. Upon further inspection I see that it has the 0x1000, 0x1100 and 0x1200 vectors implemented, but not the 7450 ones. It implements the 6xx SW TLB handler, i.e. it accesses SPR 976 instead of 980.
- macosx10 w/ -M mac99 and -cpu 7450
Shows the apple logo and then spins. Looking at the asm I don't see anything resembling the 7450 software TLB code. I'm calling it unsupported.
- debian 10 w/ -M mac99 and -cpu 7450
Bevare that -M mac99 is not matching the device tree as it has ADB instead of USB but claims to be PowerMac3,1 nevertheless. This is a silly default preserved for compatibility with some older OS X versions but to avoid problems it's better to always use -M mac99,via=pmu which is the closest to a real PowerMac3,1. There is some info on this and which option to use with which version at https://www.emaculation.com/doku.php/ppc-osx-on-qemu-for-osx
Boots linux and then spins. It has the vectors implemented, but it also uses different SPRs. The data misses come via 976, which is different from 7450, which uses only 980 (tlbmiss) for instruction and data.
- morphos w/ -M pegasos2 and -cpu 7447|7450
Hangs. It also has a different software TLB model implemented: Trying to read invalid spr 978 (0x3d2) at 00001100 Trying to read invalid spr 977 (0x3d1) at 0000110c Trying to read invalid spr 979 (0x3d3) at 0000115c Trying to read invalid spr 976 (0x3d0) at 00001188
So my initial impression that no OS supports the 7450 software TLB seems to match these findings and what people have said elsewhere in the thread.
I'm getting this with MorphOS on pegasos2: $ qemu-system-ppc -M pegasos2 -device ati-vga,romfile="" -kernel boot.img -cdrom morphos-3.15.iso -serial stdio -d unimp,guest_errors,int 2>&1 | grep --line-buffered -v '^Invalid read\|^hypercall\|^syscall\|^Raise exception.*0000000[48a]' Memory used before SYS_Init: 9MB i8259: level sensitive irq not supported i8259: level sensitive irq not supported unsupported keyboard cmd=0xaf PCI ATA/ATAPI Driver@2: PIO Mode 4 PCI ATA/ATAPI Driver@2: UDMA Mode 5 ide.device@2: QEMU QEMU DVD-ROM <CDROM> and it boots with the default 7400 CPU but with -cpu 7447 I get: Memory used before SYS_Init: 9MB i8259: level sensitive irq not supported i8259: level sensitive irq not supported Raise exception at 00051500 => 0000004f (00) Trying to read invalid spr 978 (0x3d2) at 00001100 Trying to read invalid spr 977 (0x3d1) at 0000110c Trying to read invalid spr 979 (0x3d3) at 0000115c Trying to read invalid spr 976 (0x3d0) at 00001188 ------------------------------------------------------------------------------ ExceptionThread: Exception Thread 0xbc8030 <System Init> ExceptionThread: Type 0x3 <Data Access> ExceptionThread: Illegal Data Access at 0x40000000 DSISR 0x40000000 <Read Access to a not existing Page> PC(SRR0) 0x00051500 MSR(SRR1) 0x0000b030 CR 00000884 XER 20000000 LR 0002166c CTR 0001a880 General Register Dump: GPR[0] 00021000 40000000 0000b030 0200b030 008a1c9c 1fc59000 00000000 00000000 GPR[8] 00be8718 00000000 00000000 00053e3c 0200b030 0d0d0d0d 0e0e0e0e 0f0f0f0f GPR[16] 10101010 11111111 12121212 13131313 14141414 15151515 16161616 00000002 GPR[24] 0081c1d0 00000000 10020010 00000000 10000013 00000004 00030000 00000001 FPSCR 00000000 FPECR 00000000 FPR[0] 0x0000000020000002 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[4] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[8] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[12] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[16] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[20] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[24] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 FPR[28] 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 Special Register Dump: IABR 00000000 DABR 00000000 PCM1 00000000 PMC2 00000000 PMC3 00000000 PMC4 00000000 OS Register Dump: SysCall_SRR0 00053e3c SysCall_SRR1 0200b030 SysCall_GPR2 008a17b0 SysCall_LR 0d0d0d0d SysCall_CR 00052624 SRR0[] 00000000 00000000 00000000 00000000 ------------------------------------------------------------------------------ StackFrame History: Stack 0x40000000 StackEnd 0xbc7030 LR 0x2166c Exception 4f seems to be POWERPC_EXCP_DLTLB which does not seem to happen with 7400 CPU, neither the SPR reads so there is some difference with 7447 but not sure what do these mean, I don't know much about these CPUs and did not read the docs. There is some debug code that can be enabled with DEBUG_SOFTWARE_TLB in target/ppc/excp_helper.c to get more info but I haven't tried that, looks like MorphOS is not expecting these so normally would not use it. The pegasos2 firmware seems to work but detects the CPU differently. With 7400: SmartFirmware: cpu0: PowerPC,G4 CPUClock 599 Mhz BUSClock 133 Mhz (Version 0x000C,0x0209) The 600 Mhz CPU clock is wrong but since the only two CPUs available were a 600 MHz G3 or a 1GHz 7447 it probably thinks anything not the 7447 or newer is a G3 that was running at 600 MHz. But with the 7447 it's also wrong: cpu0: PowerPC,74x7 CPUClock 1533 Mhz BUSClock 133 Mhz (Version 0x8002,0x0101) which should be 1 GHz (or 999 MHz), maybe the clock multiplier is not set correctly but not sure where it gets that from. Otherwise the firmware seems to work with 7447 CPU as well and get the same exception as above when tring boot MorphOS. Regards, BALATON Zoltan
On 24/11/2021 22:00, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
============================================================= OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
My experience of anything other than the default CPUs used on the PPC Mac machines is basically zero, so you're certainly in new territory :) I could live with your proposed solution c) although it would be nice to guard the extra vectors so that they remain uninitialised for the non-7450 CPUs. My main question is if the kernel itself doesn't support software TLBs then does adding the new code help at all? Or are you eventually planning for solution b) to improve QEMU's 7450 CPU emulation for developers without real hardware? ATB, Mark.
On 11/26/21 09:01, Mark Cave-Ayland wrote:
On 24/11/2021 22:00, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
>> ============================================================= >> OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
My experience of anything other than the default CPUs used on the PPC Mac machines is basically zero, so you're certainly in new territory :)
I could live with your proposed solution c) although it would be nice to guard the extra vectors so that they remain uninitialised for the non-7450 CPUs. My main question is if the kernel itself doesn't support software TLBs then does adding the new code help at all?
yes, it helps to boot Linux and MacOS (9 and 10) on those CPUs but you still need to replace the mmu model to POWERPC_MMU_32B in QEMU.
Or are you eventually planning for solution b) to improve QEMU's 7450 CPU emulation for developers without real hardware?
b) would be nice to have but since we don't have any images using it, may be it's time to drop support from QEMU. Thanks C.
On 26/11/2021 08:40, Cédric Le Goater wrote:
On 11/26/21 09:01, Mark Cave-Ayland wrote:
On 24/11/2021 22:00, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
>> ============================================================= >> OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
My experience of anything other than the default CPUs used on the PPC Mac machines is basically zero, so you're certainly in new territory :)
I could live with your proposed solution c) although it would be nice to guard the extra vectors so that they remain uninitialised for the non-7450 CPUs. My main question is if the kernel itself doesn't support software TLBs then does adding the new code help at all?
yes, it helps to boot Linux and MacOS (9 and 10) on those CPUs but you still need to replace the mmu model to POWERPC_MMU_32B in QEMU.
Or are you eventually planning for solution b) to improve QEMU's 7450 CPU emulation for developers without real hardware?
b) would be nice to have but since we don't have any images using it, may be it's time to drop support from QEMU.
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU. ATB, Mark.
Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> writes:
On 26/11/2021 08:40, Cédric Le Goater wrote:
On 11/26/21 09:01, Mark Cave-Ayland wrote:
On 24/11/2021 22:00, Fabiano Rosas wrote:
Fabiano Rosas <farosas@linux.ibm.com> writes:
Hi all,
We have this bug in QEMU which indicates that we haven't been able to run openbios on a 7450 cpu for quite a long time:
https://gitlab.com/qemu-project/qemu/-/issues/86
OK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7410
>> ============================================================= >> OpenBIOS 1.1 [Nov 1 2021 20:36] ...
NOK: $ ./qemu-system-ppc -serial mon:stdio -nographic -cpu 7450 -d int Raise exception at fff08cc4 => 0000004e (00) QEMU: Terminated
The actual issue is straightforward. There is a non-architected feature that QEMU has enabled by default that openbios doesn't know about. From the user manual:
"The MPC7540 has a set of implementation-specific registers, exceptions, and instructions that facilitate very efficient software searching of the page tables in memory for when software table searching is enabled (HID0[STEN] = 1). This section describes those resources and provides three example code sequences that can be used in a MPC7540 system for an efficient search of the translation tables in software. These three code sequences can be used as handlers for the three exceptions requiring access to the PTEs in the page tables in memory in this case-instruction TLB miss, data TLB miss on load, and data TLB miss on store exceptions."
The current state:
1) QEMU does not check HID0[STEN] and makes the feature always enabled by setting these cpus with the POWERPC_MMU_SOFT_74xx MMU model, instead of the generic POWERPC_MMU_32B.
2) openbios does not recognize the PVRs for those cpus and also does not have any handlers for the software TLB exceptions (vectors 0x1000, 0x1100, 0x1200).
Some assumptions (correct me if I'm wrong please):
- openbios is the only firmware we use for the following cpus: 7441, 7445, 7450, 7451, 7455, 7457, 7447, 7447a, 7448. - without openbios, we cannot have a guest running on these cpus.
So to bring 7450 back to life we would need to either:
a) find another firmware/guest OS code that supports the feature;
b) implement the switching of the feature in QEMU and have the guest code enable it only when supported. That would take some fiddling with the MMU code to: merge POWERPC_MMU_SOFT_74xx into POWERPC_MMU_32B, check the HID0[STEN] bit, figure out how to switch from HW TLB miss to SW TLB miss on demand, block access to the TLBMISS register (and others) when the feature is off, and so on;
c) leave the feature enabled in QEMU and implement the software TLB miss handlers in openbios. The UM provides sample code, so this is easy;
d) remove support for software TLB search for the 7450 family and switch the cpus to the POWERPC_MMU_32B model. This is by far the easiest solution, but could cause problems for any (which?) guest OS code that actually uses the feature. All of the existing code for the POWERPC_MMU_SOFT_74xx MMU model would probably be removed since it would be dead code then;
Option (c) seemed to me like a good compromise so this is a patch series for openbios doing that and also adding the necessary PVRs so we can get a working guest with these cpus without too much effort.
I have also a patch for QEMU adding basic sanity check tests for the 7400 and 7450 families. I'll send that separately to the QEMU ml.
Fabiano Rosas (2): ppc: Add support for MPC7450 software TLB miss interrupts ppc: Add PVRs for the MPC7450 family
arch/ppc/qemu/init.c | 52 ++++++++++ arch/ppc/qemu/start.S | 236 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 285 insertions(+), 3 deletions(-)
(Adding Mark because his email got somehow dropped from the original message)
So with these patches in OpenBIOS we could get a bit further and call into the Linux kernel using the same image as the one used for the 7400. However there seems to be no support for the 7450 software TLB in the kernel. There are only handlers for the 4xx, 8xx and 603 which are different code altogether. There's no mention of the TLBMISS and PTEHI/LO registers in the code as well.
Do we know of any guest OS that implements the 7450 software TLB at vectors 0x1000, 0x1100 and 0x1200? Otherwise replacing the POWERPC_MMU_SOFT_74xx model with POWERPC_MMU_32B might be the only way of getting an OS to run in the 7450 family.
My experience of anything other than the default CPUs used on the PPC Mac machines is basically zero, so you're certainly in new territory :)
I could live with your proposed solution c) although it would be nice to guard the extra vectors so that they remain uninitialised for the non-7450 CPUs. My main question is if the kernel itself doesn't support software TLBs then does adding the new code help at all?
The point of c) is to allow _something_ to run on the CPU with software TLB enabled. I think that would be enough to make sure the code doesn't rot too much. So more of a maintenance concern. About improving the OpenBIOS patch, no worries, I can certainly do that once we define a course of action.
yes, it helps to boot Linux and MacOS (9 and 10) on those CPUs but you still need to replace the mmu model to POWERPC_MMU_32B in QEMU.
Or are you eventually planning for solution b) to improve QEMU's 7450 CPU emulation for developers without real hardware?
b) would be nice to have but since we don't have any images using it, may be it's time to drop support from QEMU.
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU.
Yeah, that is a good point. Although the software TLB is well contained, so we could certainly document that our 7450s don't have that feature and call it a day. Does QEMU have any policy on how much of a machine is required to be implemented? I am more inclined to apply c) for now as I said, just to have some code running on the CPU and maybe document in a gitlab issue that we're lacking the runtime switch and eventually implement that. It's not like this is high traffic code anyway. It has been broken for 10+ years. That said, if Cédric and Daniel see more value in moving the 7450s to the POWERPC_MMU_32B I won't oppose.
ATB,
Mark.
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU.
Yeah, that is a good point. Although the software TLB is well contained, so we could certainly document that our 7450s don't have that feature and call it a day. Does QEMU have any policy on how much of a machine is required to be implemented?
I am more inclined to apply c) for now as I said, just to have some code running on the CPU and maybe document in a gitlab issue that we're lacking the runtime switch and eventually implement that. It's not like this is high traffic code anyway. It has been broken for 10+ years.
That said, if Cédric and Daniel see more value in moving the 7450s to the POWERPC_MMU_32B I won't oppose.
I am in favor of dropping unused code in QEMU and keeping the CPUs for which we have support in Linux using the POWERPC_MMU_32B in QEMU and the openbios patch. If we need SoftTLB support for the 74x CPUs in QEMU, we can always dig in the history. We can give FreeBSB a try also since they had support for the G4 : https://people.freebsd.org/~arved/stuff/minimac With the openbios patch, Linux boots fine under 7450, 7455, 7447 CPUs. Under 7448, it drops in xmon with a : kernel tried to execute exec-protected page (c07fdd98) - exploit attempt? (uid: 0) BUG: Unable to handle kernel instruction fetch Faulting instruction address: 0xc07fdd98 Vector: 400 (Instruction Access) at [f1019d30] pc: c07fdd98: __do_softirq+0x0/0x2f0 lr: c00516a4: irq_exit+0xbc/0xf8 sp: f1019df0 msr: 10001032 current = 0xc0d00000 pid = 1, comm = swapper This should be fixable. Thanks, C.
On Fri, 26 Nov 2021, Cédric Le Goater wrote:
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU.
Yeah, that is a good point. Although the software TLB is well contained, so we could certainly document that our 7450s don't have that feature and call it a day. Does QEMU have any policy on how much of a machine is required to be implemented?
I am more inclined to apply c) for now as I said, just to have some code running on the CPU and maybe document in a gitlab issue that we're lacking the runtime switch and eventually implement that. It's not like this is high traffic code anyway. It has been broken for 10+ years.
That said, if Cédric and Daniel see more value in moving the 7450s to the POWERPC_MMU_32B I won't oppose.
I am in favor of dropping unused code in QEMU and keeping the CPUs for which we have support in Linux using the POWERPC_MMU_32B in QEMU and the openbios patch. If we need SoftTLB support for the 74x CPUs in QEMU, we can always dig in the history.
If we can't find a guest that needs it and can be used to test with it may be OK to remove it for now but digging the history may not be easy if somebody later comes along with a guest that would need this. Likely they would just go away when finding it's not supported or maybe try to redo it from scratch and not think of checking history first. So if you drop it maybe leave a one line comment at some obvious place saying something like "74xx soft TLB removed in commit xxxxx" to make it simpler for those who may want to resurrect it later. Regards, BALATON Zoltan
We can give FreeBSB a try also since they had support for the G4 :
https://people.freebsd.org/~arved/stuff/minimac
With the openbios patch, Linux boots fine under 7450, 7455, 7447 CPUs.
Under 7448, it drops in xmon with a : kernel tried to execute exec-protected page (c07fdd98) - exploit attempt? (uid: 0) BUG: Unable to handle kernel instruction fetch Faulting instruction address: 0xc07fdd98 Vector: 400 (Instruction Access) at [f1019d30] pc: c07fdd98: __do_softirq+0x0/0x2f0 lr: c00516a4: irq_exit+0xbc/0xf8 sp: f1019df0 msr: 10001032 current = 0xc0d00000 pid = 1, comm = swapper
This should be fixable.
Thanks,
C.
Cédric Le Goater <clg@kaod.org> writes:
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU.
Yeah, that is a good point. Although the software TLB is well contained, so we could certainly document that our 7450s don't have that feature and call it a day. Does QEMU have any policy on how much of a machine is required to be implemented?
I am more inclined to apply c) for now as I said, just to have some code running on the CPU and maybe document in a gitlab issue that we're lacking the runtime switch and eventually implement that. It's not like this is high traffic code anyway. It has been broken for 10+ years.
That said, if Cédric and Daniel see more value in moving the 7450s to the POWERPC_MMU_32B I won't oppose.
I am in favor of dropping unused code in QEMU and keeping the CPUs for which we have support in Linux using the POWERPC_MMU_32B in QEMU and the openbios patch. If we need SoftTLB support for the 74x CPUs in QEMU, we can always dig in the history.
Ack. I'll send a v2.
We can give FreeBSB a try also since they had support for the G4 :
https://people.freebsd.org/~arved/stuff/minimac
With the openbios patch, Linux boots fine under 7450, 7455, 7447 CPUs.
Under 7448, it drops in xmon with a :
kernel tried to execute exec-protected page (c07fdd98) - exploit attempt? (uid: 0) BUG: Unable to handle kernel instruction fetch Faulting instruction address: 0xc07fdd98 Vector: 400 (Instruction Access) at [f1019d30] pc: c07fdd98: __do_softirq+0x0/0x2f0 lr: c00516a4: irq_exit+0xbc/0xf8 sp: f1019df0 msr: 10001032 current = 0xc0d00000 pid = 1, comm = swapper
I see two possible issues: 1) The 7448 is configured as a 7400 in QEMU (cpu-models.c), so it will behave differently from the 7450s. The user manual seems to indicate it is closer to a 7445 than a 7400. We need to double check what is correct. 2) OpenBIOS already has support for the 7448 PVR without my patch, but given that no other cpu of the 7450 family is supported, I'd say this is accidental. The mask that OpenBIOS uses for e600/MPC86xx is: .iu_version = 0x80040000, .name = "PowerPC,MPC86xx", And the verification: iu_version = mfpvr() & 0xffff0000; for (i = 0; i < sizeof(ppc_defs) / sizeof(struct cpudef); i++) { if (iu_version == ppc_defs[i].iu_version) return &ppc_defs[i]; } printk("Unknown cpu (pvr %x), freezing!\n", iu_version); But QEMU says the PVRs are as follows: CPU_POWERPC_e600 = 0x80040010, #define CPU_POWERPC_MPC8610 CPU_POWERPC_e600 #define CPU_POWERPC_MPC8641 CPU_POWERPC_e600 #define CPU_POWERPC_MPC8641D CPU_POWERPC_e600 CPU_POWERPC_7448_v10 = 0x80040100, CPU_POWERPC_7448_v11 = 0x80040101, CPU_POWERPC_7448_v20 = 0x80040200, CPU_POWERPC_7448_v21 = 0x80040201, So by applying the mask, OpenBIOS is matching both 0x80040100 and 0x80040010 when it looks like it only wants to match the latter.
On 11/29/21 16:28, Fabiano Rosas wrote:
Cédric Le Goater <clg@kaod.org> writes:
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU.
Yeah, that is a good point. Although the software TLB is well contained, so we could certainly document that our 7450s don't have that feature and call it a day. Does QEMU have any policy on how much of a machine is required to be implemented?
I am more inclined to apply c) for now as I said, just to have some code running on the CPU and maybe document in a gitlab issue that we're lacking the runtime switch and eventually implement that. It's not like this is high traffic code anyway. It has been broken for 10+ years.
That said, if Cédric and Daniel see more value in moving the 7450s to the POWERPC_MMU_32B I won't oppose.
I am in favor of dropping unused code in QEMU and keeping the CPUs for which we have support in Linux using the POWERPC_MMU_32B in QEMU and the openbios patch. If we need SoftTLB support for the 74x CPUs in QEMU, we can always dig in the history.
Ack. I'll send a v2.
We can give FreeBSB a try also since they had support for the G4 :
https://people.freebsd.org/~arved/stuff/minimac
With the openbios patch, Linux boots fine under 7450, 7455, 7447 CPUs.
Under 7448, it drops in xmon with a :
kernel tried to execute exec-protected page (c07fdd98) - exploit attempt? (uid: 0) BUG: Unable to handle kernel instruction fetch Faulting instruction address: 0xc07fdd98 Vector: 400 (Instruction Access) at [f1019d30] pc: c07fdd98: __do_softirq+0x0/0x2f0 lr: c00516a4: irq_exit+0xbc/0xf8 sp: f1019df0 msr: 10001032 current = 0xc0d00000 pid = 1, comm = swapper
I see two possible issues:
1) The 7448 is configured as a 7400 in QEMU (cpu-models.c), so it will behave differently from the 7450s. The user manual seems to indicate it is closer to a 7445 than a 7400. We need to double check what is correct.
2) OpenBIOS already has support for the 7448 PVR without my patch, but given that no other cpu of the 7450 family is supported, I'd say this is accidental. The mask that OpenBIOS uses for e600/MPC86xx is:
.iu_version = 0x80040000, .name = "PowerPC,MPC86xx",
And the verification:
iu_version = mfpvr() & 0xffff0000;
for (i = 0; i < sizeof(ppc_defs) / sizeof(struct cpudef); i++) { if (iu_version == ppc_defs[i].iu_version) return &ppc_defs[i]; } printk("Unknown cpu (pvr %x), freezing!\n", iu_version);
But QEMU says the PVRs are as follows:
CPU_POWERPC_e600 = 0x80040010, #define CPU_POWERPC_MPC8610 CPU_POWERPC_e600 #define CPU_POWERPC_MPC8641 CPU_POWERPC_e600 #define CPU_POWERPC_MPC8641D CPU_POWERPC_e600
CPU_POWERPC_7448_v10 = 0x80040100, CPU_POWERPC_7448_v11 = 0x80040101, CPU_POWERPC_7448_v20 = 0x80040200, CPU_POWERPC_7448_v21 = 0x80040201,
So by applying the mask, OpenBIOS is matching both 0x80040100 and 0x80040010 when it looks like it only wants to match the latter.
Hello, a quick status on how QEMU has evolved regarding this topic. We have merged : a09410ed1fb8 ("target/ppc: Remove the software TLB model of 7450 CPUs") and switched MMU model to POWERPC_MMU_32B in : 1da666cd8e79 ("target/ppc: Disable software TLB for the 7450 family") With the two patches of this series, we can boot the latest linux on all 7450 CPUs. With an extra small fix on the CPU table, 7448 boots. Thanks, C.
Cédric Le Goater <clg@kaod.org> writes:
On 11/29/21 16:28, Fabiano Rosas wrote:
Cédric Le Goater <clg@kaod.org> writes:
Right. If we're doing this to say "I can boot a kernel with a 7450 cpu in QEMU" but the implementation is different from real hardware, then I'm not sure what the real value is. That effectively leaves option b) if someone is willing to do the work, or as you say to simply remove the code from QEMU.
Yeah, that is a good point. Although the software TLB is well contained, so we could certainly document that our 7450s don't have that feature and call it a day. Does QEMU have any policy on how much of a machine is required to be implemented?
I am more inclined to apply c) for now as I said, just to have some code running on the CPU and maybe document in a gitlab issue that we're lacking the runtime switch and eventually implement that. It's not like this is high traffic code anyway. It has been broken for 10+ years.
That said, if Cédric and Daniel see more value in moving the 7450s to the POWERPC_MMU_32B I won't oppose.
I am in favor of dropping unused code in QEMU and keeping the CPUs for which we have support in Linux using the POWERPC_MMU_32B in QEMU and the openbios patch. If we need SoftTLB support for the 74x CPUs in QEMU, we can always dig in the history.
Ack. I'll send a v2.
We can give FreeBSB a try also since they had support for the G4 :
https://people.freebsd.org/~arved/stuff/minimac
With the openbios patch, Linux boots fine under 7450, 7455, 7447 CPUs.
Under 7448, it drops in xmon with a :
kernel tried to execute exec-protected page (c07fdd98) - exploit attempt? (uid: 0) BUG: Unable to handle kernel instruction fetch Faulting instruction address: 0xc07fdd98 Vector: 400 (Instruction Access) at [f1019d30] pc: c07fdd98: __do_softirq+0x0/0x2f0 lr: c00516a4: irq_exit+0xbc/0xf8 sp: f1019df0 msr: 10001032 current = 0xc0d00000 pid = 1, comm = swapper
I see two possible issues:
1) The 7448 is configured as a 7400 in QEMU (cpu-models.c), so it will behave differently from the 7450s. The user manual seems to indicate it is closer to a 7445 than a 7400. We need to double check what is correct.
2) OpenBIOS already has support for the 7448 PVR without my patch, but given that no other cpu of the 7450 family is supported, I'd say this is accidental. The mask that OpenBIOS uses for e600/MPC86xx is:
.iu_version = 0x80040000, .name = "PowerPC,MPC86xx",
And the verification:
iu_version = mfpvr() & 0xffff0000;
for (i = 0; i < sizeof(ppc_defs) / sizeof(struct cpudef); i++) { if (iu_version == ppc_defs[i].iu_version) return &ppc_defs[i]; } printk("Unknown cpu (pvr %x), freezing!\n", iu_version);
But QEMU says the PVRs are as follows:
CPU_POWERPC_e600 = 0x80040010, #define CPU_POWERPC_MPC8610 CPU_POWERPC_e600 #define CPU_POWERPC_MPC8641 CPU_POWERPC_e600 #define CPU_POWERPC_MPC8641D CPU_POWERPC_e600
CPU_POWERPC_7448_v10 = 0x80040100, CPU_POWERPC_7448_v11 = 0x80040101, CPU_POWERPC_7448_v20 = 0x80040200, CPU_POWERPC_7448_v21 = 0x80040201,
So by applying the mask, OpenBIOS is matching both 0x80040100 and 0x80040010 when it looks like it only wants to match the latter.
Hello,
a quick status on how QEMU has evolved regarding this topic.
We have merged :
a09410ed1fb8 ("target/ppc: Remove the software TLB model of 7450 CPUs")
and switched MMU model to POWERPC_MMU_32B in :
1da666cd8e79 ("target/ppc: Disable software TLB for the 7450 family")
With the two patches of this series, we can boot the latest linux on all 7450 CPUs. With an extra small fix on the CPU table, 7448 boots.
A small correction, we only need OpenBIOS to include patch 2: "ppc: Add PVRs for the MPC7450 family".
Hello,
a quick status on how QEMU has evolved regarding this topic.
We have merged :
a09410ed1fb8 ("target/ppc: Remove the software TLB model of 7450 CPUs")
and switched MMU model to POWERPC_MMU_32B in :
1da666cd8e79 ("target/ppc: Disable software TLB for the 7450 family")
With the two patches of this series, we can boot the latest linux on all 7450 CPUs. With an extra small fix on the CPU table, 7448 boots.
A small correction, we only need OpenBIOS to include patch 2: "ppc: Add PVRs for the MPC7450 family".
Yep we don't need the vectors anymore. But I think I remember Mark saying it would be nice to keep the implementation. I have an extra patch for 7448 tough. Thanks, C.
On 07/01/2022 13:51, Cédric Le Goater wrote:
Hello,
a quick status on how QEMU has evolved regarding this topic.
We have merged :
a09410ed1fb8 ("target/ppc: Remove the software TLB model of 7450 CPUs")
and switched MMU model to POWERPC_MMU_32B in :
1da666cd8e79 ("target/ppc: Disable software TLB for the 7450 family")
With the two patches of this series, we can boot the latest linux on all 7450 CPUs. With an extra small fix on the CPU table, 7448 boots.
A small correction, we only need OpenBIOS to include patch 2: "ppc: Add PVRs for the MPC7450 family".
Yep we don't need the vectors anymore. But I think I remember Mark saying it would be nice to keep the implementation.
I have an extra patch for 7448 tough.
Could you send the updated OpenBIOS series as a v2 (even if it's part of an existing series) just to make sure there is no ambiguity here? If the vectors aren't now being used with your latest QEMU changes then I'm less keen to merge them, simply because there is no easy way to confirm if they are working correctly without having access to real hardware. ATB, Mark.
participants (5)
-
BALATON Zoltan -
Cédric Le Goater -
Fabiano Rosas -
Mark Cave-Ayland -
Segher Boessenkool