On 2011-08-03 12:07, Vasilis Liaskovitis wrote:
When rebooting after a CPU hotplug in qemu-kvm, Seabios can get stuck in smp_probe(). Normally cmos_smp_count is read from cmos and the smp_ap_boot_code is run on all cpus except bootstrap. The expected result is CountCPUs == cmos_smp_count + 1. After a cpu hotplug, more than cmos_smp_count cpus are active, so we get a situation where CountCPUs > cmos_smp_count + 1 and Seabios keeps looping forever in smp_probe. In some cases, the while loop exit condition is tested before CountCPUs gets larger (i.e. before smp_ap_boot_code runs on all CPUs), so the hang is not always reproducible.
This patch introduces a new fw_cfg variable called hotplugged_cpus that gets updated from qemu-kvm hoplug code. Seabios reads this variable on each call to smp_probe() and adjusts the expected number of online CPUs.
The qemu-kvm part of this patch is against Jan Kiszka's cpu-hotplug tree: git://git.kiszka.org/qemu-kvm.git queues/cpu-hotplug tested with qemu-system-x86_64.
An alternative to this patch would be to update the smp_cpus variable in qemu-kvm and do a "cmos update" to 0x5f from the cpu-hotplug code. Access to the rtc_state (cmos device) would be required in hw/acpi_piix4.c. This way no change to Seabios would be required.
...
src/paravirt.c | 12 ++++++++++++ src/paravirt.h | 2 ++ src/smp.c | 6 ++++-- 3 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/paravirt.c b/src/paravirt.c index 9cf77de..3367609 100644 --- a/src/paravirt.c +++ b/src/paravirt.c @@ -305,6 +305,18 @@ u16 qemu_cfg_get_max_cpus(void) return cnt; }
+u16 qemu_cfg_get_hplug_cpus(void) +{
- u16 cnt;
- if (!qemu_cfg_present)
return 0;- qemu_cfg_read_entry(&cnt, QEMU_CFG_HPLUG_CPUS, sizeof(cnt));
Why can't Seabios read to true number online CPUs from the PIIX4 device? The information is there already, no need for addition PV here.
Jan
On Wed, Aug 03, 2011 at 12:37:13PM +0200, Jan Kiszka wrote:
On 2011-08-03 12:07, Vasilis Liaskovitis wrote:
When rebooting after a CPU hotplug in qemu-kvm, Seabios can get stuck in smp_probe(). Normally cmos_smp_count is read from cmos and the smp_ap_boot_code is run on all cpus except bootstrap. The expected result is CountCPUs == cmos_smp_count + 1. After a cpu hotplug, more than cmos_smp_count cpus are active, so we get a situation where CountCPUs > cmos_smp_count + 1 and Seabios keeps looping forever in smp_probe. In some cases, the while loop exit condition is tested before CountCPUs gets larger (i.e. before smp_ap_boot_code runs on all CPUs), so the hang is not always reproducible.
This patch introduces a new fw_cfg variable called hotplugged_cpus that gets updated from qemu-kvm hoplug code. Seabios reads this variable on each call to smp_probe() and adjusts the expected number of online CPUs.
The qemu-kvm part of this patch is against Jan Kiszka's cpu-hotplug tree: git://git.kiszka.org/qemu-kvm.git queues/cpu-hotplug tested with qemu-system-x86_64.
An alternative to this patch would be to update the smp_cpus variable in qemu-kvm and do a "cmos update" to 0x5f from the cpu-hotplug code. Access to the rtc_state (cmos device) would be required in hw/acpi_piix4.c. This way no change to Seabios would be required.
...
src/paravirt.c | 12 ++++++++++++ src/paravirt.h | 2 ++ src/smp.c | 6 ++++-- 3 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/paravirt.c b/src/paravirt.c index 9cf77de..3367609 100644 --- a/src/paravirt.c +++ b/src/paravirt.c @@ -305,6 +305,18 @@ u16 qemu_cfg_get_max_cpus(void) return cnt; }
+u16 qemu_cfg_get_hplug_cpus(void) +{
- u16 cnt;
- if (!qemu_cfg_present)
return 0;- qemu_cfg_read_entry(&cnt, QEMU_CFG_HPLUG_CPUS, sizeof(cnt));
Why can't Seabios read to true number online CPUs from the PIIX4 device? The information is there already, no need for addition PV here.
Where is it in PIIX4 device?
-- Gleb.
On 2011-08-03 12:38, Gleb Natapov wrote:
On Wed, Aug 03, 2011 at 12:37:13PM +0200, Jan Kiszka wrote:
On 2011-08-03 12:07, Vasilis Liaskovitis wrote:
When rebooting after a CPU hotplug in qemu-kvm, Seabios can get stuck in smp_probe(). Normally cmos_smp_count is read from cmos and the smp_ap_boot_code is run on all cpus except bootstrap. The expected result is CountCPUs == cmos_smp_count + 1. After a cpu hotplug, more than cmos_smp_count cpus are active, so we get a situation where CountCPUs > cmos_smp_count + 1 and Seabios keeps looping forever in smp_probe. In some cases, the while loop exit condition is tested before CountCPUs gets larger (i.e. before smp_ap_boot_code runs on all CPUs), so the hang is not always reproducible.
This patch introduces a new fw_cfg variable called hotplugged_cpus that gets updated from qemu-kvm hoplug code. Seabios reads this variable on each call to smp_probe() and adjusts the expected number of online CPUs.
The qemu-kvm part of this patch is against Jan Kiszka's cpu-hotplug tree: git://git.kiszka.org/qemu-kvm.git queues/cpu-hotplug tested with qemu-system-x86_64.
An alternative to this patch would be to update the smp_cpus variable in qemu-kvm and do a "cmos update" to 0x5f from the cpu-hotplug code. Access to the rtc_state (cmos device) would be required in hw/acpi_piix4.c. This way no change to Seabios would be required.
...
src/paravirt.c | 12 ++++++++++++ src/paravirt.h | 2 ++ src/smp.c | 6 ++++-- 3 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/paravirt.c b/src/paravirt.c index 9cf77de..3367609 100644 --- a/src/paravirt.c +++ b/src/paravirt.c @@ -305,6 +305,18 @@ u16 qemu_cfg_get_max_cpus(void) return cnt; }
+u16 qemu_cfg_get_hplug_cpus(void) +{
- u16 cnt;
- if (!qemu_cfg_present)
return 0;- qemu_cfg_read_entry(&cnt, QEMU_CFG_HPLUG_CPUS, sizeof(cnt));
Why can't Seabios read to true number online CPUs from the PIIX4 device? The information is there already, no need for addition PV here.
Where is it in PIIX4 device?
PROC registers (or however they are called).
Jan
On Wed, Aug 03, 2011 at 12:42:11PM +0200, Jan Kiszka wrote:
Why can't Seabios read to true number online CPUs from the PIIX4 device? The information is there already, no need for addition PV here.
Where is it in PIIX4 device?
PROC registers (or however they are called).
In qemu-kvm, the cpus_sts bitmap array in PIIX4PMState/ACPIGPE has the true number of online CPUS. This is accessed from the DSDT hotplug method in Seabios as "OperationRegion SystemIO" with address 0xaf00. Is this i/o address in the piix4 spec? How can it be accessed from the rest of SeaBIOS? It seems to reside in ACPI_PM space.
- Vasilis
-
Gleb Natapov -
Jan Kiszka -
Vasilis Liaskovitis