SeaBIOS September 2012

seabios@seabios.org

27 participants
112 discussions

[RFC PATCH v3 09/19] pc: Add dimm paravirt SRAT info
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

The numa_fw_cfg paravirt interface is extended to include SRAT information for all hotplug-able dimms. There are 3 words for each hotplug-able memory slot, denoting start address, size and node proximity. The new info is appended after existing numa info, so that the fw_cfg layout does not break. This information is used by Seabios to build hotplug memory device objects at runtime. nb_numa_nodes is set to 1 by default (not 0), so that we always pass srat info to SeaBIOS. v1->v2: Dimm SRAT info (#dimms) is appended at end of existing numa fw_cfg in order not to break existing layout Documentation of the new fwcfg layout is included in docs/specs/fwcfg.txt Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- docs/specs/fwcfg.txt | 28 ++++++++++++++++++++++++++++ hw/pc.c | 14 ++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-) create mode 100644 docs/specs/fwcfg.txt diff --git a/docs/specs/fwcfg.txt b/docs/specs/fwcfg.txt new file mode 100644 index 0000000..55f96d9 --- /dev/null +++ b/docs/specs/fwcfg.txt @@ -0,0 +1,28 @@ +QEMU<->BIOS Paravirt Documentation +-------------------------------------- + +This document describes paravirt data structures passed from QEMU to BIOS. + +FW_CFG_NUMA paravirt info +-------------------- +The SRAT info passed from QEMU to BIOS has the following layout: + +----------------------------------------------------------------------------------------------- +#nodes | cpu0_pxm | cpu1_pxm | ... | cpulast_pxm | node0_mem | node1_mem | ... | nodelast_mem + +----------------------------------------------------------------------------------------------- +#dimms | dimm0_start | dimm0_sz | dimm0_pxm | ... | dimmlast_start | dimmlast_sz | dimmlast_pxm + +Entry 0 contains the number of numa nodes (nb_numa_nodes). + +Entries 1..max_cpus: The next max_cpus entries describe node proximity for each +one of the vCPUs in the system. + +Entries max_cpus+1..max_cpus+nb_numa_nodes+1: The next nb_numa_nodes entries +describe the memory size for each one of the NUMA nodes in the system. + +Entry max_cpus+nb_numa_nodes+1 contains the number of memory dimms (nb_hp_dimms) + +The last 3 * nb_hp_dimms entries are organized in triplets: Each triplet contains +the physical address offset, size (in bytes), and node proximity for the +respective dimm. diff --git a/hw/pc.c b/hw/pc.c index 2c9664d..f2604ae 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -598,6 +598,7 @@ static void *bochs_bios_init(void) uint8_t *smbios_table; size_t smbios_len; uint64_t *numa_fw_cfg; + uint64_t *hp_dimms_fw_cfg; int i, j; register_ioport_write(0x400, 1, 2, bochs_bios_write, NULL); @@ -632,8 +633,10 @@ static void *bochs_bios_init(void) /* allocate memory for the NUMA channel: one (64bit) word for the number * of nodes, one word for each VCPU->node and one word for each node to * hold the amount of memory. + * Finally one word for the number of hotplug memory slots and three words + * for each hotplug memory slot (start address, size and node proximity). */ - numa_fw_cfg = g_malloc0((1 + max_cpus + nb_numa_nodes) * 8); + numa_fw_cfg = g_malloc0((2 + max_cpus + nb_numa_nodes + 3 * nb_hp_dimms) * 8); numa_fw_cfg[0] = cpu_to_le64(nb_numa_nodes); for (i = 0; i < max_cpus; i++) { for (j = 0; j < nb_numa_nodes; j++) { @@ -646,8 +649,15 @@ static void *bochs_bios_init(void) for (i = 0; i < nb_numa_nodes; i++) { numa_fw_cfg[max_cpus + 1 + i] = cpu_to_le64(node_mem[i]); } + + numa_fw_cfg[1 + max_cpus + nb_numa_nodes] = cpu_to_le64(nb_hp_dimms); + + hp_dimms_fw_cfg = numa_fw_cfg + 2 + max_cpus + nb_numa_nodes; + if (nb_hp_dimms) + setup_fwcfg_hp_dimms(hp_dimms_fw_cfg); + fw_cfg_add_bytes(fw_cfg, FW_CFG_NUMA, (uint8_t *)numa_fw_cfg, - (1 + max_cpus + nb_numa_nodes) * 8); + (2 + max_cpus + nb_numa_nodes + 3 * nb_hp_dimms) * 8); return fw_cfg; } -- 1.7.9

1 0

[RFC PATCH v3 08/19] pc: calculate dimm physical addresses and adjust memory map
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

Dimm physical address offsets are calculated automatically and memory map is adjusted accordingly. If a DIMM can fit before the PCI_HOLE_START (currently 0xe0000000), it will be added normally, otherwise its physical address will be above 4GB. Also create memory bus on i440fx-pcihost device. Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- hw/pc.c | 41 +++++++++++++++++++++++++++++++++++++++++ hw/pc.h | 6 ++++++ hw/pc_piix.c | 20 ++++++++++++++------ vl.c | 1 + 4 files changed, 62 insertions(+), 6 deletions(-) diff --git a/hw/pc.c b/hw/pc.c index 112739a..2c9664d 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -52,6 +52,7 @@ #include "arch_init.h" #include "bitmap.h" #include "vga-pci.h" +#include "dimm.h" /* output Bochs bios info messages */ //#define DEBUG_BIOS @@ -93,6 +94,9 @@ struct e820_table { static struct e820_table e820_table; struct hpet_fw_config hpet_cfg = {.count = UINT8_MAX}; +ram_addr_t below_4g_hp_mem_size = 0; +ram_addr_t above_4g_hp_mem_size = 0; +extern target_phys_addr_t ram_hp_offset; void gsi_handler(void *opaque, int n, int level) { GSIState *s = opaque; @@ -1160,3 +1164,40 @@ void pc_pci_device_init(PCIBus *pci_bus) pci_create_simple(pci_bus, -1, "lsi53c895a"); } } + + +/* Function to configure memory offsets of hotpluggable dimms */ + +target_phys_addr_t pc_set_hp_memory_offset(uint64_t size) +{ + target_phys_addr_t ret; + + /* on first call, initialize ram_hp_offset */ + if (!ram_hp_offset) { + if (ram_size >= PCI_HOLE_START ) { + ram_hp_offset = 0x100000000LL + (ram_size - PCI_HOLE_START); + } else { + ram_hp_offset = ram_size; + } + } + + if (ram_hp_offset >= 0x100000000LL) { + ret = ram_hp_offset; + above_4g_hp_mem_size += size; + ram_hp_offset += size; + } + /* if dimm fits before pci hole, append it normally */ + else if (ram_hp_offset + size <= PCI_HOLE_START) { + ret = ram_hp_offset; + below_4g_hp_mem_size += size; + ram_hp_offset += size; + } + /* otherwise place it above 4GB */ + else { + ret = 0x100000000LL; + above_4g_hp_mem_size += size; + ram_hp_offset = 0x100000000LL + size; + } + + return ret; +} diff --git a/hw/pc.h b/hw/pc.h index e4db071..f3304fc 100644 --- a/hw/pc.h +++ b/hw/pc.h @@ -10,6 +10,7 @@ #include "memory.h" #include "ioapic.h" +#define PCI_HOLE_START 0xe0000000 /* PC-style peripherals (also used by other machines). */ /* serial.c */ @@ -214,6 +215,11 @@ static inline bool isa_ne2000_init(ISABus *bus, int base, int irq, NICInfo *nd) /* pc_sysfw.c */ void pc_system_firmware_init(MemoryRegion *rom_memory); +/* memory hotplug */ +target_phys_addr_t pc_set_hp_memory_offset(uint64_t size); +extern ram_addr_t below_4g_hp_mem_size; +extern ram_addr_t above_4g_hp_mem_size; + /* e820 types */ #define E820_RAM 1 #define E820_RESERVED 2 diff --git a/hw/pc_piix.c b/hw/pc_piix.c index 88ff041..d1fd276 100644 --- a/hw/pc_piix.c +++ b/hw/pc_piix.c @@ -43,6 +43,7 @@ #include "xen.h" #include "memory.h" #include "exec-memory.h" +#include "dimm.h" #ifdef CONFIG_XEN # include <xen/hvm/hvm_info_table.h> #endif @@ -155,9 +156,9 @@ static void pc_init1(MemoryRegion *system_memory, kvmclock_create(); } - if (ram_size >= 0xe0000000 ) { - above_4g_mem_size = ram_size - 0xe0000000; - below_4g_mem_size = 0xe0000000; + if (ram_size >= PCI_HOLE_START ) { + above_4g_mem_size = ram_size - PCI_HOLE_START; + below_4g_mem_size = PCI_HOLE_START; } else { above_4g_mem_size = 0; below_4g_mem_size = ram_size; @@ -172,6 +173,9 @@ static void pc_init1(MemoryRegion *system_memory, rom_memory = system_memory; } + /* adjust memory map for hotplug dimms */ + dimm_calc_offsets(pc_set_hp_memory_offset); + /* allocate ram and load rom/bios */ if (!xen_enabled()) { fw_cfg = pc_memory_init(system_memory, @@ -192,9 +196,11 @@ static void pc_init1(MemoryRegion *system_memory, if (pci_enabled) { pci_bus = i440fx_init(&i440fx_state, &piix3_devfn, &isa_bus, gsi, system_memory, system_io, ram_size, - below_4g_mem_size, - 0x100000000ULL - below_4g_mem_size, - 0x100000000ULL + above_4g_mem_size, + below_4g_mem_size + below_4g_hp_mem_size, + 0x100000000ULL - below_4g_mem_size + - below_4g_hp_mem_size, + 0x100000000ULL + above_4g_mem_size + + above_4g_hp_mem_size, (sizeof(target_phys_addr_t) == 4 ? 0 : ((uint64_t)1 << 62)), @@ -223,6 +229,8 @@ static void pc_init1(MemoryRegion *system_memory, ioapic_init(gsi_state); } + main_memory_bus_create(object_resolve_path("i440fx", NULL)); + pc_register_ferr_irq(gsi[13]); pc_vga_init(isa_bus, pci_enabled ? pci_bus : NULL); diff --git a/vl.c b/vl.c index af1745c..2282910 100644 --- a/vl.c +++ b/vl.c @@ -184,6 +184,7 @@ DisplayType display_type = DT_DEFAULT; int display_remote = 0; const char* keyboard_layout = NULL; ram_addr_t ram_size; +ram_addr_t ram_hp_offset; const char *mem_path = NULL; #ifdef MAP_POPULATE int mem_prealloc = 0; /* force preallocation of physical target memory */ -- 1.7.9

1 0

[RFC PATCH v3 07/19] acpi_piix4: Implement memory device hotplug registers
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

A 32-byte register is used to present up to 256 hotplug-able memory devices to BIOS and OSPM. Hot-add and hot-remove functions trigger an ACPI hotplug event through these. Only reads are allowed from these registers. An ACPI hot-remove event but needs to wait for OSPM to eject the device. We use a single-byte register to know when OSPM has called the _EJ function for a particular dimm. A write to this byte will depopulate the respective dimm. Only writes are allowed to this byte. v1->v2: mems_sts address moved from 0xaf20 to 0xaf80 (to accomodate more space for cpu-hotplugging in the future). _EJ array is reduced to a single byte. Add documentation in docs/specs/acpi_hotplug.txt v2->v3: minor name changes Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- docs/specs/acpi_hotplug.txt | 22 +++++++++++++ hw/acpi_piix4.c | 73 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 91 insertions(+), 4 deletions(-) create mode 100644 docs/specs/acpi_hotplug.txt diff --git a/docs/specs/acpi_hotplug.txt b/docs/specs/acpi_hotplug.txt new file mode 100644 index 0000000..cf86242 --- /dev/null +++ b/docs/specs/acpi_hotplug.txt @@ -0,0 +1,22 @@ +QEMU<->ACPI BIOS hotplug interface +-------------------------------------- +This document describes the interface between QEMU and the ACPI BIOS for non-PCI +space. For the PCI interface please look at docs/specs/acpi_pci_hotplug.txt + +QEMU<->ACPI BIOS memory hotplug interface +-------------------------------------- + +Memory Dimm status array (IO port 0xaf80-0xaf9f, 1-byte access): +--------------------------------------------------------------- +Dimm hot-plug notification pending. One bit per slot. + +Read by ACPI BIOS GPE.3 handler to notify OS of memory hot-add or hot-remove +events. Read-only. + +Memory Dimm ejection success notification (IO port 0xafa0, 1-byte access): +--------------------------------------------------------------- +Dimm hot-remove _EJ0 notification. Byte value indicates Dimm slot that was +ejected. + +Written by ACPI memory device _EJ0 method to notify qemu of successfull +hot-removal. Write-only. diff --git a/hw/acpi_piix4.c b/hw/acpi_piix4.c index c56220b..8776669 100644 --- a/hw/acpi_piix4.c +++ b/hw/acpi_piix4.c @@ -28,6 +28,8 @@ #include "range.h" #include "ioport.h" #include "fw_cfg.h" +#include "sysbus.h" +#include "dimm.h" //#define DEBUG @@ -45,9 +47,15 @@ #define PCI_DOWN_BASE 0xae04 #define PCI_EJ_BASE 0xae08 #define PCI_RMV_BASE 0xae0c +#define MEM_BASE 0xaf80 +#define MEM_EJ_BASE 0xafa0 +#define PIIX4_MEM_HOTPLUG_STATUS 8 #define PIIX4_PCI_HOTPLUG_STATUS 2 +struct gpe_regs { + uint8_t mems_sts[DIMM_BITMAP_BYTES]; +}; struct pci_status { uint32_t up; /* deprecated, maintained for migration compatibility */ uint32_t down; @@ -69,6 +77,7 @@ typedef struct PIIX4PMState { Notifier machine_ready; /* for pci hotplug */ + struct gpe_regs gperegs; struct pci_status pci0_status; uint32_t pci0_hotplug_enable; uint32_t pci0_slot_device_present; @@ -93,8 +102,8 @@ static void pm_update_sci(PIIX4PMState *s) ACPI_BITMASK_POWER_BUTTON_ENABLE | ACPI_BITMASK_GLOBAL_LOCK_ENABLE | ACPI_BITMASK_TIMER_ENABLE)) != 0) || - (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) - & PIIX4_PCI_HOTPLUG_STATUS) != 0); + (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) & + (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_MEM_HOTPLUG_STATUS)) != 0); qemu_set_irq(s->irq, sci_level); /* schedule a timer interruption if needed */ @@ -499,7 +508,16 @@ type_init(piix4_pm_register_types) static uint32_t gpe_readb(void *opaque, uint32_t addr) { PIIX4PMState *s = opaque; - uint32_t val = acpi_gpe_ioport_readb(&s->ar, addr); + uint32_t val = 0; + struct gpe_regs *g = &s->gperegs; + + switch (addr) { + case MEM_BASE ... MEM_BASE+DIMM_BITMAP_BYTES: + val = g->mems_sts[addr - MEM_BASE]; + break; + default: + val = acpi_gpe_ioport_readb(&s->ar, addr); + } PIIX4_DPRINTF("gpe read %x == %x\n", addr, val); return val; @@ -509,7 +527,13 @@ static void gpe_writeb(void *opaque, uint32_t addr, uint32_t val) { PIIX4PMState *s = opaque; - acpi_gpe_ioport_writeb(&s->ar, addr, val); + switch (addr) { + case MEM_EJ_BASE: + dimm_notify(val, DIMM_REMOVE_SUCCESS); + break; + default: + acpi_gpe_ioport_writeb(&s->ar, addr, val); + } pm_update_sci(s); PIIX4_DPRINTF("gpe write %x <== %d\n", addr, val); @@ -560,9 +584,11 @@ static uint32_t pcirmv_read(void *opaque, uint32_t addr) static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev, PCIHotplugState state); +static int piix4_dimm_hotplug(DeviceState *qdev, DimmDevice *dev, int add); static void piix4_acpi_system_hot_add_init(PCIBus *bus, PIIX4PMState *s) { + int i = 0; register_ioport_write(GPE_BASE, GPE_LEN, 1, gpe_writeb, s); register_ioport_read(GPE_BASE, GPE_LEN, 1, gpe_readb, s); @@ -576,7 +602,15 @@ static void piix4_acpi_system_hot_add_init(PCIBus *bus, PIIX4PMState *s) register_ioport_read(PCI_RMV_BASE, 4, 4, pcirmv_read, s); + register_ioport_read(MEM_BASE, DIMM_BITMAP_BYTES, 1, gpe_readb, s); + register_ioport_write(MEM_EJ_BASE, 1, 1, gpe_writeb, s); + + for(i = 0; i < DIMM_BITMAP_BYTES; i++) { + s->gperegs.mems_sts[i] = 0; + } + pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev); + dimm_bus_hotplug(piix4_dimm_hotplug, &s->dev.qdev); } static void enable_device(PIIX4PMState *s, int slot) @@ -591,6 +625,37 @@ static void disable_device(PIIX4PMState *s, int slot) s->pci0_status.down |= (1U << slot); } +static void enable_mem_device(PIIX4PMState *s, int memdevice) +{ + struct gpe_regs *g = &s->gperegs; + s->ar.gpe.sts[0] |= PIIX4_MEM_HOTPLUG_STATUS; + g->mems_sts[memdevice/8] |= (1 << (memdevice%8)); +} + +static void disable_mem_device(PIIX4PMState *s, int memdevice) +{ + struct gpe_regs *g = &s->gperegs; + s->ar.gpe.sts[0] |= PIIX4_MEM_HOTPLUG_STATUS; + g->mems_sts[memdevice/8] &= ~(1 << (memdevice%8)); +} + +static int piix4_dimm_hotplug(DeviceState *qdev, DimmDevice *dev, int + add) +{ + PCIDevice *pci_dev = DO_UPCAST(PCIDevice, qdev, qdev); + PIIX4PMState *s = DO_UPCAST(PIIX4PMState, dev, pci_dev); + DimmDevice *slot = DIMM(dev); + + if (add) { + enable_mem_device(s, slot->idx); + } + else { + disable_mem_device(s, slot->idx); + } + pm_update_sci(s); + return 0; +} + static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev, PCIHotplugState state) { -- 1.7.9

1 0

[RFC PATCH v3 06/19] Implement "-dimm" command line option
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

Example: "-dimm id=dimm0,size=512M,node=0,populated=off" will define a 512M memory slot belonging to numa node 0. When "populated=on", a DimmDevice is created and hot-plugged at system startup. Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- hw/Makefile.objs | 2 +- qemu-config.c | 25 +++++++++++++++++++++++++ qemu-options.hx | 5 +++++ sysemu.h | 1 + vl.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 82 insertions(+), 1 deletions(-) diff --git a/hw/Makefile.objs b/hw/Makefile.objs index 6dfebd2..8c5c39a 100644 --- a/hw/Makefile.objs +++ b/hw/Makefile.objs @@ -26,7 +26,7 @@ hw-obj-$(CONFIG_I8254) += i8254_common.o i8254.o hw-obj-$(CONFIG_PCSPK) += pcspk.o hw-obj-$(CONFIG_PCKBD) += pckbd.o hw-obj-$(CONFIG_FDC) += fdc.o -hw-obj-$(CONFIG_ACPI) += acpi.o acpi_piix4.o +hw-obj-$(CONFIG_ACPI) += acpi.o acpi_piix4.o dimm.o hw-obj-$(CONFIG_APM) += pm_smbus.o apm.o hw-obj-$(CONFIG_DMA) += dma.o hw-obj-$(CONFIG_I82374) += i82374.o diff --git a/qemu-config.c b/qemu-config.c index eba977e..4022d64 100644 --- a/qemu-config.c +++ b/qemu-config.c @@ -646,6 +646,30 @@ QemuOptsList qemu_boot_opts = { }, }; +static QemuOptsList qemu_dimm_opts = { + .name = "dimm", + .head = QTAILQ_HEAD_INITIALIZER(qemu_dimm_opts.head), + .desc = { + { + .name = "id", + .type = QEMU_OPT_STRING, + .help = "id of this dimm device", + },{ + .name = "size", + .type = QEMU_OPT_SIZE, + .help = "memory size for this dimm", + },{ + .name = "populated", + .type = QEMU_OPT_BOOL, + .help = "populated for this dimm", + },{ + .name = "node", + .type = QEMU_OPT_NUMBER, + .help = "NUMA node number (i.e. proximity) for this dimm", + }, + { /* end of list */ } + }, +}; static QemuOptsList *vm_config_groups[32] = { &qemu_drive_opts, &qemu_chardev_opts, @@ -662,6 +686,7 @@ static QemuOptsList *vm_config_groups[32] = { &qemu_boot_opts, &qemu_iscsi_opts, &qemu_sandbox_opts, + &qemu_dimm_opts, NULL, }; diff --git a/qemu-options.hx b/qemu-options.hx index 804a2d1..3687722 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -2842,3 +2842,8 @@ HXCOMM This is the last statement. Insert new options before this line! STEXI @end table ETEXI + +DEF("dimm", HAS_ARG, QEMU_OPTION_dimm, + "-dimm id=dimmid,size=sz,node=nd,populated=on|off\n" + "specify memory dimm device with name dimmid, size sz on node nd", + QEMU_ARCH_ALL) diff --git a/sysemu.h b/sysemu.h index 65552ac..7baf9c9 100644 --- a/sysemu.h +++ b/sysemu.h @@ -139,6 +139,7 @@ extern QEMUClock *rtc_clock; extern int nb_numa_nodes; extern uint64_t node_mem[MAX_NODES]; extern unsigned long *node_cpumask[MAX_NODES]; +extern int nb_hp_dimms; #define MAX_OPTION_ROMS 16 typedef struct QEMUOptionRom { diff --git a/vl.c b/vl.c index 7c577fa..af1745c 100644 --- a/vl.c +++ b/vl.c @@ -126,6 +126,7 @@ int main(int argc, char **argv) #include "hw/xen.h" #include "hw/qdev.h" #include "hw/loader.h" +#include "hw/dimm.h" #include "bt-host.h" #include "net.h" #include "net/slirp.h" @@ -248,6 +249,7 @@ QTAILQ_HEAD(, FWBootEntry) fw_boot_order = QTAILQ_HEAD_INITIALIZER(fw_boot_order int nb_numa_nodes; uint64_t node_mem[MAX_NODES]; unsigned long *node_cpumask[MAX_NODES]; +int nb_hp_dimms; uint8_t qemu_uuid[16]; @@ -530,6 +532,37 @@ static void configure_rtc_date_offset(const char *startdate, int legacy) } } +static void configure_dimm(QemuOpts *opts) +{ + const char *id; + uint64_t size, node; + bool populated; + QemuOpts *devopts; + char buf[256]; + if (nb_hp_dimms == MAX_DIMMS) { + fprintf(stderr, "qemu: maximum number of DIMMs (%d) exceeded\n", + MAX_DIMMS); + exit(1); + } + id = qemu_opts_id(opts); + size = qemu_opt_get_size(opts, "size", DEFAULT_DIMMSIZE); + populated = qemu_opt_get_bool(opts, "populated", 0); + node = qemu_opt_get_number(opts, "node", 0); + + dimm_config_create((char*)id, size, node, nb_hp_dimms, 0); + + if (populated) { + devopts = qemu_opts_create(qemu_find_opts("device"), id, 0, NULL); + qemu_opt_set(devopts, "driver", "dimm"); + snprintf(buf, sizeof(buf), "%lu", size); + qemu_opt_set(devopts, "size", buf); + snprintf(buf, sizeof(buf), "%lu", node); + qemu_opt_set(devopts, "node", buf); + qemu_opt_set(devopts, "bus", "membus"); + } + nb_hp_dimms++; +} + static void configure_rtc(QemuOpts *opts) { const char *value; @@ -2354,6 +2387,8 @@ int main(int argc, char **argv, char **envp) DisplayChangeListener *dcl; int cyls, heads, secs, translation; QemuOpts *hda_opts = NULL, *opts, *machine_opts; + QemuOpts *dimm_opts[MAX_DIMMS]; + int nb_dimm_opts = 0; QemuOptsList *olist; int optind; const char *optarg; @@ -3288,6 +3323,18 @@ int main(int argc, char **argv, char **envp) exit(0); } break; + case QEMU_OPTION_dimm: + if (nb_dimm_opts == MAX_DIMMS) { + fprintf(stderr, "qemu: maximum number of DIMMs (%d) exceeded\n", + MAX_DIMMS); + } + dimm_opts[nb_dimm_opts] = + qemu_opts_parse(qemu_find_opts("dimm"), optarg, 0); + if (!dimm_opts[nb_dimm_opts]) { + exit(1); + } + nb_dimm_opts++; + break; default: os_parse_cmd_args(popt->index, optarg); } @@ -3611,6 +3658,9 @@ int main(int argc, char **argv, char **envp) } qemu_add_globals(); + for (i = 0; i < nb_dimm_opts; i++) + configure_dimm(dimm_opts[i]); + qdev_machine_init(); machine->init(ram_size, boot_devices, -- 1.7.9

1 0

[RFC PATCH v3 05/19] Implement dimm device abstraction
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

Each hotplug-able memory slot is a DimmDevice. All DimmDevices are attached to a new bus called DimmBus. This bus is introduced so that we no longer depend on hotplug-capability of main system bus (the main bus does not allow hotplugging). The DimmBus should be attached to a chipset Device (i440fx in case of the pc) A hot-add operation for a particular dimm: - creates a new DimmDevice and attaches it to the DimmBus - creates a new MemoryRegion of the given physical address offset, size and node proximity, and attaches it to main system memory as a sub_region. A successful hot-remove operation detaches and frees the MemoryRegion from system memory, and removes the DimmDevice from the DimmBus. Hotplug operations are done through normal device_add /device_del commands. Also add properties to DimmDevice. Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- hw/dimm.c | 305 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/dimm.h | 90 ++++++++++++++++++ 2 files changed, 395 insertions(+), 0 deletions(-) create mode 100644 hw/dimm.c create mode 100644 hw/dimm.h diff --git a/hw/dimm.c b/hw/dimm.c new file mode 100644 index 0000000..288b997 --- /dev/null +++ b/hw/dimm.c @@ -0,0 +1,305 @@ +/* + * Dimm device for Memory Hotplug + * + * Copyright ProfitBricks GmbH 2012 + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see <http://www.gnu.org/licenses/> + */ + +#include "trace.h" +#include "qdev.h" +#include "dimm.h" +#include <time.h> +#include "../exec-memory.h" +#include "qmp-commands.h" + +/* the system-wide memory bus. */ +static DimmBus *main_memory_bus; +/* the following list is used to hold dimm config info before machine + * initialization. After machine init, the list is emptied and not used anymore.*/ +static DimmConfiglist dimmconfig_list = QTAILQ_HEAD_INITIALIZER(dimmconfig_list); + +static void dimmbus_dev_print(Monitor *mon, DeviceState *dev, int indent); +static char *dimmbus_get_fw_dev_path(DeviceState *dev); + +static Property dimm_properties[] = { + DEFINE_PROP_UINT64("start", DimmDevice, start, 0), + DEFINE_PROP_UINT64("size", DimmDevice, size, DEFAULT_DIMMSIZE), + DEFINE_PROP_UINT32("node", DimmDevice, node, 0), + DEFINE_PROP_END_OF_LIST(), +}; + +static void dimmbus_dev_print(Monitor *mon, DeviceState *dev, int indent) +{ +} + +static char *dimmbus_get_fw_dev_path(DeviceState *dev) +{ + char path[40]; + + snprintf(path, sizeof(path), "%s", qdev_fw_name(dev)); + return strdup(path); +} + +static void dimm_bus_class_init(ObjectClass *klass, void *data) +{ + BusClass *k = BUS_CLASS(klass); + + k->print_dev = dimmbus_dev_print; + k->get_fw_dev_path = dimmbus_get_fw_dev_path; +} + +static void dimm_bus_initfn(Object *obj) +{ + DimmConfig *dimm_cfg, *next_dimm_cfg; + DimmBus *bus = DIMM_BUS(obj); + QTAILQ_INIT(&bus->dimmconfig_list); + QTAILQ_INIT(&bus->dimmlist); + + QTAILQ_FOREACH_SAFE(dimm_cfg, &dimmconfig_list, nextdimmcfg, next_dimm_cfg) { + QTAILQ_REMOVE(&dimmconfig_list, dimm_cfg, nextdimmcfg); + QTAILQ_INSERT_TAIL(&bus->dimmconfig_list, dimm_cfg, nextdimmcfg); + } +} + +static const TypeInfo dimm_bus_info = { + .name = TYPE_DIMM_BUS, + .parent = TYPE_BUS, + .instance_size = sizeof(DimmBus), + .instance_init = dimm_bus_initfn, + .class_init = dimm_bus_class_init, +}; + +void main_memory_bus_create(Object *parent) +{ + main_memory_bus = g_malloc0(dimm_bus_info.instance_size); + main_memory_bus->qbus.glib_allocated = true; + qbus_create_inplace(&main_memory_bus->qbus, TYPE_DIMM_BUS, DEVICE(parent), + "membus"); +} + +static void dimm_populate(DimmDevice *s) +{ + DeviceState *dev= (DeviceState*)s; + MemoryRegion *new = NULL; + + new = g_malloc(sizeof(MemoryRegion)); + memory_region_init_ram(new, dev->id, s->size); + vmstate_register_ram_global(new); + memory_region_add_subregion(get_system_memory(), s->start, new); + s->mr = new; +} + +static void dimm_depopulate(DimmDevice *s) +{ + assert(s); + vmstate_unregister_ram(s->mr, NULL); + memory_region_del_subregion(get_system_memory(), s->mr); + memory_region_destroy(s->mr); + s->mr = NULL; +} + +void dimm_config_create(char *id, uint64_t size, uint64_t node, uint32_t + dimm_idx, uint32_t populated) +{ + DimmConfig *dimm_cfg; + dimm_cfg = (DimmConfig*) g_malloc0(sizeof(DimmConfig)); + dimm_cfg->name = id; + dimm_cfg->idx = dimm_idx; + dimm_cfg->start = 0; + dimm_cfg->size = size; + dimm_cfg->node = node; + dimm_cfg->populated = populated; + + QTAILQ_INSERT_TAIL(&dimmconfig_list, dimm_cfg, nextdimmcfg); +} + +void dimm_bus_hotplug(dimm_hotplug_fn hotplug, DeviceState *qdev) +{ + DimmBus *bus = main_memory_bus; + bus->qbus.allow_hotplug = 1; + bus->dimm_hotplug_qdev = qdev; + bus->dimm_hotplug = hotplug; +} + +static void dimm_plug_device(DimmDevice *slot) +{ + DimmBus *bus = main_memory_bus; + + dimm_populate(slot); + if (bus->dimm_hotplug) + bus->dimm_hotplug(bus->dimm_hotplug_qdev, slot, 1); +} + +static int dimm_unplug_device(DeviceState *qdev) +{ + DimmBus *bus = main_memory_bus; + + if (bus->dimm_hotplug) + bus->dimm_hotplug(bus->dimm_hotplug_qdev, DIMM(qdev), 0); + return 1; +} + +static DimmConfig *dimmcfg_find_from_name(const char *name) +{ + DimmConfig *slot; + DimmBus *bus = main_memory_bus; + + QTAILQ_FOREACH(slot, &bus->dimmconfig_list, nextdimmcfg) { + if (!strcmp(slot->name, name)) { + return slot; + } + } + return NULL; +} + +static DimmDevice *dimm_find_from_idx(uint32_t idx) +{ + DimmDevice *slot; + DimmBus *bus = main_memory_bus; + + QTAILQ_FOREACH(slot, &bus->dimmlist, nextdimm) { + if (slot->idx == idx) { + return slot; + } + } + return NULL; +} + +/* used to create a dimm device, only on incoming migration of a hotplugged + * RAMBlock + */ +int dimm_add(char *id) +{ + DimmConfig *slotcfg = NULL; + QemuOpts *devopts; + char buf[256]; + + if (!id) { + fprintf(stderr, "ERROR %s invalid id\n",__FUNCTION__); + return 1; + } + + slotcfg = dimmcfg_find_from_name(id); + + if (!slotcfg) { + fprintf(stderr, "%s no slot %s found\n", __FUNCTION__, id); + return 1; + } + + devopts = qemu_opts_create(qemu_find_opts("device"), id, 0, NULL); + qemu_opt_set(devopts, "driver", "dimm"); + + snprintf(buf, sizeof(buf), "%lu", slotcfg->size); + qemu_opt_set(devopts, "size", buf); + snprintf(buf, sizeof(buf), "%u", slotcfg->node); + qemu_opt_set(devopts, "node", buf); + qdev_device_add(devopts); + + return 0; +} + +/* used to calculate physical address offsets for all dimms */ +void dimm_calc_offsets(dimm_calcoffset_fn calcfn) +{ + DimmConfig *slot; + QTAILQ_FOREACH(slot, &dimmconfig_list, nextdimmcfg) { + if (!slot->start) { + slot->start = calcfn(slot->size); + } + } +} + +void setup_fwcfg_hp_dimms(uint64_t *fw_cfg_slots) +{ + DimmConfig *slot; + + QTAILQ_FOREACH(slot, &dimmconfig_list, nextdimmcfg) { + assert(slot->start); + fw_cfg_slots[3 * slot->idx] = cpu_to_le64(slot->start); + fw_cfg_slots[3 * slot->idx + 1] = cpu_to_le64(slot->size); + fw_cfg_slots[3 * slot->idx + 2] = cpu_to_le64(slot->node); + } +} + +void dimm_notify(uint32_t idx, uint32_t event) +{ + DimmBus *bus = main_memory_bus; + DimmDevice *s; + s = dimm_find_from_idx(idx); + assert(s != NULL); + + switch(event) { + case DIMM_REMOVE_SUCCESS: + dimm_depopulate(s); + qdev_simple_unplug_cb((DeviceState*)s); + QTAILQ_REMOVE(&bus->dimmlist, s, nextdimm); + break; + default: + break; + } +} + +static int dimm_init(DeviceState *s) +{ + DimmBus *bus = main_memory_bus; + DimmDevice *slot; + DimmConfig *slotcfg; + + slot = DIMM(s); + slot->mr = NULL; + + slotcfg = dimmcfg_find_from_name(s->id); + + if (!slotcfg) { + fprintf(stderr, "%s no config for slot %s found\n", + __FUNCTION__, s->id); + return 1; + } + + slot->idx = slotcfg->idx; + assert(slotcfg->start); + slot->start = slotcfg->start; + slot->size = slotcfg->size; + slot->node = slotcfg->node; + + QTAILQ_INSERT_TAIL(&bus->dimmlist, slot, nextdimm); + dimm_plug_device(slot); + + return 0; +} + + +static void dimm_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + dc->props = dimm_properties; + dc->unplug = dimm_unplug_device; + dc->init = dimm_init; +} + +static TypeInfo dimm_info = { + .name = TYPE_DIMM, + .parent = TYPE_DEVICE, + .instance_size = sizeof(DimmDevice), + .class_init = dimm_class_init, +}; + +static void dimm_register_types(void) +{ + type_register_static(&dimm_bus_info); + type_register_static(&dimm_info); +} + +type_init(dimm_register_types) diff --git a/hw/dimm.h b/hw/dimm.h new file mode 100644 index 0000000..5e991a6 --- /dev/null +++ b/hw/dimm.h @@ -0,0 +1,90 @@ +#ifndef QEMU_DIMM_H +#define QEMU_DIMM_H + +#include "qemu-common.h" +#include "memory.h" +#include "sysbus.h" +#include "qapi-types.h" +#include "qemu-queue.h" +#include "cpus.h" +#define MAX_DIMMS 255 +#define DIMM_BITMAP_BYTES (MAX_DIMMS + 7) / 8 +#define DEFAULT_DIMMSIZE 1024*1024*1024 + +typedef enum { + DIMM_REMOVE_SUCCESS = 0, + DIMM_REMOVE_FAIL = 1, + DIMM_ADD_SUCCESS = 2, + DIMM_ADD_FAIL = 3 +} dimm_hp_result_code; + +#define TYPE_DIMM "dimm" +#define DIMM(obj) \ + OBJECT_CHECK(DimmDevice, (obj), TYPE_DIMM) +#define DIMM_CLASS(klass) \ + OBJECT_CLASS_CHECK(DimmDeviceClass, (klass), TYPE_DIMM) +#define DIMM_GET_CLASS(obj) \ + OBJECT_GET_CLASS(DimmDeviceClass, (obj), TYPE_DIMM) + +typedef struct DimmDevice DimmDevice; +typedef QTAILQ_HEAD(DimmConfiglist, DimmConfig) DimmConfiglist; + +typedef struct DimmDeviceClass { + DeviceClass parent_class; + + int (*init)(DimmDevice *dev); +} DimmDeviceClass; + +typedef struct DimmDevice { + DeviceState qdev; + uint32_t idx; /* index in memory hotplug register/bitmap */ + ram_addr_t start; /* starting physical address */ + ram_addr_t size; + uint32_t node; /* numa node proximity */ + MemoryRegion *mr; /* MemoryRegion for this slot. !NULL only if populated */ + QTAILQ_ENTRY (DimmDevice) nextdimm; +} DimmDevice; + +typedef struct DimmConfig +{ + const char *name; + uint32_t idx; /* index in memory hotplug register/bitmap */ + ram_addr_t start; /* starting physical address */ + ram_addr_t size; + uint32_t node; /* numa node proximity */ + uint32_t populated; /* 1 means device has been hotplugged. Default is 0. */ + QTAILQ_ENTRY (DimmConfig) nextdimmcfg; +} DimmConfig; + +typedef int (*dimm_hotplug_fn)(DeviceState *qdev, DimmDevice *dev, int add); +typedef target_phys_addr_t (*dimm_calcoffset_fn)(uint64_t size); + +#define TYPE_DIMM_BUS "dimmbus" +#define DIMM_BUS(obj) OBJECT_CHECK(DimmBus, (obj), TYPE_DIMM_BUS) + +typedef struct DimmBus { + BusState qbus; + DeviceState *dimm_hotplug_qdev; + dimm_hotplug_fn dimm_hotplug; + dimm_calcoffset_fn dimm_calcoffset; + DimmConfiglist dimmconfig_list; + QTAILQ_HEAD(Dimmlist, DimmDevice) dimmlist; +} DimmBus; + +struct dimm_hp_result { + const char *dimmname; + dimm_hp_result_code ret; + QTAILQ_ENTRY (dimm_hp_result) next; +}; + +void dimm_calc_offsets(dimm_calcoffset_fn calcfn); +void dimm_notify(uint32_t idx, uint32_t event); +void dimm_bus_hotplug(dimm_hotplug_fn hotplug, DeviceState *qdev); +void setup_fwcfg_hp_dimms(uint64_t *fw_cfg_slots); +int dimm_add(char *id); +void main_memory_bus_create(Object *parent); +void dimm_config_create(char *id, uint64_t size, uint64_t node, + uint32_t dimm_idx, uint32_t populated); + + +#endif -- 1.7.9

1 0

[RFC PATCH v3 04/19] acpi: generate hotplug memory devices
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

The memory device generation is guided by qemu paravirt info. Seabios first uses the info to setup SRAT entries for the hotplug-able memory slots. Afterwards, build_memssdt uses the created SRAT entries to generate appropriate memory device objects. One memory device (and corresponding SRAT entry) is generated for each hotplug-able qemu memslot. Currently no SSDT memory device is created for initial system memory. We only support up to 255 DIMMs for now (PackageOp used for the MEON array can only describe an array of at most 255 elements. VarPackageOp would be needed to support more than 255 devices) v1->v2: Seabios reads mems_sts from qemu to build e820_map SSDT size and some offsets are calculated with extraction macros. v2->v3: Minor name changes Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- src/acpi.c | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 152 insertions(+), 6 deletions(-) diff --git a/src/acpi.c b/src/acpi.c index 6d239fa..1223b52 100644 --- a/src/acpi.c +++ b/src/acpi.c @@ -13,6 +13,7 @@ #include "pci_regs.h" // PCI_INTERRUPT_LINE #include "ioport.h" // inl #include "paravirt.h" // qemu_cfg_irq0_override +#include "memmap.h" /****************************************************/ /* ACPI tables init */ @@ -416,11 +417,26 @@ encodeLen(u8 *ssdt_ptr, int length, int bytes) #define PCIHP_AML (ssdp_pcihp_aml + *ssdt_pcihp_start) #define PCI_SLOTS 32 +/* 0x5B 0x82 DeviceOp PkgLength NameString DimmID */ +#define MEM_BASE 0xaf80 +#define MEM_AML (ssdm_mem_aml + *ssdt_mem_start) +#define MEM_SIZEOF (*ssdt_mem_end - *ssdt_mem_start) +#define MEM_OFFSET_HEX (*ssdt_mem_name - *ssdt_mem_start + 2) +#define MEM_OFFSET_ID (*ssdt_mem_id - *ssdt_mem_start) +#define MEM_OFFSET_PXM 31 +#define MEM_OFFSET_START 55 +#define MEM_OFFSET_END 63 +#define MEM_OFFSET_SIZE 79 + +u64 nb_hp_memslots = 0; +struct srat_memory_affinity *mem; + #define SSDT_SIGNATURE 0x54445353 // SSDT #define SSDT_HEADER_LENGTH 36 #include "ssdt-susp.hex" #include "ssdt-pcihp.hex" +#include "ssdt-mem.hex" #define PCI_RMV_BASE 0xae0c @@ -472,6 +488,111 @@ static void patch_pcihp(int slot, u8 *ssdt_ptr, u32 eject) } } +static void build_memdev(u8 *ssdt_ptr, int i, u64 mem_base, u64 mem_len, u8 node) +{ + memcpy(ssdt_ptr, MEM_AML, MEM_SIZEOF); + ssdt_ptr[MEM_OFFSET_HEX] = getHex(i >> 4); + ssdt_ptr[MEM_OFFSET_HEX+1] = getHex(i); + ssdt_ptr[MEM_OFFSET_ID] = i; + ssdt_ptr[MEM_OFFSET_PXM] = node; + *(u64*)(ssdt_ptr + MEM_OFFSET_START) = mem_base; + *(u64*)(ssdt_ptr + MEM_OFFSET_END) = mem_base + mem_len; + *(u64*)(ssdt_ptr + MEM_OFFSET_SIZE) = mem_len; +} + +static void* +build_memssdt(void) +{ + u64 mem_base; + u64 mem_len; + u8 node; + int i; + struct srat_memory_affinity *entry = mem; + u64 nb_memdevs = nb_hp_memslots; + u8 memslot_status, enabled; + + int length = ((1+3+4) + + (nb_memdevs * MEM_SIZEOF) + + (1+2+5+(12*nb_memdevs)) + + (6+2+1+(1*nb_memdevs))); + u8 *ssdt = malloc_high(sizeof(struct acpi_table_header) + length); + if (! ssdt) { + warn_noalloc(); + return NULL; + } + u8 *ssdt_ptr = ssdt + sizeof(struct acpi_table_header); + + // build Scope(_SB_) header + *(ssdt_ptr++) = 0x10; // ScopeOp + ssdt_ptr = encodeLen(ssdt_ptr, length-1, 3); + *(ssdt_ptr++) = '_'; + *(ssdt_ptr++) = 'S'; + *(ssdt_ptr++) = 'B'; + *(ssdt_ptr++) = '_'; + + for (i = 0; i < nb_memdevs; i++) { + mem_base = (((u64)(entry->base_addr_high) << 32 )| entry->base_addr_low); + mem_len = (((u64)(entry->length_high) << 32 )| entry->length_low); + node = entry->proximity[0]; + build_memdev(ssdt_ptr, i, mem_base, mem_len, node); + ssdt_ptr += MEM_SIZEOF; + entry++; + } + + // build "Method(MTFY, 2) {If (LEqual(Arg0, 0x00)) {Notify(CM00, Arg1)} ...}" + *(ssdt_ptr++) = 0x14; // MethodOp + ssdt_ptr = encodeLen(ssdt_ptr, 2+5+(12*nb_memdevs), 2); + *(ssdt_ptr++) = 'M'; + *(ssdt_ptr++) = 'T'; + *(ssdt_ptr++) = 'F'; + *(ssdt_ptr++) = 'Y'; + *(ssdt_ptr++) = 0x02; + for (i=0; i<nb_memdevs; i++) { + *(ssdt_ptr++) = 0xA0; // IfOp + ssdt_ptr = encodeLen(ssdt_ptr, 11, 1); + *(ssdt_ptr++) = 0x93; // LEqualOp + *(ssdt_ptr++) = 0x68; // Arg0Op + *(ssdt_ptr++) = 0x0A; // BytePrefix + *(ssdt_ptr++) = i; + *(ssdt_ptr++) = 0x86; // NotifyOp + *(ssdt_ptr++) = 'M'; + *(ssdt_ptr++) = 'P'; + *(ssdt_ptr++) = getHex(i >> 4); + *(ssdt_ptr++) = getHex(i); + *(ssdt_ptr++) = 0x69; // Arg1Op + } + + // build "Name(MEON, Package() { One, One, ..., Zero, Zero, ... })" + *(ssdt_ptr++) = 0x08; // NameOp + *(ssdt_ptr++) = 'M'; + *(ssdt_ptr++) = 'E'; + *(ssdt_ptr++) = 'O'; + *(ssdt_ptr++) = 'N'; + *(ssdt_ptr++) = 0x12; // PackageOp + ssdt_ptr = encodeLen(ssdt_ptr, 2+1+(1*nb_memdevs), 2); + *(ssdt_ptr++) = nb_memdevs; + + entry = mem; + memslot_status = 0; + + for (i = 0; i < nb_memdevs; i++) { + enabled = 0; + if (i % 8 == 0) + memslot_status = inb(MEM_BASE + i/8); + enabled = memslot_status & 1; + mem_base = (((u64)(entry->base_addr_high) << 32 )| entry->base_addr_low); + mem_len = (((u64)(entry->length_high) << 32 )| entry->length_low); + *(ssdt_ptr++) = enabled ? 0x01 : 0x00; + if (enabled) + add_e820(mem_base, mem_len, E820_RAM); + memslot_status = memslot_status >> 1; + entry++; + } + build_header((void*)ssdt, SSDT_SIGNATURE, ssdt_ptr - ssdt, 1); + + return ssdt; +} + static void* build_ssdt(void) { @@ -644,9 +765,6 @@ build_srat(void) { int nb_numa_nodes = qemu_cfg_get_numa_nodes(); - if (nb_numa_nodes == 0) - return NULL; - u64 *numadata = malloc_tmphigh(sizeof(u64) * (MaxCountCPUs + nb_numa_nodes)); if (!numadata) { warn_noalloc(); @@ -655,10 +773,11 @@ build_srat(void) qemu_cfg_get_numa_data(numadata, MaxCountCPUs + nb_numa_nodes); + qemu_cfg_get_numa_data(&nb_hp_memslots, 1); struct system_resource_affinity_table *srat; int srat_size = sizeof(*srat) + sizeof(struct srat_processor_affinity) * MaxCountCPUs + - sizeof(struct srat_memory_affinity) * (nb_numa_nodes + 2); + sizeof(struct srat_memory_affinity) * (nb_numa_nodes + nb_hp_memslots + 2); srat = malloc_high(srat_size); if (!srat) { @@ -693,7 +812,7 @@ build_srat(void) * from 640k-1M and possibly another one from 3.5G-4G. */ struct srat_memory_affinity *numamem = (void*)core; - int slots = 0; + int slots = 0, node; u64 mem_len, mem_base, next_base = 0; acpi_build_srat_memory(numamem, 0, 640*1024, 0, 1); @@ -720,10 +839,36 @@ build_srat(void) next_base += (1ULL << 32) - RamSize; } acpi_build_srat_memory(numamem, mem_base, mem_len, i-1, 1); + numamem++; slots++; + } - for (; slots < nb_numa_nodes + 2; slots++) { + mem = (void*)numamem; + + if (nb_hp_memslots) { + u64 *hpmemdata = malloc_tmphigh(sizeof(u64) * (3 * nb_hp_memslots)); + if (!hpmemdata) { + warn_noalloc(); + free(hpmemdata); + free(numadata); + return NULL; + } + + qemu_cfg_get_numa_data(hpmemdata, 3 * nb_hp_memslots); + + for (i = 1; i < nb_hp_memslots + 1; ++i) { + mem_base = *hpmemdata++; + mem_len = *hpmemdata++; + node = *hpmemdata++; + acpi_build_srat_memory(numamem, mem_base, mem_len, node, 1); + numamem++; + slots++; + } + free(hpmemdata); + } + + for (; slots < nb_numa_nodes + nb_hp_memslots + 2; slots++) { acpi_build_srat_memory(numamem, 0, 0, 0, 0); numamem++; } @@ -774,6 +919,7 @@ acpi_bios_init(void) ACPI_INIT_TABLE(build_madt()); ACPI_INIT_TABLE(build_hpet()); ACPI_INIT_TABLE(build_srat()); + ACPI_INIT_TABLE(build_memssdt()); u16 i, external_tables = qemu_cfg_acpi_additional_tables(); -- 1.7.9

1 0

[RFC PATCH v3 03/19] acpi-dsdt: Implement functions for memory hotplug
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

Extend the DSDT to include methods for handling memory hot-add and hot-remove notifications and memory device status requests. These functions are called from the memory device SSDT methods. Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- src/acpi-dsdt.dsl | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 68 insertions(+), 2 deletions(-) diff --git a/src/acpi-dsdt.dsl b/src/acpi-dsdt.dsl index 2060686..5d3e92b 100644 --- a/src/acpi-dsdt.dsl +++ b/src/acpi-dsdt.dsl @@ -737,6 +737,71 @@ DefinitionBlock ( } Return(One) } + /* Objects filled in by run-time generated SSDT */ + External(MTFY, MethodObj) + External(MEON, PkgObj) + + Method (CMST, 1, NotSerialized) { + // _STA method - return ON status of memdevice + // Local0 = MEON flag for this cpu + Store(DerefOf(Index(MEON, Arg0)), Local0) + If (Local0) { Return(0xF) } Else { Return(0x0) } + } + + /* Memory hotplug notify array */ + OperationRegion(MEST, SystemIO, 0xaf80, 32) + Field (MEST, ByteAcc, NoLock, Preserve) + { + MES, 256 + } + + /* Memory eject byte */ + OperationRegion(MEMJ, SystemIO, 0xafa0, 1) + Field (MEMJ, ByteAcc, NoLock, Preserve) + { + MPE, 8 + } + + Method(MESC, 0) { + // Local5 = active memdevice bitmap + Store (MES, Local5) + // Local2 = last read byte from bitmap + Store (Zero, Local2) + // Local0 = memory device iterator + Store (Zero, Local0) + While (LLess(Local0, SizeOf(MEON))) { + // Local1 = MEON flag for this memory device + Store(DerefOf(Index(MEON, Local0)), Local1) + If (And(Local0, 0x07)) { + // Shift down previously read bitmap byte + ShiftRight(Local2, 1, Local2) + } Else { + // Read next byte from memdevice bitmap + Store(DerefOf(Index(Local5, ShiftRight(Local0, 3))), Local2) + } + // Local3 = active state for this memory device + Store(And(Local2, 1), Local3) + + If (LNotEqual(Local1, Local3)) { + // State change - update MEON with new state + Store(Local3, Index(MEON, Local0)) + // Do MEM notify + If (LEqual(Local3, 1)) { + MTFY(Local0, 1) + } Else { + MTFY(Local0, 3) + } + } + Increment(Local0) + } + Return(One) + } + + Method (MPEJ, 2, NotSerialized) { + // _EJ0 method - eject callback + Store(Arg0, MPE) + Sleep(200) + } } @@ -759,8 +824,9 @@ DefinitionBlock ( // CPU hotplug event Return(\_SB.PRSC()) } - Method(_L03) { - Return(0x01) + Method(_E03) { + // Memory hotplug event + Return(\_SB.MESC()) } Method(_L04) { Return(0x01) -- 1.7.9

1 0

[RFC PATCH v3 02/19] Add SSDT memory device support
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

Define SSDT hotplug-able memory devices in _SB namespace. The dynamically generated SSDT includes per memory device hotplug methods. These methods just call methods defined in the DSDT. Also dynamically generate a MTFY method and a MEON array of the online/available memory devices. ACPI extraction macros are used to place the AML code in variables later used by src/acpi. The design is taken from SSDT cpu generation. Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- Makefile | 2 +- src/ssdt-mem.dsl | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+), 1 deletions(-) create mode 100644 src/ssdt-mem.dsl diff --git a/Makefile b/Makefile index 5486f88..e82cfc9 100644 --- a/Makefile +++ b/Makefile @@ -233,7 +233,7 @@ $(OUT)%.hex: src/%.dsl ./tools/acpi_extract_preprocess.py ./tools/acpi_extract.p $(Q)$(PYTHON) ./tools/acpi_extract.py $(OUT)$*.lst > $(OUT)$*.off $(Q)cat $(OUT)$*.off > $@ -$(OUT)ccode32flat.o: $(OUT)acpi-dsdt.hex $(OUT)ssdt-proc.hex $(OUT)ssdt-pcihp.hex $(OUT)ssdt-susp.hex +$(OUT)ccode32flat.o: $(OUT)acpi-dsdt.hex $(OUT)ssdt-proc.hex $(OUT)ssdt-pcihp.hex $(OUT)ssdt-susp.hex $(OUT)ssdt-mem.hex ################ Kconfig rules diff --git a/src/ssdt-mem.dsl b/src/ssdt-mem.dsl new file mode 100644 index 0000000..ee322f0 --- /dev/null +++ b/src/ssdt-mem.dsl @@ -0,0 +1,65 @@ +/* This file is the basis for the ssdt_mem[] variable in src/acpi.c. + * It is similar in design to the ssdt_proc variable. + * It defines the contents of the per-cpu Processor() object. At + * runtime, a dynamically generated SSDT will contain one copy of this + * AML snippet for every possible memory device in the system. The + * objects will * be placed in the \_SB_ namespace. + * + * In addition to the aml code generated from this file, the + * src/acpi.c file creates a MEMNTFY method with an entry for each memdevice: + * Method(MTFY, 2) { + * If (LEqual(Arg0, 0x00)) { Notify(MP00, Arg1) } + * If (LEqual(Arg0, 0x01)) { Notify(MP01, Arg1) } + * ... + * } + * and a MEON array with the list of active and inactive memory devices: + * Name(MEON, Package() { One, One, ..., Zero, Zero, ... }) + */ +ACPI_EXTRACT_ALL_CODE ssdm_mem_aml + +DefinitionBlock ("ssdt-mem.aml", "SSDT", 0x02, "BXPC", "CSSDT", 0x1) +/* v------------------ DO NOT EDIT ------------------v */ +{ + ACPI_EXTRACT_DEVICE_START ssdt_mem_start + ACPI_EXTRACT_DEVICE_END ssdt_mem_end + ACPI_EXTRACT_DEVICE_STRING ssdt_mem_name + Device(MPAA) { + ACPI_EXTRACT_NAME_BYTE_CONST ssdt_mem_id + Name(ID, 0xAA) +/* ^------------------ DO NOT EDIT ------------------^ + * + * The src/acpi.c code requires the above layout so that it can update + * MPAA and 0xAA with the appropriate MEMDEVICE id (see + * SD_OFFSET_MEMHEX/MEMID1/MEMID2). Don't change the above without + * also updating the C code. + */ + Name(_HID, EISAID("PNP0C80")) + Name(_PXM, 0xAA) + + External(CMST, MethodObj) + External(MPEJ, MethodObj) + + Name(_CRS, ResourceTemplate() { + QwordMemory( + ResourceConsumer, + , + MinFixed, + MaxFixed, + Cacheable, + ReadWrite, + 0x0, + 0xDEADBEEF, + 0xE6ADBEEE, + 0x00000000, + 0x08000000, + ) + }) + Method (_STA, 0) { + Return(CMST(ID)) + } + Method (_EJ0, 1, NotSerialized) { + MPEJ(ID, Arg0) + } + } +} + -- 1.7.9

1 0

[RFC PATCH v3 01/19] Add ACPI_EXTRACT_DEVICE* macros
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

This allows to extract the beginning, end and name of a Device object. Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis(a)profitbricks.com> --- tools/acpi_extract.py | 28 ++++++++++++++++++++++++++++ 1 files changed, 28 insertions(+), 0 deletions(-) diff --git a/tools/acpi_extract.py b/tools/acpi_extract.py index 167a322..cb2540e 100755 --- a/tools/acpi_extract.py +++ b/tools/acpi_extract.py @@ -195,6 +195,28 @@ def aml_package_start(offset): offset += 1 return offset + aml_pkglen_bytes(offset) + 1 +def aml_device_start(offset): + #0x5B 0x82 DeviceOp PkgLength NameString ProcID + if ((aml[offset] != 0x5B) or (aml[offset + 1] != 0x82)): + die( "Name offset 0x%x: expected 0x5B 0x83 actual 0x%x 0x%x" % + (offset, aml[offset], aml[offset + 1])); + return offset + +def aml_device_string(offset): + #0x5B 0x82 DeviceOp PkgLength NameString ProcID + start = aml_device_start(offset) + offset += 2 + pkglenbytes = aml_pkglen_bytes(offset) + offset += pkglenbytes + return offset + +def aml_device_end(offset): + start = aml_device_start(offset) + offset += 2 + pkglenbytes = aml_pkglen_bytes(offset) + pkglen = aml_pkglen(offset) + return offset + pkglen + lineno = 0 for line in fileinput.input(): # Strip trailing newline @@ -279,6 +301,12 @@ for i in range(len(asl)): offset = aml_processor_end(offset) elif (directive == "ACPI_EXTRACT_PKG_START"): offset = aml_package_start(offset) + elif (directive == "ACPI_EXTRACT_DEVICE_START"): + offset = aml_device_start(offset) + elif (directive == "ACPI_EXTRACT_DEVICE_STRING"): + offset = aml_device_string(offset) + elif (directive == "ACPI_EXTRACT_DEVICE_END"): + offset = aml_device_end(offset) else: die("Unsupported directive %s" % directive) -- 1.7.9

1 0

[RFC PATCH v3 00/19] ACPI memory hotplug
by Vasilis Liaskovitis 21 Sep '12

21 Sep '12

This is v3 of the ACPI memory hotplug functionality. Only x86_64 target is supported for now. Overview: Dimm device layout is modeled with a new qemu command line "-dimm id=name,size=sz,node=pxm,populated=on|off" The starting physical address for all dimms is calculated automatically from top of memory, skipping the pci hole at [PCI_HOLE_START, 4G). Node is defining numa proximity for this dimm. When not defined it defaults to zero. "-dimm id=dimm0,size=512M,node=0,populated=off" will define a 512M memory slot belonging to numa node 0. Dimms are added or removed with normal device_add, device_del operations: Hot-add syntax: "device_add dimm,id=mydimm0" Hot-remove syntax: "dimm_del dimm,id=mydimm0" Changes v2->v3 - qdev integration. Dimms are attached to a dimmbus. The dimmbus is a child of i440fx device in the pc machine. Hot-add and hot-remove are done with normal device_add / device_del operations on the dimmbus. New commands "dimm_add" and "dimm_del" are obsolete. (In previous versions, dimms were always present on the qdev tree, and dimm_add/del simply meant allocating or deallocating memory for the devices. This version actually does hot-operations on the qdev tree) - Add _PS3 method to allow OSPM-induced hot operations. - pci-window calculation in Seabios takes dimms into account(for both 32-bit and 64-bit windows) - rename new qmp commands: query-memory-total and query-memory-hotplug - balloon driver can see the hotplugged memory Changes v1->v2 - memory map is automatically calculated for hotplug dimms. Dimms are added from top-of-memory skipping the pci hole at [PCI_HOLE_START, 4G). - Renamed from "-memslot" to "-dimm". Commands changed to "dimm_add", "dimm_del". - Seabios ejection array reduced to a byte. Use extraction macros for dimm ssdt. - additional SRAT paravirt info does not break previous SRAT fw_cfg layout. - Documentation of new acpi_piix4 registers and paravirt data. - add ACPI _OST support for _OST enabled guests. This allows qemu to receive notification for success / failure of memory hot-add and hot-remove operations. Guest needs to support _OST (https://lkml.org/lkml/2012/6/25/321) - add monitor info command to report total guest memory (initial + hot-added) - add command line options and monitor commands for batch dimm creation/population (obsolete from v3 onwards) Issues: - A main blocker issue is windows guest functionality. The patchset does not work for windows currently. My guess is the windows pnpmem driver does not like the seabios dimm device implementation (or the seabios dimm implementation is not fully ACPI-compliant). If someone can review the seabios patches or has any ideas to debug this, let me know. Testing on win2012 server RC or windows2008 consumer prerelease. When adding a DIMM, the device shows up in DeviceManager but does not work. Relevant messages: " This device cannot start. (Code 10) Device configured(memory.inf) (UserPnP eventID 400) Device installed (memory.inf) ACPI/PNP0C80\2&daba3ff&1 was configured Device not started(PNPMEM) (Kernel-PnP eventID 411, kernelID) Device ACPI\PNP0C80\2&daba3ff&1 had a problem starting Driver Name: memory.inf (c:\Windows\system32\DRIVERS\pnpmem.sys 6.2.8400 winmain_win8rc)) Memory range:0x80000000 - 0x90000000 (Initial memory of VM is 2GB. The hotplugged DIMM was a 256GB with physical address range starting at 2GB ) Conflicting device list: No conflicts. " Adding a 2nd or more dimms causes a crash (PNP_DETECTED_FATAL_ERROR with blue screen of death) and makes windows reboot. After this, the VM keeps rebooting with ACPI_BIOS_ERROR. The VM refuses to boot anymore once a 2nd (or more) extra dimm is plugged-in. - Is the dimmbus the correct way to go about integrating into qdev/qom? In a v1 comment, Anthony mentioned attaching dimms directly to an i440fx device as children. Is this possible without a bus? - Live migration works as long as the dimm layout (-dimm command line args) are identical at the source and destination qemu command line. Patch 10/19 creates the DimmDevice that corresponds to the unknown incoming ramblock. Ramblocks are migrated before qdev VMStates are migrated (the DimmDevice structure currently does not define a VMStateDescription). So the DimmDevice is handled diferrently than other devices. If this is not acceptable, any suggestions on how should it be reworked? - Hot-operation notification lists need to be added to migration state. Please review. Could people state which other issues they consider blocker for including this upstream? Does this patchset need to wait for 1.4 or could this be considered for 1.3 (assuming blockers are resolved)? The patchset has been revised every few months, but I will provide quicker version updates onwards. I can also bring this up on a weekly meeting agenda if needed. series is based on uq/master for qemu-kvm, and master for seabios. Can be found also at: http://github.com/vliaskov/qemu-kvm/commits/memhp-v3 http://github.com/vliaskov/seabios/commits/memhp-v3 Vasilis Liaskovitis (12): Implement dimm device abstraction Implement "-dimm" command line option acpi_piix4: Implement memory device hotplug registers pc: calculate dimm physical addresses and adjust memory map pc: Add dimm paravirt SRAT info fix live-migration when "populated=on" is missing Implement qmp and hmp commands for notification lists Implement "info memory-total" and "query-memory-total" balloon: update with hotplugged memory Add _OST dimm support Update dimm state on reset Implement _PS3 for dimm arch_init.c | 24 ++- docs/specs/acpi_hotplug.txt | 54 ++++++ docs/specs/fwcfg.txt | 28 +++ hmp-commands.hx | 4 + hmp.c | 24 +++ hmp.h | 2 + hw/Makefile.objs | 2 +- hw/acpi_piix4.c | 114 +++++++++++- hw/dimm.c | 435 +++++++++++++++++++++++++++++++++++++++++++ hw/dimm.h | 101 ++++++++++ hw/pc.c | 55 ++++++- hw/pc.h | 6 + hw/pc_piix.c | 20 ++- hw/virtio-balloon.c | 13 +- monitor.c | 14 ++ qapi-schema.json | 37 ++++ qemu-config.c | 25 +++ qemu-options.hx | 5 + qmp-commands.hx | 57 ++++++ sysemu.h | 1 + vl.c | 51 +++++ 21 files changed, 1051 insertions(+), 21 deletions(-) create mode 100644 docs/specs/acpi_hotplug.txt create mode 100644 docs/specs/fwcfg.txt create mode 100644 hw/dimm.c create mode 100644 hw/dimm.h Vasilis Liaskovitis (7): Add ACPI_EXTRACT_DEVICE* macros Subject: [PATCH 02/18] Add SSDT memory device support acpi-dsdt: Implement functions for memory hotplug acpi: generate hotplug memory devices Add _OST dimm method Implement _PS3 method for memory device Calculate pcimem_start and pcimem64_start from SRAT entries Makefile | 2 +- src/acpi-dsdt.dsl | 135 ++++++++++++++++++++++++++++++- src/acpi.c | 216 ++++++++++++++++++++++++++++++++++++++++++++---- src/acpi.h | 3 + src/pciinit.c | 6 +- src/post.c | 3 + src/smp.c | 4 + src/ssdt-mem.dsl | 73 +++++++++++++++++ tools/acpi_extract.py | 28 +++++++ 9 files changed, 447 insertions(+), 23 deletions(-) create mode 100644 src/ssdt-mem.dsl -- 1.7.9

1 0

← Newer
1
2
3
4
5
6
7
...
12
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

SeaBIOS September 2012