Hello,
I am working on QEMU to pass-through a graphics device. I noticed few problems with PCI enumeration hence I compiled SeaBIOS with more debug information.
The PCI enumeration problem seems to be related to BUILS_PCIMEM_SIZE. I keep on getting the error 'increase BUILD_PCIMEM_SIZE and recompile'. This happens when BIOS tries to allocate 128MB for one of the BAR. The code seems to be printing the correct error as, the 128MB naturally aligned memory request can not be satisfied. No problem with this.
As I notice on my host machine the same case happens, but host BIOS seems to be allocating 128MB memory from some other region (c0000000-c7ffffff)
02:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B64 [FireGL V3100 (PCIE)] (rev 80) (prog-if 00 [VGA controller]) Region 0: Memory at c0000000 (32-bit, prefetchable) [size=128M] Region 1: I/O ports at a000 [size=256] Region 2: Memory at f9cf0000 (32-bit, non-prefetchable) [size=64K] Expansion ROM at f9cc0000 [disabled] [size=128K]
prasad@prasad-kvm:~$ grep '0000:02' /proc/iomem c0000000-c7ffffff : PCI Bus 0000:02 c0000000-c7ffffff : 0000:02:00.0 f9c00000-f9cfffff : PCI Bus 0000:02 f9cc0000-f9cdffff : 0000:02:00.0 f9ce0000-f9ceffff : 0000:02:00.1 f9cf0000-f9cfffff : 0000:02:00.0
Can SeaBIOS do the same thing? or to put in other words Can SeaBIOS allocate memory from some other region instead of fixed f0000000 to (0xfec00000-1)?
Thanks and Regards, Prasad
On Mon, Feb 07, 2011 at 07:17:52PM +0000, Prasad Joshi wrote:
Hello,
I am working on QEMU to pass-through a graphics device. I noticed few problems with PCI enumeration hence I compiled SeaBIOS with more debug information.
The PCI enumeration problem seems to be related to BUILS_PCIMEM_SIZE. I keep on getting the error 'increase BUILD_PCIMEM_SIZE and recompile'. This happens when BIOS tries to allocate 128MB for one of the BAR. The code seems to be printing the correct error as, the 128MB naturally aligned memory request can not be satisfied. No problem with this.
As I notice on my host machine the same case happens, but host BIOS seems to be allocating 128MB memory from some other region (c0000000-c7ffffff)
[...]
Can SeaBIOS do the same thing? or to put in other words Can SeaBIOS allocate memory from some other region instead of fixed f0000000 to (0xfec00000-1)?
It should be possible to alter BUILD_PCIMEM_START in src/config.h. The reason it is at 0xf0000000 is because regular memory can go up to 0xe0000000, and the bochs vga bios had soe dependency on the address space at 0xe0000000 - though this may no longer be the case.
-Kevin
Thanks a lot for your reply.
On Thu, Feb 10, 2011 at 3:31 PM, Kevin O'Connor kevin@koconnor.net wrote:
On Mon, Feb 07, 2011 at 07:17:52PM +0000, Prasad Joshi wrote:
Hello,
As I notice on my host machine the same case happens, but host BIOS seems to be allocating 128MB memory from some other region (c0000000-c7ffffff)
[...]
Can SeaBIOS do the same thing? or to put in other words Can SeaBIOS allocate memory from some other region instead of fixed f0000000 to (0xfec00000-1)?
It should be possible to alter BUILD_PCIMEM_START in src/config.h. The reason it is at 0xf0000000 is because regular memory can go up to 0xe0000000, and the bochs vga bios had soe dependency on the address space at 0xe0000000 - though this may no longer be the case.
I modified the memory range according the lspci -vv output. Atleast on my machine, the memory range 0xf0000000 to 0xfec00000 is reserved for non-prefectable memory.
Here is a small modification. As long as it keeps the behavior for old pci assignments, I guess this would not be a problem.
diff --git a/src/config.h b/src/config.h index 5b40488..19729cb 100644 --- a/src/config.h +++ b/src/config.h @@ -49,20 +49,18 @@
// Support old pci mem assignment behaviour //#define CONFIG_OLD_PCIMEM_ASSIGNMENT 1 -#if CONFIG_OLD_PCIMEM_ASSIGNMENT #define BUILD_PCIMEM_START 0xf0000000 #define BUILD_PCIMEM_SIZE (BUILD_PCIMEM_END - BUILD_PCIMEM_START) #define BUILD_PCIMEM_END 0xfec00000 /* IOAPIC is mapped at */ + +#if CONFIG_OLD_PCIMEM_ASSIGNMENT #define BUILD_PCIPREFMEM_START 0 #define BUILD_PCIPREFMEM_SIZE 0 #define BUILD_PCIPREFMEM_END 0 #else -#define BUILD_PCIMEM_START 0xf0000000 -#define BUILD_PCIMEM_SIZE 0x08000000 /* half- of pci window */ -#define BUILD_PCIMEM_END (BUILD_PCIMEM_START + BUILD_PCIMEM_SIZE) -#define BUILD_PCIPREFMEM_START BUILD_PCIMEM_END +#define BUILD_PCIPREFMEM_START 0xc0000000 #define BUILD_PCIPREFMEM_SIZE (BUILD_PCIPREFMEM_END - BUILD_PCIPREFMEM_START) -#define BUILD_PCIPREFMEM_END 0xfec00000 /* IOAPIC is mapped at */ +#define BUILD_PCIPREFMEM_END 0xdfffffff #endif
#define BUILD_APIC_ADDR 0xfee00000
Also, in the function pci_bios_allocate_region(), discards the prefectable memory region and places it in the non-prefectable memory region, if the bus is 0.
} else if ((val & PCI_BASE_ADDRESS_MEM_PREFETCH) && /* keep behaviour on bus = 0 */ pci_bdf_to_bus(bdf) != 0 && /* If pci_bios_prefmem_addr == 0, keep old behaviour */ pci_region_addr(&pci_bios_prefmem_region) != 0) {
I am not sure if this can be changed, can you please shade more light on this?
-Kevin
On Thu, Feb 10, 2011 at 03:51:04PM +0000, Prasad Joshi wrote:
Thanks a lot for your reply.
On Thu, Feb 10, 2011 at 3:31 PM, Kevin O'Connor kevin@koconnor.net wrote:
On Mon, Feb 07, 2011 at 07:17:52PM +0000, Prasad Joshi wrote:
Hello,
As I notice on my host machine the same case happens, but host BIOS seems to be allocating 128MB memory from some other region (c0000000-c7ffffff)
[...]
Can SeaBIOS do the same thing? or to put in other words Can SeaBIOS allocate memory from some other region instead of fixed f0000000 to (0xfec00000-1)?
It should be possible to alter BUILD_PCIMEM_START in src/config.h. The reason it is at 0xf0000000 is because regular memory can go up to 0xe0000000, and the bochs vga bios had soe dependency on the address space at 0xe0000000 - though this may no longer be the case.
I modified the memory range according the lspci -vv output. Atleast on my machine, the memory range 0xf0000000 to 0xfec00000 is reserved for non-prefectable memory.
Here is a small modification. As long as it keeps the behavior for old pci assignments, I guess this would not be a problem.
diff --git a/src/config.h b/src/config.h index 5b40488..19729cb 100644 --- a/src/config.h +++ b/src/config.h @@ -49,20 +49,18 @@
// Support old pci mem assignment behaviour //#define CONFIG_OLD_PCIMEM_ASSIGNMENT 1 -#if CONFIG_OLD_PCIMEM_ASSIGNMENT #define BUILD_PCIMEM_START 0xf0000000 #define BUILD_PCIMEM_SIZE (BUILD_PCIMEM_END - BUILD_PCIMEM_START) #define BUILD_PCIMEM_END 0xfec00000 /* IOAPIC is mapped at */
+#if CONFIG_OLD_PCIMEM_ASSIGNMENT #define BUILD_PCIPREFMEM_START 0 #define BUILD_PCIPREFMEM_SIZE 0 #define BUILD_PCIPREFMEM_END 0 #else -#define BUILD_PCIMEM_START 0xf0000000 -#define BUILD_PCIMEM_SIZE 0x08000000 /* half- of pci window */ -#define BUILD_PCIMEM_END (BUILD_PCIMEM_START + BUILD_PCIMEM_SIZE) -#define BUILD_PCIPREFMEM_START BUILD_PCIMEM_END +#define BUILD_PCIPREFMEM_START 0xc0000000
As before, I believe real memory could be at 0xc0000000-0xe0000000, so this range can't be used for PCI bars.
#define BUILD_PCIPREFMEM_SIZE (BUILD_PCIPREFMEM_END - BUILD_PCIPREFMEM_START) -#define BUILD_PCIPREFMEM_END 0xfec00000 /* IOAPIC is mapped at */ +#define BUILD_PCIPREFMEM_END 0xdfffffff #endif
#define BUILD_APIC_ADDR 0xfee00000
Also, in the function pci_bios_allocate_region(), discards the prefectable memory region and places it in the non-prefectable memory region, if the bus is 0.
} else if ((val & PCI_BASE_ADDRESS_MEM_PREFETCH) && /* keep behaviour on bus = 0 */ pci_bdf_to_bus(bdf) != 0 && /* If pci_bios_prefmem_addr == 0, keep old behaviour */ pci_region_addr(&pci_bios_prefmem_region) != 0) {I am not sure if this can be changed, can you please shade more light on this?
I think Isaku is more familiar with this code.
-Kevin
On Thu, Feb 10, 2011 at 11:11:41AM -0500, Kevin O'Connor wrote:
On Thu, Feb 10, 2011 at 03:51:04PM +0000, Prasad Joshi wrote:
Thanks a lot for your reply.
On Thu, Feb 10, 2011 at 3:31 PM, Kevin O'Connor kevin@koconnor.net wrote:
On Mon, Feb 07, 2011 at 07:17:52PM +0000, Prasad Joshi wrote:
Hello,
As I notice on my host machine the same case happens, but host BIOS seems to be allocating 128MB memory from some other region (c0000000-c7ffffff)
[...]
Can SeaBIOS do the same thing? or to put in other words Can SeaBIOS allocate memory from some other region instead of fixed f0000000 to (0xfec00000-1)?
It should be possible to alter BUILD_PCIMEM_START in src/config.h. The reason it is at 0xf0000000 is because regular memory can go up to 0xe0000000, and the bochs vga bios had soe dependency on the address space at 0xe0000000 - though this may no longer be the case.
I modified the memory range according the lspci -vv output. Atleast on my machine, the memory range 0xf0000000 to 0xfec00000 is reserved for non-prefectable memory.
Here is a small modification. As long as it keeps the behavior for old pci assignments, I guess this would not be a problem.
diff --git a/src/config.h b/src/config.h index 5b40488..19729cb 100644 --- a/src/config.h +++ b/src/config.h @@ -49,20 +49,18 @@
// Support old pci mem assignment behaviour //#define CONFIG_OLD_PCIMEM_ASSIGNMENT 1 -#if CONFIG_OLD_PCIMEM_ASSIGNMENT #define BUILD_PCIMEM_START 0xf0000000 #define BUILD_PCIMEM_SIZE (BUILD_PCIMEM_END - BUILD_PCIMEM_START) #define BUILD_PCIMEM_END 0xfec00000 /* IOAPIC is mapped at */
+#if CONFIG_OLD_PCIMEM_ASSIGNMENT #define BUILD_PCIPREFMEM_START 0 #define BUILD_PCIPREFMEM_SIZE 0 #define BUILD_PCIPREFMEM_END 0 #else -#define BUILD_PCIMEM_START 0xf0000000 -#define BUILD_PCIMEM_SIZE 0x08000000 /* half- of pci window */ -#define BUILD_PCIMEM_END (BUILD_PCIMEM_START + BUILD_PCIMEM_SIZE) -#define BUILD_PCIPREFMEM_START BUILD_PCIMEM_END +#define BUILD_PCIPREFMEM_START 0xc0000000
As before, I believe real memory could be at 0xc0000000-0xe0000000, so this range can't be used for PCI bars.
Qemu may assign RAM on [0xc0000000, 0xe0000000), so the area can't be used blindly for PCI bars. To enlarge PCI bar area < 4G, I suppose the followings are necessary. - enhance qemu RAM assignment logic not to allocate the region. i.e. customize top of low memory(TOLM). Maybe qemu property or machine specific option. - make the initialization for pci bar area, pci_bios_mem_region and pci_bios_prefmem_region, in seabios aware of RAM region. Maybe calculate those region dynamically based on TOLM
To allocate BARs more efficiently, they can be sorted in its size. I haven't implemented it because it requires one more pass to scan pci bus. One for sorting BARs and another for allocating BARs.
#define BUILD_PCIPREFMEM_SIZE (BUILD_PCIPREFMEM_END - BUILD_PCIPREFMEM_START) -#define BUILD_PCIPREFMEM_END 0xfec00000 /* IOAPIC is mapped at */ +#define BUILD_PCIPREFMEM_END 0xdfffffff #endif
#define BUILD_APIC_ADDR 0xfee00000
Also, in the function pci_bios_allocate_region(), discards the prefectable memory region and places it in the non-prefectable memory region, if the bus is 0.
} else if ((val & PCI_BASE_ADDRESS_MEM_PREFETCH) && /* keep behaviour on bus = 0 */ pci_bdf_to_bus(bdf) != 0 && /* If pci_bios_prefmem_addr == 0, keep old behaviour */ pci_region_addr(&pci_bios_prefmem_region) != 0) {I am not sure if this can be changed, can you please shade more light on this?
Seabios wasn't aware of prefetchable memory before my patches. I kept the old behavior for compatibility with the compile time option because I was afraid that there might be something which depends on the logic. If we don't care of it, we can drop the compat code.
So far no one has complained about the new behavior, it might be safe to drop it.
-
Isaku Yamahata -
Kevin O'Connor -
Prasad Joshi