Myles Watson wrote:
Ok thank you for all your help. I can boot and run Linux 2.6.33 now and I see all four CPU cores in /proc/cpuinfo. I compiled and added SeaBIOS as payload for coreboot and it works. I also compiled and added GPXE to the image and was able to PXE-boot. However, there are a few problems:
- The server has 4x1GB memory, but I only see 1GB. Why? What can I do?
Send the complete boot log as an attachment and see if anyone can spot
the
reason.
See attachment for my coreboot output.
I'd use SPEW for the console log level until you get it all figured out. That way you can see some more of what's happening.
The server has 4x512MB (not 4x1GB) memory, and only half the memory is visible. The /var/log/messages in Linux says:
Have you tried different configurations? Coreboot is only seeing the RAM on node 0. Where is the RAM on your board?
Scanning NUMA topology in Northbridge 24 Number of physical nodes 2 Node 0 MemBase 0000000000000000 Limit 000000003ffef000 Skipping disabled node 1 Using node hash shift of 63
I can see all four cores in /proc/cpuinfo and I can use them, but "numactl --hardware" shows that there is only one numa node detected, so something is wrong from BIOS.
I think it just means that there's only memory on one node.
- There is something wrong with the interrupts. When the kernel is
initializing the ethernet interfaces it says
"Determining IP information for eth0...Disabling IRQ #19"
Did you modify mptable.c and irq_tables.c for your board? The IRQ
routing
is probably different than what the s2881 has. You can run getpir and
look
in /proc/interrupts and lspci when you boot with the factory BIOS.
I'm not sure how those numbers and bitmasks in mptables.c, lspci, /proc/interrupts and irq_table.c correlate, but I'll keep trying to figure it out ... I have run getpir and mptable utilitys, but taking the output from these utilitys and putting it into the source tree will not make the server boot. I guess I need to study more exactly how this kind of hardware works, and how to interpret the tables and outputs from lspci.
You'll need more verbose output from lspci to see interrupts. lspci -vv will show you the interrupt pin and which IRQ it's routed to.
The output of lspci and /proc/interrupts from a factory BIOS DL145G1 (with two single-core Opterons) looks like this:
[root@cl200 ~]# lspci -tv -[0000:00]-+-01.0-[0000:03]-- +-01.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC +-02.0-[0000:02]--+-03.0 Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet | -03.1 Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet +-02.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC +-06.0-[0000:01]--+-00.0 Advanced Micro Devices [AMD] AMD-8111 USB
...
The same output from my dualcore DL145G1 with coreboot (booted with irqpoll kernel argument) looks as follows:
[root@cl199 ~]# lspci -tv -[0000:00]-+-01.0-[0000:01]----01.0 MYRICOM Inc. Myrinet 2000 Scalable Cluster Interconnect
This device doesn't show up with the factory BIOS.
+-01.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC +-02.0-[0000:02]--+-03.0 Broadcom Corporation NetXtremeBCM5704 Gigabit Ethernet | -03.1 Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet +-02.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC +-03.0-[0000:03]--+-00.0 Advanced Micro Devices [AMD] AMD-8111 USB
These devices have a different offset. It may make it easier to debug interrupt settings if the HyperTransport bus gets enumerated the same way.
I think switching this back to 6 in Kconfig will make it match again.
config HT_CHAIN_END_UNITID_BASE hex # default 0x6 default 0x20 depends on BOARD_HP_DL145_G1
Miscellaneous Control> [root@cl200 ~]# cat /proc/interrupts CPU0 CPU1 0: 125 0 IO-APIC-edge timer 1: 13 27 IO-APIC-edge i8042 4: 0 1 IO-APIC-edge 8: 0 0 IO-APIC-edge rtc0 9: 0 0 IO-APIC-fasteoi acpi 14: 5819 6709 IO-APIC-edge ide0 15: 0 0 IO-APIC-edge ide1 19: 0 0 IO-APIC-fasteoi ohci_hcd:usb1, ohci_hcd:usb2 31: 1220580 993 IO-APIC-fasteoi eth0
[root@cl199 ~]# cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 0: 218 0 0 0 IO-APIC-edge timer 1: 0 0 0 42 IO-APIC-edge i8042 2: 0 0 0 0 XT-PIC-XT cascade 4: 0 0 0 1 IO-APIC-edge 6: 0 0 0 3 IO-APIC-edge floppy 8: 0 0 0 0 IO-APIC-edge rtc0 12: 0 0 0 4 IO-APIC-edge i8042 14: 704 0 3 11547 IO-APIC-edge ide0 15: 0 0 0 0 IO-APIC-edge ide1 19: 0 1132 518 941 IO-APIC-fasteoi ohci_hcd:usb1, ohci_hcd:usb2 28: 0 0 0 0 IO-APIC-fasteoi eth0
I haven't played with mptables or pirq very much. A couple of things from mptable.c: /*I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# */ //8111 LPC ???? smp_write_intsrc(mc, mp_INT, MP_IRQ_TRIGGER_LEVEL|MP_IRQ_POLARITY_LOW, bus_8111_0, ((sysconf.sbdn+1)<<2)|0, apicid_8111, 0x13);
pin A = 0 pin B = 1 pin C = 2 pin D = 3
IRQ = (device <<2) | pin
The things you should have to play with most are the IRQ and the PIN#. The bus number should be a little more obvious. So this entry says to write an interrupt entry for the device at sysconf.sbdn+1 pin A to the APIC for the 81111, interrupt 0x13.
sysconf.sbdn should be the base device of the 8111 (6 if you switch it back.)
For your eth0, you should be able to modify one of these pieces:
This one assigns the interrupts from bus_8131_2, device 3 (A-D) to pins 0-3 on the APIC apicid_8131_2, which starts at 28 because of previous APICs and the interrupts they've taken.
//Slot 1 PCI-X 133/100/66 or Side 1 on raiser card for(i=0;i<4;i++) { smp_write_intsrc(mc, mp_INT, MP_IRQ_TRIGGER_LEVEL|MP_IRQ_POLARITY_LOW, bus_8131_2, (3<<2)|i, apicid_8131_2, (0+i)%4); //28 }
You might want to do ACPI instead. I've been able to find documentation and tools more easily for ACPI.
Thanks, Myles