[coreboot] Re: Intel Xeon D-1577 (16-core)

[Frans Hendriks]

This relocation is performed in a later stage, so some output is expected.
Is (correct) microcode included for D-1577?

I assume wbinvd() must be included in  smm_relocation() function.
(We have implemented coreboot on several Broadwell-DE boards with >4 cores 
without using wbinv() of 8:SPEW.)

Best regards,
Frans Hendriks
Eltan B.V.

-----Original Message-----
From: Jay Talbott [mailto:[e-mail]] 
Sent: woensdag 27 februari 2019 22:19
To: [e-mail]; [e-mail]
Subject: [coreboot] Re: Intel Xeon D-1577 (16-core)

There's a bug in the SMM relocation code for Broadwell-DE that causes a race
condition resulting in the SoC locking up during the MP init. With the stock
4-core SKU that comes in most of the Camelback Mountain CRBs, it's not a
problem, which is why Intel didn't find it when they developed the original
coreboot implementation for the CRB. But as has been reported previously on
this list, it becomes a problem with more than 4 cores. We actually have an
open ticket with Intel to see if they are willing to diagnose the root cause
and fix it, but I have zero expectation that any action will ever be done on
their part at this point.

If you turn up your console output level to 8:SPEW, the problem will go
away, as the extra printks that get enabled in that case result in
serialization of the SMM relocation code on each core, thus masking the race
condition (try this first!).

Another workaround is to insert a call to wbinvd() in function
smm_relocation_handler() in file
src/soc/intel/fsp_broadwell_de/smmrelocate.c. This will also result in
serialization that masks the race condition (fixed it for us on a 12-core
SKU) without needing to have the console turned all the way up.

At some point somebody needs to dig into the actual code in smmrelocate.c
and identify the root cause of the actual race condition. We just haven't
had the time to do any further investigation into the root cause since we
have a working workaround.

Hope that helps...

- Jay

Jay Talbott
Principal Consulting Engineer
SysPro Consulting, LLC
3057 E. Muirfield St.
Gilbert, AZ 85298
[phone]
[phone] (FAX)
[e-mail]
http://www.sysproconsulting.com
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]]
> Sent: Wednesday, February 27, 2019 12:46 PM
> To: [e-mail]
> Subject: [coreboot] Intel Xeon D-1577 (16-core)
>
> Hello,
>
> I got a daughter-card (DC) based on the Intel's Camelback Mountain CRB.
> Coreboot won't boot when a DC is populated with a 16-core Xeon D-1577
> processor. Nothing is printed in the boot process, so it doesn't seem to

be
> getting very far. However, if i load/program the boot SPI with AMI BIOS
> instead of coreboot, then everything is hunky-dory. It boots up all the

way
> into linux (see below for the platform information when AMI is loaded). In
> addition, if DC is populated with a 4-core D-1527 or 2-core D-1508 then
> coreboot has no issues (see below for info).
>
> Is there any configuration that i need to change in coreboot to support

the D-
> 1577?
>
> thanks!
>
> ## AMI BIOS
> ##
> @unassigned:~$ inxi -F
> System:    Host: unassigned Kernel: 4.13-platina-mk1 x86_64 (64 bit)
>            Console: tty 0 Distro: Debian GNU/Linux 8
> Machine:   Mobo: Default string model: Default string v: Default string
>            Bios: American Megatrends v: 5.11 date: 05/31/2017
> CPU:       Octa core Intel Xeon D-1577 (-HT-MCP-) cache: 24576 KB
>            Clock Speeds: 1: 1300 MHz 2: 1300 MHz 3: 1300 MHz 4: 1300 MHz
>            5: 1300 MHz 6: 1300 MHz 7: 1300 MHz 8: 1300 MHz
> Graphics:  Card: Failed to Detect Video Card!
>            Display Server: N/A driver: N/A
>            tty size: 80x24 Advanced Data: N/A out of X
> Network:   Card-1: Broadcom Device b960
>            IF: N/A state: N/A speed: N/A duplex: N/A mac: N/A
>            Card-2: Broadcom Device b960
>            IF: N/A state: N/A speed: N/A duplex: N/A mac: N/A
>            Card-3: Intel Device 15ab driver: ixgbe
>            IF: eth1 state: down mac: 00:a0:c9:00:00:00
>            Card-4: Intel Device 15ab driver: ixgbe
>            IF: eth2 state: down mac: 34:12:78:56:01:00
>            Card-5: Intel I210 Gigabit Network Connection driver: igb
>            IF: eth0 state: up speed: 1000 Mbps duplex: full
>            mac: 50:18:4c:00:16:a1
> Drives:    HDD Total Size: 520.1GB (4.0% used)
>            ID-1: /dev/sda model: TS512ZBTDM1500T size: 512.1GB
>            ID-2: USB /dev/sdb model: Echo size: 8.0GB
> Partition: ID-1: / size: 451G used: 942M (1%) fs: ext4 dev: /dev/sda1
>            ID-2: swap-1 size: 20.75GB used: 0.00GB (0%) fs: swap dev:

/dev/sda5
> RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
> Sensors:   System Temperatures: cpu: 56.0C mobo: N/A
>            Fan Speeds (in rpm): cpu: N/A
> Info:      Processes: 118 Uptime: 17:49 Memory: 110.9/32087.7MB
>            Init: systemd runlevel: 5 Client: Shell (bash) inxi: 2.1.28
>
> ## Coreboot on 4-core D-1527:
> ##
> root@invader0:~# POST: 0x4a
> romstage_main_continue status: 0  hob_list_ptr: 7f100000
> FSP Status: 0x0
> POST: 0x4b
> POST: 0x4c
> POST: 0x4d
> CBMEM:
> IMD: root @ 7efff000 254 entries.
> IMD: root @ 7effec00 62 entries.
> POST: 0x4e
> CBFS: 'Master Header Locator' located CBFS at [800100:ffffc0)
> CBFS: Locating 'fallback/ramstage'
> CBFS: Found @ offset 33b80 size d857
>
> coreboot-v0.4-5-g0e4829a5b5 Wed Jun 20 18:38:46 UTC 2018 ramstage
> starting...
> POST: 0x39
> Moving GDT to 7effe9e0...ok
> POST: 0x80
>
> ##
> root@invader0:~# inxi -F
> System:    Host: invader0 Kernel: 4.13-platina-mk1 x86_64 (64 bit)
>            Console: tty 0 Distro: Debian GNU/Linux 8
> Machine:   Mobo: Intel model: Camelback Mountain Platina DC v: 1.0 serial:
> 123456789
>            Bios: coreboot v: v0.4-5-g0e4829a5b5 date: 06/20/2018
> CPU:       Quad core Intel Xeon D-1527 (-HT-MCP-) cache: 6144 KB
>            Clock Speeds: 1: 2194 MHz 2: 2194 MHz 3: 2194 MHz 4: 2194 MHz
>            5: 2194 MHz 6: 2194 MHz 7: 2194 MHz 8: 2194 MHz
> Graphics:  Card: Failed to Detect Video Card!
>            Display Server: N/A driver: N/A
>            tty size: 80x24 Advanced Data: N/A for root out of X
> Network:   Card-1: Broadcom Device b960 driver: vfio-pci
>            IF: N/A state: N/A speed: N/A duplex: N/A mac: N/A
>            Card-2: Broadcom Device b960 driver: vfio-pci
>            IF: N/A state: N/A speed: N/A duplex: N/A mac: N/A
>            Card-3: Intel Device 15ab driver: ixgbe
>            IF: eth1 state: down mac: 50:18:4c:00:16:a2
>            Card-4: Intel Device 15ab driver: ixgbe
>            IF: eth2 state: down mac: 50:18:4c:00:16:a3
>            Card-5: Intel I210 Gigabit Network Connection driver: igb
>            IF: eth0 state: up speed: 1000 Mbps duplex: full
>            mac: 50:18:4c:00:16:a1
> Drives:    HDD Total Size: 136.1GB (5.0% used)
>            ID-1: /dev/sda model: SanDisk_SD8SMAT1 size: 128.0GB
>            ID-2: USB /dev/sdb model: Echo size: 8.0GB
> Partition: ID-1: / size: 98G used: 1.4G (2%) fs: ext4 dev: /dev/sda6
>            ID-2: swap-1 size: 5.70GB used: 0.00GB (0%) fs: swap dev:

/dev/sda5
> RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
> Sensors:   System Temperatures: cpu: 56.0C mobo: N/A
>            Fan Speeds (in rpm): cpu: N/A
> Info:      Processes: 133 Uptime: 1 min Memory: 149.1/16078.1MB
>            Init: systemd runlevel: 5 Client: Shell (bash) inxi: 2.1.28
> root@invader0:~#
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

_______________________________________________
coreboot mailing list -- [e-mail]
To unsubscribe send an email to [e-mail]