(smc_x11) x11ssh-tf Board, Documentation

[Andreas]

I spent today trying to get the Supermicro X11SSH-TF working with
the latest Coreboot 4.14 from git.

Board is at vendor firmware 2.5 (latest) and also most recent BMC firmware.

Luckily, writing the image has been possible via the vendor BMC (which
preferably I would like to also replace with u-bmc once Coreboot runs).

After reboot into CB, the screen stays blank. I do not have a serial
connection available at the moment. I do have the BMC, though:

The vendor BMC recognizes the Coreboot image and correctly displays
header information such as build time and version (4.14). Also, the
BMC does read the POST code from Coreboot. It rather quickly cycles
through values and ends up at 0xE3 every time. And stops there.

According to the sources, that step is related to memory init. So to
be sure, I swapped out the Samsung 2x16GB ECC modules with standard
Non-ECC Corsair DDR4. Both functional with the Supermicro firmware,
both POSTing to 0xE3 with Coreboot.

Is there any chance a recent commit has broken stuff with this board?
I read reports that this board worked for some people, so I must have
made a mistake?

Basically I followed the documentation, used the provided configuration
options and supplied the me.bin and descriptor.bin extracted from the
original firmware and kept the rest to defaults. Fsp is taken from the
3rd party repo. Is that fine for this platform?

I would appreciate any pointers or ideas what might hinder success. Will
there be debug output written to the serial console?

If anybody has the board working, could he/she share his
configuration, Coreboot version and 3rd party blobs used?

Also, once I get to run I would love to improve the documentation, so
that people new to the subject like me have an easier time. I went
through a couple of pitfalls that I would like to explain so others
can follow in my footsteps.

-Andreas

[Valentin]

Hi,

We had a working coreboot on an x11ssh-ctf but since switched to the 
x11ssm-f due to issues with the Intel x550 NICs in our Network.

it was a bit of hassle to get it working and we needed some help from 
the list here to get it to boot.
i'll attach the config we used.
my notes from back then (2019-12) are:

`For this board two binary blobs have to be extracted from the original 
firmware:
  - descriptor.bin
  - me.bin
Those are checked saved under blobs in this repo, for building coreboot 
they are expected in the subdirectory 
3rdparty/blobs/mainboard/supermicro/x11-lga1151-series.`

Hope this helps you,
Valentin

On 21/06/2021 18:49, Andreas wrote:
> I spent today trying to get the Supermicro X11SSH-TF working with
> the latest Coreboot 4.14 from git.
>
> Board is at vendor firmware 2.5 (latest) and also most recent BMC firmware.
>
> Luckily, writing the image has been possible via the vendor BMC (which
> preferably I would like to also replace with u-bmc once Coreboot runs).
>
> After reboot into CB, the screen stays blank. I do not have a serial
> connection available at the moment. I do have the BMC, though:
>
> The vendor BMC recognizes the Coreboot image and correctly displays
> header information such as build time and version (4.14). Also, the
> BMC does read the POST code from Coreboot. It rather quickly cycles
> through values and ends up at 0xE3 every time. And stops there.
>
> According to the sources, that step is related to memory init. So to
> be sure, I swapped out the Samsung 2x16GB ECC modules with standard
> Non-ECC Corsair DDR4. Both functional with the Supermicro firmware,
> both POSTing to 0xE3 with Coreboot.
>
> Is there any chance a recent commit has broken stuff with this board?
> I read reports that this board worked for some people, so I must have
> made a mistake?
>
> Basically I followed the documentation, used the provided configuration
> options and supplied the me.bin and descriptor.bin extracted from the
> original firmware and kept the rest to defaults. Fsp is taken from the
> 3rd party repo. Is that fine for this platform?
>
> I would appreciate any pointers or ideas what might hinder success. Will
> there be debug output written to the serial console?
>
> If anybody has the board working, could he/she share his
> configuration, Coreboot version and 3rd party blobs used?
>
> Also, once I get to run I would love to improve the documentation, so
> that people new to the subject like me have an easier time. I went
> through a couple of pitfalls that I would like to explain so others
> can follow in my footsteps.
>
> -Andreas
> _______________________________________________
> coreboot mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Andreas]

Thanks to Valentin's config the board is booting. Now the main issue
are the Intel X550 NICs that will go into firmware-recovery mode after
some time and traffic. I am looking into that now.

Has someone ever had a X550 working with(after) coreboot? I am trying
to identify the area to look in. Is it some Bringup code in the BIOS, or is it
related to the BMC as according to the Intel datasheet the BMC has a
side channel directly to the NIC. Ugh :(

On a sidenote, coreboot output can only be found in CBMEM, VGA output
is only seen once SeaBIOS initializes. Is there still a configuration option to
tweak or is that related to the ASpeed device? Not a deal breaker, but it
would be nice to have.

Thanks for all contributions to coreboot. What is the process to update the
documentation for the board?

-Andreas

[11889.895067] ixgbe 0000:03:00.0: Adapter removed
[11889.895069] ixgbe 0000:03:00.0: Warning firmware error detected
FWSM: 0xFFFFFFFF
[11889.895071] ixgbe 0000:03:00.0: Firmware recovery mode detected.
Limiting functionality. Refer to the Intel(R) Ethernet Adapters and
Devices User Guide for details on firmware recovery mode.

Am Di., 22. Juni 2021 um 13:31 Uhr schrieb Valentin :
> Hi,
>
> We had a working coreboot on an x11ssh-ctf but since switched to the
> x11ssm-f due to issues with the Intel x550 NICs in our Network.
>
> it was a bit of hassle to get it working and we needed some help from
> the list here to get it to boot.
> i'll attach the config we used.
> my notes from back then (2019-12) are:
>
> `For this board two binary blobs have to be extracted from the original
> firmware:
> descriptor.bin
> me.bin
>
> Those are checked saved under blobs in this repo, for building coreboot
> they are expected in the subdirectory
> 3rdparty/blobs/mainboard/supermicro/x11-lga1151-series.`
>
> Hope this helps you,
> Valentin
>
> On 21/06/2021 18:49, Andreas wrote:
> > I spent today trying to get the Supermicro X11SSH-TF working with
> > the latest Coreboot 4.14 from git.
> >
> > Board is at vendor firmware 2.5 (latest) and also most recent BMC firmware.
> >
> > Luckily, writing the image has been possible via the vendor BMC (which
> > preferably I would like to also replace with u-bmc once Coreboot runs).
> >
> > After reboot into CB, the screen stays blank. I do not have a serial
> > connection available at the moment. I do have the BMC, though:
> >
> > The vendor BMC recognizes the Coreboot image and correctly displays
> > header information such as build time and version (4.14). Also, the
> > BMC does read the POST code from Coreboot. It rather quickly cycles
> > through values and ends up at 0xE3 every time. And stops there.
> >
> > According to the sources, that step is related to memory init. So to
> > be sure, I swapped out the Samsung 2x16GB ECC modules with standard
> > Non-ECC Corsair DDR4. Both functional with the Supermicro firmware,
> > both POSTing to 0xE3 with Coreboot.
> >
> > Is there any chance a recent commit has broken stuff with this board?
> > I read reports that this board worked for some people, so I must have
> > made a mistake?
> >
> > Basically I followed the documentation, used the provided configuration
> > options and supplied the me.bin and descriptor.bin extracted from the
> > original firmware and kept the rest to defaults. Fsp is taken from the
> > 3rd party repo. Is that fine for this platform?
> >
> > I would appreciate any pointers or ideas what might hinder success. Will
> > there be debug output written to the serial console?
> >
> > If anybody has the board working, could he/she share his
> > configuration, Coreboot version and 3rd party blobs used?
> >
> > Also, once I get to run I would love to improve the documentation, so
> > that people new to the subject like me have an easier time. I went
> > through a couple of pitfalls that I would like to explain so others
> > can follow in my footsteps.
> >
> > -Andreas
> > _______________________________________________
> > coreboot mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]

[Valentin]

Hi,
> Thanks to Valentin's config the board is booting. Now the main issue
> are the Intel X550 NICs that will go into firmware-recovery mode after
> some time and traffic. I am looking into that now.

Good to hear. Unfortunately we had exactly the same issue with the x550 
NICs.
It always starts with the message:

[ 2041.230487] ixgbe 0000:08:00.0 eth4: Detected Tx Unit Hang
                  Tx Queue             <1>
                  TDH, TDT             <1bb>, <1e5>
                  next_to_use          <1e5>
                  next_to_clean        <1b9>
                tx_buffer_info[next_to_clean]
                  time_stamp           <100069e13>
                  jiffies              <10006a370>
[ 2041.265172] ixgbe 0000:08:00.0 eth4: tx hang 1 detected on queue 1, 
resetting adapter
> Has someone ever had a X550 working with(after) coreboot? I am trying
> to identify the area to look in. Is it some Bringup code in the BIOS, or is it
> related to the BMC as according to the Intel datasheet the BMC has a
> side channel directly to the NIC. Ugh :(

I've read many datasheets and tried to reflash it but the firmware and 
memory structure of this NIC seem quite complicated to me and we 
couldn't even figure out which chip on the motherboard holds the firmware...
Anyways we used the ixgbe module option "debug=16" (16 is all output, 
i'm not suer anymore if we really used 16 or some lower level...) to 
find out more, I attached dmesg output from the error including register 
dumps of the x550.
> On a sidenote, coreboot output can only be found in CBMEM, VGA output
> is only seen once SeaBIOS initializes. Is there still a configuration option to
> tweak or is that related to the ASpeed device? Not a deal breaker, but it
> would be nice to have.

We only used the serial connection which should just work on the rear 
dsub port with our config.

Cheers,
Valentin

[Andreas]

Quoting myself:
> Thanks to Valentin's config the board is booting. Now the main issue
> are the Intel X550 NICs that will go into firmware-recovery mode after
> some time and traffic. I am looking into that now.

After some playing around, it seems the Ethernet is now stable. I am not
sure what action is responsible for this outcome. It could be that just a
power cut was responsible (because I flashed the image with the BMC,
the board never went actually completely power-off).

Again, I would appreciate any feedback from people who also have the
board (-TF or -CTF) working if they encountered any trouble with the X550.

-Andreas