Porting Qotom Q355G4 SBC (similar to Librem 13)

[John Keates]

Hi,

I’m new at the board porting game but I’m interested in making a port for a single board computer that is running very similar hardware to the Librem 13 laptop and the Google Pixel 2015.
It seems this board is also rather close to Intel’s own Eval Kit  hardware setup, both in hardware and software regard (almost bare defaults AMI Aptio firmware).

Looking at the coreboot archives, this board might have been mentioned before: https://mail.coreboot.org/pipermail/coreboot/2018-July/087050.html https://mail.coreboot.org/pipermail/coreboot/2018-July/087050.html but that was HTML formatted in an attachment for some reason.

Board specs:

- CPU: i5-5250U
- SuperIO: IT8784E-I 
- HDA: ALC662
- Flash: 8MByte Winbond SPI SOIC8 (but flashrom detects it as opaque when using internal programmer on-device, works correctly using external SPI flasher)
- 4x Intel I211 Ethernet controllers

more specifically, inteltool sees:

CPU: ID 0x306d4, Processor Type 0x0, Family 0x6, Model 0x3d, Stepping 0x4
Northbridge: 8086:1604 (5th generation (Broadwell family) Core Processor ULT)
Southbridge: 8086:9cc3 (unknown)
IGD: 8086:1626 (unknown)

Linux itself sees it as Wildcat Point-LP and  Broadwell-U which makes sense.

It has a running ME, but there is an unpopulated header (ME1) which when shorted seems to kill it or put it in manufacturing mode (soldered in a pin header and plugged in a jumper):

MEI found: [8086:9cba] Wildcat Point-LP MEI Controller #1

ME Status   : 0x1e040185
ME Status 2 : 0x10522106

ME: FW Partition Table      : OK
ME: Bringup Loader Failure  : NO
ME: Firmware Init Complete  : NO
ME: Manufacturing Mode      : NO
ME: Boot Options Present    : NO
ME: Update In Progress      : NO
ME: Current Working State   : Normal
ME: Current Operation State : Bring up
ME: Current Operation Mode  : Security Override via Jumper
ME: Error Code              : No Error
ME: Progress Phase          : BUP Phase
ME: Power Management Event  : Clean Moff->Mx wake
ME: Progress Phase State    : 0x52

When not using the jumper, flashrom can’t read anything using the internal programmer mode, but when plugging the jumper in, it can read it. Output is not the same as reading the SPI flash chip offline externally, so some of the flash it probably still hidden by the PCH as it always does.
There are exposed pads near the Realtek ALC chip which makes it really easy to pull GPIO33 to DVDD and disable flash descriptor security.

So far so good, the hardware has easy-to-reach points to get to the flash, ME jumper and PCH straps. The firmware isn’t very custom, and neither is the board (except the four ethernet controllers/ports).

My next steps were: cp -R the Purism directory to Qotom, remove all the Librem15 stuff, rename the Librem13 to Q3XXG4 and remove the EC references as this board doesn’t have an EC.
At this point, I’m not sure what to do next, besides the text in board_info.txt (both the vendor directory and the variant directories) I’ll probably have to make sure:

- acpi_tables.c has the correct tables, unless I can use the (generic looking) contents the Librem 13 uses, which doe acpi_init_gnvs, and acpi_create_madt_lapics + acpi_create_madt_ioapic
- there is fadt.c which seems rather generic as well. 
- gpio.h is probably depending on the PCH used? Seems to be generic for this PCH as well
- hda_verb.c seems to relate to the audio system, but I don’t care about it all that much at this time. The Librem 13 uses the ALC269 which has some differences to the ALC662, configuring it wrong probably just makes the chip sad and audio pin routing not work
- mainboard.c contains mainboard_enable and some calls for the GMA chip (install_intel_vga_int15_handler and some constants), probably the same accross this broadwell series
- romstage.c initialises the GPIOs does some PEI data copy/memset and then romstage_common (which probably is around or before the CAR stage?)
- acpi/mainboard.asl is empty, not sure what part of the existing ACPI I should dump in there

I turned the board into a variant, because Qotom has a number of boards that mostly differ in i3 vs i5 and more ethernet ports vs. more serial ports models. I named the variant q3xxg4 but I think we can do better as the PCB has a revision number printed on it, indicating more variants.
It looks like I’ll need to build some sort of devicetree.cb, perhaps by hand or converted from a ACPI dump from a running system? Then there is pei_data.c which seems to set the specifics for the hardware, the labels and locations.
While it is pretty close to the Librem 13 there as well (USB3 ports, USB2 ports) some of the locations need to be changed, and the camera, bluetooth, and speakers removed). Not sure how this relates to what is in the device tree and any *.asl files.

The old wiki and it’s porting guide isn’t up to date as far as I know, but I haven’t found a new guide or set of docs that shows how to read/source information from a running system and write it into a new mainboard definition.
I’ll probably need to get the GbE region from the old firmware, ME too, but what other parts of data (ACPI tables? PEI data?)I need to get and how to fill out that data into the mainboard definition isn’t very clear to me.

This device has a GPIO header, and two serial ports (one external DB9, one internal header), so getting debug data out should be possible, but just randomly building an image and flashing it in to the SPI chip probably gets me nothing until I have at least configured the serial port for coreboot to spit out debug data. I’m not sure if I need to get the FSP from Intel or if I should extract the one from the current firmware.

Any guidance would be appreciated.

Regards,
John

[Angel Pons]

Hello John,

Silly but crucial question: is Intel Boot Guard enabled on this board? If
it is enabled in Verified Mode, I am afraid replacing the firmware with
coreboot is impossible and proceeding any further is futile.
To make sure, you can check Intel Boot Guard's status with
coreboot/util/intelmetool. IIRC, another way to check would to change the
reset vector (last 16 bytes of the firmware image) on the vendor firmware
without changing what it actually does. The reset vector usually (if not
always) contains a JMP instruction, if you change what comes after it the
board should boot fine.

Please check this before doing anything else to avoid wasting time.

Best regards,

Angel Pons

[John Keates]

Hello Angel,

Thank you for your reply! Sadly, Boot Guard is enabled in Verified Boot mode. I’ll ask if Qotom can spin up a batch without any public key burned into the CPU, or perhaps share the private key. (which is obviously unlikely — but one can try)

Regards,
John
> On 28 Aug 2018, at 23:49, Angel Pons  wrote:
>
> Hello John,
>
> Silly but crucial question: is Intel Boot Guard enabled on this board? If it is enabled in Verified Mode, I am afraid replacing the firmware with coreboot is impossible and proceeding any further is futile.
> To make sure, you can check Intel Boot Guard's status with coreboot/util/intelmetool. IIRC, another way to check would to change the reset vector (last 16 bytes of the firmware image) on the vendor firmware without changing what it actually does. The reset vector usually (if not always) contains a JMP instruction, if you change what comes after it the board should boot fine.
>
> Please check this before doing anything else to avoid wasting time.
>
> Best regards,
>
> Angel Pons

[Nico Huber]

Hi John,

Am 29.08.18 um 04:02 schrieb John Keates:
> Thank you for your reply! Sadly, Boot Guard is enabled in Verified Boot
> mode. I’ll ask if Qotom can spin up a batch without any public key
> burned into the CPU, or perhaps share the private key. (which is
> obviously unlikely — but one can try)

there are other alternatives; BootGuard has not a single key but a
key hierarchy (probably for more profitable licensing schemes). So
you could try to get your own certificate signed by Qotom. But...
I can only advice against using BootGuard at all. If you don't have
one, you'd be looking forward to an NDA with Intel*. Also, you'd have
to integrate Intel's BootGuard ACM (piece of software that runs be-
fore host firmware) into your coreboot image and might also have to
adapt the early boot process of coreboot itself.

About the port itself. There is another pitfall: Intel FSP for Broad-
well is not integrated into upstream coreboot. What is integrated are
Google's blobs that do about the same as FSP but seem to be only licen-
sable as part of Chromebooks. IANAL, but I assume you could get into
trouble if you'd sell these blobs as part of your own product (not sure
if you want to sell anything). If you just want to use it in-house, that
might be possible, but again IANAL. (Welcome to the blob mess, btw.)

The alternative (with a much better license situation now) would be to
use Intel FSP. It does exist for Broadwell, it's "just" not integrated
upstream.

And some comments to your original mail:
> gpio.h is probably depending on the PCH used? Seems to be generic for
>
> this PCH as well

No, GPIO configuration is not generic. Never use the GPIO configuration
of another board! You might provoke short-circuits (I guess the PCH is
protected, but you never know what you break on the other side).

If you have another already working (reference) firmware for your board
(it seems so) you can dump the GPIO settings using `inteltool`.
> romstage.c initialises the GPIOs does some PEI data copy/memset and
>
> then romstage_common (which probably is around or before the CAR stage?)

This code runs with CAR enabled (most C code before ramstage does). The
PEI data is passed to the Google blob. The GPIO config and what happens
in `romstage.c` are the most crucial parts to get your board booting.
Alas, I can't tell you much about PEI data for Broadwell. Some settings
are board specific, e.g. memory configuration. You have to figure the
correct settings for your board (some questions may be answered by its
schematics).

Nico

* Well, I have to admit, an NDA has become useful anyway when porting
  coreboot. Mostly because Intel doesn't document what they are doing
  to coreboot.

[John Keates]

> On 31 Aug 2018, at 12:07, Nico Huber  wrote:
>
> Hi John,
>
> Am 29.08.18 um 04:02 schrieb John Keates:
> > Thank you for your reply! Sadly, Boot Guard is enabled in Verified Boot
> > mode. I’ll ask if Qotom can spin up a batch without any public key
> > burned into the CPU, or perhaps share the private key. (which is
> > obviously unlikely — but one can try)
>
> there are other alternatives; BootGuard has not a single key but a
> key hierarchy (probably for more profitable licensing schemes). So
> you could try to get your own certificate signed by Qotom. But...
> I can only advice against using BootGuard at all. If you don't have
> one, you'd be looking forward to an NDA with Intel*. Also, you'd have
> to integrate Intel's BootGuard ACM (piece of software that runs be-
> fore host firmware) into your coreboot image and might also have to
> adapt the early boot process of coreboot itself.

I thought the ACM was part of the ME or part of the Microcode itself, but I can be wrong.
From what I’ve read, the CPU has the hash of a public key burned in using fuses, but that can only be done during manufacturing with the PCH attached.
This was because the fuses are blown using the ME, and the ME has control over the first address the CPU will start executing code from. So the CPU has some minimal code besides the microcode, both is signed by Intel. That code is also used to verify with the ME that the SPI flash has signed contents and because the check is done super early, only the result is taken, not the action that might follow from the result; so good signature/bad signature only stops the CPU if the ME is active and a (signed) Boot Guard policy exists to do anything about this.
That is why the exploit with the older Gigabyte motherboards worked where you remove the BootGuard DXE (or was it a PEI module?) and then the Boot Guard init still works and lets the CPU run, but the follow-up (halting/continuing) is never executed.
The same was with a TXE bug where you can drop the ME into debug mode and access it’s JTAG via USB3, which can also be replaced in existing firmwares. When using JTAG, you can store permanent changes in the ME firmware using the ME CPU itself to disable Boot Guard pre-flash and also-post flash.

Another thing is the manufacturing mode PCH hard strap which puts the ME in insecure/BUP only mode, but I haven’t found any reference to what that does to Boot Guard. It would be fine for me if that disables Boot Guard follow-up (it still runs the init checks), because I can permanently bridge the ME mode pads to keep it in manufacturing mode. (this board has in fact two options, one is ME1 which is a jumper, and puts it in Jumper Security Override mode, another is a GPOP33 pin strap which either does ‘more’ or something similar.

I suppose I could check this by flipping a few bits in the SPI flash and checking if it still boots, and then pull the jumper and check again.
> About the port itself. There is another pitfall: Intel FSP for Broad-
> well is not integrated into upstream coreboot. What is integrated are
> Google's blobs that do about the same as FSP but seem to be only licen-
> sable as part of Chromebooks. IANAL, but I assume you could get into
> trouble if you'd sell these blobs as part of your own product (not sure
> if you want to sell anything). If you just want to use it in-house, that
> might be possible, but again IANAL. (Welcome to the blob mess, btw.)
>
> The alternative (with a much better license situation now) would be to
> use Intel FSP. It does exist for Broadwell, it's "just" not integrated
> upstream.

So first try, I’d have to use google blobs just to get it running (is that that Purism does as well?),
and when I get that running I can try the bare Intel FSP instead. I did know about the license changes, that was good news indeed (much more liberal and easier to interpret).
I’m probably not going to publicly sell the stuff (but I will share it for free), so the commercial part and distribution aspect won’t affect me there.
> And some comments to your original mail:
> > gpio.h is probably depending on the PCH used? Seems to be generic for
> >
> > this PCH as well
>
> No, GPIO configuration is not generic. Never use the GPIO configuration
> of another board! You might provoke short-circuits (I guess the PCH is
> protected, but you never know what you break on the other side).
>
> If you have another already working (reference) firmware for your board
> (it seems so) you can dump the GPIO settings using `inteltool`.

I figured as much; I did dump all of the current data I could using the various tools, It’s running Aptio UEFI in practically template mode with very little changes from the demo board vs. this production board.
> > romstage.c initialises the GPIOs does some PEI data copy/memset and
> >
> > then romstage_common (which probably is around or before the CAR stage?)
>
> This code runs with CAR enabled (most C code before ramstage does). The
> PEI data is passed to the Google blob. The GPIO config and what happens
> in `romstage.c` are the most crucial parts to get your board booting.
> Alas, I can't tell you much about PEI data for Broadwell. Some settings
> are board specific, e.g. memory configuration. You have to figure the
> correct settings for your board (some questions may be answered by its
> schematics).

Ah, so the PEI data I probably have to extract from the existing firmware and pass that to Google’s blobs to make it work. The memory configuration/training data is regenerated at first boot AFAIK, so as long as the Google blob or the standard Intel FSP does that, I should be fine.
> Nico
> Well, I have to admit, an NDA has become useful anyway when porting
>
> coreboot. Mostly because Intel doesn't document what they are doing
>  to coreboot.
> <0xBD56B4A4138B3CE3.asc>-- 
> coreboot mailing list: [e-mail]
> https://mail.coreboot.org/mailman/listinfo/coreboot

I think I’ll do some more experiments since it’s a good starter project to learn from for me, even it in the end I can’t get past Boot Guard, I’ll still have more knowledge than I did before. At that point I’ll probably move to ARM based boards anyway, unless I can by the U-series CPU’s myself from a shady vendor without any blown fuses, I do have a reball & SMT rework lab nearby where they have the equipment, steel stencils, paste and experience to replace SoC’s, GPU’s, CPU’s and the likes. As long as they can program a heating curve for this Intel part it should be fine, and as far as I know, they can X-Ray larger BGA’s as well, so checking the solder joint status on the balls should be OK. This is a stretch  (and a somewhat expensive hobby project at that point), but could be fun nonetheless.

Regards,
John

[Lil Evil]

Hi John,

I am the original poster of the thread you have quoted. all, apologies for the html formatted mail from my webmailer...
Great to see that I am not alone with my endeavor. However, due to other projects and lack of time, I have put this project temporarily on hold.
I have flashed a couple of coreboot trials, but the machine just spins up the fan in various patterns. Debug via serial console didn't work in this early stage. Getting some output there would be a great step ahead since I don't see any other debug option on this non-dev board.

regarding BootGuard:
I dont think bootguard is enforced, but its a while since I looked into the guts how this is acutally working.
However, I have patched my BIOS to include updated microcode, disabled ME and enabled all configuration options in the AMI bios. 
As a side effect I got an menuentry to enable secure boot and tinker with the key management, which I have not touched so far.
But based on the modifications I have done, I am pretty sure this shouldnt be an issue.

regarding FSP:
the Qotom board is very close to the intel dev board mentioned in my initial post and there are outdated coreboot patches under intel NDA that I couldnt get my hands on.
There is actually support for thie board in coreboot and you can get the FSP from intel's website, but coreboot is missing the FSP integration for those.
For FSP support the most probable option is to extract those from the firmware updates of google images, which I haven't started as I am tight up in other projects at the moment.

Let me know how I can best support your efforts
cheers
lIl

Gesendet: Samstag, 01. September 2018 um 21:01 Uhr
Von: "John Keates" 
An: "Nico Huber" 
Cc: coreboot 
Betreff: Re: [coreboot] Porting Qotom Q355G4 SBC (similar to Librem 13)
> On 31 Aug 2018, at 12:07, Nico Huber  wrote:
>
> Hi John,
>
> Am 29.08.18 um 04:02 schrieb John Keates:
> > Thank you for your reply! Sadly, Boot Guard is enabled in Verified Boot
> > mode. I’ll ask if Qotom can spin up a batch without any public key
> > burned into the CPU, or perhaps share the private key. (which is
> > obviously unlikely — but one can try)
>
> there are other alternatives; BootGuard has not a single key but a
> key hierarchy (probably for more profitable licensing schemes). So
> you could try to get your own certificate signed by Qotom. But...
> I can only advice against using BootGuard at all. If you don't have
> one, you'd be looking forward to an NDA with Intel*. Also, you'd have
> to integrate Intel's BootGuard ACM (piece of software that runs be-
> fore host firmware) into your coreboot image and might also have to
> adapt the early boot process of coreboot itself.

I thought the ACM was part of the ME or part of the Microcode itself, but I can be wrong.
> From what I’ve read, the CPU has the hash of a public key burned in using fuses, but that can only be done during manufacturing with the PCH attached.

This was because the fuses are blown using the ME, and the ME has control over the first address the CPU will start executing code from. So the CPU has some minimal code besides the microcode, both is signed by Intel. That code is also used to verify with the ME that the SPI flash has signed contents and because the check is done super early, only the result is taken, not the action that might follow from the result; so good signature/bad signature only stops the CPU if the ME is active and a (signed) Boot Guard policy exists to do anything about this.
That is why the exploit with the older Gigabyte motherboards worked where you remove the BootGuard DXE (or was it a PEI module?) and then the Boot Guard init still works and lets the CPU run, but the follow-up (halting/continuing) is never executed.
The same was with a TXE bug where you can drop the ME into debug mode and access it’s JTAG via USB3, which can also be replaced in existing firmwares. When using JTAG, you can store permanent changes in the ME firmware using the ME CPU itself to disable Boot Guard pre-flash and also-post flash.

Another thing is the manufacturing mode PCH hard strap which puts the ME in insecure/BUP only mode, but I haven’t found any reference to what that does to Boot Guard. It would be fine for me if that disables Boot Guard follow-up (it still runs the init checks), because I can permanently bridge the ME mode pads to keep it in manufacturing mode. (this board has in fact two options, one is ME1 which is a jumper, and puts it in Jumper Security Override mode, another is a GPOP33 pin strap which either does ‘more’ or something similar.

I suppose I could check this by flipping a few bits in the SPI flash and checking if it still boots, and then pull the jumper and check again.
> About the port itself. There is another pitfall: Intel FSP for Broad-
> well is not integrated into upstream coreboot. What is integrated are
> Google's blobs that do about the same as FSP but seem to be only licen-
> sable as part of Chromebooks. IANAL, but I assume you could get into
> trouble if you'd sell these blobs as part of your own product (not sure
> if you want to sell anything). If you just want to use it in-house, that
> might be possible, but again IANAL. (Welcome to the blob mess, btw.)
>
> The alternative (with a much better license situation now) would be to
> use Intel FSP. It does exist for Broadwell, it's "just" not integrated
> upstream.

So first try, I’d have to use google blobs just to get it running (is that that Purism does as well?),
and when I get that running I can try the bare Intel FSP instead. I did know about the license changes, that was good news indeed (much more liberal and easier to interpret).
I’m probably not going to publicly sell the stuff (but I will share it for free), so the commercial part and distribution aspect won’t affect me there.
> And some comments to your original mail:
> > gpio.h is probably depending on the PCH used? Seems to be generic for
> >
> > this PCH as well
>
> No, GPIO configuration is not generic. Never use the GPIO configuration
> of another board! You might provoke short-circuits (I guess the PCH is
> protected, but you never know what you break on the other side).
>
> If you have another already working (reference) firmware for your board
> (it seems so) you can dump the GPIO settings using `inteltool`.

I figured as much; I did dump all of the current data I could using the various tools, It’s running Aptio UEFI in practically template mode with very little changes from the demo board vs. this production board.
> > romstage.c initialises the GPIOs does some PEI data copy/memset and
> >
> > then romstage_common (which probably is around or before the CAR stage?)
>
> This code runs with CAR enabled (most C code before ramstage does). The
> PEI data is passed to the Google blob. The GPIO config and what happens
> in `romstage.c` are the most crucial parts to get your board booting.
> Alas, I can't tell you much about PEI data for Broadwell. Some settings
> are board specific, e.g. memory configuration. You have to figure the
> correct settings for your board (some questions may be answered by its
> schematics).

Ah, so the PEI data I probably have to extract from the existing firmware and pass that to Google’s blobs to make it work. The memory configuration/training data is regenerated at first boot AFAIK, so as long as the Google blob or the standard Intel FSP does that, I should be fine.
> Nico
> Well, I have to admit, an NDA has become useful anyway when porting
>
> coreboot. Mostly because Intel doesn't document what they are doing
> to coreboot.
> <0xBD56B4A4138B3CE3.asc>--
> coreboot mailing list: [e-mail]
> https://mail.coreboot.org/mailman/listinfo/coreboot

I think I’ll do some more experiments since it’s a good starter project to learn from for me, even it in the end I can’t get past Boot Guard, I’ll still have more knowledge than I did before. At that point I’ll probably move to ARM based boards anyway, unless I can by the U-series CPU’s myself from a shady vendor without any blown fuses, I do have a reball & SMT rework lab nearby where they have the equipment, steel stencils, paste and experience to replace SoC’s, GPU’s, CPU’s and the likes. As long as they can program a heating curve for this Intel part it should be fine, and as far as I know, they can X-Ray larger BGA’s as well, so checking the solder joint status on the balls should be OK. This is a stretch (and a somewhat expensive hobby project at that point), but could be fun nonetheless.

Regards,
John
--
coreboot mailing list: [e-mail]
https://mail.coreboot.org/mailman/listinfo/coreboot%5Bhttps://mail.coreboot....]

[Peter Stuge]

John Keates wrote:
> unless I can by the U-series CPU’s myself from a shady vendor without
> any blown fuses, I do have a reball & SMT rework lab nearby

Consider buying a donor mainboard for whatever system with the CPU you need
(in the state you need) and asking the lab to replace the one on your board.

I think it will be difficult to source a single CPU in a specific mode.

//Peter

[Piotr Król]

On 9/3/18 8:34 AM, Lil Evil wrote:
> Hi John,

Hi Lil,

(...)
> the Qotom board is very close to the intel dev board mentioned in my
> initial post and there are outdated coreboot patches under intel NDA
> that I couldnt get my hands on.

This sound very weird. coreboot is GPLv2 how Intel can keep source code
secret - I'm not a lawyer, but sounds very weird and I believe we can
address that through some channels in Intel. Can you point me to
document in Intel Research and Design Center so I can look at that? I
have correct CNDA signed with Intel.

Best Regards,

[John Keates]

> On 28 Dec 2018, at 00:36, Piotr Król  wrote:
>
> On 8/19/18 4:06 PM, John Keates wrote:
> > Hi,
>
> Hi John,

Hi!
> > I’m new at the board porting game but I’m interested in making a port
> > for a single board computer that is running very similar hardware to the
> > Librem 13 laptop and the Google Pixel 2015.
> > It seems this board is also rather close to Intel’s own Eval Kit
> >  hardware setup, both in hardware and software regard (almost bare
> > defaults AMI Aptio firmware).
> >
> > Looking at the coreboot archives, this board might have been mentioned
> > before: https://mail.coreboot.org/pipermail/coreboot/2018-July/087050.html but
> > that was HTML formatted in an attachment for some reason.
>
> Qotom are definitely interesting platforms :) And there are some OEM
> that build on top of their designs.
>
> I can't analyze whole your post, as probably most of people making
> coreboot professionally, but I have some pointers to you and would be
> glad to get some feedback from you if possible.
> > It has a running ME, but there is an unpopulated header (ME1) which when
> > shorted seems to kill it or put it in manufacturing mode (soldered in a
> > pin header and plugged in a jumper):
> >
> > MEI found: [8086:9cba] Wildcat Point-LP MEI Controller #1
> >
> > ME Status   : 0x1e040185
> > ME Status 2 : 0x10522106
> >
> > ME: FW Partition Table      : OK
> > ME: Bringup Loader Failure  : NO
> > ME: Firmware Init Complete  : NO
> > ME: Manufacturing Mode      : NO
> > ME: Boot Options Present    : NO
> > ME: Update In Progress      : NO
> > ME: Current Working State   : Normal
> > ME: Current Operation State : Bring up
> > ME: Current Operation Mode  : Security Override via Jumper
> > ME: Error Code              : No Error
> > ME: Progress Phase          : BUP Phase
> > ME: Power Management Event  : Clean Moff->Mx wake
> > ME: Progress Phase State    : 0x52
>
> I'm not sure if killing ME at porting time is good idea it can cause
> more problems then gain at this point. I would leave it as it is to keep
> things stable.

Yeah, I figured I’d better start off with a working firmware first anyway, but BootGuard still prevents that AFAIK.
> > When not using the jumper, flashrom can’t read anything using the
> > internal programmer mode, but when plugging the jumper in, it can read
> > it. Output is not the same as reading the SPI flash chip offline
> > externally, so some of the flash it probably still hidden by the PCH as
> > it always does.
>
> This is normal situation, flash descriptor, ME and maybe some SMI
> protect access to some regions. You should have external programmer and
> read firmware when platform is off - that can be tricky depending on
> platform schematics around SPI.
>
> What you should make sure:
> You have stock firmware version that can boot platform - this can be
>
> just offline read that I described above, online read for sure will
> cause platform brick.
> 2. You should care about flashing only BIOS region of flash and leave ME
> as it is.
> 3. Reading offline (let's call it golden stock version) and writing
> offline is good first test - it is risky and may end up in brick, but
> that's porting work so risk have to be calculated. If platform boot
> after that it is very good indicator, more to that you can think about
> writing random binary to bios region and verify that it is not booting
> and finally recover with your golden stock firmware version. If platform
> boot after that your are good to go with development.
>
> (…)

I did have a go at reading and writing the SPI flash offline (I’m used to embedded ARM and MIPS platforms),
It’s located reasonably accessible, just take the case off and find it next to the CPU (often the RTC battery is taped on top of it).
It has multiple footprint options it seems (or perhaps a socketed option), but the default SOIC-8 is easy enough to read, even with the cheap (and ever so fragile) pomona clip.

So at this point, I do have a few verified offline reads (so SMM, MMU, ME etc. don’t mess with it) which write back fine as well.
> > My next steps were: cp -R the Purism directory to Qotom, remove all the
> > Librem15 stuff, rename the Librem13 to Q3XXG4 and remove the EC
> > references as this board doesn’t have an EC.
> > At this point, I’m not sure what to do next, besides the text in
> > board_info.txt (both the vendor directory and the variant directories)
> > I’ll probably have to make sure:
> > acpi_tables.c has the correct tables, unless I can use the (generic
> >
> > looking) contents the Librem 13 uses, which doe acpi_init_gnvs,
> > and acpi_create_madt_lapics + acpi_create_madt_ioapic
> > there is fadt.c which seems rather generic as well.
> > gpio.h is probably depending on the PCH used? Seems to be generic for
> >
> > this PCH as well
> > hda_verb.c seems to relate to the audio system, but I don’t care about
> >
> > it all that much at this time. The Librem 13 uses the ALC269 which has
> > some differences to the ALC662, configuring it wrong probably just makes
> > the chip sad and audio pin routing not work
> > mainboard.c contains mainboard_enable and some calls for the GMA chip
> >
> > (install_intel_vga_int15_handler and some constants), probably the same
> > accross this broadwell series
> > romstage.c initialises the GPIOs does some PEI data copy/memset and
> >
> > then romstage_common (which probably is around or before the CAR stage?)
> > acpi/mainboard.asl is empty, not sure what part of the existing ACPI I
> >
> > should dump in there
>
> Probably hard to figure out without reverse engineering, schematics or
> experiments. The most scary thing is initial GPIO configuration which
> can cause hardware damage if set incorrectly. If you don't have
> schematics you should risk anyway - at least we do.
>
> (…)

Yeah, for this work it is reasonable (as far as risks go). The platform doesn’t have a lot of peripherals, it seems only the SuperIO and the HDA chip have some sidecar components with attached GPIOs.
> > The old wiki and it’s porting guide isn’t up to date as far as I know,
> > but I haven’t found a new guide or set of docs that shows how to
> > read/source information from a running system and write it into a new
> > mainboard definition.
> > I’ll probably need to get the GbE region from the old firmware, ME too,
> > but what other parts of data (ACPI tables? PEI data?)I need to get and
> > how to fill out that data into the mainboard definition isn’t very clear
> > to me.
>
> PEI is not important. ACPI defintely should be dumped, but if you can
> get back to golden stock firmware then it doesn't matter.

I got everything dumped, and I can go back, so that’s good.
> > This device has a GPIO header, and two serial ports (one external DB9,
> > one internal header), so getting debug data out should be possible, but
> > just randomly building an image and flashing it in to the SPI chip
> > probably gets me nothing until I have at least configured the serial
> > port for coreboot to spit out debug data. I’m not sure if I need to get
> > the FSP from Intel or if I should extract the one from the current firmware.
>
> Here are couple problems with that hardware:
> It doesn't seem that Intel provide FSP for Broadwell-U, there is
>
> something for Broadewll-DE:
> https://github.com/IntelFsp/FSP/tree/master/BroadwellDEFspBinPkg
> 2. Always use Intel most recent FSP
> 3. 90% chance that your stock BIOS doesn't use FSP, this is BIOS built
> from Intel Reference Code that AMI gets directly
> 4. There is cool debug option that can wear SPI, but if you have
> external programmer you can dump logs to SPI and read it from there:
> https://mail.coreboot.org/pipermail/coreboot/2017-June/084538.html
> https://github.com/coreboot/coreboot/blob/master/src/console/Kconfig#L293

Ah, the SPI logging is pretty cool, didn’t know about that. I did look in to the CCA tool, which apparently is just an USB3 cable with the USB2 Data pins and VCC pin removed. This does seem to require cooperation from the ME, so probably not worth a try at this time.
> Hope this will help.

The extra pointers are helpful indeed. I have started searches for ULV and mobile series single board computers that don’t have BootGuard enforcement enabled, but no luck so far.
But if I find one, I can definitely continue.
> Couple questions from me:
> What is the motivation behind this port?

Multiple; on one hand, I’m interested in the booting of more complex systems like x86 systems and large ARM SoCs, only scratched the surface on those, but it looks like fun.

On the other hand, I find the concept of hardware that you cannot boot with software of your own choice ridiculous, including low level boot roms and firmware.
I do understand that if a large corporation has a business model based on secrets, it’s a hard problem to solve to give less than 1% of customers something they are happy with while exposing some of those secrets.
Perhaps an option to get Intel to sign a FOSS ME engine with a key that is recognised by the on-die SoC/CPU boot ROM would be nice, but that is not likely to happen. Having only the on-die boot rom as a black box would be much more acceptable to me.
> Are you active in pfSense, OPNSense or any other UTM/firewall OS?

I’m active in OPNSense these days. I used to be rather active in pfSense, but the environment has gotten too toxic for me over the past years. I do have some side projects using Linux-based SDN setups on this type of hardware.
> Is this commercial activity or learning experience?

For me it is learning, but I imagine if I were to succeed, someone would be happy to make commercial use of it. Perhaps an integrator that takes Qotom whitelabel boxes and preloads them with an OS and firmware of choice.
Plenty of people would like a local cache of hardware so they don’t have to wait for a china shipment (that said, Qotom usually arrives within 48hrs here in western europe), and have local people to talk to.

So on one hand, if some vendor would use it to make things better for everyone, that’d be cool. On the other hand, if a company like Sophos were to start blindly reselling Qotom boxes, slap a Sophos logo on it, preinstall the free UTM and sell it at 1000% the worth of the product, I’d probably have an opinion about that, but then again, so would most people building stuff in their spare time ;-)
> As 3mdeb we are official maintainers of PC Engines and looking to
> support whole network appliance environment. Recently we working also on
> some support for Qotom based platforms.

I suspect you then have to get around the BootGuard issue, right? Because as far as I know, you cannot boot coreboot with BootGard fuses blown in enforcement mode.
PC Engines is a pretty cool company, used a lot of ALIX boards in the past (the ones with the tinyBIOS) which had a few issues between versions but always turned out great in the end.
I did switch to ARM based systems (often i.MX) and Intel based systems before the APU platforms became available locally because at that point, white label import became a real option.

Nice to see you guys on the list!
> Best Regards,
>
> Piotr Król
> Embedded Systems Consultant
> https://3mdeb.com | @3mdeb_com

Regards,
John

[Piotr Król]

On 12/28/18 3:48 AM, John Keates wrote:
> > On 28 Dec 2018, at 00:36, Piotr Król  wrote:
> >
> > On 8/19/18 4:06 PM, John Keates wrote:
> > > Hi,

Hi John,
> The extra pointers are helpful indeed. I have started searches for
> ULV and mobile series single board computers that don’t have
> BootGuard enforcement enabled, but no luck so far.

I would like to learn more about requirements here from my understanding
there should be big selection of devices that match your requirements.
Appollo Lake platforms should not have BootGuard since those mostly
target embedded market. Maybe Denverton-based hardware would also be
interesting to you. I'm not sure if it match your price point.

(...)
> > Are you active in pfSense, OPNSense or any other UTM/firewall OS?
>
> I’m active in OPNSense these days. I used to be rather active in
> pfSense, but the environment has gotten too toxic for me over the
> past years. I do have some side projects using Linux-based SDN setups
> on this type of hardware.

Understood. It looks like Qotom is pretty popular in UTM/firewall community.

(...)
> > As 3mdeb we are official maintainers of PC Engines and looking to
> > support whole network appliance environment. Recently we working also on
> > some support for Qotom based platforms.
> I suspect you then have to get around the BootGuard issue, right?
> Because as far as I know, you cannot boot coreboot with BootGard
> fuses blown in enforcement mode.

PC Engines is AMD based and there is no BootGuard there. There is
Hardware Validated Boot but hardware is shipped without enabling it,
truly it would be great if it would be possible to leverage that using
open source tools, but probably without PSP keys not much can be done.

(...)
> I did switch to ARM based systems (often i.MX) and Intel based
> systems before the APU platforms became available locally because at
> that point, white label import became a real option.

Yes, we also did some work with NXP but their mindset is very similar to
Intel - lot of stuff rely on proprietary tools that you can get only
signing NDAs and getting through their verification process.