Hi everybody,
Me and Federico Amedeo Izzo (federico.izzo42@gmail.com) started looking into our Intel ME firmwares after reading the thread "Experiments with disabling the ME on Sandy Bridge x230" started by Trammell Hudson.
We have two Thinkpad X220 (Sandy Bridge) and a X201 (Nehalem), and we obtained the following results:
* Sandy Bridge accepts an Intel ME firmware with just the FTPR partition, both with and without a valid FPT (the partition table of the Intel ME image). The system doesn't power off after 30 minutes, and the ME reports a successful initialization (see attached cbmem). To be extra safe we wrote a small Python script that removes all the non-fundamental partitions and creates a new FPT with a single partition entry (following the structure in [1] and some hints from Igor Skochinsky).
* Sandy Bridge boots successfully without a good FPT (but with a valid FTPR), so it probably has a "backup" on-chip FPT. The system boots fine and doesn't power itself off after 30 minutes, even if the cbmem entry "ME: FW Partition Table" has the value "BAD" (that, we suppose, means: "I found a broken partition table, I'm using the fallback one").
* According to Trammell Hudson's tests, this behaviour is the same in Ivy Bridge (the thread's subject says "Sandy Bridge" but his cbmem confirms that his device is actually an Ivy Bridge)
* Nehalem (on our X201) seems to lack a "backup" on-chip FPT, as removing the FPT partition bricks the device. Moreover using an ME image with just FPT and FTPR results in a brick, Nehalem probably needs more parts of the blob than just FPT and FTPR. Unfortunately our X201 is in a bad state and its flash is about to die, so we can't test further on Nehalem.
* We couldn't remove in any way the FTPR partition on Sandy Bridge: setting the entries number to 0, the UMA size to 0 MB, creating an invalid FPT with a bad checksum or creating a "fake" FTPR entry which points to an empty region leads to a bricked device. Apparently, no matter which trick is used, if ME can't find the FTPR partition it refuses to boot. We didn't try with a 0-sized partition, but the result is probably the same.
* Probably the UMA size can be reduced from the "stock" 16 MB or 32 MB to few MB, we'll investigate on this.
* Shrinking the "reduced" ME partition in IFD to the minimum size allowed (FTPR offset + FTPR size + 4 kB alignment) bricks the device too (but we're not sure about this, maybe we made a mistake during the image creation).
* Relocating the FTPR image doesn't work, even if the FPT entry points to the new location.
* We haven't understood yet how to remove the unneeded modules from the FTPR partition, any help is appreciated.
* The cbmem's ME entries * ME: Current Working State : Normal / Recovery * ME: Bringup Loader Failure : YES / NO * ME: Progress Phase : ROM Phase / BUP Phase seem to be interesting, and should be studied further.
Attached you can find the script we used to "clean" the ME images, the cbmem log of an untouched ME image (the 2.1 MB firmware extracted from a Acer Chromebook C710) and the cbmem log of the same image after executing the script on it (therefore with just the FPT with a single entry and the FTPR).
Some help testing the further options would be nice, in particular on the Nehalem architecture. Be careful: messing with ME may cause a hard brick, without the 30 minutes window, requiring an external programmer to unbrick your PC.
Nicola Corna Federico Amedeo Izzo
Hi Nicola,
Nice work! I would like to add some comments inline.
Friday, November 4, 2016, 10:20:24 PM, you wrote:
NC> * Sandy Bridge boots successfully without a good FPT (but with a valid FTPR), NC> so it probably has a "backup" on-chip FPT. The system boots fine and doesn't NC> power itself off after 30 minutes, even if the cbmem entry "ME: FW NC> Partition Table" has the value "BAD" (that, we suppose, means: "I found a NC> broken partition table, I'm using the fallback one").
AFAIK there is no "backup FPT", there is just the internal boot ROM which tries to load the main fimware from the ME region and fails if FPT is bad/missing. On failure, it does set the FW status registers which are visible on the host side in the PCI device config space, which is how you can see the "FPT bad" bit and the boot phase. I think it just happens that on this platform the boot rom does just enough hw initialization for the rest of the boot to work, even without the BUP and all other code.
Incidentally, I believe that FTPR stands for "Factory partition (&recovery)" and contains the minimal modules required for the successful boot of the host and firmware update process (during firmware update, the FTPR is updated last, so that the system can still boot if the update fails at any earlier stage).
NC> * According to Trammell Hudson's tests, this behaviour is the same in Ivy NC> Bridge (the thread's subject says "Sandy Bridge" but his cbmem confirms that NC> his device is actually an Ivy Bridge)
NC> * Nehalem (on our X201) seems to lack a "backup" on-chip FPT, as removing the NC> FPT partition bricks the device. NC> Moreover using an ME image with just FPT and FTPR results in a brick, NC> Nehalem probably needs more parts of the blob than just FPT and FTPR. NC> Unfortunately our X201 is in a bad state and its flash is about to die, so NC> we can't test further on Nehalem.
NC> * We couldn't remove in any way the FTPR partition on Sandy Bridge: setting the NC> entries number to 0, the UMA size to 0 MB, creating an invalid FPT with a NC> bad checksum or creating a "fake" FTPR entry which points to an empty region NC> leads to a bricked device. Apparently, no matter which trick is used, if ME NC> can't find the FTPR partition it refuses to boot. NC> We didn't try with a 0-sized partition, but the result is probably the same.
NC> * Probably the UMA size can be reduced from the "stock" 16 MB or 32 MB to few NC> MB, we'll investigate on this. NC> NC> * Shrinking the "reduced" ME partition in IFD to the minimum size allowed (FTPR NC> offset + FTPR size + 4 kB alignment) bricks the device too (but we're not NC> sure about this, maybe we made a mistake during the image creation).
You can try using Intel's Flash Image Tool to construct the flash image from the ME and BIOS regions and see if it makes a different IFD.
NC> * Relocating the FTPR image doesn't work, even if the FPT entry points to the NC> new location.
This is probably due to the Huffman-compressed modules, which use a table of offsets to compressed chunks and those offsets are from the start of the ME region. This table will need to be relocated too. For more info, see references to "LUT" in unhuffme[1] source code.
NC> * We haven't understood yet how to remove the unneeded modules from the FTPR NC> partition, any help is appreciated.
You can't do that as the whole partition header ("manifest") is protected by the RSA signature, including the module table. I've just uploaded to my github[2] a script (me_sigcheck.py) which allows to check the validity of the signature, so you can try various modifications yourself.
NC> * The cbmem's ME entries NC> * ME: Current Working State : Normal / Recovery NC> * ME: Bringup Loader Failure : YES / NO NC> * ME: Progress Phase : ROM Phase / BUP Phase NC> seem to be interesting, and should be studied further.
It seems "Recovery" in your case may be caused by the missing MFS partition (MFS failure : 0x1). What you can try is to copy a clean MFS partition from a non-initialized ME image[3] - this way it will pass the check and maybe not go into the recovery. It may then be possible to shrink the MFS partition to a minimal size to gain more space.
[1] https://io.netgarage.org/me/ [2] https://github.com/skochinsky/me-tools [3] http://www.win-raid.com/t1658f39-Guide-Clean-the-Engine-Initialized-DATA-sec...
NC> Attached you can find the script we used to "clean" the ME images, the cbmem log NC> of an untouched ME image (the 2.1 MB firmware extracted from a Acer Chromebook NC> C710) and the cbmem log of the same image after executing the script on it NC> (therefore with just the FPT with a single entry and the FTPR).
NC> Some help testing the further options would be nice, in particular on the NC> Nehalem architecture. NC> Be careful: messing with ME may cause a hard brick, without the 30 minutes NC> window, requiring an external programmer to unbrick your PC.
NC> Nicola Corna NC> Federico Amedeo Izzo
NC> [1] http://me.bios.io/ME_blob_format
Hi everybody,
- Sandy Bridge accepts an Intel ME firmware with just the FTPR partition, both with and without a valid FPT (the partition table of the Intel ME image). The system doesn't power off after 30 minutes, and the ME reports a successful initialization (see attached cbmem). To be extra safe we wrote a small Python script that removes all the non-fundamental partitions and creates a new FPT with a single partition entry (following the structure in [1] and some hints from Igor Skochinsky).
I failed to build a usable coreboot image for my own X220, so I made a "crippled/cleansed" ME from factory bios image via Nicola's script and insert the crippled ME back to the factory bios via coreboot's ifdtool. Then I flashed the modified factory bios to the laptop.
Now my x220 has survived nearly one hour, and the MEI has disappeared from PCI bus, but, additionally, the integrated NIC ceased to be functional with such error seen from dmesg:
[ 2.608150] e1000e: probe of 0000:00:19.0 failed with error -3
I do not know whether this phenominon has been observed, but I believe it may be natural after ME gets crippled, since I have experienced a broken integrated NIC on an HP eliteboot 8460p whose ME is broken.
This phenominon may suggest on such platform, the functionality of integrated NIC is actually implemented by the overlord called ME, and thus the integrated NIC is ME's backdoor.
The output of lspci(8) is attached.
On 11/11/2016 10:16 AM, Persmule wrote:
Hi everybody, After a manual shutdown and repower, the integrated NIC is functional again. Has similar phenominon been observed yet?
Now my x220 has survived nearly one hour, and the MEI has disappeared from PCI bus, but, additionally, the integrated NIC ceased to be functional with such error seen from dmesg:
Hi Persmule, Thanks for trying out Nicola's script to clean the ME blob.
Be careful whenever you try changing ME blob to poweroff completely the computer, and start again, because apparently if you don't reboot the ME won't load the new firmware from the flash. In fact it happened to me that after flashing a non valid ME firmware on my X201 the computer survived the reboot but became a brick after shutdown.
The possibility of replacing the ME blob inside an official (Lenovo or other) bios is very interesting because it extends also to hardware not supported by coreboot, and probabily to CPUs newer than Ivy Bridge (Trammell Hudson tested it on a Skylake mobile CPU https://www.coreboot.org/pipermail/coreboot/2016-November/082335.html)
On Sat, Nov 12, 2016 at 03:41:17PM +0100, Federico Amedeo Izzo wrote:
Be careful whenever you try changing ME blob to poweroff completely the computer, and start again, because apparently if you don't reboot the ME won't load the new firmware from the flash.
I've also found that the PCH sometimes keeps the SPI flash lines occupied on some platforms if there is any battery or AC power supplied. So I almost always have a full power-down between flashing to avoid having multiple drivers on those pins.
[...] The possibility of replacing the ME blob inside an official (Lenovo or other) bios is very interesting because it extends also to hardware not supported by coreboot, and probabily to CPUs newer than Ivy Bridge (Trammell Hudson tested it on a Skylake mobile CPU https://www.coreboot.org/pipermail/coreboot/2016-November/082335.html)
The mobile Skylake works fine with the reduced ME image, although I'm not sure if there are power management issues with it. The system is still drawing nearly 5W, when I think it should be closer to 2-3. Next week I plan to update to a 4.8 kernel to see if that makes a difference since I've read that there were changes.
Another bit of research: the ME image has the Bootguard profile and key hash, but it is ignored if the CPU has exited manufacturing mode. During manufacturing mode the OEM can set these and they will be copied into the fuses in the CPU and not be changed afterwards. My Skylake does not have any bootguard profile enabled, so I haven't been able to experiment with how the boot ACM is affected by the ME changes.
This might be a worthwhile experiment if anyone has a T450/460 with bootguard and wants to try flashing their ME image.
Hi Federico,
You guys seem to have successfully built a usable coreboot image for x220. Could you share its revision and config file? I have never built such usable image till now.
Besides, is script file me_cleaner.py downloadable somewhere?
Persmule.
在 2016年11月12日 22:41, Federico Amedeo Izzo 写道:
On 11/11/2016 10:16 AM, Persmule wrote:
Hi everybody, After a manual shutdown and repower, the integrated NIC is functional again. Has similar phenominon been observed yet?
Now my x220 has survived nearly one hour, and the MEI has disappeared from PCI bus, but, additionally, the integrated NIC ceased to be functional with such error seen from dmesg:
Hi Persmule, Thanks for trying out Nicola's script to clean the ME blob.
Be careful whenever you try changing ME blob to poweroff completely the computer, and start again, because apparently if you don't reboot the ME won't load the new firmware from the flash. In fact it happened to me that after flashing a non valid ME firmware on my X201 the computer survived the reboot but became a brick after shutdown.
The possibility of replacing the ME blob inside an official (Lenovo or other) bios is very interesting because it extends also to hardware not supported by coreboot, and probabily to CPUs newer than Ivy Bridge (Trammell Hudson tested it on a Skylake mobile CPU https://www.coreboot.org/pipermail/coreboot/2016-November/082335.html)
Hi Federico,You guys seem to have successfully built a usable coreboot image for x220. Could you share its revision and config file? I have never built such usable image till now.Besides, is script file me_cleaner.py downloadable somewhere?Persmule.
Hi,
you can find all the info about my build of coreboot in the board_status repository (here [1] you can find the latest report). I suggest you to change the payload from GRUB (external ELF) to SeaBIOS.
I've uploaded the me_cleaner.py in a GitHub repo [2].
Nicola
[1] https://review.coreboot.org/cgit/board-status.git/tree/lenovo/x220/ 4.5-260-g3d302b0/2016-11-14T00_09_12Z [2] https://github.com/corna/me_cleaner
Hi Federico,
Today I succeeded in building a usable coreboot image for my x220. With coreboot, the integrated NIC is still misfunctional after powering up, with ME cleansed, and gets fully functional after a reboot.
Besides, how small could the ME section be after ME gets cleansed? It seems that the ME in board-status is 3MiB smaller than mine.
Persmule
在 2016年11月14日 21:06, Nicola Corna 写道:
Hi Federico,You guys seem to have successfully built a usable coreboot image for x220. Could you share its revision and config file? I have never built such usable image till now.Besides, is script file me_cleaner.py downloadable somewhere?Persmule.
Hi,
you can find all the info about my build of coreboot in the board_status repository (here [1] you can find the latest report). I suggest you to change the payload from GRUB (external ELF) to SeaBIOS.
I've uploaded the me_cleaner.py in a GitHub repo [2].
Nicola
[1] https://review.coreboot.org/cgit/board-status.git/tree/lenovo/x220/ 4.5-260-g3d302b0/2016-11-14T00_09_12Z [2] https://github.com/corna/me_cleaner
Hi Persmule,
for the ME in the board status repo I used a ME image extracted from a Chromebook C710 BIOS dump, you can find the same dump somewhere in the web. Once we repair our spare X220 we can resume the work on ME and, hopefully, reduce further the firmware size.
Nicola
On Fri, Nov 04, 2016 at 09:20:24PM +0000, Nicola Corna wrote:
[...]
- Sandy Bridge accepts an Intel ME firmware with just the FTPR partition, both with and without a valid FPT (the partition table of the Intel ME image). The system doesn't power off after 30 minutes, and the ME reports a successful initialization (see attached cbmem).
I'm really glad to hear that you've been able to replicate my results and have similar results on slightly different hardware. Since my original post I've also replicated the results on a Skylake mobile CPU, which has a very different ME architecture, although with a similar set of partitions.
- Sandy Bridge boots successfully without a good FPT (but with a valid FTPR), so it probably has a "backup" on-chip FPT. The system boots fine and doesn't power itself off after 30 minutes, even if the cbmem entry "ME: FW Partition Table" has the value "BAD" (that, we suppose, means: "I found a broken partition table, I'm using the fallback one").
My guess is that it either has a fallback, a hard coded address or scans looking for the $FTPR string.
- According to Trammell Hudson's tests, this behaviour is the same in Ivy Bridge (the thread's subject says "Sandy Bridge" but his cbmem confirms that his device is actually an Ivy Bridge)
Right. I was confused since the coreboot architecture directory for the x230 is named sandybridge.
[...]
- We couldn't remove in any way the FTPR partition on Sandy Bridge: [...]
- Relocating the FTPR image doesn't work, even if the FPT entry points to the new location.
Likewise. I also tried relocating it, but was not able to do so.
- Shrinking the "reduced" ME partition in IFD to the minimum size allowed (FTPR offset + FTPR size + 4 kB alignment) bricks the device too (but we're not sure about this, maybe we made a mistake during the image creation).
I was able to reduce it to cover 0x3000 to 1 MB. There appears to be some data being written into part of the SPI flash by the ME on each boot, logging someting about the startup?, so the IFD must include that portion. I did change the IFD to allow the CPU to read the ME region so that I can include it in my TPM measurements.
- We haven't understood yet how to remove the unneeded modules from the FTPR partition, any help is appreciated.
Any of the LZMA compressed files in the FTPR partitoin can be replaced with 0xFF. The Huffman ones seem to share a common table, so I wasn't able to replace any of them individually.
Again, I'm really glad to see that this has started additional research into this area.
To Whom It May Concern!
I sincerely hope that Google designers, reading this thread, and are involved in the R&D of multi-core silicon ARM server, with Coreboot designers, will NOT replicate/copy such an opaque design as INTEL did (and why, really???), and they'll get rid of ME entity overall/completely, making simpler lookalike (hopefully Coreboot based) BIOS containing all the components from INTEL BIOS + INTEL ME functionalities/features in one (in other words migrate all ME functionalities in Google "BIOS", whatever this will be).
The benefits of this design are obvious, aren't they? ;-)
Thank you, Zoran
On Sat, Nov 5, 2016 at 3:33 PM, Trammell Hudson hudson@trmm.net wrote:
On Fri, Nov 04, 2016 at 09:20:24PM +0000, Nicola Corna wrote:
[...]
- Sandy Bridge accepts an Intel ME firmware with just the FTPR
partition, both
with and without a valid FPT (the partition table of the Intel MEimage).
The system doesn't power off after 30 minutes, and the ME reports a successful initialization (see attached cbmem).I'm really glad to hear that you've been able to replicate my results and have similar results on slightly different hardware. Since my original post I've also replicated the results on a Skylake mobile CPU, which has a very different ME architecture, although with a similar set of partitions.
- Sandy Bridge boots successfully without a good FPT (but with a valid
FTPR),
so it probably has a "backup" on-chip FPT. The system boots fine anddoesn't
power itself off after 30 minutes, even if the cbmem entry "ME: FW Partition Table" has the value "BAD" (that, we suppose, means: "Ifound a
broken partition table, I'm using the fallback one").My guess is that it either has a fallback, a hard coded address or scans looking for the $FTPR string.
- According to Trammell Hudson's tests, this behaviour is the same in
Ivy
Bridge (the thread's subject says "Sandy Bridge" but his cbmemconfirms that
his device is actually an Ivy Bridge)Right. I was confused since the coreboot architecture directory for the x230 is named sandybridge.
[...]
- We couldn't remove in any way the FTPR partition on Sandy Bridge:
[...]
- Relocating the FTPR image doesn't work, even if the FPT entry points
to the
new location.Likewise. I also tried relocating it, but was not able to do so.
- Shrinking the "reduced" ME partition in IFD to the minimum size
allowed (FTPR
offset + FTPR size + 4 kB alignment) bricks the device too (butwe're not
sure about this, maybe we made a mistake during the image creation).I was able to reduce it to cover 0x3000 to 1 MB. There appears to be some data being written into part of the SPI flash by the ME on each boot, logging someting about the startup?, so the IFD must include that portion. I did change the IFD to allow the CPU to read the ME region so that I can include it in my TPM measurements.
- We haven't understood yet how to remove the unneeded modules from the
FTPR
partition, any help is appreciated.Any of the LZMA compressed files in the FTPR partitoin can be replaced with 0xFF. The Huffman ones seem to share a common table, so I wasn't able to replace any of them individually.
Again, I'm really glad to see that this has started additional research into this area.
-- Trammell
-- coreboot mailing list: coreboot@coreboot.org https://www.coreboot.org/mailman/listinfo/coreboot
-
Federico Amedeo Izzo -
Igor Skochinsky -
Nicola Corna -
Nicola Corna -
Persmule -
persmule
-
Trammell Hudson -
Zoran Stojsavljevic