GSOC 2011

[Stefan Tauner]

hello

since no one on irc even replied with "go away intel nazi!!11", i have
to propose my idea here ;)

i tried to get flashrom to work on my two main computers[1], but they
don't work because of the EC/Intel's management engine (ME).
i don't know for sure how the vendor tools do the flashing, but i
presume that they talk to the ME via HECI to coordinate the process.
There _was_ an attempt to get basic HECI and QST (intel quiet system
technology) into the kernel to allow temperature readings etc. Intel
abandoned it in 2009 [2].
my (project) idea is to try to reverse engineer some vendor tools to
see how they work and try to reproduce it in flashrom to eventually
support ME enabled motherboards in flashrom. i dunno how big the impact
would be, but i guess this would allow even many laptops to be
supported?
what do you think about this idea and its feasibility? any volunteers
for mentoring? :)

1: http://patchwork.coreboot.org/patch/2798/
2:
http://driverdev.linuxdriverproject.org/pipermail/devel/2009-September/00245...

also i have some questions about the "official" ideas:
UI:
 - With TUI do you mean (something like) ncurses?
 - the "current" code is [3]? i have not looked into it yet.
   - has it to use qt and/or that code (i like java *hide* and have a
     bit of gdk experience)?

recovery stuff:
guess it would be mandatory to have a working coreboot setup with real
hardware for that?

bitbanging support: this would also require available hardware. would i
have to buy that myself? can it be borrowed from someone?
these projects would probably profit from my open workbench logic
analyzer...

the infrastructure things sound a bit boring, but from what i saw in
the source some refactoring would (also) improve the codebase
somewhat. ;) so the infrastructure-related projects may be even the most
valuable contribution for longterm goals. ok that was not really a
question, sorry.

if there are other ideas i would love to hear them (soon).

thanks!

3: http://gitorious.org/flashrom/flashrom-gui/commits/master

[Joshua Roys]

On 03/19/2011 01:00 PM, Stefan Tauner wrote:
> hello
>
> since no one on irc even replied with "go away intel nazi!!11", i have
> to propose my idea here ;)
>
> i tried to get flashrom to work on my two main computers[1], but they
> don't work because of the EC/Intel's management engine (ME).
> i don't know for sure how the vendor tools do the flashing, but i
> presume that they talk to the ME via HECI to coordinate the process.
> There _was_ an attempt to get basic HECI and QST (intel quiet system
> technology) into the kernel to allow temperature readings etc. Intel
> abandoned it in 2009 [2].
> my (project) idea is to try to reverse engineer some vendor tools to
> see how they work and try to reproduce it in flashrom to eventually
> support ME enabled motherboards in flashrom. i dunno how big the impact
> would be, but i guess this would allow even many laptops to be
> supported?
> what do you think about this idea and its feasibility? any volunteers
> for mentoring? :)

Hello,

My 2 cents are that this would be very useful...  There are a few ICH10 
systems at work that are fairly locked down through the chipset 
restrictions.  Other than flashing externally or attempting to 
circumvent the chipset, your plan is the only other option.  I don't 
think Intel will remove restrictions in the future (although they may 
convince BIOS folk to use less restrictive defaults) so it would be a 
good idea, I think, to learn to play nice with them.

Josh

[Stefan Tauner]

On Mon, 21 Mar 2011 12:41:11 -0400
Joshua Roys  wrote:
> My 2 cents are that this would be very useful...  There are a few
> ICH10 systems at work that are fairly locked down through the chipset 
> restrictions.  Other than flashing externally or attempting to 
> circumvent the chipset, your plan is the only other option.  I don't 
> think Intel will remove restrictions in the future (although they may 
> convince BIOS folk to use less restrictive defaults) so it would be a 
> good idea, I think, to learn to play nice with them.

hi

i talked to peter, joshua and marc on irc yesterday about it.
the condensed agreement is, that we don't really know what this would
get us exactly and we should know that before we really decide.

idwer at least said "I think choosing to work on Intel's ME as a
gsoc project will be more rewarding than focusing on ECs, which vary".

so i am trying to research this further in respect to impact on
different platforms and if the heci thing is really the one responsible
for this.

interesting bits so far:
there has been efforts to push a new heci driver upstream this year [1].

intel signs their firmware images with a PKI. the signature/public key
is checked on every boot with a sha-1 hash in rom [2]. not important
for my task but interesting nonetheless imho.

there seem to be two main architectures for accessing system flash
chips (in notebooks) these days. i'll give an example for each:
 - 1. system flash behind an external EC e.g. thinkpad SL410 [3],
 - 2. system flash directly connected to the southbridge, ec has its
   own flash for its firmware (either embedded or another external
   flash chip) thinkpad t400s (predecessor of my t410s) [4].

the first case is the known and feared case were the EC can
interfere in various ways with flashrom's interactions with the
flash. i'm pretty sure that we can make flashrom support the other
case with heci, but this would most probably only allow to flash the
bios flash not the EC firmware. updating the bios only is probably a
bad idea but this has to be solved later.

for non-mobile boards there seem to be a variety of ME configurations in
respect to flash settings in the wild:

flashrom does not work:
 - Zotac H55-itx (H55) [5]:
    Flash Configuration Lock-Down: disabled
    Flash descriptors: valid
    descriptors r/o, ME and platform locked
 - Intel DG45ID (my desktop) (ICH10R) [11]:
    Flash Configuration Lock-Down: enabled
    Flash descriptors: valid
    descriptors r/o, ME locked

flashrom works:
 - Supermicro C2SEA (ICH10R) [6]:
   Asus P5E-VM (ICH9R) [7]:
   Gigabyte EP45-DS4 (ICH10R) [8]:
   Z8NA-D6(C) (ICH10R) [9]:
    Flash Configuration Lock-Down: disabled
    Flash descriptors: disabled/not valid
 - EVGA X58 SLI (ICH10R) [10]:
    Flash Configuration Lock-Down: disabled
    Flash descriptors: valid
    all descriptors r/w

so the majority of desktop boards seem to not use the flash descriptors
at all or don't lock them and therefore work with flashrom already.
beside my intel board only that zotac boards locks flash regions.
i doubt that these are the only ones in the wild, but that's all i
could dig up yet.

i'd like to get a comment from carl-daniel, before i try dissecting
intel's flash program for the dg45id (for which i could use some help
btw).

1: http://linux.derkeiler.com/Mailing-Lists/Kernel/2011-02/msg04015.html
2:
http://software.intel.com/en-us/articles/architecture-guide-intel-active-man...
3: http://notebookschematic.com/wp-content/uploads/2010/12/SL410K.png
4: http://notebookschematic.com/wp-content/uploads/2010/11/T400S_2.png
5: http://www.flashrom.org/pipermail/flashrom/2010-December/005578.html
6: http://www.flashrom.org/pipermail/flashrom/2010-November/005428.html
7: http://www.flashrom.org/pipermail/flashrom/2011-January/005705.html
8: http://paste.flashrom.org/view.php?id=429
9: http://www.flashrom.org/pipermail/flashrom/2010-September/004671.html
10: http://www.flashrom.org/pipermail/flashrom/2010-November/005409.html
11: http://www.flashrom.org/pipermail/flashrom/2011-March/006012.html

[SPAM?] Re: GSOC 2011

[Stefan Tauner]

On Tue, 22 Mar 2011 06:46:46 +0100
Stefan Tauner  wrote:
> there has been efforts to push a new heci driver upstream this year
> [1].

just for the record.
there was a second try:
http://lists-archives.org/linux-kernel/thrd6.html#27456456

it _may_ get into 2.6.40 (maybe its staging tree)

[Stefan Tauner]

On Fri, 8 Apr 2011 01:36:14 +0200
Stefan Tauner  wrote:
> On Tue, 22 Mar 2011 06:46:46 +0100
> Stefan Tauner  wrote:
> > there has been efforts to push a new heci driver upstream this year
> > [1].
>
> just for the record.
> there was a second try:
> http://lists-archives.org/linux-kernel/thrd6.html#27456456
>
> it _may_ get into 2.6.40 (maybe its staging tree)

that's probably what will happen.
greg kh has committed mei to his staging-next tree.
http://git.kernel.org/?p=linux/kernel/git/gregkh/staging-2.6.git;a=history;f...

https://lkml.org/lkml/2011/5/15/44
http://thread.gmane.org/gmane.linux.kernel/1140197

thanks to xvilka for mentioning this!

[Carl-Daniel Hailfinger]

Am 19.03.2011 18:00 schrieb Stefan Tauner:
> i tried to get flashrom to work on my two main computers[1], but they
> don't work because of the EC/Intel's management engine (ME).
> i don't know for sure how the vendor tools do the flashing, but i
> presume that they talk to the ME via HECI to coordinate the process.

Locked region reflashing is considerably harder than handling the rest 
of the flash. Most Intel chipsets released after ICH7 (except later ICH7 
derivatives) support two flash interfaces: One interface which lets 
flashrom do all the heavy lifting (which we use right now), and one 
interface outsources read/write/erase commands to the chipset and just 
tells the chipset in abtract terms what to do. Most locked down boards 
still allow the outsourced interface to read some regions, but that 
interface is not supported in flashrom right now. One of the interfaces 
is called hardware sequencing, the other one software sequencing, but I 
can't remember which one is which. If we can add a driver for the 
currently unsupported interface (which is fully documented), we'd 
already achieve a major step forward.
> my (project) idea is to try to reverse engineer some vendor tools to
> see how they work and try to reproduce it in flashrom to eventually
> support ME enabled motherboards in flashrom. i dunno how big the impact
> would be, but i guess this would allow even many laptops to be
> supported?
> what do you think about this idea and its feasibility?

Having the ME reflash some stuff would definitely be interesting, but I 
have no idea if there is even a consistent interface for that, and the 
problem space is similar to having an EC reflash some stuff. It can be 
done, but each machine would have to be reverse engineered individually. 
Not fun. If you attempt any of this during GSoC, please make sure you 
first support the interface I mentioned above, and if any time is left, 
reverse that for reversing any ME-controlled flashing.
> any volunteers for mentoring? :)

I could serve as mentor.
> also i have some questions about the "official" ideas:
> UI:
> With TUI do you mean (something like) ncurses?
> the "current" code is [3]? i have not looked into it yet.
> has it to use qt and/or that code (i like java *hide* and have a
> bit of gdk experience)?

To be honest, a GUI would be nice, but right now the bigger problem is 
to get the command line interface and flashrom internals into shape 
before it can be useful to the general population (especially the locked 
down chipset issue needs to be addressed).
> recovery stuff:
> guess it would be mandatory to have a working coreboot setup with real
> hardware for that?

Strongly suggested, yes.
> bitbanging support: this would also require available hardware. would i
> have to buy that myself? can it be borrowed from someone?
> these projects would probably profit from my open workbench logic
> analyzer...

We already support bitbanging for SPI, and some bitbanging code for 
LPC/FWH is available, someone just has to clean it up and get it merged. 
Michael Karcher recently posted bitbanging code for Parallel, so most 
interfaces are covered.
> the infrastructure things sound a bit boring, but from what i saw in
> the source some refactoring would (also) improve the codebase
> somewhat. ;) so the infrastructure-related projects may be even the most
> valuable contribution for longterm goals. ok that was not really a
> question, sorry.

Heh, yes. Our big bottleneck right now is reviewer time. There are quite 
a few refactoring patches waiting for review, and there is more in the 
queue.
> if there are other ideas i would love to hear them (soon).

I'll try to update our GSoC page with ideas.

Regards,
Carl-Daniel

[Stefan Tauner]

On Tue, 22 Mar 2011 08:36:50 +0100
Carl-Daniel Hailfinger  wrote:
> Locked region reflashing is considerably harder than handling the
> rest of the flash. Most Intel chipsets released after ICH7 (except
> later ICH7 derivatives) support two flash interfaces: One interface
> which lets flashrom do all the heavy lifting (which we use right
> now), and one interface outsources read/write/erase commands to the
> chipset and just tells the chipset in abtract terms what to do. Most
> locked down boards still allow the outsourced interface to read some
> regions, but that interface is not supported in flashrom right now.
> One of the interfaces is called hardware sequencing, the other one
> software sequencing, but I can't remember which one is which. If we
> can add a driver for the currently unsupported interface (which is
> fully documented), we'd already achieve a major step forward.

flashrom supports software sequencing, hardware sequencing is the
interface were the chipset handles "everything". from what i have read
in the ICH datasheets (while i tried to understand sw sequencing/what
flashrom does) hardware sequencing won't get us anywhere. it seems it's
just a simpler interface for compatible/supported flash chips. the main
difference is that in software sequencing the software can define the
spi commands to be used. my impression can be wrong of course.

adding support for hardware sequencing shouldn't be that hard, because
the software has less options than with sw sequencing. but... i don't
know what we would get. you said that the "outsourced" interface (=hw
sequencing) would allow to read "some regions" on locked boards. sw
sequencing allows that too: flashrom can read the first region (=flash
descriptors) of my boards. it just stops when it reaches the first
locked region (and does not write the date to file it has read until
then).
> Having the ME reflash some stuff would definitely be interesting, but
> I have no idea if there is even a consistent interface for that, and
> the problem space is similar to having an EC reflash some stuff. It
> can be done, but each machine would have to be reverse engineered
> individually. Not fun. If you attempt any of this during GSoC, please
> make sure you first support the interface I mentioned above, and if
> any time is left, reverse that for reversing any ME-controlled
> flashing.

the question is if each machine is really different in that respect. i
doubt that because that would mean that every vendor has to provide its
own firmware for the ME microcontroller that is included in the
MCH*/ICH/SCH.
* including it in the mch makes sense because it is cheaper there due
  to different manufacturing processes and it is mentioned here
  explicitly:
http://software.intel.com/en-us/articles/architecture-guide-intel-active-man...

so i'm pretty sure if we can RE the heci flash protocol (if it
really exists :) once, we would be able to support all locked down
mainboards.