Hi Stefan,
I have waited very long to comment on this patch, but I simply don't have a good answer for it. I do think that OTP/security memory is outside the scope of flashrom because it behaves totally different (and testing could easily kill a dozen chips during inital implementation, and a few chips each time someone wants to retest). Now if something is outside the scope of flashrom, should flashrom care at all? Some flash chips have GPI pins (GPIO without output) and those are readable with commands very similar to those used by flashrom. Should we tell the user about that feature as well? We had such requests in the past, and I was not really happy about them.
Am 14.08.2011 05:36 schrieb Stefan Tauner:
Some flash chips contain OTP memory that we cannot read or write (yet). This prohibits us from cloning them, hence warn the user if we detect it. Not all variations of the tagged chips contain OTP memory. They are often only enabled on request or have there own ordering numbers. There is usually no way to distinguish them afaict.
Well, it's worse than that. I've seen datasheets for flash chips with the following features: - OTP memory, fully readable, fully writable once in one block (no partial writes) - OTP memory, fully readable, fully writable once per byte (partial writes work just fine) - OTP memory, fully readable, fully writable once per 2^n-byte block - OTP memory, fully readable, only half of it writable, the other half is programmed at the factory - OTP memory, fully readable, completely pre-written at the factory (so it's rather ROM than OTP) - Hidden OTP memory which can only be read if you know the correct security ID which is part of the OTP - (The complete flash chip behaves like OTP (well, like ROM) once a specific write-once status register is set)... not relevant here
And then you have the problem that multiple chip generations often share the same device ID, so probing can't differentiate between a chip with OTP and one without unless you're extremely lucky. Do we want FEATURE_OTP and FEATURE_MAYBE_OTP?
Do we warn if a chip has a readonly serial number? That means the chip can't be cloned. People who care about OTP for clonability reasons probably care about other readonly contents as well. OTOH, other people who don't use the OTP at all (for them, OTP is just an accidental feature of a cheap flash chip) don't want to be bothered by yet another line of output from flashrom which has no relevancy for them.
The manpage is extended to describe the backgrounds a bit. While i touched that section, i also reformatted it a bit and removed the reference to the trac bug reporting facility.
This patch is based on the idea and code of Daniel Lenski.
the reason i removed the trac reference is that it is not used, i have no (working) login and it does not really fits our work flow anyway. the only user that used it in my active flashrom time ranted about it. would anyone miss it?
To be honest, trac is something I never got along with. Even if I'm logged in there is no way for me to read the email address of the person who opened the bug, so communication is pretty difficult. I'm all for checking all old trac tickets, closing what can be closed, and moving all other tickets in some way or another to the mailing list. However, we should mention that it's OK to report bugs via IRC as well as long as the reporter mentions a way to contact hi/her (i.e. email address).
Removing the trac reference and adding the IRC reference should be a separate patch, though, which is Acked-by: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net
One comment about the man page formatting: This part of the patch may conflict with the other man page formatting patch you sent. By the way, do you know any good reference about man page formatting? I had trouble finding out what .RE and .RS do.
the new default verbosity output is: This chip may contain one-time programmable memory (see man page for details).
the new dbg1/-V output is: This chip may contain one-time programmable memory. flashrom cannot read, and may never be able to write it. flashrom may not be able to completely clone the contents of this chip (see man page for details).
there are certainly more chips with OTP memories out there. additions are welcome! i have only tagged chips with user modifiably OTP/security memories. there are some chips out there which have factory written IDs, but no OTP memory. a similar warning could/should be printed for those, but i did not want to abuse the OTP tag for this. also, there is nothing we can do, but read those IDs. duplicating chips using them is not possible, so a warning in the man page might suffice?
since OTP memory (or any other data outside the linear address range of the "main" memory) can not be handled very well in flashrom right now, it is not clear how OTP memory access could be developed further. of course one could start to implement the basic methods needed to read them, but how they should be integrated then is not clear. one could start by adding an otp_print field to the struct flashchip and just dump the bytes on probing similar to the lock printing (not on default verbosity and possibly only if there are bytes different from 0x00 and 0xff).
carl-daniel seems to be quite sceptical regarding handling OTP at all in flashrom. he stated "it does not really fit our device model at all", which is right, but could be changed. he also asked "should we really handle all features present in flash chips?" and the answer is probably "no", although i don't see a reason why we should not, if the effort is not too much and the architectural changes needed aren't obvious NOGOs. but OTP regions are somewhat special, they are not just *some* feature.
Indeed.
one argument is the one stated in the message printed by this patch. flashrom is not able to clone chips that have some kind of OTP memory written right now. imo "cloning" a flash chip seems to be a valid use case for a tool like flashrom as long as it is feasible (which is not for chips with unique, preconfigured IDs).
the other more theoretical argument i have is: OTP memory is just some memory in the flash chip. it may need other access patterns, but it is not much different from other write protected memories apart from that. some chips implement it in a way that it is even possible to erase the OTP regions. those regions are just normal flash and are made unwriteable by fuses in a register or another addressable byte.
Signed-off-by: Daniel Lenski dlenski@gmail.com Signed-off-by: Stefan Tauner stefan.tauner@student.tuwien.ac.at
Stefan: I don't want to veto this patch, and although I think that OTP handling is not really a flashrom feature, I think that this implementation satisfies the quality criteria for merging, so the patch is Acked-by: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net
Regards, Carl-Daniel
On Wed, 15 Feb 2012 02:55:38 +0100 Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Hi Stefan,
I have waited very long to comment on this patch, but I simply don't have a good answer for it. I do think that OTP/security memory is outside the scope of flashrom because it behaves totally different (and testing could easily kill a dozen chips during inital implementation, and a few chips each time someone wants to retest).
flashrom "bricks" complete mainboards now and then because the user did not prepare well, did not read documentation or obeyed ridiculously obvious warnings. putting this into relation with a developer (or advanced user) that tests flashrom's OTP capabilities reveals what i think about this argument. ;) Of course the comparison is not completely fair because unerasable OTP bits are gone forever and mainboards can be repaired… OTOH a new chip costs not much in relation to fix a mainboard for must of our users.
Now if something is outside the scope of flashrom, should flashrom care at all?
depends... i dont think of "the scope" as a clearly bounded area. everything related to flash chips is somewhat in its scope (else we would not talk about this), heck we even discussed EEPROM handling multiple times... we dont need to support any and all feature found in any flash chip out there, but we should integrate knowledge and code for the more common ones that might be useful, if there is someone willing to provide them (and maintain them if necessary).
Some flash chips have GPI pins (GPIO without output) and those are readable with commands very similar to those used by flashrom. Should we tell the user about that feature as well? We had such requests in the past, and I was not really happy about them.
this is not about GPIO or other non-memory features (one can stretch the words "read and write" to accessing IO streams, but let's pretend i did not write that ;).
Am 14.08.2011 05:36 schrieb Stefan Tauner:
Some flash chips contain OTP memory that we cannot read or write (yet). This prohibits us from cloning them, hence warn the user if we detect it. Not all variations of the tagged chips contain OTP memory. They are often only enabled on request or have there own ordering numbers. There is usually no way to distinguish them afaict.
Well, it's worse than that. I've seen datasheets for flash chips with the following features:
- OTP memory, fully readable, fully writable once in one block (no
partial writes)
- OTP memory, fully readable, fully writable once per byte (partial
writes work just fine)
- OTP memory, fully readable, fully writable once per 2^n-byte block
- OTP memory, fully readable, only half of it writable, the other half
is programmed at the factory
- OTP memory, fully readable, completely pre-written at the factory (so
it's rather ROM than OTP)
- Hidden OTP memory which can only be read if you know the correct
security ID which is part of the OTP
- (The complete flash chip behaves like OTP (well, like ROM) once a
specific write-once status register is set)... not relevant here
And then you have the problem that multiple chip generations often share the same device ID, so probing can't differentiate between a chip with OTP and one without unless you're extremely lucky. Do we want FEATURE_OTP and FEATURE_MAYBE_OTP?
FEATURE_MAYBE_NOT_CLONEABLE half serious... at least this is the semantics i would like to tag and convey to the user for now.
Do we warn if a chip has a readonly serial number? That means the chip can't be cloned. People who care about OTP for clonability reasons probably care about other readonly contents as well. OTOH, other people who don't use the OTP at all (for them, OTP is just an accidental feature of a cheap flash chip) don't want to be bothered by yet another line of output from flashrom which has no relevancy for them.
do you agree to lowering the verbosity of the whole message to dbg level?
The manpage is extended to describe the backgrounds a bit. While i touched that section, i also reformatted it a bit and removed the reference to the trac bug reporting facility.
This patch is based on the idea and code of Daniel Lenski.
the reason i removed the trac reference is that it is not used, i have no (working) login and it does not really fits our work flow anyway. the only user that used it in my active flashrom time ranted about it. would anyone miss it?
To be honest, trac is something I never got along with. Even if I'm logged in there is no way for me to read the email address of the person who opened the bug, so communication is pretty difficult. I'm all for checking all old trac tickets, closing what can be closed, and moving all other tickets in some way or another to the mailing list. However, we should mention that it's OK to report bugs via IRC as well as long as the reporter mentions a way to contact hi/her (i.e. email address).
Removing the trac reference and adding the IRC reference should be a separate patch, though, which is Acked-by: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net
ok, this will go into the recently posted manpage improvement patch (planned to do that anyway because i did not think that you would look at this one now :)
One comment about the man page formatting: This part of the patch may conflict with the other man page formatting patch you sent. By the way, do you know any good reference about man page formatting? I had trouble finding out what .RE and .RS do.
i have to look that stuff up every time myself, sorry. yesterday i have used http://gnustep.made-it.com/man-groff.html but it looks like that would not answer your question...
the new default verbosity output is: This chip may contain one-time programmable memory (see man page for details).
the new dbg1/-V output is: This chip may contain one-time programmable memory. flashrom cannot read, and may never be able to write it. flashrom may not be able to completely clone the contents of this chip (see man page for details).
there are certainly more chips with OTP memories out there. additions are welcome! i have only tagged chips with user modifiably OTP/security memories. there are some chips out there which have factory written IDs, but no OTP memory. a similar warning could/should be printed for those, but i did not want to abuse the OTP tag for this. also, there is nothing we can do, but read those IDs. duplicating chips using them is not possible, so a warning in the man page might suffice?
since OTP memory (or any other data outside the linear address range of the "main" memory) can not be handled very well in flashrom right now, it is not clear how OTP memory access could be developed further. of course one could start to implement the basic methods needed to read them, but how they should be integrated then is not clear. one could start by adding an otp_print field to the struct flashchip and just dump the bytes on probing similar to the lock printing (not on default verbosity and possibly only if there are bytes different from 0x00 and 0xff).
carl-daniel seems to be quite sceptical regarding handling OTP at all in flashrom. he stated "it does not really fit our device model at all", which is right, but could be changed. he also asked "should we really handle all features present in flash chips?" and the answer is probably "no", although i don't see a reason why we should not, if the effort is not too much and the architectural changes needed aren't obvious NOGOs. but OTP regions are somewhat special, they are not just *some* feature.
Indeed.
one argument is the one stated in the message printed by this patch. flashrom is not able to clone chips that have some kind of OTP memory written right now. imo "cloning" a flash chip seems to be a valid use case for a tool like flashrom as long as it is feasible (which is not for chips with unique, preconfigured IDs).
the other more theoretical argument i have is: OTP memory is just some memory in the flash chip. it may need other access patterns, but it is not much different from other write protected memories apart from that. some chips implement it in a way that it is even possible to erase the OTP regions. those regions are just normal flash and are made unwriteable by fuses in a register or another addressable byte.
Signed-off-by: Daniel Lenski dlenski@gmail.com Signed-off-by: Stefan Tauner stefan.tauner@student.tuwien.ac.at
Stefan: I don't want to veto this patch, and although I think that OTP handling is not really a flashrom feature, I think that this implementation satisfies the quality criteria for merging, so the patch is Acked-by: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net
thanks! ill wait for your response regarding message verbosity while merging the (independent) manpage change into my other patch.
Am 15.02.2012 14:11 schrieb Stefan Tauner:
On Wed, 15 Feb 2012 02:55:38 +0100 Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Now if something is outside the scope of flashrom, should flashrom care at all?
depends... i dont think of "the scope" as a clearly bounded area. everything related to flash chips is somewhat in its scope (else we would not talk about this), heck we even discussed EEPROM handling multiple times... we dont need to support any and all feature found in any flash chip out there, but we should integrate knowledge and code for the more common ones that might be useful, if there is someone willing to provide them (and maintain them if necessary).
If we target non-flash EEPROMs, we might as well support OTP. I'd say such support is post-1.0 material, though (and no, I don't plan to delay flashrom 1.0 like Wine 1.0 was delayed).
And then you have the problem that multiple chip generations often share the same device ID, so probing can't differentiate between a chip with OTP and one without unless you're extremely lucky. Do we want FEATURE_OTP and FEATURE_MAYBE_OTP?
FEATURE_MAYBE_NOT_CLONEABLE half serious... at least this is the semantics i would like to tag and convey to the user for now.
Heh. I think FEATURE_OTP is OK for now, and postpone a split between _OTP and _MAYBE_OTP.
Do we warn if a chip has a readonly serial number? That means the chip can't be cloned. People who care about OTP for clonability reasons probably care about other readonly contents as well. OTOH, other people who don't use the OTP at all (for them, OTP is just an accidental feature of a cheap flash chip) don't want to be bothered by yet another line of output from flashrom which has no relevancy for them.
do you agree to lowering the verbosity of the whole message to dbg level?
Yes.
Removing the trac reference and adding the IRC reference should be a separate patch, though, which is Acked-by: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net
ok, this will go into the recently posted manpage improvement patch (planned to do that anyway because i did not think that you would look at this one now :)
Thanks!
do you know any good reference about man page formatting? I had trouble finding out what .RE and .RS do.
i have to look that stuff up every time myself, sorry. yesterday i have used http://gnustep.made-it.com/man-groff.html but it looks like that would not answer your question...
Thanks, it's a good start anyway.
the other more theoretical argument i have is: OTP memory is just some memory in the flash chip. it may need other access patterns, but it is not much different from other write protected memories apart from that. some chips implement it in a way that it is even possible to erase the OTP regions. those regions are just normal flash and are made unwriteable by fuses in a register or another addressable byte.
Signed-off-by: Daniel Lenski dlenski@gmail.com Signed-off-by: Stefan Tauner stefan.tauner@student.tuwien.ac.at
Stefan: I don't want to veto this patch, and although I think that OTP handling is not really a flashrom feature, I think that this implementation satisfies the quality criteria for merging, so the patch is Acked-by: Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net
thanks! ill wait for your response regarding message verbosity while merging the (independent) manpage change into my other patch.
Go ahead.
Regards, Carl-Daniel
On Thu, 16 Feb 2012 00:03:00 +0100 Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Am 15.02.2012 14:11 schrieb Stefan Tauner:
On Wed, 15 Feb 2012 02:55:38 +0100 Carl-Daniel Hailfinger c-d.hailfinger.devel.2006@gmx.net wrote:
Now if something is outside the scope of flashrom, should flashrom care at all?
depends... i dont think of "the scope" as a clearly bounded area. everything related to flash chips is somewhat in its scope (else we would not talk about this), heck we even discussed EEPROM handling multiple times... we dont need to support any and all feature found in any flash chip out there, but we should integrate knowledge and code for the more common ones that might be useful, if there is someone willing to provide them (and maintain them if necessary).
If we target non-flash EEPROMs, we might as well support OTP. I'd say such support is post-1.0 material, though (and no, I don't plan to delay flashrom 1.0 like Wine 1.0 was delayed).
no, you will tell us that you may tag 1.0 next week - every week :P
Do we warn if a chip has a readonly serial number? That means the chip can't be cloned. People who care about OTP for clonability reasons probably care about other readonly contents as well. OTOH, other people who don't use the OTP at all (for them, OTP is just an accidental feature of a cheap flash chip) don't want to be bothered by yet another line of output from flashrom which has no relevancy for them.
do you agree to lowering the verbosity of the whole message to dbg level?
Yes.
done and committed in r1493, one less yay!
-
Carl-Daniel Hailfinger -
Stefan Tauner