Dear flashrom contributors,
you might have noticed that there hasn't been a release of flashrom in the past 1.5 years. That doesn't mean that there wasn't any progress, though. Development has continued and some patches were merged into a common branch [1]. Alas, not in the `stable` branch of the official repository. I'd rather not discuss now, why (it's complicated).
As the original plan was to have two separate branches `stable` and `staging` anyway [2] and we finally got the first commit for `stable` together (which was stalling the branch), I thought it would be a good idea to move the already baked and tested commits from `staging` to `stable`.
Sadly, even this got stalled again. Plus, there is doubt if the two branches model is feasible at all:
* How to maintain patch compatibility between them? * Should everything be first submitted to `staging`? * If not, who'd be allowed to decide to push directly to `stable` and why? * ...
So I propose the following: Forget the two branches model, start a `master` branch with either the current state of `staging` or my proposed move to `stable` [3] and release flashrom-1.0 right away.
Submission to the current `staging` branch took place within core- boot's Gerrit infrastructure. As for myself, I would like to con- tinue that for any flashrom branch. IMO, it works reasonably well to work on a branch together. Though, I know that not everybody is happy with Gerrit and am open for anything else that doesn't incur a bottleneck.
Thoughts?
Nico
[1] `staging` on https://review.coreboot.org/cgit/flashrom.git
[2] https://www.flashrom.org/Development_Guidelines#Branches
[3] The result if somebody submits the following patch queue:
https://review.coreboot.org/#/q/status:open+branch:stable+owner:%22Nico Huber"
Hello Nico,
2017-10-13 2:40 GMT+02:00 Nico Huber nico.h@gmx.de:
So I propose the following: Forget the two branches model, start a `master` branch with either the current state of `staging` or my proposed move to `stable` [3] and release flashrom-1.0 right away.
I believe staging/devel/etc branches in projects managed with git are usually a troublesome legacy from old CVS/SVN days where branching/tagging and patch management were expensive. They shouldn't have even copied into Git repositores at all. Also flashrom historically has a very flat and straightforward development model, so why making two branches at all?
From the package maintainer's point of view it would be easier and
simpler from if flashrom just have one single monotonically increasing branch (master?) with tags applied to some commits eventually (I prefer more frequent tagging though), rather than two branches where the purpose of one is to be blindly merged into another.
On 13.10.2017 02:40, Nico Huber wrote:
So I propose the following: Forget the two branches model, start a `master` branch with either the current state of `staging` or my proposed move to `stable` [3] and release flashrom-1.0 right away.
What I didn't realize last night: the `staging` branch contains valuable information in lots of fixup! commits that would be lost if we don't keep `staging`. They look ugly in the log but their messages still contain some reasoning about the changes. I prefer to use the current `staging` branch as `master` therefore.
The code state would be the same anyway. And I'm convinced that `staging` is in better shape than flashrom-0.9.9.
Nico
On Fri, Oct 13, 2017 at 11:42 PM, Nico Huber nico.h@gmx.de wrote:
On 13.10.2017 02:40, Nico Huber wrote:
So I propose the following: Forget the two branches model, start a `master` branch with either the current state of `staging` or my proposed move to `stable` [3] and release flashrom-1.0 right away.
What I didn't realize last night: the `staging` branch contains valuable information in lots of fixup! commits that would be lost if we don't keep `staging`. They look ugly in the log but their messages still contain some reasoning about the changes. I prefer to use the current `staging` branch as `master` therefore.
I'm fine with that.
I think Peter's suggestions are worth further consideration as well. As he pointed out, Flashrom has always been pretty simple and flat, and tags seem like a good way to do releases without the difficulties we've experienced with multiple branches.
Your idea to do release branches is fine, too. So long as we have something that is low-overhead, less bottlenecked, and won't cause as much friction moving forward.
And I'm convinced that `staging` is in better shape than flashrom-0.9.9.
This is somewhat subjective, but I agree largely due to the libflashrom and layout patches. And aside from code quality `staging` also has features and hardware support (either merged or pending) that I need to do work.
On Fri, 13 Oct 2017 17:42:11 +0200 Nico Huber nico.h@gmx.de wrote:
What I didn't realize last night: the `staging` branch contains valuable information in lots of fixup! commits that would be lost if we don't keep `staging`. They look ugly in the log but their messages still contain some reasoning about the changes. I prefer to use the current `staging` branch as `master` therefore.
*If* there is valuable information in them then they should get into the final commit message (i.e. the one that squashes everything to stable). I deem them basically as part of the review process of the whole change (i.e. the commit eventually committed to stable) and sure, there is information in it that might be interesting but this likewise applies to all review comments and I would not want them to be part of the commit messages, would you?
Some arguments why using staging for anything than hacking is not a good idea: - The information on a change is spread widely. For someone not involved in the respective review/development process (i.e. everybody but the author of the fixups and respective reviewer(s)) it is very hard to track what the actual outcome of all of the fixups is. Having a single commit that has a decent commit message solves this issue. - The log of a branch with lots of interwined fixups as a whole does not only look ugly, it does also no longer serve its purpose well: reading through it should give a good overview of what the code does. - The staging branch does currently and probably will continue to have some build problems, and due to the current rate of merged changes contain a significant number of undetected run-time bugs as well. Both can make bisecting for specific problems tedious if unlucky.
On 23.10.2017 09:39, Stefan Tauner wrote:
On Fri, 13 Oct 2017 17:42:11 +0200 Nico Huber nico.h@gmx.de wrote:
What I didn't realize last night: the `staging` branch contains valuable information in lots of fixup! commits that would be lost if we don't keep `staging`. They look ugly in the log but their messages still contain some reasoning about the changes. I prefer to use the current `staging` branch as `master` therefore.
*If* there is valuable information in them then they should get into the final commit message (i.e. the one that squashes everything to stable). I deem them basically as part of the review process of the whole change (i.e. the commit eventually committed to stable) and sure, there is information in it that might be interesting but this likewise applies to all review comments and I would not want them to be part of the commit messages, would you?
No, ofc not. But there is always a compromise of what to include into the commit message. Though, maybe I'd write too elaborate ones, I think yours are often too thin (it only doesn't look like it because you do too much in single commits, IMHO).
I have to admit, it's not a strong argument. But I thought, if I ask which branch to follow, I should vote for one too ;)
I actually don't care much about which branch we'll choose (current staging) or a history rewrite of it. BUT, this branch should be used for future development with appropriate write permissions for any major flashrom contributor (having few submitters just doesn't scale).
Some arguments why using staging for anything than hacking is not a good idea:
- The information on a change is spread widely.
In your notion of a change, this was and will always be true. Most commits aren't perfect and may result in future changes.
For someone not involved in the respective review/development process (i.e. everybody but the author of the fixups and respective reviewer(s)) it is very hard to track what the actual outcome of all of the fixups is.
In the git way of fixups (e.g. `fixup! <original commit>`), this is pretty easy: The outcome is what the original commit message promised.
Having a single commit that has a decent commit message solves this issue.
s/decent/perfect/?
I'd prefer decent commits and messages, too. But not if that means we reduce the rate to one commit per year. And I don't see why the tran- sition to git should be the starting point to only allow perfect com- mits. If I see something weird in flashrom code and look the commit up, it's often one that does multiple things at once and only lists what changed and not *why*. What we pushed to staging so far (including the weird looking fixups) is already more useful than commit messages that only rephrase the patch hunks.
- The log of a branch with lots of interwined fixups as a whole does not only look ugly, it does also no longer serve its purpose well: reading through it should give a good overview of what the code does.
You seem to imply perfect commits that only add new code. But in fact most commits change code. A decent commit message explains why the change was necessary and maybe why it's correct (if that is not ob- vious). For new features I agree, it should state "what the code does".
- The staging branch does currently and probably will continue to have some build problems, and due to the current rate of merged changes contain a significant number of undetected run-time bugs as well.
Feel free to compare regression rate before and after `staging`. I'm still convinced, we fixed more regressions than we have introduced so far.
Both can make bisecting for specific problems tedious if unlucky.
Lol, that's still better than squashing commits into bigger ones which makes bisecting less feasible.
Nico
On Wed, Oct 25, 2017 at 4:04 AM, Nico Huber nico.h@gmx.de wrote:
On 23.10.2017 09:39, Stefan Tauner wrote:
On Fri, 13 Oct 2017 17:42:11 +0200 Nico Huber nico.h@gmx.de wrote:
What I didn't realize last night: the `staging` branch contains valuable information in lots of fixup! commits that would be lost if we don't keep `staging`. They look ugly in the log but their messages still contain some reasoning about the changes. I prefer to use the current `staging` branch as `master` therefore.
*If* there is valuable information in them then they should get into the final commit message (i.e. the one that squashes everything to stable). I deem them basically as part of the review process of the whole change (i.e. the commit eventually committed to stable) and sure, there is information in it that might be interesting but this likewise applies to all review comments and I would not want them to be part of the commit messages, would you?
No, ofc not. But there is always a compromise of what to include into the commit message. Though, maybe I'd write too elaborate ones, I think yours are often too thin (it only doesn't look like it because you do too much in single commits, IMHO).
This is another area where Gerrit helps us. Each patch gets a "Reviewed-on:" line appended which makes it easy refer to review comments when additional context is needed. If we squash several patches into one we lose that context. We could have multiple "Reviewed-on:" lines in squashed patches, but that would be sort of awkward.
For someone not involved in the respective review/development process (i.e. everybody but the author of the fixups and respective reviewer(s)) it is very hard to track what the actual outcome of all of the fixups is.
In the git way of fixups (e.g. `fixup! <original commit>`), this is pretty easy: The outcome is what the original commit message promised.
Also, fixups and refactoring should not be conflated. Fixups should only ensure the intended outcome of the original patch is met. Refactoring code, for example simplification, cosmetic changes, etc. should be done in separate patches anyway.
Both can make bisecting for specific problems tedious if unlucky.Lol, that's still better than squashing commits into bigger ones which makes bisecting less feasible.
Squashing into uber-patches also defeats the purpose of `git blame`.
Squashing approach has a lot of significant drawbacks - it breaks `git bisect`, it breaks `git blame`, and depending on who is doing the squashing we might lose references to code reviews from "Reviewed-on:" lines. It also requires a lot of time and manual effort which leads to arbitrary decisions and possibly more errors. I just don't see why we would go thru all that when git and gerrit make things so simple and without all the manual steps.
Hi everybody,
I think it's best to put the whole story into context by giving a "brief" history of how the current state emerged (from my PoV). This first email explains the story up to fall 2016 and what the problems were that I tried to fix (and then I'll have to run and will continue later).
In October 2014 there was a coreboot (and flashrom) dev meeting in Prague where some of the most important stakeholder/devs of flashrom were present: - Carl-Daniel who at the time was already way less involved than up to about 2012. - David who was one of the main developers of the chromiumos fork of flashrom. - Nico who had a relatively small number of non-merged changes related to his work (at Secunet). - Myself who handled large portions of the upstream development the last few years.
While there was a bunch of patches that have been piled up back then it was less of a problem then the increasing divergence between the chromiumos fork and upstream. Thus we have discussed ways to converge that (by pulling changes mainly from upstream into chromium but also vice versa) and also increase the pace of merging stuff into upstream later. This was still with no intention to switch to git because of Carl-Daniel's concerns.
My workload at my day job increased quite much over time while transitioning from being a student to a full-time researcher and lecturer; Carl-Daniel got two kids, started his PhD and spent less and less time in front of the (dev) screen... the usual downward spiral of many FOSS projects I would say.
Carl-Daniel is also big part of the reason why flashrom has not switched from svn to git earlier because he is very fond of the monotonic and human-readable revision strings of svn. While I see the point in that I think git is way too superior in the daily work process but since git-svn worked so well (I did touch the repo less than a dozen timer with svn natively) this was not a major issue for myself. However, his contributions shrunk with time, everybody I spoke with was very clearly pro git (including random people creating github repositories with flashrom trees) and infrastructure maintenance burdens I derived a plan to convince even Carl-Daniel to support the switch.
By that time it had also become clear that development was quite stuck: 100s of the previous patches (including many of my own) were bitrotting while waiting for review and new contributors were kinda repelled by this state. There was also little hope that some newcomer could take over like I did in 2011. We had a kinda promising GSoC student but that did not work out either... Therefore when planing the switch to git I also proposed a change to the development process. Till then we had "a very flat and straightforward development model" - at least from the outside. Indeed there was only a single branch that got commits after they were (rigorously) reviewed by at least one of the regular contributors but I have about 100 local branches, there is the chromiumos fork etc.
The main issue regarding the development process, however, is that Carl-Daniel and myself have very strict ideas of what code quality we want to release to ordinary users that often try stupid things with flashrom (-p internal). However, this requires at least some reviews, discussion and testing, which took too much time in the past. This clashes very much with the expectations of the other type of users who are developers themselves of at least tinkering/recovering/hacking stuff and mostly use external programmers. They can live with bugs and bricks because that's most often what they have to begin with. On the other hand they would rather like to see new features that help with their respective use cases and tools.
The plan I conceived to fix both was more or less what is written down in the wiki at the moment: - a public staging tree where new patches get pushed if they make somewhat sense and go into the right direction. They would ripe there, get fixes applied as soon as someone looks closer or bugs are noticed while testing/using that branch. Naturally this would be what the devs and adventurous users checkout. - a stable branch that more or less mimics the svn's trunk branch in terms of quality: patches that seem stable enough get cherry-picked/ squashed with their fixes onto it possibly with some additional small fixes as we see fit. This would also be the branch that gets tagged, were version branches for backported fixes would be branched off and which end up in distribution releases as well.
To convince Carl-Daniel of switching to git and supporting that development strategy I invented a versioning scheme that worked around his strongest reservations against git: the completely unreadable hashes as revision indicators. The basic scheme would be: <tag>-<# commits to upstream branch>-<upstream branch>-<# commits to commit>-<sha>(-dirty) resulting in version strings like v0.9.99-3-stable-4-e4f6643-dirty which would indicate quite clearly where it is situated in the tree (4 commits from stable with some local changes and that stable branch got 3 commits since the last tag).
Keep in mind that up to that point there were no plans to switch away from mailing-list based patch submission and reviewing since this was (and still is) Carl-Daniel's and my favorite way to deal with things.
At that point I did not foresee a major change in the workflow, responsibilities and people involved. The regulars and especially myself would be spending most of the time integrating patches into stable (with the usual discussion/acking if need be) and/or fixing issues on the staging branch. Regularly merging changes that were applied to stable but not staging into the latter would keep divergence within reasonable limits. In some extreme cases changes in staging could also be completely reverted if they are discovered to be a dead end or make things worse than expected.
The following picture sums up the process and versioning scheme: http://buildbot.flashrom.org/dev_tree.png
Carl-Daniel agreed, I wrote it down in the wiki and implemented the needed changes in the build system (makefile, getrevision script and user-side git hooks). Additionally, I also saw the opportunity to rewrite the repository history to include the earliest of flashrom's commits that were not integrated in the svn tree (going back to early 2002 instead of late 2003), fix lots of typos and nonsense in old commit messages etc. This lead to a script that automatically merged these two trees and fixed the commit messages. In parallel I worked on a buildbot that is able to at least build test many of the possible configurations/target architectures and OSes of flashrom, cf. http://buildbot.flashrom.org/buildresults/flashrom-000786-HKL/results.html This was meant to help with finding build-time issues in the staging branch. Of course we also need a repository and since there are also reservations against relying on 3rd party services like github I set up a cgit repo (with initial help from Patrick) and implemented various repo-side hooks for mail and IRC announcements, github mirroring and sanity checks.
At the time this was finished (about late 2016) we were kinda ready to switch from my point of view. In the next email I'll explain what (not) happened since then.
Thanks for the detailed write-up. I suppose there's another part coming so I don't want to get too deep into discussion just yet, but one part in particular caught my eye:
On Sat, Oct 14, 2017 at 7:20 AM, Stefan Tauner stefan.tauner@gmx.at wrote:
While there was a bunch of patches that have been piled up back then it was less of a problem then the increasing divergence between the chromiumos fork and upstream. Thus we have discussed ways to converge that (by pulling changes mainly from upstream into chromium but also vice versa) and also increase the pace of merging stuff into upstream later. This was still with no intention to switch to git because of Carl-Daniel's concerns.
I'm surprised that you think that chromiumos's divergence is a *worse* problem than the huge backlog of upstream patches. The chromiumos fork is self-contained, has its own review system, its own testing, and is targeted at a narrow set of devices. I don't understand how it could have been a problem for upstream and would be interested if you can elaborate on this point.
FWIW I did try pushing some features from chromiumos to upstream, but like other patches they never really got anywhere. I also tried working with Carl-Daniel for a couple months to sync the trees, but that effort didn't get very far.
Hi again,
just to summarize my last email (since it's been over a week again): there was quite some work done behind the scenes for flashrom's git conversion that also included a proposed change in the development process in form of a second "main" branch named staging that would have been introduced to get patches up to a sane quality level faster.
What I had not anticipated was the strong wish to change the reviewing process/tools and the resulting surge of reviews and new (conflicting) changes. My plan was simply to take patches provided by any means (and thus lowering the barrier for contributing although that was only a bonus) polish them only minimally and then push them to staging to get them tested. The old (and already somewhat tested) patches would have had priority to keep conflicts low. This would have been done rather manually and without much interaction with others (there were practically no others in the last years :).
However, I was persuaded to allow Patrick to set up coreboot's Gerrit for the staging branch of flashrom back in Spring 2017. This obviously made people enthusiastic to participate in the development again and especially got some reviewing going that has not happened on the ML for a long time (although it was by no means meant to be used already). This new interest was a great though (to me) very surprising development.
I ignored that for the time being because I saw it merely as a test to see how flashrom people get along with Gerrit and if they actually do reviews unlike on the ML before. And, again, I did not expect it to get that far but thought it would die off quickly and that I could deal with it then later. Another reason was that I had no control about it whatsoever anyway. That means I (still) cannot change the configuration including permissions, various hooks including the one that syncs the upstream repo (https://www.flashrom.org/git/flashrom.git/) etc. I *still* don't even have the rights to +2/-2 changes regarding flashrom on Gerrit. So instead of paying close attention to what happened on the staging branch without my involvement I spent the little free time I diverted to flashrom into polishing up the build bot further to allow its integration into jenkins (e.g., add synchronization to make (queuing of) parallel execution possible) and refine my git conversion patch to take Gerrit into account (which of course it wasn't yet because Gerrit was not in my plans before).
Unfortunately the staging branch on Gerrit got a bit out of hand unexpectedly due to its success. There were some changes submitted that kinda made dozens if not hundreds of hours of my work obsolete without anybody even consulting me. The git conversion patch that I submitted to the ML in fall 2016 (that wasn't reviewed there) was hijacked in the sense that David took the (already outdated) version from the ML and pushed it for review on Gerrit. Then Nico did a very thorough review of it and it got submitted eventually with a number of changes that were partially not in my interest including one that completely broke the generation of tarballs. Patrick (who set up Gerrit) re-implemented a patch that was even acknowledged on the ML and sitting on my local commit queue to be committed immediately after the git patch. In the meantime patchwork (the database of old (and new) patches submitted to the ML) broke (out of my reach) and it didn't look like it would get fixed for months - but Patrick did so eventually with some efforts.
All the above wasn't helping my motivation much either, neither was the continuous passive (and sometimes not so much) aggressiveness towards any efforts on my side to improve the situation from my PoV. The situation increasingly felt like a hostile takeover... which is even worse than a hostile fork. So it pretty much looks and feels like my involvement is neither needed nor welcome by the active community. I've been doing flashrom development as a hobby but it certainly became more than that - it became a responsibility towards the users and somehow an honor (for the lack of a better word).
However, it kinda stopped being fun in the last year and I can't influence the events the way I deem necessary to fulfill the perceived responsibility without making it even less fun for me and probably everybody else at the same time which in the current situation would simply lead to escalation that I deem even worse than give way. For those reasons I am (now officially) standing down as whatever I am/was to flashrom.
FWIW, AFAICT Nico is in general doing a good job driving the development and is certainly more than qualified to lead the development even though I am definitely not agreeing with him all the time and I would have wished for a better cooperation in the past years of my involvement - they would have been way more productive that way.
I'd say if anybody would actually ask me (but the email is too long already anyway) my biggest failure was that I tried to compromise too much. If I would have changed to git earlier and ignore Carl-Daniel's sentiments towards it, putting things forward even without reviews and no (or less) consensus (there can't be consensus in the void anyway), or even switching to Gerrit and not trying to compromise with myself in this regard... things could have been much different now.
I'll probably not vanish completely and try to at least push the Patchwork patches and my private ones to Gerrit when I got time and maybe even more but I won't promise anything anymore that I possibly won't keep nor will I take any responsibilities for future releases. Let me know if you need anything else (e.g., passwords).
On Mon, Oct 23, 2017 at 7:03 PM, Stefan Tauner stefan.tauner@gmx.at wrote:
Hi again,
just to summarize my last email (since it's been over a week again): there was quite some work done behind the scenes for flashrom's git conversion that also included a proposed change in the development process in form of a second "main" branch named staging that would have been introduced to get patches up to a sane quality level faster.
What I had not anticipated was the strong wish to change the reviewing process/tools and the resulting surge of reviews and new (conflicting) changes. My plan was simply to take patches provided by any means (and thus lowering the barrier for contributing although that was only a bonus) polish them only minimally and then push them to staging to get them tested. The old (and already somewhat tested) patches would have had priority to keep conflicts low. This would have been done rather manually and without much interaction with others (there were practically no others in the last years :).
This seems to have been a simple misunderstanding with what the branches were for. The name "staging" implies that we're supposed to dump all of our patches there.
If you want those to be your own branches that you have exclusive commit rights for, we should be able to accommodate that. We can do as Antonio suggested in the other thread and use master for the typical git workflow where people send patches, and you can do with "staging" and "stable" as you please. Perhaps we should also prefix them with "stefanct-" so that there is no confusion about who the gatekeeper is.
-
David Hendricks -
Nico Huber -
Peter Lemenkov -
Stefan Tauner