Hi everybody,
I think it's best to put the whole story into context by giving a "brief" history of how the current state emerged (from my PoV). This first email explains the story up to fall 2016 and what the problems were that I tried to fix (and then I'll have to run and will continue later).
In October 2014 there was a coreboot (and flashrom) dev meeting in Prague where some of the most important stakeholder/devs of flashrom were present: - Carl-Daniel who at the time was already way less involved than up to about 2012. - David who was one of the main developers of the chromiumos fork of flashrom. - Nico who had a relatively small number of non-merged changes related to his work (at Secunet). - Myself who handled large portions of the upstream development the last few years.
While there was a bunch of patches that have been piled up back then it was less of a problem then the increasing divergence between the chromiumos fork and upstream. Thus we have discussed ways to converge that (by pulling changes mainly from upstream into chromium but also vice versa) and also increase the pace of merging stuff into upstream later. This was still with no intention to switch to git because of Carl-Daniel's concerns.
My workload at my day job increased quite much over time while transitioning from being a student to a full-time researcher and lecturer; Carl-Daniel got two kids, started his PhD and spent less and less time in front of the (dev) screen... the usual downward spiral of many FOSS projects I would say.
Carl-Daniel is also big part of the reason why flashrom has not switched from svn to git earlier because he is very fond of the monotonic and human-readable revision strings of svn. While I see the point in that I think git is way too superior in the daily work process but since git-svn worked so well (I did touch the repo less than a dozen timer with svn natively) this was not a major issue for myself. However, his contributions shrunk with time, everybody I spoke with was very clearly pro git (including random people creating github repositories with flashrom trees) and infrastructure maintenance burdens I derived a plan to convince even Carl-Daniel to support the switch.
By that time it had also become clear that development was quite stuck: 100s of the previous patches (including many of my own) were bitrotting while waiting for review and new contributors were kinda repelled by this state. There was also little hope that some newcomer could take over like I did in 2011. We had a kinda promising GSoC student but that did not work out either... Therefore when planing the switch to git I also proposed a change to the development process. Till then we had "a very flat and straightforward development model" - at least from the outside. Indeed there was only a single branch that got commits after they were (rigorously) reviewed by at least one of the regular contributors but I have about 100 local branches, there is the chromiumos fork etc.
The main issue regarding the development process, however, is that Carl-Daniel and myself have very strict ideas of what code quality we want to release to ordinary users that often try stupid things with flashrom (-p internal). However, this requires at least some reviews, discussion and testing, which took too much time in the past. This clashes very much with the expectations of the other type of users who are developers themselves of at least tinkering/recovering/hacking stuff and mostly use external programmers. They can live with bugs and bricks because that's most often what they have to begin with. On the other hand they would rather like to see new features that help with their respective use cases and tools.
The plan I conceived to fix both was more or less what is written down in the wiki at the moment: - a public staging tree where new patches get pushed if they make somewhat sense and go into the right direction. They would ripe there, get fixes applied as soon as someone looks closer or bugs are noticed while testing/using that branch. Naturally this would be what the devs and adventurous users checkout. - a stable branch that more or less mimics the svn's trunk branch in terms of quality: patches that seem stable enough get cherry-picked/ squashed with their fixes onto it possibly with some additional small fixes as we see fit. This would also be the branch that gets tagged, were version branches for backported fixes would be branched off and which end up in distribution releases as well.
To convince Carl-Daniel of switching to git and supporting that development strategy I invented a versioning scheme that worked around his strongest reservations against git: the completely unreadable hashes as revision indicators. The basic scheme would be: <tag>-<# commits to upstream branch>-<upstream branch>-<# commits to commit>-<sha>(-dirty) resulting in version strings like v0.9.99-3-stable-4-e4f6643-dirty which would indicate quite clearly where it is situated in the tree (4 commits from stable with some local changes and that stable branch got 3 commits since the last tag).
Keep in mind that up to that point there were no plans to switch away from mailing-list based patch submission and reviewing since this was (and still is) Carl-Daniel's and my favorite way to deal with things.
At that point I did not foresee a major change in the workflow, responsibilities and people involved. The regulars and especially myself would be spending most of the time integrating patches into stable (with the usual discussion/acking if need be) and/or fixing issues on the staging branch. Regularly merging changes that were applied to stable but not staging into the latter would keep divergence within reasonable limits. In some extreme cases changes in staging could also be completely reverted if they are discovered to be a dead end or make things worse than expected.
The following picture sums up the process and versioning scheme: http://buildbot.flashrom.org/dev_tree.png
Carl-Daniel agreed, I wrote it down in the wiki and implemented the needed changes in the build system (makefile, getrevision script and user-side git hooks). Additionally, I also saw the opportunity to rewrite the repository history to include the earliest of flashrom's commits that were not integrated in the svn tree (going back to early 2002 instead of late 2003), fix lots of typos and nonsense in old commit messages etc. This lead to a script that automatically merged these two trees and fixed the commit messages. In parallel I worked on a buildbot that is able to at least build test many of the possible configurations/target architectures and OSes of flashrom, cf. http://buildbot.flashrom.org/buildresults/flashrom-000786-HKL/results.html This was meant to help with finding build-time issues in the staging branch. Of course we also need a repository and since there are also reservations against relying on 3rd party services like github I set up a cgit repo (with initial help from Patrick) and implemented various repo-side hooks for mail and IRC announcements, github mirroring and sanity checks.
At the time this was finished (about late 2016) we were kinda ready to switch from my point of view. In the next email I'll explain what (not) happened since then.