(I am replying to various messages of the thread within this single email. Hence the quotes below are of different people.)
For those who missed it on IRC, Stefan (Reinauer) has proposed retiring SVN and patch work in favor of Git and (likely) Gerrit + Jenkins. Maintaining SVN and Patchwork represents a surprisingly large maintenance burden on top of Git and Gerrit that coreboot uses. Additionally, many in the community work around SVN's limitations by mirroring the project at places like Github anyway.
My counterproposal is to use gitub instead of Gerrit. - I really can't stand gerrit's interface. - We (relatively) often get patches from one-time contributors which is way easier on github because "everybody" has an account there and knows how to work with it (and if not it is straight forward compared to gerrit). - We don't really need jenkins. I am not a big fan of CI although I am a fan of build testing (hence flashrom is build tested for over 30 architecture/OS combinations manually). Depending on how it would be to interface my build bot with jenkins it would however be an improvement (not sure if it is worth it though). - The only reason to avoid github for me would be due to its proprietary nature and related problems (lock-in etc). Please give me other reasons if you prefer coreboot.org-hosted gerrit over github.
Though I know stefanct often makes small corrections/modifications to patches before submitting them (because our code review process makes even small amounts of back-and-forth comments so cumbersome). At least this way such changes can be tracked trivially on-line by diffing the N and N-1 patch revisions in gerrit.
The reason is not because the back-and-forth is cumbersome but because I am a perfectionist who does not want to bother others with my annoying views of details and I don't expect anybody to care about these changes like me (this does not always hold). The same goes for commit messages. With Carl-Daniel I do exchange my views about these kind of things *if* there is any exchange that is ;)
If there really is a large demand for a non-svn hosting, I would be open to using mercurial (as master) because it at least has some sort of usability and can provide (conceptually limited) version numbers. I have worked for a few years with mercurial and the version numbers are extremely helpful even if they are per-branch and only semi-stable.
Just to make things clear here: I won't even discuss this option - it is none. If you want to use another VCS management tool/interface then you are free to do so, but you are the only person known to prefer something else than git so you have to take on the burden and not force everybody else to as it has been up to now - this has to and will end.
The auto-tag of every commit sounds like an option I could agree with. It would preserve the convenience of the global strong svn commit ID even for git. However, I don't know if this would be acceptable for Stefan (especially from a merging perspective).
We have the infrastructure for using this kind of version strings since about 2,5 years and there is a patch that would make flashrom use them in prints which is lingering uncommented in the mailing list since August 2013. It might not be exactly what we want eventually, but if there are no comments I'll have to decide on something...
My current plan is to do a 0.9.9 release in January 2016 and switch to git afterwards. I'd like to keep patchwork at least for a while and figure out a way to keep the patches.
On 31.12.2015 19:18, Stefan Tauner wrote:
For those who missed it on IRC, Stefan (Reinauer) has proposed retiring SVN and patch work in favor of Git and (likely) Gerrit + Jenkins. Maintaining SVN and Patchwork represents a surprisingly large maintenance burden on top of Git and Gerrit that coreboot uses. Additionally, many in the community work around SVN's limitations by mirroring the project at places like Github anyway.
My counterproposal is to use gitub instead of Gerrit.
- I really can't stand gerrit's interface.
- We (relatively) often get patches from one-time contributors which is way easier on github because "everybody" has an account there and knows how to work with it (and if not it is straight forward compared to gerrit).
I don't think one-time contributors always have a github account. Also, in most cases (I might be wrong here), one-time contributors don't care much about a review, so we could just forward their patches to gerrit. Can we not?
- The only reason to avoid github for me would be due to its proprietary nature and related problems (lock-in etc). Please give me other reasons if you prefer coreboot.org-hosted gerrit over github.
Just speaking for myself here. Also I might have a github account I can't remember right now, my contributions to flashrom would be less if I have to adapt to another (non-gerrit) workflow. Just a prediction, I don't even like gerrit.
Nico
On Sat, 2 Jan 2016 16:07:23 +0100 Nico Huber nico.h@gmx.de wrote:
On 31.12.2015 19:18, Stefan Tauner wrote:
For those who missed it on IRC, Stefan (Reinauer) has proposed retiring SVN and patch work in favor of Git and (likely) Gerrit + Jenkins. Maintaining SVN and Patchwork represents a surprisingly large maintenance burden on top of Git and Gerrit that coreboot uses. Additionally, many in the community work around SVN's limitations by mirroring the project at places like Github anyway.
My counterproposal is to use gitub instead of Gerrit.
- I really can't stand gerrit's interface.
- We (relatively) often get patches from one-time contributors which is way easier on github because "everybody" has an account there and knows how to work with it (and if not it is straight forward compared to gerrit).
I don't think one-time contributors always have a github account. Also, in most cases (I might be wrong here), one-time contributors don't care much about a review, so we could just forward their patches to gerrit. Can we not?
I don't think that your premise holds. One-time contributors not necessarily dump a patch and then run away. Often they want to add support for a programmer they need and react timely on reviews but are not interested in contributing anything further than that. I deem these people quite valuable to the development of flashrom because they often show a different perspective.
So yes, we could always take patches in multiple ways (I presume you meant additionally by mail as it has been so far) and forward it to whatever reviewing platform we use, but if possible I'd like to use just one easily approachable way to keep them in one place.
- The only reason to avoid github for me would be due to its proprietary nature and related problems (lock-in etc). Please give me other reasons if you prefer coreboot.org-hosted gerrit over github.
Just speaking for myself here. Also I might have a github account I can't remember right now, my contributions to flashrom would be less if I have to adapt to another (non-gerrit) workflow. Just a prediction, I don't even like gerrit.
You have not spoken up so far. What would your preferred flow look like - apart from merging your patches waaaay faster than we did so far? ;)
On Sat, Jan 2, 2016 at 7:07 AM, Nico Huber nico.h@gmx.de wrote:
- The only reason to avoid github for me would be due to its proprietary nature and related problems (lock-in etc). Please give me other reasons if you prefer coreboot.org-hosted gerrit over github.
Just speaking for myself here. Also I might have a github account I can't remember right now, my contributions to flashrom would be less if I have to adapt to another (non-gerrit) workflow. Just a prediction, I don't even like gerrit.
I prefer using the same workflow as coreboot (I actually kinda like gerrit), as do others who filled out the Doodle poll a while back: http://doodle.com/poll/p2dxtnksmsgwmzrq
But I won't object so switching to github. It will be an improvement over the current workflow and ought to make the coreboot.org admins happy to eventually be rid of SVN+patchwork.
* Stefan Tauner stefan.tauner@alumni.tuwien.ac.at [151231 19:18]:
My counterproposal is to use gitub instead of Gerrit.
- We (relatively) often get patches from one-time contributors which is way easier on github because "everybody" has an account there and knows how to work with it (and if not it is straight forward compared to gerrit).
Unlike with gerrit, you explicitly have to sign up with github. With Gerrit you can use any Oauth2 provider that you already signed up with, significantly lowering the entry barrier
- We don't really need jenkins. I am not a big fan of CI although I am a fan of build testing (hence flashrom is build tested for over 30 architecture/OS combinations manually). Depending on how it would be to interface my build bot with jenkins it would however be an improvement (not sure if it is worth it though).
That is exactly what we are doing with the current infrastructure, and it has worked quite well for the coreboot project for about five years. Sure you can roll your own and probably end up with something slimmer, but jenkins is well tested and security reviewed, so it is very convenient and basically out of the way once you set it up.
- The only reason to avoid github for me would be due to its proprietary nature and related problems (lock-in etc). Please give me other reasons if you prefer coreboot.org-hosted gerrit over github.
I don't feel as strongly about trusting github with all of your project data, especially as you can recreate the repository from any checked out tree, but I remember the libreboot guys having strong opinions about github.
With a fair amount of overlap between developers, using one common infrastructure for both coreboot and flashrom seems to make sense to me, but honestly, any improvement of both process and tooling would be greatly appreciated. Right now we are going out of our way with one-offs to keep the environment painful.
Though I know stefanct often makes small corrections/modifications to patches before submitting them (because our code review process makes even small amounts of back-and-forth comments so cumbersome). At least this way such changes can be tracked trivially on-line by diffing the N and N-1 patch revisions in gerrit.
The reason is not because the back-and-forth is cumbersome but because I am a perfectionist who does not want to bother others with my annoying views of details and I don't expect anybody to care about these changes like me (this does not always hold). The same goes for commit messages. With Carl-Daniel I do exchange my views about these kind of things *if* there is any exchange that is ;)
I think a cleaner work flow in which it is clear which changes have been done by the original author and what changes have been done by Stefan on top of that would be very beneficial for the project. I don't think that the lack of transparency in this regard can or should be explained by perfectionism.
The auto-tag of every commit sounds like an option I could agree with. It would preserve the convenience of the global strong svn commit ID even for git. However, I don't know if this would be acceptable for Stefan (especially from a merging perspective).
We have the infrastructure for using this kind of version strings since about 2,5 years and there is a patch that would make flashrom use them in prints which is lingering uncommented in the mailing list since August 2013. It might not be exactly what we want eventually, but if there are no comments I'll have to decide on something...
You guys should explore what coreboot has done in this area, also with the work on reproducible builds. As an anecdote, I was insisting on the usefulness of sequencial commit ids just about until the day we switched coreboot to git. What you really want is a process that enables interactive development, tools to track the process, and more frequent releases. Unless you want to make sure that there are exactly two people working on this project, in which case all of this infrastructure talk is really overkill.
My current plan is to do a 0.9.9 release in January 2016 and switch to git afterwards. I'd like to keep patchwork at least for a while and figure out a way to keep the patches.
Cool. Please let me know what your preferred direction is by the end of this month then. We are currently in the process of migrating coreboot.org's services and the old infrastructure as we have it today will most likely be going away around that same time frame.
Stefan
On Thu, 7 Jan 2016 01:55:01 +0100 Stefan Reinauer stefan.reinauer@coreboot.org wrote:
- Stefan Tauner stefan.tauner@alumni.tuwien.ac.at [151231 19:18]:
My counterproposal is to use gitub instead of Gerrit.
- We (relatively) often get patches from one-time contributors which is way easier on github because "everybody" has an account there and knows how to work with it (and if not it is straight forward compared to gerrit).
Unlike with gerrit, you explicitly have to sign up with github. With Gerrit you can use any Oauth2 provider that you already signed up with, significantly lowering the entry barrier
Registering is only a small part of the problem (and that oath stuff did not work for everyone that smoothly in the past) of contributing something. As I said I think most potential flashrom contributors have a github account already.
My main point of critique is that working with gerrit is not straight forward. I had to help several people with some steps described in the wiki because they are not that intuitive. You receive some advantages due to that though like the hook but grouping changesets to topics is something that not too many newcomers initially grasp or use - on github that's the default because everything pushed upstream is packaged in pull requests. I deem that rather advantageous and important even though gerrit tries to automatically group related commits together and that seems to work quite well. We had several contributions via github in the last year without requesting that (rather the other way around "cant I just open a PR on github instead of mailing") and there were 0 problems with that regarding hand-holding the contributors (very unlike what I have experienced with gerrit). This is no deal breaker and gerrit is surely manageable I just wanted to object the narrative of registration being the main part of the barrier.
- We don't really need jenkins. I am not a big fan of CI although I am a fan of build testing (hence flashrom is build tested for over 30 architecture/OS combinations manually). Depending on how it would be to interface my build bot with jenkins it would however be an improvement (not sure if it is worth it though).
That is exactly what we are doing with the current infrastructure, and it has worked quite well for the coreboot project for about five years. Sure you can roll your own and probably end up with something slimmer, but jenkins is well tested and security reviewed, so it is very convenient and basically out of the way once you set it up.
No, you are not building coreboot for 15 different operating systems (some of them with cross compilers, some of them within virtual machines), with 5 different libcs and for 11 different hardware architectures and I am quite sure that jenkins does not support all of that out of the box :) It is not perfect at all but it was a great leap in testing flashrom for compile-time regressions - something that has bitten us a few times due to my cleanups in some core parts.
- The only reason to avoid github for me would be due to its proprietary nature and related problems (lock-in etc). Please give me other reasons if you prefer coreboot.org-hosted gerrit over github.
I don't feel as strongly about trusting github with all of your project data, especially as you can recreate the repository from any checked out tree, but I remember the libreboot guys having strong opinions about github.
If I'd be mean (which I usually are) I would say staying away from them is not a bad side effect :) but yes that's a point. On the other side I don't have control over much of flashrom's infrastructure and to be honest: you (personally, Patrick and Google's flashrom hackers as a whole) have more reasons to interfere with what I am doing than github (who would become extinct very soon if they really do something stupid - exactly for the reason you stated). I am not implying any hostility from your side at all (rather the opposite!) but you need to see it from my point of view:
When I joined the project (spring 2011) the number of contributors and contributions were already declining rapidly and many of my own patches piled up without even being commented on. About all other serious contributors vanished while or even before I became familiar with the mechanics of the community. I took over almost the whole burden of giving support on the mailing list rather quickly and tried to suggest and implement obvious but complex improvements while Carl-Daniel was still somewhat active and responsive but his reactions slowly got fainter. However, they did not completely vanish and whenever I was about to do radical changes (of the workflow as well as the source) he was there and restrained the approach by disliking some details and vanishing again after speaking up and leaving me in a rather unpleasant situation: since I became pretty much the only maintainer any criticism of flashrom's agility became effectively a direct critique of myself even though it was nothing I decided on my own alone (although I agree with many of Carl-Daniel's rationales regarding a stable product).
On the other side no one stood up and tried to help either (e.g., handle mails, reviewing or even discussing(!) any patches) which actually made me rather ignorant regarding critics. I have written about 3000 mails to the mailing list in less than 5 years, did two releases pretty much on my own, leased and set up the build bot and are the only committer in the last 1,5 years. As long as no one else shows at least some similar dedication to the project I can't take critics very serious. Using our patch and review process as excuse in this regard does not work either - only a part of what I really do is reviewing patches. Much time goes into research and support (the former often due to the latter) that is not really related to patches at all.
All of that together with my non-freetime work led to the current situation where everybody is becoming rather impatient with me. And while I am looking rather optimistically at the next months I would not be surprised if the perceived pushing away from the current model intensifies in unpleasant forms.
With a fair amount of overlap between developers, using one common infrastructure for both coreboot and flashrom seems to make sense to me, but honestly, any improvement of both process and tooling would be greatly appreciated. Right now we are going out of our way with one-offs to keep the environment painful.
I don't see that overlap to exist at all. Apart from the very few interactions with David (which I am very grateful most of the time (if it does not only contain bashing the upstream process ;)) I don't deem what chromiumos does as part of flashrom. The sources diverged so much that they really have become two distinct softwares and no change in process will change that quickly.
The only person IIRC who is a coreboot developer and who contributed to flashrom in the last year was mrnuke (via github btw).
However, the pool of coreboot developers surely have a great potential to contribute to flashrom and having a common infrastructure and similar processes would make sense, yes.
Though I know stefanct often makes small corrections/modifications to patches before submitting them (because our code review process makes even small amounts of back-and-forth comments so cumbersome). At least this way such changes can be tracked trivially on-line by diffing the N and N-1 patch revisions in gerrit.
The reason is not because the back-and-forth is cumbersome but because I am a perfectionist who does not want to bother others with my annoying views of details and I don't expect anybody to care about these changes like me (this does not always hold). The same goes for commit messages. With Carl-Daniel I do exchange my views about these kind of things *if* there is any exchange that is ;)
I think a cleaner work flow in which it is clear which changes have been done by the original author and what changes have been done by Stefan on top of that would be very beneficial for the project.
Why? What would change?
I don't think that the lack of transparency in this regard can or should be explained by perfectionism.
I don't get why it is a problem - I don't have the feeling that anybody really cares or would like to look at the respective changes. After all the years where I pushed so many changes to the mailing list often without ANY reaction I really doubt this would change if - for example - I'd make two commits for any larger foreign patch. What do you propose?
The auto-tag of every commit sounds like an option I could agree with. It would preserve the convenience of the global strong svn commit ID even for git. However, I don't know if this would be acceptable for Stefan (especially from a merging perspective).
We have the infrastructure for using this kind of version strings since about 2,5 years and there is a patch that would make flashrom use them in prints which is lingering uncommented in the mailing list since August 2013. It might not be exactly what we want eventually, but if there are no comments I'll have to decide on something...
You guys should explore what coreboot has done in this area, also with the work on reproducible builds. As an anecdote, I was insisting on the usefulness of sequencial commit ids just about until the day we switched coreboot to git. What you really want is a process that enables interactive development, tools to track the process, and more frequent releases. Unless you want to make sure that there are exactly two people working on this project, in which case all of this infrastructure talk is really overkill.
Two people would be a great improvement and thus your statement gives a hint of lack of understanding of the situation IMHO. ;) Joking aside, I don't expect a huge change in participation through a change of tooling due to the experience of the last years. Those who really want to contribute find a way to do that IMHO.
My current plan is to do a 0.9.9 release in January 2016 and switch to git afterwards. I'd like to keep patchwork at least for a while and figure out a way to keep the patches.
Cool. Please let me know what your preferred direction is by the end of this month then. We are currently in the process of migrating coreboot.org's services and the old infrastructure as we have it today will most likely be going away around that same time frame.
Do you have any suggestions regarding the patchwork database? What did coreboot do with its pw when it was switched to gerrit?
2016-01-07 9:09 GMT+01:00 Stefan Tauner stefan.tauner@alumni.tuwien.ac.at:
Do you have any suggestions regarding the patchwork database? What did coreboot do with its pw when it was switched to gerrit?
We noticed that nobody cared, so it's still lying dormant.
Patrick
-
David Hendricks -
Nico Huber -
Patrick Georgi -
Stefan Reinauer -
Stefan Tauner