A proposal to move GNOME to GitLab

The GNOME project has, after a period of contemplation, put forward a proposal to move to a GitLab installation on GNOME's infrastructure. "We are confident that GitLab is a good choice for GNOME, and we can’t wait for GNOME to modernise our developer experience with it. It will provide us with vastly more effective tools, an easier landing for newcomers, and lots of opportunities to improve the way that we work. We're ready to start working on the migration." This wiki page describes the idea in detail.

From:		Allan Day <aday-AT-gnome.org>
To:		desktop-devel-list <desktop-devel-list-AT-gnome.org>
Subject:		Proposal to deploy GitLab on gnome.org
Date:		Tue, 16 May 2017 14:22:13 +0100
Message-ID:		<CAC_wn4s3hbX=gsQm4XXVEnC3NtoiBEJuSv=aQaDMtXWkFuRdTw@mail.gmail.com>
Archive-link:		Article

[Written on behalf of Alberto Ruiz, Carlos Soriano, Andrea Veri, Emmanuele
Bassi and myself.]

Dear community,

Over the years, many of us have become increasingly frustrated about the
state of our development infrastructure, Bugzilla in particular. Pretty
much everyone we’ve spoken to doesn’t like it, and it’s not hard to see
why: it is littered with usability issues, code review is a pain, and it is
light-years behind more modern development platforms.

In the past, there haven’t been many other options, but we’re now in the
fortunate position of having viable alternatives and the sysadmin resources
to set one of them up and maintain it.

In recent months we have got together to examine the possibilities for
GNOME’s development infrastructure. We’ve spent a lot of time on this,
because we want the community to have faith in our conclusions. If you are
interested in this, you can read our research on the wiki [1].

The outcome of this evaluation process is that we are recommending that
GNOME sets up its own GitLab instance, as a replacement for Bugzilla and
cgit.

We are confident that GitLab is a good choice for GNOME, and we can’t wait
for GNOME to modernise our developer experience with it. It will provide us
with vastly more effective tools, an easier landing for newcomers, and lots
of opportunities to improve the way that we work. We're ready to start
working on the migration.

Please bear in mind that this is just a recommendation! We are not claiming
to have complete knowledge and we would like to hear questions and
comments. At the same time, we do ask that members of the community
approach this proposal with an open mind: please read the wiki pages and
try to resist making assumptions about GitLab without familiarising
yourself with it.

Yours,

Allan Day


[1] https://wiki.gnome.org/Initiatives/DevelopmentInfrastruct...
_______________________________________________
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

(Log in to post comments)

Posted May 16, 2017 14:17 UTC (Tue) by Conan_Kudo (subscriber, #103240) [Link]

This will be great for GNOME. The next generation of Free Software developers will be able to observe, play with, and contribute to GNOME more easily than ever before.

KDE, it's your turn!

A proposal to move GNOME to GitLab

Posted May 16, 2017 15:04 UTC (Tue) by boudewijn (subscriber, #14185) [Link]

"KDE, it's your turn!"

Huh? Turn to do what? We moved to Phabricator in 2014 or so.

A proposal to move GNOME to GitLab

Posted May 16, 2017 18:10 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Personally I'd say that's not that much better than Sourceforge, but my only interaction with Phabricator has been as a "drive-by" contributor (3 years ago now? I don't know how up-to-date the deployment was. It was also not to KDE either if you're curious.). Requiring either special client-side tools or limited to "paste your patch content here" for contribution to a Git repository was…less than fun.

A proposal to move GNOME to GitLab

Posted May 17, 2017 18:33 UTC (Wed) by boudewijn (subscriber, #14185) [Link]

Well, our sysadmins quickly discarded gitlab because it scaled very badly, the group system wouldn't fit a community like KDE very well -- and then they discovered that gitlab mails debug logs to a hard-coded gmail address.

I like phabricator's repo browsing, task system, possibility to create mockups and stuff, project boards and dashboards. I think its search abilities are below par. We don't use it to replace bugzilla (yet), instead users report to bugzilla, and, this is for Krita alone, I triage the bugs in bugzilla, and when I want action to be taken on a bug, I add a task in phabricator. So, phab is where our todo is, bugzilla where the user reports are.

A proposal to move GNOME to GitLab

Posted May 17, 2017 19:17 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

> they discovered that gitlab mails debug logs to a hard-coded gmail address.

Do you have details on this?

And I'm aware that one tool isn't going to fit everyone. I'm glad (but not surprised) that KDE chose a FOSS solution at least. I just didn't like Phabricator from a contributor POV when I was submitting a patch to a project using it.

A proposal to move GNOME to GitLab

Posted May 17, 2017 19:49 UTC (Wed) by pizza (subscriber, #46) [Link]

>> they discovered that gitlab mails debug logs to a hard-coded gmail address.
> Do you have details on this?

Even if true, that seems trivial to change or disable, given the full-source-code nature of gitlab...

A proposal to move GNOME to GitLab

Posted May 18, 2017 6:31 UTC (Thu) by boudewijn (subscriber, #14185) [Link]

Trivial? You'd have to do a full scan of the code for every update. Sure it wasn't malicious, probably, just someone being stupid like all of us developers are sometimes, but for our volunteer sysadmins it was a dealbreaker. For more details, you'd have to ask them... I'm just a developer.

A proposal to move GNOME to GitLab

Posted May 16, 2017 14:40 UTC (Tue) by flussence (subscriber, #85566) [Link]

I'm curious to know if they made any attempt whatsoever to take the “usability issues” mentioned to Mozilla.

A proposal to move GNOME to GitLab

Posted May 16, 2017 15:02 UTC (Tue) by ebassi (subscriber, #54855) [Link]

I'm curious to know if they made any attempt whatsoever to take the “usability issues” mentioned to Mozilla.

Of course we did. We've been using Bugzilla for more than 15 years, which means we had interactions with the Bugzilla upstream.

The main issue is that we're running with an older version of Bugzilla because we have additional modifications on top of it (and in many cases we had to drop changes from Bugzilla update to Bugzilla update) and don't have the resources to maintain a bleeding edge installation without going completely vanilla, which unfortunately is not really possible.

Additionally, Bugzilla is really not integrated with the code repositories — and that won't likely change at all — or with the additional tools and services that we have in place for documentation, translation, and design.

Finally, we found that the user experience for newcomers is still kind of a major roadblock; creating patches, attaching them, reviewing them, and applying them is not really obvious any more, in a world where pull requests on GitHub are the dominant form of contribution workflow.

Bugzilla is an incredible piece of software, and it served us well enough in the past; our requirements changed, though, so it's time to look at something different.

GitHub?

Posted May 16, 2017 15:37 UTC (Tue) by NightMonkey (subscriber, #23051) [Link]

I'm just curious on if GitHub was considered? While yes, you'd be 'hosting' code and trackers on a private company's infrastructure, that's not such a onerous burden since you can just pick up your repo and go elsewhere. Plus, perhaps you can have your sysadmin work on more fun things?

No, I don't work for GitHub. ;)

P.S. I also really like https://www.codereviewhub.com/ . I implemented it for one of my dev teams (who all came from Google HQ) and they really liked it.

GitHub?

Posted May 16, 2017 15:45 UTC (Tue) by pizza (subscriber, #46) [Link]

I can't speak for GNOME's use cases, but the Github tracker is pretty abysmal for all but the most trivial of projects. It's still not as bad as sourceforge's though...

GitHub?

Posted May 16, 2017 17:17 UTC (Tue) by jhoblitt (subscriber, #77733) [Link]

GH issues have gotten a lot better over time. Issues can now at least be assigned to a dev and milestones have proven to be useful. I have heard that GH dogfoods issues internally via dedicated "issue" repos but I have trouble imagining a workflow that would scale to a large multi-repo project.

I think the reason many "open source" projects end up using Jira is because it is relatively straight forward to setup custom workflows (which survive upgrades). I don't know of any alternative, open source or commercial, that allows for that level of customization without brittle source modifications (I believe fog bugs has supposedly improved in this area?). Don't get me wrong, I have a long list of complaints about Jira, but it seems to be the least-worst option.

GitHub?

Posted May 16, 2017 15:52 UTC (Tue) by ebassi (subscriber, #54855) [Link]

We have a full mirror on GitHub already.

To be fair: issue tracking on GitHub is pretty dire, and it's very much repository-oriented; you cannot move issues between projects under the same organisation, for instance. Virtually any complex project spanning multiple repositories that I know of — including commercial ones, though the enterprise version — hosted on GitHub end up moving away from GitHub issues to their own self-hosted issue and project management tracker.

Additionally, and this is my personal opinion, I think it's not great form for a free software project the size of GNOME to switch to a closed platform like GitHub.

GitHub?

Posted May 16, 2017 18:09 UTC (Tue) by SEJeff (subscriber, #51588) [Link]

Emanuel, that is incorrect, and has been for a bit.

https://github.com/blog/2272-introducing-projects-for-org...

https://github.com/blog/2256-a-whole-new-github-universe-...

Projects most definitely support multiple repos and their kanban board support isn't too bad honestly.

I do agree that something that goes out of the way to be libre (GNOME) shouldn't be primarily hosted on non-libre software (github) while my personal opinion is

GitHub?

Posted May 16, 2017 18:36 UTC (Tue) by ebassi (subscriber, #54855) [Link]

I know about Projects, and they are really not what I meant.

Example: if I, a user, end up filing a bug against application A, and the maintainer, after triage, decides that it's really an issue of a dependency inside GNOME, even if application A and the dependency are part of the same project, the issue is still assigned to application A; creating a project just means that you can see all the issues that are part of the project.

Of course, the way it's meant to be used is to have multiple projects inside the same organisation — say: SDK, applications, infrastructure, etc. This still does not allow me to move an issue from application A, inside the "applications" project, to the dependency inside the "SDK" project.

Issues still require me to open a new issue inside the dependency's repository (either directly or by turning a note into an issue), and then have a commit/merge message that looks like:

Fixes: organization/application-a#1234
Fixes: organization/toolkit#2345

With the chance of forgetting either issue, and then having to close issues left "dangling".

GitHub?

Posted May 16, 2017 15:56 UTC (Tue) by ebassi (subscriber, #54855) [Link]

Missed this:

> that's not such a onerous burden since you can just pick up your repo and go elsewhere

I think you're overlooking what comes along with the code — which is what makes any move onerous.

The whole point of this migration is tying up the bug tracker and other services to the Git repositories; if we move away from GitHub we'd have to move away all the issues filed, all the accounts, and disentangle the cross-references.

GitHub?

Posted May 16, 2017 16:06 UTC (Tue) by NightMonkey (subscriber, #23051) [Link]

Totally get it. Yes, their issue tracker (sans plugins or other enhancements) doesn't scale well. Thanks for the considered replies and info. It is quite helpful to see how other projects make their decisions, especially ones as large and complex as GNOME. :D

GitHub?

Posted May 16, 2017 17:33 UTC (Tue) by andreasn1 (subscriber, #88420) [Link]

"since you can just pick up your repo and go elsewhere"

I'm working on a project that's on github, and hopefully we'll never be forced to move away (due to them closing shop, or whatever the future holds), but if that was the case, is it also possible to migrate all the issues in the issue tracker and the pages in the wiki?

GitHub?

Posted May 16, 2017 17:37 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

You can use the API to fetch all of the data at least. The wiki is also a git repository itself. I don't know if they have a way to download it all as a single tarball.

GitHub?

Posted May 16, 2017 17:48 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Importing projects into Github can be lossy. For example, issues and PRs share a namespace and there's no way to turn off PRs on a project. If, during an issue migration, someone opens a PR, if your goal is to align issue numbers, you're forever off-by-one. I've imported 3 projects into Gitlab with 16000+ issues each[1] and the issue numbers today are the same as the pre-Gitlab era (Mantis *shudder*).

Other benefits of Gitlab that I've liked that Github doesn't have (or am unaware of):

- ability to move issues between repositories (given enough permissions on both repos);
- "discussions" on MRs where users can say "yes, this comment has been addressed" keeping discussions concise;
- diffs between MR pushes are readily available (not just via a notification or email link);
- terminology: "merge request" is better than "pull request" IMO since I don't care if you pull my code; I'd like it merged ;) );
- self-hosting so that you can have admin access if that's necessary;
- add "TODO" items for issues and MRs; and
- TODO/notifications are not cleared just by visiting the target page (adding a comment clears it).

Things I'd like Gitlab to implement/enable:

- emails for MR updates;
- emails for your own actions;
- MR reviews are Enterprise-only (I believe there is push to have it in CE as well);
- non-Gitlab-CI testing is a second-class citizen;
- forks between all repos in a "fork network" (I have a *really old* MR that does this, but it got stale due to a lack of time); and
- hook JSON objects are unstable and undocumented.

I can find issues/MR links for those interested in this list.

[1]They had a shared numbering namespace before.

GitHub?

Posted May 17, 2017 3:06 UTC (Wed) by raven667 (subscriber, #5198) [Link]

> non-Gitlab-CI testing is a second-class citizen;

I have been using the gitlab runner for very basic CI, using tito to build RPMs/createrepo on commit, but it looks like you could integrate with anything if you can drive it with scripts run from the gitlab runner account. Do you consider this an unusually messy way to do integration, instead expecting plugins for gitlab itself to drive the APIs of other CI tools? I'm interested in this area and just want to be explicit in my understanding of your critique.

GitHub?

Posted May 17, 2017 12:37 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

Our testing matrix has about 20 or so axes, 8 of which really matter and maybe 3 more which only apply if other options are set correctly. I don't know the best way to express such things in a flat YAML file, especially is some need extra things to be done on the machine itself (like having CUDA or a special built Homebrew package available). The projects can also take up to 45 minutes to build and test, so we don't have the hardware to test every single MR that comes in (we also need real GPUs, so Docker containers or cloud machines are generally out). Instead, we use build or mediated by another bot which checks for permission to start testing. This can be used to focus test only on certain platforms or against builds which have settings a certain way (e.g., a Windows-specific change doesn't need to waste the macOS builder's time).

On the Gitlab side, if you're using a fork-based workflow (as we are), you need Developer access to a repo to set commit statuses, so our main bot (which runs with admin privileges for other Gitlab-specific reasons), adds our buildbot account to all new repos. Neither bot with the tokens run arbitrary code from the projects, so leaking via a malicious MR isn't trivial at least. It also means that if a fork turns off the Pipelines feature on their repo, viewing the statuses is basically broken. There are existing issues for these problems as well.

GitHub?

Posted May 17, 2017 16:44 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

JFYI, both AWS and Azure have instance types with GPU.

GitHub?

Posted May 17, 2017 17:51 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

Yeah, but we still need macOS tested too :/ . I also think those GPUs tend to be homogeneous which makes testing wider ranges less viable. If we wanted to go full-cloud for testing, it'd be really expensive (either persistent machines to take advantage of incremental builds versus full builds every time) and I think we'd have to manage things across 3 or 4 providers. Just having the hardware local makes build management uniform at least even if maintenance is more of a burden (all of the builder descriptions live in one place and use a uniform description "language" rather than some being Docker containers, others being VM images, and whatever one uses for provisioning macOS testers.

And one project supports platforms that will never be in the cloud (VS 2008, macOS 10.7 (or so?), HP-UX, AIX, Solaris, etc.), so we're still back to some kind of local test infrastructure management solution.

GitHub?

Posted May 17, 2017 18:50 UTC (Wed) by excors (subscriber, #95769) [Link]

You could have short-lived VMs with persistent storage, so you can still do incremental builds despite starting a fresh VM each time. Or use separate machines for building and for running the tests. EC2's on-demand GPU instances cost ~50%-100% more than reserved ones per hour, so they're cheaper if you're only using it <50% of the time (after rounding up each use to an integer number of hours).

("Cheaper" is relative of course, it looks like EC2's cheapest modern one (with half of a GRID K520) is around $3K/year reserved, and the ones with more compute power cost more, so probably not worthwhile if you could get away with a cheap consumer GPU and don't need any of the other cloud features.)

GitHub?

Posted May 17, 2017 19:18 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

For clean builds, our build/test cycle takes about 20–40 minutes (depending on platform) machines on 10-core CPUs with hyperthreading for one project; another takes at least 45 on the same hardware. Incremental builds can be as small as 5 minutes, 15 for the larger project. The machines cost ~$2500 up front and can run multiple builds (depending on the project). I don't know what the electricity costs. There's also a benefit in being able to sit down with a machine to see what's gone wrong on it (helpful when you're dealing with things like rendering differences).

GitHub?

Posted May 17, 2017 20:00 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Yes, for GPU it might make sense to do local builds.

BTW, one of our clients uses pre-built containers to run stuff on expensive instances (with 1Tb of RAM). The build is handled on cheap instance types and only the final containers are run on the expensive instances that are spun down once the calculation is over.

You can also sometimes get GPU instances on EC2 Spot Instances for very cheap.

GitHub?

Posted May 17, 2017 10:26 UTC (Wed) by flussence (subscriber, #85566) [Link]

There's technical reasons to avoid GitHub too: they reinvent far too many wheels and their implementations all subtly suck. There's no support for modern SSH or GPG (ed25519 etc), and their syntax highlighter was switched from Pygments (what GitLab uses) to some home-grown one with spotty language support. They're also very unresponsive to pull requests for the few things that are open to be fixed, from what I've observed.

GitHub?

Posted May 17, 2017 12:33 UTC (Wed) by gracinet (subscriber, #89400) [Link]

and in particular, still no IPv6 support. At least, I suppose that with a self-hosted Gitlab or whatever instance, it's not a problem for those that'd want it.

It can be a simplification not to rely on NAT for deployments on pure IPv6 hosts / containers, especially numerous ephemeral ones. Granted, I'm not sure that would be the most meaningful feature for GNOME

GitHub?

Posted May 17, 2017 10:53 UTC (Wed) by aggelos (subscriber, #41752) [Link]

While yes, you'd be 'hosting' code and trackers on a private company's infrastructure, that's not such a onerous burden since you can just pick up your repo and go elsewhere.

To the degree that you're using github-specific features, the pain of migrating away from it increases (i.e. you are being locked in). Conversely, if you're not making full use of its features, there's little gain from using github as opposed to any other solution.

So I don't think the "it's git, you can always move your repo away" works the way you suggest.

Plus, migrating is enough of an inconvenience when you're changing domains, let alone changing people's workflows. This applies to any infrastructure setup and a fortiori for a hosted solution where the provider has an incentive and track record of focusing on "value added" features. The gitlab people are adding features as fast as they can too, which is why their license choice (for the community edition) acts as a differentiator.

A proposal to move GNOME to GitLab

Posted May 16, 2017 17:01 UTC (Tue) by jhoblitt (subscriber, #77733) [Link]

One of the big advantages of hosting repos on github is that it makes it trivial for new contributors to fork + PR. With a "private" gitlab instance, would any random dev be able create an account to fork + PR, or do they need to become officially blessed before opening an account?

A proposal to move GNOME to GitLab

Posted May 16, 2017 17:35 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Gitlab allows for Github logins, so you just need to allow instances access to your email address.

A proposal to move GNOME to GitLab

Posted May 18, 2017 17:36 UTC (Thu) by catalinuxboie (guest, #112452) [Link]

(I am the main author of RocketGit - https://rocketgit.com)

Trivial in GitHub?! No, it is not trivial: create an account (+ read and accept the license), fork, clone, change+commit, push, do a merge request. 6 steps.

Trivial is: clone, change+commit, push (automatically converted to a merge request). 3 steps.
https://rocketgit.com/op/features/anonpush

A proposal to move GNOME to GitLab

Posted May 19, 2017 3:28 UTC (Fri) by zlynx (subscriber, #2285) [Link]

That's too trivial. I don't want a pull request every time I push. Push is for backing up my local changes. I may want to rewrite the entire branch history before creating a pull request.

And the first two steps in your list are already done by almost everyone. I don't think I personally know any programmers without Github accounts.

A proposal to move GNOME to GitLab

Posted May 19, 2017 10:47 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Rewriting history after making the PR is fine too (e.g., I find rebase way better for resolving conflicts than merging, but I also squash small fixes back into the commits they belong in). I work under the assumption that it's my fork and the code hasn't really passed review yet, so "good luck" on using topic branches on their own.

A proposal to move GNOME to GitLab

Posted May 19, 2017 17:56 UTC (Fri) by catalinuxboie (guest, #112452) [Link]

Sorry, I was not clear enough. If you are authenticated, it is not an anonymous push but a normal push.
Anonymous push is for people who would not want to create an account for various reasons.
But, would be a pity to loose their contributions.

What about Tuleap libre tool?

Posted May 18, 2017 13:22 UTC (Thu) by mmanon (guest, #116011) [Link]

Have you considered Tuleap for issue and project tracking? https://www.tuleap.org/ Gitlab does a good job for Git purpose, but you will gain advanced features for tracking, custom workflow and project management with Tuleap. It is a libre project.
Look at:
- the Eclipse Fondation, they are transitioning from Bugzilla to Tuleap: https://opensource.com/article/17/1/interview-Tuleap-project
- or the open source Ring project, that moved from Redmine and Github to Tuleap : https://blog.savoirfairelinux.com/fr-ca/s/tuleap/

You can also check out comparisons:
- Tuleap versus GitLab : https://www.tuleap.org/tuleap-versus-gitlab-battle-will-n...
- Tuleap versus Jira : https://www.tuleap.org/tuleap-versus-jira-software

PS: I'm a member of the Tuleap community:-)