Automatic integration branches / merge requests

Problem to solve

One of GitLab's values is iteration - the idea that we complete the smallest possible thing, take feedback, do it again. This is a great model for us, and in practice it means that we tend to create fairly small merge requests with few dependencies on each other; or where we do have dependencies, they tend to be linear - A depends on B depends on C. In practical terms, we tend to have lots of small, short-lived feature branches that only rarely interact with each other in unexpected ways (when they do interact with each other, we can get master:broken issues).

This is quite an unusual position for a large software company to be in, however. A much more normal development model - and one that's come up as mandatory in several customer calls I've been in - is for there to be several long-lived branches other than master, into which feature branches are merged (or, more commonly, into which commits are directly pushed). The branches are long-lived for regulatory or other process reasons, but all eventually go to master.

When developing against feature branches that go into the long-lived branch, or when there are multiple long-lived branches, that eventual merge can be very painful, because changes in branches A and B, which both must go in to C, happen independently, perhaps initially branching off of C at different times.

The closest we get to this model at GitLab is for security releases - we have 3 maintained stable branches at any one time. Security fixes are developed as one MR + 3 backport copies (one per stable branch - but note the stable branches are never merged back into master). We can have tens of security fixes in a single release, and we integrate late - we only merge the MRs into their respective target branch when it's almost time to do the security release, so when two fixes interact with each other, we have to fix it in a time-pressured manner

Intended users

User experience goal

Given that, even at GitLab, there are situations where N changes must go into a long-lived branch at about the same time; and given that a number of customers have stated that it is mandatory for them to do exactly the same thing with M features going into N long-lived branches eventually going into master, GitLab should aim to make this process as convenient as possible. An MR author should be able to tell the product "the changes in this single merge request are going to have this journey" - something currently expressed in a limited way via the target_branch attribute - and GitLab should be able to take the manual labour out of ensuring that this is a success.

Proposal

The best way to check whether we will be able to successfully merge a set of MRs into a given target branch is to do exactly that - create a branch that is the product of merging all the MRs into the target branch, and then run a CI pipeline for it. This is pretty easy to script: https://gitlab.com/gitlab-org/gitlab/-/issues/218446#note_346166909 - but since we already tell GitLab what the target branch is, we should also be able to tell GitLab: Check the integration status of this set of branches.

We give it the target branch we're interested in. It finds all MRs that have that target branch, and allows us to further narrow this by excluding certain MRs or providing a more-complex filter.

Once this is configured, GitLab creates a transient branch, and perhaps a transient WIP/managed/read-only merge request for that branch. Whenever any of the selected MRs receives a change, GitLab attempts to regenerate the transient branch. We could do this every time the target branch changes too, or on a regular schedule if the target moves quickly.

If merging an MR into that target branch fails, GitLab can send a notification similar to the existing "your merge request can't be merged!" message.

If merging the MRs succeeds, but the pipeline for the merge result fails, GitLab can send a notification similar to the existing "pipeline failed for your merge request" message.

These notifications could be to all MRs in the set, but that could get very noisy, so they could also be to the author of the integration branch setting.

What this does is move the discovery of integration problems to be immediately after the offending change is pushed, rather than the discovery happening

As a further optimisation, we could allow pipelines to be disabled for all MRs that are in the group, so that their success or failure depends solely on the merge result pipeline for their group. It's an open question as to whether we'd want to use that in any context for GitLab, but I think others who currently rely on "commit directly to the long-lived branch" could be convinced to switch to a feature branches -> long-lived (auto-generated) integration branches -> master model with this.

Finally, we could allow a single MR to be registered against multiple integration branches at once, and for MR authors to provide an "overriding MR" if the canonical MR generates conflicts or test failures. This allows us to reduce the number of MRs required for a typical GitLab security release, per change, from 4 to 1, without losing any test coverage.

The integration branch is useful for other reasons too - for instance, it allows associated groups (compliance, testing, security, documentation, etc) to test a batch of changes out after, rather than before, integration has occurred, by focusing primarily on the integration branch/MR. Again, this feels-like it would be super handy at GitLab for security releases, but also for our larger customers with heavy compliance requirements.

@m_gill @danielgruesso @marin @nolith (for a delivery perspective) I wonder what you think of this feature proposal. I feel like the first step - group N merge requests that target a stable branch and run CI for them before they are merged - is something we could deliver quite quickly, and that it would be pretty helpful in the context of catching problems before merge day on GitLab security releases. As I thought it through, I realised that with a few extensions, we could eliminate the need to have 4 (manual) MRs per change, while catching problems earlier and allowing engineers to still self-serve merge conflict resolution.

Edited May 20, 2020 by Nick Thomas