Async git reference updater

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Adding/removing/updating refs in a gitaly repository is more complex than it might first appear. There are several RPCs involved:

• WriteRef - This allows setting a single ref. Often not the most efficient as each call generates a new transaction.
• DeleteRefs - This RPC was added to solve performance problems with single ref transactions. Though since it only deletes references its usage is limited. There can still be problems with transaction ordering.
• UpdateRefs - This RPC can add/update/delete a stream of refs. These refs are all changed in the same transaction. All new features should use this RPC.

On top of that in gitlab-rails we have a handful of services and workers to allow features to either manipulate refs in bulk and/or in the background:

• MergeRequests::CleanupRefWorker - This worker allows async deletion of some specific merge request refs.
• BatchedGitRefUpdates::Deletion - This model has a corresponding cronjob that applies ref deletions in bulk. This is the functionality we want here but it is limited to ref deletions.

Ideally we would encapsulate all ref operations into a single async framework.