[go: up one dir, main page]

LFS batch requests should work with proxy downloads enabled

Background

Recently we discovered that large quantities of LFS requests caused an undue strain on Gitaly storage due to the need to validate user permissions for every LFS object. With some repositories having tens of thousands of LFS objects, the permission checks were overwhelming.

This issue was resolved in %16.2 via LFS: The `batch request` for download should pr... (#413684 - closed). We have confirmed with customers that this does indeed solve the issue.

Problem

While we have confirmed that the above does work, it does not function with proxy downloads enabled. This is not ideal as it leads to LFS objects being returned from object storage having the object storage URL (such as S3). This also means that for firewall rules, the customer would need to trust and open up all S3 addresses.

Furthermore, this is enabled for GitLab Dedicated, meaning that the performance issue still exists for our dedicated customers (and we really shouldn't turn off proxy downloads in this case either).

Mitigation

Short term, we should update documentation to ensure that it's clear to users that proxy downloads must be disabled to take advantage of the performance improvements. However, long term we really should fix proxy downloads so they function with the updated LFS Batch requests.

Edited by 🤖 GitLab Bot 🤖