gitlab:lfs:migrate fails with "Killed" message (OOM)

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Label this issue
• Close this issue

Summary

When running gitlab:lfs:migrate the script hits a repository with huge LGS files (more than 16GB in one object) the script just exists with a "Killed" message.

gitlab-rake gitlab:lfs:migrate VERBOSE=1
I, 2021-12-22T10:46:24.131496 #8482  INFO – : Starting transfer of LFS files to object storage
Killed

Steps to reproduce

1. Configure a gitlab instance with local storage for LFS
2. create a repo with very large LFS file ( a file larger than the memory available in the instance that will do the migration)
3. run migrate command

gitlab-rake gitlab:lfs:migrate VERBOSE=1
I, 2021-12-22T10:46:24.131496 #8482  INFO – : Starting transfer of LFS files to object storage
Killed

What is the current bug behavior?

The message "Killed" appears in the console. After more investigation, it looks like the script is killed by linux Out Of Memory mecanism.

We think that the script must be loading the whole file into memory but could not find the actual code path that does this.

What is the expected correct behavior?

Thee script should just use the file system to store the files and not load everything in memory.

Workaround

Our current workaround is to just use a 32GB memory VM to do the migration, we shouldn't have to spawn such huge VM for this simple migration, and also this workaround is obviously not scalable to larger files.

#/label typebug