From 4afdd66009a64c5eac9386337e22de2df9265398 Mon Sep 17 00:00:00 2001 From: Katrin Leinweber Date: Fri, 6 Aug 2021 07:38:22 +0200 Subject: [PATCH 1/4] Advise about detecting repo size problems --- .markdownlint.yml | 1 + doc/user/project/repository/reducing_the_repo_size_using_git.md | 2 ++ 2 files changed, 3 insertions(+) diff --git a/.markdownlint.yml b/.markdownlint.yml index 35e8ef119be5c2..ea7492cf2087a6 100644 --- a/.markdownlint.yml +++ b/.markdownlint.yml @@ -49,6 +49,7 @@ "Geo", "Git LFS", "git-annex", + "git-sizer", "Git", "Gitaly", "GitHub", diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 323a2efce76dda..ecb32feff55290 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -13,6 +13,8 @@ Git repositories become larger over time. When large files are added to a Git re - They take up a large amount of storage space on the server. - Git repository storage limits [can be reached](#storage-limits). +Such problems can be detected with [git-sizer](https://github.com/github/git-sizer#getting-started). + Rewriting a repository can remove unwanted history to make the repository smaller. We **recommend [`git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/README.md)** over [`git filter-branch`](https://git-scm.com/docs/git-filter-branch) and -- GitLab From 4b5cf35b31e68808a8c3237fcfc9ed8e24f0f6dc Mon Sep 17 00:00:00 2001 From: Katrin Leinweber Date: Fri, 6 Aug 2021 07:39:20 +0200 Subject: [PATCH 2/4] Advise about ignoring commit messages during rebase --- doc/topics/git/git_rebase.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/topics/git/git_rebase.md b/doc/topics/git/git_rebase.md index 89dbdf54e95bd8..0962b53afd551b 100644 --- a/doc/topics/git/git_rebase.md +++ b/doc/topics/git/git_rebase.md @@ -174,18 +174,18 @@ the operation you want to perform in each commit. To do so, you need to edit the commits in your terminal's text editor. For example, if you're using [Vim](https://www.vim.org/) as the text editor in -a macOS's `ZSH` shell, and you want to **squash** all the three commits +a macOS's `ZSH` shell, and you want to **squash** or **fixup** all the three commits (join them into one): 1. Press i on your keyboard to switch to Vim's editing mode. 1. Navigate with your keyboard arrows to edit the **second** commit keyword - from `pick` to `squash` (or `s`). Do the same to the **third** commit. + from `pick` to `squash`/`fixup` (or `s`/`f`). Do the same to the **third** commit. The first commit should be left **unchanged** (`pick`) as we want to squash the second and third into the first. 1. Press Escape to leave the editing mode. 1. Type `:wq` to "write" (save) and "quit". -1. Git outputs the commit message so you have a chance to edit it: +1. When squashing, Git outputs the commit message so you have a chance to edit it: - All lines starting with `#` are ignored and not included in the commit message. Everything else is included. - To leave it as it is, type `:wq`. To edit the commit message: switch to the -- GitLab From 1c00f58a73b5ddd7c4a19f675f24335f528a4d51 Mon Sep 17 00:00:00 2001 From: Katrin Leinweber Date: Thu, 5 Aug 2021 12:27:05 +0200 Subject: [PATCH 3/4] Document workaround to import a too large repository --- doc/user/project/settings/import_export.md | 56 ++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 32f7fee0287366..592d11f957cc72 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -210,3 +210,59 @@ To help avoid abuse, by default, users are rate limited to: | Import | 6 projects per minute | GitLab.com may have [different settings](../../gitlab_com/index.md#importexport) from the defaults. + +## Troubleshooting + +### Import workaround for large repositories + +[Maximum import size limitations](#importing-the-project) +can prevent an import from being successful. +If changing the import limits is not possible, +the following local workflow can be used to temporarily +reduce therepository size for another import attempt. + +1. Create a temporary working directory from the export: + + ```shell + EXPORT= + + mkdir "$EXPORT" + tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/ + cd "$EXPORT"/ + git clone project.bundle + + # Prevent interference with recreating an importable file later + mv project.bundle ../"$EXPORT"-original.bundle + mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz + ``` + +1. To reduce the repository size, + [identify and remove large files](../repository/reducing_the_repo_size_using_git.md) + or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase) + to reduce the number of commits. + + ```shell + # Reduce the .git/objects/pack/ file size + cd project + git reflog expire --expire=now --all + git gc --prune=now --aggressive + + # Prepare recreating an importable file + git bundle create ../project.bundle main + cd .. + mv project/ ../"$EXPORT"-project + cd .. + + # Recreate an importable file + tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ . + ``` + +1. Import this new, smaller file into GitLab. +1. In a full clone of the original repository, + use `git remote set-url origin && git push --force --all` + to complete the import. +1. Update the imported repository's + [branch protection rules](../protected_branches.md) and + its [default branch](../repository/branches/default.md), and + delete the temporary, `smaller-…` branch, and + the local, temporary data. -- GitLab From 5f4f9f8c9a7576097a26b00cacf7a97566b022ec Mon Sep 17 00:00:00 2001 From: Cynthia Ng Date: Mon, 9 Aug 2021 18:00:39 +0000 Subject: [PATCH 4/4] Apply 4 suggestion(s) to 2 file(s) --- doc/topics/git/git_rebase.md | 4 ++-- doc/user/project/settings/import_export.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/topics/git/git_rebase.md b/doc/topics/git/git_rebase.md index 0962b53afd551b..f515677eacfd2a 100644 --- a/doc/topics/git/git_rebase.md +++ b/doc/topics/git/git_rebase.md @@ -174,13 +174,13 @@ the operation you want to perform in each commit. To do so, you need to edit the commits in your terminal's text editor. For example, if you're using [Vim](https://www.vim.org/) as the text editor in -a macOS's `ZSH` shell, and you want to **squash** or **fixup** all the three commits +a macOS's `ZSH` shell, and you want to `squash` or `fixup` all the three commits (join them into one): 1. Press i on your keyboard to switch to Vim's editing mode. 1. Navigate with your keyboard arrows to edit the **second** commit keyword - from `pick` to `squash`/`fixup` (or `s`/`f`). Do the same to the **third** commit. + from `pick` to `squash` or `fixup` (or `s` or `f`). Do the same to the **third** commit. The first commit should be left **unchanged** (`pick`) as we want to squash the second and third into the first. 1. Press Escape to leave the editing mode. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 592d11f957cc72..cd0fa351e2955d 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -219,7 +219,7 @@ GitLab.com may have [different settings](../../gitlab_com/index.md#importexport) can prevent an import from being successful. If changing the import limits is not possible, the following local workflow can be used to temporarily -reduce therepository size for another import attempt. +reduce the repository size for another import attempt. 1. Create a temporary working directory from the export: @@ -248,7 +248,7 @@ reduce therepository size for another import attempt. git gc --prune=now --aggressive # Prepare recreating an importable file - git bundle create ../project.bundle main + git bundle create ../project.bundle cd .. mv project/ ../"$EXPORT"-project cd .. -- GitLab