diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 3cad18dc497b92e2664529536ab7d133fd813396..a14f902db91f67d4af4148d9ae14078601fe1850 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -570,13 +570,15 @@ See the following additional guides: ## Warnings, limitations, and known issues -### Separate buckets required when using Helm +### Use separate buckets -Generally, using the same bucket for your Object Storage is fine to do -for convenience. +Using separate buckets for each data type is the recommended approach for GitLab. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. -However, if you're using or planning to use Helm, separate buckets will -be required as there is a [known limitation with restorations of Helm chart backups](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer). +Helm-based installs require separate buckets to +[handle backup restorations](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) ### S3 API compatibility issues diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index d4d522ab1b84a5b1edcacbb0b1749a4aeb09e8db..e337e449cdd0ed3a41e4074dec4b9ca16c5a2223 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -1940,13 +1940,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -2069,13 +2069,13 @@ On each node perform the following: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2303,20 +2303,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future.
diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 48c72bb930d2b8ad0ab56924c518631be0527a08..d02f7ea66ac61c952c11cede9ea99b83584ee3f9 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -1637,13 +1637,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -1769,13 +1769,13 @@ On each node perform the following: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2002,20 +2002,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future.
diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 9ad6054104a26c5be0342b4257a25562b3c882b0..62f0c993213ef455a5beb82e5deaae849548f5fd 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -668,13 +668,13 @@ On each node perform the following: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ## Uncomment and edit the following options if you have set up NFS ## @@ -920,18 +920,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is -separately configured. We have an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -for improving this, which would allow for one bucket with separate folders. - -Using a single bucket when GitLab is deployed with the Helm chart causes -restoring from a backup to -[not function properly](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer). -Although you may not be using a Helm deployment right now, if you migrate -GitLab to a Helm deployment later, GitLab would still work, but you may not -realize backups aren't working correctly until a critical requirement for -functioning backups is encountered. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future.
diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 175c4318d78dafcd30385d9d4b675110a5dea154..593bfaf7282e3e3aa1c9d539063601cc53ddb578 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -1322,13 +1322,14 @@ To configure the Sidekiq nodes, one each one: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -1489,13 +1490,13 @@ On each node perform the following: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. If you're using [Gitaly with TLS support](#gitaly-tls-support), make sure the @@ -1697,20 +1698,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future.
diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 0b22a2d3602bb0c0237a9e81994dc0ea4c460d53..04ce39046d8f8b9b7ad7712bf74981d1232cfee1 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -1637,13 +1637,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -1769,13 +1769,13 @@ On each node perform the following: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2002,20 +2002,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future.
diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 37d35c299fa425463604c00b0ca2162a3a4fd1ff..37e67b0ab733b75824589c9e55cd02b482ef1d7e 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -1319,13 +1319,13 @@ To configure the Sidekiq nodes, one each one: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -1471,13 +1471,13 @@ On each node perform the following: 'google_project' => '', 'google_json_key_location' => '' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "" ## Uncomment and edit the following options if you have set up NFS ## @@ -1694,20 +1694,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future.