From 0af831109470e676c1baf963d9acdbbf32b82103 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 5 Oct 2023 16:22:23 +0100 Subject: [PATCH 01/13] Add more guidance about Ref Arch targets in docs --- .../reference_architectures/10k_users.md | 38 +++++++++-- .../reference_architectures/1k_users.md | 37 +++++++---- .../reference_architectures/25k_users.md | 38 +++++++++-- .../reference_architectures/2k_users.md | 30 +++++++-- .../reference_architectures/3k_users.md | 40 +++++++---- .../reference_architectures/50k_users.md | 38 +++++++++-- .../reference_architectures/5k_users.md | 41 ++++++++---- .../reference_architectures/index.md | 66 ++++++++++++------- 8 files changed, 241 insertions(+), 87 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 16f1f11a6b13e0..d84e86222ce438 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -6,18 +6,21 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 10,000 users **(PREMIUM SELF)** -This page describes GitLab reference architecture for up to 10,000 users. For a -full list of reference architectures, see +This page describes the GitLab reference architecture designed for the typical load of up to 10,000 users +with notable headroom. + +For a full list of reference architectures, see [Available reference architectures](index.md#available-reference-architectures). -> - **Supported users (approximate):** 10,000 +NOTE: +Before deploying this architecture it's recommended to read through the [main documentation](index.md) first, +specifically the [Before you start](index.md#before-you-start) and [Deciding which architecture to use](index.md#deciding-which-architecture-to-use) sections. + +> - **Target load:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)** -> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). +> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use) | Service | Nodes | Configuration | GCP | AWS | |------------------------------------------|-------|-------------------------|------------------|----------------| @@ -144,6 +147,27 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. +## Testing Methodology + +The 10k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 200 RPS +- Web: 20 RPS +- Git (Pull): 20 RPS +- Git (Push): 4 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup components To set up GitLab and its components to accommodate up to 10,000 users: diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index 2f7c8209a4464f..143a12b9fbade0 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -6,24 +6,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 1,000 users **(FREE SELF)** -This page describes GitLab reference architecture for up to 1,000 users. For a -full list of reference architectures, see -[Available reference architectures](index.md#available-reference-architectures). +This page describes the GitLab reference architecture designed for the typical load of up to 1,000 users +with notable headroom (non-HA standalone). -If you are serving up to 1,000 users, and you don't have strict availability -requirements, a [standalone](index.md#standalone-non-ha) single-node solution with -frequent backups is appropriate for -many organizations. +For a full list of reference architectures, see +[Available reference architectures](index.md#available-reference-architectures). -> - **Supported users (approximate):** 1,000 +> - **Target Load:** API: 20 RPS, Web: 2 RPS, Git (Pull): 2 RPS, Git (Push): 1 RPS > - **High Availability:** No. For a highly-available environment, you can > follow a modified [3K reference architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid:** No. For a cloud native hybrid environment, you > can follow a [modified hybrid reference architecture](#cloud-native-hybrid-reference-architecture-with-helm-charts). -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 20 RPS, Web: 2 RPS, Git (Pull): 2 RPS, Git (Push): 1 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)** > - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). | Users | Configuration | GCP | AWS | Azure | @@ -73,6 +67,27 @@ WARNING: **However, if you have [large monorepos](index.md#large-monorepos) (larger than several gigabytes) or [additional workloads](index.md#additional-workloads) these can *significantly* impact the performance of the environment and further adjustments may be required.** If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. +## Testing Methodology + +The 2k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 20 RPS +- Web: 2 RPS +- Git (Pull): 2 RPS +- Git (Push): 1 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup instructions To install GitLab for this default reference architecture, use the standard diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 1745fb55b22cd0..a8a82222dd8719 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -6,18 +6,21 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 25,000 users **(PREMIUM SELF)** -This page describes GitLab reference architecture for up to 25,000 users. For a -full list of reference architectures, see +This page describes the GitLab reference architecture designed for the typical load of up to 25,000 users +with notable headroom. + +For a full list of reference architectures, see [Available reference architectures](index.md#available-reference-architectures). -> - **Supported users (approximate):** 25,000 +NOTE: +Before deploying this architecture it's recommended to read through the [main documentation](index.md) first, +specifically the [Before you start](index.md#before-you-start) and [Deciding which architecture to use](index.md#deciding-which-architecture-to-use) sections. + +> - **Target load:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)** -> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). +> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use) | Service | Nodes | Configuration | GCP | AWS | |------------------------------------------|-------|-------------------------|------------------|--------------| @@ -144,6 +147,27 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. +## Testing Methodology + +The 25k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 500 RPS +- Web: 50 RPS +- Git (Pull): 50 RPS +- Git (Push): 10 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup components To set up GitLab and its components to accommodate up to 25,000 users: diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 6528301ae149cb..0564e13f631817 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -6,18 +6,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 2,000 users **(FREE SELF)** -This page describes GitLab reference architecture for up to 2,000 users. +This page describes the GitLab reference architecture designed for the typical load of up to 2,000 users +with notable headroom (non-HA). + For a full list of reference architectures, see [Available reference architectures](index.md#available-reference-architectures). -> - **Supported users (approximate):** 2,000 +> - **Target Load:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS > - **High Availability:** No. For a highly-available environment, you can > follow a modified [3K reference architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)** > - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). | Service | Nodes | Configuration | GCP | AWS | Azure | @@ -81,6 +80,27 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. +## Testing Methodology + +The 2k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 40 RPS +- Web: 4 RPS +- Git (Pull): 4 RPS +- Git (Push): 1 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup components To set up GitLab and its components to accommodate up to 2,000 users: diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 80e2ed76839135..2b9001b718c015 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -6,27 +6,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 3,000 users **(PREMIUM SELF)** -This GitLab reference architecture can help you deploy GitLab to up to 3,000 -users, and then maintain uptime and access for those users. You can also use -this architecture to provide improved GitLab uptime and availability for fewer -than 3,000 users. For fewer users, reduce the stated node sizes as needed. +This page describes the GitLab reference architecture designed for the typical load of up to 3,000 users +with notable headroom. -If maintaining a high level of uptime for your GitLab environment isn't a -requirement, or if you don't have the expertise to maintain this sort of -environment, we recommend using the non-HA [2,000-user reference architecture](2k_users.md) -for your GitLab installation. If HA is still a requirement, there's several supported -tweaks you can make to this architecture to reduce complexity as detailed here. +This architecture is the smallest one available with HA built in. If you require HA but +have a lower user count or total load the [Supported Modifications for lower user counts](#supported-modifications-for-lower-user-counts-ha) +section details how to reduce this architecture's size while maintaining HA. For a full list of reference architectures, see [Available reference architectures](index.md#available-reference-architectures). -> - **Supported users (approximate):** 3,000 +> - **Target Load:** 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS > - **High Availability:** Yes, although [Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)** > - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). | Service | Nodes | Configuration | GCP | AWS | @@ -149,6 +142,27 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. +## Testing Methodology + +The 3k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 60 RPS +- Web: 6 RPS +- Git (Pull): 6 RPS +- Git (Push): 1 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup components To set up GitLab and its components to accommodate up to 3,000 users: diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index b0ca172893e61b..05d1f5c24481d1 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -6,18 +6,21 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 50,000 users **(PREMIUM SELF)** -This page describes GitLab reference architecture for up to 50,000 users. For a -full list of reference architectures, see +This page describes the GitLab reference architecture designed for the typical load of up to 50,000 users +with notable headroom. + +For a full list of reference architectures, see [Available reference architectures](index.md#available-reference-architectures). -> - **Supported users (approximate):** 50,000 +NOTE: +Before deploying this architecture it's recommended to read through the [main documentation](index.md) first, +specifically the [Before you start](index.md#before-you-start) and [Deciding which architecture to use](index.md#deciding-which-architecture-to-use) sections. + +> - **Target load:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)** -> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). +> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use) | Service | Nodes | Configuration | GCP | AWS | |------------------------------------------|-------|-------------------------|------------------|---------------| @@ -144,6 +147,27 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. +## Testing Methodology + +The 50k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 1000 RPS +- Web: 100 RPS +- Git (Pull): 100 RPS +- Git (Push): 20 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup components To set up GitLab and its components to accommodate up to 50,000 users: diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 5e6387359d8085..2a3505c91bb766 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -6,25 +6,21 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 5,000 users **(PREMIUM SELF)** -This page describes GitLab reference architecture for up to 5,000 users. For a -full list of reference architectures, see +This page describes the GitLab reference architecture designed for the typical load of up to 5,000 users +with notable headroom. + +For a full list of reference architectures, see [Available reference architectures](index.md#available-reference-architectures). NOTE: -This reference architecture is designed to help your organization achieve a -highly-available GitLab deployment. If you do not have the expertise or need to -maintain a highly-available environment, you can have a simpler and less -costly-to-operate environment by using the -[2,000-user reference architecture](2k_users.md). +Before deploying this architecture it's recommended to read through the [main documentation](index.md) first, +specifically the [Before you start](index.md#before-you-start) and [Deciding which architecture to use](index.md#deciding-which-architecture-to-use) sections. -> - **Supported users (approximate):** 5,000 +> - **Target load:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) > - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Validation and test results:** The Quality Engineering team does [regular smoke and performance tests](index.md#validation-and-test-results) to ensure the reference architectures remain compliant -> - **Test requests per second (RPS) rates:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS -> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)** -> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use). +> - **Unsure which Reference Architecture to use?** [Go to this guide for more info](index.md#deciding-which-architecture-to-use) | Service | Nodes | Configuration | GCP | AWS | |-------------------------------------------|-------|-------------------------|-----------------|--------------| @@ -146,6 +142,27 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. +## Testing Methodology + +The 5k architecture is designed to cover a large majority of workflows and is regularly +[smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team +against the following endpoint throughput targets: + +- API: 100 RPS +- Web: 10 RPS +- Git (Pull): 10 RPS +- Git (Push): 2 RPS + +The above targets were selected based on real customer data of total environmental loads corresponding to the user count, +including CI and other workloads along with additional substantial headroom added. + +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). +If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. + +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). + ## Setup components To set up GitLab and its components to accommodate up to 5,000 users: diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index e2fc425b5362b1..ef0320f159ec71 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -12,36 +12,38 @@ GitLab Quality Engineering and Support teams to provide recommended deployments ## Available reference architectures -Depending on your workflow, the following recommended reference architectures -may need to be adapted accordingly. Your workload is influenced by factors -including how active your users are, how much automation you use, mirroring, -and repository/change size. Additionally, the displayed memory values are -provided by [GCP machine types](https://cloud.google.com/compute/docs/machine-resource). -For different cloud vendors, attempt to select options that best match the -provided architecture. +The following Reference Architectures are available as recommended starting points for your environment. + +The architectures are named in terms of user count, which in this case means the architecture is designed against +the _total_ load that typically comes with such a user count along with substantial headroom added to cover most scenarios +such as CI or other automated workloads. + +However, it should be noted that in some cases known heavy scenarios such as [large monorepos](#large-repositories) or notable [additional workloads](#additional-workloads) may require adjustments to be made. + +For each Reference Architecture, the details of what they have been tested against can be found respectively in the `Testing Methodology` section of each page. ### GitLab package (Omnibus) -The following reference architectures, where the GitLab package is used, are available: +Below is a list of GitLab package based architectures: -- [Up to 1,000 users](1k_users.md) -- [Up to 2,000 users](2k_users.md) -- [Up to 3,000 users](3k_users.md) -- [Up to 5,000 users](5k_users.md) -- [Up to 10,000 users](10k_users.md) -- [Up to 25,000 users](25k_users.md) -- [Up to 50,000 users](50k_users.md) +- [Up to 1,000 users](1k_users.md) _API: 20 RPS, Web: 2 RPS, Git (Pull): 2 RPS, Git (Push): 1 RPS_ +- [Up to 2,000 users](2k_users.md) _API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS_ +- [Up to 3,000 users](3k_users.md) _API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS_ +- [Up to 5,000 users](5k_users.md) _API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS_ +- [Up to 10,000 users](10k_users.md) _API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS_ +- [Up to 25,000 users](25k_users.md) _API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS_ +- [Up to 50,000 users](50k_users.md) _API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS_ ### Cloud native hybrid -The following Cloud Native Hybrid reference architectures, where select recommended components can be run in Kubernetes, are available: +Below is a list of Cloud Native Hybrid reference architectures, where select recommended components can be run in Kubernetes: -- [Up to 2,000 users](2k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -- [Up to 3,000 users](3k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -- [Up to 5,000 users](5k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -- [Up to 10,000 users](10k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -- [Up to 25,000 users](25k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -- [Up to 50,000 users](50k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) +- [Up to 2,000 users](2k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS_ +- [Up to 3,000 users](3k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS_ +- [Up to 5,000 users](5k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS_ +- [Up to 10,000 users](10k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS_ +- [Up to 25,000 users](25k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS_ +- [Up to 50,000 users](50k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative)_API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS_ ## Before you start @@ -664,10 +666,24 @@ You should upgrade a Reference Architecture in the same order as you created it. ### Scaling an environment -Scaling a GitLab environment is designed to be as seamless as possible. +Scaling a GitLab environment is designed to be as flexible and seamless as possible. + +This can be done iteratively or wholesale to the next size of architecture depending on your circumstances. +So for example if you see that your Postgres node is maxing out it's resources often you can just increase its +resources accordingly while leaving the rest of the environment as is. + +On the other hand if you're expecting a large increase in users you may elect to scale up the whole environment to the next +size of architecture. + +As a general guidance, as long as the overall design is being followed it's supported for you to scale the environment vertically as required. + +Conversely, if you have robust metrics in place that show the environment is over-provisioned, you can apply the same process for +scaling downwards. You should take an iterative approach when scaling downwards, however, to ensure there are no issues. + +#### Scaling from a non-HA to a HA architecture -In terms of the Reference Architectures, you would look to the next size and adjust accordingly. -Most setups would only need vertical scaling, but there are some specific areas that can be adjusted depending on the setup: +While in most cases vertical scaling is only required to increase an environment's resources if you are moving to a HA environment +there may be some additional steps required as shown below: - If you're scaling from a non-HA environment to an HA environment, various components are recommended to be deployed in their HA forms: - Redis to multi-node Redis w/ Redis Sentinel -- GitLab From e281560083992efea792741c73aadd5ee1dec411 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 5 Oct 2023 17:11:00 +0100 Subject: [PATCH 02/13] Adjust scaling language --- doc/administration/reference_architectures/index.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index ef0320f159ec71..021720da384ffb 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -669,8 +669,7 @@ You should upgrade a Reference Architecture in the same order as you created it. Scaling a GitLab environment is designed to be as flexible and seamless as possible. This can be done iteratively or wholesale to the next size of architecture depending on your circumstances. -So for example if you see that your Postgres node is maxing out it's resources often you can just increase its -resources accordingly while leaving the rest of the environment as is. +So for example if you see that any of your GitLab Rails, Sidekiq, Gitaly, Redis or Postgres nodes are consistently oversaturated you can just increase their resources accordingly while leaving the rest of the environment as is. On the other hand if you're expecting a large increase in users you may elect to scale up the whole environment to the next size of architecture. -- GitLab From 66549838e742bb247d6cbc3d928c1576da56601f Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 12 Oct 2023 12:02:30 +0000 Subject: [PATCH 03/13] Apply 22 suggestion(s) to 7 file(s) --- .../reference_architectures/10k_users.md | 4 ++-- .../reference_architectures/25k_users.md | 4 ++-- .../reference_architectures/2k_users.md | 2 +- .../reference_architectures/3k_users.md | 4 ++-- .../reference_architectures/50k_users.md | 4 ++-- .../reference_architectures/5k_users.md | 4 ++-- .../reference_architectures/index.md | 23 ++++++++++--------- 7 files changed, 23 insertions(+), 22 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index d84e86222ce438..e21ee0c6bd8ed8 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -147,7 +147,7 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. -## Testing Methodology +## Testing methodology The 10k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team @@ -165,7 +165,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index a8a82222dd8719..be4fa6a5fee9ef 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -147,7 +147,7 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. -## Testing Methodology +## Testing methodology The 25k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team @@ -165,7 +165,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 0564e13f631817..374cfb167ba7e0 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -80,7 +80,7 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. -## Testing Methodology +## Testing methodology The 2k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 2b9001b718c015..54c577ab72c336 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -142,7 +142,7 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. -## Testing Methodology +## Testing methodology The 3k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team @@ -160,7 +160,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 05d1f5c24481d1..2db25266cbdac4 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -147,7 +147,7 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. -## Testing Methodology +## Testing methodology The 50k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team @@ -165,7 +165,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 2a3505c91bb766..3f2e583bb61e7f 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -142,7 +142,7 @@ monitor .[#7FFFD4,norank]u--> elb Before starting, see the [requirements](index.md#requirements) for reference architectures. -## Testing Methodology +## Testing methodology The 5k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team @@ -160,7 +160,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 021720da384ffb..894c2005b051e1 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -18,13 +18,13 @@ The architectures are named in terms of user count, which in this case means the the _total_ load that typically comes with such a user count along with substantial headroom added to cover most scenarios such as CI or other automated workloads. -However, it should be noted that in some cases known heavy scenarios such as [large monorepos](#large-repositories) or notable [additional workloads](#additional-workloads) may require adjustments to be made. +However, it should be noted that in some cases, known heavy scenarios such as [large monorepos](#large-repositories) or notable [additional workloads](#additional-workloads) may require adjustments to be made. -For each Reference Architecture, the details of what they have been tested against can be found respectively in the `Testing Methodology` section of each page. +For each Reference Architecture, the details of what they have been tested against can be found respectively in the "Testing Methodology" section of each page. ### GitLab package (Omnibus) -Below is a list of GitLab package based architectures: +Below is a list of Linux package based architectures: - [Up to 1,000 users](1k_users.md) _API: 20 RPS, Web: 2 RPS, Git (Pull): 2 RPS, Git (Push): 1 RPS_ - [Up to 2,000 users](2k_users.md) _API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS_ @@ -43,7 +43,7 @@ Below is a list of Cloud Native Hybrid reference architectures, where select rec - [Up to 5,000 users](5k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS_ - [Up to 10,000 users](10k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS_ - [Up to 25,000 users](25k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS_ -- [Up to 50,000 users](50k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative)_API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS_ +- [Up to 50,000 users](50k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) _API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS_ ## Before you start @@ -669,19 +669,20 @@ You should upgrade a Reference Architecture in the same order as you created it. Scaling a GitLab environment is designed to be as flexible and seamless as possible. This can be done iteratively or wholesale to the next size of architecture depending on your circumstances. -So for example if you see that any of your GitLab Rails, Sidekiq, Gitaly, Redis or Postgres nodes are consistently oversaturated you can just increase their resources accordingly while leaving the rest of the environment as is. +For example, if any of your GitLab Rails, Sidekiq, Gitaly, Redis or PostgreSQL nodes are consistently oversaturated, then increase their resources accordingly while leaving the rest of the environment as is. -On the other hand if you're expecting a large increase in users you may elect to scale up the whole environment to the next + +If expecting a large increase in users, you may elect to scale up the whole environment to the next size of architecture. -As a general guidance, as long as the overall design is being followed it's supported for you to scale the environment vertically as required. +If the overall design is being followed, you can scale the environment vertically as required. -Conversely, if you have robust metrics in place that show the environment is over-provisioned, you can apply the same process for -scaling downwards. You should take an iterative approach when scaling downwards, however, to ensure there are no issues. +If robust metrics are in place that show the environment is over-provisioned, you can apply the same process for +scaling downwards. You should take an iterative approach when scaling downwards to ensure there are no issues. -#### Scaling from a non-HA to a HA architecture +#### Scaling from a non-HA to an HA architecture -While in most cases vertical scaling is only required to increase an environment's resources if you are moving to a HA environment +While in most cases vertical scaling is only required to increase an environment's resources, if you are moving to an HA environment, there may be some additional steps required as shown below: - If you're scaling from a non-HA environment to an HA environment, various components are recommended to be deployed in their HA forms: -- GitLab From d3d2c4d756a29253d390308f03339561543abf87 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 12 Oct 2023 12:13:16 +0000 Subject: [PATCH 04/13] Apply 3 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/1k_users.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index 143a12b9fbade0..8c89eb4cdbd6b8 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -67,9 +67,9 @@ WARNING: **However, if you have [large monorepos](index.md#large-monorepos) (larger than several gigabytes) or [additional workloads](index.md#additional-workloads) these can *significantly* impact the performance of the environment and further adjustments may be required.** If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -## Testing Methodology +## Testing methodology -The 2k architecture is designed to cover a large majority of workflows and is regularly +The 1k architecture is designed to cover a large majority of workflows and is regularly [smoke and performance tested](index.md#validation-and-test-results) by the Quality Engineering team against the following endpoint throughput targets: @@ -85,7 +85,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup instructions -- GitLab From 2e56e8c572f030d449b3707dc6a2acb1bdf4ac6b Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 12 Oct 2023 13:14:30 +0100 Subject: [PATCH 05/13] Fix links --- doc/administration/reference_architectures/10k_users.md | 2 +- doc/administration/reference_architectures/1k_users.md | 2 +- doc/administration/reference_architectures/25k_users.md | 2 +- doc/administration/reference_architectures/2k_users.md | 2 +- doc/administration/reference_architectures/3k_users.md | 2 +- doc/administration/reference_architectures/50k_users.md | 2 +- doc/administration/reference_architectures/5k_users.md | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index e21ee0c6bd8ed8..26e23a735763e2 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -166,7 +166,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index 8c89eb4cdbd6b8..72c7b4774cdf3d 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -86,7 +86,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup instructions diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index be4fa6a5fee9ef..e44d9b7a64049c 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -166,7 +166,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 374cfb167ba7e0..eb6d9132589f9f 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -99,7 +99,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 54c577ab72c336..3995cb0cce1e13 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -161,7 +161,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 2db25266cbdac4..8a52d7dda3f90c 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -166,7 +166,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 3f2e583bb61e7f..96046017d2a1c9 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -161,7 +161,7 @@ or notable [additional workloads](index.md#additional-workloads) these can notab If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. -The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). +The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components -- GitLab From 9de8aaf19d862b440f50bfc770462fc9db6035ba Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 12 Oct 2023 13:16:49 +0100 Subject: [PATCH 06/13] Remove unclear verbiage --- doc/administration/reference_architectures/10k_users.md | 2 +- doc/administration/reference_architectures/1k_users.md | 2 +- doc/administration/reference_architectures/25k_users.md | 2 +- doc/administration/reference_architectures/2k_users.md | 2 +- doc/administration/reference_architectures/3k_users.md | 2 +- doc/administration/reference_architectures/50k_users.md | 2 +- doc/administration/reference_architectures/5k_users.md | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 26e23a735763e2..6b73900be94842 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 10,000 users **(PREMIUM SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 10,000 users +This page describes the GitLab reference architecture designed for the load of up to 10,000 users with notable headroom. For a full list of reference architectures, see diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index 72c7b4774cdf3d..de2f43d6b98a70 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 1,000 users **(FREE SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 1,000 users +This page describes the GitLab reference architecture designed for the load of up to 1,000 users with notable headroom (non-HA standalone). For a full list of reference architectures, see diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index e44d9b7a64049c..1f26b9fedb8099 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 25,000 users **(PREMIUM SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 25,000 users +This page describes the GitLab reference architecture designed for the load of up to 25,000 users with notable headroom. For a full list of reference architectures, see diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index eb6d9132589f9f..ec5b107f74b878 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 2,000 users **(FREE SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 2,000 users +This page describes the GitLab reference architecture designed for the load of up to 2,000 users with notable headroom (non-HA). For a full list of reference architectures, see diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 3995cb0cce1e13..a5493b2ea81c39 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 3,000 users **(PREMIUM SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 3,000 users +This page describes the GitLab reference architecture designed for the load of up to 3,000 users with notable headroom. This architecture is the smallest one available with HA built in. If you require HA but diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 8a52d7dda3f90c..6e4f1872cc3381 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 50,000 users **(PREMIUM SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 50,000 users +This page describes the GitLab reference architecture designed for the load of up to 50,000 users with notable headroom. For a full list of reference architectures, see diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 96046017d2a1c9..03603cad88b61c 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reference architecture: up to 5,000 users **(PREMIUM SELF)** -This page describes the GitLab reference architecture designed for the typical load of up to 5,000 users +This page describes the GitLab reference architecture designed for the load of up to 5,000 users with notable headroom. For a full list of reference architectures, see -- GitLab From d207436c6ddd45c46b2dcfacb6ae4c527cd10192 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 12 Oct 2023 12:17:29 +0000 Subject: [PATCH 07/13] Apply 1 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/2k_users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index ec5b107f74b878..691ee87e97deb5 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -98,7 +98,7 @@ If you have metrics to suggest that you have regularly higher throughput against or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. -Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and it's dataset, which is available for anyone to use. +Testing is done regularly via our [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance) and its dataset, which is available for anyone to use. The results of this testing are [available publicly on the GPT wiki](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest). For more information on our testing strategy [refer to this section of the documentation](index.md#validation-and-test-results). ## Setup components -- GitLab From bc84d24653cbbaf072a718d97859680abb6601f1 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 12 Oct 2023 13:19:33 +0100 Subject: [PATCH 08/13] Lint fix --- doc/administration/reference_architectures/index.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 894c2005b051e1..386c0f39fa310f 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -15,8 +15,7 @@ GitLab Quality Engineering and Support teams to provide recommended deployments The following Reference Architectures are available as recommended starting points for your environment. The architectures are named in terms of user count, which in this case means the architecture is designed against -the _total_ load that typically comes with such a user count along with substantial headroom added to cover most scenarios -such as CI or other automated workloads. +the _total_ load that comes with such a user count based on real data along with substantial headroom added to cover most scenarios such as CI or other automated workloads. However, it should be noted that in some cases, known heavy scenarios such as [large monorepos](#large-repositories) or notable [additional workloads](#additional-workloads) may require adjustments to be made. @@ -671,7 +670,6 @@ Scaling a GitLab environment is designed to be as flexible and seamless as possi This can be done iteratively or wholesale to the next size of architecture depending on your circumstances. For example, if any of your GitLab Rails, Sidekiq, Gitaly, Redis or PostgreSQL nodes are consistently oversaturated, then increase their resources accordingly while leaving the rest of the environment as is. - If expecting a large increase in users, you may elect to scale up the whole environment to the next size of architecture. -- GitLab From 0b03f1649aee8d0654c84de1c8e38a1549adf072 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Thu, 12 Oct 2023 17:18:59 +0100 Subject: [PATCH 09/13] Update links --- doc/administration/reference_architectures/10k_users.md | 2 +- doc/administration/reference_architectures/1k_users.md | 2 +- doc/administration/reference_architectures/25k_users.md | 2 +- doc/administration/reference_architectures/2k_users.md | 2 +- doc/administration/reference_architectures/3k_users.md | 2 +- doc/administration/reference_architectures/50k_users.md | 2 +- doc/administration/reference_architectures/5k_users.md | 2 +- doc/administration/reference_architectures/index.md | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 6b73900be94842..2d42b8a6be31f7 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -161,7 +161,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index de2f43d6b98a70..362da0bd7c6c15 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -81,7 +81,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 1f26b9fedb8099..b3c56c371b93f4 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -161,7 +161,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 691ee87e97deb5..fb8b9d8de45e79 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -94,7 +94,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index a5493b2ea81c39..3de21f21716dc6 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -156,7 +156,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 6e4f1872cc3381..361deaf939d2d0 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -161,7 +161,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 03603cad88b61c..663bf9e5216e5f 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -156,7 +156,7 @@ against the following endpoint throughput targets: The above targets were selected based on real customer data of total environmental loads corresponding to the user count, including CI and other workloads along with additional substantial headroom added. -If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-repositories) +If you have metrics to suggest that you have regularly higher throughput against the above endpoint targets, [large monorepos](index.md#large-monorepos) or notable [additional workloads](index.md#additional-workloads) these can notably impact the performance environment and [further adjustments may be required](index.md#scaling-an-environment). If this applies to you, we strongly recommended referring to the linked documentation as well as reaching out to your [Customer Success Manager](https://handbook.gitlab.com/job-families/sales/customer-success-management/) or our [Support team](https://about.gitlab.com/support/) for further guidance. diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 386c0f39fa310f..5c3d1d7194be24 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -17,7 +17,7 @@ The following Reference Architectures are available as recommended starting poin The architectures are named in terms of user count, which in this case means the architecture is designed against the _total_ load that comes with such a user count based on real data along with substantial headroom added to cover most scenarios such as CI or other automated workloads. -However, it should be noted that in some cases, known heavy scenarios such as [large monorepos](#large-repositories) or notable [additional workloads](#additional-workloads) may require adjustments to be made. +However, it should be noted that in some cases, known heavy scenarios such as [large monorepos](#large-monorepos) or notable [additional workloads](#additional-workloads) may require adjustments to be made. For each Reference Architecture, the details of what they have been tested against can be found respectively in the "Testing Methodology" section of each page. -- GitLab From 649472c0bd3e60ddc371cd8a18a0d0ca18f1669f Mon Sep 17 00:00:00 2001 From: Grant Young Date: Fri, 13 Oct 2023 16:57:01 +0100 Subject: [PATCH 10/13] Update Decision Tree to include methodology --- .../reference_architectures/index.md | 20 ++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 5c3d1d7194be24..34fd04e5265de1 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -64,6 +64,18 @@ As a general guide, **the more performant and/or resilient you want your environ This section explains the designs you can choose from. It begins with the least complexity, goes to the most, and ends with a decision tree. +### Expected Load (RPS) + +The first thing to check is what the expected load is your environment would be expected to serve. + +While the Reference Architectures have been designed with substantial headroom by default it's recommended to also check the +load of what each has been tested against as detailed in the "Testing Methodology" section and in turn compare those values with +what you are either already seeing on your existing GitLab instance / equivalent tooling to help select the right Reference Architecture +size. + +Load is given in terms of Requests per Section (RPS) for each endpoint type (API, Web, Git). This information on your existing infrastructure +can typically be surfaced by most reputable monitoring solutions or in some other ways such as load balancer metrics. + ### Standalone (non-HA) For environments serving 2,000 or fewer users, we generally recommend a standalone approach by deploying a non-highly available single or multi-node environment. With this approach, you can employ strategies such as [automated backups](../../administration/backup_restore/backup_gitlab.md#configuring-cron-to-make-daily-backups) for recovery to provide a good level of RPO / RTO while avoiding the complexities that come with HA. @@ -145,10 +157,11 @@ Below you can find the above guidance in the form of a decision tree. It's recom ```mermaid %%{init: { 'theme': 'base' } }%% graph TD - L1A(What Reference Architecture should I use?) + L0A(What Reference Architecture should I use?) + L1A(What is your expected load?) - L2A(3,000 users or more?) - L2B(2,000 users or less?) + L2A("Equivalent to 3,000 users or more?") + L2B("Equivalent to 2,000 users or less?") L3A("Do you need HA?
(or Zero-Downtime Upgrades)") L3B[Do you have experience with
and want additional resilience
with select components in Kubernetes?] @@ -158,6 +171,7 @@ graph TD L4C>Recommendation

Cloud Native Hybrid architecture
closest to user count] L4D>"Recommendation

Standalone 1K or 2K
architecture with Backups"] + L0A --> L1A L1A --> L2A L1A --> L2B L2A -->|Yes| L3B -- GitLab From 563b47e838429daebb14cbd7e779fb8815a96175 Mon Sep 17 00:00:00 2001 From: Nailia Iskhakova Date: Mon, 16 Oct 2023 09:36:38 +0000 Subject: [PATCH 11/13] Apply 1 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 34fd04e5265de1..589de913ae99f5 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -158,7 +158,7 @@ Below you can find the above guidance in the form of a decision tree. It's recom %%{init: { 'theme': 'base' } }%% graph TD L0A(What Reference Architecture should I use?) - L1A(What is your expected load?) + L1A(What is your expected load?) L2A("Equivalent to 3,000 users or more?") L2B("Equivalent to 2,000 users or less?") -- GitLab From 24d5eeab4809f1b5957953d718f53033b0f35710 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Mon, 16 Oct 2023 10:52:03 +0100 Subject: [PATCH 12/13] Update text in new section --- doc/administration/reference_architectures/index.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 589de913ae99f5..1a93c7f14eb6fa 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -68,13 +68,14 @@ This section explains the designs you can choose from. It begins with the least The first thing to check is what the expected load is your environment would be expected to serve. -While the Reference Architectures have been designed with substantial headroom by default it's recommended to also check the -load of what each has been tested against as detailed in the "Testing Methodology" section and in turn compare those values with -what you are either already seeing on your existing GitLab instance / equivalent tooling to help select the right Reference Architecture +The Reference Architectures have been designed with substantial headroom by default, but it's recommended to also check the +load of what each architecture has been tested against under the "Testing Methodology" section found on each page, +comparing those values with what load you are expecting against your existing GitLab environment to help select the right Reference Architecture size. Load is given in terms of Requests per Section (RPS) for each endpoint type (API, Web, Git). This information on your existing infrastructure -can typically be surfaced by most reputable monitoring solutions or in some other ways such as load balancer metrics. +can typically be surfaced by most reputable monitoring solutions or in some other ways such as load balancer metrics. For example, on existing GitLab environments, +[Prometheus metrics](../monitoring/prometheus/gitlab_metrics.md) such as `gitlab_transaction_duration_seconds` can be used to see this data. ### Standalone (non-HA) -- GitLab From ca102700901042f1b436b41b2058881eacfe7fb1 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 18 Oct 2023 11:15:08 +0100 Subject: [PATCH 13/13] Add links to HA migration docs --- doc/administration/reference_architectures/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 1a93c7f14eb6fa..fcbfaf46009747 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -699,9 +699,9 @@ While in most cases vertical scaling is only required to increase an environment there may be some additional steps required as shown below: - If you're scaling from a non-HA environment to an HA environment, various components are recommended to be deployed in their HA forms: - - Redis to multi-node Redis w/ Redis Sentinel - - Postgres to multi-node Postgres w/ Consul + PgBouncer - - Gitaly to Gitaly Cluster w/ Praefect + - [Redis to multi-node Redis w/ Redis Sentinel](../redis/replication_and_failover.md#switching-from-an-existing-single-machine-installation) + - [Postgres to multi-node Postgres w/ Consul + PgBouncer](../postgresql/moving.md) + - [Gitaly to Gitaly Cluster w/ Praefect](../gitaly/index.md#migrate-to-gitaly-cluster) - From 10k users and higher, Redis is recommended to be split into multiple HA servers as it's single threaded. Conversely, if you have robust metrics in place that show the environment is over-provisioned, you can apply the same process for -- GitLab