From bfd5e5783a86c8b5a83b94013f59eccf014423a3 Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Thu, 21 Jan 2021 14:56:13 -0500 Subject: [PATCH 1/9] Add docs on Cloud Native deplyoment for 10k users Adds a section to the 10k users reference architecture documentation that outlines general guidelines for deploying Kubernetes-eligible GitLab components in a cloud native environment. --- .../reference_architectures/10k_users.md | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index afa53b5efa8232..f7d09fcd91f2e5 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2028,3 +2028,57 @@ See the [troubleshooting documentation](troubleshooting.md). Back to setup components + +## Cloud Native Deployment (optional) + +For users looking to leverage the benefits of running workloads in a cloud native environment, +the information below outlines our recommendations for running eligible GitLab components in Kubernetes. + +We only recommend deploying the Sidekiq and Webservice components in Kubernetes, +keeping the remaining components on traditional servers as documented above. + +### Cluster topology + +As a starting point for reference, the recommended infrastructure configuration for the cluster is below. +Note that this example references Google Cloud's Kubernetes Engine (GKE) and associated machine types, but the +memory and CPU totals should translate to most systems. + +- 2x `n1-standard-4` nodes for non-GitLab-related resources, including Grafana, NGINX, and Prometheus +- 4x `n1-standard-4` nodes for GitLab Sidekiq-related resources +- 4x `n1-highcpu-32` nodes for GitLab Webservice-related resources + +### Resource usage settings + +In this section, formulas are provided to help calculate how many pods can be deployed within resource constraints. +The [ref/10k example values file](https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/examples/ref/10k.yaml) +documents the relevant configuration to provide to the Helm Chart for this reference architecture tier. + +#### Sidekiq + +Generally, each Sidekiq pod should have about 1 vCPU and 2 GB of memory. + +Given the network topology above, the four Sidekiq nodes +in the cluster total to 16 vCPUs and 60 GB of memory, excluding the overhead of Kubernetes components. Once those components +are accounted for, there is around 15.5 vCPUs and 50 GB of memory allocatable to workloads. + +With these constraints identified, the available resources allow us to deploy up to 16 Sidekiq pods. If more pods are needed, the +available resources must expand approximately at the 1 vCPU to 2 GB of memory ratio for each pod added. + +#### Webservice + +Generally, each Webservice pod should have about 1 vCPU and 1.25 GB of memory _per worker_. +Note that the default number of worker processes is currently 2. + +Given the network topology above, the four Webservice nodes +in the cluster total to 128 vCPUs and 128 GB of memory, excluding the overhead of Kubernetes components. Once those components +are accounted for, there is around 127.5 vCPUs and 104 GB of memory allocatable to workloads. + +With these constraints identified, the available resources allow us to deploy up to 28 Webservice pods. If more pods are needed, the +available resources must expand approximately at the 1 vCPU to 1.25 GB of memory _per worker_ ratio for each pod added. + +#### Further reading + +For further documentation on resource usage, see the following documentation: + +- [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources) +- [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources) -- GitLab From 2ff883b863af484c968e060ce2266a9e634c025e Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Thu, 21 Jan 2021 15:06:10 -0500 Subject: [PATCH 2/9] Move information into table format Adds a table to collect information for better readibility. --- .../reference_architectures/10k_users.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index f7d09fcd91f2e5..af073120e2d111 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2043,9 +2043,15 @@ As a starting point for reference, the recommended infrastructure configuration Note that this example references Google Cloud's Kubernetes Engine (GKE) and associated machine types, but the memory and CPU totals should translate to most systems. -- 2x `n1-standard-4` nodes for non-GitLab-related resources, including Grafana, NGINX, and Prometheus -- 4x `n1-standard-4` nodes for GitLab Sidekiq-related resources -- 4x `n1-highcpu-32` nodes for GitLab Webservice-related resources +Machine count | Machine type | Allocatable vCPUs* | Allocatable memory (GB)* | Purpose +-|-|-|-|- +2 | `n1-standard-4` | 15.5 | 50 | Non-GitLab resources, including Grafana, NGINX, and Prometheus +4 | `n1-standard-4` | 15.5 | 50 | GitLab Sidekiq pods +4 | `n1-highcpu-32` | 128 | 128 | GitLab Webservice pods + +Note that "allocatable" in this table refers to the amount of resources available to +be used by workloads deployed to Kubernetes after accounting for the overhead of +running Kubernetes itself. ### Resource usage settings @@ -2057,21 +2063,15 @@ documents the relevant configuration to provide to the Helm Chart for this refer Generally, each Sidekiq pod should have about 1 vCPU and 2 GB of memory. -Given the network topology above, the four Sidekiq nodes -in the cluster total to 16 vCPUs and 60 GB of memory, excluding the overhead of Kubernetes components. Once those components -are accounted for, there is around 15.5 vCPUs and 50 GB of memory allocatable to workloads. - -With these constraints identified, the available resources allow us to deploy up to 16 Sidekiq pods. If more pods are needed, the +Given the available resources in the table above, up to 16 Sidekiq pods can be deployed. If more pods are needed, the available resources must expand approximately at the 1 vCPU to 2 GB of memory ratio for each pod added. #### Webservice Generally, each Webservice pod should have about 1 vCPU and 1.25 GB of memory _per worker_. Note that the default number of worker processes is currently 2. - -Given the network topology above, the four Webservice nodes -in the cluster total to 128 vCPUs and 128 GB of memory, excluding the overhead of Kubernetes components. Once those components -are accounted for, there is around 127.5 vCPUs and 104 GB of memory allocatable to workloads. +This means that each Webservice pod should have about 2 vCPUs and 2.5 GB of memory with the default +number of worker processes configured. With these constraints identified, the available resources allow us to deploy up to 28 Webservice pods. If more pods are needed, the available resources must expand approximately at the 1 vCPU to 1.25 GB of memory _per worker_ ratio for each pod added. -- GitLab From 71e3f6ccf49733e05659b59c45edccd475d1d438 Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Thu, 21 Jan 2021 15:07:12 -0500 Subject: [PATCH 3/9] Set resource ratios to display in bold Displays the resource ratios and pod counts in bold so they're easier to find when scanning. --- doc/administration/reference_architectures/10k_users.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index af073120e2d111..15bfdb9fed2f30 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2061,19 +2061,19 @@ documents the relevant configuration to provide to the Helm Chart for this refer #### Sidekiq -Generally, each Sidekiq pod should have about 1 vCPU and 2 GB of memory. +Generally, each Sidekiq pod should have about **1 vCPU and 2 GB of memory**. -Given the available resources in the table above, up to 16 Sidekiq pods can be deployed. If more pods are needed, the +Given the available resources in the table above, up to **16** Sidekiq pods can be deployed. If more pods are needed, the available resources must expand approximately at the 1 vCPU to 2 GB of memory ratio for each pod added. #### Webservice -Generally, each Webservice pod should have about 1 vCPU and 1.25 GB of memory _per worker_. +Generally, each Webservice pod should have about **1 vCPU and 1.25 GB of memory _per worker_**. Note that the default number of worker processes is currently 2. This means that each Webservice pod should have about 2 vCPUs and 2.5 GB of memory with the default number of worker processes configured. -With these constraints identified, the available resources allow us to deploy up to 28 Webservice pods. If more pods are needed, the +With these constraints identified, the available resources allow us to deploy up to **28** Webservice pods. If more pods are needed, the available resources must expand approximately at the 1 vCPU to 1.25 GB of memory _per worker_ ratio for each pod added. #### Further reading -- GitLab From 73acf283e6335e2705476ad429590e8a9ea25509 Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Thu, 21 Jan 2021 15:17:26 -0500 Subject: [PATCH 4/9] Remove asterisks from table column titles Removes the asterisks from the table column titles. They were previously meant to indicate a note below the table, but it throws an error in `markdownlint` and isn't strictly necessary. --- doc/administration/reference_architectures/10k_users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 15bfdb9fed2f30..a338dc91822219 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2043,7 +2043,7 @@ As a starting point for reference, the recommended infrastructure configuration Note that this example references Google Cloud's Kubernetes Engine (GKE) and associated machine types, but the memory and CPU totals should translate to most systems. -Machine count | Machine type | Allocatable vCPUs* | Allocatable memory (GB)* | Purpose +Machine count | Machine type | Allocatable vCPUs | Allocatable memory (GB) | Purpose -|-|-|-|- 2 | `n1-standard-4` | 15.5 | 50 | Non-GitLab resources, including Grafana, NGINX, and Prometheus 4 | `n1-standard-4` | 15.5 | 50 | GitLab Sidekiq pods -- GitLab From 6ccba66dfa328b81075a2c28f545b4e66080029c Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Mon, 25 Jan 2021 14:26:06 -0500 Subject: [PATCH 5/9] Fix allocatable vCPU/memory values - Assume .125 vCPU and 2.5 GB of memory allocated to Kubernetes/overhead for each node - Use these values to adjust the allocatable values for each machine type - Fix the total vCPU/memory columns for the 2x n1-standard-4 nodes (these were calculated for 4x nodes by accident) --- doc/administration/reference_architectures/10k_users.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index a338dc91822219..e4af0932fd1d7e 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2045,9 +2045,9 @@ memory and CPU totals should translate to most systems. Machine count | Machine type | Allocatable vCPUs | Allocatable memory (GB) | Purpose -|-|-|-|- -2 | `n1-standard-4` | 15.5 | 50 | Non-GitLab resources, including Grafana, NGINX, and Prometheus -4 | `n1-standard-4` | 15.5 | 50 | GitLab Sidekiq pods -4 | `n1-highcpu-32` | 128 | 128 | GitLab Webservice pods +2 | `n1-standard-4` | 7.75 | 25 | Non-GitLab resources, including Grafana, NGINX, and Prometheus +4 | `n1-standard-4` | 15.5 | 50 | GitLab Sidekiq pods +4 | `n1-highcpu-32` | 127.5 | 118 | GitLab Webservice pods Note that "allocatable" in this table refers to the amount of resources available to be used by workloads deployed to Kubernetes after accounting for the overhead of -- GitLab From dab26c1781a5fbe99bf2e758be8e6250fe6015a7 Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Thu, 28 Jan 2021 16:55:29 +0000 Subject: [PATCH 6/9] Apply 10 suggestion(s) to 1 file(s) --- .../reference_architectures/10k_users.md | 49 ++++++++++--------- 1 file changed, 27 insertions(+), 22 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index e4af0932fd1d7e..cbdc61813bca72 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2031,17 +2031,19 @@ See the [troubleshooting documentation](troubleshooting.md). ## Cloud Native Deployment (optional) -For users looking to leverage the benefits of running workloads in a cloud native environment, -the information below outlines our recommendations for running eligible GitLab components in Kubernetes. +Hybrid installations leverage the benefits of both cloud native and traditional +deployments. We recommend shifting the Sidekiq and Webservice components into +Kubernetes to reap cloud native workload management benefits while the others +are deployed using the traditional server method already described. -We only recommend deploying the Sidekiq and Webservice components in Kubernetes, -keeping the remaining components on traditional servers as documented above. +The following sections detail this hybrid approach. ### Cluster topology -As a starting point for reference, the recommended infrastructure configuration for the cluster is below. -Note that this example references Google Cloud's Kubernetes Engine (GKE) and associated machine types, but the -memory and CPU totals should translate to most systems. +The following table provides a starting point for hybrid +deployment infrastructure. The recommendations use Google Cloud's Kubernetes Engine (GKE) +and associated machine types, but the memory and CPU requirements should +translate to most other providers. Machine count | Machine type | Allocatable vCPUs | Allocatable memory (GB) | Purpose -|-|-|-|- @@ -2049,32 +2051,35 @@ Machine count | Machine type | Allocatable vCPUs | Allocatable memory (GB) | Pur 4 | `n1-standard-4` | 15.5 | 50 | GitLab Sidekiq pods 4 | `n1-highcpu-32` | 127.5 | 118 | GitLab Webservice pods -Note that "allocatable" in this table refers to the amount of resources available to -be used by workloads deployed to Kubernetes after accounting for the overhead of -running Kubernetes itself. +NOTE: +"Allocatable" in this table refers to the amount of resources available to workloads deployed in Kubernetes **_after_** accounting for the overhead of running Kubernetes itself. + ### Resource usage settings -In this section, formulas are provided to help calculate how many pods can be deployed within resource constraints. -The [ref/10k example values file](https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/examples/ref/10k.yaml) -documents the relevant configuration to provide to the Helm Chart for this reference architecture tier. +The following formulas help when calculating how many pods may be deployed within resource constraints. +The [10k reference architecture example values file](https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/examples/ref/10k.yaml) +documents how to apply the calculated configuration to the Helm Chart. #### Sidekiq -Generally, each Sidekiq pod should have about **1 vCPU and 2 GB of memory**. +Sidekiq pods should generally have **1 vCPU and 2 GB of memory**. -Given the available resources in the table above, up to **16** Sidekiq pods can be deployed. If more pods are needed, the -available resources must expand approximately at the 1 vCPU to 2 GB of memory ratio for each pod added. +[The provided starting point](#cluster-topology) allows the deployment of up to +**16** Sidekiq pods. Expand available resources using the 1vCPU to 2GB memory +ratio for each additional pod. #### Webservice -Generally, each Webservice pod should have about **1 vCPU and 1.25 GB of memory _per worker_**. -Note that the default number of worker processes is currently 2. -This means that each Webservice pod should have about 2 vCPUs and 2.5 GB of memory with the default -number of worker processes configured. +Webservice pods typically need about **1 vCPU and 1.25 GB of memory _per worker_**. +NOTE: +Each Webservice pod will consume roughly 2 vCPUs and 2.5 GB of memory using +the [recommended topology](#cluster-topology) because two worker processes +are created by default. -With these constraints identified, the available resources allow us to deploy up to **28** Webservice pods. If more pods are needed, the -available resources must expand approximately at the 1 vCPU to 1.25 GB of memory _per worker_ ratio for each pod added. +The [provided recommendations](#cluster-topology) allow the deployment of up to **28** +Webservice pods. Expand available resources using the ratio of 1 vCPU to 1.25 GB of memory +**_per each worker process_** for each additional Webservice pod. #### Further reading -- GitLab From 7e60a2d0d1d3aa2e4b3eb2d2dd8d333ef80ee265 Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Thu, 28 Jan 2021 12:08:15 -0500 Subject: [PATCH 7/9] Remove duplicate empty line --- doc/administration/reference_architectures/10k_users.md | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index cbdc61813bca72..f7758311d9b6d0 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2054,7 +2054,6 @@ Machine count | Machine type | Allocatable vCPUs | Allocatable memory (GB) | Pur NOTE: "Allocatable" in this table refers to the amount of resources available to workloads deployed in Kubernetes **_after_** accounting for the overhead of running Kubernetes itself. - ### Resource usage settings The following formulas help when calculating how many pods may be deployed within resource constraints. -- GitLab From e88d48735e4488d481032bb1d71ee019e3b5bdfc Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Mon, 8 Feb 2021 16:10:11 +0000 Subject: [PATCH 8/9] Apply 8 suggestion(s) to 1 file(s) --- .../reference_architectures/10k_users.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index f7758311d9b6d0..fb7f03c4cbc38a 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2051,8 +2051,7 @@ Machine count | Machine type | Allocatable vCPUs | Allocatable memory (GB) | Pur 4 | `n1-standard-4` | 15.5 | 50 | GitLab Sidekiq pods 4 | `n1-highcpu-32` | 127.5 | 118 | GitLab Webservice pods -NOTE: -"Allocatable" in this table refers to the amount of resources available to workloads deployed in Kubernetes **_after_** accounting for the overhead of running Kubernetes itself. +"Allocatable" in this table refers to the amount of resources available to workloads deployed in Kubernetes _after_ accounting for the overhead of running Kubernetes itself. ### Resource usage settings @@ -2062,23 +2061,22 @@ documents how to apply the calculated configuration to the Helm Chart. #### Sidekiq -Sidekiq pods should generally have **1 vCPU and 2 GB of memory**. +Sidekiq pods should generally have 1 vCPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -**16** Sidekiq pods. Expand available resources using the 1vCPU to 2GB memory +16 Sidekiq pods. Expand available resources using the 1vCPU to 2GB memory ratio for each additional pod. #### Webservice -Webservice pods typically need about **1 vCPU and 1.25 GB of memory _per worker_**. -NOTE: +Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. Each Webservice pod will consume roughly 2 vCPUs and 2.5 GB of memory using the [recommended topology](#cluster-topology) because two worker processes are created by default. -The [provided recommendations](#cluster-topology) allow the deployment of up to **28** +The [provided recommendations](#cluster-topology) allow the deployment of up to 28 Webservice pods. Expand available resources using the ratio of 1 vCPU to 1.25 GB of memory -**_per each worker process_** for each additional Webservice pod. +_per each worker process_ for each additional Webservice pod. #### Further reading -- GitLab From 5fc3f90b7f5029d706b18385c81aa14e02040ff3 Mon Sep 17 00:00:00 2001 From: Mitchell Nielsen Date: Mon, 8 Feb 2021 16:19:29 +0000 Subject: [PATCH 9/9] Apply 3 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/10k_users.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index fb7f03c4cbc38a..e80e1e1caba1c8 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2067,6 +2067,8 @@ Sidekiq pods should generally have 1 vCPU and 2 GB of memory. 16 Sidekiq pods. Expand available resources using the 1vCPU to 2GB memory ratio for each additional pod. +For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). + #### Webservice Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. @@ -2078,9 +2080,4 @@ The [provided recommendations](#cluster-topology) allow the deployment of up to Webservice pods. Expand available resources using the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. -#### Further reading - -For further documentation on resource usage, see the following documentation: - -- [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources) -- [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources) +For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). -- GitLab