From b371dd2d4220e30184630d317c970fc1ee4cadf1 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Tue, 15 Mar 2022 16:55:58 +0000 Subject: [PATCH 1/3] Add additional Gitaly size note in Ref Arch docs --- .../reference_architectures/10k_users.md | 10 ++++++---- .../reference_architectures/25k_users.md | 10 ++++++---- .../reference_architectures/2k_users.md | 12 +++++++----- .../reference_architectures/3k_users.md | 10 ++++++---- .../reference_architectures/50k_users.md | 10 ++++++---- .../reference_architectures/5k_users.md | 10 ++++++---- 6 files changed, 37 insertions(+), 25 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index f82f23cca7d299..82b2d7dbde8dba 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -1455,10 +1455,12 @@ To configure the Praefect nodes, on each one: ### Configure Gitaly The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have -requirements that are dependent on data, specifically the number of projects -and those projects' sizes. It's recommended that a Gitaly Cluster stores -no more than 5 TB of data on each node. Depending on your -repository storage requirements, you may require additional Gitaly Clusters. +requirements that are dependent on data and load. + +NOTE: +The Reference Architecture specs have been designed with good headroom in mind +but please note that for Gitaly increased specs and / or additional +Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 6d56f67097b4fa..0a981e2cf4441f 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -1459,10 +1459,12 @@ the file of the same name on this server. If this is the first Omnibus node you ### Configure Gitaly The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have -requirements that are dependent on data, specifically the number of projects -and those projects' sizes. It's recommended that a Gitaly Cluster stores -no more than 5 TB of data on each node. Depending on your -repository storage requirements, you may require additional Gitaly Clusters. +requirements that are dependent on data and load. + +NOTE: +The Reference Architecture specs have been designed with good headroom in mind +but please note that for Gitaly increased specs and / or additional +Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 6f6c02c309a5c1..af7db100232774 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -397,11 +397,13 @@ are supported and can be added if needed. ## Configure Gitaly -[Gitaly](../gitaly/index.md) server node requirements are dependent on data, -specifically the number of projects and those projects' sizes. It's recommended -that a Gitaly server node stores no more than 5TB of data. Although this -reference architecture includes a single Gitaly server node, you may require -additional nodes depending on your repository storage requirements. +[Gitaly](../gitaly/index.md) server node requirements are dependent on data size, +specifically the number of projects and those projects' sizes. + +NOTE: +The Reference Architecture specs have been designed with good headroom in mind +but please note that for Gitaly increased specs or switching to Gitaly Cluster +may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index ec6204c21a12aa..c50d7f3b821d35 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -1399,10 +1399,12 @@ the file of the same name on this server. If this is the first Omnibus node you ### Configure Gitaly The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have -requirements that are dependent on data, specifically the number of projects -and those projects' sizes. It's recommended that a Gitaly Cluster stores -no more than 5 TB of data on each node. Depending on your -repository storage requirements, you may require additional Gitaly Clusters. +requirements that are dependent on data and load. + +NOTE: +The Reference Architecture specs have been designed with good headroom in mind +but please note that for Gitaly increased specs and / or additional +Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 9ff4e8e53c4b0f..7f9e866e6a5544 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -1468,10 +1468,12 @@ the file of the same name on this server. If this is the first Omnibus node you ### Configure Gitaly The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have -requirements that are dependent on data, specifically the number of projects -and those projects' sizes. It's recommended that a Gitaly Cluster stores -no more than 5 TB of data on each node. Depending on your -repository storage requirements, you may require additional Gitaly Clusters. +requirements that are dependent on data and load. + +NOTE: +The Reference Architecture specs have been designed with good headroom in mind +but please note that for Gitaly increased specs and / or additional +Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 50b5bbd6dc8fbe..4c854cd1e48b23 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -1397,10 +1397,12 @@ the file of the same name on this server. If this is the first Omnibus node you ### Configure Gitaly The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have -requirements that are dependent on data, specifically the number of projects -and those projects' sizes. It's recommended that a Gitaly Cluster stores -no more than 5 TB of data on each node. Depending on your -repository storage requirements, you may require additional Gitaly Clusters. +requirements that are dependent on data and load. + +NOTE: +The Reference Architecture specs have been designed with good headroom in mind +but please note that for Gitaly increased specs and / or additional +Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs -- GitLab From 0b3ce5e9563218ecfc64450f1283661587ff0ef0 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 16 Mar 2022 15:46:44 +0000 Subject: [PATCH 2/3] Add note about PG downsizing --- doc/administration/reference_architectures/3k_users.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index c50d7f3b821d35..d692749bd15c7a 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -2159,6 +2159,7 @@ but with smaller performance requirements, several modifications can be consider - Reducing the node counts: Some node types do not need consensus and can run with fewer nodes (but more than one for redundancy). This will also lead to reduced performance. - GitLab Rails and Sidekiq: Stateless services don't have a minimum node count. Two are enough for redundancy. - Gitaly and Praefect: A quorum is not strictly necessary. Two Gitaly nodes and two Praefect nodes are enough for redundancy. + - PostgreSQL and PgBouncer: A quorum is not strictly necessary. Two PostgreSQL nodes and two PgBouncer nodes are enough for redundancy. - Running select components in reputable Cloud PaaS solutions: Select components of the GitLab setup can instead be run on Cloud Provider PaaS solutions. By doing this, additional dependent components can also be removed: - PostgreSQL: Can be run on reputable Cloud PaaS solutions such as Google Cloud SQL or Amazon RDS. In this setup, the PgBouncer and Consul nodes are no longer required: - Consul may still be desired if [Prometheus](../monitoring/prometheus/index.md) auto discovery is a requirement, otherwise you would need to [manually add scrape configurations](../monitoring/prometheus/index.md#adding-custom-scrape-configurations) for all nodes. -- GitLab From ec71fb6656bd66668f466851a5e7c1236dfc571f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 18 Mar 2022 13:24:21 +0000 Subject: [PATCH 3/3] Apply 6 suggestion(s) to 6 file(s) --- doc/administration/reference_architectures/10k_users.md | 2 +- doc/administration/reference_architectures/25k_users.md | 2 +- doc/administration/reference_architectures/2k_users.md | 2 +- doc/administration/reference_architectures/3k_users.md | 2 +- doc/administration/reference_architectures/50k_users.md | 2 +- doc/administration/reference_architectures/5k_users.md | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 82b2d7dbde8dba..1fed63c3706f5c 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -1459,7 +1459,7 @@ requirements that are dependent on data and load. NOTE: The Reference Architecture specs have been designed with good headroom in mind -but please note that for Gitaly increased specs and / or additional +but for Gitaly, increased specs or additional Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 0a981e2cf4441f..cabb3c16366ead 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -1463,7 +1463,7 @@ requirements that are dependent on data and load. NOTE: The Reference Architecture specs have been designed with good headroom in mind -but please note that for Gitaly increased specs and / or additional +but for Gitaly, increased specs or additional Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index af7db100232774..f9a7420c0bcaf9 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -402,7 +402,7 @@ specifically the number of projects and those projects' sizes. NOTE: The Reference Architecture specs have been designed with good headroom in mind -but please note that for Gitaly increased specs or switching to Gitaly Cluster +but for Gitaly, increased specs or switching to Gitaly Cluster may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index d692749bd15c7a..081ed9d2be58a9 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -1403,7 +1403,7 @@ requirements that are dependent on data and load. NOTE: The Reference Architecture specs have been designed with good headroom in mind -but please note that for Gitaly increased specs and / or additional +but for Gitaly, increased specs or additional Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 7f9e866e6a5544..84ec20e308ac2b 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -1472,7 +1472,7 @@ requirements that are dependent on data and load. NOTE: The Reference Architecture specs have been designed with good headroom in mind -but please note that for Gitaly increased specs and / or additional +but for Gitaly, increased specs or additional Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 4c854cd1e48b23..a88eb402ecdf09 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -1401,7 +1401,7 @@ requirements that are dependent on data and load. NOTE: The Reference Architecture specs have been designed with good headroom in mind -but please note that for Gitaly increased specs and / or additional +but for Gitaly, increased specs or additional Gitaly Cluster arrays may be required for notably large data sets or load. Due to Gitaly having notable input and output requirements, we strongly -- GitLab