From 08c055d624e35eb946788f1920b59f96d5e28021 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 22 Dec 2021 14:37:51 +0000 Subject: [PATCH 1/6] Further update Testing details on Ref Arch docs Table is expanded and clearer --- .../reference_architectures/index.md | 29 +++++++++---------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 6bf35ba6e22ffd..79e7f8416be50b 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -208,19 +208,16 @@ Note the following about the testing process: - We aim to have a "test smart" approach where architectures tested have a good range that can also apply to others. Testing focuses on 10k Omnibus on GCP as the testing has shown this is a good bellwether for the other architectures and cloud providers as well as Cloud Native Hybrids. - Testing is done publicly and all results are shared. -Τhe following table details the testing done against the reference architectures along with the frequency and results. - -| Reference Architecture | Tests Run1 | -|------------------------|----------------------------------------------------------------------------------------------------------------------| -| 1k | [Omnibus - Daily (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)2 | -| 2k | [Omnibus - Daily (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)2 | -| 3k | [Omnibus - Weekly (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)2 | -| 5k | [Omnibus - Weekly (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)2 | -| 10k | [Omnibus - Daily (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)2
[Omnibus - Ad-Hoc (GCP, AWS, Azure)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)

[Cloud Native Hybrid - Ad-Hoc (GCP, AWS)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | -| 25k | [Omnibus - Weekly (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)2
[Omnibus - Ad-Hoc (Azure)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | -| 50k | [Omnibus - Weekly (GCP)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)2
[Omnibus - Ad-Hoc (AWS)](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - -Note that: - -1. The list above is non exhaustive. Additional testing is continuously evaluated and iterated on, and the table is updated regularly. -1. The Omnibus reference architectures are VM-based only and testing has shown that they perform similarly on equivalently specced hardware regardless of Cloud Provider or if run on premises. +Τhe following table details the testing done against the reference architectures along with the frequency and results. Note that this list above is non exhaustive - Additional testing is continuously evaluated and iterated on, and the table will be updated accordingly. + +| Reference architecture size | Bare-Metal | GCP | AWS | Azure | +|-----------------------------|------------|-----|-----|-------| +| 1k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)1 | - | - | - | +| 2k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)1 | - | - | - | +| 3k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)1 | - | - | - | +| 5k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)1 | - | - | - | +| 10k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)1
| [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) | +| 25k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)1 | - | - | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | +| 50k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)1 | - | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - | + +1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [GitLab Omnibus](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). -- GitLab From 91750736b2cf0e05e1260fb6585b3aa0b2b995cb Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 22 Dec 2021 15:24:07 +0000 Subject: [PATCH 2/6] Further updates to table --- doc/administration/reference_architectures/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 79e7f8416be50b..e2a64d77d861e3 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -212,8 +212,8 @@ Note the following about the testing process: | Reference architecture size | Bare-Metal | GCP | AWS | Azure | |-----------------------------|------------|-----|-----|-------| -| 1k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)1 | - | - | - | -| 2k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)1 | - | - | - | +| 1k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)1 | - | - | - | +| 2k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)1 | - | - | - | | 3k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)1 | - | - | - | | 5k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)1 | - | - | - | | 10k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)1
| [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) | -- GitLab From 7bc28743fe0c901a6642e800463685f98c2ea1d9 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 22 Dec 2021 15:38:48 +0000 Subject: [PATCH 3/6] Further tweaks to testing table --- .../reference_architectures/index.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index e2a64d77d861e3..6e50632ac3a027 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -212,12 +212,12 @@ Note the following about the testing process: | Reference architecture size | Bare-Metal | GCP | AWS | Azure | |-----------------------------|------------|-----|-----|-------| -| 1k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)1 | - | - | - | -| 2k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)1 | - | - | - | -| 3k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)1 | - | - | - | -| 5k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)1 | - | - | - | -| 10k | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)1
| [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) | -| 25k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)1 | - | - | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | -| 50k | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)1 | - | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - | +| 1k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)1 | - | - | +| 2k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)1 | - | - | +| 3k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)1 | - | - | +| 5k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)1 | - | - | +| 10k | Refer to GCP1 | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)1
[Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k)
[Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) | +| 25k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)1 | - | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | +| 50k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)1 | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - | 1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [GitLab Omnibus](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). -- GitLab From c01c695f0a123db914d3cfda6744bb3c084853c4 Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 22 Dec 2021 16:03:04 +0000 Subject: [PATCH 4/6] Correct name for Omnibus GitLab --- doc/administration/reference_architectures/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 6e50632ac3a027..5f7249a3f60a1a 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -220,4 +220,4 @@ Note the following about the testing process: | 25k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)1 | - | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | | 50k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)1 | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - | -1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [GitLab Omnibus](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). +1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [Omnibus GitLab](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). -- GitLab From afeaa7cfdbc25c6bce3911c337631110d3bcec7a Mon Sep 17 00:00:00 2001 From: Grant Young Date: Wed, 22 Dec 2021 16:30:33 +0000 Subject: [PATCH 5/6] Tweak table header to wrap --- doc/administration/reference_architectures/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 5f7249a3f60a1a..c99e09c51f643f 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -210,7 +210,7 @@ Note the following about the testing process: Τhe following table details the testing done against the reference architectures along with the frequency and results. Note that this list above is non exhaustive - Additional testing is continuously evaluated and iterated on, and the table will be updated accordingly. -| Reference architecture size | Bare-Metal | GCP | AWS | Azure | +| Reference
Architecture
Size | Bare-Metal | GCP | AWS | Azure | |-----------------------------|------------|-----|-----|-------| | 1k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)1 | - | - | | 2k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)1 | - | - | -- GitLab From 3220259ff41bd714b39c18e1767284e31855471e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 28 Dec 2021 13:37:45 +0000 Subject: [PATCH 6/6] Apply 2 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index c99e09c51f643f..81f07f304bc3a3 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -208,7 +208,7 @@ Note the following about the testing process: - We aim to have a "test smart" approach where architectures tested have a good range that can also apply to others. Testing focuses on 10k Omnibus on GCP as the testing has shown this is a good bellwether for the other architectures and cloud providers as well as Cloud Native Hybrids. - Testing is done publicly and all results are shared. -Τhe following table details the testing done against the reference architectures along with the frequency and results. Note that this list above is non exhaustive - Additional testing is continuously evaluated and iterated on, and the table will be updated accordingly. +Τhe following table details the testing done against the reference architectures along with the frequency and results. Note that this list above is non exhaustive. Additional testing is continuously evaluated and iterated on, and the table is updated accordingly. | Reference
Architecture
Size | Bare-Metal | GCP | AWS | Azure | |-----------------------------|------------|-----|-----|-------| @@ -220,4 +220,4 @@ Note the following about the testing process: | 25k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)1 | - | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | | 50k | Refer to GCP1 | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)1 | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - | -1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [Omnibus GitLab](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). +1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [Omnibus GitLab](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad-hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). -- GitLab