From 408d59b2390aebec8941e1de4f0065774ac4217e Mon Sep 17 00:00:00 2001 From: Caleb Cooper <6532460-calebcooper@users.noreply.gitlab.com> Date: Fri, 6 May 2022 17:57:39 +0000 Subject: [PATCH 1/5] Highlight incremental logging --- doc/administration/reference_architectures/3k_users.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index d1ee13b19401ca..243f77964b32f2 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -2122,10 +2122,6 @@ There are two ways of specifying object storage configuration in GitLab: Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. - -In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. - For configuring object storage in GitLab 13.1 and earlier, or for storage types not supported by consolidated configuration form, refer to the following guides based on what features you intend to use: @@ -2155,6 +2151,12 @@ in the future. +## Enable incremental logging + +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. + +As this reference architecture does not use NFS to share this directory between Gitlab Rails and Sidekiq nodes, it is necessary to enable [Incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/elasticsearch.md) -- GitLab From ab5a4bb0516310d0f71d519c1952ca2ac0e09924 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 11 May 2022 13:56:56 +0000 Subject: [PATCH 2/5] Apply 1 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/3k_users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 243f77964b32f2..1e86301719698d 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -2155,7 +2155,7 @@ in the future. GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. -As this reference architecture does not use NFS to share this directory between Gitlab Rails and Sidekiq nodes, it is necessary to enable [Incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. ## Configure Advanced Search -- GitLab From ae0a0b52306229e98dc29adb57b3ed835375aa40 Mon Sep 17 00:00:00 2001 From: Caleb Cooper <6532460-calebcooper@users.noreply.gitlab.com> Date: Wed, 11 May 2022 14:21:41 +0000 Subject: [PATCH 3/5] Add incremental logging section to 5k+ reference architectures --- .../reference_architectures/10k_users.md | 10 ++++++---- .../reference_architectures/25k_users.md | 10 ++++++---- .../reference_architectures/50k_users.md | 10 ++++++---- doc/administration/reference_architectures/5k_users.md | 10 ++++++---- 4 files changed, 24 insertions(+), 16 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 47eb149eb2d2e7..049b8e9f697beb 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2183,10 +2183,6 @@ There are two ways of specifying object storage configuration in GitLab: Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. - -In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. - For configuring object storage in GitLab 13.1 and earlier, or for storage types not supported by consolidated configuration form, refer to the following guides based on what features you intend to use: @@ -2216,6 +2212,12 @@ in the future. +## Enable incremental logging + +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. + +As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/elasticsearch.md) diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 225cf4b9c71591..a584e8f2203608 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -2187,10 +2187,6 @@ There are two ways of specifying object storage configuration in GitLab: Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. - -In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. - For configuring object storage in GitLab 13.1 and earlier, or for storage types not supported by consolidated configuration form, refer to the following guides based on what features you intend to use: @@ -2220,6 +2216,12 @@ in the future. +## Enable incremental logging + +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. + +As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/elasticsearch.md) diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 7fb3f158848bd1..9010c943186bc3 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -2203,10 +2203,6 @@ There are two ways of specifying object storage configuration in GitLab: Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. - -In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. - For configuring object storage in GitLab 13.1 and earlier, or for storage types not supported by consolidated configuration form, refer to the following guides based on what features you intend to use: @@ -2236,6 +2232,12 @@ in the future. +## Enable incremental logging + +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. + +As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/elasticsearch.md) diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 91a630a7eb0041..81158942d6c88f 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -2122,10 +2122,6 @@ There are two ways of specifying object storage configuration in GitLab: Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. - -In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. - For configuring object storage in GitLab 13.1 and earlier, or for storage types not supported by consolidated configuration form, refer to the following guides based on what features you intend to use: @@ -2155,6 +2151,12 @@ in the future. +## Enable incremental logging + +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. + +As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/elasticsearch.md) -- GitLab From 55402b30edfcf50437e0604c83cf53c4f91192af Mon Sep 17 00:00:00 2001 From: Grant Young Date: Mon, 23 May 2022 17:37:12 +0000 Subject: [PATCH 4/5] Apply 1 suggestion(s) to 1 file(s) --- doc/administration/reference_architectures/3k_users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 1e86301719698d..176ae033ae14b3 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -2155,7 +2155,7 @@ in the future. GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. -As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +To remove the need to use NFS it's recommended however that to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. ## Configure Advanced Search -- GitLab From 0a857a639ac0328887bbb4c584357862833a94f1 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 8 Jun 2022 20:52:30 +0000 Subject: [PATCH 5/5] Apply 5 suggestion(s) to 5 file(s) --- doc/administration/reference_architectures/10k_users.md | 4 ++-- doc/administration/reference_architectures/25k_users.md | 4 ++-- doc/administration/reference_architectures/3k_users.md | 4 ++-- doc/administration/reference_architectures/50k_users.md | 4 ++-- doc/administration/reference_architectures/5k_users.md | 4 ++-- 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 049b8e9f697beb..a2dadaf4f1631f 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -2214,9 +2214,9 @@ in the future. ## Enable incremental logging -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. -As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. ## Configure Advanced Search diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index a584e8f2203608..466182598d39b7 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -2218,9 +2218,9 @@ in the future. ## Enable incremental logging -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. -As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. ## Configure Advanced Search diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 176ae033ae14b3..2d68a253f02753 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -2153,9 +2153,9 @@ in the future. ## Enable incremental logging -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. -To remove the need to use NFS it's recommended however that to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. ## Configure Advanced Search diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 9010c943186bc3..344273797cd34f 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -2234,9 +2234,9 @@ in the future. ## Enable incremental logging -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. -As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. ## Configure Advanced Search diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 81158942d6c88f..422a12ee22cec3 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -2153,9 +2153,9 @@ in the future. ## Enable incremental logging -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. -As this reference architecture does not use NFS to share this directory between GitLab Rails and Sidekiq nodes, it is necessary to enable [incremental logging](../job_logs.md#incremental-logging-architecture) (available in GitLab 13.6 and later), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. ## Configure Advanced Search -- GitLab