From cae24815fea8510df47d37a0bf9bd21d425f2e06 Mon Sep 17 00:00:00 2001 From: Shinya Maeda Date: Thu, 12 Dec 2019 11:08:28 +0700 Subject: [PATCH 1/2] Docs for Ci Resource Group This commit adds docs for Ci resource group --- app/models/ci/build.rb | 5 ++++ .../unreleased/ci-resource-group-doc.yml | 6 ++++ doc/ci/yaml/README.md | 30 +++++++++++++++++++ .../ci/pipeline/seed/build/resource_group.rb | 2 +- 4 files changed, 42 insertions(+), 1 deletion(-) create mode 100644 changelogs/unreleased/ci-resource-group-doc.yml diff --git a/app/models/ci/build.rb b/app/models/ci/build.rb index 5324d59c1556a2..f1fbbe74e6389f 100644 --- a/app/models/ci/build.rb +++ b/app/models/ci/build.rb @@ -489,6 +489,11 @@ def has_environment? environment.present? end + def requires_resource? + Feature.enabled?(:ci_resource_group, project, default_enabled: true) && + self.resource_group_id.present? + end + def starts_environment? has_environment? && self.environment_action == 'start' end diff --git a/changelogs/unreleased/ci-resource-group-doc.yml b/changelogs/unreleased/ci-resource-group-doc.yml new file mode 100644 index 00000000000000..a675e5f2971333 --- /dev/null +++ b/changelogs/unreleased/ci-resource-group-doc.yml @@ -0,0 +1,6 @@ +--- +title: Add 'resource_group' keyword to .gitlab-ci.yml for pipeline job concurrency + limitation +merge_request: 21617 +author: +type: added diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 84e0bb873a752b..f2500b73cae164 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -119,6 +119,7 @@ The following table lists available parameters for jobs: | [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | | [`variables`](#variables) | Define job variables on a job level. | | [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | +| [`resource_group`](#resource_group) | Limit pipeline job concurrency using named semaphores. | NOTE: **Note:** Parameters `types` and `type` are [deprecated](#deprecated-parameters). @@ -2634,6 +2635,35 @@ In the example above, a new pipeline run will cause an existing running pipeline NOTE: **Note:** Once an uninterruptible job is running, the pipeline will never be canceled, regardless of the final job's state. +### `resource_group` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/15536) in GitLab 12.6. + +Sometimes running multiples jobs or pipelines at a time for the same environment +can lead to errors during the deployment. + +To avoid these errors, the attribute `resource_group` can be used for assuring that +the Runner will only pick one job at a time. + +When `resource_group` key is defined in a job in `.gitlab-ci.yml`, +running jobs are mutually exclusive across different pipelines in the same project. +If multiple jobs belong to the same resource group are enqueued simultaneously, +only one of them will be picked by the Runner and the other jobs wait until the +currently running job to finish. + +Here is a simple example: + +```yaml +deploy-to-production: + script: deploy + resource_group: production +``` + +In this case, one of the `deploy-to-production` jobs run at a time. When a new +`deploy-to-production` job is created, it doesn't run until the currently running/pending +`deploy-to-production` job has finished. As a result, you can ensure that concurrent deployments +will never happen to the production environment. + ### `include` > - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. diff --git a/lib/gitlab/ci/pipeline/seed/build/resource_group.rb b/lib/gitlab/ci/pipeline/seed/build/resource_group.rb index 100eb1d4084d45..3bec6d1e8b6429 100644 --- a/lib/gitlab/ci/pipeline/seed/build/resource_group.rb +++ b/lib/gitlab/ci/pipeline/seed/build/resource_group.rb @@ -16,7 +16,7 @@ def initialize(build, resource_group_key) end def to_resource - return unless Feature.enabled?(:ci_resource_group, build.project) + return unless Feature.enabled?(:ci_resource_group, build.project, default_enabled: true) return unless resource_group_key.present? resource_group = build.project.resource_groups -- GitLab From 32cd96f44092c08ee1d2e6857ab161a1500fa631 Mon Sep 17 00:00:00 2001 From: Orit Golowinski Date: Mon, 23 Dec 2019 07:51:32 +0000 Subject: [PATCH 2/2] Changed wording a bit and added a use case --- app/models/ci/build.rb | 7 +------ doc/ci/yaml/README.md | 32 ++++++++++++++++++-------------- 2 files changed, 19 insertions(+), 20 deletions(-) diff --git a/app/models/ci/build.rb b/app/models/ci/build.rb index f1fbbe74e6389f..5edd2f52fc8d6b 100644 --- a/app/models/ci/build.rb +++ b/app/models/ci/build.rb @@ -481,7 +481,7 @@ def expanded_kubernetes_namespace end def requires_resource? - Feature.enabled?(:ci_resource_group, project) && + Feature.enabled?(:ci_resource_group, project, default_enabled: true) && self.resource_group_id.present? end @@ -489,11 +489,6 @@ def has_environment? environment.present? end - def requires_resource? - Feature.enabled?(:ci_resource_group, project, default_enabled: true) && - self.resource_group_id.present? - end - def starts_environment? has_environment? && self.environment_action == 'start' end diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index f2500b73cae164..62a113a407c582 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -119,7 +119,7 @@ The following table lists available parameters for jobs: | [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | | [`variables`](#variables) | Define job variables on a job level. | | [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | -| [`resource_group`](#resource_group) | Limit pipeline job concurrency using named semaphores. | +| [`resource_group`](#resource_group) | Limit job concurrency. | NOTE: **Note:** Parameters `types` and `type` are [deprecated](#deprecated-parameters). @@ -2637,19 +2637,19 @@ Once an uninterruptible job is running, the pipeline will never be canceled, reg ### `resource_group` -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/15536) in GitLab 12.6. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/15536) in GitLab 12.7. -Sometimes running multiples jobs or pipelines at a time for the same environment +Sometimes running multiples jobs or pipelines at the same time in an environment can lead to errors during the deployment. -To avoid these errors, the attribute `resource_group` can be used for assuring that -the Runner will only pick one job at a time. +To avoid these errors, the `resource_group` attribute can be used to ensure that +the Runner will not run certain jobs simultaneously. -When `resource_group` key is defined in a job in `.gitlab-ci.yml`, -running jobs are mutually exclusive across different pipelines in the same project. -If multiple jobs belong to the same resource group are enqueued simultaneously, -only one of them will be picked by the Runner and the other jobs wait until the -currently running job to finish. +When the `resource_group` key is defined in a job in `.gitlab-ci.yml`, +job runs are mutually exclusive across different pipelines in the same project. +If multiple jobs belonging to the same resource group are enqueued simultaneously, +only one of them will be picked by the Runner, and the other jobs will wait until the +`resource_group` is free. Here is a simple example: @@ -2659,10 +2659,14 @@ deploy-to-production: resource_group: production ``` -In this case, one of the `deploy-to-production` jobs run at a time. When a new -`deploy-to-production` job is created, it doesn't run until the currently running/pending -`deploy-to-production` job has finished. As a result, you can ensure that concurrent deployments -will never happen to the production environment. +In this case, if a `deploy-to-production` job is running in a pipeline, and a new +`deploy-to-production` job is created in a different pipeline, it will not run until +the currently running/pending `deploy-to-production` job is finished. As a result, +you can ensure that concurrent deployments will never happen to the production environment. + +There can be multiple `resource_group`s defined per environment. A good use case for this +is when deploying to physical devices. You may have more than one phyisical device, and each +one can be deployed to, but there can be only one deployment per device at any given time. ### `include` -- GitLab