diff --git a/app/assets/javascripts/editor/schema/ci.json b/app/assets/javascripts/editor/schema/ci.json index 23fc89a4d6c847d258e34e1a595db23666581b1f..1c2613ba724d8e95959b80b19fc5eee858b9e818 100644 --- a/app/assets/javascripts/editor/schema/ci.json +++ b/app/assets/javascripts/editor/schema/ci.json @@ -2389,6 +2389,10 @@ "path_prefix": { "type": "string", "markdownDescription": "The GitLab Pages URL path prefix used in this version of pages. The given value is converted to lowercase, shortened to 63 bytes, and everything except alphanumeric characters is replaced with a hyphen. Leading and trailing hyphens are not permitted." + }, + "expire_in": { + "type": "string", + "markdownDescription": "How long the deployment should be active. Deployments that have expired are no longer available on the web. Supports a wide variety of formats, e.g. '1 week', '3 mins 4 sec', '2 hrs 20 min', '2h20min', '6 mos 1 day', '47 yrs 6 mos and 4d', '3 weeks and 2 days'. Set to 'never' to prevent extra deployments from expiring. [Learn More](https://docs.gitlab.com/ee/ci/yaml/#pagesexpire_in)." } } } diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index f65cdcacdd4a738143e58d7d812a45e7e7b9527f..bcf7a4ecbed907a033020a62085406b7524e79a2 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -3461,7 +3461,7 @@ as an artifact and published with GitLab Pages. DETAILS: **Tier:** Premium, Ultimate -**Offering:** Self-managed +**Offering:** GitLab.com, Self-managed, GitLab Dedicated **Status:** Experiment > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/129534) in GitLab 16.7 as an [experiment](../../policy/experiment-beta-support.md) [with a flag](../../user/feature_flags.md) named `pages_multiple_versions_setting`, disabled by default. @@ -3493,6 +3493,50 @@ pages: In this example, a different pages deployment is created for each branch. +### `pages:pages.expire_in` + +DETAILS: +**Tier:** Premium, Ultimate +**Offering:** GitLab.com, Self-managed, GitLab Dedicated + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/456478) in GitLab 17.4 + +Use `expire_in` to specify how long a deployment should be available before +it expires. After the deployment is expired, it's deactivated by a cron +job running every 10 minutes. + +Extra deployments expire by default. To prevent them from expiring, set the +value to `never`. + +**Keyword type**: Job keyword. You can use it only as part of a `pages` job. + +**Possible inputs**: The expiry time. If no unit is provided, the time is in seconds. +Valid values include: + +- `'42'` +- `42 seconds` +- `3 mins 4 sec` +- `2 hrs 20 min` +- `2h20min` +- `6 mos 1 day` +- `47 yrs 6 mos and 4d` +- `3 weeks and 2 days` +- `never` + +**Example of `pages:pages.expire_in`**: + +```yaml +pages: + stage: deploy + script: + - echo "Pages accessible through ${CI_PAGES_URL}" + pages: + expire_in: 1 week + artifacts: + paths: + - public +``` + ### `parallel` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336576) in GitLab 15.9, the maximum value for `parallel` is increased from 50 to 200. diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 0e9a459607880ca9336faf9b46a66ef70f241c4d..84d546ca920498fcaf0244c0efeb6a3606c56d79 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -175,6 +175,56 @@ The project maintainer can disable this feature on: 1. Deselect the **Use unique domain** checkbox. 1. Select **Save changes**. +## Expiring deployments + +DETAILS: +**Tier:** Premium, Ultimate +**Offering:** GitLab.com, Self-managed, GitLab Dedicated + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/162826) in GitLab 17.4. + +You can configure your Pages deployments to be automatically deleted after +a period of time has passed by specifying a duration at `pages.expire_in`: + +```yaml +pages: + stage: deploy + script: + - ... + pages: + expire_in: 1 week + artifacts: + paths: + - public +``` + +By default, [extra deployments](#create-multiple-deployments) expire automatically after 24 hours. +To disable this behavior, set `pages.expire_in` to `never`. + +Expired deployments are stopped by a cron job that runs every 10 minutes. +Stopped deployments are subsequently deleted by another cron job that also +runs every 10 minutes. To recover it, follow the steps described in +[Recover a stopped deployment](#recover-a-stopped-deployment). + +A stopped or deleted deployment is no longer available on the web. Users will +see a 404 Not found error page at its URL, until another deployment is created +with the same URL configuration. + +### Recover a stopped deployment + +Prerequisites: + +- You must have at least the Maintainer role for the project. + +To recover a stopped deployment that has not yet been deleted: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Deploy > Pages**. +1. Near **Deployments** turn on the **Include stopped deployments** toggle. + If your deployment has not been deleted yet, it should be included in the + list. +1. Expand the deployment you want to recover and select **Restore**. + ## Create multiple deployments DETAILS: @@ -236,7 +286,12 @@ The number of extra deployments is limited by the root-level namespace. For spec By default, extra deployments expire after 24 hours, after which they are deleted. If you're using a self-hosted instance, your instance admin can -[configure a different duration](../../../administration/pages/index.md#configure-the-default-expiry-for-extra-deployments). +[configure a different default duration](../../../administration/pages/index.md#configure-the-default-expiry-for-extra-deployments). + +To customize the expiry time, [configure `pages.expire_in`](#expiring-deployments). + +To prevent deployments from automatically expiring, set `pages.expire_in` to +`never`. ### Path clash diff --git a/ee/app/services/ee/projects/update_pages_service.rb b/ee/app/services/ee/projects/update_pages_service.rb index 146361ee67ebb818d37859c9d3ec8c17ab179ca2..b9c5c826a8e939e0c4cce1787dce346ebcf65fdb 100644 --- a/ee/app/services/ee/projects/update_pages_service.rb +++ b/ee/app/services/ee/projects/update_pages_service.rb @@ -15,11 +15,30 @@ def pages_deployment_attributes(file, build) private + def fallback_expiry_date + value = ::Gitlab::CurrentSettings.pages_extra_deployments_default_expiry_seconds + + # A value of 0 means deployments should not expire, in this case return nil + value.seconds.from_now if value.nonzero? + end + + def custom_expire_in + build.pages&.fetch(:expire_in, nil) + end + + def custom_expiry_date + ::Gitlab::Ci::Build::DurationParser.new(custom_expire_in).seconds_from_now + end + + def expiry_customised? + custom_expire_in.present? + end + + # returns a datetime for the expiry or may return nil if expire_in='never' def expires_at - return unless ::Gitlab::CurrentSettings.pages_extra_deployments_default_expiry_seconds&.nonzero? - return unless extra_deployment? + return custom_expiry_date if expiry_customised? - ::Gitlab::CurrentSettings.pages_extra_deployments_default_expiry_seconds.seconds.from_now + fallback_expiry_date if extra_deployment? end end end diff --git a/ee/spec/models/ci/build_spec.rb b/ee/spec/models/ci/build_spec.rb index c006341eb64946b3176142056bd3178b8a91a3ae..c8d7a2b6159b98d87f3e3f5d24abf6373dfaa040 100644 --- a/ee/spec/models/ci/build_spec.rb +++ b/ee/spec/models/ci/build_spec.rb @@ -1090,6 +1090,8 @@ true | { pages: { path_prefix: nil } } | { path_prefix: '' } true | { pages: { path_prefix: 'foo' } } | { path_prefix: 'foo' } true | { pages: { path_prefix: '$CI_COMMIT_BRANCH' } } | { path_prefix: 'master' } + true | { pages: { path_prefix: 'foo', expire_in: '1d' } } | { path_prefix: 'foo', expire_in: '1d' } + true | { pages: { path_prefix: 'foo', expire_in: 'never' } } | { path_prefix: 'foo', expire_in: 'never' } end with_them do diff --git a/ee/spec/services/ee/projects/update_pages_service_spec.rb b/ee/spec/services/ee/projects/update_pages_service_spec.rb index 065c4d19977ea91d300d33715d18fe655865a3e6..1afb4c7b2cd9315ac58f40bd710e52f8877ebcf4 100644 --- a/ee/spec/services/ee/projects/update_pages_service_spec.rb +++ b/ee/spec/services/ee/projects/update_pages_service_spec.rb @@ -3,12 +3,17 @@ require 'spec_helper' RSpec.describe Projects::UpdatePagesService, feature_category: :pages do + using RSpec::Parameterized::TableSyntax + let_it_be(:project) { create(:project) } let_it_be(:user) { create(:user) } let(:path_prefix) { nil } - let(:build_options) { { pages: { path_prefix: path_prefix } } } + let(:expire_in) { nil } + let(:system_default_expiry) { 86400 } + let(:build_options) { { pages: { path_prefix: path_prefix, expire_in: expire_in } } } let(:build) { create(:ci_build, :pages, project: project, user: user, options: build_options) } + let(:multiple_versions_enabled) { true } subject(:service) { described_class.new(project, build) } @@ -17,19 +22,48 @@ end before do + stub_application_setting(pages_extra_deployments_default_expiry_seconds: system_default_expiry) stub_pages_setting(enabled: true) + allow(::Gitlab::Pages) + .to receive(:multiple_versions_enabled_for?) + .with(build.project) + .and_return(multiple_versions_enabled) + end + + describe 'expiry' do + # Specify a fixed date as now, because we want to reference it in the examples + # and freeze_time does not apply during spec setup + now = Time.utc(2024, 8, 29, 13, 20, 0) + + before do + travel_to now + end + + where(:path_prefix, :system_default_expiry, :expire_in, :result) do + '/path_prefix/' | 3600 | '1 week' | (now + 1.week) # use the value from ci over the default + '/path_prefix/' | 3600 | 'never' | nil # a value of 'never' prevents the deployment from expiring + '/path_prefix/' | 3600 | nil | (now + 1.hour) # fall back to the system setting + '/path_prefix/' | 0 | nil | nil # System setting can also be set to 0 (no expiry) + '/path_prefix/' | 0 | '1 week' | (now + 1.week) # but make sure to still use the value from ci + '' | 3600 | '1 week' | (now + 1.week) # main deployments can also be set to expire + '' | 3600 | nil | nil # but they should not do so by default + end + + with_them do + it "sets the expiry date to the expected value" do + expect { expect(service.execute).to include(status: :success) } + .to change { project.pages_deployments.count }.by(1) + + expect(project.pages_deployments.last.expires_at).to eq(result) + end + end end context 'when path_prefix is not blank' do let(:path_prefix) { '/path_prefix/' } context 'and pages_multiple_versions is disabled for project' do - before do - allow(::Gitlab::Pages) - .to receive(:multiple_versions_enabled_for?) - .with(build.project) - .and_return(false) - end + let(:multiple_versions_enabled) { false } it 'does not create a new pages_deployment' do expect { expect(service.execute).to include(status: :error) } @@ -44,11 +78,9 @@ end context 'and pages_multiple_versions is enabled for project' do + let(:multiple_versions_enabled) { true } + before do - allow(::Gitlab::Pages) - .to receive(:multiple_versions_enabled_for?) - .with(build.project) - .and_return(true) stub_application_setting(pages_extra_deployments_default_expiry_seconds: 3600) end @@ -75,25 +107,4 @@ end end end - - context 'when path_prefix is blank' do - let(:path_prefix) { '' } - - context 'and pages_multiple_versions is enabled for project' do - before do - allow(::Gitlab::Pages) - .to receive(:multiple_versions_enabled_for?) - .with(build.project) - .and_return(true) - stub_application_setting(pages_extra_deployments_default_expiry_seconds: 3600) - end - - it 'does not set an expiry date', :freeze_time do - expect { expect(service.execute).to include(status: :success) } - .to change { project.pages_deployments.count }.by(1) - - expect(project.pages_deployments.last.expires_at).to be_nil - end - end - end end diff --git a/lib/gitlab/ci/config/entry/pages.rb b/lib/gitlab/ci/config/entry/pages.rb index 57d9e944f51d2df2ef5ee4b77034a318f13fcf01..e61bdb9d69389eac1d76dee8caec59376635dce4 100644 --- a/lib/gitlab/ci/config/entry/pages.rb +++ b/lib/gitlab/ci/config/entry/pages.rb @@ -9,7 +9,7 @@ module Entry # Entry that represents the pages attributes # class Pages < ::Gitlab::Config::Entry::Node - ALLOWED_KEYS = %i[path_prefix].freeze + ALLOWED_KEYS = %i[path_prefix expire_in].freeze include ::Gitlab::Config::Entry::Attributable include ::Gitlab::Config::Entry::Validatable @@ -22,6 +22,7 @@ class Pages < ::Gitlab::Config::Entry::Node with_options allow_nil: true do validates :path_prefix, type: String + validates :expire_in, duration: { parser: ::Gitlab::Ci::Build::DurationParser } end end end diff --git a/spec/lib/gitlab/ci/config/entry/pages_spec.rb b/spec/lib/gitlab/ci/config/entry/pages_spec.rb index 0ee692a443e6ece8174b8ee68c430816a9fef614..be2cbc23b3b03759154524c6fa5fd8e7c6a98f05 100644 --- a/spec/lib/gitlab/ci/config/entry/pages_spec.rb +++ b/spec/lib/gitlab/ci/config/entry/pages_spec.rb @@ -17,31 +17,78 @@ context 'when value is a hash' do context 'when the hash is valid' do - let(:config) { { path_prefix: 'prefix' } } + let(:config) { { path_prefix: 'prefix', expire_in: '1 day' } } it 'is valid' do expect(entry).to be_valid expect(entry.value).to eq({ - path_prefix: 'prefix' + path_prefix: 'prefix', + expire_in: '1 day' }) end end - context 'when path_prefix key is not a string' do - let(:config) { { path_prefix: 1 } } + context 'when hash contains not allowed keys' do + let(:config) { { unknown: 'echo' } } it 'is invalid' do expect(entry).not_to be_valid - expect(entry.errors).to include('pages path prefix should be a string') + expect(entry.errors).to include('pages config contains unknown keys: unknown') end end - context 'when hash contains not allowed keys' do - let(:config) { { unknown: 'echo' } } + context 'when it specifies path_prefix' do + context 'and it is not a string' do + let(:config) { { path_prefix: 1 } } - it 'is invalid' do - expect(entry).not_to be_valid - expect(entry.errors).to include('pages config contains unknown keys: unknown') + it 'is invalid' do + expect(entry).not_to be_valid + expect(entry.errors).to include('pages path prefix should be a string') + end + end + end + + context 'when it specifies expire_in' do + context 'and it is a duration string' do + let(:config) { { expire_in: '1 day' } } + + it 'is valid' do + expect(entry).to be_valid + expect(entry.value).to eq({ + expire_in: '1 day' + }) + end + end + + context 'and it is never' do + let(:config) { { expire_in: 'never' } } + + it 'is valid' do + expect(entry).to be_valid + expect(entry.value).to eq({ + expire_in: 'never' + }) + end + end + + context 'and it is nil' do + let(:config) { { expire_in: nil } } + + it 'is valid' do + expect(entry).to be_valid + expect(entry.value).to eq({ + expire_in: nil + }) + end + end + + context 'and it is an invalid duration' do + let(:config) { { expire_in: 'some string that cant be parsed' } } + + it 'is valid' do + expect(entry).not_to be_valid + expect(entry.errors).to include('pages expire in should be a duration') + end end end end