CI: Download repo from artifacts in lint stage pipelines
What does this MR do and why?
Replace git cloning with artifact-based repository fetching in lint stage to reduce Gitaly load and mitigate "too much load" errors.
This also reduces the CI build time by ~15 seconds
References
Contributes to GitLab Service Overload Impact on Pipelines ("u... (gitlab-org/quality/analytics/team#134)
Gitlab Doc link
List of jobs affected by this change
docs code_qualitydocs hugo_builddocs-i18n-lint japanese-valedocs-i18n-lint markdowndocs-i18n-lint pathsdocs-lint linksdocs-lint markdowndocs-lint mermaiddocs-lint redirectsdocs-i18n-lint linksfeature-flags-usagegenerate-apollo-graphql-schemahaml-lintlint-pipeline-yamllint-yamlping-appsec-for-sast-findingsqa:metadata-lintqa:orchestrator-lintqa:selectorsrubocoprubocop-docscheck-deprecated-filestemplates-shellchecksemgrep-appsec-custom-rulesstatic-verification-with-databasestatic-analysis
JOBS That are not part of this change in STAGE Lint
-
check-deprecated-files- As clone-gitlab-repo is just cloning to depth 20 , and check-deprecated-files is not able to fetch previous commit revision Job Link
Before Logs
Getting source from Git repository
Gitaly correlation ID: b0b18ba721c14de6ab2010c6966b016f
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/gitlab-org/gitlab/.git/
Created fresh repository.
remote: Enumerating objects: 752904, done.
remote: Counting objects: 100% (752904/752904), done.
remote: Compressing objects: 100% (332322/332322), done.
remote: Total 752904 (delta 535877), reused 581935 (delta 399963), pack-reused 0 (from 0)
Receiving objects: 100% (752904/752904), 522.25 MiB | 33.48 MiB/s, done.
Resolving deltas: 100% (535877/535877), done.
From https://us-east1-c.ci-gateway.int.gprd.gitlab.net:8989/gitlab-org/gitlab
* [new ref] refs/pipelines/2218033785 -> refs/pipelines/2218033785
Checking out aa8eae83 as detached HEAD (ref is refs/merge-requests/214344/merge)...
Skipping Git submodules setup
After Chnage Logs
Getting source from Git repository
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
Downloading artifacts
Downloading artifacts for clone-gitlab-repo (12457474851)...
Downloading artifacts from coordinator... ok correlation_id=21a9f157dcc7490c8d897110861063eb host=storage.googleapis.com id=12457474851 responseStatus=200 OK token=6b_XKyAqB
IMPACT on CI Speed
Related Issues
Edited by Pranshu Sharma