8.10 CI Plan

The bugs that needs to be solved.

The key features (includes a Frontend and Backend):

• Allow to add manual actions: #17010 (closed): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5297

The planned Frontend improvements (@annabeldunstone @hazelyang):

• Move Build settings and build badge to new CI/CD Pipelines settings page: https://gitlab.com/gitlab-org/gitlab-ce/issues/15343 !5244 (merged)
• Add warn_on_failure option on CI builds (@connorshea): https://gitlab.com/gitlab-org/gitlab-ce/issues/17669: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5004
• Design updates for pipelines (Frontend): https://gitlab.com/gitlab-org/gitlab-ce/issues/18479 https://gitlab.com/gitlab-org/gitlab-ce/issues/17760 (!5156 (merged))
• Don't pluralize pipeline stage headings (!5117 (merged))
• Update pending/running icons (!5222 (merged))
• Update pipeline/build state icons: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5280
• Change 'New Pipeline' button to 'Run Pipeline' (!5119 (merged))
• Shorten duration representation (!5121 (merged))
• Distinguish pipeline ref with tag or branch icon (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5239)

The planned Backend improvements (@ayufan @godfat @grzesiek @tmaczukin):

• Continue GitLabCiYamlProcessor refactoring (@grzesiek): https://gitlab.com/gitlab-org/gitlab-ce/issues/15060 (This is to be followed up)
  • Add new GitLab CI configuration facade class (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4462)
  • Add global entry with before script to new CI config (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4482)
  • Improve validations and error handling in new CI config entries (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4560)
  • Move global CI entries handling from legacy to new config (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4820)
  • Move job CI config entries from legacy to new config (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5087)
• Extend CI predefined variables (@ayufan): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4826
• Start tracking user who created a pipeline (#18992 (closed)) to make it possible to add (@ayufan): https://gitlab.com/gitlab-org/gitlab-ce/issues/18054: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5272
• Add artifacts_size to database (@godfat): https://gitlab.com/gitlab-org/gitlab-ce/issues/18869: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4964
• Use Gitlab-Workhorse-Send-Data to send entry (@godfat): #19128 (closed), #19224 (closed), !5094 (merged), !5214 (merged), gitlab-workhorse!53 (merged)
• Add an API endpoint to retrieve the latest build artifact for a branch (@godfat): #4255 (closed), #14419 (closed): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5347
• Allow Slack service to notify status of builds on a different channel: #8110 (closed): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5124
• Fix LFS for CI: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5270

The planned GitLab Runner improvements (@ayufan @tmaczukin):

• Runner's release process description: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/176
• Introduce production engineers to CI runner management: https://gitlab.com/gitlab-com/infrastructure/issues/133
• Enable privileged mode on Shared Runners: https://gitlab.com/gitlab-com/infrastructure/issues/158
• Use Sentry in GitLab Runner to monitor critical errors: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1022: MR: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/217
• Improve logging: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/218
• Extend support for caching and artifacts for other executors: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/224
• Add support for cloning VirtualBox VM snapshots: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/111
• Add integration tests: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/231
• Improve support for Docker Machine: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/233
• Improve build abort: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/232

Slipped

• Update artifacts_size for old builds (!5076 (closed))
• Add download buttons for downloading artifacts for a branch (@godfat): #4255 (closed), #14419 (closed), !5142 (merged)
• Prepare a blog post about Shared Runners and performance: gitlab-com/blog-posts#244 (closed)
• Pipelines for merge request (Frontend): #18681 (closed)
• Collapse build tabs https://gitlab.com/gitlab-org/gitlab-ce/issues/18516
• Pre-create all builds when processing the pipeline (@ayufan): this is to solved https://gitlab.com/gitlab-org/gitlab-ce/issues/18644 and https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/289: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5296, https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5295 (probably not)

Stretch; likely not to happen:

• Add a line number and permalink to each line of build trace (Frontend): https://gitlab.com/gitlab-org/gitlab-ce/issues/13205: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5327
• Make internal projects readable by all tokens / deploy keys: #18993 (closed)
• Retry all docker operations in GitLab Runner to increase stability of builds: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/122
• Implement code coverage badge and some other improvements regarding code coverage: #3714 (closed): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3994
• Show artifacts size in various places: https://gitlab.com/gitlab-org/gitlab-ce/issues/18352
• Fix issue with leaking sockets when using docker executor: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/196: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/185
• Prepare monitoring recommendations with description for our infrastructure team
• Generate coverage report for whole test suite: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5018
• Have CI automatically test migration paths (@zj): #14549 (closed): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4874

Very, very unlikely to happen:

• My Pipelines: #18054 (moved)
• Start tracking pipelines for Merge Requests and add merge-requests to only: https://gitlab.com/gitlab-org/gitlab-ce/issues/15310
• Statistics server embedded into Runner: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/219. The ideas how to improve that: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1484

The future features: CI Backlog

CI issues for 8.10: https://gitlab.com/gitlab-org/gitlab-ce/issues?assignee_id=&author_id=&label_name=CI&milestone_title=8.10&scope=all&sort=id_desc&state=opened&utf8=%E2%9C%93&weight=

Runner issues for 1.4: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues?scope=all&sort=id_desc&state=opened&utf8=%E2%9C%93&milestone_title=v1.4&label-name=