Snipped from https://hackmd.io/vdYP7DO5RDatOwG686ez3Q

Title : Reducing overall CI workload tezos/tezos

Description : The idea is to reduce the overall workload. This will save money, which can be used use faster hardware to speed up the marge-bot pipeline wall time. The major contributor to workload atm is opam tests, so most ideas are centered around those tests.

Estimated effort (in nb weeks) : 1 week

Associated KRs : CI runs consistently under 20 minutes and cost is not higher that the current one

Dependencies : Project 1

Part of : https://gitlab.com/groups/tezos/-/milestones/5

Metrics & Objectives

• Goal metrics (measured by):
  • Projected Sequential Time per Pipeline (M1.3)
  • Recorded Sequential Time per Pipeline (M2.3)
• Guardrail metric (measured by):
  • Number of opam failures in scheduled pipelines (M3.1)
  • Recorded / projected AWS cost (implied by the goal metric if sequential time is a sufficient proxy for cost and by M4)

Objectives:

• Goal metric:
  • Reduce projected sequential time per pipeline to 230 minutes (from 422 minutes).
  • Reduce recorded sequential time per pipeline to 250 minutes (from 422 minutes).
• Guardrail metric:
  • The number of opam failures in scheduled pipelines should be similar to a time period before the our changes (-+ 5%).
  • Recorded / projected AWS cost does not increase (this is implied by the goal metric if sequential time is a sufficient proxy for cost)

Task breakdown

As list:

• Only run opam jobs for leaves/top-level packages (@abate)
• nomadic-labs/marge-bot#9 (closed): Guarantee final pipeline in marge-bot (@abate)
• (Use runtest image instead of prebuild) @(Arvid, Pietro) {2-3d}

Deliverables

• A pipeline configuration that runs less opam jobs

How the objective was computed