[go: up one dir, main page]

Images/CI: adds [runtime_datadog] image

Bruno Brequested to merge
bruno-dd-ci-runtime-clean into master

What

  1. Creates an intermediate image runtime_datadog that adds datadog-ci to all images/ci images except runtime.
  2. Modifies before_script so that every job that has a non-NOOP before_script step will by default try to send info to datadog.

Why

  1. To instrument all CI jobs using one of the prebuild, build, test, e2etest images. (NB: no job uses directly runtime image.)
  2. Temporary change that we plan to reverse:
    • it's a low hanging fruit to have more jobs instrumented by datadog-ci
    • but ultimately we want all CI jobs to run scripts/ci/datadog_send_job_info.sh.

How

  • We move npm and datadog-ci from e2etest to a newly created runtime_datadog image.
    • datadog-ci needs to be installed with npm (the standalone binary does not work on alpine).
    • The authentication to our proxy NPM registry is now done in runtime_datadog.
  • We use runtime_datadog instead of runtime in prebuild, build, test, e2etest.
  • We add runtime_datadog in the variants build in the CI defined in ci.inc.sh.
  • We update the README as well as some comments in the Dockerfiles.

An alternative would have been to add datadog-ci in runtime and then to remove it when building the Octez Docker image. We chose not to do that since it was simpler to create a dedicated image and because no CI job uses the runtime image. (Many thanks @abate for this suggestion.)

Manually testing the MR

  • Test pipelines are (mostly) green: images can be built (CI images and also Octez Docker image).

  • Jobs are indeed instrumented, which means that datadog-ci is installed on prebuild, build, test, e2etest.

    • compare this widget that shows the jobs in master from a schedule_extended_test pipeline vs this second widget that lists the jobs in the MR branch run in a schedule_extended_test pipeline.
    • additionally, this widget lists three jobs that resp. use build, test and e2etest image. (no job using prebuild ran - these jobs are opam:* jobs).

  • NPM registry authentication worked: we did use our proxy in both runtime_datadog and e2etest:

npm silly packumentCache full:https://us-central1-npm.pkg.dev/nl-gitlab-runner/npm-registry/@datadog%2fdatadog-ci cache-miss
npm silly packumentCache full:https://us-central1-npm.pkg.dev/nl-gitlab-runner/npm-registry/eth-cli cache-miss
  • No datadog-ci or npm is installed in the Octez Docker image: it's a consequence of it being built on top of CI.runtime but if you want further proof, you can run:
% docker run --rm -it --entrypoint /bin/sh us-central1-docker.pkg.dev/nl-gitlab-runner/registry/tezos/tezos:bruno-dd-ci-runtime-clean 
Unable to find image 'us-central1-docker.pkg.dev/nl-gitlab-runner/registry/tezos/tezos:bruno-dd-ci-runtime-clean' locally
bruno-dd-ci-runtime-clean: Pulling from nl-gitlab-runner/registry/tezos/tezos
94e9d8af2201: Already exists 
1ee698eb3074: Pull complete 
728b256944aa: Pull complete 
a5b07fe1561e: Pull complete 
4f4fb700ef54: Pull complete 
9949625d351e: Pull complete 
34359a8c5cf3: Pull complete 
20a48dd52ba3: Pull complete 
Digest: sha256:1b1e155e74d4a0cd8bb5cfec195544368ea06c7b1ebe1eeb0e56e434560fbee3
Status: Downloaded newer image for us-central1-docker.pkg.dev/nl-gitlab-runner/registry/tezos/tezos:bruno-dd-ci-runtime-clean
~ $ npm --version
/bin/sh: npm: not found
~ $ datadog-ci --version
/bin/sh: datadog-ci: not found
~ $ exit

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR

Merge request reports