[go: up one dir, main page]

EVM Node: Support transparent tracing of workers with OpenTelemetry

Part of L2-2

What

This commit adds a wrapper around Tezos_workers.Worker.MakeSingle in Octez_telemetry. This wrapper enables automatic OpenTelemetry tracing for worker requests, capturing details like request initiation, completion, and errors. It also adapts various EVM node workers to utilize this new tracing mechanism.

Why

The primary motivation for this change is to significantly improve the observability of Octez worker operations. By integrating OpenTelemetry, developers and operators can gain deeper insights into the execution flow, identify performance bottlenecks, and facilitate debugging of asynchronous worker tasks within the EVM node. This enhances the ability to monitor and understand the system's behavior in production.

The introduction of a dedicated library instead of contributing to Tezos_workers is motivated by the lack of perspective we have on the cost of instrumentation. Besides and contrary to the current approach used by the Octez nodes, the EVM node releases will have the OpenTelemetry instrumentation. That is, it will be possible to opt-in exporting traces in production. In order to not affect the Tezos L1 binaries while we conduct the necessary experiments to validate this choice, it is reasonable not to modify the shared codebase.

How

A new Worker module is added to Octez_telemetry. This module exports a MakeSingle functor that takes a Request module and creates an instrumented worker. The new worker transparently creates OpenTelemetry spans for each request, attaching relevant attributes. The Request modules of existing EVM node workers (e.g., block producer, blueprint publisher, tx queue) are extended with a name function to provide descriptive span names. The build system configurations (dune, manifest, opam) are updated to include the new library and its dependencies.

Manually testing the MR

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Thomas Letan

Merge request reports

Loading