[go: up one dir, main page]

Tezt_cloud: Add migration scenario for Layer1 test

Gabriel Moiserequested to merge
gabriel@tezt_cloud_agnostic_baker_migration into master

What

Parent MR: !17069 (merged)

This work enables the testing of a migration scenario that is done via a UAU for all the nodes participating in the private network.

Why

  1. We want to customize the layer 1 scenario even more, by adding a common case which causes issues to our lives on the chain
  2. Particularly, we want to see the agnostic baker succeed in migrating from protocol X to X+1 in a closer to real life scenario than simply the sandbox cases

How

This is done by adding a --migration-offset optional argument that, when mentioned, will trigger the migration scenario. In a nutshell, an entry is added in the config files of the nodes participating in the chain, which contains a user_activated_upgrade JSON entry that contains the level where the migration will happen (which will be the initial level from where we start from the snapshot + the migration_offset) and the next protocol that we upgrade to (which needs to be upgraded when we upgrade the ghostnet and mainnet chains to new protocols). This way, when we reach the migration level, the bakers will do the following:

  • protocol-dependent bakers : run the next protocol binary
  • protocol-agnostic bakers : automatically switch to the next protocol process

Manually testing the MR

The localhost command that I used is: (this can be tweaked to include more/less agnostic bakers and/or to tweak the migration level)

TEZT_CLOUD=debian dune exec tezt/tests/cloud/main.exe -- LAYER1 -v --log-file /tmp/log \
   --localhost \
   --stake 2,1,1 \
   --agnostic-bakers 0,2 \
   --migration-offset 20 \
   --network ghostnet \
   --faketime "-$(($(date --utc +%s) - $(date --date $(./octez-node snapshot info --json ./ghostnet.snapshot | jq -r .snapshot_header.timestamp) +%s)))" \
   --prometheus \
   --grafana \
   --website \
   --snapshot ./ghostnet.snapshot \
   --monitoring

In the logs, one can observe the following:

  • the baker-1 switches from baker-1-baker to baker-1-baker-next at the UAU level:
[15:41:33.129] [TOP] Add UAU entry for level : 11172470
...
[15:45:50.073] [baker-1-baker] Mar 14 08:06:00.073: received new proposal BMaLFWd4zhSQmXLAQd9SrhSDzxifE7PxPRby9GkMyLueCkgk2uG at
[15:45:50.073] [baker-1-baker] Mar 14 08:06:00.073:   level 11172469, round 0
...
[15:45:54.527] [baker-1] Mar 14 08:06:04.527: the protocol table was updated: protocol PsRiotumaAMo (level 12) was
[15:45:54.527] [baker-1] Mar 14 08:06:04.527:   activated on block BL1hHJEt7er92E9RCnjJN7anacm8P2DzyPeLNYGQCkoTpku2eA8
[15:45:54.527] [baker-1] Mar 14 08:06:04.527:   (level 11172470)
[15:45:54.534] [baker-1] Mar 14 08:06:04.533: head is now BL1hHJEt7er92E9RCnjJN7anacm8P2DzyPeLNYGQCkoTpku2eA8 (11172470)
[15:45:54.536] [baker-1-baker-next] pre-emptive-forge-time optimization set to 0.600000s. Operation inclusion window is ~3.400000s. Caution: Setting this too high may result in reduced block proposal rewards.
[15:45:54.536] [baker-1-baker-next] Baker 21.0~rc3+dev (1437a95a) for PsRiotumaAMo started.
[15:45:54.537] [TOP] init_baker_next: baker-1-baker-next is ready!
[15:45:54.537] [curl#10] curl -XPOST http://localhost:9090/-/reload
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537: Baker will run with the following delegates:@[<v
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:   2>@,
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1M9b1sKoh5GB7AiBiqdMrjMMyvuhsZua5n (tz1M9b1sKoh5GB7AiBiqdMrjMMyvuhsZua5n)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1N29q5T3jJ2i1JEWHax7q1NRkDMADj6fof (tz1N29q5T3jJ2i1JEWHax7q1NRkDMADj6fof)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1PZY3tEWmXGasYeehXYqwXuw2Z3iZ6QDnA (tz1PZY3tEWmXGasYeehXYqwXuw2Z3iZ6QDnA)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1QuqvzKmUNE4KzS59vPHrvDPS3JhYsKYcm (tz1QuqvzKmUNE4KzS59vPHrvDPS3JhYsKYcm)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1ViGF3WbG8gZH6w2oizRjJkyi43BtGr1ma (tz1ViGF3WbG8gZH6w2oizRjJkyi43BtGr1ma)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1VigxFsaiceHojGNfEKMsPHuL8sxK4t7jZ (tz1VigxFsaiceHojGNfEKMsPHuL8sxK4t7jZ)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1XMiZwHpHZ8a1AfwRWKfzLskJgZNyV8PHs (tz1XMiZwHpHZ8a1AfwRWKfzLskJgZNyV8PHs)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1YzMXTGVr1o9WCHPHKvnuQZk2kxEitnLNn (tz1YzMXTGVr1o9WCHPHKvnuQZk2kxEitnLNn)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1b5zY87MngtSCEH9w6Dxqoq5tFo2iHbeGz (tz1b5zY87MngtSCEH9w6Dxqoq5tFo2iHbeGz)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz1cg5EqC3WdZgRSvGJeW328S4KQNrT4jvyv (tz1cg5EqC3WdZgRSvGJeW328S4KQNrT4jvyv)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz2FVY3z8GeFh153BkrTNz8DnEHWcqzKwr3i (tz2FVY3z8GeFh153BkrTNz8DnEHWcqzKwr3i)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3MfU16Z8hTWwixZUX4zYMbVVa5oiFKBiT5 (tz3MfU16Z8hTWwixZUX4zYMbVVa5oiFKBiT5)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3PCcdoqnh7BFNXvoo52tLdFqzbrYAcoMNz (tz3PCcdoqnh7BFNXvoo52tLdFqzbrYAcoMNz)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3ZMMXjiMSLnqfRDaZmXJwTjnLSrqHvWR7P (tz3ZMMXjiMSLnqfRDaZmXJwTjnLSrqHvWR7P)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3bqiV6sRKd6NLaC9zf79makonFHxfiZhKK (tz3bqiV6sRKd6NLaC9zf79makonFHxfiZhKK)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3btDQsDkqq2G7eBdrrLqetaAfLVw6BnPez (tz3btDQsDkqq2G7eBdrrLqetaAfLVw6BnPez)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3eP8NTdMvV1GxNqA1E6uYFX6KTWpRPLY16 (tz3eP8NTdMvV1GxNqA1E6uYFX6KTWpRPLY16)
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.537:         tz3gWm2Moxurj6dRD1nhwpyv4tUR4MxcJzC4 (tz3gWm2Moxurj6dRD1nhwpyv4tUR4MxcJzC4)@]
[15:45:54.559] [baker-1-baker-next] Mar 14 08:06:04.541: initializing irmin context at /tmp/tezt-555783/1/baker-1/context
...
  • the agnostic bakers switched correctly to the new protocol:
[15:45:54.018] [baker-0] Mar 14 08:06:04.018: the protocol table was updated: protocol PsRiotumaAMo (level 12) was
[15:45:54.018] [baker-0] Mar 14 08:06:04.018:   activated on block BL1hHJEt7er92E9RCnjJN7anacm8P2DzyPeLNYGQCkoTpku2eA8
[15:45:54.018] [baker-0] Mar 14 08:06:04.018:   (level 11172470)
[15:45:54.024] [baker-0] Mar 14 08:06:04.023: head is now BL1hHJEt7er92E9RCnjJN7anacm8P2DzyPeLNYGQCkoTpku2eA8 (11172470)
[15:45:54.024] [baker-0-agnostic-baker] Mar 14 08:06:04.024: new block on proposal period (remaining period duration 7098)
[15:45:54.025] [baker-0-agnostic-baker] Mar 14 08:06:04.026: the active protocol PsRiotumaAMo was encountered
[15:45:54.026] [baker-0-agnostic-baker] Mar 14 08:06:04.026: stopping baker for protocol PsQuebecnLBy
[15:45:54.027] [baker-0-agnostic-baker] Error:
[15:45:54.027] [baker-0-agnostic-baker]   Lwt.Resolution_loop.Canceled
[15:45:54.027] [baker-0-agnostic-baker] Mar 14 08:06:04.027: starting baker for protocol PsRiotumaAMo with arguments:
[15:45:54.027] [baker-0-agnostic-baker] Mar 14 08:06:04.027:   "--endpoint http://127.0.0.1:30052 --base-dir
[15:45:54.027] [baker-0-agnostic-baker] Mar 14 08:06:04.027: /tmp/tezt-5557[...]"
[15:45:54.027] [baker-0-agnostic-baker] Mar 14 08:06:04.027: baker for protocol PsRiotumaAMo is now running
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.029: read liquidity baking toggle vote = pass
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.029: read adaptive issuance vote = pass
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.030: No DAL node endpoint has been provided.
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.030: Not running a DAL node might result
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.030:   in losing a share of the participation rewards.
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.030: For instructions on how to
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.030:   run a DAL node, please visit
[15:45:54.029] [baker-0-agnostic-baker] Mar 14 08:06:04.030:   https://docs.tezos.com/tutorials/join-dal-baker.
[15:45:54.031] [baker-0-agnostic-baker] Node is bootstrapped.
[15:45:54.031] [baker-0-agnostic-baker] Waiting for protocol 022-PsRiotum to start...
[15:45:54.034] [baker-0-agnostic-baker] pre-emptive-forge-time optimization set to 0.600000s. Operation inclusion window is ~3.400000s. Caution: Setting this too high may result in reduced block proposal rewards.
[15:45:54.034] [baker-0-agnostic-baker] Baker 21.0~rc3+dev (1437a95a) for PsRiotumaAMo started.
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034: Baker will run with the following delegates:@[<v
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:   2>@,
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1LZVSPJw5taDFBVjvcQMUhwehdWgatKbzK (tz1LZVSPJw5taDFBVjvcQMUhwehdWgatKbzK)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1MbwKSdbL5qyqhVNw9kafZjMVZKuM7EjHj (tz1MbwKSdbL5qyqhVNw9kafZjMVZKuM7EjHj)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1NQze1hyYH86rzFxoAmb5vQvVdZwfDaQzL (tz1NQze1hyYH86rzFxoAmb5vQvVdZwfDaQzL)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1NiaviJwtMbpEcNqSP6neeoBYj8Brb3QPv (tz1NiaviJwtMbpEcNqSP6neeoBYj8Brb3QPv)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1UMMZHpwmrQBHjAdwcL8uMe3fjZSecEz3F (tz1UMMZHpwmrQBHjAdwcL8uMe3fjZSecEz3F)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1VicarbXb4uAY12pQBAREHUimYCxjfKbLa (tz1VicarbXb4uAY12pQBAREHUimYCxjfKbLa)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1XUkUxMUGJJ8ZAH82iv3Tx5j4eqsJBLNh8 (tz1XUkUxMUGJJ8ZAH82iv3Tx5j4eqsJBLNh8)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1Zt8QQ9aBznYNk5LUBjtME9DuExomw9YRs (tz1Zt8QQ9aBznYNk5LUBjtME9DuExomw9YRs)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1ccniXhg7WdHVJfvnXMVMihxitQTijQeEd (tz1ccniXhg7WdHVJfvnXMVMihxitQTijQeEd)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1e13WxtEt7v3pEcNJtDrhPQyUNWZ53B4Hu (tz1e13WxtEt7v3pEcNJtDrhPQyUNWZ53B4Hu)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz1e7uhpwmiKp8Yd2KwFwmVhoT31L478KUe3 (tz1e7uhpwmiKp8Yd2KwFwmVhoT31L478KUe3)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3PZ8RqWWBSTfhRfZnYuEsdoe5W5n3amhmQ (tz3PZ8RqWWBSTfhRfZnYuEsdoe5W5n3amhmQ)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3RtcGP6G6DFUWMXbdsNvRVe9UuyQoQ1T5b (tz3RtcGP6G6DFUWMXbdsNvRVe9UuyQoQ1T5b)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3Vgidaye1PPYGbmREaxVrV6jwBQUaPoSeg (tz3Vgidaye1PPYGbmREaxVrV6jwBQUaPoSeg)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3ZmB8oWUmi8YZXgeRpgAcPnEMD8VgUa4Ve (tz3ZmB8oWUmi8YZXgeRpgAcPnEMD8VgUa4Ve)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3aYdfwVMHSdXjRfFDNR9ZFS1rqpMPj9uCp (tz3aYdfwVMHSdXjRfFDNR9ZFS1rqpMPj9uCp)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3dWN9k5k3J58NRsA85oyLKko7Vyjr6yXhm (tz3dWN9k5k3J58NRsA85oyLKko7Vyjr6yXhm)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3e7XBCKzh9PivTHepApVmxy92bfTaysS38 (tz3e7XBCKzh9PivTHepApVmxy92bfTaysS38)
[15:45:54.034] [baker-0-agnostic-baker] Mar 14 08:06:04.034:         tz3f86ncuuXnaGBZZAXi8CDrQ4ASVgoFUT7p (tz3f86ncuuXnaGBZZAXi8CDrQ4ASVgoFUT7p)@]
[15:45:54.038] [baker-0-agnostic-baker] Mar 14 08:06:04.038: initializing irmin context at /tmp/tezt-555783/1/baker-0/context
...

ISSUE: The stresstest breaks unfortunately, I am not sure why, but this is independent of the addition of the agnostic bakers option, but this should not be blocking on the MR, because for now we can test it without the stresstest option. Here is the issue I am facing when I also enable stresstesting:

      TEZT_CLOUD=debian dune exec tezt/tests/cloud/main.exe -- LAYER1 -v --log-file /tmp/log \
         --localhost \
         --stake 2,1,1 \
         --agnostic-bakers 0,2 \
         --migration-offset 20 \
         --network ghostnet \
         --stresstest tz1KhnTgwoRRALBX6vRHRnydDGSBFsWtcJxc/edpkuFRauFAdhipQu9s4xmfNJWmtLxPKpoaoG41gYGq5AgUA43Vxqx/100/1 \
         --faketime "-$(($(date --utc +%s) - $(date --date $(./octez-node snapshot info --json ./ghostnet.snapshot | jq -r .snapshot_header.timestamp) +%s)))" \
         --prometheus \
         --grafana \
         --website \
         --snapshot ./ghostnet.snapshot \
         --monitoring

gets to this error even it progresses beyond the migration level (I had it run up to level 11172506, so 46 more levels after migration, but there were some high round blocks in that period):

[14:56:46.893] [stresstest-0-client] Node is bootstrapped.
[14:56:46.914] [stresstest-0-client] This simulation failed (force = false):
[14:56:46.914] [stresstest-0-client]   Manager signed operations:
[14:56:46.914] [stresstest-0-client]     From: tz1KhnTgwoRRALBX6vRHRnydDGSBFsWtcJxc
[14:56:46.914] [stresstest-0-client]     Fee to the baker: ꜩ0.001
[14:56:46.914] [stresstest-0-client]     Expected counter: 102350
[14:56:46.914] [stresstest-0-client]     Gas limit: 1040
[14:56:46.914] [stresstest-0-client]     Storage limit: 257 bytes
[14:56:46.914] [stresstest-0-client]     Balance updates:
[14:56:46.914] [stresstest-0-client]       tz1KhnTgwoRRALBX6vRHRnydDGSBFsWtcJxc ... -ꜩ0.001
[14:56:46.914] [stresstest-0-client]       payload fees(the block proposer) ....... +ꜩ0.001
[14:56:46.914] [stresstest-0-client]     Transaction:
[14:56:46.914] [stresstest-0-client]       Amount: ꜩ151015.1
[14:56:46.914] [stresstest-0-client]       From: tz1KhnTgwoRRALBX6vRHRnydDGSBFsWtcJxc
[14:56:46.914] [stresstest-0-client]       To: tz1iqxcfKxAFxP8VgcGakfCGjbe52WbYfE5v
[14:56:46.914] [stresstest-0-client]       This operation FAILED.
[14:56:46.914] [stresstest-0-client] Error:
[14:56:46.914] [stresstest-0-client]   The transfer simulation failed.
[14:56:46.914] [stresstest-0-client]   { "id": "proto.022-PsRiotum.gas_exhausted.operation",
[14:56:46.914] [stresstest-0-client]   "description":
[14:56:46.914] [stresstest-0-client]     "A script or one of its callee took more time than the operation said it would",
[14:56:46.914] [stresstest-0-client]   "data": {} }

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Gabriel Moise

Merge request reports