[go: up one dir, main page]

Fix cemented snapshot export

Victor Allombertrequested to merge
vicall@fix_cemented_snapshot_export into master

This MR aims to fix a snapshot export corner case that occurs while exporting a full snapshot targeting a block that is stored in the cemented store of a node that is the first block of a cycle -- likely to happen when exporting old snapshots from an archive node.

This issue was introduce by !18387 (merged) as it aims to set the savepoint to the predecessor of the imported head. This is possible as we keep the metadata in the floating store. However, when exporting a block from the cemented store, that is the first of the cycle, there is only cemented cycles exported, and thus no floating store, and thus no block metadata.

The MR aims to export a floating store containing the right metadata when hitting this corner case. As a side effect, the snapshot import is likely to be a bit longer to import (should not exceed several seconds max).

Manually testing the MR

Get an archive node directory (I recommend shadownet).

Export a snapshot of a block that is cemented (i.e, a block at least 2+ cycles in the past): for example, block 745201 from cycle 69.

$./octez-node snapshot export --block 745201 --data-dir ./shadownet_archive
Oct 29 14:49:40.488: initializing irmin context at ./shadownet_archive/context
Oct 29 14:49:41.049: exporting a snapshot (v9) in full mode, targeting block hash BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9 (level: 745201)
Oct 29 14:49:41.050: initializing irmin context at ./shadownet_archive/context
Exporting context Done
Copying cemented blocks and populating indexes: 70/70 cycles Done
Copying protocols: 3/3 Done
Oct 29 14:50:06.914: successful export: TEZOS_SHADOWNET_2025-08-07T20:00:00Z-BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9-745201.full-1

Import it

$./octez-node config init --data-dir ./shadownet_import --network https://teztnets.com/shadownet   
Oct 29 14:50:14.673: the node configuration has been successfully validated.
Created ./shadownet_import/config.json for network.

$./octez-node snapshot import ./TEZOS_SHADOWNET_2025-08-07T20:00:00Z-BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9-745201.full-1  --data-dir ./shadownet_import --no-check 
Oct 29 14:50:25.341: Warning: to speed up the import, the consistency of the imported data will not be fully checked. It is not recommended to use this option
Oct 29 14:50:25.341:   with a snapshot downloaded from an untrusted source
Oct 29 14:50:25.341: importing data from snapshot ./TEZOS_SHADOWNET_2025-08-07T20:00:00Z-BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9-745201.full-1:
Oct 29 14:50:25.341:   chain TEZOS_SHADOWNET_2025-08-07T20:00:00Z, block hash BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9 at level 745201, timestamp 2025-10-17T15:19:45-00:00 in full (snapshot version 9)
Oct 29 14:50:25.341: you may consider using the --block <block_hash> argument to ensure that the block imported is the one you expected
Oct 29 14:50:25.341: retrieving and validating data. This can take a while, please bear with us
Copying protocols: 3/3 Done
Importing context Done
Oct 29 14:50:25.531: initializing irmin context at ./shadownet_import/context
Copying cycles: 70/70 (100%) Done
Storing floating blocks: 0 blocks written Done
Oct 29 14:50:26.406: successful import from file ./TEZOS_SHADOWNET_2025-08-07T20:00:00Z-BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9-745201.full-1

Then, run a node or export a snapshot from it, expecting an error without this patch:

$./octez-node snapshot export TMP_SNAPSHOT --data-dir ./shadownet_import --block head --rolling    
Oct 29 14:50:28.221: initializing irmin context at ./shadownet_import/context
octez-node: Error:
              The block 'savepoint' (745200, BKoU4irFTS7e) metadata is unexpectedly missing from the store.

With this MR:

$./octez-node snapshot export TMP_SNAPSHOT --data-dir ./shadownet_import --block head --rolling  
Oct 29 15:14:09.722: cleaning leftover export directory 'TMP_SNAPSHOT_tmp'
Oct 29 15:14:09.724: initializing irmin context at ./shadownet_import/context
Oct 29 15:14:09.867: exporting a snapshot (v9) in rolling mode, targeting block hash BLKRqcKyXH2eBCMsU1DhUdeuje1iyVyM2fDjckxmU1tFDcdFnJ9 (level: 745201)
Oct 29 15:14:09.868: initializing irmin context at ./shadownet_import/context
Exporting context Done
Copying floating blocks: 451 blocks copied Done
Copying protocols: 2/2 Done
Oct 29 15:14:10.374: successful export: TMP_SNAPSHOT

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR

Merge request reports