[go: up one dir, main page]

Store: Create new store version to upgrade cemented files offsets

What

This is an upgrade procedure which would increase the store version from v3.1 to v3.2 and it would consist in upgrading the format of the offsets in the cemented_blocks_files from a 32-bit format to a 64-bit format.

Why

This is motivated by the following issue and present as part of a goal of the following milestone.

How

The upgrade works based on the following steps:

  • obtain the cemented_blocks_files directory
  • obtain the cemented_blocks_files (in case of an non-existent/empty directory, we are in the case of a rolling node, so there is nothing to be done)
  • for each cemented_block_file:
    • create a corresponding cemented_block_file_upgraded version
    • retrieve the list of (say N) offsets (which have a 32-bit format)
    • transform them into 64-bit format
    • add to each of them 4 bytes x N because when we will store these upgraded offsets, they will occupy more space at the beginning of the file, and there is 4 more bytes for each of these offsets, therefore we shift the blocks by 4N bytes
    • write these upgraded offsets at the beginning of the cemented_block_file_upgraded file
    • copy the blocks information from the cemented_block_file to its upgraded counterpart
    • remove the old cemented file
    • rename the upgraded file to the old cemented file name

This MR also removes the automatic upgrade of the store when the node is ran, so the node operator needs to do an explicit storage upgrade in order to continue running the node.

Manually testing the MR

Start by running a sandbox node for a few minutes (on master), so that cycles can be stored:

$ DATA_DIR=~/tezos/tmp_node/ ./src/bin_node/octez-sandboxed-node.sh 1 --connections 0 --history-mode full

In another terminal, we start baking:

$ eval `./src/bin_client/octez-init-sandboxed-client.sh 1`

$ octez-activate-alpha

$ octez-baker-alpha run with local node tmp_node

After a few minutes, we will have a lot of cemented files in tmp_node/store/chain_NetXdQprcVkpa/cemented.

Now, on this branch, we stop the node and the baker, we run make and then we upgrade the storage:

$ ./octez-node upgrade storage --data-dir ~/tezos/tmp_node/

This should work without error, and should increase the size of all cemented files by 4 * nb_blocks (as we have 4 more bytes for each offset):

$ stat -f%z tmp_node_backup/store/chain_NetXdQprcVkpa/cemented/1514_1521
10402

$ stat -f%z tmp_node/store/chain_NetXdQprcVkpa/cemented/1514_1521
10434

In the example above, we have 8 blocks in a cycle, so we increase the size with 32 bytes. Because we use a function to decide whether each cemented file uses a 32 or 64 bit format, there is no issue with sudden interruptions of the upgrade procedure, as when we resume, the process will take into account all the files again, and check if they need upgraded or not.

Edited by Gabriel Moise

Merge request reports

Loading