From e41ba9e66dee586a20943269de8c4866d592c137 Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Mon, 21 Nov 2022 12:38:42 +0100 Subject: [PATCH 01/12] doc: include the in-use numbered protocols in odoc-lite --- docs/Makefile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/Makefile b/docs/Makefile index c4b3c88cad26..02e49672be03 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -69,7 +69,9 @@ odoc: docexes odoc-lite: docexes rm -rf $(TMPDOCDIR)/ mkdir -p $(TMPDOCDIR)/ - rsync --recursive --links --perms --exclude="src/proto_0*" \ + rsync --recursive --links --perms \ + --include="src/proto_014*" --include="src/proto_015*" \ + --exclude="src/proto_0*" \ ../src ../tezt ../vendors ../dune ../dune-project $(TMPDOCDIR)/ # FIXME: https://gitlab.com/tezos/tezos/-/issues/2971 # The new version of odoc (2.1.0) is stricter than the old version (1.5.3), -- GitLab From c67670037bf19ceb88d66efdaac2086bb6fba548 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Thu, 6 Oct 2022 01:12:37 +0200 Subject: [PATCH 02/12] doc: fork operations and validation entry for the protocol doc Advances #3162, #3921 - Forks a section to describe the content of blocks and operations for each economic protocol. - Forks a section to describe the validation and application of blocks and operations for each economic protocol. - Updates protocol overview section accordingly. - Impmemented for Kathmandu, Lima and Alpha docs --- docs/alpha/blocks_ops.rst | 33 ++++++ docs/alpha/consensus.rst | 5 + docs/alpha/protocol.rst | 10 ++ docs/alpha/protocol_overview.rst | 160 ++++++++++++++++---------- docs/alpha/validation.rst | 48 ++++++++ docs/kathmandu/blocks_ops.rst | 33 ++++++ docs/kathmandu/consensus.rst | 5 + docs/kathmandu/protocol.rst | 12 +- docs/kathmandu/protocol_overview.rst | 164 ++++++++++++++++----------- docs/kathmandu/validation.rst | 48 ++++++++ docs/lima/blocks_ops.rst | 33 ++++++ docs/lima/consensus.rst | 5 + docs/lima/protocol.rst | 12 +- docs/lima/protocol_overview.rst | 161 +++++++++++++++----------- docs/lima/validation.rst | 48 ++++++++ 15 files changed, 580 insertions(+), 197 deletions(-) create mode 100644 docs/alpha/blocks_ops.rst create mode 100644 docs/alpha/validation.rst create mode 100644 docs/kathmandu/blocks_ops.rst create mode 100644 docs/kathmandu/validation.rst create mode 100644 docs/lima/blocks_ops.rst create mode 100644 docs/lima/validation.rst diff --git a/docs/alpha/blocks_ops.rst b/docs/alpha/blocks_ops.rst new file mode 100644 index 000000000000..46fcf15f12f4 --- /dev/null +++ b/docs/alpha/blocks_ops.rst @@ -0,0 +1,33 @@ +===================== +Blocks and Operations +===================== + +.. _validation_passes_alpha: + +Validation Passes +~~~~~~~~~~~~~~~~~ + +.. _consensus_operations_alpha: + +Consensus Operations +~~~~~~~~~~~~~~~~~~~~ + +.. _voting_operations_alpha: + +Voting Operations +~~~~~~~~~~~~~~~~~ + +.. _anonymous_operations_alpha: + +Anonymous Operations +~~~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_alpha: + +Manager Operations +~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_batches_alpha: + +Manager Operation Batches +""""""""""""""""""""""""" diff --git a/docs/alpha/consensus.rst b/docs/alpha/consensus.rst index 0ad1937e27da..2816d8507a10 100644 --- a/docs/alpha/consensus.rst +++ b/docs/alpha/consensus.rst @@ -441,6 +441,11 @@ These are a subset of the :ref:`protocol constants `. Shell-protocol interaction revisited ------------------------------------ +.. FIXME tezos/tezos#3914: + + Integrate protocol-specific block parts in the blocks and ops + entry. + :ref:`Recall` that, for the shell to interact with the economic protocol, two notions are defined abstractly at the level of the shell and made concrete at the level of the consensus protocol. Namely, these two notions are the protocol-specific header and the fitness. As in Emmy*, the protocol-specific header contains the fields: diff --git a/docs/alpha/protocol.rst b/docs/alpha/protocol.rst index ddd4a330ab8b..f3974f59c9c3 100644 --- a/docs/alpha/protocol.rst +++ b/docs/alpha/protocol.rst @@ -90,3 +90,13 @@ Sapling, etc), and some details about its implementation. :maxdepth: 2 event + +.. toctree:: + :maxdepth: 2 + + blocks_ops + +.. toctree:: + :maxdepth: 2 + + validation diff --git a/docs/alpha/protocol_overview.rst b/docs/alpha/protocol_overview.rst index 4760a194aca1..15c9bbf06da1 100644 --- a/docs/alpha/protocol_overview.rst +++ b/docs/alpha/protocol_overview.rst @@ -37,97 +37,131 @@ different implementations, and this is important, for instance because different to implement different transaction selection strategies. Tezos is a self-amending blockchain, in that a large part of Tezos can be -changed through a so-called amendement procedure. To this end, as mentioned in -:doc:`the big picture<../shell/the_big_picture>`, a Tezos node consists of two +changed through a so-called amendment procedure. To this end, as mentioned in +:ref:`the big picture`, a Tezos node consists of two components: - the shell, which comprises the network and storage layer, and embeds - the economic protocol component, which is the part that can be changed through amendment. -The role of the protocol -~~~~~~~~~~~~~~~~~~~~~~~~ +The role of the economic protocol +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Update for pipelined validation up to Lima. At a very high level, a protocol must: -- implement protocol-specific types, such as the type of operations or protocol-specific block header data (in addition to the shell generic header), -- define under which conditions a block is a valid extension of the current blockchain, and define an ordering on blocks to arbitrate between concurrent extensions. +- implement protocol-specific types, such as the type of operations or + protocol-specific block header data (in addition to the shell + generic header), + +- define under which conditions a block is a valid extension of the + current blockchain, and define an ordering on blocks to arbitrate + between concurrent extensions. -Validity conditions are implemented in the ``apply`` function which is called -whenever the node processes a block. The ``apply`` function takes as arguments a -*context* and a block. The context represents the *protocol state* and is -therefore protocol specific. The context may contain, for instance, a list of -accounts and their balances. More generally, the context must provide enough -information to determine the validity of a block. Given a context and a block, -the ``apply`` function returns the updated context if the block is valid and has -a higher :ref:`fitness`. The fitness determines a total ordering between blocks. +Validity conditions are implemented in the ``apply`` function which is +called whenever the node processes a block---see the dedicated +:doc:`protocol validation and operation` entry for further +detail into the validation and application process for +:ref:`blocks` and their +:ref:`operations`. .. _shell_proto_interact_alpha: Shell-protocol interaction ~~~~~~~~~~~~~~~~~~~~~~~~~~ -:doc:`Recall<../shell/the_big_picture>` that the economic protocol and the shell interact in order to ensure that the blocks being appended to the blockchain are valid. There are mainly two rules that the shell uses when receiving a new block: - -- The shell does not accept a branch whose fork point is in a cycle more than ``PRESERVED_CYCLES`` in the past. More precisely, if ``n`` is the current cycle, :ref:`the last allowed fork point` is the first level of cycle ``n-PRESERVED_CYCLES``. The parameter ``PRESERVED_CYCLES`` therefore plays a central role in Tezos: any block before the last allowed fork level is immutable. -- The shell changes the head of the chain to this new block only if the block is :doc:`valid<../shell/validation>` and has a higher fitness than the current head; a block is valid if the operations it includes are valid. - -The support provided by the protocol for validating blocks can be modulated by different :package-api:`validation modes `. -They allow using this same support for quite different use cases, as follows: - -- being able to validate a block, typically used in the :doc:`validator <../shell/validation>`; -- being able to pre-apply a block, typically used in the :doc:`validator <../shell/validation>` to precheck a block, avoiding to further consider invalid blocks; -- being able to construct a block, typically used by the baker to bake a block; -- being able to partially construct a block, typically used by the :doc:`prevalidator <../shell/prevalidation>` to determine valid operations in the mempool. - -Blocks -~~~~~~ +In the Tezos :ref:`architecture`, the economic +protocol and the shell interact in order to ensure that the blocks +being appended to the blockchain are valid. There are mainly two rules +that the shell uses when receiving a new block: + +- The shell does not accept a branch whose fork point is in a cycle + more than ``PRESERVED_CYCLES`` in the past. More precisely, if ``n`` + is the current cycle, :ref:`the last allowed fork point` is + the first level of cycle ``n-PRESERVED_CYCLES``. The parameter + ``PRESERVED_CYCLES`` therefore plays a central role in Tezos: any + block before the last allowed fork level is immutable. +- The shell changes the head of the chain to this new block only if + the block is :doc:`valid<../shell/validation>`, and it has a higher + fitness than the current head; a block is + :ref:`valid` only if all the + operations included are also + :ref:`valid`. + +The support provided by the protocol for validating blocks can be +modulated by different :ref:`validation +modes`. They allow using this same +interface for quite different use cases, as follows: + +- being able to :ref:`apply` a block, + typically used by the shell's :doc:`validator <../shell/validation>` + component; +- being able to :ref:`construct` a block, + typically used by the baker daemon to *bake* -- that is, to produce + -- a new block; +- being able to :ref:`partially construct` + a block, typically used by the :doc:`prevalidator + <../shell/prevalidation>` to determine valid operations in the + mempool; and, +- being able to :ref:`pre-apply` a + block, typically used in the :doc:`validator <../shell/validation>` + to precheck a block, avoiding to further consider invalid blocks. + +.. _block_contents_alpha: + +Blocks, Operations and their Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3914: + + Integrate protocol-specific block parts in the blocks and ops + entry. A block consists of a header and operations. A block's header is composed of two parts: :ref:`the protocol-agnostic part` and :ref:`the protocol-specific part`. This separation enables the shell to interact with different -protocols. - -.. _validation_passes_alpha: - -Operations & Validation Passes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The different kinds of operations are grouped in classes, such that operations belonging to different classes may be validated independently, and/or with different priorities. -Each class has an associated index, called a :ref:`validation pass`. -There are four classes of operations: :doc:`consensus ` operations, :doc:`voting ` operations, anonymous operations, manager operations. - -Consensus operations are endorsements, while `voting ` operations are ballot and proposal. - -Anonymous operations are operations which are not signed. There are three anonymous operations: seed nonce revelation, double baking evidence, and double endorsing evidence. The evidence for double baking and double endorsing is included in a block by the so-called accuser (see :ref:`slashing`). - -Manager operations are activation, origination (see :doc:`smart contracts`), transaction, reveal, and delegation (see :doc:`proof of stake `). Manager operations are the only fee-paying operations. - -Recall that users have associated :ref:`accounts ` which they activate before being able to participate. By means of the operation :ref:`origination`, accounts can be further associated with smart contracts in which they are called :ref:`originated accounts`. :ref:`Transactions` are used to either transfer tez between two accounts or run the code of a smart contract. Transactions are signed by an account's private key. Before making a transaction, a user must reveal her public key so that other users (not being aware of this public key) can effectively check the signature of the transaction. - -Manager operations can be grouped into batches forming a so-called group operation. A group operation satisfies: - -- atomicity: either all the operations in the batch succeed or none is applied -- efficiency: the whole batch is signed only once (by the same implicit account), thus it is much more efficient to check, and it requires much less gas -- usability: the batch only increments the counter of the signer account by one; for this reason it is easier for tools to provide sending several operations per block using operation batches than tracking counter changes. - -The list of operations can be obtained with :ref:`this rpc `. +protocols. Each Tezos economic protocol can specify different kinds of +operations, which are described further in detail in +:doc:`./blocks_ops`. + +The semantics of, respectively, operations and blocks is indeed also +dependent on each economic protocol. The :doc:`Validation and +Application` entry explains the internals of *validation* +-- that is, how to determine whether operations and blocks can be +safely be included in the Tezos blockchain -- and *application* -- +that is, how the effects of operations and blocks are taken into +account -- for this economic protocol. .. _protocol_constants_alpha: Protocol constants ~~~~~~~~~~~~~~~~~~ -Protocols are tuned by several *protocol constants*, such as the size of a nonce, or the number of blocks per cycle. -One can distinguish two kinds of protocol constants: +Protocols are tuned by several *protocol constants*, such as the size +of a nonce, or the number of blocks per cycle. One can distinguish two +kinds of protocol constants: + +- *fixed* protocol constants, such as the size of a nonce, are values + wired in the code of a protocol, and can only be changed by protocol + amendment (that is, by adopting a new protocol) -- *fixed* prototocol constants, such as the size of a nonce, are values wired in the code of a protocol, and can only be changed by protocol amendment (that is, by adopting a new protocol) -- *parametric* protocol constants, such as the number of blocks per cycle, are values maintained in a read-only data structure that can be instantiated differently, for the same protocol, from one network to another (for instance, test networks move faster). +- *parametric* protocol constants, such as the number of blocks per + cycle, are values maintained in a read-only data structure that can + be instantiated differently, for the same protocol, from one network + to another (for instance, test networks move faster). The *list* of protocol constants can be found in the OCaml APIs: -- fixed protocol constants are defined in the module :package-api:`Constants_repr ` -- parametric constants are defined in the module :package-api:`Constants_parametric_repr ` +- fixed protocol constants are defined in the module + :package-api:`Constants_repr + ` +- parametric constants are defined in the module + :package-api:`Constants_parametric_repr + ` The *values* of protocol constants in any given protocol can be found using specific RPC calls: @@ -144,6 +178,6 @@ See, for example: See also ~~~~~~~~ -An in-depth description of the inners of a protocol can be found in the blog -post `How to write a Tezos protocol +An in-depth description of the internals of developing a new Tezos +protocol can be found in the blog post: `How to write a Tezos protocol `_. diff --git a/docs/alpha/validation.rst b/docs/alpha/validation.rst new file mode 100644 index 000000000000..f1eb7bce552a --- /dev/null +++ b/docs/alpha/validation.rst @@ -0,0 +1,48 @@ +========================== +Validation and Application +========================== + +.. _validation_modes_alpha: + +Validation modes +================ + +.. _full_application_alpha: + +Full Application +~~~~~~~~~~~~~~~~ + +.. _full_construction_alpha: + +Full Construction +~~~~~~~~~~~~~~~~~ + +.. _partial_construction_alpha: + +Partial Construction +~~~~~~~~~~~~~~~~~~~~ + +.. _partial_application_alpha: + +Partial Application +~~~~~~~~~~~~~~~~~~~ + +.. _block_validation_overview_alpha: + +Block Validation +================ + +.. _operation_validity_alpha: + +Operation Validation and Application +==================================== + +.. _manager_operations_validity_alpha: + +Validity of Manager Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_application_alpha: + +Application of Manager Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/kathmandu/blocks_ops.rst b/docs/kathmandu/blocks_ops.rst new file mode 100644 index 000000000000..5488272a3089 --- /dev/null +++ b/docs/kathmandu/blocks_ops.rst @@ -0,0 +1,33 @@ +===================== +Blocks and Operations +===================== + +.. _validation_passes_kathmandu: + +Validation Passes +~~~~~~~~~~~~~~~~~ + +.. _consensus_operations_kathmandu: + +Consensus Operations +~~~~~~~~~~~~~~~~~~~~ + +.. _voting_operations_kathmandu: + +Voting Operations +~~~~~~~~~~~~~~~~~ + +.. _anonymous_operations_kathmandu: + +Anonymous Operations +~~~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_kathmandu: + +Manager Operations +~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_batches_kathmandu: + +Manager Operation Batches +""""""""""""""""""""""""" diff --git a/docs/kathmandu/consensus.rst b/docs/kathmandu/consensus.rst index 29c7a90ed32f..ef9e48cbfbaf 100644 --- a/docs/kathmandu/consensus.rst +++ b/docs/kathmandu/consensus.rst @@ -448,6 +448,11 @@ Consensus related protocol parameters Shell-protocol interaction revisited ------------------------------------ +.. FIXME tezos/tezos#3914: + + Integrate protocol-specific block parts in the blocks and ops + entry. + :ref:`Recall` that, for the shell to interact with the economic protocol, two notions are defined abstractly at the level of the shell and made concrete at the level of the consensus protocol. Namely, these two notions are the protocol-specific header and the fitness. As in Emmy*, the protocol-specific header contains the fields: diff --git a/docs/kathmandu/protocol.rst b/docs/kathmandu/protocol.rst index 1e263d2cb045..57a36ccc1b4a 100644 --- a/docs/kathmandu/protocol.rst +++ b/docs/kathmandu/protocol.rst @@ -83,5 +83,15 @@ Sapling, etc), and some details about its implementation. .. toctree:: :maxdepth: 2 - + event + +.. toctree:: + :maxdepth: 2 + + blocks_ops + +.. toctree:: + :maxdepth: 2 + + validation diff --git a/docs/kathmandu/protocol_overview.rst b/docs/kathmandu/protocol_overview.rst index ef457b858ce3..5856df6a52d5 100644 --- a/docs/kathmandu/protocol_overview.rst +++ b/docs/kathmandu/protocol_overview.rst @@ -37,29 +37,32 @@ different implementations, and this is important, for instance because different to implement different transaction selection strategies. Tezos is a self-amending blockchain, in that a large part of Tezos can be -changed through a so-called amendement procedure. To this end, as mentioned in -:doc:`the big picture<../shell/the_big_picture>`, a Tezos node consists of two +changed through a so-called amendment procedure. To this end, as mentioned in +:ref:`the big picture`, a Tezos node consists of two components: - the shell, which comprises the network and storage layer, and embeds - the economic protocol component, which is the part that can be changed through amendment. -The role of the protocol -~~~~~~~~~~~~~~~~~~~~~~~~ +The role of the economic protocol +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ At a very high level, a protocol must: -- implement protocol-specific types, such as the type of operations or protocol-specific block header data (in addition to the shell generic header), -- define under which conditions a block is a valid extension of the current blockchain, and define an ordering on blocks to arbitrate between concurrent extensions. +- implement protocol-specific types, such as the type of operations or + protocol-specific block header data (in addition to the shell + generic header), -Validity conditions are implemented in the ``apply`` function which is called -whenever the node processes a block. The ``apply`` function takes as arguments a -*context* and a block. The context represents the *protocol state* and is -therefore protocol specific. The context may contain, for instance, a list of -accounts and their balances. More generally, the context must provide enough -information to determine the validity of a block. Given a context and a block, -the ``apply`` function returns the updated context if the block is valid and has -a higher :ref:`fitness`. The fitness determines a total ordering between blocks. +- define under which conditions a block is a valid extension of the + current blockchain, and define an ordering on blocks to arbitrate + between concurrent extensions. + +Validity conditions are implemented in the ``apply`` function which is +called whenever the node processes a block---see the dedicated +:doc:`protocol validation and operation` entry for further +detail into the validation and application process for +:ref:`blocks` and their +:ref:`operations`. .. _shell_proto_interact: .. _shell_proto_interact_kathmandu: @@ -67,53 +70,69 @@ a higher :ref:`fitness`. The fitness determines a total order Shell-protocol interaction ~~~~~~~~~~~~~~~~~~~~~~~~~~ -:doc:`Recall<../shell/the_big_picture>` that the economic protocol and the shell interact in order to ensure that the blocks being appended to the blockchain are valid. There are mainly two rules that the shell uses when receiving a new block: - -- The shell does not accept a branch whose fork point is in a cycle more than ``PRESERVED_CYCLES`` in the past. More precisely, if ``n`` is the current cycle, :ref:`the last allowed fork point` is the first level of cycle ``n-PRESERVED_CYCLES``. The parameter ``PRESERVED_CYCLES`` therefore plays a central role in Tezos: any block before the last allowed fork level is immutable. -- The shell changes the head of the chain to this new block only if the block is :doc:`valid<../shell/validation>` and has a higher fitness than the current head; a block is valid if the operations it includes are valid. - -The support provided by the protocol for validating blocks can be modulated by different :package-api:`validation modes `. -They allow using this same support for quite different use cases, as follows: - -- being able to validate a block, typically used in the :doc:`validator <../shell/validation>`; -- being able to pre-apply a block, typically used in the :doc:`validator <../shell/validation>` to precheck a block, avoiding to further consider invalid blocks; -- being able to construct a block, typically used by the baker to bake a block; -- being able to partially construct a block, typically used by the :doc:`prevalidator <../shell/prevalidation>` to determine valid operations in the mempool. - -Blocks -~~~~~~ +In the Tezos :ref:`architecture`, the economic +protocol and the shell interact in order to ensure that the blocks +being appended to the blockchain are valid. There are mainly two rules +that the shell uses when receiving a new block: + +- The shell does not accept a branch whose fork point is in a cycle + more than ``PRESERVED_CYCLES`` in the past. More precisely, if ``n`` + is the current cycle, :ref:`the last allowed fork point` is + the first level of cycle ``n-PRESERVED_CYCLES``. The parameter + ``PRESERVED_CYCLES`` therefore plays a central role in Tezos: any + block before the last allowed fork level is immutable. +- The shell changes the head of the chain to this new block only if + the block is :doc:`valid<../shell/validation>`, and it has a higher + fitness than the current head; a block is + :ref:`valid` only if all the + operations included are also + :ref:`valid`. + +The support provided by the protocol for validating blocks can be +modulated by different :ref:`validation +modes`. They allow using this same +interface for quite different use cases, as follows: + +- being able to :ref:`apply` a block, + typically used by the shell's :doc:`validator <../shell/validation>` + component; +- being able to :ref:`construct` a block, + typically used by the baker daemon to *bake* -- that is, to produce + -- a new block; +- being able to :ref:`partially construct` + a block, typically used by the :doc:`prevalidator + <../shell/prevalidation>` to determine valid operations in the + mempool; and, +- being able to :ref:`pre-apply` a + block, typically used in the :doc:`validator <../shell/validation>` + to precheck a block, avoiding to further consider invalid blocks. + +.. _block_contents: +.. _block_contents_kathmandu: + +Blocks, Operations and their Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3914: + + Integrate protocol-specific block parts in the blocks and ops + entry. A block consists of a header and operations. A block's header is composed of two parts: :ref:`the protocol-agnostic part` and :ref:`the protocol-specific part`. This separation enables the shell to interact with different -protocols. - -.. _validation_passes: -.. _validation_passes_kathmandu: - -Operations & Validation Passes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The different kinds of operations are grouped in classes, such that operations belonging to different classes may be validated independently, and/or with different priorities. -Each class has an associated index, called a :ref:`validation pass`. -There are four classes of operations: :doc:`consensus ` operations, :doc:`voting ` operations, anonymous operations, manager operations. - -Consensus operations are endorsements, while `voting ` operations are ballot and proposal. - -Anonymous operations are operations which are not signed. There are three anonymous operations: seed nonce revelation, double baking evidence, and double endorsing evidence. The evidence for double baking and double endorsing is included in a block by the so-called accuser (see :ref:`slashing`). - -Manager operations are activation, origination (see :doc:`smart contracts`), transaction, reveal, and delegation (see :doc:`proof of stake `). Manager operations are the only fee-paying operations. - -Recall that users have associated :ref:`accounts ` which they activate before being able to participate. By means of the operation :ref:`origination`, accounts can be further associated with smart contracts in which they are called :ref:`originated accounts`. :ref:`Transactions` are used to either transfer tez between two accounts or run the code of a smart contract. Transactions are signed by an account's private key. Before making a transaction, a user must reveal her public key so that other users (not being aware of this public key) can effectively check the signature of the transaction. - -Manager operations can be grouped into batches forming a so-called group operation. A group operation satisfies: - -- atomicity: either all the operations in the batch succeed or none is applied -- efficiency: the whole batch is signed only once (by the same implicit account), thus it is much more efficient to check, and it requires much less gas -- usability: the batch only increments the counter of the signer account by one; for this reason it is easier for tools to provide sending several operations per block using operation batches than tracking counter changes. - -The list of operations can be obtained with :ref:`this rpc `. +protocols. Each Tezos economic protocol can specify different kinds of +operations, which are described further in detail in +:doc:`./blocks_ops`. + +The semantics of, respectively, operations and blocks is indeed also +dependent on each economic protocol. The :doc:`Validation and +Application` entry explains the internals of *validation* +-- that is, how to determine whether operations and blocks can be +safely be included in the Tezos blockchain -- and *application* -- +that is, how the effects of operations and blocks are taken into +account -- for this economic protocol. .. _protocol_constants: .. _protocol_constants_kathmandu: @@ -121,16 +140,27 @@ The list of operations can be obtained with :ref:`this rpc ` -- parametric constants are defined in the module :package-api:`Constants_parametric_repr ` +- fixed protocol constants are defined in the module + :package-api:`Constants_repr + ` +- parametric constants are defined in the module + :package-api:`Constants_parametric_repr + ` The *values* of protocol constants in any given protocol can be found using specific RPC calls: @@ -140,13 +170,13 @@ The *values* of protocol constants in any given protocol can be found using spec Further documentation of various protocol constants can be found in the subsystems where they conceptually belong. See, for example: -- :ref:`proof-of-stake parameters `. -- :ref:`consensus-related parameters ` -- :ref:`randomness generation parameters `. +- :ref:`proof-of-stake parameters `. +- :ref:`consensus-related parameters ` +- :ref:`randomness generation parameters `. See also ~~~~~~~~ -An in-depth description of the inners of a protocol can be found in the blog -post `How to write a Tezos protocol +An in-depth description of the internals of developing a new Tezos +protocol can be found in the blog post: `How to write a Tezos protocol `_. diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst new file mode 100644 index 000000000000..e330094b579c --- /dev/null +++ b/docs/kathmandu/validation.rst @@ -0,0 +1,48 @@ +========================== +Validation and Application +========================== + +.. _validation_modes_kathmandu: + +Validation modes +================ + +.. _full_application_kathmandu: + +Full Application +~~~~~~~~~~~~~~~~ + +.. _full_construction_kathmandu: + +Full Construction +~~~~~~~~~~~~~~~~~ + +.. _partial_construction_kathmandu: + +Partial Construction +~~~~~~~~~~~~~~~~~~~~ + +.. _partial_application_kathmandu: + +Partial Application +~~~~~~~~~~~~~~~~~~~ + +.. _block_validation_overview_kathmandu: + +Block Validation +================ + +.. _operation_validity_kathmandu: + +Operation Validation and Application +==================================== + +.. _manager_operations_validity_kathmandu: + +Validity of Manager Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_application_kathmandu: + +Application of Manager Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/lima/blocks_ops.rst b/docs/lima/blocks_ops.rst new file mode 100644 index 000000000000..2ca3f5f3868f --- /dev/null +++ b/docs/lima/blocks_ops.rst @@ -0,0 +1,33 @@ +===================== +Blocks and Operations +===================== + +.. _validation_passes_lima: + +Validation Passes +~~~~~~~~~~~~~~~~~ + +.. _consensus_operations_lima: + +Consensus Operations +~~~~~~~~~~~~~~~~~~~~ + +.. _voting_operations_lima: + +Voting Operations +~~~~~~~~~~~~~~~~~ + +.. _anonymous_operations_lima: + +Anonymous Operations +~~~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_lima: + +Manager Operations +~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_batches_lima: + +Manager Operation Batches +""""""""""""""""""""""""" diff --git a/docs/lima/consensus.rst b/docs/lima/consensus.rst index 18d549e21b0c..48ceaf456918 100644 --- a/docs/lima/consensus.rst +++ b/docs/lima/consensus.rst @@ -441,6 +441,11 @@ These are a subset of the :ref:`protocol constants `. Shell-protocol interaction revisited ------------------------------------ +.. FIXME tezos/tezos#3914: + + Integrate protocol-specific block parts in the blocks and ops + entry. + :ref:`Recall` that, for the shell to interact with the economic protocol, two notions are defined abstractly at the level of the shell and made concrete at the level of the consensus protocol. Namely, these two notions are the protocol-specific header and the fitness. As in Emmy*, the protocol-specific header contains the fields: diff --git a/docs/lima/protocol.rst b/docs/lima/protocol.rst index 1e263d2cb045..57a36ccc1b4a 100644 --- a/docs/lima/protocol.rst +++ b/docs/lima/protocol.rst @@ -83,5 +83,15 @@ Sapling, etc), and some details about its implementation. .. toctree:: :maxdepth: 2 - + event + +.. toctree:: + :maxdepth: 2 + + blocks_ops + +.. toctree:: + :maxdepth: 2 + + validation diff --git a/docs/lima/protocol_overview.rst b/docs/lima/protocol_overview.rst index 03e679e73f8a..9182a7496b46 100644 --- a/docs/lima/protocol_overview.rst +++ b/docs/lima/protocol_overview.rst @@ -37,97 +37,128 @@ different implementations, and this is important, for instance because different to implement different transaction selection strategies. Tezos is a self-amending blockchain, in that a large part of Tezos can be -changed through a so-called amendement procedure. To this end, as mentioned in -:doc:`the big picture<../shell/the_big_picture>`, a Tezos node consists of two +changed through a so-called amendment procedure. To this end, as mentioned in +:ref:`the big picture`, a Tezos node consists of two components: - the shell, which comprises the network and storage layer, and embeds - the economic protocol component, which is the part that can be changed through amendment. -The role of the protocol -~~~~~~~~~~~~~~~~~~~~~~~~ +The role of the economic protocol +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Update for pipelined validation up to Lima. At a very high level, a protocol must: -- implement protocol-specific types, such as the type of operations or protocol-specific block header data (in addition to the shell generic header), -- define under which conditions a block is a valid extension of the current blockchain, and define an ordering on blocks to arbitrate between concurrent extensions. +- implement protocol-specific types, such as the type of operations or + protocol-specific block header data (in addition to the shell + generic header), + +- define under which conditions a block is a valid extension of the + current blockchain, and define an ordering on blocks to arbitrate + between concurrent extensions. -Validity conditions are implemented in the ``apply`` function which is called -whenever the node processes a block. The ``apply`` function takes as arguments a -*context* and a block. The context represents the *protocol state* and is -therefore protocol specific. The context may contain, for instance, a list of -accounts and their balances. More generally, the context must provide enough -information to determine the validity of a block. Given a context and a block, -the ``apply`` function returns the updated context if the block is valid and has -a higher :ref:`fitness`. The fitness determines a total ordering between blocks. +Validity conditions are implemented in the ``apply`` function which is +called whenever the node processes a block---see the dedicated +:doc:`protocol validation and operation` entry for further +detail into the validation and application process for +:ref:`blocks` and their +:ref:`operations`. .. _shell_proto_interact_lima: Shell-protocol interaction ~~~~~~~~~~~~~~~~~~~~~~~~~~ -:doc:`Recall<../shell/the_big_picture>` that the economic protocol and the shell interact in order to ensure that the blocks being appended to the blockchain are valid. There are mainly two rules that the shell uses when receiving a new block: - -- The shell does not accept a branch whose fork point is in a cycle more than ``PRESERVED_CYCLES`` in the past. More precisely, if ``n`` is the current cycle, :ref:`the last allowed fork point` is the first level of cycle ``n-PRESERVED_CYCLES``. The parameter ``PRESERVED_CYCLES`` therefore plays a central role in Tezos: any block before the last allowed fork level is immutable. -- The shell changes the head of the chain to this new block only if the block is :doc:`valid<../shell/validation>` and has a higher fitness than the current head; a block is valid if the operations it includes are valid. - -The support provided by the protocol for validating blocks can be modulated by different :package-api:`validation modes `. -They allow using this same support for quite different use cases, as follows: - -- being able to validate a block, typically used in the :doc:`validator <../shell/validation>`; -- being able to pre-apply a block, typically used in the :doc:`validator <../shell/validation>` to precheck a block, avoiding to further consider invalid blocks; -- being able to construct a block, typically used by the baker to bake a block; -- being able to partially construct a block, typically used by the :doc:`prevalidator <../shell/prevalidation>` to determine valid operations in the mempool. - -Blocks -~~~~~~ +In the Tezos :ref:`architecture`, the economic +protocol and the shell interact in order to ensure that the blocks +being appended to the blockchain are valid. There are mainly two rules +that the shell uses when receiving a new block: + +- The shell does not accept a branch whose fork point is in a cycle + more than ``PRESERVED_CYCLES`` in the past. More precisely, if ``n`` + is the current cycle, :ref:`the last allowed fork point` is + the first level of cycle ``n-PRESERVED_CYCLES``. The parameter + ``PRESERVED_CYCLES`` therefore plays a central role in Tezos: any + block before the last allowed fork level is immutable. +- The shell changes the head of the chain to this new block only if + the block is :doc:`valid<../shell/validation>`, and it has a higher + fitness than the current head; a block is + :ref:`valid` only if all the + operations included are also + :ref:`valid`. + +The support provided by the protocol for validating blocks can be +modulated by different :ref:`validation +modes`. They allow using this same +interface for quite different use cases, as follows: + +- being able to :ref:`apply` a block, + typically used by the shell's :doc:`validator <../shell/validation>` + component; +- being able to :ref:`construct` a block, + typically used by the baker daemon to *bake* -- that is, to produce + -- a new block; +- being able to :ref:`partially construct` + a block, typically used by the :doc:`prevalidator + <../shell/prevalidation>` to determine valid operations in the + mempool; and, +- being able to :ref:`pre-apply` a + block, typically used in the :doc:`validator <../shell/validation>` + to precheck a block, avoiding to further consider invalid blocks. + +.. _block_contents_lima: + +Blocks, Operations and their Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. tezos/tezos#3914: + + Integrate protocol-specific block parts in the blocks and ops + entry. A block consists of a header and operations. A block's header is composed of two parts: :ref:`the protocol-agnostic part` -and :ref:`the protocol-specific part`. -This separation enables the shell to interact with different -protocols. - -.. _validation_passes_lima: - -Operations & Validation Passes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The different kinds of operations are grouped in classes, such that operations belonging to different classes may be validated independently, and/or with different priorities. -Each class has an associated index, called a :ref:`validation pass`. -There are four classes of operations: :doc:`consensus ` operations, :doc:`voting ` operations, anonymous operations, manager operations. - -Consensus operations are endorsements, while `voting ` operations are ballot and proposal. - -Anonymous operations are operations which are not signed. There are three anonymous operations: seed nonce revelation, double baking evidence, and double endorsing evidence. The evidence for double baking and double endorsing is included in a block by the so-called accuser (see :ref:`slashing`). - -Manager operations are activation, origination (see :doc:`smart contracts`), transaction, reveal, and delegation (see :doc:`proof of stake `). Manager operations are the only fee-paying operations. - -Recall that users have associated :ref:`accounts ` which they activate before being able to participate. By means of the operation :ref:`origination`, accounts can be further associated with smart contracts in which they are called :ref:`originated accounts`. :ref:`Transactions` are used to either transfer tez between two accounts or run the code of a smart contract. Transactions are signed by an account's private key. Before making a transaction, a user must reveal her public key so that other users (not being aware of this public key) can effectively check the signature of the transaction. - -Manager operations can be grouped into batches forming a so-called group operation. A group operation satisfies: - -- atomicity: either all the operations in the batch succeed or none is applied -- efficiency: the whole batch is signed only once (by the same implicit account), thus it is much more efficient to check, and it requires much less gas -- usability: the batch only increments the counter of the signer account by one; for this reason it is easier for tools to provide sending several operations per block using operation batches than tracking counter changes. - -The list of operations can be obtained with :ref:`this rpc `. +and :ref:`the protocol-specific part`. This +separation enables the shell to interact with different +protocols. Each Tezos economic protocol can specify different kinds of +operations, which are described further in detail in +:doc:`./blocks_ops`. + +The semantics of, respectively, operations and blocks is indeed also +dependent on each economic protocol. The :doc:`Validation and +Application` entry explains the internals of *validation* +-- that is, how to determine whether operations and blocks can be +safely be included in the Tezos blockchain -- and *application* -- +that is, how the effects of operations and blocks are taken into +account -- for this economic protocol. .. _protocol_constants_lima: Protocol constants ~~~~~~~~~~~~~~~~~~ -Protocols are tuned by several *protocol constants*, such as the size of a nonce, or the number of blocks per cycle. -One can distinguish two kinds of protocol constants: +Protocols are tuned by several *protocol constants*, such as the size +of a nonce, or the number of blocks per cycle. One can distinguish two +kinds of protocol constants: -- *fixed* protocol constants, such as the size of a nonce, are values wired in the code of a protocol, and can only be changed by protocol amendment (that is, by adopting a new protocol) -- *parametric* protocol constants, such as the number of blocks per cycle, are values maintained in a read-only data structure that can be instantiated differently, for the same protocol, from one network to another (for instance, test networks move faster). +- *fixed* protocol constants, such as the size of a nonce, are values + wired in the code of a protocol, and can only be changed by protocol + amendment (that is, by adopting a new protocol) +- *parametric* protocol constants, such as the number of blocks per + cycle, are values maintained in a read-only data structure that can + be instantiated differently, for the same protocol, from one network + to another (for instance, test networks move faster). The *list* of protocol constants can be found in the OCaml APIs: -- fixed protocol constants are defined in the module :package-api:`Constants_repr ` -- parametric constants are defined in the module :package-api:`Constants_parametric_repr ` +- fixed protocol constants are defined in the module + :package-api:`Constants_repr + ` +- parametric constants are defined in the module :package-api:`Constants_parametric_repr ` The *values* of protocol constants in any given protocol can be found using specific RPC calls: @@ -144,6 +175,6 @@ See, for example: See also ~~~~~~~~ -An in-depth description of the inners of a protocol can be found in the blog -post `How to write a Tezos protocol +An in-depth description of the internals of developing a new Tezos +protocol can be found in the blog post: `How to write a Tezos protocol `_. diff --git a/docs/lima/validation.rst b/docs/lima/validation.rst new file mode 100644 index 000000000000..f7b4ba9763e6 --- /dev/null +++ b/docs/lima/validation.rst @@ -0,0 +1,48 @@ +========================== +Validation and Application +========================== + +.. _validation_modes_lima: + +Validation modes +================ + +.. _full_application_lima: + +Full Application +~~~~~~~~~~~~~~~~ + +.. _full_construction_lima: + +Full Construction +~~~~~~~~~~~~~~~~~ + +.. _partial_construction_lima: + +Partial Construction +~~~~~~~~~~~~~~~~~~~~ + +.. _partial_application_lima: + +Partial Application +~~~~~~~~~~~~~~~~~~~ + +.. _block_validation_overview_lima: + +Block Validation +================ + +.. _operation_validity_lima: + +Operation Validation and Application +==================================== + +.. _manager_operations_validity_lima: + +Validity of Manager Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _manager_operations_application_lima: + +Application of Manager Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- GitLab From d27e5164dad7743caf44c78f99a346bb21abc381 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Thu, 6 Oct 2022 02:01:42 +0200 Subject: [PATCH 03/12] doc: add a protocol-specific entry for operations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Advances #3921, #3162. Fix 3937. Adds a protocol perspective on operations, describing all enabled operations by validation passess. Updated up to Kathmandu (exported to Lima and Alpha). Co-authored-by: Germán Delbianco Co-authored-by: Eugen Zalinescu --- docs/alpha/blocks_ops.rst | 190 +++++++++++++++++++++++ docs/alpha/glossary.rst | 15 ++ docs/alpha/randomness_generation.rst | 4 + docs/kathmandu/blocks_ops.rst | 184 ++++++++++++++++++++++ docs/kathmandu/glossary.rst | 15 ++ docs/kathmandu/randomness_generation.rst | 4 + docs/lima/blocks_ops.rst | 189 ++++++++++++++++++++++ docs/lima/glossary.rst | 15 ++ docs/lima/randomness_generation.rst | 4 + 9 files changed, 620 insertions(+) diff --git a/docs/alpha/blocks_ops.rst b/docs/alpha/blocks_ops.rst index 46fcf15f12f4..0c4e451cd713 100644 --- a/docs/alpha/blocks_ops.rst +++ b/docs/alpha/blocks_ops.rst @@ -2,32 +2,222 @@ Blocks and Operations ===================== +The content of a Tezos block is made up of operations, which implement +and reify different functionalities provided by a Tezos economic +protocol: from reaching consensus on the state of the Tezos +blockchain, to performing smart contract calls and transactions. Each +Tezos economic protocol can specify different kinds of operations. + +This entry describes the operations supported by :doc:`the economic +protocol <./protocol>` that implement *enabled* features -- that is, +those available to end-users on Tezos Mainnet. The complete list of +operations, including those corresponding to features in development +or available only on test networks, is given in the +:package-api:`OCaml Documentation +`. + .. _validation_passes_alpha: Validation Passes ~~~~~~~~~~~~~~~~~ +The different kinds of operations are grouped into classes. Each class +has an associated index, a natural number, also known as a +:ref:`validation pass`. There are currently four classes +of operations: :ref:`consensus `, +:ref:`voting `, +:ref:`anonymous`, and :ref:`manager +operations`. This order also specifies the +:ref:`validation and application` priority +of each of these classes. Consensus operations are considered the +highest priority ones, and manager operations the lowest. + +The current protocol implementation enforces the following invariant: + +- each kind of operation belongs to *at most one* validation pass; +- operations whose kind does not belong to any validation pass cannot + be :ref:`applied`. + +.. FIXME tezos/tezos#3915: + + Failing noops don't fit within any of the validation passes + below. We need to change the structure a bit to be able to list + them here. + +In the sequel, we describe the different classes of operations, and +the different kinds of operations belonging to each class. + .. _consensus_operations_alpha: Consensus Operations ~~~~~~~~~~~~~~~~~~~~ +.. TODO tezos/tezos#4204: document PCQ/PQ + +Consensus operations are administrative operations that are necessary +to implement the :doc:`consensus algorithm`. There are two +kinds of consensus operations, each belonging to the different voting +phases required to agree on the next block. + +- A ``Preendorsement`` operation implements a first vote for a + :ref:`candidate block ` with the aim of + building a :ref:`preendorsement quorum `. + +- An ``Endorsement`` operation implements a vote for a candidate block + for which a preendorsement quorum certificate (PQC) has been + observed. + .. _voting_operations_alpha: Voting Operations ~~~~~~~~~~~~~~~~~ +Voting operations are operations related to the on-chain :doc:`Tezos +Amendment` process. In this economic protocol, there are two +voting operations: + +- The ``Proposal`` operation enables delegates to submit (also known as + to "inject") protocol amendment proposals, or to up-vote previously + submitted proposals, during the Proposal period. + +- The ``Ballot`` operation enables delegates to participate in the + Exploration and Promotion periods. Delegates use this operation to + vote for (``Yay``), against (``Nay``), or to side with the majority + (``Pass``), when examining a protocol amendment proposal. + +Further details on each operation's implementation and semantics are +provided in the dedicated entry for :ref:`on-chain +governance`. + .. _anonymous_operations_alpha: Anonymous Operations ~~~~~~~~~~~~~~~~~~~~ +.. TODO: tezos/tezos#3936 integrate consensus keys operations + +This class groups all operations that do not require a signature from +a Tezos account. They allow to implement different functionalities of +the protocol, and their common characteristic is that it is desirable +that the account originating these operation remains anonymous in +order to avoid censorship. + +Two operations in this class implement functionality pertaining to the +protocol's :doc:`random seeds generation +mechanism`: + +- The ``Seed_nonce_revelation`` operation allows a baker to + anonymously reveal the nonce seed for the commitment it had included + in a previously baked block (in the previous cycle). + +- The ``Vdf_revelation`` operation allows the submission of a solution + to, and a proof of correctness of, the :ref:`VDF + challenge` corresponding to the VDF revelation period + of the randomness generation protocol. + +Further details on the latter operation's implementation and semantics +are provided in the :ref:`random seed generation +protocol`. + +Three operations in this class are used to :ref:`punish participants +which engage in Byzantine behaviour` -- notably +delegates which :ref:`"double sign" ` blocks, or emit +conflicting :ref:`consensus operations`: + +- The ``Double_preendorsement_evidence`` operation allows for accusing + a delegate of having *double-preendorsed* -- i.e., of having + preendorsed two different block candidates, at the same level and at + the same round. The bulk of the evidence, the two arguments + provided, consists of the two offending preendorsements. + +- Similarly, the ``Double_endorsement_evidence`` operation allows for + accusing a delegate of having *double-endorsed* -- i.e., of having + endorsed two different block candidates at the same level and the + same round -- by providing the two offending endorsements. + +- The ``Double_baking_evidence`` allows for accusing a delegate of + having "double-baked" a block -- i.e., of having signed two + different blocks at the same level and at same round. The bulk of + the evidence consists of the :ref:`block + headers` of each of the two offending + blocks. + +See :ref:`here` for further detail on the +semantics of evidence-providing operations. + +Finally, the ``Activation`` operation allows users which participated +in the Tezos fundraiser to make their :ref:`accounts ` +operational. + .. _manager_operations_alpha: Manager Operations ~~~~~~~~~~~~~~~~~~ +.. FIXME tezos/tezos#3936: integrate consensus keys operations. + +.. FIXME tezos/tezos#3937: + + Document increased paid storage manager operation. + +Manager operations enable end-users to interact with the Tezos +blockchain -- e.g., transferring funds or calling :doc:`smart +contracts`. A manager operation is issued by a single +*manager* account which signs the operation and pays the +:ref:`fees` to the baker for its inclusion in a block. Indeed, +manager operations are the only fee-paying and +:ref:`gas-consuming` operations. + +- The ``Reveal`` operation reveals the public key of the sending + manager. Knowing this public key is indeed necessary to check the signature + of future operations signed by this manager. +- The ``Transaction`` operation allows users to either transfer tez + between accounts and/or to invoke a smart contract. +- The ``Delegation`` operation allows users to :ref:`delegate their + stake ` to a :ref:`delegate` (a + *baker*), or to register themselves as delegates. +- The ``Origination`` operation is used to + :ref:`originate`, that is to deploy, smart contracts + in the Tezos blockchain. +- The ``Set_deposits_limit`` operation enables delegates to adjust the + amount of stake a delegate :ref:`has locked in + bonds`. +- Support for registering global constants is implemented with the + ``Register_global_constant`` operation. +- The ``Increase_paid_storage`` operation allows a sender to increase + the paid storage of some previously deployed contract. +- The ``Event`` operation enables sending event-like information to + external applications from Tezos smart contracts -- see + :doc:`Contract Events` for further detail. + +Moreover, all operations necessary to implement Tezos' *enshrined* +Layer 2 solutions into the economic protocol are also manager +operations: + +.. FIXME tezos/tezos#3916: expand documentation of TORU operations. + +- :doc:`Transaction Optimistic Rollups` are + implemented using the following manager operations: + ``Tx_rollup_origination``, ``Tx_rollup_submit_batch``, + ``Tx_rollup_commit``, ``Tx_rollup_return_bond``, + ``Tx_rollup_finalize_commitment``, ``Tx_rollup_remove_commitment``, + ``Tx_rollup_rejection``, ``Tx_rollup_rejection``, + ``Transfer_ticket``. + .. _manager_operations_batches_alpha: Manager Operation Batches """"""""""""""""""""""""" + +Manager operations can be grouped, forming a so-called +**batch**. Batches enable the inclusion of several manager operations +from the same manager in a single block. + +Batches satisfy the following properties: + +- All operations in a batch are issued by the same manager, which + provides a single signature for the entire batch. +- A batch is :ref:`applied` + atomically: all its operations are executed sequentially, without + interleaving other operations. Either all the operations in the + batch succeed, or none is applied. diff --git a/docs/alpha/glossary.rst b/docs/alpha/glossary.rst index a0c9808a8833..3deedc0fcd3b 100644 --- a/docs/alpha/glossary.rst +++ b/docs/alpha/glossary.rst @@ -100,6 +100,14 @@ _`Double signing` network: an accuser_ can provide proof of the double signing to be awarded part of the double signer's deposit. +_`Failing Noop` + The ``Failing_noop`` operation implements a *No-op*, which always + fails at :ref:`application time`, and + should never appear in :ref:`applied + blocks`. This operation allows end-users to + :ref:`sign arbitrary messages` which have no + computational semantics. + _`Fee` To ensure responsible use of computation resources of other nodes, and also to encourage active participation in the consensus protocol, there are some fees that users pay to bakers for including their operations in blocks. @@ -189,6 +197,13 @@ _`Transaction` An operation_ to transfer tez between two accounts, or to run the code of a `smart contract`_. +_`Validation pass` + An index that can be associated with different kinds of + operations, in order to group them into classes. Validation passes + enable prioritizing the :ref:`validation and + application` of certain classes of + operations. + _`Voting period` Any of the ``proposal``, ``exploration``, ``cooldown``, ``promotion`` or ``adoption`` stages in the voting procedure when diff --git a/docs/alpha/randomness_generation.rst b/docs/alpha/randomness_generation.rst index 80f9f9ab3751..417c5acaa144 100644 --- a/docs/alpha/randomness_generation.rst +++ b/docs/alpha/randomness_generation.rst @@ -46,6 +46,8 @@ if a malicious participant can make sure she is the last revealer, then she can choose whether to reveal its committed value, effectively choosing between two different predetermined seeds. +.. _vdf_alpha: + Verifiable Delay Function ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -75,6 +77,8 @@ cheaper and based on a weaker security assumption (low order assumption). Protocol -------- +.. _randomness_generation_alpha: + Randomness generation overview ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/kathmandu/blocks_ops.rst b/docs/kathmandu/blocks_ops.rst index 5488272a3089..700e86d0966f 100644 --- a/docs/kathmandu/blocks_ops.rst +++ b/docs/kathmandu/blocks_ops.rst @@ -2,32 +2,216 @@ Blocks and Operations ===================== +The content of a Tezos block is made up of operations, which implement +and reify different functionalities provided by a Tezos economic +protocol: from reaching consensus on the state of the Tezos +blockchain, to performing smart contract calls and transactions. Each +Tezos economic protocol can specify different kinds of operations. + +This entry describes the operations supported by :doc:`the economic +protocol <./protocol>` that implement *enabled* features -- that is, +those available to end-users on Tezos Mainnet. The complete list of +operations, including those corresponding to features in development +or available only on test networks, is given in the +:package-api:`OCaml Documentation +`. + .. _validation_passes_kathmandu: Validation Passes ~~~~~~~~~~~~~~~~~ +The different kinds of operations are grouped into classes. Each class +has an associated index, a natural number, also known as a +:ref:`validation pass`. There are currently four classes +of operations: :ref:`consensus `, +:ref:`voting `, +:ref:`anonymous`, and :ref:`manager +operations`. This order also specifies +the :ref:`validation and application` +priority of each of these classes. Consensus operations are +considered the highest priority ones, and manager operations the +lowest. + +The current protocol implementation enforces the following invariant: + +- each kind of operation belongs to *at most one* validation pass; +- operations whose kind does not belong to any validation pass cannot + be :ref:`applied`. + +.. FIXME tezos/tezos#3915: + + Failing noops don't fit within any of the validation passes + below. We need to change the structure a bit to be able to list + them here. + +In the sequel, we describe the different classes of operations, and +the different kinds of operations belonging to each class. + .. _consensus_operations_kathmandu: Consensus Operations ~~~~~~~~~~~~~~~~~~~~ +.. TODO tezos/tezos#4204: document PCQ/PQ + +Consensus operations are administrative operations that are necessary +to implement the :doc:`consensus algorithm`. There are two +kinds of consensus operations, each belonging to the different voting +phases required to agree on the next block. + +- A ``Preendorsement`` operation implements a first vote for a + :ref:`candidate block ` with the aim of + building a :ref:`preendorsement quorum `. + +- An ``Endorsement`` operation implements a vote for a candidate block + for which a preendorsement quorum certificate (PQC) has been + observed. + .. _voting_operations_kathmandu: Voting Operations ~~~~~~~~~~~~~~~~~ +Voting operations are operations related to the on-chain :doc:`Tezos +Amendment` process. In this economic protocol, there are two +voting operations: + +- The ``Proposal`` operation enables delegates to submit (also known as + to "inject") protocol amendment proposals, or to up-vote previously + submitted proposals, during the Proposal period. + +- The ``Ballot`` operation enables delegates to participate in the + Exploration and Promotion periods. Delegates use this operation to + vote for (``Yay``), against (``Nay``), or to side with the majority + (``Pass``), when examining a protocol amendment proposal. + +Further details on each operation's implementation and semantics are +provided in the dedicated entry for :ref:`on-chain +governance`. + .. _anonymous_operations_kathmandu: Anonymous Operations ~~~~~~~~~~~~~~~~~~~~ +This class groups all operations that do not require a signature from +a Tezos account. They implement different functionalities of the +protocol, and their common characteristic is that they allow the +account originating these operations to remain anonymous in order to +avoid censorship. + +Two operations in this class implement functionality pertaining to the +protocol's :doc:`random seeds generation +mechanism`: + +- The ``Seed_nonce_revelation`` operation allows a baker to + anonymously reveal the nonce seed for the commitment it had included + in a previously baked block (in the previous cycle). + +- The ``Vdf_revelation`` operation allows the submission of a solution + to, and a proof of correctness of, the :ref:`VDF + challenge` corresponding to the VDF revelation period + of the randomness generation protocol. + +Further details on the latter operation's implementation and semantics +are provided in the :ref:`random seed generation +protocol`. + +Three operations in this class are used to :ref:`punish participants +which engage in Byzantine behaviour` -- notably +delegates which :ref:`"double sign" ` blocks, or emit +conflicting :ref:`consensus +operations`: + +- The ``Double_preendorsement_evidence`` operation allows for accusing + a delegate of having *double-preendorsed* -- i.e., of having + preendorsed two different block candidates, at the same level and at + the same round. The bulk of the evidence, the two arguments + provided, consists of the two offending preendorsements. + +- Similarly, the ``Double_endorsement_evidence`` operation allows for + accusing a delegate of having *double-endorsed* -- i.e., of having + endorsed two different block candidates at the same level and the + same round -- by providing the two offending endorsements. + +- The ``Double_baking_evidence`` allows for accusing a delegate of + having "double-baked" a block -- i.e., of having signed two + different blocks at the same level and at same round. The bulk of + the evidence consists of the :ref:`block + headers` of each of the two offending + blocks. + +See :ref:`here` for further detail on the +semantics of evidence-providing operations. + +Finally, the ``Activation`` operation allows users which participated +in the Tezos fundraiser to make their :ref:`accounts ` +operational. + .. _manager_operations_kathmandu: Manager Operations ~~~~~~~~~~~~~~~~~~ +Manager operations enable end-users to interact with the Tezos +blockchain -- e.g., transferring funds or calling :doc:`smart +contracts`. A manager operation is issued by a single +*manager* account which signs the operation and pays the +:ref:`fees` to the baker for its inclusion in a block. Indeed, +manager operations are the only fee-paying and +:ref:`gas-consuming` operations. + +- The ``Reveal`` operation reveals the public key of the sending + manager. Knowing this public key is indeed necessary to check the signature + of future operations signed by this manager. +- The ``Transaction`` operation allows users to either transfer tez + between accounts and/or to invoke a smart contract. +- The ``Delegation`` operation allows users to :ref:`delegate their + stake ` to a :ref:`delegate` (a + *baker*), or to register themselves as delegates. +- The ``Origination`` operation is used to + :ref:`originate`, that is to deploy, smart contracts + in the Tezos blockchain. +- The ``Set_deposits_limit`` operation enables delegates to adjust the + amount of stake a delegate :ref:`has locked in + bonds`. +- Support for registering global constants is implemented with the + ``Register_global_constant`` operation. +- The ``Increase_paid_storage`` operation allows a sender to increase + the paid storage of some previously deployed contract. +- The ``Event`` operation enable sending event-like information to + external applications from Tezos smart contracts -- see + :doc:`Contract Events` for further detail. + +Moreover, all operations necessary to implement Tezos' *enshrined* +Layer 2 solutions into the economic protocol are also manager +operations: + +.. FIXME tezos/tezos#3916: expand documentation of TORU operations. + +- :doc:`Transaction Optimistic Rollups` are + implemented using the following manager operations: + ``Tx_rollup_origination``, ``Tx_rollup_submit_batch``, + ``Tx_rollup_commit``, ``Tx_rollup_return_bond``, + ``Tx_rollup_finalize_commitment``, ``Tx_rollup_remove_commitment``, + ``Tx_rollup_rejection``, ``Tx_rollup_rejection``, + ``Transfer_ticket``. + .. _manager_operations_batches_kathmandu: Manager Operation Batches """"""""""""""""""""""""" + +Manager operations can be grouped, forming a so-called +**batch**. Batches enable the inclusion of several manager operations +from the same manager in a single block. + +Batches satisfy the following properties: + +- All operations in a batch are issued by the same manager, which + provides a single signature for the entire batch. +- A batch is :ref:`applied` + atomically: all its operations are executed sequentially, without + interleaving other operations. Either all the operations in the + batch succeed, or none is applied. diff --git a/docs/kathmandu/glossary.rst b/docs/kathmandu/glossary.rst index 4a779f64c7ba..901b4c5ded96 100644 --- a/docs/kathmandu/glossary.rst +++ b/docs/kathmandu/glossary.rst @@ -100,6 +100,14 @@ _`Double signing` network: an accuser_ can provide proof of the double signing to be awarded part of the double signer's deposit. +_`Failing Noop` + The ``Failing_noop`` operation implements a *No-op*, which always + fails at :ref:`application time`, and + should never appear in :ref:`applied + blocks`. This operation allows end-users to + :ref:`sign arbitrary messages` which have no + computational semantics. + _`Fee` To ensure responsible use of computation resources of other nodes, and also to encourage active participation in the consensus protocol, there are some fees that users pay to bakers for including their operations in blocks. @@ -191,6 +199,13 @@ _`Transaction` An operation_ to transfer tez between two accounts, or to run the code of a `smart contract`_. +_`Validation pass` + An index (a natural number) that can be associated + to the different kind of operations, in order to group them into + classes. Validation passes enable prioritizing the + :ref:`validation and application` of + certain classes of operations. + _`Voting period` Any of the ``proposal``, ``exploration``, ``cooldown``, ``promotion`` or ``adoption`` stages in the voting procedure when diff --git a/docs/kathmandu/randomness_generation.rst b/docs/kathmandu/randomness_generation.rst index c76a0d323256..eaf9483fa464 100644 --- a/docs/kathmandu/randomness_generation.rst +++ b/docs/kathmandu/randomness_generation.rst @@ -46,6 +46,8 @@ if a malicious participant can make sure she is the last revealer, then she can choose whether to reveal its committed value, effectively choosing between two different predetermined seeds. +.. _vdf_kathmandu: + Verifiable Delay Function ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -75,6 +77,8 @@ cheaper and based on a weaker security assumption (low order assumption). Protocol -------- +.. _randomness_generation_kathmandu: + Randomness generation overview ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/lima/blocks_ops.rst b/docs/lima/blocks_ops.rst index 2ca3f5f3868f..1c65a17b260b 100644 --- a/docs/lima/blocks_ops.rst +++ b/docs/lima/blocks_ops.rst @@ -2,32 +2,221 @@ Blocks and Operations ===================== +The content of a Tezos block is made up of operations, which implement +and reify different functionalities provided by a Tezos economic +protocol: from reaching consensus on the state of the Tezos +blockchain, to performing smart contract calls and transactions. Each +Tezos economic protocol can specify different kinds of operations. + +This entry describes the operations supported by :doc:`the economic +protocol <./protocol>` that implement *enabled* features -- that is, +those available to end-users on Tezos Mainnet. The complete list of +operations, including those corresponding to features in development +or available only on test networks, is given in the +:package-api:`OCaml Documentation +`. + .. _validation_passes_lima: Validation Passes ~~~~~~~~~~~~~~~~~ +The different kinds of operations are grouped into classes. Each class +has an associated index, a natural number, also known as a +:ref:`validation pass`. There are currently four classes +of operations: :ref:`consensus `, +:ref:`voting `, +:ref:`anonymous`, and :ref:`manager +operations`. This order also specifies the +:ref:`validation and application` priority of +each of these classes. Consensus operations are considered the highest +priority ones, and manager operations the lowest. + +The current protocol implementation enforces the following invariant: + +- each kind of operation belongs to *at most one* validation pass; +- operations whose kind does not belong to any validation pass cannot + be :ref:`applied`. + +.. FIXME tezos/tezos#3915: + + Failing noops don't fit within any of the validation passes + below. We need to change the structure a bit to be able to list + them here. + +In the sequel, we describe the different classes of operations, and +the different kinds of operations belonging to each class. + .. _consensus_operations_lima: Consensus Operations ~~~~~~~~~~~~~~~~~~~~ +.. TODO tezos/tezos#4204: document PCQ/PQ + +Consensus operations are administrative operations that are necessary +to implement the :doc:`consensus algorithm`. There are two +kinds of consensus operations, each belonging to the different voting +phases required to agree on the next block. + +- A ``Preendorsement`` operation implements a first vote for a + :ref:`candidate block ` with the aim of + building a :ref:`preendorsement quorum `. + +- An ``Endorsement`` operation implements a vote for a candidate block + for which a preendorsement quourm cerificate (PQC) has been + observed. + .. _voting_operations_lima: Voting Operations ~~~~~~~~~~~~~~~~~ +Voting operations are operations related to the on-chain :doc:`Tezos +Amendment` process. In this economic protocol, there are two +voting operations: + +- The ``Proposal`` operation enables delegates to submit (also known as + to "inject") protocol amendment proposals, or to up-vote previously + submitted proposals, during the Proposal period. + +- The ``Ballot`` operation enables delegates to participate in the + Exploration and Promotion periods. Delegates use this operation to + vote for (``Yay``), against (``Nay``), or to side with the majority + (``Pass``), when examining a protocol amendment proposal. + +Further details on each operation's implementation and semantics are +provided in the dedicated entry for :ref:`on-chain +governance`. + .. _anonymous_operations_lima: Anonymous Operations ~~~~~~~~~~~~~~~~~~~~ +.. TODO: tezos/tezos#3936 integrate consensus keys operations + +This class groups all operations that do not require a signature from +a Tezos account. They allow to implement different functionalities of +the protocol, and their common characteristic is that it is desirable +that the account originating these operation remains anonymous in +order to avoid censorship. + +Two operations in this class implement functionality pertaining to the +protocol's :doc:`random seeds generation +mechanism`: + +- The ``Seed_nonce_revelation`` operation allows a baker to + anonymously reveal the nonce seed for the commitment it had included + in a previously baked block (in the previous cycle). + +- The ``Vdf_revelation`` operation allows the submission of a solution + to, and a proof of correctness of, the :ref:`VDF + challenge` corresponding to the VDF revelation period + of the randomness generation protocol. + +Further details on the latter operation's implementation and semantics +are provided in the :ref:`random seed generation +protocol`. + +Three operations in this class are used to :ref:`punish participants +which engage in Byzantine behaviour` -- notably +delegates which :ref:`"double sign" ` blocks, or emit +conflicting :ref:`consensus operations`: + +- The ``Double_preendorsement_evidence`` operation allows for accusing + a delegate of having *double-preendorsed* -- i.e., of having + preendorsed two different block candidates, at the same level and at + the same round. The bulk of the evidence, the two arguments + provided, consists of the two offending preendorsements. + +- Similarly, the ``Double_endorsement_evidence`` operation allows for + accusing a delegate of having *double-endorsed* -- i.e., of having + endorsed two different block candidates at the same level and the + same round -- by providing the two offending endorsements. + +- The ``Double_baking_evidence`` allows for accusing a delegate of + having "double-baked" a block -- i.e., of having signed two + different blocks at the same level and at same round. The bulk of + the evidence consists of the :ref:`block + headers` of each of the two offending blocks. + +See :ref:`here` for further detail on the +semantics of evidence-providing operations. + +Finally, the ``Activation`` operation allows users which participated +in the Tezos fundraiser to make their :ref:`accounts ` +operational. + .. _manager_operations_lima: Manager Operations ~~~~~~~~~~~~~~~~~~ +.. FIXME tezos/tezos#3936: integrate consensus keys operations. + +.. FIXME tezos/tezos#3937: + + Document increased paid storage manager operation. + +Manager operations enable end-users to interact with the Tezos +blockchain -- e.g., transferring funds or calling :doc:`smart +contracts`. A manager operation is issued by a single +*manager* account which signs the operation and pays the +:ref:`fees` to the baker for its inclusion in a block. Indeed, +manager operations are the only fee-paying and +:ref:`gas-consuming` operations. + +- The ``Reveal`` operation reveals the public key of the sending + manager. Knowing this public key is indeed necessary to check the signature + of future operations signed by this manager. +- The ``Transaction`` operation allows users to either transfer tez + between accounts and/or to invoke a smart contract. +- The ``Delegation`` operation allows users to :doc:`delegate their + stake` to a :ref:`delegate` (a *baker*), + or to register themselves as delegates. +- The ``Origination`` operation is used to + :ref:`originate`, that is to deploy, smart contracts + in the Tezos blockchain. +- The ``Set_deposits_limit`` operation enables delegates to adjust the + amount of stake a delegate :ref:`has locked in + bonds`. +- Support for registering global constants is implemented with the + ``Register_global_constant`` operation. +- The ``Increase_paid_storage`` operation allows a sender to increase + the paid storage of some previously deployed contract. +- The ``Event`` operation enables sending event-like information to + external applications from Tezos smart contracts -- see + :doc:`Contract Events` for further detail. + +Moreover, all operations necessary to implement Tezos' *enshrined* +Layer 2 solutions into the economic protocol are also manager +operations: + +.. FIXME tezos/tezos#3916: expand documentation of TORU operations. + +- :doc:`Transaction Optimistic Rollups` are + implemented using the following manager operations: + ``Tx_rollup_origination``, ``Tx_rollup_submit_batch``, + ``Tx_rollup_commit``, ``Tx_rollup_return_bond``, + ``Tx_rollup_finalize_commitment``, ``Tx_rollup_remove_commitment``, + ``Tx_rollup_rejection``, ``Tx_rollup_rejection``, + ``Transfer_ticket``. + .. _manager_operations_batches_lima: Manager Operation Batches """"""""""""""""""""""""" + +Manager operations can be grouped, forming a so-called +**batch**. Batches enable the inclusion of several manager operations +from the same manager in a single block. + +Batches satisfy the following properties: + +- All operations in a batch are issued by the same manager, which + provides a single signature for the entire batch. +- A batch is :ref:`applied` + atomically: all its operations are executed sequentially, without + interleaving other operations. Either all the operations in the + batch succeed, or none is applied. diff --git a/docs/lima/glossary.rst b/docs/lima/glossary.rst index 0661d67e7b78..0c48bd863866 100644 --- a/docs/lima/glossary.rst +++ b/docs/lima/glossary.rst @@ -100,6 +100,14 @@ _`Double signing` network: an accuser_ can provide proof of the double signing to be awarded part of the double signer's deposit. +_`Failing Noop` + The ``Failing_noop`` operation implements a *No-op*, which always + fails at :ref:`application time`, and + should never appear in :ref:`applied + blocks`. This operation allows end-users to + :ref:`sign arbitrary messages` which have no + computational semantics. + _`Fee` To ensure responsible use of computation resources of other nodes, and also to encourage active participation in the consensus protocol, there are some fees that users pay to bakers for including their operations in blocks. @@ -189,6 +197,13 @@ _`Transaction` An operation_ to transfer tez between two accounts, or to run the code of a `smart contract`_. +_`Validation pass` + An index that can be associated to the different kind of + operations, in order to group them into classes. Validation passes + enable prioritizing the :ref:`validation and + application` of certain classes of + operations. + _`Voting period` Any of the ``proposal``, ``exploration``, ``cooldown``, ``promotion`` or ``adoption`` stages in the voting procedure when diff --git a/docs/lima/randomness_generation.rst b/docs/lima/randomness_generation.rst index d340bd043edb..1b91f6b56fab 100644 --- a/docs/lima/randomness_generation.rst +++ b/docs/lima/randomness_generation.rst @@ -46,6 +46,8 @@ if a malicious participant can make sure she is the last revealer, then she can choose whether to reveal its committed value, effectively choosing between two different predetermined seeds. +.. _vdf_lima: + Verifiable Delay Function ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -75,6 +77,8 @@ cheaper and based on a weaker security assumption (low order assumption). Protocol -------- +.. _randomness_generation_lima: + Randomness generation overview ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- GitLab From 41a39b1285bd39d1265562e8dcec53c9762d99d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Tue, 7 Jun 2022 17:32:26 +0200 Subject: [PATCH 04/12] doc: add overview for protocol validation & application MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Advances #3921, #3162. Adds an overview for the block validation and application protocol entry. Implemented for Kathmandu, Lima, and Alpha. Updated up to Pipelining phase 1 (Kathmandu). Co-authored-by: vbot Co-authored-by: Germán Delbianco --- docs/alpha/validation.rst | 77 +++++++++++++++++++++++++++++++++++ docs/kathmandu/validation.rst | 68 +++++++++++++++++++++++++++++++ docs/lima/validation.rst | 70 +++++++++++++++++++++++++++++++ 3 files changed, 215 insertions(+) diff --git a/docs/alpha/validation.rst b/docs/alpha/validation.rst index f1eb7bce552a..c7e206897f83 100644 --- a/docs/alpha/validation.rst +++ b/docs/alpha/validation.rst @@ -2,6 +2,83 @@ Validation and Application ========================== +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment + +The :doc:`economic protocol` is responsible for providing +the rules that govern the Tezos network, and for enforcing that these +rules implement a correct blockchain protocol. However, it does so +:ref:`in coordination with a Tezos shell`, who +ultimately implements these rules on its behalf. To this end, a Tezos +economic protocol must provide the shell with an interface enabling +roughly the following functionalities: + +- deciding whether an operation is *valid*, and thus can be safely + included into a new block in the blockchain; + +- deciding whether a new block is *valid*, and thus can be safely + appended to the blockchain; + +- including an operation in a block, and executing it on the + blockchain state, effectively modifying the ledger state; and, + +- appending a new block to a Tezos blockchain, and computing the + updated ledger state. + +From a higher-level, *abstract* perspective, the validation system in +the Tezos protocol implements this business logic in a functional, +state-passing machine where: + +- Its state is given by the :ref:`context`, the internal + representation of the state of the Tezos ledger at a given blockchain + level. For instance, the context contains the information of all + activated accounts and contracts, and their balances. More + generally, the context must provide enough information to determine + the validity of operations, and blocks. + +- The *apply_operation* method determines whether an operation is safe + to be executed in a given context. If so, it proceeds to execute it + and outputs the context resulting from the applied operation's + effects. + +- The *apply* method takes as input a block and a context, determines + whether the block can be safely and correctly appended to the + current head of the chain. If so, it appends the block to the chain + and outputs the resulting context, reflecting the state of the + ledger after the block has successfully been applied. *apply* relies + on (i.e., it should call) *apply_operation* to validate and apply + each operation in the block, and compute intermediate states. + +.. TODO #4155: + + When creating a new environment, update references to V in the + paragraph below. + +However, the *concrete* API exported from the Tezos economic protocol +does not implement this business logic *monolithically*, as described +above, but it rather presents a more fine-grained API. The rationale +is to provide specialized variations of the core *validation* and +*application* functionality, dubbed :ref:`Validation +modes`. These modes enable the protocol to +distinguish the validation of operations "in the mempool", performed +by the :doc:`prevalidator<../shell/prevalidation>`, from the +validation of the operations included in newly received blocks, +performed by the :ref:`block validator`, in order to +localize validation rules as needed. The resulting concrete API is +specified by the :package-api:`Protocol +` +module in the :doc:`protocol +environment<../shell/protocol_environment>` ``V8``, and it is +implemented by this protocol in the +:package-api:`Main` +module. + +The rest of this document is organized as follows: we first describe +the different validation modes implemented by this Tezos economic +protocol, and then we delve deeper into the particulars of validation +and application for blocks and the operations supported. + .. _validation_modes_alpha: Validation modes diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst index e330094b579c..bfa8c693e760 100644 --- a/docs/kathmandu/validation.rst +++ b/docs/kathmandu/validation.rst @@ -2,6 +2,74 @@ Validation and Application ========================== +The :doc:`economic protocol` is responsible for providing +the rules that govern the Tezos network, and for enforcing that these +rules implement a correct blockchain protocol. However, it does so +:ref:`in coordination with a Tezos shell`, who +ultimately implements these rules on its behalf. To this end, a Tezos +economic protocol must provide the shell with an interface enabling +roughly the following functionalities: + +- deciding whether an operation is *valid*, and thus can be safely + included into a new block in the blockchain; + +- deciding whether a new block is *valid*, and thus can be safely + appended to the blockchain; + +- including an operation in a block, and executing it on the + blockchain state, effectively modifying the ledger state; and, + +- appending a new block to a Tezos blockchain, and computing the + updated ledger state. + +From a higher-level, *abstract* perspective, the validation system in +the Tezos protocol implements this business logic in a functional, +state-passing machine where: + +- Its state is given by the :ref:`context`, the internal + representation of the state of the Tezos ledger at a given blockchain + level. For instance, the context contains the information of all + activated accounts and contracts, and their balances. More + generally, the context must provide enough information to determine + the validity of operations, and blocks. + +- The *apply_operation* method determines whether an operation is safe + to be executed in a given context. If so, it proceeds to execute it + and outputs the context resulting from the applied operation's + effects. + +- The *apply* method takes as input a block and a context, determines + whether the block can be safely and correctly appended to the + current head of the chain. If so, it appends the block to the chain + and outputs the resulting context, reflecting the state of the + ledger after the block has successfully been applied. *apply* relies + on (i.e., it should call) *apply_operation* to validate and apply + each operation in the block, and compute intermediate states. + +However, the *concrete* API exported from the Tezos economic protocol +does not implement this business logic *monolithically*, as described +above, but it rather presents a more fine-grained API. The rationale +is to provide specialized variations of the core *validation* and +*application* functionality, dubbed :ref:`Validation +modes`. These modes enable the protocol to +distinguish the validation of operations "in the mempool", performed +by the :doc:`prevalidator<../shell/prevalidation>`, from the +validation of the operations included in newly received blocks, +performed by the :ref:`block validator`, in order to +localize validation rules as needed. The resulting concrete API is +specified by the :package-api:`Protocol +` +module in the :doc:`protocol +environment<../shell/protocol_environment>` ``V6``, and it is +implemented by this protocol in the +:package-api:`Main` +module. + +The rest of this document is organized as follows: we first describe +the different validation modes implemented by this Tezos economic +protocol, and then we delve deeper into the particulars of validation +and application for blocks and the operations supported. + .. _validation_modes_kathmandu: Validation modes diff --git a/docs/lima/validation.rst b/docs/lima/validation.rst index f7b4ba9763e6..59beb1838fd1 100644 --- a/docs/lima/validation.rst +++ b/docs/lima/validation.rst @@ -2,6 +2,76 @@ Validation and Application ========================== +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment + +The :doc:`economic protocol` is responsible for +providing the rules that govern the Tezos network, and for enforcing +that this rules implement a correct blockchain protocol. However, it +does so :ref:`in coordination with a Tezos shell`, +who ultimately implements these rules on its behalf. To this end, a +Tezos economic protocol must provide the shell with an interface +enabling roughly the following functionalities: + + - deciding whether an operations is *valid*, and thus can be safely + included in a new block in the blockchain; + + - deciding whether a new block is *valid*, and thus it can be safely + appended to the blockchain; + + - including an operation in a block, and executing it on the + blockchain state, effectively modifying the ledger state; and, + + - appending a new block to a Tezos blockchain, and computing the + updated ledger state. + +From a higher-level, *abstract* perspective, the validation system in +the Tezos protocol implements this business logic in a functional, +state-passing machine where: + +- Its state is given by the :ref:`context`, the internal + representation of the state of Tezos ledger at a given blockchain + level. For instance, the context contains the information of all + activated accounts and contracts, and their balances. More + generally, the context must provide enough information to determine + the validity of operations, and blocks. + +- The *apply_operation* method determines whether an operation is safe + to be executed in a given context. If so, it proceeds to execute it + and outputs the context resulting from the applied operation's + effects. + +- The *apply* method takes as input a block and a context, determines + whether the block can be safely and correctly appended to the + current head of the chain. If so, it appends the block to the chain + and outputs a the resulting context, reflecting the state of the + ledger after the block has successfully been applied. *apply* relies + on (i.e., it should call) *apply_operation* to validate and apply + each operation in the block, and compute intermediate states. + +However, the *concrete* API exported from the Tezos economic protocol +does not implement this business logic *monolithically*, as described +above, but it rather presents a more fine-grained API. The rationale +is to provide specialized variations of the core *validation* and +*application* functionality, dubbed :ref:`Validation +modes`. The latter enable, for instance, to +distinguish and nuance the validation (and application) of operations +"in the mempool" by the :doc:`prevalidator<../shell/prevalidation>`, +from the validation (and application) of the contents of newly baked +block broadcast across the network. The resulting concrete API is +specified by the :package-api:`Protocol +` +module in the :doc:`protocol +environment<../shell/protocol_environment>` ``V7``, and it is +implemented by this protocol in the +:package-api:`Main` +module. + +The rest of this document is organized as follows: we first describe +the different validation modes implemented by this Tezos economic +protocol, and then we delve deeper into the particulars of the validation and application of blocks and the operations supported. + .. _validation_modes_lima: Validation modes -- GitLab From 62007acb0eaa38cd9cb04f5bc9a536cfb953497e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Tue, 7 Jun 2022 16:39:19 +0200 Subject: [PATCH 05/12] doc: add validation modes to protocol validation entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Advances #3921, #3162. Expand description of validation modes. Implemented for Kathmandu, Lima, Alpha -- updated up to Kathmandu. Co-authored-by: vbot Co-authored-by: Germán Delbianco --- docs/alpha/plugins.rst | 2 +- docs/alpha/validation.rst | 112 +++++++++++++++++++++++++++++++-- docs/kathmandu/plugins.rst | 1 + docs/kathmandu/validation.rst | 112 +++++++++++++++++++++++++++++++-- docs/lima/validation.rst | 113 ++++++++++++++++++++++++++++++++-- docs/shell/validation.rst | 6 +- 6 files changed, 328 insertions(+), 18 deletions(-) diff --git a/docs/alpha/plugins.rst b/docs/alpha/plugins.rst index c103825ea0a8..8d4ffb7befd7 100644 --- a/docs/alpha/plugins.rst +++ b/docs/alpha/plugins.rst @@ -97,7 +97,7 @@ block proposal as follows: This filtering strategy is implemented in the ``prefilter`` (see :doc:`../shell/prevalidation`). - +.. _precheck_filter_alpha: Prechecking of manager operations ................................. diff --git a/docs/alpha/validation.rst b/docs/alpha/validation.rst index c7e206897f83..566681fe154f 100644 --- a/docs/alpha/validation.rst +++ b/docs/alpha/validation.rst @@ -60,11 +60,11 @@ does not implement this business logic *monolithically*, as described above, but it rather presents a more fine-grained API. The rationale is to provide specialized variations of the core *validation* and *application* functionality, dubbed :ref:`Validation -modes`. These modes enable the protocol to -distinguish the validation of operations "in the mempool", performed -by the :doc:`prevalidator<../shell/prevalidation>`, from the -validation of the operations included in newly received blocks, -performed by the :ref:`block validator`, in order to +modes`. For example, these modes enable the +protocol to distinguish operations "in the mempool", whose validation +is triggered by the :doc:`prevalidator<../shell/prevalidation>`, from +operations included in newly received blocks, whose validation is +triggered by the :ref:`block validator`, in order to localize validation rules as needed. The resulting concrete API is specified by the :package-api:`Protocol ` @@ -84,26 +84,128 @@ and application for blocks and the operations supported. Validation modes ================ +The Tezos protocol provides different validation modes, intended to be +used by Tezos *shell* and *baker* software implementations when +needing to apply (or to assert the validity) of blocks and operations +under different, or specialized, circumstances -- for example, in +order to *bake* a block. For each of these validation modes, the API +specified by the protocol environment offers an entry point so that +protocol-agnostic components, the Octez shell for instance, are able +to use these different modes. + .. _full_application_alpha: Full Application ~~~~~~~~~~~~~~~~ +The ``Full application`` mode is intended to be used to *fully* +validate and apply blocks. In particular, this mode is used to +validate and apply a **known** block, with a known operation trace. A +Tezos shell implementation should use the full application mode to +decide whether an incoming block can be safely included in the +blockchain. That is, all validity checks are enabled: the block's +signature is correct, and **all** operations included in the block are +valid; the correct amount of consensus operations have been included +in order to satisfy the consensus' threshold, etc. + .. _full_construction_alpha: Full Construction ~~~~~~~~~~~~~~~~~ +The ``Full construction`` mode is intended to be used when a +*delegate* is trying to bake a block and therefore needs to filter the +validity of the desired operation trace, to include only valid +operations. This mode is mostly similar to the ``Full application`` +mode except that *some* global block validity checks are disabled, and +consensus operations are validated with slightly different +preconditions. For instance, since a delegate cannot sign a block +while it is being built, the signature check is disabled, and it will +be left to the baker to correctly sign the resulting block after its +construction is finalized. + +In Octez, this mode is mainly used by the baker daemon. + .. _partial_construction_alpha: Partial Construction ~~~~~~~~~~~~~~~~~~~~ +The ``Partial construction`` mode, also known as ``Mempool mode`` is +used by the :doc:`prevalidator component<../shell/prevalidation>` of +an Octez node to validate incoming operations -- that is, those +not-yet included into blocks. This mode's business-logic is very close +to the ``Full construction`` mode, and the differences boil down to +the intended usage. The partial construction mode does not try to +fully bake a block, but rather to inform the Octez prevalidator on the +potential validity of operations (and whether they can safely included +into a block), so that the latter can **classify** incoming +operations, and further decide how to process them accordingly. + +.. _protocol_classification_alpha: + +The protocol provides the shell with the following classification of +an operation, consisting of one valid kind -- ``Applied`` --, and +:ref:`four error kinds ` defined by the +protocol environment: + +- ``Applied``: the operation is valid and can be included in a + potential block in the current context. + +- ``Temporary``: the operation is invalid in the current context, but + it could *later* become valid -- in the context associated to a + successor block of the current head. For instance, a manager + operation whose counter value is greater than the one expected (a + *"counter-in-the-future"* error), or the manager's balance is + insufficient to pay the operation's fees, etc. + +- ``Branch``: the operation is invalid in the current context and in + any possible context from its future successors, but it might still + be valid in an alternative branch. For example: a manager operation + with a smaller counter than the one expected (a + *"counter-in-the-past"* error), an unexpected endorsement for the + current level, etc. + +- ``Permanent``: the operation is invalid in the current context, and + there isn't any plausible context where it might be or become + valid. For example, an operation carrying an invalid signature. + +- ``Outdated``: the operation is *too old* to be included in a + block. Furthermore, there might be still some value in the + information provided by an ``Outdated`` operation. An example is the + case of an endorsement which was received *too late*, but that could + still be used to form a consensus quorum. + .. _partial_application_alpha: Partial Application ~~~~~~~~~~~~~~~~~~~ +The ``Partial application`` mode is used for :ref:`multi-pass +validation`. Its aim is to provide Tezos shell +implementations with a light-weight (read "fast") block application +mechanism, which can determine whether a block has a *chance* of being +valid or not, in a situation when the provided context is *not a +recent one*. That is, when the block candidate succeeds neither the +head of the chain, nor a close ancestor. + +This validation mode is typically used when the node receives a +significantly large branch -- for instance, while bootstrapping. To +check whether this branch is plausibly valid or potentially malicious +spam, the shell retrieves the context from the most recent common +ancestor between its current head and the announced branch, and +proceeds to "partially apply" each block of this branch using the +common ancestor's context. + +Indeed, by relying on the ancestor context, this mode can *only* +assert the validity of consensus-related preconditions (endorsing +power, block fitness, etc.), as future consensus slots are known in +advance -- how much in advance being specified by the +```` protocol constant. Thus, the `Partial +application` mode provides an over-approximation of the branch's +validity, and as a result intermediate results are not committed on +disk in order to prevent potential attacks. + .. _block_validation_overview_alpha: Block Validation diff --git a/docs/kathmandu/plugins.rst b/docs/kathmandu/plugins.rst index 057080a8beb5..a83f94745920 100644 --- a/docs/kathmandu/plugins.rst +++ b/docs/kathmandu/plugins.rst @@ -101,6 +101,7 @@ This filtering strategy is implemented in the ``prefilter`` (see :doc:`../shell/prevalidation`). +.. _precheck_filter_kathmandu: Prechecking of manager operations ................................. diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst index bfa8c693e760..d38af329a822 100644 --- a/docs/kathmandu/validation.rst +++ b/docs/kathmandu/validation.rst @@ -51,11 +51,11 @@ does not implement this business logic *monolithically*, as described above, but it rather presents a more fine-grained API. The rationale is to provide specialized variations of the core *validation* and *application* functionality, dubbed :ref:`Validation -modes`. These modes enable the protocol to -distinguish the validation of operations "in the mempool", performed -by the :doc:`prevalidator<../shell/prevalidation>`, from the -validation of the operations included in newly received blocks, -performed by the :ref:`block validator`, in order to +modes`. For example, these modes enable the +protocol to distinguish operations "in the mempool", whose validation +is triggered by the :doc:`prevalidator<../shell/prevalidation>`, from +operations included in newly received blocks, whose validation is +triggered by the :ref:`block validator`, in order to localize validation rules as needed. The resulting concrete API is specified by the :package-api:`Protocol ` @@ -75,26 +75,128 @@ and application for blocks and the operations supported. Validation modes ================ +The Tezos protocol provides different validation modes, intended to be +used by Tezos *shell* and *baker* software implementations when +needing to apply (or to assert the validity) of blocks and operations +under different, or specialized, circumstances -- for example, in +order to *bake* a block. For each of these validation modes, the API +specified by the protocol environment offers an entry point so that +protocol-agnostic components, the Octez shell for instance, are able +to use these different modes. + .. _full_application_kathmandu: Full Application ~~~~~~~~~~~~~~~~ +The ``Full application`` mode is intended to be used to *fully* +validate and apply blocks. In particular, this mode is used to +validate and apply a **known** block, with a known operation trace. A +Tezos shell implementation should use the full application mode to +decide whether an incoming block can be safely included in the +blockchain. That is, all validity checks are enabled: the block's +signature is correct, and **all** operations included in the block are +valid; the correct amount of consensus operations have been included +in order to satisfy the consensus' threshold, etc. + .. _full_construction_kathmandu: Full Construction ~~~~~~~~~~~~~~~~~ +The ``Full construction`` mode is intended to be used when a +*delegate* is trying to bake a block and therefore needs to filter the +validity of the desired operation trace, to include only valid +operations. This mode is mostly similar to the ``Full application`` +mode except that *some* global block validity checks are disabled, and +consensus operations are validated with slightly different +preconditions. For instance, since a delegate cannot sign a block +while it is being built, the signature check is disabled, and it will +be left to the baker to correctly sign the resulting block after its +construction is finalized. + +In Octez, this mode is mainly used by the baker daemon. + .. _partial_construction_kathmandu: Partial Construction ~~~~~~~~~~~~~~~~~~~~ +The ``Partial construction`` mode, also known as ``Mempool mode`` is +used by the :doc:`prevalidator component<../shell/prevalidation>` of +an Octez node to validate incoming operations -- that is, those +not-yet included into blocks. This mode's business-logic is very close +to the ``Full construction`` mode, and the differences boil down to +the intended usage. The partial construction mode does not try to +fully bake a block, but rather to inform the Octez prevalidator on the +potential validity of operations (and whether they can safely included +into a block), so that the latter can **classify** incoming +operations, and further decide how to process them accordingly. + +.. _protocol_classification_kathmandu: + +The protocol provides the shell with the following classification of +an operation, consisting of one valid kind -- ``Applied`` --, and +:ref:`four error kinds ` defined by the +protocol environment: + +- ``Applied``: the operation is valid and can be included in a + potential block in the current context. + +- ``Temporary``: the operation is invalid in the current context, but + it could *later* become valid -- in the context associated to a + successor block of the current head. For instance, a manager + operation whose counter value is greater than the one expected (a + *"counter-in-the-future"* error), or the manager's balance is + insufficient to pay the operation's fees, etc. + +- ``Branch``: the operation is invalid in the current context and in + any possible context from its future successors, but it might still + be valid in an alternative branch. For example: a manager operation + with a smaller counter than the one expected (a + *"counter-in-the-past"* error), an unexpected endorsement for the + current level, etc. + +- ``Permanent``: the operation is invalid in the current context, and + there isn't any plausible context where it might be or become + valid. For example, an operation carrying an invalid signature. + +- ``Outdated``: the operation is *too old* to be included in a + block. Furthermore, there might be still some value in the + information provided by an ``Outdated`` operation. An example is the + case of an endorsement which was received *too late*, but that could + still be used to form a consensus quorum. + .. _partial_application_kathmandu: Partial Application ~~~~~~~~~~~~~~~~~~~ +The ``Partial application`` mode is used for :ref:`multi-pass +validation`. Its aim is to provide Tezos shell +implementations with a light-weight (read "fast") block application +mechanism, which can determine whether a block has a *chance* of being +valid or not, in a situation when the provided context is *not a +recent one*. That is, when the block candidate succeeds neither the +head of the chain, nor a close ancestor. + +This validation mode is typically used when the node receives a +significantly large branch -- for instance, while bootstrapping. To +check whether this branch is plausibly valid or potentially malicious +spam, the shell retrieves the context from the most recent common +ancestor between its current head and the announced branch, and +proceeds to "partially apply" each block of this branch using the +common ancestor's context. + +Indeed, by relying on the ancestor context, this mode can *only* +assert the validity of consensus-related preconditions (endorsing +power, block fitness, etc.), as future consensus slots are known in +advance -- how much in advance being specified by the +```` protocol constant. Thus, the `Partial +application` mode provides an over-approximation of the branch's +validity, and as a result intermediate results are not committed on +disk in order to prevent potential attacks. + .. _block_validation_overview_kathmandu: Block Validation diff --git a/docs/lima/validation.rst b/docs/lima/validation.rst index 59beb1838fd1..c99cb5a85678 100644 --- a/docs/lima/validation.rst +++ b/docs/lima/validation.rst @@ -55,11 +55,12 @@ does not implement this business logic *monolithically*, as described above, but it rather presents a more fine-grained API. The rationale is to provide specialized variations of the core *validation* and *application* functionality, dubbed :ref:`Validation -modes`. The latter enable, for instance, to -distinguish and nuance the validation (and application) of operations -"in the mempool" by the :doc:`prevalidator<../shell/prevalidation>`, -from the validation (and application) of the contents of newly baked -block broadcast across the network. The resulting concrete API is +modes`. For example, these modes enable the +protocol to distinguish operations "in the mempool", whose validation +is triggered by the :doc:`prevalidator<../shell/prevalidation>`, from +operations included in newly received blocks, whose validation is +triggered by the :ref:`block validator`, in order to +localize validation rules as needed. The resulting concrete API is specified by the :package-api:`Protocol ` module in the :doc:`protocol @@ -77,26 +78,128 @@ protocol, and then we delve deeper into the particulars of the validation and ap Validation modes ================ +The Tezos protocol provides different validation modes, intended to be +used by Tezos *shell* and *baker* software implementations when +needing to apply (or to assert the validity) of blocks and operations +under different, or specialized, circumstances -- for example, in +order to *bake* a block . For each of these validation modes, the API +specified by the protocol environment offers an entry-point so that +protocol agnostic components, the Octez shell for instance, are able +to use these different modes. + .. _full_application_lima: Full Application ~~~~~~~~~~~~~~~~ +The ``Full application`` mode is intended to be used to *fully* +validate and apply blocks. In particular, this mode is used to +validate and apply a **known** block, with a known operation trace. A +Tezos shell implementation should use the full application mode to +decide whether an incoming block can be safely included in the +blockchain. Thus, all validity checks are enabled: the block's +signature is correct, and **all** operations included in the block are +valid; the correct amount of consensus operations have been included +in order to satisfy the consensus' threshold, etc. + .. _full_construction_lima: Full Construction ~~~~~~~~~~~~~~~~~ +The ``Full construction`` mode is intended to be used when a +*delegate* is trying to bake a block and therefore needs to filter the +validity of the desired operation trace, to include only valid +operations. This mode is mostly similar to the ``Full application`` +mode except that *some* global block validity checks are disabled, and +consensus operations are validated with slightly different +preconditions. For instance, since a delegate cannot sign a block +while it is being built, the signature check is disabled, and it will +be left to the baker to correctly sign the resulting block after its +construction is finalized. + +In Octez, this mode is mainly used by the baker daemon. + .. _partial_construction_lima: Partial Construction ~~~~~~~~~~~~~~~~~~~~ +The ``Partial construction`` mode, also known as ``Mempool mode`` is +used by the :doc:`prevalidator component<../shell/prevalidation>` of +an Octez node to validate incoming operations -- that is, those +not-yet included into blocks. This mode's business-logic is very close +to the ``Full construction`` mode, and the differences boil down to +the intended usage. The partial construction mode does not try to +fully bake a block, but rather to inform the Octez prevalidator on the +potential validity of operations (and whether they can safely included +into a block), so that the later can **classify** incoming operations, +and further decide how to process them accordingly. + +.. _protocol_classification_lima: + +The protocol provides the shell with the following classification of +an operation, consisting of one valid kind -- ``Applied`` --, and +:ref:`four error category kinds ` defined +by the protocol environment: + +- ``Applied``: the operation is valid and can be included in a + potential block in the current context. + +- ``Temporary``: the operation is invalid in the current context, but + it could *later* become valid -- in the context associated to a + successor of the current head. For instance, a manager operation + whose counter value is greater than the one expected (a + *"counter-in-the-future"* error), or the manager's balance is + insufficient to pay the operation's fees, etc. + +- ``Branch``: the operation is invalid in the current context and in + all possible context from its future successors, but it might still + be valid in an alternative branch. For example: a manager operation + with a smaller counter than the one expected (a + *"counter-in-the-past"* error), an unexpected endorsement for the + current level, etc. + +- ``Permanent``: the operation is invalid in the current context, + and there isn't any plausible context where it might become + valid. For example, an operation carrying an invalid signature. + +- ``Outdated``: the operation is *too old* to be included in a + block. Furthermore, there might be still some value in the + information provided by an ``Outdated`` operation. An example is the + case of an endorsement which was received *too late*, but that could + still be used to form a consensus quorum. + .. _partial_application_lima: Partial Application ~~~~~~~~~~~~~~~~~~~ +The ``Partial application`` mode is used for :ref:`multi-pass +validation`. Its aim is to provide Tezos shell +implementations with a light-weight (read, fast) block application +mechanism, which can determine whether a block has a *chance* of being +valid or not, in a situation when the provided context is *not a +recent one*. That is, when the block candidate succeeds neither the +head of the chain, nor a close ancestor. + +This is validation mode is typically used when the node receives a +significantly large branch -- for instance, while bootstrapping. To +check whether this branch is plausibly valid or potentially malicious +spam, the shell retrieves the context from the most recent common +ancestor between its current head and the announced branch, and +proceeds to "partially apply" each block of this branch using the +common ancestor's context. + +Indeed, by relying on the ancestor context, this mode can *only* +assert the validity of consensus-related preconditions (endorsing +power, block fitness, etc.), as future consensus slots are known in +advance -- how much in advance being specified by +the ```` protocol constant. Thus, the `Partial +application` modes provides an over-approximation of the branch's +validity, and as a result intermediate results are not committed on +disk in order to prevent potential attacks. + .. _block_validation_overview_lima: Block Validation diff --git a/docs/shell/validation.rst b/docs/shell/validation.rst index 0c058488bc3d..f103492a19f3 100644 --- a/docs/shell/validation.rst +++ b/docs/shell/validation.rst @@ -103,8 +103,10 @@ difference between the current head and the examined proposal is more than one block, mostly during the initial bootstrap phase, the peer worker launches a *bootstrap pipeline* task. +.. _multi_pass_validation: + A third scheme is planned (but not yet implemented) for validating -alternative chains: the *multipass validator*. This method is quite +alternative chains: the *multi-pass validator*. This method is quite more complex, its goal is to detect erroneous blocks as soon as possible, without having to download all the chain data. This process will work by first validating the block headers, then the operations @@ -123,7 +125,7 @@ signatures. For that to be sound, the fork limit is set to not allow rewinding before the baking and endorsing slots are set. Each of these three peer validator tasks (head increment, bootstrap -pipeline, or multipass) will interact with the distributed DB to get +pipeline, or multi-pass) will interact with the distributed DB to get the data they need (block headers and operations). When they have everything needed for a block, they will call the *block validator*. -- GitLab From 8477127b488ae7100985b4bb55cd373543a0469e Mon Sep 17 00:00:00 2001 From: vbot Date: Wed, 8 Jun 2022 11:44:25 +0200 Subject: [PATCH 06/12] doc: add block validity section to protocol validation entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Advances #3921, #3162. Adds block validation overview subsection to protocol validation entries for Kathmandu, Lima, Alpha -- updated up to Kathmandu. Co-authored-by: Germán Delbianco Co-authored-by: vbot --- docs/alpha/validation.rst | 42 +++++++++++++++++++++++++++++++++ docs/kathmandu/validation.rst | 38 ++++++++++++++++++++++++++++++ docs/lima/validation.rst | 44 ++++++++++++++++++++++++++++++++++- docs/shell/glossary.rst.h | 9 ++++--- 4 files changed, 127 insertions(+), 6 deletions(-) diff --git a/docs/alpha/validation.rst b/docs/alpha/validation.rst index 566681fe154f..3d2f3256cae7 100644 --- a/docs/alpha/validation.rst +++ b/docs/alpha/validation.rst @@ -211,6 +211,48 @@ disk in order to prevent potential attacks. Block Validation ================ +.. FIXME tezos/tezos#3921: + + Adapt to pipelined block validation up to Lima and v7 environment. + +The validity of a blocks depends on a set of precondition checks +implemented in different steps, which happen at different stages of +the application (and the construction) of a block. + +The first step in the process is to decide whether a candidate block +is *well-formed*, that is, that it has the expected "shape" of a valid +block under the current Tezos economic protocol. Given a block +candidate, the block validation process will then verify that the +candidate block declares consistent :ref:`level`, +:ref:`round`, and timestamp values; that it carries a valid +signature, etc. At this step, the block validation process will also +initialize the data-structures required for subsequent steps. + +The second step iterates over the block's operations and proceeds to +apply them sequentially. When at least one operation is found to be +invalid, under the conditions described in +:ref:`operation_validity_alpha` further below, the whole block is +considered as invalid. + +The last step in the block validation process, known as "block +finalization", aims to verify that the collected consensus operations +constitute a sufficiently large :ref:`quorum`. That is, +it will verify that the total endorsing power present in the block is +greater than the ``CONSENSUS_THRESHOLD`` constant. + +This sequence of three steps also yields a new context -- the +resulting state of the Tezos ledger after the application of the +candidate block. The shell may decide to commit this context to disk. + +The Tezos economic protocol also offers a cheap (read "faster") +alternative to determine an over-approximation of the validity of a +block (see :ref:`partial_application_alpha` above). This feature +allows the shell to propagate blocks faster without needing to fully +validate them, speeding-up block propagation over the network. Of +course, as this is an over-approximation, this feature cannot be +considered to provide a safe guarantee that a block will be valid: in +particular, it does not validate all kinds of operations. + .. _operation_validity_alpha: Operation Validation and Application diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst index d38af329a822..c22d55d5a3f1 100644 --- a/docs/kathmandu/validation.rst +++ b/docs/kathmandu/validation.rst @@ -202,6 +202,44 @@ disk in order to prevent potential attacks. Block Validation ================ +The validity of a blocks depends on a set of precondition checks +implemented in different steps, which happen at different stages of +the application (and the construction) of a block. + +The first step in the process is to decide whether a candidate block +is *well-formed*, that is, that it has the expected "shape" of a valid +block under the current Tezos economic protocol. Given a block +candidate, the block validation process will then verify that the +candidate block declares consistent :ref:`level`, +:ref:`round`, and timestamp values; that it carries a valid +signature, etc. At this step, the block validation process will also +initialize the data-structures required for subsequent steps. + +The second step iterates over the block's operations and proceeds to +apply them sequentially. When at least one operation is found to be +invalid, under the conditions described in +:ref:`operation_validity_alpha` further below, the whole block is +considered as invalid. + +The last step in the block validation process, known as "block +finalization", aims to verify that the collected consensus operations +constitute a sufficiently large :ref:`quorum`. That is, +it will verify that the total endorsing power present in the block is +greater than the ``CONSENSUS_THRESHOLD`` constant. + +This sequence of three steps also yields a new context -- the +resulting state of the Tezos ledger after the application of the +candidate block. The shell may decide to commit this context to disk. + +The Tezos economic protocol also offers a cheap (read "faster") +alternative to determine an over-approximation of the validity of a +block (see :ref:`partial_application_kathmandu` above). This feature +allows the shell to propagate blocks faster without needing to fully +validate them, speeding-up block propagation over the network. Of +course, as this is an over-approximation, this feature cannot be +considered to provide a safe guarantee that a block will be valid: in +particular, it does not validate all kinds of operations. + .. _operation_validity_kathmandu: Operation Validation and Application diff --git a/docs/lima/validation.rst b/docs/lima/validation.rst index c99cb5a85678..56192bd0e00c 100644 --- a/docs/lima/validation.rst +++ b/docs/lima/validation.rst @@ -82,7 +82,7 @@ The Tezos protocol provides different validation modes, intended to be used by Tezos *shell* and *baker* software implementations when needing to apply (or to assert the validity) of blocks and operations under different, or specialized, circumstances -- for example, in -order to *bake* a block . For each of these validation modes, the API +order to *bake* a block. For each of these validation modes, the API specified by the protocol environment offers an entry-point so that protocol agnostic components, the Octez shell for instance, are able to use these different modes. @@ -205,6 +205,48 @@ disk in order to prevent potential attacks. Block Validation ================ +.. FIXME tezos/tezos#3921: + + Adapt to pipelined block validation up to Lima and v7 environment. + +The validity of a blocks depends on a set of precondition checks +implemented in different steps, which happen at different stages of +the application (and the construction) of a block. + +The first step in the process is to decide whether a candidate block +is *well-formed*, that is, that it has the expected "shape" of a valid +block under the current Tezos economic protocol. Given a block +candidate, the block validation process will then verify that the +candidate block declares consistent :ref:`level`, +:ref:`round`, and timestamp values; that it carries a valid +signature, etc. At this step, the block validation process will also +initialize the data-structures required for subsequent steps. + +The second step iterates over the block's operations and proceeds to +apply them sequentially. When at least one operation is found to be +invalid, under the conditions described in +:ref:`operation_validity_lima` further below, the whole block is +considered as invalid. + +The last step in the block validation process, known as "block +finalization", aims to verify that the collected consensus operations +constitute a sufficiently large :ref:`quorum`. That is, +it will verify that the total endorsing power present in the block is +greater than the ``CONSENSUS_THRESHOLD`` constant. + +This last step also yields a new context -- the resulting state of the +Tezos ledger after the application of the candidate block. The shell +may decide to commit this context to disk. + +The Tezos economic protocol also offers a cheap (read "faster") +alternative to determine an over-approximation of the validity of a +block (see :ref:`partial_application_lima` above). This feature +allows the shell to propagate blocks faster without needing to fully +validate them, speeding-up block propagation over the network. As this +is an over-approximation, this feature cannot be considered to provide +a safe guarantee that a block will be valid: in particular, it does +not validate all kinds of operations. + .. _operation_validity_lima: Operation Validation and Application diff --git a/docs/shell/glossary.rst.h b/docs/shell/glossary.rst.h index 3926a385b513..4b41bf55ee21 100644 --- a/docs/shell/glossary.rst.h +++ b/docs/shell/glossary.rst.h @@ -7,9 +7,8 @@ _`Block` Blocks conceptually contain a header and a list of Operations_, which are specific to the `economic protocol`_. - The header itself decomposes into a shell header (common to all protocols) and a protocol-specific header. - The shell header contains `protocol`-agnostic data such as the - block predecessor's hash and the block's timestamp. + The header itself decomposes into a :ref:`shell header` (common to all Tezos economic protocols), and a protocol-specific header. + The shell header contains protocol-agnostic data such as the predecessor's block hash and the block's timestamp. _`Context` The state of the blockchain. The context is defined by the @@ -38,8 +37,8 @@ _`Fitness` (a.k.a. score, a.k.a. weight) _`Height` See level_. -_`Level` (a.k.a. height) - The position of a block in the chain, that is, the number of blocks +_`Level` (a.k.a. block height) + The position of a block in the blockchain, that is, the number of blocks since the genesis block, where the genesis block is at level 0. _`Mempool` -- GitLab From 963439e0f6d3808f29f078aa90365fe8a2742bae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Mon, 3 Oct 2022 19:01:46 +0200 Subject: [PATCH 07/12] doc: add operation validation and application section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Contributes to #3162. - Details the validation of manager operations. - Details the application of manager operations. - Explains Backtracked/Applied/Failed/Skipped semantics (Fixes #3111). Updates Kathmandu, and Alpha up to Kathmandu. Co-authored-by: Germán Delbianco Co-authored-by: vbot --- docs/alpha/validation.rst | 132 ++++++++++++++++++++++++++++++ docs/kathmandu/validation.rst | 148 ++++++++++++++++++++++++++++++++++ docs/lima/validation.rst | 132 +++++++++++++++++++++++++++++- 3 files changed, 411 insertions(+), 1 deletion(-) diff --git a/docs/alpha/validation.rst b/docs/alpha/validation.rst index 3d2f3256cae7..0d11724be1f2 100644 --- a/docs/alpha/validation.rst +++ b/docs/alpha/validation.rst @@ -258,12 +258,144 @@ particular, it does not validate all kinds of operations. Operation Validation and Application ==================================== +In the Tezos economic protocol, we dissociate the notion of *validity* +from the notion of *applicability* for operations. A valid operation +is an operation that can be included safely in a block without +affecting the block's validity. Applying an operation, on the other +hand, actually performs the operation's side-effects which can be: +registering a new delegate, executing a smart contract, voting for a +new protocol amendment proposal, etc. + +Note that an operation may fail during the application phase, even +though it has been checked as valid. For example, a smart contract +call that exceeds its gas limit can be included in a block even if an +error is raised at run-time. The application (that is, the operation's +side-effects) will not take effect, but fees will nonetheless be taken +from the account submitting the smart contract call. + +In the sequel we refine the validity conditions and describe the +application process for each of the different validation passes. + +.. FIXME tezos/tezos#3921: + + Expand validity and application for other validation classes. + .. _manager_operations_validity_alpha: Validity of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +In this sub-section, we explain the conditions for manager operations +(and batches of managers operations) to be considered valid and hence +suitable for inclusion in a block. + +Validity of Individual Manager Operations +......................................... + +:ref:`Manager operation` are a class of +operations, issued by a single *manager* account which signs the +operation and pays their fees. The different manager operation kinds +share several common fields: + +- ``source``: the public key's hash of the *source* account of the + manager operation -- that is, the *manager*. +- ``fee``: the amount of tez paid to the baker which decides to + include this operation; +- ``counter``: the manager account's counter, incremented each time + this account executes a manager operation, to prevent + replay-attacks. +- ``gas_limit``: the maximum amount of gas that this operation may + consume before failing. +- ``storage_limit``: the maximum amount of storage that this operation + may require before failing. +- ``operation``: the actual operation(s) -- e.g., transfers, + smart-contract calls, originations, delegations, etc. +- ``signature``: the manager's signature used to prove its identity. + +A manager operation is **valid** if and only if all of the following +conditions hold: + +- The operation source's public key has been previously *revealed*, + unless the operation is itself a ``Reveal`` operation. +- The operation's signature is correct with regard to the manager + account's public key. +- The provided ``counter`` value is the expected one for the manager. +- Depending on the operation's kind, the ``gas_limit`` is high enough + to cover the minimal cost of parsing the operation and further + minimal treatment. +- The manager account is solvent to pay the announced fees. + +Validity of Manager Operation Batches +..................................... + +A :ref:`batch` of manager operations +includes one or more manager operations for sequential and atomic +execution. The atomicity property imposes that the validity of a batch +should entail the validity of each individual operation in the batch, +as defined above. However, it also entails some additional *global* +constraints on manager batches. + +For each of the operations present in a batch, the validation process +must check that the individual constraints above are satisfied, *with +the exception of* the signature constraint. Given that the signature +concerns the whole batch, as all operations in the batch are signed by +the same manager, it suffices to verify the signature only once. + +The **global batch validity** constraint for this economic protocol is +defined as the conjunction of the following conditions: + +- When a ``Reveal`` operation is present, **it must only occur once**, + and **it must be placed at the head** of the batch -- that is, the + ``Reveal`` operation must be the first operation in the batch. +- Every operation in the batch should declare the same `source`. +- Each of the individual operation counters must be incremented + correctly and sequentially. +- The sum of each individual operation's declared fees must be lower + than the account's balance. That is, the manager account must be + solvent to pay the announced fees for all the operations in the + batch. + .. _manager_operations_application_alpha: Application of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the validity of a manager operation (or, a batch of manager +operations) is established, the protocol proceeds to apply the +operation. This first step in this application phase is to transfer +the operation's fees to the baker that included this operation. Then, +the actual application depends on the operation kind. For instance, +this could be a smart contract execution, enacting a delegation, or +multiple actions executed as a batch. The application of a batch of +manager operations consists of the sequential application of each +operation in the batch, following their inclusion order -- the head of +the batch being the first manager operation being applied. + +The application of each individual manager operation may either +succeed -- and therefore be reported as ``Applied`` --, or indeed fail +with an error. In both cases, the fees are taken and the counter for +the operation's manager is incremented. + +When a manager operation fails, every side-effect which was previously +performed is backtracked. Moreover, the (rest of the) batch has to be +aborted. Thus, depending on the position of the manager operation in +a batch, its failure has to be propagated accordingly: + +- If there were other successfully applied operations in the batch + prior to the offending one, the effect of each of them has to be + reverted, and each of them will be reported as ``Backtracked``. + +- If there were other operations pending application after the + offending one, their application is aborted, and they are reported + as ``Skipped``. + +For example, let's consider a simple batch of three manager operations +``[op1, op2, op3]``, if ``op1`` is successful but ``op2`` fails, the +ticket result for the application of the manager operation batch will +report: + +* ``op1`` -- ``Backtracked``, ```op1`` was applied successfully, but + after ``op2`` failed, the operation was canceled; +* ``op2`` -- ``Failed``, the application of this particular operation failed; +* ``op3`` -- ``Skipped``, this operation was never executed because + ``op2`` had previously failed. diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst index c22d55d5a3f1..eb768a834605 100644 --- a/docs/kathmandu/validation.rst +++ b/docs/kathmandu/validation.rst @@ -245,12 +245,160 @@ particular, it does not validate all kinds of operations. Operation Validation and Application ==================================== +In the Tezos economic protocol, we dissociate the notion of *validity* +from the notion of *applicability* for operations. A valid operation +is an operation that can be included safely in a block without +affecting the block's validity. Applying an operation, on the other +hand, actually performs the operation's side-effects which can be: +registering a new delegate, executing a smart contract, voting for a +new protocol amendment proposal, etc. + +Note that an operation may fail during the application phase, even +though it has been checked as valid. For example, a smart contract +call that exceeds its gas limit can be included in a block even if an +error is raised at run-time. The application (that is, the operation's +side-effects) will not take effect, but fees will nonetheless be taken +from the account submitting the smart contract call. + +Unfortunately, the implementation of these two morally distinct phases +in previous Tezos economic protocols has been entangled within the +``apply_operation`` method. As a result, using the latter method did +not allow in practice to distinguish between invalid and +valid-yet-inapplicable operations, and external *ad hoc* mechanisms +were developed to circumvent this limitation -- see the :ref:`precheck +prevalidator filter` entry. + +This economic protocol implements the first stage of a pipelined +validation discipline which targets to reflect, operationally, this +separation of concerns. However, in this economic protocol, only +manager operations are allowed to fail during the application phase +and yet still be included in a block. + +The explicit conditions for a manager operation to be considered as +valid, are given in the :ref:`manager_operations_validity_alpha` +sub-section below. + +.. FIXME tezos/tezos#3921: + + Expand validity and application for other validation classes. + .. _manager_operations_validity_kathmandu: Validity of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +In this sub-section, we explain the conditions for manager operations +(and batches of managers operations) to be considered valid and hence +suitable for inclusion in a block. + + +Validity of Individual Manager Operations +......................................... + +:ref:`Manager operation` are a class of +operations, issued by a single *manager* account which signs the +operation and pays their fees. The different manager operation kinds +share several common fields: + +- ``source``: the public key's hash of the *source* account of the + manager operation -- that is, the *manager*. +- ``fee``: the amount of tez paid to the baker which decides to + include this operation; +- ``counter``: the manager account's counter, incremented each time + this account executes a manager operation, to prevent + replay-attacks. +- ``gas_limit``: the maximum amount of gas that this operation may + consume before failing. +- ``storage_limit``: the maximum amount of storage that this operation + may require before failing. +- ``operation``: the actual operation(s) -- e.g., transfers, + smart-contract calls, originations, delegations, etc. +- ``signature``: the manager's signature used to prove its identity. + +A manager operation is **valid** if and only if all of the following +conditions hold: + +- The operation source's public key has been previously *revealed*, + unless the operation is itself a ``Reveal`` operation. +- The operation's signature is correct with regard to the manager + account's public key. +- The provided ``counter`` value is the expected one for the manager. +- Depending on the operation's kind, the ``gas_limit`` is high enough + to cover the minimal cost of parsing the operation and further + minimal treatment. +- The manager account is solvent to pay the announced fees. + +Validity of Manager Operation Batches +..................................... + +A :ref:`batch` of manager +operations includes one or more manager operations for sequential and +atomic execution. The atomicity property imposes that the validity of +a batch should entail the validity of each individual operation in the +batch, as defined above. However, it also entails some additional +*global* constraints on manager batches. + +For each of the operations present in a batch, the validation process +must check that the individual constraints above are satisfied, *with +the exception of* the signature constraint. Given that the signature +concerns the whole batch, as all operations in the batch are signed by +the same manager, it suffices to verify the signature only once. + +The **global batch validity** constraint for this economic protocol is +defined as the conjunction of the following conditions: + +- When a ``Reveal`` operation is present, **it must only occur once**, + and **it must be placed at the head** of the batch -- that is, the + ``Reveal`` operation must be the first operation in the batch. +- Every operation in the batch should declare the same `source`. +- Each of the individual operation counters must be incremented + correctly and sequentially. +- The sum of each individual operation's declared fees must be lower + than the account's balance. That is, the manager account must be + solvent to pay the announced fees for all the operations in the + batch. + .. _manager_operations_application_kathmandu: Application of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the validity of a manager operation (or, a batch of manager +operations) is established, the protocol proceeds to apply the +operation. This first step in this application phase is to transfer +the operation's fees to the baker that included this operation. Then, +the actual application depends on the operation kind. For instance, +this could be a smart contract execution, enacting a delegation, or +multiple actions executed as a batch. The application of a batch of +manager operations consists of the sequential application of each +operation in the batch, following their inclusion order -- the head of +the batch being the first manager operation being applied. + +The application of each individual manager operation may either +succeed -- and therefore be reported as ``Applied`` --, or indeed fail +with an error. In both cases, the fees are taken and the counter for +the operation's manager is incremented. + +When a manager operation fails, every side-effect which was previously +performed is backtracked. Moreover, the (rest of the) batch has to be +aborted. Thus, depending on the position of the manager operation in +a batch, its failure has to be propagated accordingly: + +- If there were other successfully applied operations in the batch + prior to the offending one, the effect of each of them has to be + reverted, and each of them will be reported as ``Backtracked``. + +- If there were other operations pending application after the + offending one, their application is aborted, and they are reported + as ``Skipped``. + +For example, let's consider a simple batch of three manager operations +``[op1, op2, op3]``, if ``op1`` is successful but ``op2`` fails, the +ticket result for the application of the manager operation batch will +report: + +* ``op1`` -- ``Backtracked``, ```op1`` was applied successfully, but + after ``op2`` failed, the operation was canceled; +* ``op2`` -- ``Failed``, the application of this particular operation failed; +* ``op3`` -- ``Skipped``, this operation was never executed because + ``op2`` had previously failed. diff --git a/docs/lima/validation.rst b/docs/lima/validation.rst index 56192bd0e00c..4505f9ab1592 100644 --- a/docs/lima/validation.rst +++ b/docs/lima/validation.rst @@ -252,12 +252,142 @@ not validate all kinds of operations. Operation Validation and Application ==================================== -.. _manager_operations_validity_lima: +In the Tezos economic protocol, we dissociate the notion of *validity* +from the notion of *applicability* for operations. A valid operation +is an operation that can be included safely in a block without +affecting the block's validity. Applying an operation, on the other +hand, actually performs the operation's side-effects which can be: +registering to a new delegate, executing a smart contract, voting for +a new protocol amendment proposal, etc. + +While an operation may fail during the application phase, it could +still be considered as valid. For example, a smart contract call that +exceeds its gas limit can be included in a block even if an error is +raised at run-time. The application (that is, the operation's +side-effects) will not take effect, but fees will nonetheless be taken +from the account submitting the smart contract call. + +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment: add + validity and application for other validation classes. Validity of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +In this sub-section, we explain the conditions for manager operations +(and batches of managers operations) to be considered valid and hence +includible in a block. + +Validity of Individual Manager Operations +......................................... + +:ref:`Manager operation` are a class of +operations, issued by a single *manager* account which signs the +operation and pays their fees. The different manager operation kinds +share several common fields: + +- ``source``: the public key's hash of the *source* account of the + manager operation -- that is, the *manager*. +- ``fee``: the amount of tez paid to the baker which decides to + include this operation; +- ``counter``: the manager account's counter, incremented each time + this account executes a manager operation, to prevent + replay-attacks. +- ``gas_limit``: the maximum amount of gas that this operation may + consume before failing. +- ``storage_limit``: the maximum amount of storage that this operation + may require before failing. +- ``operation``: the actual operation(s) -- e.g., transfers, + smart-contract calls, originations, delegations, etc. +- ``signature``: the manager's signature used to prove its identity. + +A manager operation is **valid** if and only if all of the following +conditions hold: + +- The operation source's public key has been previously *revealed*, + unless the operation is itself is a ``Reveal`` operation. +- The operation's signature is correct with regard to the manager + account's public key. +- The provided ``counter`` value is the expected one for the manager. +- Depending on the operation's kind, the ``gas_limit`` is high enough + to cover the minimal cost of the parsing the operation and further + minimal treatment. +- The manager account is solvent enough to be able to pay the + announced fees. + +Validity of Manager Operation Batches +..................................... + +A :ref:`batch` of manager +operations includes one or more manager operations for sequential and +atomic execution. The atomicity property imposes that the validity of +a batch should entail the validity of each individual operation in the +batch. As a result, the validity of a batch entails the lifting of the +individual validity defined above, and additional *global* constraints +on manager batches. + +For each of the operations present in a batch, the validation process +must check that the individual constraints above are satisfied, *with +the exception of* the signature constraint. Given that the signature +concerns the whole batch, as all operations in the batch are signed by +the same manager, it suffices to verify the signature only once. + +The **global batch validity** constraint for this economic protocol is +defined as the conjunction of the following conditions: + +- When a ``Reveal`` operation is present, **it must only occur once**, + and **it must be placed at the head** of the batch -- that is, the + ``Reveal`` operation must be the first operation in the batch. +- Every operation in the batch should declare the same `source`. +- Each of the individual operation counters must be incremented + correctly and sequentially. +- The sum of each individual operation's declared fees must be lower + than the account's balance. That is, the manager account must be + solvent enough to cover the announced fees for all the operations in + the batch. + .. _manager_operations_application_lima: Application of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the validity of a manager operation (or, a batch of manager +operations) is established, the protocol proceeds to apply the +operation. This first step in this application phase is to transfer +the operation's fees to the baker that included this operation. Then, +the actual application depends on the operation kind. For instance, +this could be a smart contract execution, enacting a delegation or +multiple actions executed as a batch. The application of a batch of +manager consists of the sequential application of each operation in +the batch, following their inclusion order -- the head of the batch +being the first manager operation being applied. + +The application of each individual manager operation may either +succeed -- and therefore reported as ``Applied`` --, or indeed fail +with an error. In both cases, the fees are taken and the counter for +the operation's manager is incremented. + +When a manager operation fails, every side-effect which was previously +performed is backtracked. Moreover, depending on the position of the +manager operation in a batch, this failure has to be propagated +accordingly: + +- If there were other successfully applied operations in the batch + prior to the offending one, the effect of each of them has to be + reverted, and each of them will be reported as ``Backtracked``. + +- If there were other operations pending application after the + offending one, their application is aborted, and they are reported + as ``Skipped``. + +For example, let's consider a simple batch of three manager operations +``[op1, op2, op3]``, if ``op1`` is successful but ``op2`` fails, the +ticket result for the application of the manager operation batch will +report: + +* ``op1`` -- ``Backtracked``, ```op1`` was applied successfully, but + after ``op2`` failed, the operation was cancelled; +* ``op2`` -- ``Failed``, the application of this particular operation failed; +* ``op3`` -- ``Skipped``, this operation was never executed because + ``op2`` had previously failed. -- GitLab From f562b7cc79d9a004d4ec0d844623871228225691 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Mon, 3 Oct 2022 19:02:33 +0200 Subject: [PATCH 08/12] doc: make a few nits to protocol glossary Kathmandu, Alpha --- docs/alpha/glossary.rst | 58 ++++++++++++++++++++------------- docs/kathmandu/glossary.rst | 64 ++++++++++++++++++++++--------------- docs/lima/glossary.rst | 60 +++++++++++++++++++++------------- docs/user/various.rst | 2 ++ 4 files changed, 114 insertions(+), 70 deletions(-) diff --git a/docs/alpha/glossary.rst b/docs/alpha/glossary.rst index 3deedc0fcd3b..7d50b69d4479 100644 --- a/docs/alpha/glossary.rst +++ b/docs/alpha/glossary.rst @@ -74,9 +74,13 @@ _`Cycle` A cycle is a set of consecutive blocks. E.g., cycle 12 started at block_ level 49152 and ended at block_ level 53248. - Cycles are used as a unit of “time” in the block_ chain. For example, the - different phases in the amendment voting procedures are defined based on - cycles. + Cycles are used as a unit of “time” in the blockchain. For + example, the different phases in the amendment voting procedures + are defined based on cycles. + + The length of a cycle is a (parametric) protocol + :ref:`constant`, and thus might change across different + Tezos protocols. _`Delegate` An `implicit account`_ to which an account_ has delegated their @@ -92,13 +96,16 @@ _`Delegation` the account_. _`Double signing` - When a baker_ signs two different blocks at the same level and same round, - it is called double baking. Double baking is detrimental to the network and might be - indicative of an attempt to double spend. - The same goes for signing two different endorsements at the same level and the same round. - As such, double signing is punished by the - network: an accuser_ can provide proof of the double signing to be awarded - part of the double signer's deposit. + The action of a baker_ signing two different blocks at the same + level and same round is called *double baking*. Double baking + is detrimental to the network and might be indicative of an + attempt to double spend. The same goes for signing two different + *endorsements* at the same level and the same round. + + Double signing (i.e. double baking or double endorsing) is + punished by the network: an accuser_ can provide proof of the + double signing to be awarded part of the double signer's deposit + -- see :ref:`Slashing`. _`Failing Noop` The ``Failing_noop`` operation implements a *No-op*, which always @@ -109,11 +116,16 @@ _`Failing Noop` computational semantics. _`Fee` - To ensure responsible use of computation resources of other nodes, and also to encourage active participation in the consensus protocol, there are some - fees that users pay to bakers for including their operations in blocks. - For example, fees are paid to a baker for operations such as a transaction_ or a revelation of a public key. + To ensure responsible use of computation resources of other + nodes, and also to encourage active participation in the consensus + protocol, users pay fees to bakers for including (some of) their + operations in blocks. For example, fees are paid to a baker for + operations such as a transaction_ or a revelation of a public key. + + Currently, only :ref:`manager operations` + require collecting fees from its sender account_. - See also `burn`_. + See also `burn`_. _`Gas` A measure of the number of elementary operations_ performed during @@ -125,7 +137,7 @@ _`Implicit account` contract`_, an `Implicit account`_ cannot include a script and it cannot reject incoming transactions. - If registered, an `implicit account`_ can act as a delegate_. + If *registered*, an `implicit account`_ can act as a delegate_. The address of an `implicit account`_ always starts with the letters `tz` followed by `1`, `2` or `3` (depending on the @@ -169,7 +181,8 @@ _`Originated account` See `smart contract`_. _`Origination` - An operation_ to create a `smart contract`_. + A manager operation_ whose purpose is to create -- that + is, to deploy -- a `smart contract`_ on the Tezos blockchain. _`Round` An attempt to reach consensus on a block at a given level. @@ -183,10 +196,11 @@ _`Roll` deprecated; see `Minimal stake`_. _`Smart contract` - Account_ which is associated to a :ref:`Michelson ` script. They are - created with an explicit origination_ operation and are therefore - sometimes called originated accounts. The address of a smart - contract always starts with the letters ``KT1``. + Account_ which is associated to a :ref:`Michelson + ` script. They are created with an + explicit origination_ operation and are therefore sometimes called + originated accounts. The address of a smart contract always starts + with the letters ``KT1``. _`Smart Optimistic Rollups` Smart optimistic rollups constitute a `layer 2`_ solution that can be used to deploy either a general-purpose polyvalent layer 2 blockchain @@ -198,8 +212,8 @@ _`Transaction` `smart contract`_. _`Validation pass` - An index that can be associated with different kinds of - operations, in order to group them into classes. Validation passes + An index (a natural number) associated with a particular kind of + operations, allowing to group them into classes. Validation passes enable prioritizing the :ref:`validation and application` of certain classes of operations. diff --git a/docs/kathmandu/glossary.rst b/docs/kathmandu/glossary.rst index 901b4c5ded96..6d7183b4026b 100644 --- a/docs/kathmandu/glossary.rst +++ b/docs/kathmandu/glossary.rst @@ -74,9 +74,13 @@ _`Cycle` A cycle is a set of consecutive blocks. E.g., cycle 12 started at block_ level 49152 and ended at block_ level 53248. - Cycles are used as a unit of “time” in the block_ chain. For example, the - different phases in the amendment voting procedures are defined based on - cycles. + Cycles are used as a unit of “time” in the blockchain. For + example, the different phases in the amendment voting procedures + are defined based on cycles. + + The length of a cycle is a (parametric) protocol + :ref:`constant`, and thus might change across different + Tezos protocols. _`Delegate` An `implicit account`_ to which an account_ has delegated their @@ -92,13 +96,16 @@ _`Delegation` the account_. _`Double signing` - When a baker_ signs two different blocks at the same level and same round, - it is called double baking. Double baking is detrimental to the network and might be - indicative of an attempt to double spend. - The same goes for signing two different endorsements at the same level and the same round. - As such, double signing is punished by the - network: an accuser_ can provide proof of the double signing to be awarded - part of the double signer's deposit. + The action of a baker_ signing two different blocks at the same + level and same round is called *double baking*. Double baking + is detrimental to the network and might be indicative of an + attempt to double spend. The same goes for signing two different + *endorsements* at the same level and the same round. + + Double signing (i.e. double baking or double endorsing) is + punished by the network: an accuser_ can provide proof of the + double signing to be awarded part of the double signer's deposit + -- see :ref:`Slashing`. _`Failing Noop` The ``Failing_noop`` operation implements a *No-op*, which always @@ -109,11 +116,16 @@ _`Failing Noop` computational semantics. _`Fee` - To ensure responsible use of computation resources of other nodes, and also to encourage active participation in the consensus protocol, there are some - fees that users pay to bakers for including their operations in blocks. - For example, fees are paid to a baker for operations such as a transaction_ or a revelation of a public key. + To ensure responsible use of computation resources of other + nodes, and also to encourage active participation in the consensus + protocol, users pay fees to bakers for including (some of) their + operations in blocks. For example, fees are paid to a baker for + operations such as a transaction_ or a revelation of a public key. + + Currently, only :ref:`manager operations` + require collecting fees from its sender account_. - See also `burn`_. + See also `burn`_. _`Gas` A measure of the number of elementary operations_ performed during @@ -125,7 +137,7 @@ _`Implicit account` contract`_, an `Implicit account`_ cannot include a script and it cannot reject incoming transactions. - If registered, an `implicit account`_ can act as a delegate_. + If *registered*, an `implicit account`_ can act as a delegate_. The address of an `implicit account`_ always starts with the letters `tz` followed by `1`, `2` or `3` (depending on the @@ -171,7 +183,8 @@ _`Originated account` See `smart contract`_. _`Origination` - An operation_ to create a `smart contract`_. + A manager operation_ whose purpose is to create -- that + is, to deploy -- a `smart contract`_ on the Tezos blockchain. _`Round` An attempt to reach consensus on a block at a given level. @@ -185,10 +198,11 @@ _`Roll` deprecated; see `Minimal stake`_. _`Smart contract` - Account_ which is associated to a :ref:`Michelson ` script. They are - created with an explicit origination_ operation and are therefore - sometimes called originated accounts. The address of a smart - contract always starts with the letters ``KT1``. + Account_ which is associated to a :ref:`Michelson + ` script. They are created with an + explicit origination_ operation and are therefore sometimes called + originated accounts. The address of a smart contract always starts + with the letters ``KT1``. _`Smart Optimistic Rollups` Smart optimistic rollups constitute a `layer 2`_ solution that can be used to deploy either a general-purpose polyvalent layer 2 blockchain @@ -200,11 +214,11 @@ _`Transaction` `smart contract`_. _`Validation pass` - An index (a natural number) that can be associated - to the different kind of operations, in order to group them into - classes. Validation passes enable prioritizing the - :ref:`validation and application` of - certain classes of operations. + An index (a natural number) associated with a particular kind of + operations, allowing to group them into classes. Validation passes + enable prioritizing the :ref:`validation and + application` of certain classes of + operations. _`Voting period` Any of the ``proposal``, ``exploration``, ``cooldown``, diff --git a/docs/lima/glossary.rst b/docs/lima/glossary.rst index 0c48bd863866..ddd11a51e204 100644 --- a/docs/lima/glossary.rst +++ b/docs/lima/glossary.rst @@ -74,9 +74,13 @@ _`Cycle` A cycle is a set of consecutive blocks. E.g., cycle 12 started at block_ level 49152 and ended at block_ level 53248. - Cycles are used as a unit of “time” in the block_ chain. For example, the - different phases in the amendment voting procedures are defined based on - cycles. + Cycles are used as a unit of “time” in the blockchain. For + example, the different phases in the amendment voting procedures + are defined based on cycles. + + The length of a cycle is a (parametric) protocol + :ref:`constant`, and thus might change across different + Tezos protocols. _`Delegate` An `implicit account`_ to which an account_ has delegated their @@ -92,13 +96,16 @@ _`Delegation` the account_. _`Double signing` - When a baker_ signs two different blocks at the same level and same round, - it is called double baking. Double baking is detrimental to the network and might be - indicative of an attempt to double spend. - The same goes for signing two different endorsements at the same level and the same round. - As such, double signing is punished by the - network: an accuser_ can provide proof of the double signing to be awarded - part of the double signer's deposit. + The action of a baker_ signing two different blocks at the same + level and same round is called *double baking*. Double baking + is detrimental to the network and might be indicative of an + attempt to double spend. The same goes for signing two different + *endorsements* at the same level and the same round. + + Double signing (i.e. double baking or double endorsing) is + punished by the network: an accuser_ can provide proof of the + double signing to be awarded part of the double signer's deposit + -- see :ref:`Slashing`. _`Failing Noop` The ``Failing_noop`` operation implements a *No-op*, which always @@ -109,11 +116,16 @@ _`Failing Noop` computational semantics. _`Fee` - To ensure responsible use of computation resources of other nodes, and also to encourage active participation in the consensus protocol, there are some - fees that users pay to bakers for including their operations in blocks. - For example, fees are paid to a baker for operations such as a transaction_ or a revelation of a public key. + To ensure responsible use of computation resources of other + nodes, and also to encourage active participation in the consensus + protocol, users pay fees to bakers for including (some of) their + operations in blocks. For example, fees are paid to a baker for + operations such as a transaction_ or a revelation of a public key. + + Currently, only :ref:`manager operations` + require collecting fees from its sender account_. - See also `burn`_. + See also `burn`_. _`Gas` A measure of the number of elementary operations_ performed during @@ -125,7 +137,7 @@ _`Implicit account` contract`_, an `Implicit account`_ cannot include a script and it cannot reject incoming transactions. - If registered, an `implicit account`_ can act as a delegate_. + If *registered*, an `implicit account`_ can act as a delegate_. The address of an `implicit account`_ always starts with the letters `tz` followed by `1`, `2` or `3` (depending on the @@ -169,7 +181,8 @@ _`Originated account` See `smart contract`_. _`Origination` - An operation_ to create a `smart contract`_. + A manager operation_ whose purpose is to create -- that + is, to deploy -- a `smart contract`_ on the Tezos blockchain. _`Round` An attempt to reach consensus on a block at a given level. @@ -183,10 +196,11 @@ _`Roll` deprecated; see `Minimal stake`_. _`Smart contract` - Account_ which is associated to a :ref:`Michelson ` script. They are - created with an explicit origination_ operation and are therefore - sometimes called originated accounts. The address of a smart - contract always starts with the letters ``KT1``. + Account_ which is associated to a :ref:`Michelson + ` script. They are created with an + explicit origination_ operation and are therefore sometimes called + originated accounts. The address of a smart contract always starts + with the letters ``KT1``. _`Smart Optimistic Rollups` Smart optimistic rollups constitute a `layer 2`_ solution that can be used to deploy either a general-purpose polyvalent layer 2 blockchain @@ -198,10 +212,10 @@ _`Transaction` `smart contract`_. _`Validation pass` - An index that can be associated to the different kind of - operations, in order to group them into classes. Validation passes + An index (a natural number) associated with a particular kind of + operations, allowing to group them into classes. Validation passes enable prioritizing the :ref:`validation and - application` of certain classes of + application` of certain classes of operations. _`Voting period` diff --git a/docs/user/various.rst b/docs/user/various.rst index 135d688a0fce..02c049737712 100644 --- a/docs/user/various.rst +++ b/docs/user/various.rst @@ -63,6 +63,8 @@ indicates a recommended course of action. | 255 | like 127 but an error and an error occurred during exit (e.g., ``kill -9``) | check output/logs, clean-up leftover files | +-------------+----------------------------------------------------------------------------------+------------------------------------------------------------------+ +.. _failing_noop: + Failing_noop operation ---------------------- -- GitLab From 9782d16f230e402af4dbb6966f78516ae5802578 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Delbianco?= Date: Thu, 29 Sep 2022 18:38:41 +0200 Subject: [PATCH 09/12] doc: update precheck filters and plugins entries wrt to K - Clarifies that 1M is enforced globally. - Leaves work for Lima and Alpha as TODO --- docs/alpha/plugins.rst | 12 +++++++----- docs/alpha/precheck.rst | 7 +++++++ docs/kathmandu/plugins.rst | 7 +------ docs/kathmandu/precheck.rst | 34 +++++++++++++++------------------- docs/lima/plugins.rst | 17 ++++++++++------- docs/lima/precheck.rst | 7 +++++++ 6 files changed, 47 insertions(+), 37 deletions(-) diff --git a/docs/alpha/plugins.rst b/docs/alpha/plugins.rst index 8d4ffb7befd7..26647e5ccdd7 100644 --- a/docs/alpha/plugins.rst +++ b/docs/alpha/plugins.rst @@ -102,6 +102,13 @@ This filtering strategy is implemented in the ``prefilter`` (see Prechecking of manager operations ................................. +.. FIXME tezos/tezos#3938: + + This section doesn't make much sense after the pipelining project + has plugged validate into the plugin for Lima. Parts of this + section be integrated into plugin.rst, and the relevant definitions + should point to the validation entry. + The aim of the ``precheck`` filter is to avoid fully executing manager operations before deciding whether to gossip them to the network. @@ -120,11 +127,6 @@ increased the chain's fitness), only one operation per manager is propagated. All other received operations originating from the same manager will be classified as ``Branch_delayed`` and will not be propagated. -This criterion is used only by the prevalidator to decide the propagation of -operations. A baker can still include several operations originating from the same -manager in a single block, provided that it gets them in time (note that they can be -propagated by nodes using different versions or implementations). - Alternatively, a user can inject an operation with the same manager and the same counter, but with a higher fee to replace an already existing operation in the prevalidator. Only one of the two operations will be eventually diff --git a/docs/alpha/precheck.rst b/docs/alpha/precheck.rst index 2a37ed54e6b7..9f9786127292 100644 --- a/docs/alpha/precheck.rst +++ b/docs/alpha/precheck.rst @@ -1,6 +1,13 @@ Prechecking of manager operations ================================= +.. FIXME tezos/tezos#3938: + + This section doesn't make much sense after the pipelining project + has plugged validate into the plugin for Lima. Parts of this + section be integrated into plugin.rst, and the relevant definitions + should point to the validation entry. + The prevalidator of Octez prevents the propagation of non-valid operations by trying to apply (or execute) them. However, executing diff --git a/docs/kathmandu/plugins.rst b/docs/kathmandu/plugins.rst index a83f94745920..4ef29a90b5a6 100644 --- a/docs/kathmandu/plugins.rst +++ b/docs/kathmandu/plugins.rst @@ -124,11 +124,6 @@ increased the chain's fitness), only one operation per manager is propagated. All other received operations originating from the same manager will be classified as ``Branch_delayed`` and will not be propagated. -This criterion is used only by the prevalidator to decide the propagation of -operations. A baker can still include several operations originating from the same -manager in a single block, provided that it gets them in time (note that they can be -propagated by nodes using different versions or implementations). - Alternatively, a user can inject an operation with the same manager and the same counter, but with a higher fee to replace an already existing operation in the prevalidator. Only one of the two operations will be eventually @@ -156,7 +151,7 @@ In addition to quick detection of operations that have no chance to be prechecked or applied in the current context, the mempool's ``prefilter`` provides a priority for each successfully filtered operation. Concretely, the priority is either ``High``, ``Medium`` or ``Low`` in the current implementation, depending -on the :ref:`validation pass`.Some extra information (like the fees, or the gas/fees +on their :ref:`validation pass`.Some extra information (like the fees, or the gas/fees ratio of manager operations) are also provided along the priorities to enable fine-grained operations ordering. This extra information is similar to the one used by the baker's diff --git a/docs/kathmandu/precheck.rst b/docs/kathmandu/precheck.rst index a4339589255d..0a4b3a1c5eba 100644 --- a/docs/kathmandu/precheck.rst +++ b/docs/kathmandu/precheck.rst @@ -2,22 +2,21 @@ Prechecking of manager operations ================================= The prevalidator of Octez prevents the propagation of non-valid -operations by trying to apply (or execute) them. -However, executing -manager operations (like complex smart contract calls) can be -time-consuming. -To ensure a high throughput of the Tezos protocol, the -propagation of operations should be as fast as possible. This calls -for a lighter approach. +operations by trying to :ref:`apply` +them. However, executing :ref:`manager +operations` can be time-consuming. To +ensure a high throughput of the Tezos protocol, the propagation of +operations should be as fast as possible. This calls for a lighter +approach. Prechecking manager operations implements such a lighter approach, by discarding most invalid operations, without executing any of them. -This is possible because the validity of an operation (in the sense, -being able to include it in a block) -depends on its solvability (see below :ref:`solvability_kathmandu`). +This is possible because the :ref:`validity of a manager +operation` depends on its +*solvability* -- :ref:`defined further below`. -Fortunately, it turns out that deciding whether an operation is solvable can be achieved without -applying it. +Fortunately, it turns out that deciding whether an operation is +solvable can be achieved without applying it. However, restricting the propagation to solvable operations is not enough to ensure resilience against DDoS attacks. @@ -33,23 +32,21 @@ block. Note however that one can still inject successive operations with the same counter using the ``replace-by-fee`` feature. - .. _solvability: .. _solvability_kathmandu: Solvable operations ------------------- -The solvability of an operation depends on the content of the Tezos -context in which it is applied. - +The solvability of a manager operation depends on the content of the +Tezos context in which it is applied. In the following, we consider the context kept by the prevalidator (see :doc:`../shell/prevalidation`). Checking operation solvability with this context is lighter than with the contexts for block validation or block construction: it can be decided without checking whether the gas announced by the operation -is below the remaining gas for the whole block. +is below the remaining gas for the whole block -- . Checking solvability consists in verifying that the operation is well-formed and that fees can be paid: no execution of operations (especially of smart-contract code) is done to determine @@ -68,7 +65,6 @@ Roughly, a manager operation is solvable when the following conditions hold: operation; and - the signature of the operation is valid. - Co-precheckable operations -------------------------- @@ -86,7 +82,7 @@ In this case, the operations could be included in the next block in any order, modulo block limits (eg. maximum gas, block size limit, etc). -The precheck of a manger operation, with respect to a context and a +The precheck of a manager operation, with respect to a context and a set of co-precheckable operations, consists in checking whether adding the operation to the set preserves the operations of the set being co-precheckable. diff --git a/docs/lima/plugins.rst b/docs/lima/plugins.rst index 3972ee212532..6027fc72b44c 100644 --- a/docs/lima/plugins.rst +++ b/docs/lima/plugins.rst @@ -102,13 +102,21 @@ This filtering strategy is implemented in the ``prefilter`` (see Prechecking of manager operations ................................. +.. FIXME tezos/tezos#3938: + + This section doesn't make much sense after the pipelining project + has plugged validate into the plugin for Lima. Parts of this + section be integrated into plugin.rst, and the relevant definitions + should point to the validation entry. + The aim of the ``precheck`` filter is to avoid fully executing manager operations before deciding whether to gossip them to the network. The detailed description of this feature is given in :doc:`./precheck`. For operations other than manager operations, the -``precheck`` filter is a no-op, which entails that these operations need to be -fully executed to decide their propagation (see :doc:`../shell/prevalidation`). +``precheck`` filter is a no-op, which entails that these operations +need to be fully executed to decide their propagation (see +:doc:`../shell/prevalidation`). One manager operation per manager per block @@ -120,11 +128,6 @@ increased the chain's fitness), only one operation per manager is propagated. All other received operations originating from the same manager will be classified as ``Branch_delayed`` and will not be propagated. -This criterion is used only by the prevalidator to decide the propagation of -operations. A baker can still include several operations originating from the same -manager in a single block, provided that it gets them in time (note that they can be -propagated by nodes using different versions or implementations). - Alternatively, a user can inject an operation with the same manager and the same counter, but with a higher fee to replace an already existing operation in the prevalidator. Only one of the two operations will be eventually diff --git a/docs/lima/precheck.rst b/docs/lima/precheck.rst index c65cd5f03c3f..5ea3b537e792 100644 --- a/docs/lima/precheck.rst +++ b/docs/lima/precheck.rst @@ -1,6 +1,13 @@ Prechecking of manager operations ================================= +.. FIXME tezos/tezos#3938: + + This section doesn't make much sense after the pipelining project + has plugged validate into the plugin for Lima. Parts of this + section be integrated into plugin.rst, and the relevant definitions + should point to the validation entry. + The prevalidator of Octez prevents the propagation of non-valid operations by trying to apply (or execute) them. However, executing -- GitLab From f4fe81f6bb3bf296a81976c687f51ddf8d17be98 Mon Sep 17 00:00:00 2001 From: vbot Date: Fri, 14 Oct 2022 15:04:41 +0200 Subject: [PATCH 10/12] doc: add description for consensus key related operations Co-authored-by: vbot Co-authored-by: Eugen Zalinescu --- docs/alpha/blocks_ops.rst | 36 +++++++++++++++++++++--------------- docs/lima/blocks_ops.rst | 35 +++++++++++++++++++++-------------- 2 files changed, 42 insertions(+), 29 deletions(-) diff --git a/docs/alpha/blocks_ops.rst b/docs/alpha/blocks_ops.rst index 0c4e451cd713..f4c25e2416b7 100644 --- a/docs/alpha/blocks_ops.rst +++ b/docs/alpha/blocks_ops.rst @@ -94,13 +94,11 @@ governance`. Anonymous Operations ~~~~~~~~~~~~~~~~~~~~ -.. TODO: tezos/tezos#3936 integrate consensus keys operations - This class groups all operations that do not require a signature from -a Tezos account. They allow to implement different functionalities of -the protocol, and their common characteristic is that it is desirable -that the account originating these operation remains anonymous in -order to avoid censorship. +a Tezos account (with an exception, detailed below). They implement +different functionalities of the protocol, and their common +characteristic is that they allow the account originating these +operations to remain anonymous in order to avoid censorship. Two operations in this class implement functionality pertaining to the protocol's :doc:`random seeds generation @@ -112,8 +110,8 @@ mechanism`: - The ``Vdf_revelation`` operation allows the submission of a solution to, and a proof of correctness of, the :ref:`VDF - challenge` corresponding to the VDF revelation period - of the randomness generation protocol. + challenge` corresponding to the VDF revelation period of + the randomness generation protocol. Further details on the latter operation's implementation and semantics are provided in the :ref:`random seed generation @@ -139,15 +137,20 @@ conflicting :ref:`consensus operations`: having "double-baked" a block -- i.e., of having signed two different blocks at the same level and at same round. The bulk of the evidence consists of the :ref:`block - headers` of each of the two offending - blocks. + headers` of each of the two offending blocks. + +See :ref:`here` for further detail on the semantics of +evidence-providing operations. -See :ref:`here` for further detail on the -semantics of evidence-providing operations. +The ``Activation`` operation allows users which participated in the +Tezos fundraiser to make their :ref:`accounts ` operational. -Finally, the ``Activation`` operation allows users which participated -in the Tezos fundraiser to make their :ref:`accounts ` -operational. +Finally, the ``Drain_delegate`` operation allows an active +consensus-key account, i.e., an account to which a baker delegated its +consensus-signing responsibility, to **empty** its delegate +account. This operation is used as a deterrent to ensure that a +delegate secures its consensus key as much as its manager (or main) +key. .. _manager_operations_alpha: @@ -176,6 +179,9 @@ manager operations are the only fee-paying and - The ``Delegation`` operation allows users to :ref:`delegate their stake ` to a :ref:`delegate` (a *baker*), or to register themselves as delegates. +- The ``Update_consensus_key`` operation allows users to delegate the + responsibility of signing blocks and consensus-related operations to + another account. - The ``Origination`` operation is used to :ref:`originate`, that is to deploy, smart contracts in the Tezos blockchain. diff --git a/docs/lima/blocks_ops.rst b/docs/lima/blocks_ops.rst index 1c65a17b260b..71f8c1874bc5 100644 --- a/docs/lima/blocks_ops.rst +++ b/docs/lima/blocks_ops.rst @@ -94,13 +94,11 @@ governance`. Anonymous Operations ~~~~~~~~~~~~~~~~~~~~ -.. TODO: tezos/tezos#3936 integrate consensus keys operations - This class groups all operations that do not require a signature from -a Tezos account. They allow to implement different functionalities of -the protocol, and their common characteristic is that it is desirable -that the account originating these operation remains anonymous in -order to avoid censorship. +a Tezos account (with an exception, detailed below). They implement +different functionalities of the protocol, and their common +characteristic is that they allow the account originating these +operations to remain anonymous in order to avoid censorship. Two operations in this class implement functionality pertaining to the protocol's :doc:`random seeds generation @@ -141,12 +139,18 @@ conflicting :ref:`consensus operations`: the evidence consists of the :ref:`block headers` of each of the two offending blocks. -See :ref:`here` for further detail on the -semantics of evidence-providing operations. +See :ref:`here` for further detail on the semantics of +evidence-providing operations. + +The ``Activation`` operation allows users which participated in the +Tezos fundraiser to make their :ref:`accounts ` operational. -Finally, the ``Activation`` operation allows users which participated -in the Tezos fundraiser to make their :ref:`accounts ` -operational. +Finally, the ``Drain_delegate`` operation allows an active +consensus-key account, i.e., an account to which a baker delegated its +consensus-signing responsibility, to **empty** its delegate +account. This operation is used as a deterrent to ensure that a +delegate secures its consensus key as much as its manager (or main) +key. .. _manager_operations_lima: @@ -172,9 +176,12 @@ manager operations are the only fee-paying and of future operations signed by this manager. - The ``Transaction`` operation allows users to either transfer tez between accounts and/or to invoke a smart contract. -- The ``Delegation`` operation allows users to :doc:`delegate their - stake` to a :ref:`delegate` (a *baker*), - or to register themselves as delegates. +- The ``Delegation`` operation allows users to :ref:`delegate their + stake ` to a :ref:`delegate` (a + *baker*), or to register themselves as delegates. +- The ``Update_consensus_key`` operation allows users to delegate the + responsibility of signing blocks and consensus-related operations to + another account. - The ``Origination`` operation is used to :ref:`originate`, that is to deploy, smart contracts in the Tezos blockchain. -- GitLab From d2fb6ab2a543d5852dcff2499dff0ffae0501918 Mon Sep 17 00:00:00 2001 From: German Date: Thu, 6 Oct 2022 15:56:12 +0200 Subject: [PATCH 11/12] WIP doc other validation passes [sip-ci] --- docs/alpha/validation.rst | 20 +++++++++++++++++++- docs/kathmandu/validation.rst | 19 ++++++++++++++++++- docs/lima/validation.rst | 27 +++++++++++++++++++++++++++ 3 files changed, 64 insertions(+), 2 deletions(-) diff --git a/docs/alpha/validation.rst b/docs/alpha/validation.rst index 0d11724be1f2..0814f9b2b992 100644 --- a/docs/alpha/validation.rst +++ b/docs/alpha/validation.rst @@ -278,7 +278,25 @@ application process for each of the different validation passes. .. FIXME tezos/tezos#3921: - Expand validity and application for other validation classes. + Adapt to pipelined validation up to Lima and v7 environment. + +.. _voting_operations_validity_alpha: + +Validity and Application of Voting Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment. + +.. _anonymous_operations_validity_alpha: + +Validity and Application of Anonymous Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment. .. _manager_operations_validity_alpha: diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst index eb768a834605..d4282b32b739 100644 --- a/docs/kathmandu/validation.rst +++ b/docs/kathmandu/validation.rst @@ -276,12 +276,29 @@ and yet still be included in a block. The explicit conditions for a manager operation to be considered as valid, are given in the :ref:`manager_operations_validity_alpha` -sub-section below. +sub-section below. For the remaining acceptable :ref:`validation +passes`, we provide a quick overview of +their global validity in the sequel. .. FIXME tezos/tezos#3921: Expand validity and application for other validation classes. +.. _consensus_operations_validity_kathmandu: + +Validity and Application of Consensus Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _voting_operations_validity_kathmandu: + +Validity and Application of Voting Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. _anonymous_operations_validity_kathmandu: + +Validity and Application of Anonymous Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + .. _manager_operations_validity_kathmandu: Validity of Manager Operations diff --git a/docs/lima/validation.rst b/docs/lima/validation.rst index 4505f9ab1592..6d3c838a6619 100644 --- a/docs/lima/validation.rst +++ b/docs/lima/validation.rst @@ -272,6 +272,33 @@ from the account submitting the smart contract call. Adapt to pipelined validation up to Lima and v7 environment: add validity and application for other validation classes. + .. _consensus_operations_validity_lima: + +Validity and Application of Consensus Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment. + +.. _voting_operations_validity_lima: + +Validity and Application of Voting Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment. + +.. _anonymous_operations_validity_lima: + +Validity and Application of Anonymous Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. FIXME tezos/tezos#3921: + + Adapt to pipelined validation up to Lima and v7 environment. + Validity of Manager Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- GitLab From 9f303407935fec2ee7d5e912cf187ed97dcf9618 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20Andr=C3=A9s=20Delbianco?= Date: Fri, 14 Oct 2022 17:15:07 +0200 Subject: [PATCH 12/12] WIP working on consensus ops validation --- docs/kathmandu/blocks_ops.rst | 8 ++++ docs/kathmandu/validation.rst | 85 +++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) diff --git a/docs/kathmandu/blocks_ops.rst b/docs/kathmandu/blocks_ops.rst index 700e86d0966f..0fcfef7a8511 100644 --- a/docs/kathmandu/blocks_ops.rst +++ b/docs/kathmandu/blocks_ops.rst @@ -68,6 +68,14 @@ phases required to agree on the next block. for which a preendorsement quorum certificate (PQC) has been observed. +.. _consensus_content_kathmandu: + +These operations carry a similar "consensus content" -- consisting of +the delegate's first :ref:`slot`, the target +:ref:`level` and :ref:`round`, and the hash of the +(pre)endorsed block' payload. + + .. _voting_operations_kathmandu: Voting Operations diff --git a/docs/kathmandu/validation.rst b/docs/kathmandu/validation.rst index d4282b32b739..d77fb96c5f78 100644 --- a/docs/kathmandu/validation.rst +++ b/docs/kathmandu/validation.rst @@ -289,6 +289,91 @@ their global validity in the sequel. Validity and Application of Consensus Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The validity and applicability of consensus operation is determined +monolithically on this protocol by ``apply_operation`` method. + +:ref:`Preendorsements and +endorsements` share a common +:ref:`consensus content` -- notably the +delegate's first :ref:`slots`, and the targeted +:ref:`level` and :ref:`round` values. Naturally they +share several preconditions, and the validation process exploits these +common patterns. + +Still, the whole process is nuanced by the validation mode in use, and +the + +The first step in the validation process consists on + +Exploits these common pattern by the differences in treatment of this +cont + +The first step in the validation process of consensus operations +depends on + +First, it attempts to validate the operation's "consensus content" by +checking that the delegate signing the operation has rights for the +proposed level and round values. + +slot, + +level and round values. + +A key difference in the process is that preendorsements arises from +the fact that preendorsements are not intended to be included in +blocks, but are rather an artefact of the implementation of the +Tenderbake consensus algorithm. As a result, pre-endorsements can only +be applied in the Partial Construction (or Mempool) mode. + +1. Validation:validate consensus content: + +(* Validate the 'operation.shell.branch' field of the operation. It + MUST point to the grandfather: the block hash used in the + payload_hash. Otherwise we could produce a preendorsement pointing + to the direct proposal. This preendorsement wouldn't be able to + propagate for a subsequent proposal using it as a locked_round + evidence. *) + +1.1 compute expected consensus content (* The [Alpha_context] is +modified only in [Full_construction] mode when we check a +preendorsement if the [preendorsement_quorum_round] was not set. *) + + Endorsements: need to have an endorsement branch. + + Preendorsements: pre-endorsements cannot appear in full application + mode (they are not included in blocks which are included in the + blockchain ledger). Partial Construction: mempool. they need to + check that there is an expected branch (i.e. they don't target a + future level), + + In Full construction mode: we check if there was a pqc round set, + otherwise we set it. + +1.2 check consensus content +: check level, + - Application/FullConsturction: match provided = expected +: check round, + - +op branch, payload hash. + +1.3 check funds, signature etc. + +Endorsements: depending on application mode (Partial construction) +needs to differentiate parents from grand parents: +- if is_parent_endorsement then validate_grand_parent_endorsement + +Refusals: + +Endorsements: doesn't target locked round; targets different level, +round, branch (grandparent) or payload; incorrect delegate for slot, +or doesn't have locked founds, signature doesn't match expected one, + + assert (not mem(op.slot, state.endorsements_seen)) (* Refused *) + +PreEndorsements: doesn't target locked round; targets different level, +round, branch (parent) or payload; incorrect delegate for slot, +or doesn't have locked founds, signature doesn't match expected one, + .. _voting_operations_validity_kathmandu: Validity and Application of Voting Operations -- GitLab