From a4e2d5f28ec914b456cefab1b6822e69db9e5acd Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Wed, 7 Dec 2022 18:08:09 +0100 Subject: [PATCH 1/7] doc: proofread light mode --- docs/user/light.rst | 67 ++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 34 deletions(-) diff --git a/docs/user/light.rst b/docs/user/light.rst index 1daef4d64255..4ede48a42a1b 100644 --- a/docs/user/light.rst +++ b/docs/user/light.rst @@ -1,29 +1,33 @@ Light mode ---------- -The ``octez-client`` described in -:ref:`its own tutorial ` forwards all RPCs to a node. -This page describes the *light* mode, a mode where the client -performs protocol RPCs locally; like the :doc:`proxy mode`. +The proxy mode, described in :doc:`a dedicated tutorial <./proxy>`, +is an execution mode where the :ref:`Octez client ` +avoids some RPC calls to the node, especially computation-intensive RPCs. +It does so by requesting the data it needs from the node using RPCs (that are not computation-intensive), and uses +this data locally to perform computations by itself, whenever possible. + +This page describes the *light* mode, a variant of the proxy mode, +where the client also +performs RPCs locally whenever possible. However, contrary to the proxy mode, the light mode provides -a high level of security. For that it obtains data from multiple -(hopefully unrelated) endpoints and makes sure all endpoints send -the same data, by using *Merkle proofs*. Such proofs make very hard +a high level of security. For that, it obtains its data from multiple +(hopefully unrelated) node endpoints, and makes sure that all endpoints send +the same data, by using *Merkle proofs*. Such proofs make it very hard for unrelated endpoints to craft fake data. This mode is akin to a light client or *thin client* in Bitcoin terms. While the existing implementation of the light mode is entirely functional, -it still has room for improvement, in particular communications over -the network can be reduced. Users are encouraged to manifest themselves, -so that enhancements to the light mode are made high priority: please -submit issues `here on GitLab `_. +it still has room for improvement. For instance, communications over +the network can be reduced. Users are encouraged to share their experience, +by submitting issues `here on GitLab `_. Executing commands in light mode ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The CLI interface of the client in light mode (the *light client* in short) -is the same as the default client. To turn light mode ON, you must +The CLI interface of the client in light mode (the *light client* for short) +is the same as the default client. To turn the light mode on, you must pass two arguments to ``octez-client``: * ``--mode light``, and @@ -45,7 +49,7 @@ The ``sources.json`` file contains: endpoint is tolerated). -Here is an example valid ``--sources`` file: +Here is an example of a valid ``--sources`` file: :: @@ -56,17 +60,12 @@ Here is an example valid ``--sources`` file: ] } -Because computations done locally are protocol dependent, the light mode -does not support all protocols. However, at any given time, it should -support the protocol being developed (``alpha``) and the three most -recent protocols. Execute ``octez-client list light protocols`` -to see the supported protocols. - -If ``--protocol`` is omitted when calling the light client, it -tries to match the node's protocol. On the one hand, this is handy when -testing. On the other hand, in a production environment, it is recommended -to specify ``--protocol`` if the protocol is known, to avoid an extra -RPC at **every** call ``octez-client --mode light ...``. +Because computations done locally are protocol dependent, the light mode has to be configured for a specific protocol. +However, the light mode does not support all protocols. +Execute ``octez-client list light protocols`` to see the supported protocols. +It is expected that, at any given time, it should support ``Alpha``, +the current protocol of Mainnet, and the current protocol proposal on Mainnet at +the time of release, if any. Examples with the sandbox ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -74,8 +73,9 @@ Examples with the sandbox In this section, we show examples of usage of the light mode when using the :doc:`sandboxed node`. For convenience we repeat instructions for the sandboxed mode here, but refer the reader to the -sandboxed mode page for further details. In a terminal, -start a sandboxed node: +sandboxed mode page for further details. + +In a terminal, start a sandboxed node: :: @@ -114,7 +114,7 @@ To avoid warnings being printed in upcoming commands (optional): $ export TEZOS_CLIENT_UNSAFE_DISABLE_DISCLAIMER=y The last step before being able to use the light client is to prepare -the JSON file to pass to ``--sources``. In our scenario, this file +the JSON file passed to ``--sources``. In our scenario, this file specifies the two endpoints to use: :: @@ -126,11 +126,10 @@ You're now ready to use the light client. For example, bake a block: :: $ octez-client --endpoint http://localhost:18731 --mode light --sources sources.json bake for bootstrap1 - protocol of light mode unspecified, using the node's protocol: ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK Apr 8 16:42:24.202 - alpha.baking.forge: found 0 valid operations (0 refused) for timestamp 2021-04-08T14:42:24.000-00:00 (fitness 01::0000000000000004) Injected block BMAHozsNCos2 -Well that doesn't seem very different from what the default client would return. +Well, that doesn't seem very different from what the default client would return. Indeed, it's the same; that was the point! To see what the light client is doing differently, you may use the environment variable ``TEZOS_LOG``. Set it as follows: @@ -147,7 +146,7 @@ keystrokes and the ``protocol of light mode unspecified`` warning: :: - $ alias light-client="octez-client --endpoint http://localhost:18731 --mode light --protocol ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK --sources sources.json" + $ alias light-client="octez-client --endpoint http://localhost:18731 --mode light --sources sources.json" And then bake a new block: @@ -169,7 +168,7 @@ And then bake a new block: ... Injected block BMdbKufTymQJ -Here's the meaning of these lines: +Here is the meaning of these lines: * Line ``light mode's core created`` indicates that the light mode was initialized. It should be printed once per block being inspected. @@ -180,7 +179,7 @@ Here's the meaning of these lines: to fetch Merkle proofs for this key from other endpoints. * Lines ``API call: get ...`` indicate that ``octez-client`` is requesting data from the light mode's cache. In this snippet, after the light mode - gathered data for key ``v1``; the client is requesting data for the children + gathered data for key ``v1``, the client is requesting data for the children keys ``v1;constants`` and ``v1;first_level`` (the ``;`` indicates nesting). This example shows how the light mode sometimes batches requests, to avoid querying many keys in a row. Here it did a single request for ``v1`` instead @@ -196,5 +195,5 @@ How to deploy to relieve nodes from some RPCs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Again, we refer to the corresponding section in the page of -the proxy mode :doc:`proxy mode`. The exact same recommendations +the :doc:`proxy mode`. The exact same recommendations apply for the light mode. -- GitLab From d9388a3c7126fd4302cfd87173dab5a1a1ffb9f9 Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Mon, 12 Dec 2022 10:28:25 +0100 Subject: [PATCH 2/7] doc: bake for --minimal-timestamp instead of for bootstrap1 --- docs/user/light.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/user/light.rst b/docs/user/light.rst index 4ede48a42a1b..98f2bcf6fc15 100644 --- a/docs/user/light.rst +++ b/docs/user/light.rst @@ -121,7 +121,7 @@ specifies the two endpoints to use: $ echo '{ "uris": [ "http://localhost:18731", "http://localhost:18732" ] }' > sources.json -You're now ready to use the light client. For example, bake a block: +You're now ready to use the light client. For example, bake a block (note that this command may take up to a few minutes to complete): :: @@ -129,6 +129,10 @@ You're now ready to use the light client. For example, bake a block: Apr 8 16:42:24.202 - alpha.baking.forge: found 0 valid operations (0 refused) for timestamp 2021-04-08T14:42:24.000-00:00 (fitness 01::0000000000000004) Injected block BMAHozsNCos2 +As you may have noticed, the block is baked when the ``boostrap1`` baker has rights to bake, which explains the possible delay of up to a few minutes. +If you are in a hurry, you may want to bake for any baker, by replacing ``bake for bootstrap1`` with ``bake for --minimal-timestamp``. +We will use this option for the next times. + Well, that doesn't seem very different from what the default client would return. Indeed, it's the same; that was the point! To see what the light client is doing differently, you may use the environment variable ``TEZOS_LOG``. @@ -152,7 +156,7 @@ And then bake a new block: :: - $ light-client bake for bootstrap1 + $ light-client bake for --minimal-timestamp Apr 8 16:49:28.172 - light_mode: light mode's core created for chain main and block head Apr 8 16:49:28.173 - light_mode: API call: do_rpc v1 Apr 8 16:49:28.175 - light_mode: integrated data for key v1 from one endpoint, about to validate from 1 other -- GitLab From 267baecb81098ed5e338dc205d87aaaa5466d0e6 Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Mon, 12 Dec 2022 18:57:23 +0100 Subject: [PATCH 3/7] doc: proofread and update the doc of the proxy server --- docs/user/proxy-server.rst | 128 ++++++++++++++++++------------------- 1 file changed, 63 insertions(+), 65 deletions(-) diff --git a/docs/user/proxy-server.rst b/docs/user/proxy-server.rst index f9660ccb7753..657b1221b577 100644 --- a/docs/user/proxy-server.rst +++ b/docs/user/proxy-server.rst @@ -10,46 +10,43 @@ things: * Regular HTTP proxies, as proxy servers are meant to be deployed in front of a node, to lower the node's load. -While the proxy server can only serve a subset of the RPCs a full node can serve -(detailed :ref:`below `), one can transparently -use a proxy server as a replacement for a full node, in the manner described -:ref:`below `: it will use HTTP redirects to redirect clients -to the node when it cannot handle a certain request. +Even though the proxy server only serves a subset of the RPCs a full node can serve (detailed :ref:`below `), it can transparently act as a replacement for a full node, by redirecting to the node the HTTP requests that it cannot handle itself, as described :ref:`below `. Launching a proxy server ~~~~~~~~~~~~~~~~~~~~~~~~ -The minimal arguments to launch a proxy server are ``--endpoint`` +The proxy server is implemented by the ``octez-proxy-server`` executable. +The minimal arguments to the proxy server are ``--endpoint`` and ``--rpc-addr``: -* ``--endpoint`` specifies the URL of the RPC server of the node - to do requests to obtain data (RPCs of the form - ``/chains//blocks//context/raw/bytes``). +* ``--endpoint`` specifies the URL of the RPC server of the underlying node, that is uses + for obtaining data (via RPCs of the form + ``/chains//blocks//context/raw/bytes``), + and for redirecting the requests it cannot handle. * ``--rpc-addr`` specifies the URL that the proxy server should serve. -Another important argument is ``--data-dir``: +An important option is ``--data-dir``: * ``--data-dir`` specifies the directory of the node from - which to obtain data. This argument can only be used when the proxy server - has actual access to the specified data directory. - In this case, this option will reduce the number of RPC calls to the - node, thereby reducing its IO consumption. + which to obtain data directly (without RPCs). This option can only be used when the proxy server + has access to the node's data directory. + In this case, the proxy server will reduce the number of RPC calls to the + node, thereby reducing its I/O consumption. -The full list of arguments is detailed :ref:`below `. +The full command-line API is detailed :ref:`below `. .. _sandbox_example: -Example with the sandbox -~~~~~~~~~~~~~~~~~~~~~~~~ +Examples with the sandbox +~~~~~~~~~~~~~~~~~~~~~~~~~ -In this section, we show examples of usage of a proxy server when using -the :doc:`sandboxed node`. For convenience we repeat +In this section, we show examples of using a proxy server in +the :doc:`sandboxed node`. For convenience, we repeat instructions for the sandboxed mode here, but refer the reader to the sandboxed mode page for further details. First, edit ``src/proto_alpha/parameters/sandbox-parameters.json`` to set the first value of ``time_between_blocks`` to ``15`` seconds (it's -the time between two blocks in the chain, we set it to ``15`` seconds, -so that the scenario below is easier to understand): +the time between two blocks in the chains): :: @@ -58,7 +55,7 @@ so that the scenario below is easier to understand): ... } -In a terminal, start a sandboxed node: +In a first terminal, start a sandboxed node: :: @@ -67,17 +64,14 @@ In a terminal, start a sandboxed node: Created /tmp/octez-node.Uzq5aGAN/config.json for network: sandbox. ... -Leave that terminal running. In a second terminal, prepare the appropriate -environment for using a proxy server: - -:: - - $ eval `./src/bin_client/octez-init-sandboxed-client.sh 1` +Note in the trace above that the sandbox node wrote some configuration file in a working directory, that you may need to know later on. -Then upgrade the node to protocol alpha: +Leave this first terminal running. In a second terminal, prepare the appropriate +environment for running a proxy server: :: + $ eval `./src/bin_client/octez-init-sandboxed-client.sh 1` $ octez-activate-alpha $ octez-client bake for bootstrap1 @@ -87,19 +81,17 @@ To avoid warnings being printed in upcoming commands (optional): $ export TEZOS_CLIENT_UNSAFE_DISABLE_DISCLAIMER=y -You're now ready to use a proxy server. Open a third terminal, and -launch a proxy server as shown below. We specify debug variables -in ``TEZOS_LOG`` to have debug output showing what the proxy server +You're now ready to run a proxy server, as shown below. +We specify debug variables in ``TEZOS_LOG`` to see what the proxy server is doing (see the :doc:`proxy mode` page for more details). :: $ export TEZOS_LOG="proxy_rpc_ctxt->debug; proxy_rpc->debug; proxy_server_run->debug; proxy_getter->debug; proxy_services->debug" $ ./octez-proxy-server --endpoint http://127.0.0.1:18731 --rpc-addr http://127.0.0.1:18732 - protocol of proxy unspecified, using the node's protocol: ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK Apr 21 11:09:22.092 - proxy_server_run: starting proxy RPC server on 127.0.0.1:18732 -Now, start a fourth terminal, and make a client request data from the proxy server: +Now, start a third terminal, and ask the client to request data from the proxy server: :: @@ -131,7 +123,7 @@ In the proxy server's terminal, you should see this output (tree sizes may vary) Apr 21 11:10:07.479 - proxy_rpc: received tree of size 115 Lines of the form ``proxy_rpc: /chains/
/blocks//context/raw/bytes/...`` -show requests that the proxy server does to the node to obtain data. +show requests that the proxy server sends to the node to obtain data. ``15`` seconds after the previous command, the proxy server should clear the data it obtained, because ``time_between_blocks`` was set to ``15`` @@ -141,7 +133,7 @@ seconds at the beginning of this scenario: Apr 21 11:10:22.478 - proxy_services: clearing data for chain main and block head -Now, in the fourth terminal, retrieve the contracts again, but twice in a row: +Now, in the third terminal, retrieve the contracts again, but twice in a row: :: @@ -194,15 +186,15 @@ and restart it as follows: :: - $ ./octez-proxy-server --endpoint http://127.0.0.1:18731 --rpc-addr http://127.0.0.1:18732 --data-dir /tmp/octez-node.Uzq5aGAN + $ ./octez-proxy-server --endpoint http://127.0.0.1:18731 --rpc-addr http://127.0.0.1:18732 --data-dir protocol of proxy unspecified, using the node's protocol: ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK Apr 21 11:09:22.092 - proxy_server_run: starting proxy RPC server on 127.0.0.1:18732 -The value of the ``--data-dir`` argument was obtained by looking at the +You can obtain the value for the ``--data-dir`` argument by looking at the output of the terminal where ``octez-node`` was launched (see :ref:`above `). -Now, in the fourth terminal (the client's terminal), redo the request +Now, in the third terminal (the client's terminal), redo the request to retrieve contracts: :: @@ -228,7 +220,7 @@ Additional arguments We describe the entire list of arguments of the proxy server. This documentation is also available with ``./octez-proxy-server --help``. -Here is the list of possible arguments: +Here is the list of possible options: * ``-c`` and ``--config`` specify the JSON file to use an input for the configuration. This JSON file is an object like this: @@ -266,51 +258,56 @@ Here is the list of possible arguments: ``time_between_blocks`` is hence regularly requested from the node, this incurs a higher load of the node. -All arguments are optional as they can either be specified in the configuration +All these options can either be specified in the configuration file or on the command line. However, the union of the configuration file and the command line should specify the endpoint to use and the RPC address to serve. .. _what_the_proxy_server_serves: -What the proxy server serves -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Supported RPCs +~~~~~~~~~~~~~~ -The proxy server itself serves protocol-specific RPCs, which are listed -`here `_ for protocol Alpha, +The proxy server itself serves protocol-specific RPCs (listed +`here `_ for protocol Alpha), but not all of them: since the proxy server is a readonly frontend for the underlying node, it only serves the readonly requests (``GET`` requests, as well as a subset of the ``POST`` requests). -Because computations done by the proxy server are protocol dependent, the proxy server -does not support all protocols. However, it is expected that, at any +Because computations done by the proxy server are protocol dependent, the proxy mode must choose a specific protocol: the same as the underlying node. +However, the proxy mode does not support all protocols. +Execute ``octez-client list proxy protocols`` to see the supported protocols. +It is expected that, at any given time, the proxy server supports ``Alpha``, the current protocol of Mainnet and the current protocol proposal on Mainnet at the time of release. -In doubt, execute ``octez-client list proxy protocols`` to see the supported protocols. .. _unsupported_rpcs: Unsupported RPCs ~~~~~~~~~~~~~~~~ -Requests that are not readonly can only be handled by a full node. However, it -is possible to *send* any RPC to the proxy server: if the RPC is not supported +Requests that are not readonly can only be handled by a full node. However, the proxy server accepts any RPC: if the RPC is not supported by the proxy server, it will redirect clients to the appropriate endpoint on the underlying node using an HTTP redirect (``301 Moved Permanently``), and the node will then handle the request. This can be easily demonstrated with a simple test: start a proxy server, and make a request to it with ``curl -vL /``. -(For example, ``/chains/main/blocks/head/header`` is one such RPC.) The output +For example:: + + curl -vL http://127.0.0.1:18732/chains/main/blocks/head/header + +The output from ``curl`` will show that the proxy server asks curl to follow a redirect to the node's endpoint, which it will do because of the ``-L`` flag, and then it is finally responded to by the node. Any RPC that can be handled by the proxy server itself will of course not show this behaviour. Clearly, making such requests to the proxy server does not decrease the load of -the node. (To be precise, it in fact also adds a slight delay to the HTTP -request if the redirect is not cached by the client.) However, it does allow the -use of a single endpoint for all RPC requests, which may be more convenient for -certain use-cases. +the node. However, it does allow the +use of a single endpoint for all RPC requests, which may be convenient for +some use cases. +In turn, it adds a slight delay to the HTTP +request, unless the redirect is cached by the client. Deployment ~~~~~~~~~~ @@ -322,20 +319,21 @@ Because the proxy server is protocol-dependent, if the node it talks to changes protocol; the proxy server will start failing for RPCs concerning blocks of the new protocol. We hereby recommend to automatically restart proxy servers that have a high ratio of failures. -Restarting a proxy server is always fine, they can be thrown away at any +Restarting a proxy server is always fine; they can be thrown away at any moment. Heuristics ~~~~~~~~~~ -The proxy server has heuristics. For example there is an heuristic -to make big map queries faster, when many queries to siblings keys of a given -big map are done in burst. The list of heuristics is -visible for protocol Alpha in -`proxy.ml `_. -The heuristic is implemented in function ``split_key``. For example, -any request of the form ``rolls/owner/snapshot/i/j/tail`` is transformed -into a request of the form ``rolls/owner/snapshot/i/j`` to obtain data for all +The proxy server uses several heuristics to optimize its own work and/or decrease the node's load. +For example, there is a heuristic +to make big map queries faster, which is useful when many queries to siblings keys of a given +big map are done in burst. + +The heuristics for protocol Alpha are implemented in file +:src:`proxy.ml `, in function ``split_key`` and associates. For example, +any request of the form ``big_maps/index/i/contents/tail`` is transformed +into a request of the form ``big_maps/index/i/contents`` to obtain data for all possible values of ``tail`` at once. -For the moment the heuristics cannot be specified on the command line, but +For the moment, the heuristics cannot be specified on the command line, but this can be implemented in the future. -- GitLab From eb4eb9366f14c6b721ea9efce5278c0320065dcc Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Thu, 19 Jan 2023 10:17:07 +0100 Subject: [PATCH 4/7] doc: explain when the proxy may not swittch to a new protocol --- docs/user/proxy-server.rst | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/user/proxy-server.rst b/docs/user/proxy-server.rst index 657b1221b577..8585b2e4f1c7 100644 --- a/docs/user/proxy-server.rst +++ b/docs/user/proxy-server.rst @@ -267,7 +267,7 @@ and the command line should specify the endpoint to use and the RPC address to s Supported RPCs ~~~~~~~~~~~~~~ -The proxy server itself serves protocol-specific RPCs (listed +The proxy server itself only serves protocol-specific RPCs (listed `here `_ for protocol Alpha), but not all of them: since the proxy server is a readonly frontend for the underlying node, it only serves the readonly requests (``GET`` requests, as @@ -315,9 +315,15 @@ Deployment As a proxy server is a readonly frontend to a node, you can spawn multiple proxy servers in front of a single node. -Because the proxy server is protocol-dependent, if the node it talks to -changes protocol; the proxy server will start failing for RPCs -concerning blocks of the new protocol. We hereby recommend to automatically +As described above, the proxy server configures himself to the same protocol as the underlying node. As a consequence, +when the underlying node changes protocol, the proxy server will also switch to the new protocol, **unless** the proxy server executable does not contain the new protocol. +This may happen, for instance, if the executable was compiled before this protocol was even injected. +As there is no dynamic linking of new protocols in the proxy server, it will +start failing for RPCs +concerning blocks of the new protocol. +The solution in this case is to restart the proxy server using a more recent executable. + +More generally, we recommend to automatically restart proxy servers that have a high ratio of failures. Restarting a proxy server is always fine; they can be thrown away at any moment. -- GitLab From 5722ca3f12f142a7aa046939607dfcc64428f005 Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Thu, 19 Jan 2023 11:08:18 +0100 Subject: [PATCH 5/7] doc: update logs as cache is no more cleared after 15s --- docs/user/proxy-server.rst | 46 +++++--------------------------------- 1 file changed, 6 insertions(+), 40 deletions(-) diff --git a/docs/user/proxy-server.rst b/docs/user/proxy-server.rst index 8585b2e4f1c7..df53dcea811e 100644 --- a/docs/user/proxy-server.rst +++ b/docs/user/proxy-server.rst @@ -28,7 +28,7 @@ and ``--rpc-addr``: An important option is ``--data-dir``: * ``--data-dir`` specifies the directory of the node from - which to obtain data directly (without RPCs). This option can only be used when the proxy server + which to obtain data directly (without RPCs). Of course, this option can only be used when the proxy server has access to the node's data directory. In this case, the proxy server will reduce the number of RPC calls to the node, thereby reducing its I/O consumption. @@ -43,17 +43,7 @@ Examples with the sandbox In this section, we show examples of using a proxy server in the :doc:`sandboxed node`. For convenience, we repeat instructions for the sandboxed mode here, but refer the reader to the -sandboxed mode page for further details. First, edit -``src/proto_alpha/parameters/sandbox-parameters.json`` -to set the first value of ``time_between_blocks`` to ``15`` seconds (it's -the time between two blocks in the chains): - -:: - - { ..., - "time_between_blocks": [ "15", "0" ], - ... - } +sandboxed mode page for further details. In a first terminal, start a sandboxed node: @@ -125,15 +115,7 @@ In the proxy server's terminal, you should see this output (tree sizes may vary) Lines of the form ``proxy_rpc: /chains/
/blocks//context/raw/bytes/...`` show requests that the proxy server sends to the node to obtain data. -``15`` seconds after the previous command, the proxy server should clear -the data it obtained, because ``time_between_blocks`` was set to ``15`` -seconds at the beginning of this scenario: - -:: - - Apr 21 11:10:22.478 - proxy_services: clearing data for chain main and block head - -Now, in the third terminal, retrieve the contracts again, but twice in a row: +Now, in the third terminal, retrieve the contracts again: :: @@ -150,31 +132,16 @@ In the meantime, in the proxy server's terminal, you should see: :: - Apr 21 11:14:04.262 - proxy_rpc: chains/
/blocks//header - Apr 21 11:14:04.263 - proxy_rpc: proxy cache created for chain main and block head - Apr 21 11:14:04.266 - proxy_getter: Cache miss: (v1/constants) - Apr 21 11:14:04.266 - proxy_getter: split_key heuristic triggers, getting v1 instead of v1/constants - Apr 21 11:14:04.266 - proxy_rpc: /chains/
/blocks//context/raw/bytes/v1 - Apr 21 11:14:04.266 - proxy_rpc: received tree of size 2 - Apr 21 11:14:04.267 - proxy_getter: Cache hit: (v1/cycle_eras) - Apr 21 11:14:04.267 - proxy_getter: Cache miss: (pending_migration_balance_updates) - Apr 21 11:14:04.267 - proxy_rpc: /chains/
/blocks//context/raw/bytes/pending_migration_balance_updates - Apr 21 11:14:04.267 - proxy_getter: Cache miss: (pending_migration_operation_results) - Apr 21 11:14:04.267 - proxy_rpc: /chains/
/blocks//context/raw/bytes/pending_migration_operation_results - Apr 21 11:14:04.267 - proxy_getter: Cache miss: (contracts/index) - Apr 21 11:14:04.268 - proxy_rpc: /chains/
/blocks//context/raw/bytes/contracts/index - Apr 21 11:14:04.269 - proxy_rpc: received tree of size 115 Apr 21 11:14:06.511 - proxy_getter: Cache hit: (v1/constants) Apr 21 11:14:06.512 - proxy_getter: Cache hit: (v1/cycle_eras) Apr 21 11:14:06.512 - proxy_getter: Cache hit: (pending_migration_balance_updates) Apr 21 11:14:06.512 - proxy_getter: Cache hit: (pending_migration_operation_results) Apr 21 11:14:06.512 - proxy_getter: Cache hit: (contracts/index) -The last four lines show that the proxy server is answering the request +This show that the proxy server is answering the request without delegating anything to the node: there is no ``proxy_rpc`` line. The proxy server is reusing the data it obtained for ```` from -the first request, because less than ``time_between_block`` (``15`` seconds) -have passed. +the first request. Reducing RPC calls: ``--data-dir`` """""""""""""""""""""""""""""""""" @@ -208,7 +175,6 @@ Now the output in the proxy server terminal should be: Apr 21 11:22:44.359 - proxy_rpc: chains/
/blocks//header Apr 21 11:22:44.360 - proxy_rpc: proxy cache created for chain main and block head - Apr 21 11:22:59.362 - proxy_services: clearing data for chain main and block head There are far fewer ``proxy_rpc`` lines! That is because the proxy server obtained its data by reading the node's data-dir, instead of performing RPC calls. @@ -273,7 +239,7 @@ but not all of them: since the proxy server is a readonly frontend for the underlying node, it only serves the readonly requests (``GET`` requests, as well as a subset of the ``POST`` requests). -Because computations done by the proxy server are protocol dependent, the proxy mode must choose a specific protocol: the same as the underlying node. +Because computations done by the proxy server are protocol-dependent, the proxy mode must choose a specific protocol: the same as the underlying node. However, the proxy mode does not support all protocols. Execute ``octez-client list proxy protocols`` to see the supported protocols. It is expected that, at any -- GitLab From f60a662f2a006b46c93e778524decdac32c58141 Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Fri, 27 Jan 2023 17:20:42 +0100 Subject: [PATCH 6/7] doc: checking that endpoint belongs to sources is not systematic --- docs/user/light.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/user/light.rst b/docs/user/light.rst index 98f2bcf6fc15..b48bb2f5478d 100644 --- a/docs/user/light.rst +++ b/docs/user/light.rst @@ -41,7 +41,6 @@ The ``sources.json`` file contains: To avoid being surprised by the automatic addition of the default endpoint, it is recommended to supply option ``--endpoint`` whenever ``--sources`` is supplied, valued as one member of the ``uris`` field in ``--sources`` (e.g., the first member). - An automatic check makes sure this recommendation is ensured. * An optional ``min_agreement`` field, which must be a float from ``0.0`` (excluded) to ``1.0`` (included). This field specifies the ratio of endpoints that must agree for data to be accepted. The default value is ``1.0``, which means that -- GitLab From 71d6bd9b83570f2ba9c0fa093de082ebd497eb68 Mon Sep 17 00:00:00 2001 From: Nic Volanschi Date: Mon, 30 Jan 2023 12:25:29 +0100 Subject: [PATCH 7/7] doc: move proxy-server page within setup-node --- docs/index.rst | 1 - docs/user/setup-node.rst | 7 +++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/index.rst b/docs/index.rst index 0f1bc852d25a..6268fabbeb17 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -175,7 +175,6 @@ Core developers are also provided reference materials for internal APIs of Octez user/versioning user/setup-client user/setup-node - user/proxy-server user/multisig user/fa12 user/logging diff --git a/docs/user/setup-node.rst b/docs/user/setup-node.rst index 5e821e88db7f..445ba71105f1 100644 --- a/docs/user/setup-node.rst +++ b/docs/user/setup-node.rst @@ -7,6 +7,8 @@ A node in the Tezos network provides different configuration possibilities: - specify the Tezos network to connect to, which can be the Mainnet or different test networks. - configure the amount of history kept by the node according to different tradeoffs, using history modes - rapidly catch up with a given (main or test) network by loading network snapshots +- set up one or several proxy servers to decrease the load of the node by serving some RPC requests themselves +- set up infrastructure for continuosly monitoring the node to help spotting operational or efficiency issues These possibilities are described in the following pages. @@ -30,6 +32,11 @@ These possibilities are described in the following pages. snapshots +.. toctree:: + :maxdepth: 2 + + proxy-server + .. toctree:: :maxdepth: 2 -- GitLab