Tezlink: end-to-end demonstration of Tezos Layer 1

Context

To build Tezlink, we want to target as early as possible an end-to-end demo in which a Tezos wallet (such as Umami), a Tezos indexer (such as TzKT), and a Tezos application (such as Kanvas) are used. To know which Michelson instructions, RPC endpoints, and blockchain operations are needed for this demo, we will first run it on the layer 1 and use coverage testing tools to list what needs to be supported first.

Work breakdown

@luciano.freitas

Run a sandbox Tezos network (using octez-node and octez-baker),
Deploy an instance of TzKT for the sandbox network, list the called RPCs. Note that Umami depends on TzKT, thus it needs to be deployed first.
Configure Umami to use the sandbox network,
Import in Umami the secret key of a bootstrap account,
Use Umami to get the balance of the bootstrap account, list the RPCs called by Umami
Use Umami to transfer one tez from the bootstrap account to a fresh implicit account, list the called RPCs,
Deploy an instance of Teia, mint and send a token, list the called RPCs, operations, and Michelson instructions.

Steps Taken

Launch Tezos Sandbox

The Tezos sandbox setup is well-documented in the Octez documentation.
The node is initially created with the genesis protocol, but we need RPC endpoints that are not available in this version. Normally, we could use the alpha protocol; however, the tools we plan to connect might not be compatible with the latest protocol version. A good alternative is to activate a released protocol version, such as Paris.

To activate the Paris protocol, apply the following patch to the octez-init-sandboxed-client.sh script:

diff --git a/src/bin_client/octez-init-sandboxed-client.sh b/src/bin_client/octez-init-sandboxed-client.sh
index 4ddab088c0..7aac0f9398 100755
--- a/src/bin_client/octez-init-sandboxed-client.sh
+++ b/src/bin_client/octez-init-sandboxed-client.sh
@@ -94,6 +94,7 @@ main() {
     local_compiler="${local_compiler:-$bin_dir/../../_build/default/src/lib_protocol_compiler/bin/main_native.exe}"

     quebec_parameters_file="$bin_dir/../../_build/default/src/proto_021_PsQuebec/lib_parameters/sandbox-parameters.json"
+    paris_parameters_file="$bin_dir/../../_build/default/src/proto_020_PsParisC/lib_parameters/sandbox-parameters.json"
     parameters_file="$bin_dir/../../_build/default/src/proto_alpha/lib_parameters/sandbox-parameters.json"

   else
@@ -170,6 +171,7 @@ main() {
   cat << EOF
 if type octez-client-reset >/dev/null 2>&1 ; then octez-client-reset; fi ;
 PATH="$client_dir/bin:\$PATH" ; export PATH ;
+alias octez-activate-paris="$client -block genesis activate protocol PsParisCZo7KAh1Z1smVd9ZMZ1HHn5gkzbM94V3PLCpknFWhUAi with fitness 1 and key activator and parameters $paris_parameters_file";
 alias octez-activate-quebec="$client -block genesis activate protocol PsQuebecnLByd3JwTiGadoG4nGWi3HYiLXUjkibeFV8dCFeVMUg with fitness 1 and key activator and parameters $quebec_parameters_file";
 alias octez-activate-alpha="$client  -block genesis activate protocol ProtoALphaALphaALphaALphaALphaALphaALphaALphaDdp3zK with fitness 1 and key activator and parameters $parameters_file" ;
 alias octez-client-reset="rm -rf \"$client_dir\"; unalias octez-activate-alpha octez-client-reset" ;

Deploying TzKT

The TzKT source code was obtained from https://github.com/baking-bad/tzkt.git.
Unfortunately, the current codebase does not work out of the box and requires a one-line change in its codebase. Additionally, setting environment variables prevents us from using the provided Docker images and make scripts.

Apply the following patch to fix the issue:

diff --git a/Tzkt.Sync/Protocols/Handlers/Proto1/Rpc/Rpc.cs b/Tzkt.Sync/Protocols/Handlers/Proto1/Rpc/Rpc.cs
index c30586c9..9b1f0fff 100644
--- a/Tzkt.Sync/Protocols/Handlers/Proto1/Rpc/Rpc.cs
+++ b/Tzkt.Sync/Protocols/Handlers/Proto1/Rpc/Rpc.cs
@@ -11,7 +11,7 @@ namespace Tzkt.Sync.Protocols.Proto1

     #region indexer
     public virtual Task<JsonElement> GetBlockAsync(int level)
-        => Node.GetAsync($"chains/main/blocks/{level}?version=0");
+        => Node.GetAsync($"chains/main/blocks/{level}?version=1");

     public virtual Task<JsonElement> GetBakingRightsAsync(int block, int cycle)
         => Node.GetAsync($"chains/main/blocks/{block}/helpers/baking_rights?cycle={cycle}&max_priority=8&all=true");

The code is written for .NET 7.0, which is no longer supported. Replace all occurrences of <TargetFramework>net7.0</TargetFramework> with:
```
<TargetFramework>net8.0</TargetFramework>
```
Replace all instances of https://rpc.tzkt.io/mainnet with http://localhost:18731 (assuming this is the address of the sandbox Tezos node).
Replace host=db with host=localhost.

Database Setup

Assuming it's the first time deploying the system, execute the following commands to set up the PostgreSQL database:

sudo -u postgres psql

postgres=# CREATE DATABASE tzkt_db;
postgres=# CREATE USER tzkt WITH ENCRYPTED PASSWORD 'qwerty';
postgres=# GRANT ALL PRIVILEGES ON DATABASE tzkt_db TO tzkt;
postgres=# \q

sudo -u postgres psql tzkt_db

tzkt_db=# GRANT ALL ON SCHEMA public TO tzkt;
tzkt_db=# \q

If re-deploying and the database already exists, run these commands instead:

sudo -u postgres psql

postgres=# DROP DATABASE tzkt_db;
postgres=# CREATE DATABASE tzkt_db;
postgres=# GRANT ALL PRIVILEGES ON DATABASE tzkt_db TO tzkt;
postgres=# \q

sudo -u postgres psql tzkt_db

tzkt_db=# GRANT ALL ON SCHEMA public TO tzkt;
tzkt_db=# \q

Finally, follow the steps in the README to deploy the API and sync runners.

Deploying Umami

Attempts to deploy the services with a self-signed HTTPS locally resulted in CORS issues. Therefore, it was necessary to modify Umami and deploy it locally.
Umami was obtained from https://github.com/trilitech/umami-v2.git.
The following modifications enable the program to accept a localhost URL and add a button for the sandbox:

diff --git a/packages/components/src/utils/validationSchemes.ts b/packages/components/src/utils/validationSchemes.ts
index 8022aaa5..44e50197 100644
--- a/packages/components/src/utils/validationSchemes.ts
+++ b/packages/components/src/utils/validationSchemes.ts
@@ -3,14 +3,25 @@ import { type Network } from "@umami/tezos";
 import { z } from "zod";

 const URL_REGEX =
-  /^(https?:\/\/)?(www\.)?([a-zA-Z0-9_-]+(\.[a-zA-Z0-9_-]+)+)(\/[a-zA-Z0-9_-]+)*(\/?|\?[a-zA-Z0-9_-]+=?[a-zA-Z0-9_-]*(&[a-zA-Z0-9_-]+=?[a-zA-Z0-9_-]*)*)?(#[a-zA-Z0-9_-]+)?$/;
+  /^(https?:\/\/)?(www\.)?([a-zA-Z0-9_-]+(\.[a-zA-Z0-9_-]+)*|localhost|(?:\d{1,3}\.){3}\d{1,3})(:\d+)?(\/[a-zA-Z0-9_-]*)*(\/?|\?[a-zA-Z0-9_&=.-]*)?(#[a-zA-Z0-9_-]+)?$/;

 const urlScheme = (urlType: string) =>
   z
     .string()
     .min(1, `${urlType} URL is required`)
     .regex(URL_REGEX, `Enter a valid ${urlType} URL`)
-    .startsWith("https://", { message: `${urlType} URL must be secure and start with 'https://'` });
+    .refine(
+      (url) =>
+        url.startsWith("https://") ||
+        url.startsWith("http://localhost") ||
+        url.startsWith("http://127.0.0.1") ||
+        url.startsWith("http://0.0.0.0"),
+      {
+        message: `${urlType} URL must start with 'https://' or be a localhost URL`,
+      }
+    );

 export const getNetworkValidationScheme = (availableNetworks: Network[], network?: Network) =>
   z.object({
diff --git a/packages/tezos/src/Network.ts b/packages/tezos/src/Network.ts
index 6eb13f80..bea6f8d3 100644
--- a/packages/tezos/src/Network.ts
+++ b/packages/tezos/src/Network.ts
@@ -16,6 +16,14 @@ export const GHOSTNET: Network = {
   buyTezUrl: "https://faucet.ghostnet.teztnets.com/",
 };

+export const SANDBOX: Network = {
+  name: "sandbox",
+  rpcUrl: "http://localhost:8080",
+  tzktApiUrl: "http://localhost:8081",
+  tzktExplorerUrl: "https://ghostnet.tzkt.io",
+  buyTezUrl: "",
+};
+
 export const isDefault = (network: Network) => !!DefaultNetworks.find(n => n.name === network.name);

-export const DefaultNetworks: Network[] = [MAINNET, GHOSTNET];
+export const DefaultNetworks: Network[] = [MAINNET, GHOSTNET, SANDBOX];

It was necessary to set up a proxy to adapt CORS headers for local deployment. We used nginx for this purpose.
Below is the required configuration for nginx.

Tezos RPC Node Proxy Configuration

Create an nginx server block listening on port 8080:

server {
    listen 8080;
    # Uncomment the following lines if using SSL
    # listen 443 ssl;
    # server_name tezosnode.local;
    # ssl_certificate     tezos/tezosnode.crt;
    # ssl_certificate_key tezos/tezosnode.key;

    location / {
        # Hide upstream CORS headers to prevent duplication
        proxy_hide_header Access-Control-Allow-Origin;

        # Add CORS headers to all responses
        add_header 'Access-Control-Allow-Origin' '*' always;
        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
        add_header 'Access-Control-Allow-Headers' 'Origin, Content-Type, Accept, Authorization' always;
        add_header 'Access-Control-Max-Age' 1728000 always;

        # Handle OPTIONS method
        if ($request_method = OPTIONS) {
            return 204;
        }

        proxy_pass http://localhost:18731;
        proxy_set_header Host $host;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'keep-alive';
    }
}

TzKT API Proxy Configuration

Create another nginx server block listening on port 8081:

server {
    listen 8081;
    # Uncomment the following lines if using SSL
    # listen 443 ssl;
    # server_name tzktapi.local;
    # ssl_certificate     tezos/tzktapi.crt;
    # ssl_certificate_key tezos/tzktapi.key;

    location / {
        # Hide upstream CORS headers to prevent duplication
        proxy_hide_header Access-Control-Allow-Origin;

        # Add CORS headers to all responses
        add_header 'Access-Control-Allow-Origin' '*' always;
        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
        add_header 'Access-Control-Allow-Headers' 'Origin, Content-Type, Accept, Authorization' always;
        add_header 'Access-Control-Max-Age' 1728000 always;

        # Handle OPTIONS method
        if ($request_method = OPTIONS) {
            return 204;
        }

        proxy_pass http://localhost:5000;
        proxy_set_header Host $host;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'keep-alive';
    }
}

Teia-ui

To introduce Teia-ui into our sandbox environment, you must modify the configuration file containing the deployed contract addresses. This adjustment currently prevents the use of the official Docker and web-based implementations of Teia-ui. Additionally, the repository in its current state does not compile from source due to dependency vulnerabilities.

To resolve these issues, apply the following patch:

teia_diff.txt

By applying this patch, Teia-ui should successfully build and run in the demo.

Demo scripts

After making the required changes, you can set up the Tezos sandbox, deploy TzKT, and run Umami locally. The following scripts automate these steps, assuming you have the necessary repositories, have applied the requested patches, and have a running nginx process:

demo_browser.py
demo.sh

List of used RPCs

We gathered a list of all RPC calls by parsing the Tezos node logs in debug mode. After removing duplicates, the cleaned list of RPCs is available here:

demo_rpc.txt

Summary:

chain_id:=main|NetXdQprcVkpaWU
block:=(genesis|head|N|B...)(~N)?

/chains/<chain_id>/blocks/<block>
/chains/<chain_id>/blocks/<block>/context/constants
/chains/<chain_id>/blocks/<block>/context/contracts/<KT1...>
/chains/<chain_id>/blocks/<block>/context/contracts/<KT1...>/entrypoints
/chains/<chain_id>/blocks/<block>/context/contracts/<tz1...>
/chains/<chain_id>/blocks/<block>/context/contracts/<tz1...>/balance
/chains/<chain_id>/blocks/<block>/context/contracts/<tz1...>/counter
/chains/<chain_id>/blocks/<block>/context/contracts/<tz1...>/manager_key
/chains/<chain_id>/blocks/<block>/context/issuance/expected_issuance
/chains/<chain_id>/blocks/<block>/context/raw/json/cycle/<N>
/chains/<chain_id>/blocks/<block>/hash
/chains/<chain_id>/blocks/<block>/header
/chains/<chain_id>/blocks/<block>/header/raw
/chains/<chain_id>/blocks/<block>/header/shell
/chains/<chain_id>/blocks/<block>/helpers/current_level?offset=1
/chains/<chain_id>/blocks/<block>/helpers/preapply/block?timestamp=<N>
/chains/<chain_id>/blocks/<block>/helpers/preapply/operations
/chains/<chain_id>/blocks/<block>/helpers/preapply/operations?version=1
/chains/<chain_id>/blocks/<block>/helpers/scripts/simulate_operation
/chains/<chain_id>/blocks/<block>/helpers/scripts/simulate_operation?version=1
/chains/<chain_id>/blocks/<block>/helpers/validators?level=1
/chains/<chain_id>/blocks/<block>/live_blocks
/chains/<chain_id>/blocks/<block>/operation_hashes
/chains/<chain_id>/blocks/<block>/operations/3/0?version=1
/chains/<chain_id>/blocks/<block>/operations?version=1
/chains/<chain_id>/blocks/<block>/protocols
/chains/<chain_id>/blocks/<block>/resulting_context_hash
/chains/<chain_id>/blocks/<block>?version=1
/chains/<chain_id>/chain_id
/chains/<chain_id>/mempool/monitor_operations?version=1&validated=yes&refused=no&outdated=no&branch_refused=no&branch_delayed=yes
/chains/<chain_id>/mempool/pending_operations?version=2&validated=yes&refused=no&outdated=no&branch_refused=no&branch_delayed=yes
/injection/block
/injection/block?async=true&chain=NetXdQprcVkpaWU
/injection/operation
/injection/operation?chain=main
/monitor/bootstrapped
/monitor/heads/main
/monitor/heads/main?next_protocol=PsParisCZo7KAh1Z1smVd9ZMZ1HHn5gkzbM94V3PLCpknFWhUAi
/version

Using coverage tests

Following the official Tezos documentation on measuring test coverage, we performed a series of actions during our demo:

Executed the initialization scripts
Linked the dapp to the wallet
Minted a batch of NFTs
Transferred NFTs to another account
Created a new account
Executed transfers to and from the new account
Performed unstaking operations in the wallet

The resulting coverage report is provided here:

_coverage_report.zip