Teztale: Integrate shard assignments
What
This MR adds end-to-end support in Teztale for recording and exporting DAL shard assignments per delegate and level, without yet wiring any DAL node / RPC producer.
Why
The broader goal is to be able to inspect DAL shard workflow per delegate in Teztale (and eventually in OTel), as explained in the related milestone.
How
Steps followed:
- Extends the Teztale data model so that
Data.Delegate_operations.tcarries the list of DAL shard indices assigned to that delegate at the given level. - Extends the Teztale SQL schema with a new table
dal_shard_assignments(*id*, endorsing_right, shard_index). - Adds a new HTTP endpoint on the Teztale server
POST /<LEVEL>/dal_shardswhich accepts shard assignment payloads and stores them in the database. - Updates the exporter so that when exporting
/LEVEL.jsonand<MIN>-<MAX>.json,Data.Delegate_operationsentries now include the assigned shard indices reconstructed from the database.
Existing Teztale functionality is kept intact and remains backward compatible.
Manually testing the MR
I used the following configuration teztale.json:
{
"db": "sqlite3:/Users/gabrielmoise/tezos/teztale.db",
"interfaces": [
{
"port": 8080
}
],
"admins": [],
"users": [
{
"login": "gabriel",
"password": "secret"
}
],
"max_batch_size": 1000,
"with_transaction": "FULL",
"with_metrics": false,
"verbosity": "INFO"
}
And I got:
./octez-teztale-server ./teztale.json
sqlite3 teztale.db "SELECT name, sql FROM sqlite_master WHERE name = 'dal_shard_assignments';"
dal_shard_assignments|CREATE TABLE dal_shard_assignments(
id INTEGER PRIMARY KEY,
endorsing_right INTEGER NOT NULL,
shard_index INTEGER NOT NULL,
FOREIGN KEY (endorsing_right) REFERENCES endorsing_rights(id),
UNIQUE (endorsing_right, shard_index))
So, the table is properly created and of course, empty for now:
sqlite3 /Users/gabrielmoise/tezos/teztale.db \
"SELECT * FROM dal_shard_assignments;"
Start the Teztale archiver to populate the tables to have proper delegates:
./octez-teztale-archiver --endpoint http://127.0.0.1:18733 feed http://gabriel:secret@127.0.0.1:8080
Now, we obtain an endorsing_right id from:
sqlite3 teztale.db "
SELECT
er.id,
er.level,
hex(d.address) AS delegate_hex
FROM endorsing_rights AS er
JOIN delegates AS d ON d.id = er.delegate
ORDER BY er.level
LIMIT 5;
"
59|16584604|00A31E81AC3425310E3274A4698A793B2839DC0AFA
33|16584604|0129C603B0F52CA6BB1A164465B7B01CF4DB0A70BB
38|16584604|0210833DB28B9940D7F7C45929B5A394AD7B81E02A
39|16584604|002A27457AEB6AA11653F5687848212AA91B53B1D2
44|16584604|00CAB813F2F1698C74EF2FC13E315B36407D8386A4
So, using this 59 id, we do a manual insertion (as the automatic way will be implemented in the next MR):
sqlite3 teztale.db "
INSERT INTO dal_shard_assignments (endorsing_right, shard_index)
VALUES
(59, 0),
(59, 3),
(59, 7);
"
sqlite3 teztale.db "
SELECT
s.id,
s.endorsing_right,
er.level,
s.shard_index
FROM dal_shard_assignments AS s
JOIN endorsing_rights AS er ON er.id = s.endorsing_right
ORDER BY s.id;
"
1|59|16584604|0
2|59|16584604|3
3|59|16584604|7
Checklist
- Document the interface of any function added or modified (see the coding guidelines)
- Document any change to the user interface, including configuration parameters (see node configuration)
- Provide automatic testing (see the testing guide).
-
For new features and bug fixes, add an item in the appropriate changelog (
docs/protocols/alpha.rstfor the protocol and the environment,CHANGES.rstat the root of the repository for everything else). -
Select suitable reviewers using the
Reviewersfield below. -
Select as
Assigneethe next person who should take action on that MR
Edited by Gabriel Moise