DAL_node: Add GET attestable_slots streamed RPC

What

Parent MR: !19434 (merged)

Add the streaming RPC GET attestable_slots, which is only parametrised by the pkh given by the delegate.

Why

On top of this work, we will adapt the baker code such that it will retrieve the attestation status without delay, at the moment when it’s needed, instead of querying the DAL node and waiting for an answer.

How

By using push-based interface that lets the consumer (the baker in the future) subscribe to newly attestable DAL slots for a given delegate (pkh). Instead of polling at every level, the baker receives a stream of slot_ids as soon as the DAL node has enough shards for that pkh to attest.

New RPC: GET /profiles/<pkh>/monitor/attestable_slots (stream)

No backfill, no explicit starting level: the stream begins from subscription time.

Important notes

• Ordering (in terms of published slot levels) is not guaranteed, the items are pushed in streams as soon as they are ready, the consumers need to not depend on order

• In the baker, if we use this RPC, we will no longer have the shards received messages, as in the new "monitoring" RPC, we do not gather all the slots for a particular delegate, we simply push a slot when it is attestable. So, this could be done in the baker, I guess, can be discussed. From what I discussed with @eugenz , this is not a blocker.

[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.575 NOTICE │ For slots
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.575 NOTICE │   1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 26, 39, 45, 46, 48, 49, 53, 58, 60
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.575 NOTICE │   published at level 19, tz1LQbTo4cRjs got all its shards.
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │ For slots
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │   0, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 25, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44, 47, 50, 51, 52, 54, 55, 56, 57, 59, 61, 62, 63
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │   published at level 19, tz1LQbTo4cRjs missed
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │   shards:
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │  For slot index 0, 0 shards out of 2 were received
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │  For slot index 4, 1 shards out of 2 were received
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │  For slot index 5, 0 shards out of 2 were received
[14:59:51.576] [baker-dal-node-51] Sep 25 14:59:51.577   WARN │  For slot index 6, 0 shards out of 2 were received

• Refactoring code from Node_context.Attestable_slots and RPC_server.Profile_handlers is not trivial, and the MR is quite big, so this is not really necessary.

Manually testing the MR

Run an Octez node on ghostnet:

./octez-node config init --data-dir ~/.tezos-node-ghostnet --network "ghostnet"

./octez-node identity generate --data-dir ~/.tezos-node-ghostnet

./octez-node snapshot import ghostnet.snapshot --data-dir ~/.tezos-node-ghostnet --no-check

./octez-node run --data-dir ~/.tezos-node-ghostnet --rpc-addr 127.0.0.1:18733 --net-addr 127.0.0.1:19732

Run an Octez DAL node on ghostnet (with attester profile given by tz1Zt8QQ9aBznYNk5LUBjtME9DuExomw9YRs) :

./octez-dal-node config init --endpoint http://127.0.0.1:18733 --attester-profiles=tz1Zt8QQ9aBznYNk5LUBjtME9DuExomw9YRs --operator-profiles=3

./octez-dal-node run --rpc-addr 127.0.0.1:11832

Oct 03 13:08:06.177 NOTICE │ tz1Zt8QQ9aBzn attested slot(s) 0, 1, 4, 7, 10 at attested level 15392731.
Oct 03 13:08:06.177 NOTICE │ Finalized block BL22y2dkSiaKh33Gr6sue8Ji9y7ucip6yUhY1Z7MMJWK8MwfFSt at level
Oct 03 13:08:06.177 NOTICE │   15392731, round 0
Oct 03 13:08:10.531 NOTICE │ tz1Zt8QQ9aBzn attested slot(s) 0, 1, 4, 7, 10 at attested level 15392732.
Oct 03 13:08:10.531 NOTICE │ Finalized block BKoE37w1DsxARyxZGNkTt1ssdfKf2Ec9YEXiv8auQiL4dsuGWL5 at level
Oct 03 13:08:10.531 NOTICE │   15392732, round 0
...

Now, to get the streaming RPC, do:

curl -N "http://127.0.0.1:11832/profiles/tz1Zt8QQ9aBznYNk5LUBjtME9DuExomw9YRs/monitor/attestable_slots"
{"slot_level":15392731,"slot_index":1}
{"slot_level":15392731,"slot_index":4}
{"slot_level":15392731,"slot_index":7}
{"slot_level":15392731,"slot_index":10}
{"slot_level":15392731,"slot_index":0}
{"slot_level":15392732,"slot_index":1}
{"slot_level":15392732,"slot_index":10}
{"slot_level":15392732,"slot_index":4}
{"slot_level":15392732,"slot_index":7}
{"slot_level":15392732,"slot_index":0}
...

On ghostnet, we can observe that we get a new entry with slot_level = X when the node's head is at X+2, because the stream is designed to emit as soon as your node has all shards assigned to your PKH for that slot — we do not wait until the attested level L. Emitting early lets the baker cache “this slot will be attestable at L” and use it when the chain actually reaches L.

Checklist

• Document the interface of any function added or modified (see the coding guidelines)
• Document any change to the user interface, including configuration parameters (see node configuration)
• Provide automatic testing (see the testing guide).
• For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
• Select suitable reviewers using the Reviewers field below.
• Select as Assignee the next person who should take action on that MR