lib_shell_benchmarks: add intercept benchmarks for DECODING_(INT|NAT)
Context
Resolves #5798 (closed)
The following commands were tested on the Snoop server.
$ ./octez-snoop list all benchmarks | grep -E "DECODING_(INT|NAT)"
encoding/DECODING_INT: Benchmarking encoding/DECODING_INT
encoding/DECODING_INT/intercept: Benchmarking encoding/DECODING_INT/intercept (intercept case)
encoding/DECODING_NAT: Benchmarking encoding/DECODING_NAT
encoding/DECODING_NAT/intercept: Benchmarking encoding/DECODING_NAT/intercept (intercept case)
$ ./octez-snoop generate code for benchmarks encoding/DECODING_INT encoding/DECODING_NAT --out-dir dir
...
$ cat dir/encoding.sol.txt
dir/encoding.sol:
inference_model_name: encoding
free_variables:
builtin/Timer_latency = 24
encoding/DECODING_INT_coeff = 13.4035169156
encoding/DECODING_INT_const = 18
encoding/DECODING_NAT_coeff = 13.3660403858
encoding/DECODING_NAT_const = 9
scores:
builtin/TIMER_LATENCY : R2: NA , RMSE: 1.000000 , T-Timer_latency: 1179.479153
encoding/DECODING_INT : R2: 0.999924 , RMSE: 1073.454634 , T-DECODING_INT_coeff: 22241.809772
encoding/DECODING_INT/intercept : R2: NA , RMSE: 1.000000 , T-DECODING_INT_const: 377.521230
encoding/DECODING_NAT : R2: 0.999975 , RMSE: 628.440998 , T-DECODING_NAT_coeff: 62320.023344
encoding/DECODING_NAT/intercept : R2: NA , RMSE: 1.000000 , T-DECODING_NAT_const: 232.486626
c.f: The reference values from the biweekly CSV
-
encoding/DECODING_INT_coeff:= 13.3234013448 -
encoding/DECODING_NAT_coeff:= 13.2910436478
Plots
DECODING_INT_encoding_validation-0.pdf DECODING_NAT_encoding_validation-0.pdf
Note
As experimented on the Snoop server, I observed variance on the _const parameters. (Both of the constants varied around 9 - 17.) This might raise false alerts at biweekly runs.
I think this problem is not specific to the encoding benchmarks, but is the same as the problem observed in some interpreter benchmarks (e.g. #5027). So I think fixing the variance is out of the scope of this MR, and need to be investigated in the issue 5027.
Checklist
-
Select suitable reviewers using the Reviewersfield below. -
Select as Assigneethe next person who should take action on that MR
Edited by satos