Result Reporting and Structured Data Formats
What
The TPS calculating part of the benchmark cli, along with checks ensuring errors didn't occur across various levels of abstraction. Provides a cli identical to jstz for easily incorporating the remv benchmark into existing workflows.
Why
Actual TPS numbers finally, and ensuring as much as possible that results are reliable and correct.
How
A data format is defined which is LogType found in the utils/src/data_interface.rs. This defines the communication format between the kernel and the results.rs which handles the results command of cli (found in revm/bench). In line with this the kernel is restructured to use a custom InboxResult enum, abstracting
- kernel_sdk specific things into the pure function
get_inbox_messageand - revm specific things into the pure function
handle_res
get_inbox_message doesn't return a Result so we can't use ?. I've used a continuation passing style with the function to_inbox_result emulating the ? syntax. Hopefully this is readable but if not I can replace it with match statements throughout, or return a Result type mapping Err variants to Log(LogType::Error) in the caller (main loop).
Each revm execution returns an abi encoded byte sequence which is logged verbatim and interpreted by results.rs. results.rs ensures the correct number of mints, transfers and balance checks, and at the end, checks that every balance is correct by simulating the transfer generation strategy (don't know if there's a quicker algorithm, but it's not important as it doesn't run in the kernel).
Manually Testing
From the riscv/revm run make build. Then from riscv/scripts run ./revm-bench -t 100 or ./revm-bench -t 10000 -sn`
make -C src/riscv all
Benchmarking
| riscv | native | |
|---|---|---|
| 13th Gen Intel(R) Core(TM) i7-13700HX | 362.909 TPS | 18931.638 TPS |
Tasks for the Author
-
Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes). -
Eliminate dead code and other spurious artefacts introduced in your changes. -
Document new public functions, methods and types. -
Make sure the documentation for updated functions, methods, and types is correct. -
Add tests for bugs that have been fixed. -
Put in reasonable effort to ensure that CI will pass. make -C src/riscvdune build src/lib_riscvdune build src/rust_deps
-
Benchmark performance and populate the table above if needed. -
Explain changes to regression test captures when applicable. -
Write commit messages to reflect the changes they're about. -
Self-review your changes to ensure they are high-quality. -
Complete all of the above before assigning this MR to reviewers.