Alcotezt-UX: Protocol-specific test naming and tagging
Context
This MR builds on !7965 (merged).
It adds protocol-specific Alcotezt wrappers that automatically prefix test titles with the protocol names, and adds an protocol-appropriate tag.
Manually testing the MR
While not strictly necessary, this MR is best experienced with the development version of tezt discussed in !7965 (merged) (that this MR is based on). You can install it through:
$ opam pin tezt git@gitlab.com:nomadic-labs/tezt.git#arvid@filename-ux-improvements
Then try:
$ tezt -f liquidity_baking_pbt.ml --list
+----------------------------------------------------------------------+----------------------------------------------------------------------+------------------+
| FILE | TITLE | TAGS |
+----------------------------------------------------------------------+----------------------------------------------------------------------+------------------+
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | Alpha: protocol > pbt > liquidity baking: Machines Cross-Validation | alcotest, alpha |
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | Alpha: protocol > pbt > liquidity baking: Economic Properties | alcotest, alpha |
| src/proto_016_PtMumbai/lib_protocol/test/pbt/liquidity_baking_pbt.ml | Mumbai: protocol > pbt > liquidity baking: Machines Cross-Validation | alcotest, mumbai |
| src/proto_016_PtMumbai/lib_protocol/test/pbt/liquidity_baking_pbt.ml | Mumbai: protocol > pbt > liquidity baking: Economic Properties | alcotest, mumbai |
| src/proto_015_PtLimaPt/lib_protocol/test/pbt/liquidity_baking_pbt.ml | Lima: protocol > pbt > liquidity baking: Machines Cross-Validation | alcotest, lima |
| src/proto_015_PtLimaPt/lib_protocol/test/pbt/liquidity_baking_pbt.ml | Lima: protocol > pbt > liquidity baking: Economic Properties | alcotest, lima |
+----------------------------------------------------------------------+----------------------------------------------------------------------+------------------+
(tezos)
Reviewing this MR
This MR ended up being a bit complicated and big. I've tried to split it into logical units, so reviewing commit-by-commit is recommended. I'll also happily split it if we can agree first that this is the correct direction.
Things might seem more complicated than necessary, because I tried to take all of this into account:
- Snapshotting should be simple
- Tags and title prefixes should be uniform between Tezt integration tests and Tezt protocol unit tests
- You shouldn't have to hard code prefixes or tags to protocol unit tests
- It should be hard to accidentally forget prefixing a test or adding the appropriate tag.
- I didn't want to introduce a lot more opam packages
The MR adds a lot of automation to the Alcotezt part of things, but the enforcing prefixing and tagging of "regular" Tezt unit tests is more maunal. See test_logging.ml for an example of this. More automation / static enforcements can be added here once we have more such tests.
Another complication comes from the proto_*/lib_client tests. They are not removed when a protocol is snapshotted. Moreover, they do not (and cannot) depend on the protocol test helpers (in lib_protocol/test/helpers) as those are removed when freezing. Also, they test more protocols than what is represented by Tezt's tezt/lib_tezos/protocol.ml. Therefore, I had a choice between extending the latter or adding a hack. I did the latter.
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Document any change to the user interface, including configuration parameters (see node configuration) -
Provide automatic testing (see the testing guide). -
For new features and bug fixes, add an item in the appropriate changelog ( docs/protocols/alpha.rstfor the protocol and the environment,CHANGES.rstat the root of the repository for everything else). -
Select suitable reviewers using the Reviewersfield below. -
Select as Assigneethe next person who should take action on that MR