Alcotezt-UX: register Alcotests with [__FILE__]
Context
Related: #5084 (closed)
This modifies the Alcotezts so that they are registred with their __FILE__ instead of setting __FILE__ to library name as before.
This requires us to modify the signature of Alcotezt's run functions, meaning that Alcotezt is no longer signature compatible with
Alcotest. I intentionally made the ~__FILE__ argument mandatory, so that we do not forget setting the filename of some file.
As it stands, this make little difference, as Tezt 3.0.0 takes the basename of the __FILE__ argument passed to Test.register.
However, with an upcoming change in Tezt, nomadic-labs/tezt!13 (merged), the full path will be used, and it will be possible to match on the full path (or a suffix thereof). The changes in:
- nomadic-labs/tezt!17 (merged) also makes it possible to use match to match on the test's full path
- nomadic-labs/tezt!14 (merged) makes it possible to deselect tests based on their path
You can experiment with upcoming these upcoming Tezt features through pinning or vendoring.
Manually testing the MR
On tezos/tezos master:
$ dune exec src/lib_clic/test/main.exe -- --list
+------------+----------------------------+----------+
| FILE | TITLE | TAGS |
+------------+----------------------------+----------+
| Tezos_clic | dispatch | alcotest |
| Tezos_clic | auto-completion-parameters | alcotest |
+------------+----------------------------+----------+
With this change:
$ dune exec src/lib_clic/test/main.exe -- --list
+--------------------------------+----------------------------------------+
| FILE | TITLE | TAGS |
+--------------------------------+----------------------------------------+
| test_clic.ml | Tezos_clic: dispatch | alcotest |
| test_clic.ml | Tezos_clic: auto-completion-parameters | alcotest |
+--------------------------------+----------------------------------------+
(suite name moved into title, I changed this as it was inconsistent between Alcotest.run and Alcotest_lwt.run.
Then, try pinning tezt to the development version that contains all the above changes:
$ opam pin tezt git@gitlab.com:nomadic-labs/tezt.git#arvid@filename-ux-improvements
Now --list will show the full path:
$ dune exec src/lib_clic/test/main.exe -- --list
+--------------------------------+----------------------------------------+----------+
| FILE | TITLE | TAGS |
+--------------------------------+----------------------------------------+----------+
| src/lib_clic/test/test_clic.ml | Tezos_clic: dispatch | alcotest |
| src/lib_clic/test/test_clic.ml | Tezos_clic: auto-completion-parameters | alcotest |
+--------------------------------+----------------------------------------+----------+
As before, you can select a file by it's basename:
tezt -f test_clic.ml --list
+--------------------------------+----------------------------------------+----------+
| FILE | TITLE | TAGS |
+--------------------------------+----------------------------------------+----------+
| src/lib_clic/test/test_clic.ml | Tezos_clic: dispatch | alcotest |
| src/lib_clic/test/test_clic.ml | Tezos_clic: auto-completion-parameters | alcotest |
+--------------------------------+----------------------------------------+----------+
However, note that it will select all tests with that basename:
$ tezt -f liquidity_baking_pbt.ml --list
+----------------------------------------------------------------------+----------------------------------------------------------------------------+----------+
| FILE | TITLE | TAGS |
+----------------------------------------------------------------------+----------------------------------------------------------------------------+----------+
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: alpha: Machines Cross-Validation | alcotest |
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: alpha: Economic Properties | alcotest |
| src/proto_016_PtMumbai/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 016-PtMumbai: Machines Cross-Validation | alcotest |
| src/proto_016_PtMumbai/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 016-PtMumbai: Economic Properties | alcotest |
| src/proto_015_PtLimaPt/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 015-PtLimaPt: Machines Cross-Validation | alcotest |
| src/proto_015_PtLimaPt/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 015-PtLimaPt: Economic Properties | alcotest |
+----------------------------------------------------------------------+----------------------------------------------------------------------------+----------+
You can narrow the search down by providing a longer suffix of the tests path (or the full path):
$ tezt -f src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml --list
+---------------------------------------------------------------+---------------------------------------------------------------------+----------+
| FILE | TITLE | TAGS |
+---------------------------------------------------------------+---------------------------------------------------------------------+----------+
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: alpha: Machines Cross-Validation | alcotest |
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: alpha: Economic Properties | alcotest |
+---------------------------------------------------------------+---------------------------------------------------------------------+----------+
You can select all the tests of a given folder using --match:
$ tezt --match src/lib_shell --list
+-------------------------------------------------------------------+------------------------------------------------------------------+----------+
| FILE | TITLE | TAGS |
+-------------------------------------------------------------------+------------------------------------------------------------------+----------+
| src/lib_shell_services/test/test_block_services.ml | tezos-lib-shell-services: block-services | alcotest |
| src/lib_shell/test/test_synchronisation_heuristic_fuzzy.ml | synchronisation heuristic fuzzy: synchronisation heuristic fuzzy | alcotest |
| src/lib_shell/test/test_shell_operation.ml | Shell_operation: Corner cases | alcotest |
...
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Document any change to the user interface, including configuration parameters (see node configuration) -
Provide automatic testing (see the testing guide). -
For new features and bug fixes, add an item in the appropriate changelog ( docs/protocols/alpha.rstfor the protocol and the environment,CHANGES.rstat the root of the repository for everything else). -
Select suitable reviewers using the Reviewersfield below. -
Select as Assigneethe next person who should take action on that MR