[go: up one dir, main page]

Alcotezt-UX: register Alcotests with [__FILE__]

Context

Related: #5084 (closed)

This modifies the Alcotezts so that they are registred with their __FILE__ instead of setting __FILE__ to library name as before.

This requires us to modify the signature of Alcotezt's run functions, meaning that Alcotezt is no longer signature compatible with Alcotest. I intentionally made the ~__FILE__ argument mandatory, so that we do not forget setting the filename of some file.

As it stands, this make little difference, as Tezt 3.0.0 takes the basename of the __FILE__ argument passed to Test.register. However, with an upcoming change in Tezt, nomadic-labs/tezt!13 (merged), the full path will be used, and it will be possible to match on the full path (or a suffix thereof). The changes in:

You can experiment with upcoming these upcoming Tezt features through pinning or vendoring.

Manually testing the MR

On tezos/tezos master:

$ dune exec src/lib_clic/test/main.exe -- --list
+------------+----------------------------+----------+
|    FILE    |           TITLE            |   TAGS   |
+------------+----------------------------+----------+
| Tezos_clic | dispatch                   | alcotest |
| Tezos_clic | auto-completion-parameters | alcotest |
+------------+----------------------------+----------+

With this change:

$ dune exec src/lib_clic/test/main.exe -- --list
+--------------------------------+----------------------------------------+
| FILE         | TITLE                                  | TAGS            |
+--------------------------------+----------------------------------------+
| test_clic.ml | Tezos_clic: dispatch                   | alcotest        |
| test_clic.ml | Tezos_clic: auto-completion-parameters | alcotest        |
+--------------------------------+----------------------------------------+

(suite name moved into title, I changed this as it was inconsistent between Alcotest.run and Alcotest_lwt.run.

Then, try pinning tezt to the development version that contains all the above changes:

$ opam pin tezt git@gitlab.com:nomadic-labs/tezt.git#arvid@filename-ux-improvements

Now --list will show the full path:

$ dune exec src/lib_clic/test/main.exe -- --list                           
+--------------------------------+----------------------------------------+----------+
|              FILE              |                 TITLE                  |   TAGS   |
+--------------------------------+----------------------------------------+----------+
| src/lib_clic/test/test_clic.ml | Tezos_clic: dispatch                   | alcotest |
| src/lib_clic/test/test_clic.ml | Tezos_clic: auto-completion-parameters | alcotest |
+--------------------------------+----------------------------------------+----------+

As before, you can select a file by it's basename:

tezt -f test_clic.ml --list
+--------------------------------+----------------------------------------+----------+
|              FILE              |                 TITLE                  |   TAGS   |
+--------------------------------+----------------------------------------+----------+
| src/lib_clic/test/test_clic.ml | Tezos_clic: dispatch                   | alcotest |
| src/lib_clic/test/test_clic.ml | Tezos_clic: auto-completion-parameters | alcotest |
+--------------------------------+----------------------------------------+----------+

However, note that it will select all tests with that basename:

$ tezt -f liquidity_baking_pbt.ml --list
+----------------------------------------------------------------------+----------------------------------------------------------------------------+----------+
|                                 FILE                                 |                                   TITLE                                    |   TAGS   |
+----------------------------------------------------------------------+----------------------------------------------------------------------------+----------+
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml        | protocol > pbt > liquidity baking: alpha: Machines Cross-Validation        | alcotest |
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml        | protocol > pbt > liquidity baking: alpha: Economic Properties              | alcotest |
| src/proto_016_PtMumbai/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 016-PtMumbai: Machines Cross-Validation | alcotest |
| src/proto_016_PtMumbai/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 016-PtMumbai: Economic Properties       | alcotest |
| src/proto_015_PtLimaPt/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 015-PtLimaPt: Machines Cross-Validation | alcotest |
| src/proto_015_PtLimaPt/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: 015-PtLimaPt: Economic Properties       | alcotest |
+----------------------------------------------------------------------+----------------------------------------------------------------------------+----------+

You can narrow the search down by providing a longer suffix of the tests path (or the full path):

$ tezt -f src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml --list
+---------------------------------------------------------------+---------------------------------------------------------------------+----------+
|                             FILE                              |                                TITLE                                |   TAGS   |
+---------------------------------------------------------------+---------------------------------------------------------------------+----------+
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: alpha: Machines Cross-Validation | alcotest |
| src/proto_alpha/lib_protocol/test/pbt/liquidity_baking_pbt.ml | protocol > pbt > liquidity baking: alpha: Economic Properties       | alcotest |
+---------------------------------------------------------------+---------------------------------------------------------------------+----------+

You can select all the tests of a given folder using --match:

$ tezt --match src/lib_shell --list
+-------------------------------------------------------------------+------------------------------------------------------------------+----------+
|                               FILE                                |                              TITLE                               |   TAGS   |
+-------------------------------------------------------------------+------------------------------------------------------------------+----------+
| src/lib_shell_services/test/test_block_services.ml                | tezos-lib-shell-services: block-services                         | alcotest |
| src/lib_shell/test/test_synchronisation_heuristic_fuzzy.ml        | synchronisation heuristic fuzzy: synchronisation heuristic fuzzy | alcotest |
| src/lib_shell/test/test_shell_operation.ml                        | Shell_operation: Corner cases                                    | alcotest |
...

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Arvid Jakobsson

Merge request reports

Loading