Manifest + Tezt = easy to define local Tezt tests
Context
The idea is to be able to define Tezt tests in libraries, in particular unit tests.
Devs would just:
- create test files like
src/lib_stdlib/tezt/test_stdlib.ml; - use
Test.registerin those files; - declare those tests in the manifest with the
teztfunction of moduleManifest.
Then one could use dune build @src/lib_stdlib/runtezt to run those tests and get something like:
tezos_stdlib_test_something_tezt_exe alias src/lib_stdlib/tezt/runtezt
[13:44:53.708] [SUCCESS] (1/1) dummy test for tezos-stdlib (something)
tezos_stdlib_test_stdlib_tezt_exe alias src/lib_stdlib/tezt/runtezt
[13:44:53.719] [SUCCESS] (1/1) dummy test for tezos-stdlib
Or one can also run the test from the main Tezt executable which gathers all tests.
Each test only depends on the dependencies that are declared in the manifest + the tezt library, i.e. they can be quick to compile if they don't depend on, say, tezos-base. For unit tests that do not depend on tezt-tezos for instance, it should result in faster compilation times.
The runtezt Alias
In order not to run Tezt tests in the CI twice (once with dune runtest src/lib_X and once with dune exec tezt/tests/main.exe), Tezt tests are not added to the runtest alias. They are added to the runtezt alias instead. Which means that one cannot run dune runtest src/lib_X to run Tezt tests: one has to run dune build @src/lib_X/runtezt.
Additionally, for packages which have Tezt tests, we add dune exec runtezt {with-test} to the .opam files.
Having a different command (runtest vs build @runtezt) is not ideal, but it was the easiest way I could find to exclude Tezt tests from the CI unit tests.
Caveat
Tests in src/proto_alpha will eventually be duplicated during the snapshot, and there is some code to make sure their ~title will change when this happens. This piece of code should probably, eventually, be put somewhere where it can be shared among other protocol tests, maybe even enforced somehow.
Manually testing the MR
This MR ported src/proto_alpha/lib_protocol/test/regression. To test it:
make -C manifest
# There is a runtezt alias.
dune build @src/proto_alpha/lib_protocol/test/regression/runtezt
# You can also just run the executable (this allows to pass command-line arguments).
dune exec src/proto_alpha/lib_protocol/test/regression/main.exe
# Tests are also available from "the big Tezt executable".
# This is how the CI runs tests, to benefit from auto-balancing.
dune exec tezt/tests/main.exe -- -f test_logging.ml
# Also works from other directories.
cd src/proto_alpha/lib_protocol/test/regression
dune build @runtezt
dune exec ./main.exe
You should also check that the CI actually runs the test_logging.ml tests.
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Document any change to the user interface, including configuration parameters (see node configuration) -
Provide automatic testing (see the testing guide). -
For new features and bug fixes, add an item in the appropriate changelog ( docs/protocols/alpha.rstfor the protocol and the environment,CHANGES.rstat the root of the repository for everything else). -
Select suitable reviewers using the Reviewersfield below. -
Select as Assigneethe next person who should take action on that MR