Tezt: add Tag module and flaky tag
Context
Tag ci_disable (also known as ci_disabled — the latter being used in the LTF, the former being used in the regular CI) is starting to be used. But it does not say why a test is disabled. Often, the reason is that the test is flaky. We propose to introduce a dedicated tag for that: flaky.
The advantages of a dedicated tag are that one can then run tools to list flaky tests, measure the flakiness of flaky tests, and one immediately knows why the test was disabled. This alleviates the need to create a dedicated issue for all tests.
This MR also unifies ci_disable and ci_disabled and introduces a Tag module. The module:
- allows devs to jump to the definition of tags (with merlin for instance) to see the documentation of the tag;
- and prevents devs from making a typo thanks to the type-checker.
Manually testing the MR
Try something like:
tezt --list flaky
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Document any change to the user interface, including configuration parameters (see node configuration) -
Provide automatic testing (see the testing guide). -
Select suitable reviewers using the Reviewersfield below. -
Select as Assigneethe next person who should take action on that MRw