Detect and report on flaky tests

Description

Flaky tests are a huge problem in many teams' CI/CD pipelines. Sometimes flaky tests are avoidable, sometimes they aren't. Even when they are, making them stable can be hard and sometimes prohibitive. We should make it easy to detect flaky tests and report on them so that people can prioritize which tests to fix.

Previous research

🕊 Dovetail project

Proposal

Links / references

• Part of https://gitlab.com/gitlab-org/gitlab-ee/issues/3583

Documentation blurb

Overview

What is it? Why should someone use this feature? What is the underlying (business) problem? How do you use this feature?

Use cases

Who is this for? Provide one or more use cases.

Feature checklist

Make sure these are completed before closing the issue, with a link to the relevant commit.

• Feature assurance
• Documentation
• Added to features.yml

Customers

https://gitlab.my.salesforce.com/00161000004bZEK
https://gitlab.my.salesforce.com/00161000002xBbJ

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.