Add Test Execution Data to Project Quality Summary UI
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Problem to solve
As an app dev team lead, I want to see how quality is trending in my projects, so I can prevent escaped defects and spend time shipping features not bug fixes.
Additional information / use cases
Hypothesis: If we provide high level data from more features to team leads, they will get their team to use the features driving usage.
Experiment: Add data views for data we have about test execution and code coverage, add teaser links that can be tracked for other categories to inform what to develop next.
Proposal
Add this as a tab in Analytics -> CI/CD.
Applies to all cards
- Show all metrics based on the latest pipeline results using single stat component
- Add question-circle icons to each card header with more information in a popover on what each means (see #337307[Popover.png] for content that lives within it)
Test runs
- Test runs include percentage of successful tests, failed tests and skipped tests
- See full report links to https://gitlab.com/gitlab-org/gitlab/-/pipelines/396801566/test_report for the latest pipeline
Test coverage
- Test coverage includes percentage of test coverage
- See project Code Coverage Statistics links to https://gitlab.com/gitlab-org/gitlab/-/graphs/master/charts
- The addition of the badge should only be added if #344231 (closed) is completed beforehand. We will only show the percentage if that is not completed.
- If test coverage is not set up, show the empty state
Future Iterations
Add Code quality
- Critical and High Severity Quality Violations - Number of Critical and High violations in the default pipeline's latest report. Link to the Code Quality report view on the latest pipeline on the /default branch.
- Repeat Failing Tests - Number of tests in the default pipeline that have failed multiple times over the last 14 days. This will link to the report at some point in the future.
- Load Testing / Browser Performance -> Link to a landing page / blog post
- Accessibility / Usability -> Link to a landing page / blog post
Links to explore other features
- Add a "nudge" empty state to explore:
- Load Testing / Browser Performance
- Accessibility / Usability -> Link to a landing page / blog post