[go: up one dir, main page]

Fix flaky test: Avoid nested finding creation in security policy scan finding spec

Summary

This MR fixes a flaky test that fails with a PG::ForeignKeyViolation error approximately 5% of the time.

Test: Merge request > User sees security policy with scan finding rule with scan findings when policy is defined for pre-existing vulnerabilities behaves like blocks the MR merge blocked text is present

File: ee/spec/features/merge_request/user_sees_security_policy_rules_scan_findings_spec.rb:183

Issue: https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/2215


Root Cause Analysis

The Error

PG::ForeignKeyViolation: ERROR:  insert or update on table "internal_ids" violates foreign key constraint "fk_rails_f7d46b66c6"
DETAIL:  Key (project_id)=(48) is not present in table "projects".

What Was Happening

The test was calling:

create(:vulnerabilities_finding, :detected, project: project)

The :detected trait creates a vulnerability after the finding:

trait :detected do
  after(:create) do |finding|
    create(
      :vulnerability, :detected,
      project: finding.project,
      vulnerability_finding: finding,
      findings: [finding]
    )
  end
end

However, the vulnerability factory has this line:

vulnerability_finding { association(:vulnerabilities_finding, project: project) }

This always creates a new finding, even when we're already providing one via the :detected trait. The nested finding creation uses an auto-incremented project_id that may not exist in the database, causing a foreign key violation.

Why It's Flaky (5% failure rate)

Project IDs are auto-incremented. When the sequence reaches certain values (like 48), the implicitly created finding references a non-existent project_id. This depends on:

  • Test execution order
  • Number of projects created in previous tests
  • Database sequence state

The Fix

Instead of using the :detected trait (which creates a vulnerability that tries to create its own finding), we:

  1. Create the finding explicitly
  2. Pass it directly to the vulnerability factory

Before:

create(:vulnerabilities_finding, :detected, project: project)

After:

finding = create(:vulnerabilities_finding, project: project)
create(:vulnerability, :detected, project: project, vulnerability_finding: finding, findings: [finding])

This ensures the vulnerability uses the finding we created, rather than trying to create a second one with a potentially invalid project_id.

Pattern Classification

Proof

Execution Trace

Failing Path:

  1. create(:vulnerabilities_finding, :detected, project: project) → Finding created with project_id=1 (valid)
  2. :detected trait triggers after(:create) callback
  3. Callback creates vulnerability with vulnerability_finding: finding
  4. Vulnerability factory evaluates default association: vulnerability_finding { association(:vulnerabilities_finding, project: project) }
  5. NEW finding created with project_id=48 (INVALID) → Foreign key violation

Fixed Path:

  1. finding = create(:vulnerabilities_finding, project: project) → Finding created with project_id=1 (valid)
  2. create(:vulnerability, :detected, project: project, vulnerability_finding: finding, findings: [finding]) → Vulnerability created
  3. Explicit vulnerability_finding: finding parameter overrides the default association
  4. No new finding created → Success

Expected Output

The test expects to see "Merge blocked" text on the page. With this fix:

  • The vulnerability finding is created with the correct project
  • The vulnerability is created with the correct finding
  • No orphaned finding creation attempts
  • No foreign key violations

Both scenarios now pass:

  • Common case: Test passes as before
  • Rare case: No more foreign key violations

Changelog

title: Fix flaky test for security policy scan finding with pre-existing vulnerabilities
merge_request: 216847
author:
type: fixed
ee: true
Edited by Marc Shaw

Merge request reports

Loading