Add parameter to filter findings by scan_mode

What does this MR do and why?

This MR updates the Security::FindingsFinder so that it can include or exclude partial scans based on the value of the scan_mode parameter. This will allow us to present partial scan and full scan results in separate areas of the MR widget. In order to make this change, I had to create a separate ReactiveCaching implementation so that controller parameters can be passed through the cache.

Relates to: #543637 (closed)

SQL

Before: https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/42010/commands/128859

Click to expand

SELECT
    security_findings.*
FROM
    security_scans,
    unnest( '{1,2,4,5,6,7}'::pg_catalog.int2[] ) AS severities ( severity ),
    LATERAL (
        SELECT
            security_findings.*
        FROM
            security_findings
            LEFT JOIN vulnerability_occurrences ON vulnerability_occurrences.uuid = security_findings.uuid
        WHERE
            security_findings.scan_id = security_scans.id AND
            COALESCE(
                vulnerability_occurrences.severity,
                security_findings.severity
            ) = severities.severity AND
            security_findings.partition_number = 183 AND
            security_findings.deduplicated = true AND
            (
                NOT EXISTS (
                    SELECT
                        1
                    FROM
                        vulnerabilities
                        JOIN vulnerability_occurrences ON vulnerability_occurrences.vulnerability_id = vulnerabilities.id
                    WHERE
                        vulnerabilities.state = 2 AND
                        vulnerability_occurrences.uuid = security_findings.uuid
                )
            )
        ORDER BY
            security_findings.severity DESC,
            security_findings.id ASC
        LIMIT 21
    ) AS security_findings
WHERE
    security_scans.pipeline_id = 1958065763 AND
    security_scans.latest = true AND
    security_scans.status = 1
ORDER BY
    security_findings.severity DESC,
    security_findings.id ASC
LIMIT 11;

After (without partial scans): https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/42010/commands/128855

Click to expand

SELECT
    security_findings.*
FROM
    security_scans,
    unnest( '{1,2,4,5,6,7}'::pg_catalog.int2[] ) AS severities ( severity ),
    LATERAL (
        SELECT
            security_findings.*
        FROM
            security_findings
            LEFT JOIN vulnerability_occurrences ON vulnerability_occurrences.uuid = security_findings.uuid
        WHERE
            security_findings.scan_id = security_scans.id AND
            COALESCE(
                vulnerability_occurrences.severity,
                security_findings.severity
            ) = severities.severity AND
            security_findings.partition_number = 183 AND
            security_findings.deduplicated = true AND
            (
                NOT EXISTS (
                    SELECT
                        1
                    FROM
                        vulnerabilities
                        JOIN vulnerability_occurrences ON vulnerability_occurrences.vulnerability_id = vulnerabilities.id
                    WHERE
                        vulnerabilities.state = 2 AND
                        vulnerability_occurrences.uuid = security_findings.uuid
                )
            ) AND
            (
                NOT EXISTS (
                    SELECT
                        1
                    FROM
                        vulnerability_partial_scans
                    WHERE
                        vulnerability_partial_scans.scan_id = security_scans.id
                )
            )
        ORDER BY
            security_findings.severity DESC,
            security_findings.id ASC
        LIMIT 21
    ) AS security_findings
WHERE
    security_scans.pipeline_id = 1958065763 AND
    security_scans.latest = true AND
    security_scans.status = 1
ORDER BY
    security_findings.severity DESC,
    security_findings.id ASC
LIMIT 11;

After (with partial scans): https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/42010/commands/128857

This returns 0 rows since the feature is not enabled on production, but we can expect similar performance to the inverse query above.

Click to expand

SELECT
    security_findings.*
FROM
    security_scans,
    unnest( '{1,2,4,5,6,7}'::pg_catalog.int2[] ) AS severities ( severity ),
    LATERAL (
        SELECT
            security_findings.*
        FROM
            security_findings
            LEFT JOIN vulnerability_occurrences ON vulnerability_occurrences.uuid = security_findings.uuid
        WHERE
            security_findings.scan_id = security_scans.id AND
            COALESCE(
                vulnerability_occurrences.severity,
                security_findings.severity
            ) = severities.severity AND
            security_findings.partition_number = 183 AND
            security_findings.deduplicated = true AND
            (
                NOT EXISTS (
                    SELECT
                        1
                    FROM
                        vulnerabilities
                        JOIN vulnerability_occurrences ON vulnerability_occurrences.vulnerability_id = vulnerabilities.id
                    WHERE
                        vulnerabilities.state = 2 AND
                        vulnerability_occurrences.uuid = security_findings.uuid
                )
            ) AND
            (
                EXISTS (
                    SELECT
                        1
                    FROM
                        vulnerability_partial_scans
                    WHERE
                        vulnerability_partial_scans.scan_id = security_scans.id
                )
            )
        ORDER BY
            security_findings.severity DESC,
            security_findings.id ASC
        LIMIT 21
    ) AS security_findings
WHERE
    security_scans.pipeline_id = 1958065763 AND
    security_scans.latest = true AND
    security_scans.status = 1
ORDER BY
    security_findings.severity DESC,
    security_findings.id ASC
LIMIT 11;

References

Screenshots or screen recordings

How to set up and validate locally

1. Enable the feature flag: echo 'Feature.enable(:vulnerability_partial_scans)' | bundle exec rails c

2. Create a new project

3. Add this file to the project in a merge request:

   sast:
     stage: test
     script: wget https://gitlab.com/gitlab-org/gitlab/-/raw/5d7bc4e75ae688b968a6debc0b8f62e7eb1f54d1/ee/spec/fixtures/security_reports/master/gl-sast-report-differential.json
     artifacts:
       reports:
         sast: gl-sast-report-differential.json

4. Go to <gdk_url>/:project_path/-/merge_requests/:id/security_reports?type=sast&scan_mode=partial and observe that results from the report are returned. You will have to do this twice since the first request updates the cache and returns an empty response.

5. Set scan_mode to full and no results should be returned.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.