[go: up one dir, main page]

Draft: Spike: Add KEV and EPSS score filtering for MR approval policies

Imam Hossainrequested to merge
581332-spike-access-cev-id-for-security-findings-in-mrap-policies into master

What does this MR do and why?

Introduces a new cache table for security finding enrichment data, including CISA KEV and EPSS scores. This enables filtering vulnerabilities in scan result policies based on these new attributes, allowing for more precise and risk-aware policy enforcement. The cache is populated when security findings are stored.

Tasks

  • Update security orchestration json schema with the new vulnerability attributes
  • Create security_finding_enrichment_caches table and synchronize the data
  • Update approval rules evaluation to consider the new policy attributes for Newly Detected findings
  • Implement approval rules evaulations for the kev and epss score for Previously Existing vulnerabilities
  • Support reachability attribute filter for DS scan vulnerabilities/findings

Query Plans

Security Finding Filter

Note: The join on pm_cve_enrichment and security_finding_enrichments are newly introduced.

  SELECT DISTINCT "security_findings".*
  FROM "security_findings"
  INNER JOIN "security_scans"
    ON "security_findings"."scan_id" = "security_scans"."id"
  INNER JOIN "security_finding_enrichments"
    ON "security_finding_enrichments"."finding_uuid" = "security_findings"."uuid"
  INNER JOIN "pm_cve_enrichment"
    ON "pm_cve_enrichment"."id" = "security_finding_enrichments"."cve_enrichment_id"
  WHERE "security_scans"."pipeline_id" = 2062
    AND "security_findings"."partition_number" = 4
    AND "security_scans"."latest" = TRUE
    AND "security_scans"."status" = 1
    AND "pm_cve_enrichment"."is_known_exploit" = TRUE
    AND "pm_cve_enrichment"."epss_score" > 0.1
Query Plan 
Unique  (cost=27.15..27.18 rows=1 width=1292) (actual time=0.207..0.207 rows=1 loops=1)
  ->  Sort  (cost=27.15..27.16 rows=1 width=1292) (actual time=0.207..0.207 rows=1 loops=1)
        Sort Key: security_findings.id, security_findings.scan_id, security_findings.scanner_id, security_findings.severity, security_findings.deduplicated, security_findings.uuid, security_findings.overridden_uuid, security_findings.finding_data, security_findings.project_id
        Sort Method: quicksort  Memory: 25kB
        ->  Nested Loop  (cost=0.99..27.14 rows=1 width=1292) (actual time=0.186..0.193 rows=1 loops=1)
              ->  Nested Loop  (cost=0.57..16.47 rows=5 width=1300) (actual time=0.066..0.092 rows=16 loops=1)
                    ->  Nested Loop  (cost=0.42..16.07 rows=1 width=1292) (actual time=0.036..0.054 rows=16 loops=1)
                          ->  Index Scan using index_security_scans_on_pipeline_id_and_scan_type on security_scans  (cost=0.28..3.28 rows=1 width=8) (actual time=0.018..0.021 rows=1 loops=1)
                                Index Cond: (pipeline_id = 2062)
                                Filter: (latest AND (status = 1))
                                Rows Removed by Filter: 6
                          ->  Index Scan using security_findings_4_scan_id_id_idx on security_findings_4 security_findings (cost=0.14..12.52 rows=28 width=1292) (actual time=0.017..0.031 rows=16 loops=1)
                                Index Cond: (scan_id = security_scans.id)
                                Filter: (partition_number = 4)
                    ->  Index Only Scan using index_sec_finding_enrichments_on_finding_and_enrichment on security_finding_enrichments  (cost=0.15..0.35 rows=5 width=24) (actual time=0.002..0.002 rows=1 loops=16)
                          Index Cond: (finding_uuid = security_findings.uuid)
                          Heap Fetches: 16
              ->  Index Scan using pm_cve_enrichment_pkey on pm_cve_enrichment  (cost=0.42..2.13 rows=1 width=8) (actual time=0.006..0.006 rows=0 loops=16)
                    Index Cond: (id = security_finding_enrichments.cve_enrichment_id)
                    Filter: (is_known_exploit AND (epss_score > '0.1'::double precision))
                    Rows Removed by Filter: 1
Planning Time: 0.421 ms
Execution Time: 0.245 ms

Vulnerability Filter

Plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/46142/commands/140998

SELECT 
  DISTINCT "vulnerabilities".* 
FROM 
  "vulnerabilities" 
  INNER JOIN "vulnerability_occurrences" ON "vulnerability_occurrences"."vulnerability_id" = "vulnerabilities"."id" 
  INNER JOIN "vulnerability_occurrence_identifiers" ON "vulnerability_occurrence_identifiers"."occurrence_id" = "vulnerability_occurrences"."id" 
  INNER JOIN "vulnerability_identifiers" ON "vulnerability_identifiers"."id" = "vulnerability_occurrence_identifiers"."identifier_id" 
  AND (LOWER(vulnerability_identifiers.external_type) = 'cve') 
  INNER JOIN "pm_cve_enrichment" ON "pm_cve_enrichment"."cve" = "vulnerability_identifiers"."name" 
WHERE 
  "vulnerabilities"."project_id" = 68128652 
  AND "vulnerabilities"."present_on_default_branch" = TRUE 
  AND "pm_cve_enrichment"."is_known_exploit" = TRUE 
  AND "pm_cve_enrichment"."epss_score" > 0.1

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. Create the following security policies for the project
  • Copy the following policies in .gitlab/security-policies/policy.yml
  • Navigate to Project -> Policies -> Edit policy project and assign the project to itself as the security policy project
Policy YAML

.gitlab/security-policies/policy.yml

approval_policy:
  - name: Security Scan - Newly Detected
    description: ""
    enabled: true
    rules:
      - type: scan_finding
        scanners:
          - dependency_scanning
        vulnerabilities_allowed: 0
        severity_levels: []
        vulnerability_states:
          - new_needs_triage
        branch_type: protected
        vulnerability_attributes:
          epss_score:
            operator: greater_than
            value: 0.8
          known_exploited: true
    actions:
      - type: require_approval
        approvals_required: 1
        role_approvers:
          - developer
          - maintainer
          - owner
      - type: send_bot_message
        enabled: true
    approval_settings:
      block_branch_modification: false
      prevent_pushing_and_force_pushing: false
      prevent_approval_by_author: false
      prevent_approval_by_commit_author: false
      remove_approvals_with_new_commit: false
      require_password_to_approve: false
    fallback_behavior:
      fail: closed
  - name: Security Scan - Preexisting
    description: ""
    enabled: true
    enforcement_type: enforce
    rules:
      - type: scan_finding
        scanners:
          - dependency_scanning
        vulnerabilities_allowed: 0
        severity_levels: []
        vulnerability_states:
          - detected
          - confirmed
          - dismissed
          - resolved
        branch_type: protected
        vulnerability_attributes:
          known_exploited: false
          epss_score:
            operator: greater_than
            value: 0.1
    actions:
      - type: require_approval
        approvals_required: 1
        role_approvers:
          - developer
          - maintainer
          - owner
      - type: send_bot_message
        enabled: true
    approval_settings:
      block_branch_modification: false
      prevent_pushing_and_force_pushing: false
      prevent_approval_by_author: false
      prevent_approval_by_commit_author: false
      remove_approvals_with_new_commit: false
      require_password_to_approve: false
    fallback_behavior:
      fail: closed
  1. Add the required scanners in '.gitlab-ci.yml'
# See https://docs.gitlab.com/ee/ci/yaml/ for all available options
image: busybox:latest
include:
  - template: 'Jobs/Dependency-Scanning.gitlab-ci.yml'
  1. Create a requirements.txt file in the main branch
PyYAML==3.12 ## "CVE-2020-14343"
  1. Wait for pipeline finish then create a MR to update the requirements.txt file with:
PyYAML==3.12 ## "CVE-2020-14343"
requests==2.19 ## "CVE-2023-32681"

Observation: MR is blocked by Preexisting security policy

  1. Update the package metadata with the following epss_score to match with the policy requirement
PackageMetadata::CveEnrichment.find_by(cve: "CVE-2023-32681").update!(epss_score: 0.9, is_known_exploit: true)
  1. Run the MR pipeline again

Observation: MR is blocked by the security policy Security Scan - Newly Detected

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #581332

Edited by Imam Hossain

Merge request reports