From 72f47d638ca2c57702235c0fdc799e82d04dd4b3 Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Wed, 17 Dec 2025 09:48:08 +0000 Subject: [PATCH 1/7] Add prometheus metrics configuration --- doc/install/install_ai_gateway.md | 55 +++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index f6e47fb4621c0f..008cc6c8387606 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -561,6 +561,61 @@ These tests are performed for offline environments: | License | Tests whether your license has the ability to access Code Suggestions feature. | | System exchange | Tests whether Code Suggestions can be used in your instance. If the system exchange assessment fails, users might not be able to use GitLab Duo features. | + +## Monitoring and Observability + +### Set Up Prometheus Metrics for AI Gateway Monitoring + +To enable **Prometheus metrics** on your AI Gateway, set the required environment variables, open port **8082** for metric collection, and restart the AI Gateway container. + +``` +-e AIGW_FASTAPI__METRICS_HOST=0.0.0.0 +-e AIGW_FASTAPI__METRICS_PORT=8082 +``` + +### Set Up Prometheus Metrics for Duo Workflow Service Monitoring + +To enable **Prometheus metrics** on your Duo Worflow Service, set the required environment variables, open port **8083** for metric collection, and restart the AI Gateway container. + +``` +-e PROMETHEUS_METRICS__ADDR=0.0.0.0 +-e PROMETHEUS_METRICS__PORT=8083 +``` + +### Configure Prometheus to Scrape Metrics + +Add the following `prometheus.yml` configuration to your Prometheus instance to collect metrics from your AI Gateway and Duo Workflow Service: + +```yaml +global: + scrape_interval: 15s + +scrape_configs: + - job_name: 'ai-gateway' + static_configs: + - targets: [':8082'] + scheme: 'http' + metrics_path: '/metrics' + + - job_name: 'grpc-service' + static_configs: + - targets: [':8083'] + scheme: 'http' + metrics_path: '/metrics' +``` + +Prometheus will scrape metrics from both services every 15 seconds. + +### Verify Metrics Collection + +1. In the Prometheus UI, navigate to `Status > Targets` to verify that both the **AI Gateway** and **Duo Workflow Service** targets are actively being scraped. +2. Check **Alerts** or **Graph** tabs to query metrics +3. The AI Gateway and Duo Workflow Service expose metrics at these endpoints: + +**AI Gateway metrics**: `http://:8082/metrics` +**Duo Workflow Service metrics**: `http://:8083/metrics` + + ## Does the AI gateway need to autoscale? Autoscaling is not mandatory but is recommended for environments with variable workloads, high concurrency requirements, or unpredictable usage patterns. In the GitLab production environment: -- GitLab From abe13abeb44910e9d87a229c03c04a2c921587ba Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Wed, 17 Dec 2025 10:48:13 +0000 Subject: [PATCH 2/7] Edit install_ai_gateway.md --- doc/install/install_ai_gateway.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index 008cc6c8387606..03de6e8c860400 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -561,14 +561,13 @@ These tests are performed for offline environments: | License | Tests whether your license has the ability to access Code Suggestions feature. | | System exchange | Tests whether Code Suggestions can be used in your instance. If the system exchange assessment fails, users might not be able to use GitLab Duo features. | - ## Monitoring and Observability ### Set Up Prometheus Metrics for AI Gateway Monitoring To enable **Prometheus metrics** on your AI Gateway, set the required environment variables, open port **8082** for metric collection, and restart the AI Gateway container. -``` +```shell -e AIGW_FASTAPI__METRICS_HOST=0.0.0.0 -e AIGW_FASTAPI__METRICS_PORT=8082 ``` @@ -577,7 +576,7 @@ To enable **Prometheus metrics** on your AI Gateway, set the required environmen To enable **Prometheus metrics** on your Duo Worflow Service, set the required environment variables, open port **8083** for metric collection, and restart the AI Gateway container. -``` +```shell -e PROMETHEUS_METRICS__ADDR=0.0.0.0 -e PROMETHEUS_METRICS__PORT=8083 ``` @@ -609,13 +608,12 @@ Prometheus will scrape metrics from both services every 15 seconds. ### Verify Metrics Collection 1. In the Prometheus UI, navigate to `Status > Targets` to verify that both the **AI Gateway** and **Duo Workflow Service** targets are actively being scraped. -2. Check **Alerts** or **Graph** tabs to query metrics -3. The AI Gateway and Duo Workflow Service expose metrics at these endpoints: +1. Check **Alerts** or **Graph** tabs to query metrics. +1. The AI Gateway and Duo Workflow Service expose metrics at these endpoints: **AI Gateway metrics**: `http://:8082/metrics` **Duo Workflow Service metrics**: `http://:8083/metrics` - ## Does the AI gateway need to autoscale? Autoscaling is not mandatory but is recommended for environments with variable workloads, high concurrency requirements, or unpredictable usage patterns. In the GitLab production environment: -- GitLab From be825632da9cebe092088ddf991a9da75d3aebf5 Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Wed, 17 Dec 2025 12:34:53 +0000 Subject: [PATCH 3/7] Edit install_ai_gateway.md --- doc/install/install_ai_gateway.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index 03de6e8c860400..1c68da641403cf 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -581,6 +581,24 @@ To enable **Prometheus metrics** on your Duo Worflow Service, set the required e -e PROMETHEUS_METRICS__PORT=8083 ``` +To expose the metrics ports from the `gitlab-ai-gateway` container to the host, add the following port mappings: +For Docker CLI: + +```shell +-p 8082:8082 \ +-p 8083:8083 \ +``` + +For Docker Compose, add to the `gitlab-ai-gateway` service: + +```shell +ports: + - "8082:8082" + - "8083:8083" +``` + +This exposes the **AI Gateway metrics endpoint** on port `8082` and the **Duo Workflow Service metrics endpoint** on port `8083`. + ### Configure Prometheus to Scrape Metrics Add the following `prometheus.yml` configuration to your Prometheus instance to collect metrics from your AI Gateway and Duo Workflow Service: -- GitLab From f7b724be1807f2454055f0a8f14e6ad4d89774c6 Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Wed, 17 Dec 2025 12:35:05 +0000 Subject: [PATCH 4/7] Apply 1 suggestion(s) to 1 file(s) Co-authored-by: Manoj M J --- doc/install/install_ai_gateway.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index 1c68da641403cf..216149afd2d0c3 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -616,7 +616,7 @@ scrape_configs: - job_name: 'grpc-service' static_configs: - - targets: [':8083'] + - targets: [':8083'] scheme: 'http' metrics_path: '/metrics' ``` -- GitLab From 099726cff12512dcb9eb7c189ede8ef175046e75 Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Wed, 17 Dec 2025 12:35:14 +0000 Subject: [PATCH 5/7] Apply 1 suggestion(s) to 1 file(s) Co-authored-by: Manoj M J --- doc/install/install_ai_gateway.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index 216149afd2d0c3..43eacb2964f996 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -630,7 +630,7 @@ Prometheus will scrape metrics from both services every 15 seconds. 1. The AI Gateway and Duo Workflow Service expose metrics at these endpoints: **AI Gateway metrics**: `http://:8082/metrics` -**Duo Workflow Service metrics**: `http://:8083/metrics` +**Duo Workflow Service metrics**: `http://:8083/metrics` ## Does the AI gateway need to autoscale? -- GitLab From b139efc7f6045b78777b5bf04a69600ab21e03fb Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Thu, 18 Dec 2025 10:48:37 +0000 Subject: [PATCH 6/7] Apply 13 suggestion(s) to 1 file(s) Co-authored-by: Fiona Neill --- doc/install/install_ai_gateway.md | 86 ++++++++++++++++--------------- 1 file changed, 44 insertions(+), 42 deletions(-) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index 43eacb2964f996..c6d48287a133ed 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -561,47 +561,50 @@ These tests are performed for offline environments: | License | Tests whether your license has the ability to access Code Suggestions feature. | | System exchange | Tests whether Code Suggestions can be used in your instance. If the system exchange assessment fails, users might not be able to use GitLab Duo features. | -## Monitoring and Observability +## Monitor the AI Gateway -### Set Up Prometheus Metrics for AI Gateway Monitoring +Use Prometheus to gather metrics about your AI Gateway usage and performance. +### Set up Prometheus metrics for the AI Gateway -To enable **Prometheus metrics** on your AI Gateway, set the required environment variables, open port **8082** for metric collection, and restart the AI Gateway container. +To set up Prometheus metrics on the AI Gateway: -```shell --e AIGW_FASTAPI__METRICS_HOST=0.0.0.0 --e AIGW_FASTAPI__METRICS_PORT=8082 -``` - -### Set Up Prometheus Metrics for Duo Workflow Service Monitoring - -To enable **Prometheus metrics** on your Duo Worflow Service, set the required environment variables, open port **8083** for metric collection, and restart the AI Gateway container. - -```shell --e PROMETHEUS_METRICS__ADDR=0.0.0.0 --e PROMETHEUS_METRICS__PORT=8083 -``` - -To expose the metrics ports from the `gitlab-ai-gateway` container to the host, add the following port mappings: -For Docker CLI: +1. Set the required environment variables and open port **8082**: -```shell --p 8082:8082 \ --p 8083:8083 \ -``` - -For Docker Compose, add to the `gitlab-ai-gateway` service: - -```shell -ports: - - "8082:8082" - - "8083:8083" -``` + ```shell + -e AIGW_FASTAPI__METRICS_HOST=0.0.0.0 + -e AIGW_FASTAPI__METRICS_PORT=8082 + ``` +1. Restart the AI Gateway container. -This exposes the **AI Gateway metrics endpoint** on port `8082` and the **Duo Workflow Service metrics endpoint** on port `8083`. +### Set up Prometheus for the GitLab Duo Workflow service -### Configure Prometheus to Scrape Metrics +To set up Prometheus metrics on the GitLab Duo Workflow service: -Add the following `prometheus.yml` configuration to your Prometheus instance to collect metrics from your AI Gateway and Duo Workflow Service: +1. Set the required environment variables and open port **8083**: + ```shell + -e PROMETHEUS_METRICS__ADDR=0.0.0.0 + -e PROMETHEUS_METRICS__PORT=8083 + ``` +1. Restart the AI Gateway container. +1. Expose the metrics ports from the `gitlab-ai-gateway` container to the host: + - For Docker CLI: + ```shell + -p 8082:8082 \ + -p 8083:8083 \ + ``` + - For Docker Compose, add to the `gitlab-ai-gateway` service: + ```shell + ports: + - "8082:8082" + - "8083:8083" + ``` + This exposes the AI Gateway metrics endpoint on port `8082` and the GitLab Duo Workflow Service metrics endpoint on port `8083`. + +### Configure Prometheus to scrape metrics + +To collect metrics from the AI Gateway and GitLab Duo Workflow service, add the following +`prometheus.yml` configuration to your Prometheus instance. In this configuration, Prometheus +scrapes metrics from both services every 15 seconds. ```yaml global: @@ -621,16 +624,15 @@ scrape_configs: metrics_path: '/metrics' ``` -Prometheus will scrape metrics from both services every 15 seconds. - -### Verify Metrics Collection +### Verify metrics collection -1. In the Prometheus UI, navigate to `Status > Targets` to verify that both the **AI Gateway** and **Duo Workflow Service** targets are actively being scraped. -1. Check **Alerts** or **Graph** tabs to query metrics. -1. The AI Gateway and Duo Workflow Service expose metrics at these endpoints: +To verify that the targets for AI Gateway and GitLab Duo Workflow service are being collected: -**AI Gateway metrics**: `http://:8082/metrics` -**Duo Workflow Service metrics**: `http://:8083/metrics` +1. In the Prometheus UI, go to **Status > Targets**. +1. Go to the **Alerts** or **Graph** tabs to query metrics. The AI Gateway and GitLab Duo Workflow service +expose metrics at the following endpoints: + - AI Gateway: `http://:8082/metrics` + - GitLab Duo Workflow service: `http://:8083/metrics` ## Does the AI gateway need to autoscale? -- GitLab From 93b95ddf3bbf115aef984b03659f049ce26df661 Mon Sep 17 00:00:00 2001 From: Vidhya Hariharan <7292346-vidhya.h@users.noreply.gitlab.com> Date: Thu, 18 Dec 2025 13:35:33 +0000 Subject: [PATCH 7/7] Edit install_ai_gateway.md --- doc/install/install_ai_gateway.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index c6d48287a133ed..702393e670d002 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -564,29 +564,32 @@ These tests are performed for offline environments: ## Monitor the AI Gateway Use Prometheus to gather metrics about your AI Gateway usage and performance. + ### Set up Prometheus metrics for the AI Gateway To set up Prometheus metrics on the AI Gateway: 1. Set the required environment variables and open port **8082**: + ```shell -e AIGW_FASTAPI__METRICS_HOST=0.0.0.0 -e AIGW_FASTAPI__METRICS_PORT=8082 ``` -1. Restart the AI Gateway container. ### Set up Prometheus for the GitLab Duo Workflow service To set up Prometheus metrics on the GitLab Duo Workflow service: 1. Set the required environment variables and open port **8083**: + ```shell -e PROMETHEUS_METRICS__ADDR=0.0.0.0 -e PROMETHEUS_METRICS__PORT=8083 ``` -1. Restart the AI Gateway container. + 1. Expose the metrics ports from the `gitlab-ai-gateway` container to the host: + - For Docker CLI: ```shell -p 8082:8082 \ @@ -600,6 +603,8 @@ To set up Prometheus metrics on the GitLab Duo Workflow service: ``` This exposes the AI Gateway metrics endpoint on port `8082` and the GitLab Duo Workflow Service metrics endpoint on port `8083`. +1. Restart the AI Gateway container + ### Configure Prometheus to scrape metrics To collect metrics from the AI Gateway and GitLab Duo Workflow service, add the following -- GitLab