From 1bfea66c5c0107c2c1db5603181b9ab580e4efdc Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Tue, 24 Jan 2017 21:25:48 +0000 Subject: [PATCH 1/4] Update Prometheus documentation to caveat SSL/HSTS limitation, and note that they are not authenticated services. --- doc/administration/monitoring/performance/prometheus.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/doc/administration/monitoring/performance/prometheus.md b/doc/administration/monitoring/performance/prometheus.md index 51c63325064c..75b176e78492 100644 --- a/doc/administration/monitoring/performance/prometheus.md +++ b/doc/administration/monitoring/performance/prometheus.md @@ -6,6 +6,7 @@ them yourself. Over subsequent releases additional GitLab metrics will be captured. - Prometheus services are off by default but will be on starting with GitLab 9.0. +- Prometheus and it's exporters do not authenticate users, and will be available to anyone who can access them. [Prometheus] is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products. @@ -21,8 +22,9 @@ connect directly to Prometheus or utilize a dashboard tool like [Grafana]. ## Configuring Prometheus >**Note:** -Available since Omnibus GitLab 8.16. For installations from source you'll +- Available since Omnibus GitLab 8.16. For installations from source you'll have to install and configure it yourself. +- Prometheus and it's exporters do not authenticate users, and will be available for anyone who can access them. To enable Prometheus: @@ -46,6 +48,9 @@ Prometheus. After you have [enabled Prometheus](#configuring-prometheus), you can visit `:9090` for the dashboard that Prometheus offers by default. +>**Note:** +Currently if SSL has been enabled, you will not be able to access Prometheus on the same hostname as GitLab due to HSTS. Accessing via another method with a different hostname like SSH should be possible. + The performance data collected by Prometheus can be viewed directly in the Prometheus console or through a compatible dashboard tool. The Prometheus interface provides a [flexible query language][prom-query] to work -- GitLab From 32fbacbebbd6049902a8159e26b9053d22da9560 Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Tue, 24 Jan 2017 22:12:28 +0000 Subject: [PATCH 2/4] More clearly state HSTS/SSL configuration is not currently supported, and it will be in a future release. --- doc/administration/monitoring/performance/prometheus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/administration/monitoring/performance/prometheus.md b/doc/administration/monitoring/performance/prometheus.md index 75b176e78492..b4d7ca62990b 100644 --- a/doc/administration/monitoring/performance/prometheus.md +++ b/doc/administration/monitoring/performance/prometheus.md @@ -49,7 +49,7 @@ After you have [enabled Prometheus](#configuring-prometheus), you can visit `:9090` for the dashboard that Prometheus offers by default. >**Note:** -Currently if SSL has been enabled, you will not be able to access Prometheus on the same hostname as GitLab due to HSTS. Accessing via another method with a different hostname like SSH should be possible. +Currently if SSL has been enabled, you will not be able to access Prometheus on the same hostname as GitLab due to HSTS. This configuration will be supported in a future release. The performance data collected by Prometheus can be viewed directly in the Prometheus console or through a compatible dashboard tool. -- GitLab From 876ee8c33bfa462e6e29a3651f53f4d56b887208 Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Fri, 27 Jan 2017 01:21:12 +0000 Subject: [PATCH 3/4] Update prometheus.md --- .../monitoring/performance/prometheus.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/doc/administration/monitoring/performance/prometheus.md b/doc/administration/monitoring/performance/prometheus.md index b4d7ca62990b..fa3a0fd0f1a5 100644 --- a/doc/administration/monitoring/performance/prometheus.md +++ b/doc/administration/monitoring/performance/prometheus.md @@ -24,7 +24,6 @@ connect directly to Prometheus or utilize a dashboard tool like [Grafana]. >**Note:** - Available since Omnibus GitLab 8.16. For installations from source you'll have to install and configure it yourself. -- Prometheus and it's exporters do not authenticate users, and will be available for anyone who can access them. To enable Prometheus: @@ -39,17 +38,17 @@ To enable Prometheus: take effect By default, Prometheus will run as the `gitlab-prometheus` user and listen on -TCP port `9090` under localhost. If the [node exporter](#node-exporter) service +TCP port `9090`. If the [node exporter](#node-exporter) service has been enabled, it will automatically be set up as a monitoring target for Prometheus. ## Viewing Performance Metrics After you have [enabled Prometheus](#configuring-prometheus), you can visit -`:9090` for the dashboard that Prometheus offers by default. +`http://:9090` for the dashboard that Prometheus offers by default. >**Note:** -Currently if SSL has been enabled, you will not be able to access Prometheus on the same hostname as GitLab due to HSTS. This configuration will be supported in a future release. +If SSL has been enabled, you will not be able to access Prometheus on the same hostname as GitLab due to [HSTS](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security). We recommend setting up a new DNS entry for Prometheus to access the console, for example `http://prometheus.example.com:9090/`. The performance data collected by Prometheus can be viewed directly in the Prometheus console or through a compatible dashboard tool. @@ -58,6 +57,12 @@ with the collected data where you can visualize their output. For a more fully featured dashboard, Grafana can be used and has [official support for Prometheus][prom-grafana]. +Sample Prometheus Queries: +* % Memory Used: `(1 - ((node_memory_MemFree + node_memory_Cached) / node_memory_MemTotal)) * 100` +* % CPU Load: `1 - rate(node_cpu{mode="idle"}[5m])` +* Data Transmitted: `irate(node_network_transmit_bytes[5m])` +* Data Received: `irate(node_network_receive_bytes[5m])` + ## Prometheus exporters There are a number of libraries and servers which help in exporting existing -- GitLab From 8214640317dcbbd6fb91018301c97c283ddcf202 Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Mon, 6 Feb 2017 17:30:11 +0000 Subject: [PATCH 4/4] Update Prometheus documentation to clarify HSTS and Localhost access. --- .../monitoring/performance/prometheus.md | 25 ++++++++----------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/doc/administration/monitoring/performance/prometheus.md b/doc/administration/monitoring/performance/prometheus.md index fa3a0fd0f1a5..91c0dd32c1fe 100644 --- a/doc/administration/monitoring/performance/prometheus.md +++ b/doc/administration/monitoring/performance/prometheus.md @@ -1,12 +1,12 @@ # GitLab Prometheus >**Notes:** -- Prometheus and the node exporter are bundled in the Omnibus GitLab package +- Prometheus and Node Exporter have been bundled in the Omnibus GitLab package since GitLab 8.16. For installations from source you will have to install them yourself. Over subsequent releases additional GitLab metrics will be captured. -- Prometheus services are off by default but will be on starting with GitLab 9.0. -- Prometheus and it's exporters do not authenticate users, and will be available to anyone who can access them. +- Prometheus and its exporters are off by default but will be on starting with GitLab 9.0. +- Prometheus and its exporters do not authenticate users, and will be available to anyone who can access them. [Prometheus] is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products. @@ -21,10 +21,6 @@ connect directly to Prometheus or utilize a dashboard tool like [Grafana]. ## Configuring Prometheus ->**Note:** -- Available since Omnibus GitLab 8.16. For installations from source you'll -have to install and configure it yourself. - To enable Prometheus: 1. Edit `/etc/gitlab/gitlab.rb` @@ -38,17 +34,17 @@ To enable Prometheus: take effect By default, Prometheus will run as the `gitlab-prometheus` user and listen on -TCP port `9090`. If the [node exporter](#node-exporter) service +`http://localhost:9090`. If the [node exporter](#node-exporter) service has been enabled, it will automatically be set up as a monitoring target for Prometheus. ## Viewing Performance Metrics After you have [enabled Prometheus](#configuring-prometheus), you can visit -`http://:9090` for the dashboard that Prometheus offers by default. +`http://localhost:9090` for the dashboard that Prometheus offers by default. >**Note:** -If SSL has been enabled, you will not be able to access Prometheus on the same hostname as GitLab due to [HSTS](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security). We recommend setting up a new DNS entry for Prometheus to access the console, for example `http://prometheus.example.com:9090/`. +If SSL has been enabled, you may not be able to access Prometheus on the same browser as GitLab due to [HSTS][hsts]. We plan to [provide access via GitLab][multi-user-prometheus], but in the interim there are some workarounds: using a separate browser for Prometheus, resetting HSTS, or having [nginx proxy it][nginx-custom-config]. The performance data collected by Prometheus can be viewed directly in the Prometheus console or through a compatible dashboard tool. @@ -77,10 +73,6 @@ making it easy to configure and use. ### Node exporter ->**Note:** -Available since Omnibus GitLab 8.16. For installations from source you'll -have to install and configure it yourself. - The [node exporter] allows you to measure various machine resources such as memory, disk and CPU utilization. @@ -98,7 +90,7 @@ To enable the node exporter: take effect Prometheus it will now automatically begin collecting performance data from -the node exporter. You can visit `:9100/metrics` for a real +the node exporter. You can visit `http://localhost:9100/metrics` for a real time representation of the metrics that are collected. Refresh the page and you will see the data change. @@ -110,3 +102,6 @@ you will see the data change. [scrape-config]: https://prometheus.io/docs/operating/configuration/#%3Cscrape_config%3E [prom-exporters]: https://prometheus.io/docs/instrumenting/exporters/ [reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure +[hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security +[multi-user-prometheus]: https://gitlab.com/gitlab-org/multi-user-prometheus +[nginx-custom-config]: https://docs.gitlab.com/omnibus/settings/configuration.html#inserting-custom-nginx-settings-into-the-gitlab-server-block \ No newline at end of file -- GitLab