Connecting to Prometheus Data Sources¶

Grafana provides native support for querying Prometheus servers, enabling you to visualize metrics from local, remote write, or federated Prometheus instances. This section guides you through configuring Grafana to connect to a Prometheus data source.

Adding a Prometheus Data Source¶

To add a Prometheus data source in Grafana, follow these steps:

Navigate to Data Source Settings
In the Grafana UI, go to Configuration > Data Sources and click Add data source.
Select Prometheus
Choose Prometheus from the list of available data sources.
Configure Connection Settings
Enter the following parameters:
URL: The Prometheus server endpoint (e.g., http://localhost:9090 for a local instance or https://prometheus.example.com for a remote server).
Access: Choose between Server (default), Proxy, or Browser. Use Server for direct queries, Proxy for Grafana to act as an intermediary, or Browser for browser-based access (less common).
HTTP Method: Select GET or POST based on your Prometheus setup (typically GET for most use cases).

Authentication: Configure basic auth, API keys, or TLS certificates if required. For example:

[datasources]
my-prometheus = {
    type = prometheus
    url = http://localhost:9090
    access = server
    http_method = GET
    isDefault = true
}

Test the Connection
Click Save & Test to verify the connection. Grafana will attempt to query a sample metric (e.g., up{job="prometheus"}).

Advanced Configuration Options¶

1. Remote Write and Federated Queries¶

For remote write setups, ensure the Prometheus URL points to the remote write endpoint (e.g., http://remote-write-server:9009). Federated queries can be configured using the federate endpoint, with the URL formatted as http://prometheus-federate:9090/api/v1/federate.

2. TLS/SSL Configuration¶

Enable TLS by specifying the CA certificate path or using a trusted certificate bundle:

[datasources]
my-prometheus = {
    type = prometheus
    url = https://secure-prometheus.example.com
    tls_server_name = prometheus.example.com
    tls_skip_verify = false
    tls_ca_cert = /path/to/ca.crt
}

3. Query Timeout and Max Samples¶

Adjust query timeouts and sample limits in the Grafana configuration file (grafana.ini):

[prometheus]
query_timeout = 30s
max_samples = 100000

Example: Querying Prometheus Metrics¶

Once connected, use PromQL to query metrics. For example:

rate(http_requests_total{job="api-server"}[5m])

This query calculates the rate of HTTP requests over the last 5 minutes for the api-server job.

Best Practices¶

Use Proxy Mode for Security: Avoid exposing Prometheus directly to the internet by using Grafana as a proxy.
Monitor Query Performance: Track slow queries in Prometheus using the query_duration_seconds metric.
Secure Authentication: Always use HTTPS and disable insecure HTTP methods unless necessary.
Version Compatibility: Ensure Grafana and Prometheus versions are compatible (e.g., Grafana 10.x supports Prometheus 2.38+).

Key takeaways¶

Configure Prometheus data sources via Grafana's UI or grafana.ini file.
Use Server access for direct queries, Proxy for security, and Browser for limited use cases.
Enable TLS and authentication to secure remote connections.
Test connections with sample metrics and validate query performance.
Leverage PromQL for advanced metric analysis and visualization.