Skip to main content

Grafana Dashboard Creation

Introduction​

Grafana is a powerful open-source visualization tool that pairs perfectly with Prometheus to create interactive, dynamic dashboards for your metrics. While Prometheus provides the data collection and storage, Grafana transforms this data into visually appealing, informative dashboards that make monitoring your systems intuitive and effective.

In this guide, you'll learn how to create Grafana dashboards for visualizing Prometheus metrics. We'll cover everything from setting up your first dashboard to creating complex panels and organizing them effectively.

Prerequisites​

Before starting with Grafana dashboard creation, ensure you have:

  • A running Prometheus server collecting metrics
  • Grafana installed and configured
  • Prometheus configured as a data source in Grafana

If you haven't set these up yet, please refer to our previous sections on [Prometheus Installation] and [Grafana Setup].

Basic Dashboard Creation​

Accessing the Grafana Interface​

First, let's access the Grafana web interface:

  1. Open your web browser and navigate to your Grafana instance (typically http://localhost:3000 if running locally)
  2. Log in with your credentials (default: admin/admin)
  3. You'll be directed to the Grafana home dashboard

Creating Your First Dashboard​

Let's create a new dashboard to visualize Prometheus metrics:

  1. Click the "+" icon in the left sidebar
  2. Select "Dashboard" from the dropdown menu
  3. Click "Add new panel"

At this point, you'll see the panel edit view where you can configure your visualization.

Adding Your First Panel​

Panels are the building blocks of Grafana dashboards. Here's how to create your first panel:

text
1. In the query editor, select "Prometheus" as your data source
2. Enter a PromQL query in the query field, for example:
   rate(node_cpu_seconds_total{mode="user"}[1m])
3. Select the visualization type from the right side (Graph, Gauge, etc.)
4. Click "Apply" to add the panel to your dashboard
5. Click the disk icon or press Ctrl+S to save your dashboard

Here's an example of a simple CPU usage panel configuration:

text
Data Source: Prometheus
Query: sum by (instance) (rate(node_cpu_seconds_total{mode!="idle"}[1m])) / count by (instance) (node_cpu_seconds_total{mode="system"})
Legend: {{instance}} - CPU Usage
Visualization: Time series

This query shows the CPU usage percentage across all your monitored instances.

Understanding Panel Types​

Grafana offers various visualization types for different use cases:

Time Series​

The most common visualization type, perfect for showing how metrics change over time.

text
Example Query: rate(http_requests_total[5m])
Use Case: Tracking request rates, CPU usage, memory consumption over time

Gauge​

Ideal for showing a single value within a range, like percentage of disk used.

text
Example Query: (node_filesystem_avail_bytes / node_filesystem_size_bytes) * 100
Use Case: Disk space usage, CPU utilization percentage

Stat​

Shows a single big number with optional sparkline, perfect for key performance indicators.

text
Example Query: sum(up)
Use Case: Number of online instances, total error count

Bar Gauge​

Displays values as a gradient bar, useful for comparing multiple values.

text
Example Query: sum by(job) (up)
Use Case: Service health across multiple jobs

Table​

Shows raw data in tabular format.

text
Example Query: topk(10, http_requests_total)
Use Case: Top 10 endpoints by request count

Advanced Dashboard Features​

Now let's explore some more advanced features to enhance your dashboards.

Variables and Templates​

Variables make your dashboards dynamic and interactive:

  1. Click the dashboard settings icon (gear) in the top right
  2. Select "Variables" from the left menu
  3. Click "Add variable"

Here's an example of setting up an instance variable:

text
Name: instance
Label: Instance
Type: Query
Data source: Prometheus
Query: label_values(node_exporter_build_info, instance)

You can now use this variable in your queries with $instance:

text
node_memory_MemAvailable_bytes{instance="$instance"} / node_memory_MemTotal_bytes{instance="$instance"} * 100

This allows users to switch between different instances from a dropdown menu.

Dashboard Annotations​

Annotations allow you to mark points in time on your graphs:

  1. Go to Dashboard settings > Annotations
  2. Click "Add Annotation Query"
  3. Configure a Prometheus query that returns events

Example annotation query:

text
changes(process_start_time_seconds{job="prometheus"}[1m]) > 0

This will mark points in time when Prometheus was restarted.

Organizing Panels with Rows​

For complex dashboards, organize panels into collapsible rows:

  1. Hover over the dashboard
  2. Click "Add panel" (+ icon)
  3. Select "Add a new row"
  4. Drag panels into the row
  5. Click the arrow on the row header to collapse/expand

Dashboard Time Range Controls​

Grafana provides powerful time range controls:

  • Use the time picker in the top right to select predefined ranges
  • Drag on a graph to zoom into a specific time period
  • Hold Shift and drag to pan through time

Real-World Example: Application Monitoring Dashboard​

Let's create a practical dashboard for monitoring a web application:

  1. Create a new dashboard
  2. Add a row titled "Application Health"
  3. Add these panels:

Request Rate Panel:

text
Query: sum(rate(http_requests_total[5m])) by (route)
Visualization: Time series
Title: HTTP Request Rate

Error Rate Panel:

text
Query: sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) * 100
Visualization: Time series
Title: Error Rate %
Thresholds: 1=yellow, 5=red

Response Time Panel:

text
Query: histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, route))
Visualization: Time series
Title: 95th Percentile Response Time

Active Users Panel:

text
Query: sum(active_users)
Visualization: Stat
Title: Active Users
  1. Add another row titled "System Resources"
  2. Add system resource panels (CPU, Memory, Disk, Network)

This creates a comprehensive dashboard that monitors both application metrics and system resources.

Creating Multi-Service Dashboards​

For monitoring multiple services:

  1. Set up variables for filtering by service:
text
Name: service
Type: Query
Query: label_values(up, job)
  1. Create panels using the variable:
text
Query: sum(rate(http_requests_total{job="$service"}[5m])) by (status)
Title: $service Request Rate by Status
  1. Duplicate panels for different metrics but the same services

This approach lets you monitor all your services from a single dashboard.

Sharing and Exporting Dashboards​

Grafana dashboards can be easily shared and reused:

  1. To export: Dashboard settings > JSON Model > Copy to clipboard
  2. To import: + > Import > Paste JSON
  3. To share: Click "Share" in the top menu for link or snapshot options

Best Practices for Dashboard Design​

When creating Grafana dashboards, follow these best practices:

  1. Keep it simple: Start with essential metrics and add complexity as needed
  2. Use consistent naming: Maintain a naming convention for panels and dashboards
  3. Group related metrics: Use rows to organize panels by service or function
  4. Set appropriate thresholds: Configure alerts and color thresholds to highlight issues
  5. Add documentation: Include text panels explaining dashboard purpose and panel meanings
  6. Use templates: Make dashboards reusable with variables instead of hardcoded values
  7. Consider performance: Too many complex queries can slow down dashboard loading

Troubleshooting Common Issues​

No Data Displayed​

If your panel shows "No data":

  1. Check that Prometheus is running and accessible
  2. Verify your PromQL query in Prometheus' web UI
  3. Check time range settings in Grafana
  4. Ensure metric names and label filters are correct

Dashboard is Slow​

If your dashboard loads slowly:

  1. Reduce the number of panels or queries
  2. Optimize complex PromQL queries
  3. Increase your browser's memory limits
  4. Consider using recording rules in Prometheus for complex queries

Summary​

In this guide, you've learned how to:

  • Create basic Grafana dashboards for Prometheus metrics
  • Work with different panel types for various visualization needs
  • Use advanced features like variables, annotations, and rows
  • Build practical, real-world monitoring dashboards
  • Apply best practices for dashboard design

Grafana dashboards are powerful tools for visualizing your Prometheus metrics. With the knowledge from this guide, you can create effective, informative dashboards that help you monitor your systems and applications with ease.

Additional Resources​

To continue learning about Grafana dashboards:

Exercises​

  1. Create a dashboard showing system metrics (CPU, memory, disk, network) for your servers
  2. Build a dashboard for monitoring a specific application with request rates, errors, and latencies
  3. Create a variable that allows filtering by environment (prod, staging, dev)
  4. Set up annotations to mark deployment events on your dashboards
  5. Export your dashboard and share it with a colleague

Remember, the best way to learn is by doing. Start building your own dashboards, experiment with different visualizations, and discover what works best for your specific monitoring needs.


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)