Templating Variables

Introduction

Templating variables are a powerful feature in Grafana that allow you to create dynamic dashboards. Instead of hardcoding values like server names, time ranges, or application components in your queries, you can use variables that change based on user input. When working with Grafana Loki, templating variables enable you to create flexible log exploration interfaces that can adapt to different scenarios without modifying the dashboard itself.

In this guide, you'll learn how to create and use templating variables specifically for Loki data sources, understand how they can make your dashboards more powerful, and see practical examples of their application.

Understanding Templating Variables in Grafana

Templating variables act as placeholders in your dashboard that can be filled with different values. They appear as dropdown menus at the top of your dashboard, allowing you to select different options and have your queries automatically update.

Types of Variables in Grafana

Grafana offers several types of variables that you can use with Loki:

• Query variables: Populate options by running a Loki query
• Custom variables: Define a list of key-value pairs manually
• Text box variables: Allow free-form text input
• Interval variables: For time ranges (like 1h, 6h, 1d)
• Data source variables: Switch between different data sources
• Constant variables: Hidden variables that can be referenced in queries

Creating Loki Variables

Let's create a basic variable for a Loki data source to understand how it works:

Example: Creating a Label-Value Variable

In Loki, one common use case is to create a variable that lists all values for a specific label. Here's how to do it:

1. Navigate to your dashboard settings
2. Select "Variables" from the left menu
3. Click "Add variable"
4. Configure the variable:

Name: app
Label: Application
Type: Query
Data source: Loki
Query: label_values(app)

This will create a dropdown that shows all values of the app label found in your logs.

Using Variables in Loki Queries

Once you've created variables, you can use them in your LogQL queries. Variables are referenced using the syntax $variableName or ${variableName} (the latter is required when the variable is part of a larger string).

Basic Variable Usage

Here's a basic Loki query using our app variable:

{app="$app"}

When a user selects "frontend" from the dropdown, the query becomes:

{app="frontend"}

Multi-Value Selection

Grafana variables can also allow selecting multiple values. When multiple values are selected, Grafana generates a regex pattern. For example, if the user selects both "frontend" and "backend", the query becomes:

{app=~"frontend|backend"}

Notice the =~ operator which means "matches regex" in LogQL.

Advanced Templating Techniques for Loki

Chained Variables

You can create dependencies between variables. For example, let's create a service variable that depends on the app selection:

Name: service
Label: Service
Type: Query
Data source: Loki
Query: label_values({app="$app"}, service)

This variable will show only services related to the selected application.

Using Variables in Labels and Expressions

Variables can be used in both label selectors and in log processing expressions:

{app="$app"} | json | response_time > $threshold

If $threshold is set to "100", the query will filter for logs with response times greater than 100.

Practical Example: Building a Dynamic Log Explorer

Let's build a complete example of a dynamic log explorer for a microservice architecture:

1. First, create an environment variable:

Name: environment
Label: Environment
Type: Custom
Options: production, staging, development

2. Create an application variable that depends on the environment:

Name: application
Label: Application
Type: Query
Data source: Loki
Query: label_values({environment="$environment"}, app)

3. Create a service variable that depends on both:

Name: service
Label: Service
Type: Query
Data source: Loki
Query: label_values({environment="$environment", app="$application"}, service)

4. Create a panel with this Loki query:

{environment="$environment", app="$application", service="$service"} 
| json 
| line_format "{{.timestamp}} {{.level}} {{.message}}"

With this setup, users can:

1. Select an environment
2. See only applications in that environment
3. Then see only services for the selected application
4. View formatted logs specifically for their selection

Creating Template Variables Dynamically with LogQL

Loki's LogQL can be leveraged to create more sophisticated variable options. Here are some useful patterns:

Finding Unique Error Types

Name: errorTypes
Label: Error Types
Type: Query
Data source: Loki
Query: label_values(count_over_time({level="error"}[24h]) > 0, error_type)

This shows only error types that have appeared in the last 24 hours.

Time-Based Variables

You can create a variable that represents different time spans:

Name: timeSpan
Label: Time Span
Type: Custom
Options: 5m, 15m, 30m, 1h, 3h, 6h, 12h, 24h

Then use it in your queries:

{app="$app"} | logfmt | rate[${timeSpan}]

Using Variables in Dashboard Links

Variables can also be used in dashboard links to maintain context when moving between dashboards:

Dashboard: Application Detail
Link URL: /d/app-details?var-app=${app}&var-environment=${environment}

Common Pitfalls and Best Practices

Performance Considerations

• Avoid expensive variables: Some variable queries might be expensive to compute. Keep them efficient.
• Use caching: Enable variable caching in Grafana settings for better performance.
• Limit the number of options: Try to keep the number of variable options reasonable.

Syntax Issues

• Variable escaping: Use ${var} syntax when the variable is part of a larger string.
• Regex-sensitive characters: Be careful with special regex characters if using regex selection mode.

Variable Refresh

Variables can be set to refresh:

• On dashboard load: Default behavior
• On time range change: Updates when the dashboard time range changes
• Never: Only manual refresh

Choose the appropriate setting based on how frequently the variable options might change.

Summary

Templating variables transform static Loki dashboards into dynamic, interactive tools for log exploration. By leveraging these variables, you can:

• Create flexible dashboards that adapt to different environments and services
• Allow users to drill down into specific components without creating multiple dashboards
• Build sophisticated filtering mechanisms based on log content
• Improve dashboard performance by focusing queries on relevant data

As you build more complex log analytics solutions with Grafana and Loki, mastering templating variables will significantly enhance your ability to create useful, user-friendly monitoring tools.

Additional Resources

• Experiment with different variable types and refresh options
• Try building a dashboard that uses chained variables to narrow down from environment to specific log lines
• Create a dashboard that uses template variables to switch between different log visualization methods