Skip to main content

Binary Operations in LogQL Metrics

Introduction​

Binary operations in LogQL metrics allow you to combine, compare, and manipulate different metric values using mathematical operators. This powerful feature enables you to create complex calculations between metrics, derive new insights from your logging data, and build more sophisticated monitoring dashboards in Grafana Loki.

In this guide, we'll explore how binary operations work in LogQL, their syntax, common use cases, and practical examples to help you leverage these operations in your observability workflows.

What Are Binary Operations?​

Binary operations in LogQL are mathematical operations that take two operands (metrics) and produce a result. These operations follow the standard order of operations (PEMDAS - Parentheses, Exponents, Multiplication/Division, Addition/Subtraction).

Key Concepts​

  1. Operands: The metrics on which operations are performed
  2. Operators: Mathematical symbols that define the operation to perform
  3. Vector Matching: How metrics with different labels are combined
  4. Type Conversion: How LogQL handles different data types in operations

Supported Binary Operators​

LogQL supports several binary operators for metrics:

OperatorDescriptionExample
+Additionmetric_a + metric_b
-Subtractionmetric_a - metric_b
*Multiplicationmetric_a * metric_b
/Divisionmetric_a / metric_b
%Modulometric_a % metric_b
^Exponentiationmetric_a ^ metric_b
==Equalmetric_a == metric_b
!=Not equalmetric_a != metric_b
>Greater thanmetric_a > metric_b
<Less thanmetric_a < metric_b
>=Greater than or equalmetric_a >= metric_b
<=Less than or equalmetric_a <= metric_b

Basic Syntax​

The general syntax for binary operations in LogQL is:

<left_operand> <operator> <right_operand>

Where:

  • <left_operand> is a metric query or a literal value
  • <operator> is one of the supported binary operators
  • <right_operand> is another metric query or a literal value

Simple Binary Operations​

Let's start with some basic examples of binary operations in LogQL:

Example 1: Adding Two Metrics​

sum(rate({app="frontend"} |= "error" [5m])) + sum(rate({app="backend"} |= "error" [5m]))

This query adds the error rates from frontend and backend applications, giving you the total error rate across both systems.

Example 2: Calculating Percentage​

sum(rate({app="api"} |= "error" [5m])) / sum(rate({app="api"} [5m])) * 100

This calculates the percentage of error logs relative to all logs for the API application.

Vector Matching in Binary Operations​

When performing binary operations on metrics with labels, LogQL needs to know how to match the different time series together. This is called vector matching.

Types of Vector Matching​

  1. One-to-one matching (default): Requires that the label sets on both sides match exactly
  2. Many-to-one/one-to-many matching: One side can match with multiple entries on the other side
  3. Group matching: Defines specific labels to consider for matching

Example: One-to-One Matching​

sum by (method) (rate({app="api"} | json | unwrap duration_ms [5m])) / 
sum by (method) (count_over_time({app="api"} [5m]))

This calculates the average duration per HTTP method by dividing the sum of durations by the count of requests.

Using Binary Operations with Scalar Values​

You can also perform operations between metrics and scalar (constant) values:

sum(rate({app="payment"} |= "transaction" [5m])) * 60

This multiplies the transaction rate by 60 to convert from "per second" to "per minute".

Practical Use Cases​

Let's explore some real-world applications of binary operations in LogQL metrics:

Case 1: Error Budget Calculation​

1 - (sum(rate({app="api"} |= "error" [1h])) / sum(rate({app="api"} [1h])))

This calculates the error budget consumption by determining the ratio of successful requests to total requests.

Case 2: Comparing Service Performance​

sum by (service) (rate({env="prod"} | json | unwrap latency_ms [5m])) / 
sum by (service) (rate({env="baseline"} | json | unwrap latency_ms [5m]))

This compares the latency in production against a baseline for each service, showing which services have degraded performance.

Case 3: Resource Utilization Ratio​

sum(rate({app="database"} | json | unwrap memory_used [5m])) / 
sum(rate({app="database"} | json | unwrap memory_allocated [5m])) * 100

This calculates the percentage of allocated memory being utilized by database instances.

Advanced Binary Operations​

Using Multiple Operations​

You can chain multiple binary operations together in LogQL:

sum(rate({app="web"} |= "error" [5m])) / 
(sum(rate({app="web"} [5m])) * (1 - 0.05))

This complex expression calculates if errors exceed 5% of the expected traffic threshold.

Combining with Functions​

Binary operations can be combined with LogQL functions for more complex queries:

avg_over_time(
  (sum by (service) (rate({env="prod"} | json | unwrap response_time_ms [5m])) -
   sum by (service) (rate({env="prod"} | json | unwrap processing_time_ms [5m])))
  [1h]
)

This calculates the average network latency (response time minus processing time) over the last hour for each service.

Common Mistakes and Troubleshooting​

When working with binary operations in LogQL, be aware of these common issues:

  1. Label Matching Problems: Operations fail if labels don't match properly

    • Solution: Use by and without clauses to manage label sets

  2. Division by Zero:

    • Solution: Use conditional operators or add a small constant to denominators

  3. Type Mismatches:

    • Solution: Ensure operands are of compatible types

Summary​

Binary operations in LogQL metrics provide a powerful way to combine, compare, and derive insights from your log data in Grafana Loki. By mastering these operations, you can:

  • Create complex calculations between different metrics
  • Derive meaningful business KPIs from technical metrics
  • Build more sophisticated monitoring dashboards
  • Perform comparative analysis across different environments or time periods

As you continue your journey with Grafana Loki, binary operations will become an essential tool in your observability toolkit, enabling you to extract more value from your logging data.

Exercises​

To reinforce your understanding of binary operations in LogQL metrics, try these exercises:

  1. Calculate the error rate ratio between your production and staging environments
  2. Create a query that shows the percentage of CPU time spent on garbage collection
  3. Build a query to compare the 95th percentile response time today versus yesterday
  4. Develop an alert expression that triggers when errors exceed 2% of traffic and the total traffic is above a certain threshold

Additional Resources​


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)