Kotlin Monitoring

Introduction

Monitoring is a critical aspect of maintaining a healthy and reliable backend application. For Kotlin backends, proper monitoring allows you to track performance metrics, identify bottlenecks, detect errors, and ensure your application meets service level objectives (SLOs). In this guide, we'll explore how to implement monitoring in Kotlin backend applications, from basic logging to advanced observability solutions.

Why Monitoring Matters

Before diving into the technical details, let's understand why monitoring is essential:

• Early Problem Detection: Identify issues before they affect your users
• Performance Optimization: Discover and eliminate bottlenecks
• Resource Planning: Make informed decisions about scaling
• Business Intelligence: Gain insights into user behavior and system usage
• Security: Detect unusual patterns that might indicate security breaches

Basic Monitoring with Logging

Setting Up a Logger

The simplest form of monitoring begins with proper logging. Kotlin applications typically use SLF4J with an implementation like Logback or Log4j2.

First, add the required dependencies to your build.gradle.kts:

kotlin

dependencies {
    implementation("org.slf4j:slf4j-api:1.7.36")
    implementation("ch.qos.logback:logback-classic:1.2.11")
}

Create a basic Logback configuration file (src/main/resources/logback.xml):

xml

<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>logs/application.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/application.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    
    <root level="INFO">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="FILE" />
    </root>
</configuration>

Now, you can use the logger in your Kotlin code:

kotlin

import org.slf4j.LoggerFactory

class UserService {
    private val logger = LoggerFactory.getLogger(UserService::class.java)
    
    fun createUser(username: String): User {
        logger.info("Creating new user: $username")
        try {
            // User creation logic
            val user = User(id = generateId(), username = username)
            logger.info("Successfully created user: ${user.id}")
            return user
        } catch (e: Exception) {
            logger.error("Failed to create user: $username", e)
            throw e
        }
    }
}

Structured Logging

For more advanced analysis, structured logging in JSON format can be useful:

xml

<configuration>
    <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
    </appender>
    
    <root level="INFO">
        <appender-ref ref="JSON" />
    </root>
</configuration>

Remember to add the Logstash encoder dependency:

kotlin

implementation("net.logstash.logback:logstash-logback-encoder:7.2")

Metrics Collection with Micrometer

Micrometer provides a simple facade for the most popular monitoring systems, allowing you to instrument your code with dimensional metrics.

Add Micrometer to your project:

kotlin

dependencies {
    implementation("io.micrometer:micrometer-registry-prometheus:1.9.2")
}

Here's how to use Micrometer to track application metrics:

kotlin

import io.micrometer.core.instrument.MeterRegistry
import io.micrometer.core.instrument.Timer
import org.springframework.stereotype.Service

@Service
class OrderService(private val registry: MeterRegistry) {
    private val orderProcessingTimer = registry.timer("order.processing")
    
    fun processOrder(order: Order) {
        // Record how long order processing takes
        Timer.Sample.start(registry).use { sample ->
            try {
                // Process the order
                // ...
                
                // Increment a counter for successful orders
                registry.counter("orders.success").increment()
                
                sample.stop(orderProcessingTimer)
            } catch (e: Exception) {
                // Increment a counter for failed orders
                registry.counter("orders.failure").increment()
                throw e
            }
        }
    }
    
    fun trackInventoryLevel(productId: String, quantity: Int) {
        // Report current inventory level as a gauge
        registry.gauge("inventory.quantity", listOf(Tag.of("product", productId)), quantity)
    }
}

Integrating with Prometheus and Grafana

Prometheus is a popular open-source monitoring system that works well with Kotlin applications. Grafana provides visualization for Prometheus metrics.

Setting up Prometheus with Spring Boot

If you're using Spring Boot, add the Actuator and Prometheus dependencies:

kotlin

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-actuator")
    implementation("io.micrometer:micrometer-registry-prometheus")
}

Configure your application.properties or application.yml file:

yaml

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,prometheus
  metrics:
    export:
      prometheus:
        enabled: true

This exposes a /actuator/prometheus endpoint that Prometheus can scrape.

Basic Prometheus Configuration

Create a prometheus.yml file:

yaml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'kotlin-app'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['localhost:8080']

Visualizing with Grafana

Once Prometheus is collecting your metrics, you can set up Grafana dashboards to visualize them. Here's a sample JSON for a basic dashboard:

json

{
  "annotations": {
    "list": []
  },
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "id": 1,
  "links": [],
  "panels": [
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "Prometheus",
      "fill": 1,
      "fillGradient": 0,
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "hiddenSeries": false,
      "id": 2,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "null",
      "options": {
        "dataLinks": []
      },
      "percentage": false,
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "order_processing_seconds_count",
          "legendFormat": "Order Processing Count",
          "refId": "A"
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeRegions": [],
      "timeShift": null,
      "title": "Order Processing",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    }
  ],
  "schemaVersion": 22,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "Kotlin App Dashboard",
  "uid": "kotlin-app",
  "version": 1
}

Health Checks with Spring Boot Actuator

For basic application health monitoring, Spring Boot Actuator provides a /actuator/health endpoint:

kotlin

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-actuator")
}

You can customize health checks by implementing your own health indicators:

kotlin

import org.springframework.boot.actuate.health.Health
import org.springframework.boot.actuate.health.HealthIndicator
import org.springframework.stereotype.Component

@Component
class DatabaseHealthIndicator(private val dataSource: DataSource) : HealthIndicator {
    override fun health(): Health {
        return try {
            val connection = dataSource.connection
            connection.use {
                if (it.isValid(1000)) {
                    Health.up()
                        .withDetail("database", "PostgreSQL")
                        .withDetail("status", "Connected")
                        .build()
                } else {
                    Health.down()
                        .withDetail("error", "Failed to validate database connection")
                        .build()
                }
            }
        } catch (e: Exception) {
            Health.down()
                .withDetail("error", e.message)
                .build()
        }
    }
}

Distributed Tracing with OpenTelemetry

For microservices architectures, distributed tracing helps you understand the flow of requests across multiple services.

Add OpenTelemetry dependencies:

kotlin

dependencies {
    implementation("io.opentelemetry:opentelemetry-api:1.15.0")
    implementation("io.opentelemetry:opentelemetry-sdk:1.15.0")
    implementation("io.opentelemetry:opentelemetry-exporter-jaeger:1.15.0")
}

Configure OpenTelemetry in your application:

kotlin

import io.opentelemetry.api.GlobalOpenTelemetry
import io.opentelemetry.api.trace.Tracer
import io.opentelemetry.exporter.jaeger.JaegerGrpcSpanExporter
import io.opentelemetry.sdk.OpenTelemetrySdk
import io.opentelemetry.sdk.trace.SdkTracerProvider
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor

fun initTracing(): Tracer {
    val jaegerExporter = JaegerGrpcSpanExporter.builder()
        .setEndpoint("http://localhost:14250")
        .build()
    
    val tracerProvider = SdkTracerProvider.builder()
        .addSpanProcessor(SimpleSpanProcessor.create(jaegerExporter))
        .build()
    
    val openTelemetry = OpenTelemetrySdk.builder()
        .setTracerProvider(tracerProvider)
        .build()
    
    GlobalOpenTelemetry.set(openTelemetry)
    
    return openTelemetry.getTracer("my-kotlin-app")
}

Use the tracer in your code:

kotlin

class OrderProcessingService(private val tracer: Tracer) {
    fun processOrder(order: Order) {
        tracer.spanBuilder("processOrder").startSpan().use { span ->
            // Add contextual information to the span
            span.setAttribute("order.id", order.id)
            span.setAttribute("customer.id", order.customerId)
            
            try {
                // Process the order
                validateOrder(order)
                checkInventory(order)
                processPayment(order)
                
                span.setAttribute("status", "success")
            } catch (e: Exception) {
                span.recordException(e)
                span.setAttribute("status", "failed")
                throw e
            }
        }
    }
    
    private fun validateOrder(order: Order) {
        tracer.spanBuilder("validateOrder").startSpan().use { span ->
            // Validation logic
        }
    }
    
    private fun checkInventory(order: Order) {
        tracer.spanBuilder("checkInventory").startSpan().use { span ->
            // Inventory check logic
        }
    }
    
    private fun processPayment(order: Order) {
        tracer.spanBuilder("processPayment").startSpan().use { span ->
            // Payment processing logic
        }
    }
}

Real-World Example: Monitoring a Kotlin Web Service

Let's put everything together in a comprehensive example of monitoring a Kotlin web service built with Spring Boot:

kotlin

import io.micrometer.core.instrument.MeterRegistry
import io.micrometer.core.instrument.Timer
import org.slf4j.LoggerFactory
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.web.bind.annotation.*
import java.util.concurrent.atomic.AtomicInteger

@SpringBootApplication
class MonitoredAppApplication

fun main(args: Array<String>) {
    runApplication<MonitoredAppApplication>(*args)
}

data class Product(val id: String, val name: String, val price: Double)

@RestController
@RequestMapping("/api/products")
class ProductController(private val productService: ProductService) {
    private val logger = LoggerFactory.getLogger(ProductController::class.java)
    
    @GetMapping
    fun getAllProducts(): List<Product> {
        logger.info("Fetching all products")
        return productService.getAllProducts()
    }
    
    @GetMapping("/{id}")
    fun getProduct(@PathVariable id: String): Product {
        logger.info("Fetching product with ID: $id")
        return productService.getProduct(id)
    }
    
    @PostMapping
    fun createProduct(@RequestBody product: Product): Product {
        logger.info("Creating new product: ${product.name}")
        return productService.createProduct(product)
    }
}

@Service
class ProductService(private val meterRegistry: MeterRegistry) {
    private val logger = LoggerFactory.getLogger(ProductService::class.java)
    private val products = mutableMapOf<String, Product>()
    private val productCount = AtomicInteger(0)
    
    // Register gauges
    init {
        meterRegistry.gauge("products.count", productCount)
    }
    
    fun getAllProducts(): List<Product> {
        val timer = meterRegistry.timer("product.service.get.all")
        return timer.record<List<Product>> { 
            logger.debug("Returning ${products.size} products")
            products.values.toList()
        }
    }
    
    fun getProduct(id: String): Product {
        val timer = meterRegistry.timer("product.service.get.one", "productId", id)
        return timer.record<Product> {
            logger.debug("Looking up product with ID: $id")
            products[id] ?: throw ProductNotFoundException(id)
        }
    }
    
    fun createProduct(product: Product): Product {
        val timer = meterRegistry.timer("product.service.create")
        return timer.record<Product> {
            logger.info("Creating product: ${product.name}")
            products[product.id] = product
            productCount.incrementAndGet()
            meterRegistry.counter("product.created").increment()
            product
        }
    }
}

class ProductNotFoundException(id: String) : RuntimeException("Product not found: $id")

@RestControllerAdvice
class GlobalExceptionHandler(private val meterRegistry: MeterRegistry) {
    private val logger = LoggerFactory.getLogger(GlobalExceptionHandler::class.java)
    
    @ExceptionHandler(ProductNotFoundException::class)
    @ResponseStatus(HttpStatus.NOT_FOUND)
    fun handleProductNotFound(ex: ProductNotFoundException): Map<String, String> {
        logger.warn("Product not found: ${ex.message}")
        meterRegistry.counter("errors", "type", "product_not_found").increment()
        return mapOf("error" to ex.message!!)
    }
    
    @ExceptionHandler(Exception::class)
    @ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR)
    fun handleGenericError(ex: Exception): Map<String, String> {
        logger.error("Unexpected error", ex)
        meterRegistry.counter("errors", "type", "server_error").increment()
        return mapOf("error" to "An unexpected error occurred")
    }
}

Alerts and Notifications

Once you have monitoring in place, you'll want to set up alerts to notify you when things go wrong. Most monitoring systems like Prometheus support alert rules.

Here's a sample alert rule for Prometheus that will trigger if request latency gets too high:

yaml

groups:
- name: example
  rules:
  - alert: HighRequestLatency
    expr: http_request_duration_seconds{path="/api/products"} > 0.5
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "High request latency on {{ $labels.instance }}"
      description: "Request latency is above 500ms (current value: {{ $value }}s)"

Summary

Effective monitoring is crucial for maintaining reliable Kotlin backend services. In this guide, we covered:

1. Basic Logging: Using SLF4J and Logback for application logs
2. Metrics Collection: Implementing Micrometer for gathering application metrics
3. Prometheus Integration: Exposing metrics for Prometheus to collect
4. Grafana Visualization: Creating dashboards to visualize application metrics
5. Health Checks: Implementing custom health indicators with Spring Boot Actuator
6. Distributed Tracing: Using OpenTelemetry to trace requests across services
7. Alerting: Setting up notifications for when things go wrong

By implementing these monitoring strategies, you'll gain better visibility into your Kotlin backend applications and be able to respond quickly to issues.

Additional Resources

• Spring Boot Actuator Documentation
• Micrometer Documentation
• Prometheus Getting Started Guide
• Grafana Documentation
• OpenTelemetry for Java/Kotlin

Exercises

1. Set up basic logging in a simple Kotlin application and experiment with different log levels.
2. Implement Micrometer metrics in a Spring Boot application to track API request counts.
3. Configure Prometheus to scrape metrics from your application and create a simple Grafana dashboard.
4. Write a custom health indicator that checks the connectivity to an external service.
5. Implement distributed tracing in a multi-service application using OpenTelemetry.