Gin Monitoring

In production environments, tracking the health, performance, and behavior of your Gin applications is crucial. Effective monitoring allows you to identify issues early, optimize performance, and make data-driven decisions about future development. This guide will walk you through implementing monitoring solutions for your Gin web applications.

Why Monitor Your Gin Application?

Monitoring provides valuable insights into:

Performance metrics: Response times, request rates, resource usage
Error tracking: Frequency and types of errors occurring
User behavior: Most used endpoints, traffic patterns
System health: Server load, memory usage, database connections

Getting Started with Basic Logging

The simplest form of monitoring starts with proper logging. Gin provides built-in middleware for logging requests.

Basic Request Logging

go
package main

import (
    "github.com/gin-gonic/gin"
    "log"
    "time"
)

func main() {
    // Create a gin router with default middleware (logger and recovery)
    r := gin.Default()
    
    // Define a route
    r.GET("/ping", func(c *gin.Context) {
        c.JSON(200, gin.H{
            "message": "pong",
        })
    })
    
    r.Run(":8080")
}

The gin.Default() function includes the Logger middleware, which outputs logs like:

[GIN] 2023/07/15 - 12:34:56 | 200 |     123.456µs |      127.0.0.1 | GET      "/ping"

Custom Logging Format

You can create a custom logger to capture specific information:

go
package main

import (
    "github.com/gin-gonic/gin"
    "log"
    "time"
)

func main() {
    // Create a gin router without default middleware
    r := gin.New()
    
    // Use Recovery middleware
    r.Use(gin.Recovery())
    
    // Custom logger
    r.Use(func(c *gin.Context) {
        // Start timer
        start := time.Now()
        
        // Process request
        c.Next()
        
        // Log request details after processing
        latency := time.Since(start)
        log.Printf("| %3d | %13v | %15s | %-7s %s",
            c.Writer.Status(),
            latency,
            c.ClientIP(),
            c.Request.Method,
            c.Request.URL.Path,
        )
    })
    
    // Define a route
    r.GET("/ping", func(c *gin.Context) {
        c.JSON(200, gin.H{
            "message": "pong",
        })
    })
    
    r.Run(":8080")
}

Advanced Monitoring with Prometheus

For production applications, you'll want more comprehensive metrics. Prometheus is a popular monitoring solution that works well with Go applications.

Setting Up Prometheus with Gin

First, install the required packages:

bash
go get github.com/prometheus/client_golang/prometheus
go get github.com/prometheus/client_golang/prometheus/promauto
go get github.com/prometheus/client_golang/prometheus/promhttp

Then implement the metrics collection:

go
package main

import (
    "github.com/gin-gonic/gin"
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promauto"
    "github.com/prometheus/client_golang/prometheus/promhttp"
    "net/http"
    "strconv"
    "time"
)

var (
    // Define metrics
    httpRequestsTotal = promauto.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests",
        },
        []string{"method", "endpoint", "status"},
    )
    
    httpRequestDuration = promauto.NewHistogramVec(
        prometheus.HistogramOpts{
            Name:    "http_request_duration_seconds",
            Help:    "Duration of HTTP requests in seconds",
            Buckets: prometheus.DefBuckets,
        },
        []string{"method", "endpoint", "status"},
    )
)

// PrometheusMiddleware collects metrics for each HTTP request
func PrometheusMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        start := time.Now()
        
        // Process request
        c.Next()
        
        // Skip metrics for the metrics endpoint
        if c.Request.URL.Path == "/metrics" {
            return
        }
        
        // Record metrics after completing request
        status := strconv.Itoa(c.Writer.Status())
        elapsed := time.Since(start).Seconds()
        
        httpRequestsTotal.WithLabelValues(
            c.Request.Method,
            c.Request.URL.Path,
            status,
        ).Inc()
        
        httpRequestDuration.WithLabelValues(
            c.Request.Method,
            c.Request.URL.Path,
            status,
        ).Observe(elapsed)
    }
}

func main() {
    r := gin.New()
    
    // Add middleware
    r.Use(gin.Recovery())
    r.Use(PrometheusMiddleware())
    
    // Expose metrics endpoint for Prometheus to scrape
    r.GET("/metrics", gin.WrapH(promhttp.Handler()))
    
    // Sample endpoints
    r.GET("/ping", func(c *gin.Context) {
        c.JSON(200, gin.H{"message": "pong"})
    })
    
    r.GET("/slow", func(c *gin.Context) {
        time.Sleep(500 * time.Millisecond)
        c.JSON(200, gin.H{"message": "slow response"})
    })
    
    r.GET("/error", func(c *gin.Context) {
        c.JSON(500, gin.H{"message": "server error"})
    })
    
    r.Run(":8080")
}

With this setup, you can access metrics at /metrics endpoint, which Prometheus can scrape to collect data.

Implementing Health Checks

Health checks are endpoints that allow monitoring systems or load balancers to verify that your application is functioning properly.

go
package main

import (
    "github.com/gin-gonic/gin"
    "net/http"
)

func main() {
    r := gin.Default()
    
    // Simple health check endpoint
    r.GET("/health", func(c *gin.Context) {
        c.JSON(http.StatusOK, gin.H{
            "status": "UP",
        })
    })
    
    // Advanced health check with dependency checks
    r.GET("/health/detailed", func(c *gin.Context) {
        // Check database connection
        dbStatus := checkDatabase()
        
        // Check cache connection
        cacheStatus := checkCache()
        
        // Determine overall status
        overallStatus := "UP"
        httpStatus := http.StatusOK
        
        if !dbStatus || !cacheStatus {
            overallStatus = "DEGRADED"
            httpStatus = http.StatusServiceUnavailable
        }
        
        c.JSON(httpStatus, gin.H{
            "status": overallStatus,
            "components": gin.H{
                "database": dbStatus,
                "cache":    cacheStatus,
            },
        })
    })
    
    r.Run(":8080")
}

func checkDatabase() bool {
    // Replace with actual database connection check
    // For example: try to ping the database
    return true
}

func checkCache() bool {
    // Replace with actual cache connection check
    return true
}

Request Tracing

For complex applications, it's important to trace requests as they flow through different services. This can be implemented using OpenTelemetry or similar tools.

Here's a basic example using a request ID for tracing:

go
package main

import (
    "github.com/gin-gonic/gin"
    "github.com/google/uuid"
)

// RequestIDMiddleware adds a unique request ID to each request
func RequestIDMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        // Check for existing request ID (from upstream service)
        requestID := c.GetHeader("X-Request-ID")
        if requestID == "" {
            // Create new request ID if none exists
            requestID = uuid.New().String()
        }
        
        // Add request ID to context and headers
        c.Set("RequestID", requestID)
        c.Writer.Header().Set("X-Request-ID", requestID)
        
        c.Next()
    }
}

func main() {
    r := gin.New()
    r.Use(gin.Logger())
    r.Use(gin.Recovery())
    r.Use(RequestIDMiddleware())
    
    r.GET("/api/resource", func(c *gin.Context) {
        // Get request ID from context
        requestID := c.GetString("RequestID")
        
        // Log with request ID
        // log.Printf("[%s] Processing request for /api/resource", requestID)
        
        c.JSON(200, gin.H{
            "message": "Request processed",
            "request_id": requestID,
        })
    })
    
    r.Run(":8080")
}

Real-world Example: Complete Monitoring Setup

Let's create a more comprehensive example combining multiple monitoring techniques:

go
package main

import (
    "database/sql"
    "github.com/gin-gonic/gin"
    "github.com/google/uuid"
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promauto"
    "github.com/prometheus/client_golang/prometheus/promhttp"
    "log"
    "net/http"
    "os"
    "strconv"
    "time"
    
    _ "github.com/go-sql-driver/mysql"
)

var (
    // Logger
    logger = log.New(os.Stdout, "[GIN-APP] ", log.Ldate|log.Ltime)
    
    // Database connection
    db *sql.DB
    
    // Prometheus metrics
    httpRequestsTotal = promauto.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests",
        },
        []string{"method", "endpoint", "status"},
    )
    
    httpRequestDuration = promauto.NewHistogramVec(
        prometheus.HistogramOpts{
            Name:    "http_request_duration_seconds",
            Help:    "Duration of HTTP requests in seconds",
            Buckets: prometheus.DefBuckets,
        },
        []string{"method", "endpoint", "status"},
    )
    
    databaseQueriesTotal = promauto.NewCounter(
        prometheus.CounterOpts{
            Name: "database_queries_total",
            Help: "Total number of database queries",
        },
    )
)

// Middleware for request tracing
func TracingMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        // Add request ID
        requestID := c.GetHeader("X-Request-ID")
        if requestID == "" {
            requestID = uuid.New().String()
        }
        c.Set("RequestID", requestID)
        c.Writer.Header().Set("X-Request-ID", requestID)
        
        // Process request
        c.Next()
    }
}

// Middleware for logging
func LoggingMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        start := time.Now()
        
        // Get request ID
        requestID, exists := c.Get("RequestID")
        if !exists {
            requestID = "unknown"
        }
        
        // Process request
        c.Next()
        
        // Log request details
        latency := time.Since(start)
        status := c.Writer.Status()
        
        logger.Printf("[%s] %s %s | %d | %v | IP: %s",
            requestID,
            c.Request.Method,
            c.Request.URL.Path,
            status,
            latency,
            c.ClientIP(),
        )
        
        // Log any errors
        if len(c.Errors) > 0 {
            logger.Printf("[%s] Errors: %v", requestID, c.Errors)
        }
    }
}

// Middleware for prometheus metrics
func MetricsMiddleware() gin.HandlerFunc {
    return func(c *gin.Context) {
        start := time.Now()
        
        // Process request
        c.Next()
        
        // Skip metrics for the metrics endpoint itself
        if c.Request.URL.Path == "/metrics" {
            return
        }
        
        // Record metrics
        status := strconv.Itoa(c.Writer.Status())
        elapsed := time.Since(start).Seconds()
        
        httpRequestsTotal.WithLabelValues(
            c.Request.Method,
            c.Request.URL.Path,
            status,
        ).Inc()
        
        httpRequestDuration.WithLabelValues(
            c.Request.Method,
            c.Request.URL.Path,
            status,
        ).Observe(elapsed)
    }
}

func main() {
    // Initialize database connection
    var err error
    db, err = sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/dbname")
    if err != nil {
        logger.Fatalf("Failed to connect to database: %v", err)
    }
    defer db.Close()
    
    // Set up Gin
    r := gin.New()
    r.Use(gin.Recovery())
    r.Use(TracingMiddleware())
    r.Use(LoggingMiddleware())
    r.Use(MetricsMiddleware())
    
    // Health checks
    r.GET("/health", func(c *gin.Context) {
        c.JSON(http.StatusOK, gin.H{
            "status": "UP",
        })
    })
    
    r.GET("/health/detailed", func(c *gin.Context) {
        // Check database
        dbErr := db.Ping()
        dbStatus := dbErr == nil
        
        // Determine overall status
        status := "UP"
        httpStatus := http.StatusOK
        
        if !dbStatus {
            status = "DOWN"
            httpStatus = http.StatusServiceUnavailable
            logger.Printf("Health check failed: Database connection error: %v", dbErr)
        }
        
        c.JSON(httpStatus, gin.H{
            "status": status,
            "components": gin.H{
                "database": dbStatus,
            },
        })
    })
    
    // Metrics endpoint
    r.GET("/metrics", gin.WrapH(promhttp.Handler()))
    
    // API endpoints
    api := r.Group("/api")
    {
        api.GET("/users", func(c *gin.Context) {
            requestID, _ := c.Get("RequestID")
            logger.Printf("[%s] Fetching users from database", requestID)
            
            // Record database query metric
            databaseQueriesTotal.Inc()
            
            // Simulate database query
            time.Sleep(100 * time.Millisecond)
            
            c.JSON(http.StatusOK, gin.H{
                "users": []string{"user1", "user2", "user3"},
            })
        })
    }
    
    // Start server
    logger.Println("Starting server on :8080")
    if err := r.Run(":8080"); err != nil {
        logger.Fatalf("Failed to start server: %v", err)
    }
}

Integrating with Monitoring Systems

Prometheus + Grafana Setup

Install Prometheus and configure it to scrape your Gin application's /metrics endpoint:

yaml
# prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'gin-app'
    static_configs:
      - targets: ['localhost:8080']

Set up Grafana and configure a dashboard with panels for:
- Request rate
- Error rate
- Response time percentiles
- Database query count
- System resources (CPU, memory)

Logging to ELK Stack (Elasticsearch, Logstash, Kibana)

To send your logs to an ELK stack, you can use a log library that supports structured logging:

go
import (
    "github.com/gin-gonic/gin"
    "github.com/sirupsen/logrus"
    "gopkg.in/sohlich/elogrus.v7"
    "github.com/olivere/elastic/v7"
)

func setupLogging() *logrus.Logger {
    logger := logrus.New()
    logger.SetFormatter(&logrus.JSONFormatter{})
    
    // Connect to Elasticsearch
    client, err := elastic.NewClient(
        elastic.SetURL("http://localhost:9200"),
        elastic.SetSniff(false),
    )
    if err != nil {
        logger.Fatal(err)
    }
    
    // Hook to send logs to Elasticsearch
    hook, err := elogrus.NewAsyncElasticHook(
        client, 
        "localhost", 
        logrus.InfoLevel, 
        "gin-logs",
    )
    if err != nil {
        logger.Fatal(err)
    }
    logger.AddHook(hook)
    
    return logger
}

Summary

Monitoring your Gin application is essential for ensuring reliability, performance, and user satisfaction. In this guide, we've covered:

Basic logging techniques built into Gin
Metrics collection with Prometheus for quantitative monitoring
Health checks to verify application functionality
Request tracing to follow requests through your system
Integration with monitoring systems like Prometheus/Grafana and ELK Stack

By implementing these techniques, you'll gain visibility into your application's behavior, catch issues early, and make informed decisions about optimizations and improvements.

Additional Resources

Exercises

Implement the Prometheus monitoring example and create a simple Grafana dashboard to visualize the metrics.
Add custom metrics to track important business metrics for your application (e.g., user registrations, orders placed).
Create a detailed health check system that verifies all external dependencies your application relies on.
Implement distributed tracing for a multi-service application using OpenTelemetry.
Set up alerts in Prometheus/Grafana to notify you when error rates exceed a threshold or when response times are too high.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

Why Monitor Your Gin Application?​

Getting Started with Basic Logging​

Basic Request Logging​

Custom Logging Format​

Advanced Monitoring with Prometheus​

Setting Up Prometheus with Gin​

Implementing Health Checks​

Request Tracing​

Real-world Example: Complete Monitoring Setup​

Integrating with Monitoring Systems​

Prometheus + Grafana Setup​

Logging to ELK Stack (Elasticsearch, Logstash, Kibana)​

Summary​

Additional Resources​

Exercises​