.NET Monitoring Techniques

Introduction

Monitoring is a critical aspect of maintaining healthy .NET applications in production environments. Effective monitoring allows developers and operations teams to identify issues before they impact users, troubleshoot problems quickly, and optimize application performance.

In this guide, we'll explore various techniques and tools for monitoring .NET applications, from basic logging to advanced Application Performance Monitoring (APM) solutions. Whether you're deploying a simple web application or a complex microservices architecture, these monitoring approaches will help ensure your .NET applications run smoothly.

Why Monitoring Matters

Before diving into specific techniques, let's understand why monitoring is essential:

• Early Issue Detection: Identify problems before they affect users
• Performance Optimization: Pinpoint bottlenecks and resource-intensive operations
• Capacity Planning: Make informed decisions about scaling resources
• Security Monitoring: Detect unusual access patterns or potential breaches
• User Experience Insights: Understand how real users interact with your application

Basic Logging Techniques

Using the Built-in Logging Framework

.NET provides a robust logging infrastructure through the Microsoft.Extensions.Logging namespace. This is the foundation of most monitoring strategies.

First, add the required package to your project:

dotnet add package Microsoft.Extensions.Logging

Here's a basic example of configuring and using the logger:

using Microsoft.Extensions.Logging;

public class OrderService
{
    private readonly ILogger<OrderService> _logger;

    public OrderService(ILogger<OrderService> logger)
    {
        _logger = logger;
    }

    public void ProcessOrder(Order order)
    {
        _logger.LogInformation("Starting to process order {OrderId}", order.Id);
        
        try
        {
            // Order processing logic
            _logger.LogInformation("Order {OrderId} processed successfully", order.Id);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to process order {OrderId}", order.Id);
            throw;
        }
    }
}

Configuring Logging in ASP.NET Core

In ASP.NET Core applications, you can configure logging in the Program.cs file:

var builder = WebApplication.CreateBuilder(args);

// Configure logging
builder.Logging.ClearProviders();
builder.Logging.AddConsole();
builder.Logging.AddDebug();
builder.Logging.AddEventLog();

// Set minimum log level
builder.Logging.SetMinimumLevel(LogLevel.Information);

var app = builder.Build();
// Rest of your application setup

Structured Logging with Serilog

For more advanced logging capabilities, Serilog is a popular choice among .NET developers:

dotnet add package Serilog.AspNetCore

Configure Serilog in your application:

var builder = WebApplication.CreateBuilder(args);

// Setup Serilog
builder.Host.UseSerilog((ctx, lc) => lc
    .ReadFrom.Configuration(ctx.Configuration)
    .Enrich.FromLogContext()
    .WriteTo.Console()
    .WriteTo.File("logs/app.log", rollingInterval: RollingInterval.Day));

// Services and middleware setup
var app = builder.Build();

Example appsettings.json configuration:

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    }
  }
}

Health Checks

Health checks provide a simple way to report the health status of your application and its dependencies.

Implementing Health Checks in ASP.NET Core

Add the health checks package:

dotnet add package Microsoft.AspNetCore.Diagnostics.HealthChecks

Configure health checks in your application:

var builder = WebApplication.CreateBuilder(args);

// Add health checks
builder.Services.AddHealthChecks()
    .AddSqlServer(builder.Configuration.GetConnectionString("DefaultConnection"))
    .AddRedis(builder.Configuration.GetConnectionString("Redis"))
    .AddCheck<CustomHealthCheck>("custom_check");

var app = builder.Build();

// Configure middleware
app.UseHealthChecks("/health", new HealthCheckOptions
{
    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});

app.Run();

Example of a custom health check:

public class CustomHealthCheck : IHealthCheck
{
    public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, 
        CancellationToken cancellationToken = default)
    {
        // Check some business-critical dependency or functionality
        var isHealthy = await CheckCriticalServiceAsync();
        
        if (isHealthy)
        {
            return HealthCheckResult.Healthy("The service is functioning normally.");
        }
        
        return HealthCheckResult.Unhealthy("The service is experiencing issues.");
    }
    
    private async Task<bool> CheckCriticalServiceAsync()
    {
        // Implementation of your critical service check
        return true;
    }
}

Performance Monitoring

Performance Counters

.NET provides access to system performance counters which can be used to monitor various aspects of your application:

using System.Diagnostics;

public class PerformanceMonitor
{
    public void MonitorCpuUsage()
    {
        try 
        {
            using var cpuCounter = new PerformanceCounter("Processor", "% Processor Time", "_Total");
            var firstValue = cpuCounter.NextValue();
            
            // Need to wait a short period to get accurate reading
            Thread.Sleep(1000);
            
            var secondValue = cpuCounter.NextValue();
            Console.WriteLine($"CPU Usage: {secondValue}%");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to monitor CPU: {ex.Message}");
        }
    }
    
    public void MonitorMemoryUsage()
    {
        try
        {
            using var memCounter = new PerformanceCounter("Memory", "Available MBytes");
            var availableMb = memCounter.NextValue();
            Console.WriteLine($"Available Memory: {availableMb} MB");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to monitor memory: {ex.Message}");
        }
    }
}

.NET EventCounters

EventCounters provide a modern, lightweight alternative to performance counters:

using System.Diagnostics.Tracing;

[EventSource(Name = "MyCompany-MyApp-Monitoring")]
public sealed class MyAppEventSource : EventSource
{
    public static readonly MyAppEventSource Log = new MyAppEventSource();
    
    private IncrementingEventCounter _requestCounter;
    private EventCounter _requestDuration;
    
    private MyAppEventSource()
    {
        _requestCounter = new IncrementingEventCounter("request-count", this)
        {
            DisplayName = "Request Count",
            DisplayRateTimeScale = TimeSpan.FromSeconds(1)
        };
        
        _requestDuration = new EventCounter("request-duration", this)
        {
            DisplayName = "Request Duration",
            DisplayUnits = "ms"
        };
    }
    
    public void RequestStart()
    {
        _requestCounter.Increment();
    }
    
    public void RequestEnd(float duration)
    {
        _requestDuration.WriteMetric(duration);
    }
    
    protected override void Dispose(bool disposing)
    {
        _requestCounter?.Dispose();
        _requestCounter = null;
        
        _requestDuration?.Dispose();
        _requestDuration = null;
        
        base.Dispose(disposing);
    }
}

To use this event source in your application:

public async Task ProcessRequestAsync()
{
    MyAppEventSource.Log.RequestStart();
    var stopwatch = Stopwatch.StartNew();
    
    try
    {
        // Process request
        await Task.Delay(100); // Simulating work
    }
    finally
    {
        stopwatch.Stop();
        MyAppEventSource.Log.RequestEnd(stopwatch.ElapsedMilliseconds);
    }
}

Diagnostic Tools

Using dotnet-trace

dotnet-trace is a command-line tool for collecting traces from .NET applications:

# Install the tool
dotnet tool install --global dotnet-trace

# Collect a trace
dotnet trace collect --process-id <PID> --output trace.nettrace

# Convert the trace to a format readable by perfview
dotnet trace convert trace.nettrace --format speedscope

Using dotnet-counters

dotnet-counters allows you to monitor .NET Core applications using EventCounters:

# Install the tool
dotnet tool install --global dotnet-counters

# Monitor system.runtime counters
dotnet counters monitor --process-id <PID> System.Runtime

# Monitor custom counters
dotnet counters monitor --process-id <PID> MyCompany-MyApp-Monitoring

Example output:

Press p to pause, r to resume, q to quit.
    Status: Running

[System.Runtime]
    CPU Usage (%)                                      24
    Working Set (MB)                                  112
    GC Heap Size (MB)                                  54
    Gen 0 GC Count                                    115
    Gen 1 GC Count                                     49
    Gen 2 GC Count                                      5
    Exception Count                                     0

Using dotnet-dump

For analyzing memory issues and crashes:

# Install the tool
dotnet tool install --global dotnet-dump

# Collect a memory dump
dotnet dump collect --process-id <PID>

# Analyze the dump
dotnet dump analyze core_20201016_143440

Application Performance Monitoring (APM) Tools

Application Insights Integration

Azure Application Insights is a powerful APM solution for .NET applications. Here's how to integrate it:

dotnet add package Microsoft.ApplicationInsights.AspNetCore

Configure in Program.cs:

var builder = WebApplication.CreateBuilder(args);

// Add Application Insights
builder.Services.AddApplicationInsightsTelemetry(builder.Configuration["ApplicationInsights:ConnectionString"]);

var app = builder.Build();
// Rest of application setup

Then configure the connection string in appsettings.json:

{
  "ApplicationInsights": {
    "ConnectionString": "InstrumentationKey=your-key;IngestionEndpoint=https://regionname.in.applicationinsights.azure.com/"
  }
}

Custom Telemetry with Application Insights

Track custom events and metrics:

public class OrderController : Controller
{
    private readonly TelemetryClient _telemetryClient;
    
    public OrderController(TelemetryClient telemetryClient)
    {
        _telemetryClient = telemetryClient;
    }
    
    [HttpPost]
    public IActionResult PlaceOrder(OrderViewModel model)
    {
        try
        {
            // Process order
            
            // Track successful order with custom properties
            _telemetryClient.TrackEvent("OrderPlaced", new Dictionary<string, string>
            {
                ["OrderId"] = order.Id.ToString(),
                ["CustomerType"] = order.CustomerType,
                ["TotalAmount"] = order.TotalAmount.ToString()
            });
            
            // Track business metrics
            _telemetryClient.TrackMetric("OrderValue", order.TotalAmount);
            
            return Ok();
        }
        catch (Exception ex)
        {
            // Track failed orders
            _telemetryClient.TrackException(ex, new Dictionary<string, string>
            {
                ["OrderId"] = model.OrderId.ToString()
            });
            
            return StatusCode(500);
        }
    }
}

Distributed Tracing

For microservices architectures, distributed tracing is essential:

OpenTelemetry Integration

dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.Console

Configure in your application:

var builder = WebApplication.CreateBuilder(args);

// Add OpenTelemetry
builder.Services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation()
        .AddConsoleExporter());

var app = builder.Build();
// Rest of application setup

Monitoring Best Practices

1. Start Simple: Begin with basic logging and health checks before implementing complex monitoring
2. Monitor the Right Metrics: Focus on metrics that directly impact user experience
3. Set Appropriate Alerting Thresholds: Avoid alert fatigue by only alerting on actionable issues
4. Use Correlation IDs: Implement request correlation for tracing requests across services
5. Include Business Metrics: Monitor not just technical metrics but also business KPIs
6. Regularly Review Dashboards: Don't just set up monitoring—actively use it to improve your application
7. Consider Privacy and Security: Ensure sensitive data isn't included in logs or metrics

Real-World Monitoring Example

Let's see a complete example of monitoring setup for a typical web API:

using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using Serilog;
using Serilog.Events;

var builder = WebApplication.CreateBuilder(args);

// Configure Serilog
Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .Enrich.FromLogContext()
    .Enrich.WithMachineName()
    .Enrich.WithEnvironmentName()
    .WriteTo.Console()
    .WriteTo.Seq("http://seq-server:5341")
    .CreateLogger();

builder.Host.UseSerilog();

// Configure services
builder.Services.AddControllers();

// Add health checks
builder.Services.AddHealthChecks()
    .AddSqlServer(builder.Configuration.GetConnectionString("DefaultConnection"))
    .AddRedis(builder.Configuration.GetConnectionString("Redis"))
    .AddUrlGroup(new Uri("https://api.external-service.com/health"), "external-service");

// Add Application Insights
builder.Services.AddApplicationInsightsTelemetry();

// Add OpenTelemetry
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService("MyServiceName"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation()
        .AddOtlpExporter(options => options.Endpoint = new Uri("http://otel-collector:4317")))
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddRuntimeInstrumentation()
        .AddOtlpExporter(options => options.Endpoint = new Uri("http://otel-collector:4317")));

// Build application
var app = builder.Build();

// Configure middlewares
app.UseSerilogRequestLogging();

// Configure health check endpoints
app.MapHealthChecks("/health");
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready")
});
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false
});

app.MapControllers();
app.Run();

Summary

Effective monitoring is crucial for maintaining reliable .NET applications in production. In this guide, we've covered:

• Basic logging techniques using built-in .NET logging and Serilog
• Health checks to monitor application and dependency status
• Performance monitoring with counters and EventCounters
• Diagnostic tools for investigating issues
• Application Performance Monitoring (APM) with Azure Application Insights
• Distributed tracing with OpenTelemetry
• Best practices for implementing monitoring in real-world applications

By implementing these monitoring techniques, you'll gain insight into your application's behavior, detect issues early, and ensure optimal performance for your users.

Additional Resources

• Microsoft Documentation on .NET Logging
• Health Checks in ASP.NET Core
• Serilog Documentation
• Azure Application Insights Documentation
• OpenTelemetry .NET Documentation

Exercises

1. Set up basic logging with Serilog in a new ASP.NET Core application
2. Implement health checks for your database and external services
3. Use EventCounters to track custom metrics in your application
4. Create a simple dashboard in Application Insights to monitor your application
5. Implement distributed tracing in a multi-service application using OpenTelemetry

By working through these exercises, you'll gain hands-on experience with .NET monitoring techniques that will help you maintain robust, high-performing applications.