.NET Monitoring Techniques

Introduction

Monitoring is a critical aspect of maintaining healthy .NET applications in production environments. Effective monitoring allows developers and operations teams to identify issues before they impact users, troubleshoot problems quickly, and optimize application performance.

In this guide, we'll explore various techniques and tools for monitoring .NET applications, from basic logging to advanced Application Performance Monitoring (APM) solutions. Whether you're deploying a simple web application or a complex microservices architecture, these monitoring approaches will help ensure your .NET applications run smoothly.

Why Monitoring Matters

Before diving into specific techniques, let's understand why monitoring is essential:

Early Issue Detection: Identify problems before they affect users
Performance Optimization: Pinpoint bottlenecks and resource-intensive operations
Capacity Planning: Make informed decisions about scaling resources
Security Monitoring: Detect unusual access patterns or potential breaches
User Experience Insights: Understand how real users interact with your application

Basic Logging Techniques

Using the Built-in Logging Framework

.NET provides a robust logging infrastructure through the Microsoft.Extensions.Logging namespace. This is the foundation of most monitoring strategies.

First, add the required package to your project:

bash
dotnet add package Microsoft.Extensions.Logging

Here's a basic example of configuring and using the logger:

csharp
using Microsoft.Extensions.Logging;

public class OrderService
{
    private readonly ILogger<OrderService> _logger;

    public OrderService(ILogger<OrderService> logger)
    {
        _logger = logger;
    }

    public void ProcessOrder(Order order)
    {
        _logger.LogInformation("Starting to process order {OrderId}", order.Id);
        
        try
        {
            // Order processing logic
            _logger.LogInformation("Order {OrderId} processed successfully", order.Id);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to process order {OrderId}", order.Id);
            throw;
        }
    }
}

Configuring Logging in ASP.NET Core

In ASP.NET Core applications, you can configure logging in the Program.cs file:

csharp
var builder = WebApplication.CreateBuilder(args);

// Configure logging
builder.Logging.ClearProviders();
builder.Logging.AddConsole();
builder.Logging.AddDebug();
builder.Logging.AddEventLog();

// Set minimum log level
builder.Logging.SetMinimumLevel(LogLevel.Information);

var app = builder.Build();
// Rest of your application setup

Structured Logging with Serilog

For more advanced logging capabilities, Serilog is a popular choice among .NET developers:

bash
dotnet add package Serilog.AspNetCore

Configure Serilog in your application:

csharp
var builder = WebApplication.CreateBuilder(args);

// Setup Serilog
builder.Host.UseSerilog((ctx, lc) => lc
    .ReadFrom.Configuration(ctx.Configuration)
    .Enrich.FromLogContext()
    .WriteTo.Console()
    .WriteTo.File("logs/app.log", rollingInterval: RollingInterval.Day));

// Services and middleware setup
var app = builder.Build();

Example appsettings.json configuration:

json
{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    }
  }
}

Health Checks

Health checks provide a simple way to report the health status of your application and its dependencies.

Implementing Health Checks in ASP.NET Core

Add the health checks package:

bash
dotnet add package Microsoft.AspNetCore.Diagnostics.HealthChecks

Configure health checks in your application:

csharp
var builder = WebApplication.CreateBuilder(args);

// Add health checks
builder.Services.AddHealthChecks()
    .AddSqlServer(builder.Configuration.GetConnectionString("DefaultConnection"))
    .AddRedis(builder.Configuration.GetConnectionString("Redis"))
    .AddCheck<CustomHealthCheck>("custom_check");

var app = builder.Build();

// Configure middleware
app.UseHealthChecks("/health", new HealthCheckOptions
{
    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});

app.Run();

Example of a custom health check:

csharp
public class CustomHealthCheck : IHealthCheck
{
    public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, 
        CancellationToken cancellationToken = default)
    {
        // Check some business-critical dependency or functionality
        var isHealthy = await CheckCriticalServiceAsync();
        
        if (isHealthy)
        {
            return HealthCheckResult.Healthy("The service is functioning normally.");
        }
        
        return HealthCheckResult.Unhealthy("The service is experiencing issues.");
    }
    
    private async Task<bool> CheckCriticalServiceAsync()
    {
        // Implementation of your critical service check
        return true;
    }
}

Performance Monitoring

Performance Counters

.NET provides access to system performance counters which can be used to monitor various aspects of your application:

csharp
using System.Diagnostics;

public class PerformanceMonitor
{
    public void MonitorCpuUsage()
    {
        try 
        {
            using var cpuCounter = new PerformanceCounter("Processor", "% Processor Time", "_Total");
            var firstValue = cpuCounter.NextValue();
            
            // Need to wait a short period to get accurate reading
            Thread.Sleep(1000);
            
            var secondValue = cpuCounter.NextValue();
            Console.WriteLine($"CPU Usage: {secondValue}%");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to monitor CPU: {ex.Message}");
        }
    }
    
    public void MonitorMemoryUsage()
    {
        try
        {
            using var memCounter = new PerformanceCounter("Memory", "Available MBytes");
            var availableMb = memCounter.NextValue();
            Console.WriteLine($"Available Memory: {availableMb} MB");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to monitor memory: {ex.Message}");
        }
    }
}

.NET EventCounters

EventCounters provide a modern, lightweight alternative to performance counters:

csharp
using System.Diagnostics.Tracing;

[EventSource(Name = "MyCompany-MyApp-Monitoring")]
public sealed class MyAppEventSource : EventSource
{
    public static readonly MyAppEventSource Log = new MyAppEventSource();
    
    private IncrementingEventCounter _requestCounter;
    private EventCounter _requestDuration;
    
    private MyAppEventSource()
    {
        _requestCounter = new IncrementingEventCounter("request-count", this)
        {
            DisplayName = "Request Count",
            DisplayRateTimeScale = TimeSpan.FromSeconds(1)
        };
        
        _requestDuration = new EventCounter("request-duration", this)
        {
            DisplayName = "Request Duration",
            DisplayUnits = "ms"
        };
    }
    
    public void RequestStart()
    {
        _requestCounter.Increment();
    }
    
    public void RequestEnd(float duration)
    {
        _requestDuration.WriteMetric(duration);
    }
    
    protected override void Dispose(bool disposing)
    {
        _requestCounter?.Dispose();
        _requestCounter = null;
        
        _requestDuration?.Dispose();
        _requestDuration = null;
        
        base.Dispose(disposing);
    }
}

To use this event source in your application:

csharp
public async Task ProcessRequestAsync()
{
    MyAppEventSource.Log.RequestStart();
    var stopwatch = Stopwatch.StartNew();
    
    try
    {
        // Process request
        await Task.Delay(100); // Simulating work
    }
    finally
    {
        stopwatch.Stop();
        MyAppEventSource.Log.RequestEnd(stopwatch.ElapsedMilliseconds);
    }
}

Diagnostic Tools

Using dotnet-trace

dotnet-trace is a command-line tool for collecting traces from .NET applications:

bash
# Install the tool
dotnet tool install --global dotnet-trace

# Collect a trace
dotnet trace collect --process-id <PID> --output trace.nettrace

# Convert the trace to a format readable by perfview
dotnet trace convert trace.nettrace --format speedscope

Using dotnet-counters

dotnet-counters allows you to monitor .NET Core applications using EventCounters:

bash
# Install the tool
dotnet tool install --global dotnet-counters

# Monitor system.runtime counters
dotnet counters monitor --process-id <PID> System.Runtime

# Monitor custom counters
dotnet counters monitor --process-id <PID> MyCompany-MyApp-Monitoring

Example output:

Press p to pause, r to resume, q to quit.
    Status: Running

[System.Runtime]
    CPU Usage (%)                                      24
    Working Set (MB)                                  112
    GC Heap Size (MB)                                  54
    Gen 0 GC Count                                    115
    Gen 1 GC Count                                     49
    Gen 2 GC Count                                      5
    Exception Count                                     0

Using dotnet-dump

For analyzing memory issues and crashes:

bash
# Install the tool
dotnet tool install --global dotnet-dump

# Collect a memory dump
dotnet dump collect --process-id <PID>

# Analyze the dump
dotnet dump analyze core_20201016_143440

Application Performance Monitoring (APM) Tools

Application Insights Integration

Azure Application Insights is a powerful APM solution for .NET applications. Here's how to integrate it:

bash
dotnet add package Microsoft.ApplicationInsights.AspNetCore

Configure in Program.cs:

csharp
var builder = WebApplication.CreateBuilder(args);

// Add Application Insights
builder.Services.AddApplicationInsightsTelemetry(builder.Configuration["ApplicationInsights:ConnectionString"]);

var app = builder.Build();
// Rest of application setup

Then configure the connection string in appsettings.json:

json
{
  "ApplicationInsights": {
    "ConnectionString": "InstrumentationKey=your-key;IngestionEndpoint=https://regionname.in.applicationinsights.azure.com/"
  }
}

Custom Telemetry with Application Insights

Track custom events and metrics:

csharp
public class OrderController : Controller
{
    private readonly TelemetryClient _telemetryClient;
    
    public OrderController(TelemetryClient telemetryClient)
    {
        _telemetryClient = telemetryClient;
    }
    
    [HttpPost]
    public IActionResult PlaceOrder(OrderViewModel model)
    {
        try
        {
            // Process order
            
            // Track successful order with custom properties
            _telemetryClient.TrackEvent("OrderPlaced", new Dictionary<string, string>
            {
                ["OrderId"] = order.Id.ToString(),
                ["CustomerType"] = order.CustomerType,
                ["TotalAmount"] = order.TotalAmount.ToString()
            });
            
            // Track business metrics
            _telemetryClient.TrackMetric("OrderValue", order.TotalAmount);
            
            return Ok();
        }
        catch (Exception ex)
        {
            // Track failed orders
            _telemetryClient.TrackException(ex, new Dictionary<string, string>
            {
                ["OrderId"] = model.OrderId.ToString()
            });
            
            return StatusCode(500);
        }
    }
}

Distributed Tracing

For microservices architectures, distributed tracing is essential:

OpenTelemetry Integration

bash
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.Console

Configure in your application:

csharp
var builder = WebApplication.CreateBuilder(args);

// Add OpenTelemetry
builder.Services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation()
        .AddConsoleExporter());

var app = builder.Build();
// Rest of application setup

Monitoring Best Practices

Start Simple: Begin with basic logging and health checks before implementing complex monitoring
Monitor the Right Metrics: Focus on metrics that directly impact user experience
Set Appropriate Alerting Thresholds: Avoid alert fatigue by only alerting on actionable issues
Use Correlation IDs: Implement request correlation for tracing requests across services
Include Business Metrics: Monitor not just technical metrics but also business KPIs
Regularly Review Dashboards: Don't just set up monitoring—actively use it to improve your application
Consider Privacy and Security: Ensure sensitive data isn't included in logs or metrics

Real-World Monitoring Example

Let's see a complete example of monitoring setup for a typical web API:

csharp
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using Serilog;
using Serilog.Events;

var builder = WebApplication.CreateBuilder(args);

// Configure Serilog
Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .Enrich.FromLogContext()
    .Enrich.WithMachineName()
    .Enrich.WithEnvironmentName()
    .WriteTo.Console()
    .WriteTo.Seq("http://seq-server:5341")
    .CreateLogger();

builder.Host.UseSerilog();

// Configure services
builder.Services.AddControllers();

// Add health checks
builder.Services.AddHealthChecks()
    .AddSqlServer(builder.Configuration.GetConnectionString("DefaultConnection"))
    .AddRedis(builder.Configuration.GetConnectionString("Redis"))
    .AddUrlGroup(new Uri("https://api.external-service.com/health"), "external-service");

// Add Application Insights
builder.Services.AddApplicationInsightsTelemetry();

// Add OpenTelemetry
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService("MyServiceName"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation()
        .AddOtlpExporter(options => options.Endpoint = new Uri("http://otel-collector:4317")))
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddRuntimeInstrumentation()
        .AddOtlpExporter(options => options.Endpoint = new Uri("http://otel-collector:4317")));

// Build application
var app = builder.Build();

// Configure middlewares
app.UseSerilogRequestLogging();

// Configure health check endpoints
app.MapHealthChecks("/health");
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready")
});
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false
});

app.MapControllers();
app.Run();

Summary

Effective monitoring is crucial for maintaining reliable .NET applications in production. In this guide, we've covered:

Basic logging techniques using built-in .NET logging and Serilog
Health checks to monitor application and dependency status
Performance monitoring with counters and EventCounters
Diagnostic tools for investigating issues
Application Performance Monitoring (APM) with Azure Application Insights
Distributed tracing with OpenTelemetry
Best practices for implementing monitoring in real-world applications

By implementing these monitoring techniques, you'll gain insight into your application's behavior, detect issues early, and ensure optimal performance for your users.

Additional Resources

Exercises

Set up basic logging with Serilog in a new ASP.NET Core application
Implement health checks for your database and external services
Use EventCounters to track custom metrics in your application
Create a simple dashboard in Application Insights to monitor your application
Implement distributed tracing in a multi-service application using OpenTelemetry

By working through these exercises, you'll gain hands-on experience with .NET monitoring techniques that will help you maintain robust, high-performing applications.

If you spot any mistakes on this website, please let me know at feedback@compilenrun.com. I’d greatly appreciate your feedback! :)

Introduction​

Why Monitoring Matters​

Basic Logging Techniques​

Using the Built-in Logging Framework​

Configuring Logging in ASP.NET Core​

Structured Logging with Serilog​

Health Checks​

Implementing Health Checks in ASP.NET Core​

Performance Monitoring​

Performance Counters​

.NET EventCounters​

Diagnostic Tools​

Using dotnet-trace​

Using dotnet-counters​

Using dotnet-dump​

Application Performance Monitoring (APM) Tools​

Application Insights Integration​

Custom Telemetry with Application Insights​

Distributed Tracing​

OpenTelemetry Integration​

Monitoring Best Practices​

Real-World Monitoring Example​

Summary​

Additional Resources​

Exercises​