.NET Background Workers

Introduction

Background workers in .NET allow your applications to perform time-consuming operations without freezing the user interface or blocking the main thread. They are essential for creating responsive applications, particularly those with graphical interfaces or web services that need to handle multiple requests simultaneously.

In this tutorial, we'll explore different ways to implement background processing in .NET applications. We'll cover the classic BackgroundWorker component, the newer IHostedService and BackgroundService abstractions, and discuss when to use each approach.

Why Use Background Workers?

Before diving into implementations, let's understand when and why you should use background workers:

Responsiveness: Keep your UI responsive while processing intensive tasks
Parallel processing: Execute multiple operations concurrently
Long-running operations: Handle tasks that take substantial time to complete
Scheduled tasks: Execute jobs at regular intervals
Resource management: Better control over CPU and memory usage

The Classic BackgroundWorker Component

The BackgroundWorker class has been part of .NET Framework since version 2.0 and is still available in modern .NET. It's particularly useful for Windows Forms and WPF applications where you need to run operations without freezing the UI.

Basic Implementation

Here's a simple example using BackgroundWorker to count numbers in the background:

csharp
using System;
using System.ComponentModel;
using System.Windows.Forms;

namespace BackgroundWorkerDemo
{
    public partial class MainForm : Form
    {
        private BackgroundWorker _worker;
        
        public MainForm()
        {
            InitializeComponent();
            
            // Initialize the BackgroundWorker
            _worker = new BackgroundWorker();
            _worker.WorkerReportsProgress = true;
            _worker.WorkerSupportsCancellation = true;
            
            // Define what happens during the background operation
            _worker.DoWork += Worker_DoWork;
            
            // Define what happens when progress is reported
            _worker.ProgressChanged += Worker_ProgressChanged;
            
            // Define what happens when the operation completes
            _worker.RunWorkerCompleted += Worker_RunWorkerCompleted;
        }
        
        private void StartButton_Click(object sender, EventArgs e)
        {
            if (!_worker.IsBusy)
            {
                // Start the asynchronous operation
                _worker.RunWorkerAsync(100); // Pass 100 as the maximum count
                StartButton.Enabled = false;
                CancelButton.Enabled = true;
            }
        }
        
        private void CancelButton_Click(object sender, EventArgs e)
        {
            if (_worker.IsBusy)
            {
                _worker.CancelAsync();
                CancelButton.Enabled = false;
            }
        }
        
        private void Worker_DoWork(object sender, DoWorkEventArgs e)
        {
            // Get the BackgroundWorker that raised this event
            BackgroundWorker worker = sender as BackgroundWorker;
            
            // Get the maximum count from the argument
            int max = (int)e.Argument;
            
            // Perform a time-consuming operation
            for (int i = 1; i <= max; i++)
            {
                // Check for cancellation request
                if (worker.CancellationPending)
                {
                    e.Cancel = true;
                    return;
                }
                
                // Simulate work being done
                System.Threading.Thread.Sleep(100);
                
                // Report progress
                int percentComplete = (i * 100) / max;
                worker.ReportProgress(percentComplete, i);
            }
            
            // Set the result which will be available in RunWorkerCompleted
            e.Result = $"Counted to {max}";
        }
        
        private void Worker_ProgressChanged(object sender, ProgressChangedEventArgs e)
        {
            // This runs on the UI thread
            progressBar.Value = e.ProgressPercentage;
            statusLabel.Text = $"Processing... Current number: {e.UserState}";
        }
        
        private void Worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
            // This runs on the UI thread
            if (e.Cancelled)
            {
                statusLabel.Text = "Operation was cancelled";
            }
            else if (e.Error != null)
            {
                statusLabel.Text = $"Error: {e.Error.Message}";
            }
            else
            {
                statusLabel.Text = $"Completed! {e.Result}";
            }
            
            StartButton.Enabled = true;
            CancelButton.Enabled = false;
        }
    }
}

When to Use BackgroundWorker

The BackgroundWorker component is best suited for:

Desktop applications (Windows Forms, WPF) where you need UI updates
Scenarios where you need built-in progress reporting
Simple background tasks with clear start and end points
Cases where you need cancellation support

Modern Background Processing: IHostedService and BackgroundService

For more modern .NET applications, especially ASP.NET Core and console apps using the .NET Generic Host, Microsoft introduced IHostedService and BackgroundService abstractions.

IHostedService Interface

The IHostedService interface defines methods that are called when the application starts and stops:

csharp
public interface IHostedService
{
    Task StartAsync(CancellationToken cancellationToken);
    Task StopAsync(CancellationToken cancellationToken);
}

Using BackgroundService Abstract Class

The BackgroundService abstract class implements IHostedService and provides a convenient base for creating services that perform work in the background:

csharp
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System;
using System.Threading;
using System.Threading.Tasks;

namespace BackgroundServiceDemo
{
    public class TimedBackgroundService : BackgroundService
    {
        private readonly ILogger<TimedBackgroundService> _logger;
        private readonly TimeSpan _period = TimeSpan.FromSeconds(5);
        
        public TimedBackgroundService(ILogger<TimedBackgroundService> logger)
        {
            _logger = logger;
        }
        
        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            _logger.LogInformation("Timed Background Service is starting.");
            
            using PeriodicTimer timer = new PeriodicTimer(_period);
            
            try
            {
                while (await timer.WaitForNextTickAsync(stoppingToken))
                {
                    await DoWorkAsync(stoppingToken);
                }
            }
            catch (OperationCanceledException)
            {
                _logger.LogInformation("Timed Background Service is stopping.");
            }
        }
        
        private async Task DoWorkAsync(CancellationToken stoppingToken)
        {
            _logger.LogInformation("Timed Background Service is working.");
            
            // Simulate a time-consuming operation
            await Task.Delay(1000, stoppingToken);
            
            _logger.LogInformation("Work completed at: {time}", DateTimeOffset.Now);
        }
    }
}

Registering the Background Service

To use your background service, register it in the dependency injection container:

csharp
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using System.Threading.Tasks;

namespace BackgroundServiceDemo
{
    public class Program
    {
        public static async Task Main(string[] args)
        {
            IHost host = Host.CreateDefaultBuilder(args)
                .ConfigureServices((hostContext, services) =>
                {
                    services.AddHostedService<TimedBackgroundService>();
                })
                .Build();

            await host.RunAsync();
        }
    }
}

When to Use BackgroundService

The BackgroundService and IHostedService are best suited for:

ASP.NET Core applications that need background processing
Console applications using the .NET Generic Host
Services that need to start as soon as the application starts
Long-running or recurring tasks that should run for the lifetime of your application
Scenarios where you need dependency injection

Practical Example: Processing Queue Items

Here's a real-world example of a background service that processes items from a queue:

csharp
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

public class QueueProcessingService : BackgroundService
{
    private readonly ILogger<QueueProcessingService> _logger;
    private readonly ConcurrentQueue<WorkItem> _workItems = new ConcurrentQueue<WorkItem>();
    private readonly SemaphoreSlim _signal = new SemaphoreSlim(0);
    
    public QueueProcessingService(ILogger<QueueProcessingService> logger)
    {
        _logger = logger;
    }
    
    public void QueueWorkItem(WorkItem workItem)
    {
        if (workItem == null)
            throw new ArgumentNullException(nameof(workItem));
            
        _workItems.Enqueue(workItem);
        _signal.Release();
    }
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Queue Processing Service is starting.");
        
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                // Wait until there's work to do or we're being cancelled
                await _signal.WaitAsync(stoppingToken);
                
                // Process the work item
                if (_workItems.TryDequeue(out WorkItem workItem))
                {
                    try
                    {
                        await ProcessWorkItemAsync(workItem, stoppingToken);
                    }
                    catch (Exception ex)
                    {
                        _logger.LogError(ex, "Error occurred while processing work item {WorkItemId}", workItem.Id);
                    }
                }
            }
            catch (OperationCanceledException)
            {
                // Graceful shutdown
                break;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error occurred in queue processing service");
                
                // Add a delay before retrying to avoid tight loops in error cases
                await Task.Delay(1000, stoppingToken);
            }
        }
        
        _logger.LogInformation("Queue Processing Service is stopping.");
    }
    
    private async Task ProcessWorkItemAsync(WorkItem workItem, CancellationToken stoppingToken)
    {
        _logger.LogInformation("Processing work item {WorkItemId}", workItem.Id);
        
        // Simulate processing time
        await Task.Delay(workItem.ProcessingTime, stoppingToken);
        
        _logger.LogInformation("Completed work item {WorkItemId}", workItem.Id);
    }
}

public class WorkItem
{
    public Guid Id { get; } = Guid.NewGuid();
    public int ProcessingTime { get; set; } = 1000; // milliseconds
    public string Data { get; set; }
}

To use this service:

csharp
// In Startup.ConfigureServices or Program.cs
services.AddSingleton<QueueProcessingService>();
services.AddHostedService(provider => provider.GetRequiredService<QueueProcessingService>());

// Then inject and use it in your controllers or other services
public class MyController : ControllerBase
{
    private readonly QueueProcessingService _queueService;
    
    public MyController(QueueProcessingService queueService)
    {
        _queueService = queueService;
    }
    
    [HttpPost("process")]
    public IActionResult EnqueueWork([FromBody] string data)
    {
        var workItem = new WorkItem
        {
            Data = data,
            ProcessingTime = 5000 // 5 seconds
        };
        
        _queueService.QueueWorkItem(workItem);
        
        return Accepted(new { id = workItem.Id });
    }
}

Advanced Patterns: Worker Pools

For more complex scenarios, you might need a pool of workers to process tasks in parallel. Here's a simple implementation of a worker pool:

csharp
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

public class WorkerPoolService : BackgroundService
{
    private readonly ILogger<WorkerPoolService> _logger;
    private readonly ConcurrentQueue<Func<CancellationToken, Task>> _workItems = new ConcurrentQueue<Func<CancellationToken, Task>>();
    private readonly SemaphoreSlim _signal = new SemaphoreSlim(0);
    private readonly int _maxDegreeOfParallelism;
    
    public WorkerPoolService(ILogger<WorkerPoolService> logger, int maxDegreeOfParallelism = 4)
    {
        _logger = logger;
        _maxDegreeOfParallelism = maxDegreeOfParallelism;
    }
    
    public void QueueTask(Func<CancellationToken, Task> workItem)
    {
        _workItems.Enqueue(workItem);
        _signal.Release();
    }
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Worker Pool Service is starting with {Count} workers.", _maxDegreeOfParallelism);
        
        // Create multiple worker tasks
        var workers = new Task[_maxDegreeOfParallelism];
        
        for (int i = 0; i < _maxDegreeOfParallelism; i++)
        {
            int workerId = i; // Capture for logging
            workers[i] = WorkerAsync(workerId, stoppingToken);
        }
        
        // Wait for all workers to complete
        await Task.WhenAll(workers);
        
        _logger.LogInformation("Worker Pool Service is stopping.");
    }
    
    private async Task WorkerAsync(int workerId, CancellationToken stoppingToken)
    {
        _logger.LogInformation("Worker {WorkerId} starting", workerId);
        
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await _signal.WaitAsync(stoppingToken);
                
                if (_workItems.TryDequeue(out var workItem))
                {
                    _logger.LogDebug("Worker {WorkerId} processing a task", workerId);
                    
                    try
                    {
                        await workItem(stoppingToken);
                    }
                    catch (Exception ex) when (ex is not OperationCanceledException)
                    {
                        _logger.LogError(ex, "Worker {WorkerId} encountered an error", workerId);
                    }
                    
                    _logger.LogDebug("Worker {WorkerId} completed a task", workerId);
                }
            }
            catch (OperationCanceledException)
            {
                break;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Worker {WorkerId} encountered an unexpected error", workerId);
                await Task.Delay(1000, stoppingToken);
            }
        }
        
        _logger.LogInformation("Worker {WorkerId} stopping", workerId);
    }
}

Best Practices

When working with background workers in .NET, follow these best practices:

Error handling: Always catch and log exceptions in your background workers to prevent silent failures.
Graceful shutdown: Implement proper cancellation handling to ensure your workers can be stopped cleanly when the application shuts down.
Resource management: Be mindful of memory and thread usage, especially in long-running services.
Stateless design: Design your background services to be as stateless as possible to avoid issues with stale data.
Configurable intervals: Make timing parameters configurable rather than hardcoded.
Scalability: Consider scalability requirements when designing your background processing system.
Monitoring: Implement proper logging and monitoring to track performance and detect issues.
Testing: Write unit tests for your background workers, mocking any dependencies.

Common Pitfalls to Avoid

Blocking the main thread: Never block the main thread with synchronous calls in background processing.
Infinite loops without delays: Always include delays in polling loops to avoid CPU spikes.
Ignoring cancellation tokens: Always respect cancellation tokens to enable clean shutdown.
Unhandled exceptions: Uncaught exceptions can crash your worker or even your entire application.
Thread safety issues: Be careful with shared state in background workers.

Summary

In this tutorial, we've covered several approaches to background processing in .NET:

The classic BackgroundWorker component for desktop applications
Modern BackgroundService and IHostedService patterns for ASP.NET Core and console apps
Advanced patterns like worker pools for parallel processing

These tools allow you to build responsive, efficient applications that can handle long-running tasks without blocking the main application flow. By choosing the right background processing pattern for your needs, you can significantly improve user experience and application performance.

Further Resources

Exercises

Create a Windows Forms application that uses BackgroundWorker to download multiple files simultaneously with progress reporting.
Implement a BackgroundService in an ASP.NET Core application that periodically checks for expired user sessions and cleans them up.
Build a worker pool service that processes items from a database queue, with configurable retry logic for failed items.
Design a background service that monitors a directory for new files and processes them as they arrive.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

Introduction​

Why Use Background Workers?​

The Classic BackgroundWorker Component​

Basic Implementation​

When to Use BackgroundWorker​

Modern Background Processing: IHostedService and BackgroundService​

IHostedService Interface​

Using BackgroundService Abstract Class​

Registering the Background Service​

When to Use BackgroundService​

Practical Example: Processing Queue Items​

Advanced Patterns: Worker Pools​

Best Practices​

Common Pitfalls to Avoid​

Summary​

Further Resources​

Exercises​