.NET Parallel Loops

Introduction

When processing large collections or performing CPU-intensive operations, sequential execution can be slow and inefficient. .NET provides powerful parallel programming capabilities through the Task Parallel Library (TPL), allowing you to execute loop iterations concurrently across multiple processor cores.

Parallel loops are one of the easiest ways to improve performance in your applications, especially when you have independent iterations that can safely run at the same time. In this tutorial, we'll explore how to use parallel loops in .NET to speed up your data processing tasks.

Prerequisites

Basic knowledge of C# and .NET
Understanding of loops in C# (for, foreach)
.NET 6.0 or later (though parallel loops work with .NET Framework 4.0+)

Understanding Parallel Loops

In traditional sequential loops, each iteration executes one after another:

csharp
for (int i = 0; i < 1000; i++)
{
    // Process item i
}

Parallel loops, on the other hand, distribute the workload across multiple threads:

csharp
Parallel.For(0, 1000, i =>
{
    // Process item i in parallel
});

The TPL handles the complexity of thread creation, scheduling, and synchronization, allowing you to focus on the business logic.

Basic Parallel Loop Examples

Let's start with some simple examples to understand the syntax and behavior of parallel loops.

Parallel.For

The Parallel.For method is similar to a standard for loop but executes iterations in parallel:

csharp
using System;
using System.Threading;
using System.Threading.Tasks;

public class ParallelForExample
{
    public static void BasicParallelFor()
    {
        Console.WriteLine("Starting parallel processing...");
        
        Parallel.For(0, 10, (int i) =>
        {
            Console.WriteLine($"Processing item {i} on thread {Thread.CurrentThread.ManagedThreadId}");
            // Simulate work
            Thread.Sleep(100);
        });
        
        Console.WriteLine("Parallel processing completed.");
    }
}

Output:

Starting parallel processing...
Processing item 1 on thread 4
Processing item 0 on thread 1
Processing item 3 on thread 5
Processing item 2 on thread 6
Processing item 4 on thread 7
Processing item 5 on thread 8
Processing item 6 on thread 9
Processing item 7 on thread 10
Processing item 8 on thread 11
Processing item 9 on thread 12
Parallel processing completed.

Note that the output order may vary between runs because iterations execute in parallel.

Parallel.ForEach

Parallel.ForEach is the parallel equivalent of the foreach loop:

csharp
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

public class ParallelForEachExample
{
    public static void BasicParallelForEach()
    {
        List<string> items = new List<string> { "Item1", "Item2", "Item3", "Item4", "Item5" };
        
        Console.WriteLine("Starting parallel foreach...");
        
        Parallel.ForEach(items, item =>
        {
            Console.WriteLine($"Processing {item} on thread {Thread.CurrentThread.ManagedThreadId}");
            // Simulate work
            Thread.Sleep(100);
        });
        
        Console.WriteLine("Parallel foreach completed.");
    }
}

Output:

Starting parallel foreach...
Processing Item1 on thread 1
Processing Item2 on thread 4
Processing Item3 on thread 5
Processing Item4 on thread 6
Processing Item5 on thread 7
Parallel foreach completed.

Performance Benefits

To demonstrate the performance benefits, let's compare sequential and parallel processing of a CPU-intensive task:

csharp
using System;
using System.Diagnostics;
using System.Threading.Tasks;

public class PerformanceComparison
{
    public static void ComparePerformance()
    {
        int arraySize = 10000000;
        double[] numbers = new double[arraySize];
        
        // Initialize the array
        for (int i = 0; i < arraySize; i++)
        {
            numbers[i] = i;
        }
        
        // Sequential processing
        Stopwatch sequentialSw = Stopwatch.StartNew();
        for (int i = 0; i < arraySize; i++)
        {
            numbers[i] = Math.Sqrt(numbers[i]);
        }
        sequentialSw.Stop();
        
        // Reset array
        for (int i = 0; i < arraySize; i++)
        {
            numbers[i] = i;
        }
        
        // Parallel processing
        Stopwatch parallelSw = Stopwatch.StartNew();
        Parallel.For(0, arraySize, i =>
        {
            numbers[i] = Math.Sqrt(numbers[i]);
        });
        parallelSw.Stop();
        
        Console.WriteLine($"Sequential processing time: {sequentialSw.ElapsedMilliseconds}ms");
        Console.WriteLine($"Parallel processing time: {parallelSw.ElapsedMilliseconds}ms");
        Console.WriteLine($"Performance improvement: {(double)sequentialSw.ElapsedMilliseconds / parallelSw.ElapsedMilliseconds:F2}x");
    }
}

Output (will vary based on your hardware):

Sequential processing time: 542ms
Parallel processing time: 112ms
Performance improvement: 4.84x

On a modern multi-core processor, you can expect significant performance improvements, especially for CPU-bound operations.

Advanced Parallel Loop Features

Controlling Parallelism

You can control the degree of parallelism using ParallelOptions:

csharp
using System;
using System.Threading;
using System.Threading.Tasks;

public class ParallelOptions
{
    public static void LimitedParallelism()
    {
        // Limit to 2 concurrent tasks
        ParallelOptions options = new ParallelOptions
        {
            MaxDegreeOfParallelism = 2
        };
        
        Console.WriteLine("Starting limited parallel processing...");
        
        Parallel.For(0, 10, options, i =>
        {
            Console.WriteLine($"Processing item {i} on thread {Thread.CurrentThread.ManagedThreadId}");
            Thread.Sleep(100);
        });
        
        Console.WriteLine("Limited parallel processing completed.");
    }
}

Cancellation Support

You can cancel parallel operations using a CancellationToken:

csharp
using System;
using System.Threading;
using System.Threading.Tasks;

public class CancellationExample
{
    public static void CancellableParallelLoop()
    {
        CancellationTokenSource cts = new CancellationTokenSource();
        
        // Cancel after 500ms
        cts.CancelAfter(500);
        
        ParallelOptions options = new ParallelOptions
        {
            CancellationToken = cts.Token
        };
        
        try
        {
            Parallel.For(0, 1000, options, i =>
            {
                Console.WriteLine($"Processing item {i}");
                Thread.Sleep(100); // Simulate work
                
                // Optionally check for cancellation manually
                options.CancellationToken.ThrowIfCancellationRequested();
            });
        }
        catch (OperationCanceledException)
        {
            Console.WriteLine("Operation was canceled!");
        }
    }
}

Loop State and Early Termination

You can break out of parallel loops early:

csharp
using System;
using System.Threading.Tasks;

public class EarlyTermination
{
    public static void BreakParallelLoop()
    {
        Console.WriteLine("Starting parallel loop with early termination...");
        
        Parallel.For(0, 1000, (int i, ParallelLoopState loopState) =>
        {
            if (i >= 10)
            {
                Console.WriteLine($"Breaking at iteration {i}");
                loopState.Break(); // Stop processing new items
                return;
            }
            
            Console.WriteLine($"Processing iteration {i}");
        });
        
        Console.WriteLine("Parallel loop completed.");
    }
}

Real-World Example: Parallel Image Processing

Let's look at a practical example where parallel loops can be useful: image processing. In this example, we'll apply a simple grayscale filter to an image:

csharp
using System;
using System.Diagnostics;
using System.Drawing;
using System.Threading.Tasks;

public class ImageProcessing
{
    public static void ParallelImageProcessing(string inputPath, string outputPath)
    {
        using (Bitmap originalImage = new Bitmap(inputPath))
        {
            Bitmap processedImage = new Bitmap(originalImage.Width, originalImage.Height);
            
            // Sequential processing
            Stopwatch sequentialSw = Stopwatch.StartNew();
            for (int x = 0; x < originalImage.Width; x++)
            {
                for (int y = 0; y < originalImage.Height; y++)
                {
                    Color originalPixel = originalImage.GetPixel(x, y);
                    int grayValue = (int)(originalPixel.R * 0.3 + originalPixel.G * 0.59 + originalPixel.B * 0.11);
                    Color grayPixel = Color.FromArgb(originalPixel.A, grayValue, grayValue, grayValue);
                    processedImage.SetPixel(x, y, grayPixel);
                }
            }
            sequentialSw.Stop();
            processedImage.Save(outputPath.Replace(".jpg", "_sequential.jpg"));
            
            // Parallel processing
            processedImage = new Bitmap(originalImage.Width, originalImage.Height);
            Stopwatch parallelSw = Stopwatch.StartNew();
            
            Parallel.For(0, originalImage.Width, x =>
            {
                for (int y = 0; y < originalImage.Height; y++)
                {
                    Color originalPixel = originalImage.GetPixel(x, y);
                    int grayValue = (int)(originalPixel.R * 0.3 + originalPixel.G * 0.59 + originalPixel.B * 0.11);
                    Color grayPixel = Color.FromArgb(originalPixel.A, grayValue, grayValue, grayValue);
                    processedImage.SetPixel(x, y, grayPixel);
                }
            });
            
            parallelSw.Stop();
            processedImage.Save(outputPath);
            
            Console.WriteLine($"Sequential processing time: {sequentialSw.ElapsedMilliseconds}ms");
            Console.WriteLine($"Parallel processing time: {parallelSw.ElapsedMilliseconds}ms");
            Console.WriteLine($"Performance improvement: {(double)sequentialSw.ElapsedMilliseconds / parallelSw.ElapsedMilliseconds:F2}x");
        }
    }
}

note

For production image processing, consider using libraries that support hardware acceleration or low-level pixel manipulation like unsafe code with pointers, as the GetPixel/SetPixel methods have significant overhead.

Best Practices and Considerations

When to Use Parallel Loops

Parallel loops are most effective when:

You have CPU-intensive operations
Loop iterations are independent (no interdependencies)
The work per iteration is substantial (to overcome threading overhead)
Your data size is large enough to benefit from parallelism

When to Avoid Parallel Loops

Avoid parallel loops when:

Iterations are IO-bound (use async/await instead)
Iterations depend on each other
The work per iteration is very small
You need deterministic ordering

Thread Safety Considerations

Be careful with shared resources in parallel loops:

csharp
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

public class ThreadSafety
{
    public static void DemonstrateProblem()
    {
        List<int> results = new List<int>();
        
        // INCORRECT: Not thread-safe
        Parallel.For(0, 100, i =>
        {
            results.Add(i * 2); // Race condition!
        });
        
        Console.WriteLine($"Expected 100 items, got {results.Count}"); // Likely less than 100
    }
    
    public static void DemonstrateSolution()
    {
        // Option 1: Use thread-safe collection
        var threadSafeList = new System.Collections.Concurrent.ConcurrentBag<int>();
        
        Parallel.For(0, 100, i =>
        {
            threadSafeList.Add(i * 2); // Thread-safe
        });
        
        Console.WriteLine($"Thread-safe collection has {threadSafeList.Count} items");
        
        // Option 2: Use local aggregation with ToArray
        var result = Parallel.ForAll(Enumerable.Range(0, 100), i => i * 2);
        
        Console.WriteLine($"Local aggregation has {result.Length} items");
    }
}

Summary

Parallel loops in .NET provide an easy way to improve performance by utilizing multiple CPU cores. Key points to remember:

Use Parallel.For and Parallel.ForEach for simple parallelization
Consider the independence of iterations and overhead of threading
Use thread-safe collections and techniques when working with shared state
Use ParallelOptions to control parallelism and support cancellation
Parallel loops are best for CPU-bound operations on reasonably large datasets

With these tools, you can make your .NET applications more responsive and efficient, especially when processing large amounts of data.

Exercises

Create a program that generates prime numbers up to 10 million using both sequential and parallel approaches, and compare their performance.
Implement a parallel word counter that processes multiple text files concurrently and aggregates the results.
Create a simple image batch processor that applies various filters to multiple images in parallel.
Implement a custom partitioner for a parallel loop that divides work more efficiently for uneven workloads.

Additional Resources

Happy coding with parallel loops!

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

Introduction​

Prerequisites​

Understanding Parallel Loops​

Basic Parallel Loop Examples​

Parallel.For​

Parallel.ForEach​

Performance Benefits​

Advanced Parallel Loop Features​

Controlling Parallelism​

Cancellation Support​

Loop State and Early Termination​

Real-World Example: Parallel Image Processing​

Best Practices and Considerations​

When to Use Parallel Loops​

When to Avoid Parallel Loops​

Thread Safety Considerations​

Summary​

Exercises​

Additional Resources​