Nginx CPU Affinity
Introduction
When running high-traffic web servers, getting the most out of your hardware becomes crucial. One powerful optimization technique for Nginx is CPU affinity (also called CPU pinning), which allows you to bind specific Nginx worker processes to particular CPU cores.
By default, the operating system decides which CPU core runs each process, potentially moving processes between cores. This can cause cache misses and context switching overhead. CPU affinity gives you control over this process distribution, potentially improving performance in high-load environments.
What is CPU Affinity?
CPU affinity refers to the ability to bind or "pin" a process to one or more specific CPU cores, ensuring that:
- The process runs only on the designated core(s)
- The CPU cache is used more efficiently
- Context switching overhead is reduced
Configuring CPU Affinity in Nginx
Nginx provides the worker_cpu_affinity
directive to control which CPU cores each worker process can use. This directive is only available in the Nginx main context within your configuration.
Basic Syntax
worker_processes 4;
worker_cpu_affinity 0001 0010 0100 1000;
In this example:
- We set up 4 worker processes
- Each binary mask (0001, 0010, 0100, 1000) assigns one worker to one specific CPU core
The binary mask representation works as follows:
- Reading from right to left, each position represents a CPU core (Core 0, Core 1, etc.)
- A
1
means the worker can use that core - A
0
means the worker cannot use that core
Practical Example 1: One-to-One Mapping
For a server with 4 CPU cores:
# One worker per CPU core
worker_processes 4;
worker_cpu_affinity 0001 0010 0100 1000;
This configuration:
- Worker 1 runs on CPU core 0 (0001)
- Worker 2 runs on CPU core 1 (0010)
- Worker 3 runs on CPU core 2 (0100)
- Worker 4 runs on CPU core 3 (1000)
Practical Example 2: Assigning Multiple Cores
You can assign multiple cores to a worker by setting multiple bits to 1:
# Two workers, each with access to 2 cores
worker_processes 2;
worker_cpu_affinity 0011 1100;
This configuration:
- Worker 1 can use CPU cores 0 and 1 (0011)
- Worker 2 can use CPU cores 2 and 3 (1100)
Simplified Syntax for Many Cores
For systems with many CPU cores, Nginx offers a simplified syntax:
# For an 8-core system
worker_processes 8;
worker_cpu_affinity auto;
The auto
parameter automatically distributes workers across all available cores.
When to Use CPU Affinity
CPU affinity is most beneficial in these scenarios:
- High-traffic production servers where maximum performance is critical
- NUMA (Non-Uniform Memory Access) architectures where memory access times depend on memory location relative to the processor
- When you need deterministic performance with predictable latency
It may not be necessary for:
- Development environments
- Low-traffic servers
- Virtual machines with limited resources
Implementation Steps
Let's walk through a complete implementation:
Step 1: Determine your CPU core count
# Linux
nproc
# or
grep -c processor /proc/cpuinfo
# FreeBSD/macOS
sysctl -n hw.ncpu
Output example:
8
Step 2: Configure Nginx
Edit your nginx.conf
file:
sudo nano /etc/nginx/nginx.conf
Add or modify these directives:
worker_processes 8; # Match your core count
worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;
Step 3: Test and reload Nginx
sudo nginx -t
sudo nginx -s reload
Expected output:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Advanced Considerations
Reserving Cores for System Tasks
In some cases, it's beneficial to reserve some CPU cores for the operating system and other processes:
# 8-core system, reserve core 0 for the OS
worker_processes 7;
worker_cpu_affinity 00000010 00000100 00001000 00010000 00100000 01000000 10000000;
Combining with Process Priority
For even more control, combine CPU affinity with the worker_priority
directive:
worker_processes 4;
worker_cpu_affinity 0001 0010 0100 1000;
worker_priority -5; # Higher priority (lower number)
Real-world Tuning Example
For a server handling primarily CPU-bound SSL traffic:
worker_processes 8;
worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;
worker_priority -10;
worker_connections 4096;
ssl_session_cache shared:SSL:50m;
Monitoring and Validating CPU Affinity
To verify your CPU affinity settings are working:
# Get Nginx worker PIDs
ps aux | grep nginx
# Check CPU affinity for a specific PID
taskset -p [PID]
Example output:
pid 12345's current affinity mask: 1
This confirms the process is bound to CPU core 0.
Performance Benchmarking
Always benchmark before and after CPU affinity changes. A simple benchmark with ab
(Apache Benchmark):
# Before CPU affinity
ab -n 10000 -c 100 https://your-server/
# After CPU affinity
ab -n 10000 -c 100 https://your-server/
Compare the requests per second and response times.
Common Pitfalls
- Over-optimization: Excessive tuning can sometimes degrade performance
- Ignoring workload characteristics: CPU-bound vs I/O-bound workloads need different strategies
- Not accounting for hyperthreading: Logical cores ≠ physical cores
Summary
CPU affinity in Nginx gives you precise control over which CPU cores handle your web server's workload. By properly configuring CPU affinity, you can:
- Improve cache utilization
- Reduce context switching
- Optimize performance for specific workloads
- Achieve more predictable performance characteristics
Remember that CPU affinity is just one aspect of server optimization. It works best when combined with other tuning parameters and proper monitoring.
Additional Resources
- Nginx Official Documentation
- Linux commands:
taskset
,mpstat
, andhtop
for CPU monitoring - Consider learning about related concepts like NUMA architecture and processor cache hierarchy
Practice Exercises
- Configure CPU affinity for a 4-core system, mapping each worker to a single core
- Create a configuration for an 8-core system where you reserve 2 cores for the operating system
- Set up CPU affinity for a dual-socket NUMA system (advanced)
- Benchmark your server before and after implementing CPU affinity to measure performance impact
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)