Express Graceful Shutdown
When running Express.js applications in production, properly handling the application shutdown process is critical. A "graceful shutdown" ensures that your server completes any in-flight requests, closes database connections properly, and releases resources before exiting. This prevents data loss, connection leaks, and provides a better user experience.
Why Graceful Shutdown Matters
Imagine your Express application is handling hundreds of user requests when suddenly it needs to be restarted for a deployment. Without proper shutdown handling:
- Active HTTP requests might be terminated abruptly
- Database transactions might be left incomplete
- File uploads could be interrupted
- Connection pools might leak
- Users would experience errors
By implementing graceful shutdown patterns, you can mitigate these issues and ensure a smooth experience during deployments or server restarts.
Understanding Process Signals
Node.js processes respond to various system signals that can trigger shutdown:
SIGTERM- The standard signal for termination, sent by process managers like PM2, Docker, or KubernetesSIGINT- Sent when the user presses Ctrl+C in the terminalSIGUSR2- Often used by development tools like Nodemon for restartuncaughtException- When an unhandled error occurs
We'll create handlers for these signals to orchestrate our graceful shutdown.
Basic Graceful Shutdown Pattern
Let's implement a simple graceful shutdown pattern:
javascript
const express = require('express');
const app = express();
// Regular Express setup
app.get('/', (req, res) => {
res.send('Hello World!');
});
// Start the server
const server = app.listen(3000, () => {
console.log('Server running on port 3000');
});
// Graceful shutdown function
function gracefulShutdown(signal) {
console.log(`Received ${signal}, shutting down gracefully`);
// Stop accepting new connections
server.close(() => {
console.log('HTTP server closed');
// Additional cleanup logic would go here (database connections, etc)
console.log('Process terminating...');
process.exit(0);
});
// Force shutdown after 10 seconds if it cannot close gracefully
setTimeout(() => {
console.error('Could not close connections in time, forcefully shutting down');
process.exit(1);
}, 10000);
}
// Listen for termination signals
process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
process.on('SIGINT', () => gracefulShutdown('SIGINT'));
How This Works:
- We create an Express server as usual
- We save a reference to the server instance returned by
app.listen() - We define a
gracefulShutdownfunction that:- Logs the received signal
- Calls
server.close()to stop accepting new connections and wait for existing ones to finish - Sets a timeout that forces exit if connections don't close in time
- We register this function as a handler for both
SIGTERMandSIGINTsignals
This basic pattern already provides significant protection against abrupt shutdowns.
Advanced Graceful Shutdown with Database Connections
Most real-world applications need to handle database connections and other resources. Here's a more complete example with MongoDB:
javascript
const express = require('express');
const mongoose = require('mongoose');
const app = express();
// Connect to MongoDB
mongoose.connect('mongodb://localhost:27017/myapp')
.then(() => console.log('MongoDB connected'))
.catch(err => console.error('MongoDB connection error:', err));
app.get('/', (req, res) => {
res.send('Hello World!');
});
// Start the server
const server = app.listen(3000, () => {
console.log('Server running on port 3000');
});
// Track open connections
let connections = [];
server.on('connection', connection => {
connections.push(connection);
connection.on('close', () => {
connections = connections.filter(curr => curr !== connection);
});
});
// Graceful shutdown function
async function gracefulShutdown(signal) {
console.log(`Received ${signal}, shutting down gracefully`);
// Stop accepting new connections
server.close(() => {
console.log('HTTP server closed');
});
// Close MongoDB connection
try {
await mongoose.connection.close();
console.log('MongoDB connection closed');
} catch (err) {
console.error('Error closing MongoDB connection', err);
}
// Close any remaining connections
if (connections.length > 0) {
console.log(`Destroying ${connections.length} open connections`);
connections.forEach(connection => connection.destroy());
}
console.log('Process terminating...');
process.exit(0);
}
// Listen for termination signals
process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
process.on('SIGINT', () => gracefulShutdown('SIGINT'));
// Handle uncaught exceptions and rejections
process.on('uncaughtException', (error) => {
console.error('Uncaught Exception:', error);
gracefulShutdown('uncaughtException');
});
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled Rejection at:', promise, 'reason:', reason);
gracefulShutdown('unhandledRejection');
});
This implementation offers several improvements:
- Tracks and closes all open connections
- Properly closes the MongoDB connection
- Handles uncaught exceptions and unhandled promise rejections
Handling Long-Running Requests
Sometimes your Express application might have long-running requests, such as file uploads or complex database operations. Let's implement a strategy to handle them:
javascript
const express = require('express');
const app = express();
// Track active requests
let activeRequests = 0;
// Middleware to count active requests
app.use((req, res, next) => {
activeRequests++;
// Decrement counter when response finished
res.on('finish', () => {
activeRequests--;
});
next();
});
// Simulate a long-running request
app.get('/long-task', (req, res) => {
console.log('Starting long task...');
// Simulate work that takes 5 seconds
setTimeout(() => {
console.log('Long task completed');
res.send('Task completed');
}, 5000);
});
const server = app.listen(3000, () => {
console.log('Server running on port 3000');
});
async function gracefulShutdown(signal) {
console.log(`Received ${signal}, shutting down gracefully`);
// Stop accepting new connections
server.close(() => {
console.log('HTTP server closed');
});
// Wait for active requests to finish (with timeout)
if (activeRequests > 0) {
console.log(`Waiting for ${activeRequests} active requests to finish...`);
const checkInterval = setInterval(() => {
if (activeRequests === 0) {
clearInterval(checkInterval);
console.log('All requests finished, shutting down');
process.exit(0);
} else {
console.log(`Still waiting for ${activeRequests} requests...`);
}
}, 1000);
// Force exit after 30 seconds max
setTimeout(() => {
console.error('Could not finish all requests in time, forcefully shutting down');
process.exit(1);
}, 30000);
} else {
process.exit(0);
}
}
// Listen for termination signals
process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
process.on('SIGINT', () => gracefulShutdown('SIGINT'));
This implementation:
- Counts all active requests using a middleware
- When shutting down, it waits for active requests to complete
- Provides status updates during shutdown
- Forces termination after a maximum timeout period
Real-World Example: Complete Application with Multiple Resources
Here's a more comprehensive example that integrates all the concepts in a real-world scenario:
javascript
const express = require('express');
const mongoose = require('mongoose');
const Redis = require('ioredis');
const amqp = require('amqplib');
// Create Express app
const app = express();
// Track active requests
let activeRequests = 0;
app.use((req, res, next) => {
activeRequests++;
res.on('finish', () => { activeRequests--; });
next();
});
// Connect to databases and message queues
const redis = new Redis();
let rabbitmqChannel, mongoConnection;
// Initialize connections
async function initializeConnections() {
try {
// Connect MongoDB
mongoConnection = await mongoose.connect('mongodb://localhost:27017/myapp');
console.log('MongoDB connected');
// Connect RabbitMQ
const rabbitmqConnection = await amqp.connect('amqp://localhost');
rabbitmqChannel = await rabbitmqConnection.createChannel();
console.log('RabbitMQ connected');
} catch (error) {
console.error('Error initializing connections:', error);
process.exit(1);
}
}
// Set up routes
app.get('/', (req, res) => {
res.send('Server is running');
});
app.get('/data', async (req, res) => {
try {
// Simulate complex data operation
await new Promise(resolve => setTimeout(resolve, 500));
res.json({ message: 'Data retrieved successfully' });
} catch (error) {
res.status(500).json({ error: error.message });
}
});
// Server startup
async function startServer() {
await initializeConnections();
const server = app.listen(3000, () => {
console.log('Server running on port 3000');
});
// Track connections
let connections = [];
server.on('connection', connection => {
connections.push(connection);
connection.on('close', () => {
connections = connections.filter(curr => curr !== connection);
});
});
// Shutdown function
async function gracefulShutdown(signal) {
console.log(`\n${signal} received. Starting graceful shutdown...`);
// 1. Stop accepting new connections
server.close(() => {
console.log('✓ HTTP server closed to new connections');
});
// 2. Wait for active requests (with timeout)
if (activeRequests > 0) {
console.log(`Waiting for ${activeRequests} active requests to complete...`);
let shutdownInterval = setInterval(() => {
if (activeRequests === 0) {
clearInterval(shutdownInterval);
console.log('✓ All pending requests completed');
} else {
console.log(`Still waiting for ${activeRequests} active requests...`);
}
}, 1000);
// Wait maximum 30 seconds for requests to finish
await new Promise(resolve => setTimeout(resolve, 30000));
clearInterval(shutdownInterval);
}
// 3. Close all resource connections
try {
console.log('Closing database connections...');
// Close Redis connection
await redis.quit();
console.log('✓ Redis connection closed');
// Close RabbitMQ connection
if (rabbitmqChannel) {
await rabbitmqChannel.close();
console.log('✓ RabbitMQ connection closed');
}
// Close MongoDB connection
if (mongoose.connection.readyState !== 0) {
await mongoose.connection.close();
console.log('✓ MongoDB connection closed');
}
// 4. Close remaining sockets
if (connections.length > 0) {
console.log(`Destroying ${connections.length} socket connections...`);
connections.forEach(connection => connection.destroy());
console.log('✓ All connections destroyed');
}
console.log('Graceful shutdown completed');
process.exit(0);
} catch (error) {
console.error('Error during shutdown:', error);
process.exit(1);
}
}
// Register signal handlers
process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
process.on('SIGINT', () => gracefulShutdown('SIGINT'));
process.on('uncaughtException', (error) => {
console.error('Uncaught Exception:', error);
gracefulShutdown('uncaughtException');
});
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled Rejection at:', promise, 'reason:', reason);
gracefulShutdown('unhandledRejection');
});
}
// Start everything
startServer().catch(err => {
console.error('Failed to start server:', err);
process.exit(1);
});
This implementation:
- Connects to multiple services (MongoDB, Redis, RabbitMQ)
- Tracks active requests and open connections
- Implements a detailed shutdown sequence with proper cleanup
- Provides informative logging throughout the shutdown process
- Handles various termination signals
- Sets appropriate timeouts for various shutdown phases
Best Practices for Graceful Shutdown
When implementing graceful shutdown in Express, keep these best practices in mind:
- Always set timeouts: Never wait indefinitely for connections to close
- Log the shutdown process: Having detailed logs helps troubleshooting
- Close resources in the correct order: Some resources may depend on others
- Handle all termination signals: SIGTERM, SIGINT, uncaughtException, etc.
- Test your shutdown logic: Run tests that simulate abrupt shutdowns
- Use health checks: Integrate with container orchestrators like Kubernetes
- Add shutdown hooks in process managers: If using PM2, add shutdown hooks
Working with Process Managers and Containers
When running Express in production, you're likely using process managers like PM2 or containers like Docker. Here's how to ensure graceful shutdown works with them:
PM2 Configuration
javascript
// ecosystem.config.js
module.exports = {
apps: [{
name: "my-express-app",
script: "./server.js",
instances: "max",
exec_mode: "cluster",
kill_timeout: 5000, // Give the app time to close connections
wait_ready: true, // Wait for app to send 'ready' signal
listen_timeout: 30000 // Wait this many ms for the app to start
}]
};
Docker Configuration
dockerfile
FROM node:16-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
# Set signal handling for proper shutdown in Docker
STOPSIGNAL SIGTERM
CMD ["node", "server.js"]
Kubernetes Configuration
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: express-app
spec:
replicas: 3
template:
spec:
containers:
- name: express-app
image: my-express-app:latest
ports:
- containerPort: 3000
lifecycle:
preStop:
exec:
command: ["sh", "-c", "sleep 5"]
terminationGracePeriodSeconds: 30
Summary
Implementing graceful shutdown in Express applications is essential for production environments. It ensures:
- Ongoing requests are completed before the server stops
- Database connections and other resources are closed properly
- Server resources are released cleanly
- Users experience minimal disruption during deployments
By following the patterns shown in this guide, you can make your Express applications more resilient, maintain data integrity, and provide a better user experience even during restarts, deployments, or when recovering from errors.
Additional Resources
- Express.js Documentation
- Node.js Process Documentation
- PM2 Graceful Shutdown
- Kubernetes Pod Lifecycle
Exercises
- Implement a basic graceful shutdown for a simple Express server
- Add MongoDB connection handling to the shutdown process
- Create a solution that tracks and waits for long-running file upload requests
- Build a health check endpoint that reports if the server is shutting down
- Implement a solution that works with PM2 in cluster mode, ensuring all workers shutdown gracefully
By mastering graceful shutdown patterns, you'll be well-prepared to build robust, production-ready Express applications that can handle real-world operational challenges.
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)