Express Memory Leaks

Memory leaks in Express applications can cause your server to slow down over time and eventually crash. Understanding how to identify and fix these issues is critical for maintaining performant and reliable web applications.

What is a Memory Leak?

A memory leak occurs when your application allocates memory but fails to release it when it's no longer needed. In JavaScript and Node.js, this typically happens when objects are unintentionally kept in memory even after they should be garbage collected.

In Express applications, memory leaks are particularly problematic because:

Servers are expected to run for long periods without restarting
Each user request adds to memory usage
Under high load, memory leaks can cause catastrophic failures

Common Causes of Memory Leaks in Express

1. Unbounded Caches

javascript
// PROBLEM: This cache will grow indefinitely
const requestCache = {};

app.get('/api/data/:id', (req, res) => {
  const id = req.params.id;
  
  if (requestCache[id]) {
    return res.json(requestCache[id]);
  }
  
  // Fetch data
  const data = fetchData(id);
  // Store in cache without limits
  requestCache[id] = data;
  
  return res.json(data);
});

2. Event Listeners Not Being Removed

javascript
// PROBLEM: Event listeners pile up
const db = require('./database');

app.get('/api/listen', (req, res) => {
  // This adds a new listener on EVERY request
  db.on('update', (data) => {
    // Handle update
    console.log('Database updated', data);
  });
  
  res.send('Listening for updates');
});

3. Closures Capturing Large Objects

javascript
// PROBLEM: Large data captured in closure
app.get('/api/process', (req, res) => {
  const hugeData = loadHugeDataSet(); // Several MB of data
  
  const intervalId = setInterval(() => {
    // This closure captures hugeData in memory
    processChunk(hugeData);
    
    // If we forget to clear the interval, hugeData stays in memory
  }, 1000);
  
  res.send('Processing started');
});

4. Global Variables Growing Over Time

javascript
// PROBLEM: Global array that only grows
const allRequests = [];

app.use((req, res, next) => {
  // Track all requests
  allRequests.push({
    url: req.url,
    method: req.method,
    time: Date.now(),
    body: req.body // This could be large!
  });
  
  next();
});

How to Detect Memory Leaks

Using Built-in Node.js Tools

Node.js provides tools to check memory usage:

javascript
// Add this endpoint to check memory usage
app.get('/debug/memory', (req, res) => {
  const memoryUsage = process.memoryUsage();
  
  res.json({
    rss: `${Math.round(memoryUsage.rss / 1024 / 1024)} MB`,
    heapTotal: `${Math.round(memoryUsage.heapTotal / 1024 / 1024)} MB`,
    heapUsed: `${Math.round(memoryUsage.heapUsed / 1024 / 1024)} MB`,
    external: `${Math.round(memoryUsage.external / 1024 / 1024)} MB`
  });
});

Using External Tools

There are several tools you can use to monitor memory usage:

Node.js Inspector: Use Chrome DevTools to analyze heap snapshots
Clinic.js: A suite of tools for diagnosing performance issues
PM2: Process manager with memory monitoring

Fixing Common Memory Leaks

1. Implement Bounded Caches

javascript
// SOLUTION: Using a bounded LRU cache
const LRU = require('lru-cache');
const cache = new LRU({
  max: 500, // Maximum items
  maxAge: 1000 * 60 * 60 // Items expire after 1 hour
});

app.get('/api/data/:id', (req, res) => {
  const id = req.params.id;
  
  if (cache.has(id)) {
    return res.json(cache.get(id));
  }
  
  // Fetch data
  const data = fetchData(id);
  // Store in bounded cache
  cache.set(id, data);
  
  return res.json(data);
});

2. Clean Up Event Listeners

javascript
// SOLUTION: Remove event listeners when done
const db = require('./database');

app.get('/api/listen', (req, res) => {
  const handler = (data) => {
    // Handle update
    console.log('Database updated', data);
  };
  
  db.on('update', handler);
  
  // Set timeout to remove listener after some time
  setTimeout(() => {
    db.removeListener('update', handler);
  }, 60000); // Remove after 1 minute
  
  res.send('Listening for updates (for 1 minute only)');
});

3. Avoid Capturing Large Objects in Closures

javascript
// SOLUTION: Process data then release it
app.get('/api/process', (req, res) => {
  // Start a background job without keeping data in memory
  startProcessingJob(req.params.jobId)
    .then(() => {
      console.log('Job completed');
    });
  
  res.send('Processing started');
});

// This function doesn't keep the data in a closure
function startProcessingJob(jobId) {
  return new Promise((resolve) => {
    const intervalId = setInterval(() => {
      // Load only the data we need for this chunk
      const dataChunk = loadNextChunk(jobId);
      
      if (!dataChunk) {
        clearInterval(intervalId);
        resolve();
        return;
      }
      
      processChunk(dataChunk);
      // dataChunk goes out of scope here
    }, 1000);
  });
}

4. Limit Collection Sizes

javascript
// SOLUTION: Limit the collection size
const allRequests = [];
const MAX_REQUESTS = 1000;

app.use((req, res, next) => {
  // Only keep essential information
  allRequests.push({
    url: req.url,
    method: req.method,
    time: Date.now()
    // Don't store the body!
  });
  
  // Remove old entries if we exceed our limit
  if (allRequests.length > MAX_REQUESTS) {
    allRequests.shift(); // Remove oldest entry
  }
  
  next();
});

Real-World Example: API Server with Memory Leak Prevention

Let's build a small but complete Express API server that implements memory leak prevention techniques:

javascript
const express = require('express');
const LRU = require('lru-cache');
const EventEmitter = require('events');

const app = express();
app.use(express.json());

// Create bounded cache
const userCache = new LRU({
  max: 1000,
  maxAge: 1000 * 60 * 15 // 15 minutes
});

// Event bus with leak protection
class SafeEventBus extends EventEmitter {
  constructor() {
    super();
    // Set a maximum number of listeners to detect potential leaks
    this.setMaxListeners(20);
  }
}

const eventBus = new SafeEventBus();

// Track active connections for cleanup
const activeConnections = new Set();

// Monitor memory and emit warning at thresholds
function checkMemory() {
  const memUsage = process.memoryUsage().heapUsed / 1024 / 1024;
  console.log(`Memory usage: ${Math.round(memUsage)} MB`);
  
  if (memUsage > 1024) { // 1GB
    console.error('CRITICAL: Memory usage too high, potential leak');
    // In production, you might want to notify your monitoring system
  }
}

// Check memory usage every 30 seconds
setInterval(checkMemory, 30000);

// API endpoints
app.get('/api/users/:id', (req, res) => {
  const userId = req.params.id;
  
  if (userCache.has(userId)) {
    return res.json(userCache.get(userId));
  }
  
  // Simulate database call
  setTimeout(() => {
    const user = { 
      id: userId, 
      name: `User ${userId}`, 
      email: `user${userId}@example.com` 
    };
    
    userCache.set(userId, user);
    res.json(user);
  }, 100);
});

// Example of SSE (Server-Sent Events) with proper cleanup
app.get('/api/events', (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  
  // Send initial message
  res.write('data: Connected to event stream\n\n');
  
  // Create event handler
  const eventHandler = (event) => {
    res.write(`data: ${JSON.stringify(event)}\n\n`);
  };
  
  // Register handler
  eventBus.on('newEvent', eventHandler);
  
  // Track this connection
  activeConnections.add(res);
  
  // Clean up when client disconnects
  req.on('close', () => {
    eventBus.removeListener('newEvent', eventHandler);
    activeConnections.delete(res);
    res.end();
  });
});

// Simulate event generation
setInterval(() => {
  eventBus.emit('newEvent', { 
    time: new Date().toISOString(),
    message: 'Periodic update'
  });
}, 10000);

// Graceful shutdown to prevent memory leaks
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);

function shutdown() {
  console.log('Shutting down gracefully...');
  
  // Close all active connections
  for (const connection of activeConnections) {
    connection.end();
  }
  activeConnections.clear();
  
  // Clear cache
  userCache.reset();
  
  // Remove all event listeners
  eventBus.removeAllListeners();
  
  console.log('Cleanup complete, exiting');
  process.exit(0);
}

// Start server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

Best Practices Summary

Use Bounded Caches: Always set limits on your caches using libraries like lru-cache
Clean Up Event Listeners: Always remove event listeners when they're no longer needed
Watch Closure Scope: Be careful not to capture large objects in closures, especially in timers
Limit Collection Sizes: For logging or tracking, always limit array or object sizes
Use WeakMaps/WeakSets: For associating data with objects without preventing garbage collection
Monitor Memory Usage: Set up regular memory usage checks
Implement Graceful Shutdown: Clean up resources when your app terminates
Set Memory Limits: Use the --max-old-space-size flag when starting Node.js

Memory Profiling in Production

For production environments, consider implementing a memory profiling endpoint that's only accessible to administrators:

javascript
// SECURITY NOTE: Protect this endpoint in production!
app.get('/admin/heap-snapshot', authenticateAdmin, (req, res) => {
  const heapdump = require('heapdump');
  const filename = `/tmp/heap-${Date.now()}.heapsnapshot`;
  
  heapdump.writeSnapshot(filename, (err) => {
    if (err) {
      return res.status(500).send('Failed to generate heap snapshot');
    }
    
    res.download(filename, () => {
      // Delete file after download
      require('fs').unlink(filename, () => {});
    });
  });
});

Summary

Memory leaks in Express.js applications can significantly impact performance and reliability. By understanding common causes and implementing preventative measures, you can keep your applications running smoothly even under heavy load.

Remember the key principles:

Set boundaries on all data storage
Clean up resources when they're no longer needed
Monitor memory usage regularly
Implement graceful shutdown procedures

With these practices, your Express applications will be more resilient and performant in production environments.

Additional Resources

Exercises

Create an Express application and implement a memory leak on purpose (e.g., unbounded cache), then use Node.js memory profiling tools to identify the leak.
Convert an existing Express route that uses a simple object as a cache to use lru-cache instead.
Write a middleware that monitors request counts and response times without causing a memory leak.
Implement a graceful shutdown procedure for an Express application that cleans up resources properly.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

What is a Memory Leak?​

Common Causes of Memory Leaks in Express​

1. Unbounded Caches​

2. Event Listeners Not Being Removed​

3. Closures Capturing Large Objects​

4. Global Variables Growing Over Time​

How to Detect Memory Leaks​

Using Built-in Node.js Tools​

Using External Tools​

Fixing Common Memory Leaks​

1. Implement Bounded Caches​

2. Clean Up Event Listeners​

3. Avoid Capturing Large Objects in Closures​

4. Limit Collection Sizes​

Real-World Example: API Server with Memory Leak Prevention​

Best Practices Summary​

Memory Profiling in Production​

Summary​

Additional Resources​

Exercises​