Skip to main content

MongoDB Full Document Updates

When working with MongoDB Change Streams, one powerful feature is the ability to access the complete updated document rather than just the changes that occurred. This capability is essential for applications that need the entire context of a change rather than just the modified fields.

Introduction to Full Document Updates​

By default, MongoDB Change Streams provide information about what changed in a document but don't include the complete updated document. For many applications, having access to the full document after an update is crucial for properly processing the change event.

Consider a user profile update scenario: you might want to know not just that the user changed their email address, but also need access to all their current profile information to update a search index or send a notification with complete user details.

Enabling Full Document Access​

To retrieve the full updated document in a change stream, you need to specify the fullDocument option when creating the change stream.

javascript
const changeStream = collection.watch([], {
  fullDocument: 'updateLookup'
});

The updateLookup value instructs MongoDB to look up the current state of the document and include it in the change event.

Available fullDocument Options​

MongoDB provides several options for the fullDocument parameter:

  • 'default': Does not include the full document (default behavior)
  • 'updateLookup': Looks up the current state of the document
  • 'required' (MongoDB 6.0+): Similar to updateLookup but throws an error if document not found
  • 'whenAvailable' (MongoDB 6.0+): Returns the document if available

Basic Example: Tracking User Profile Updates​

Let's look at a practical example tracking user profile updates:

javascript
const { MongoClient } = require('mongodb');

async function monitorProfileUpdates() {
  const client = new MongoClient('mongodb://localhost:27017');
  
  try {
    await client.connect();
    const db = client.db('userDB');
    const users = db.collection('users');
    
    // Create a change stream with full document lookup
    const changeStream = users.watch([], {
      fullDocument: 'updateLookup'
    });
    
    console.log('Monitoring user profile updates...');
    
    // Process change events
    changeStream.on('change', (change) => {
      if (change.operationType === 'update') {
        console.log('User updated:');
        console.log(`ID: ${change.documentKey._id}`);
        console.log('Updated fields:', JSON.stringify(change.updateDescription.updatedFields));
        console.log('Current full profile:', JSON.stringify(change.fullDocument));
      }
    });
    
    // Keep the application running
    // In a real application, you'd handle process termination properly
  } catch (error) {
    console.error('Error:', error);
    await client.close();
  }
}

monitorProfileUpdates();

What happens when you run this code:​

  1. You connect to MongoDB and select the "users" collection
  2. You create a change stream with fullDocument: 'updateLookup'
  3. When a user document is updated, the change event includes:
    • Information about what changed (updateDescription.updatedFields)
    • The complete updated document (fullDocument)

Comparing Change Events With and Without Full Document​

Let's examine the difference in the output of change events with and without the full document option:

Without Full Document (default)​

javascript
// Change event structure without fullDocument option
{
  _id: { ... },
  operationType: 'update',
  ns: {
    db: 'userDB',
    coll: 'users'
  },
  documentKey: {
    _id: ObjectId('60a1e2f9f7834510b41d7a12')
  },
  updateDescription: {
    updatedFields: {
      email: '[email protected]',
      lastUpdated: ISODate('2023-09-15T12:30:45.123Z')
    },
    removedFields: []
  }
}

With Full Document (updateLookup)​

javascript
// Change event structure with fullDocument: 'updateLookup'
{
  _id: { ... },
  operationType: 'update',
  ns: {
    db: 'userDB',
    coll: 'users'
  },
  documentKey: {
    _id: ObjectId('60a1e2f9f7834510b41d7a12')
  },
  updateDescription: {
    updatedFields: {
      email: '[email protected]',
      lastUpdated: ISODate('2023-09-15T12:30:45.123Z')
    },
    removedFields: []
  },
  fullDocument: {
    _id: ObjectId('60a1e2f9f7834510b41d7a12'),
    username: 'johndoe',
    email: '[email protected]',
    firstName: 'John',
    lastName: 'Doe',
    createdAt: ISODate('2023-01-10T09:15:23.456Z'),
    lastUpdated: ISODate('2023-09-15T12:30:45.123Z'),
    preferences: { ... }
  }
}

Notice how the event with fullDocument includes the complete current state of the document after the update.

Accessing Previous Document States (MongoDB 6.0+)​

Starting with MongoDB 6.0, you can also access the document's state before the update using the fullDocumentBeforeChange option:

javascript
const changeStream = collection.watch([], {
  fullDocument: 'updateLookup',
  fullDocumentBeforeChange: 'required'
});

This provides both the previous and current states of the document, which is invaluable for tracking what exactly changed.

Example: Tracking Before and After States​

javascript
const changeStream = users.watch([], {
  fullDocument: 'updateLookup',
  fullDocumentBeforeChange: 'required'
});

changeStream.on('change', (change) => {
  if (change.operationType === 'update') {
    console.log('User updated:');
    console.log('Previous state:', JSON.stringify(change.fullDocumentBeforeChange));
    console.log('Current state:', JSON.stringify(change.fullDocument));
    
    // You can now perform detailed comparisons between old and new values
  }
});

Real-world Application: Audit Logging System​

Let's implement a more complete example: an audit logging system that tracks all changes to sensitive user data:

javascript
const { MongoClient } = require('mongodb');

async function setupAuditLogging() {
  const client = new MongoClient('mongodb://localhost:27017');
  
  try {
    await client.connect();
    const db = client.db('companyDB');
    const users = db.collection('employees');
    const auditLogs = db.collection('auditLogs');
    
    // Create a change stream with both before and after documents
    const changeStream = users.watch([], {
      fullDocument: 'updateLookup',
      fullDocumentBeforeChange: 'required' // Requires MongoDB 6.0+
    });
    
    console.log('Audit logging system active...');
    
    changeStream.on('change', async (change) => {
      if (change.operationType === 'update') {
        const sensitiveFields = ['salary', 'role', 'accessLevel', 'personalInfo'];
        const changedFields = Object.keys(change.updateDescription.updatedFields);
        
        // Check if any sensitive fields were modified
        const sensitiveFieldsChanged = changedFields.filter(field => 
          sensitiveFields.some(sf => field === sf || field.startsWith(`${sf}.`))
        );
        
        if (sensitiveFieldsChanged.length > 0) {
          // Create audit log entry
          await auditLogs.insertOne({
            timestamp: new Date(),
            employeeId: change.documentKey._id,
            changedFields: sensitiveFieldsChanged,
            oldValues: extractFieldValues(change.fullDocumentBeforeChange, sensitiveFieldsChanged),
            newValues: extractFieldValues(change.fullDocument, sensitiveFieldsChanged),
            userId: getCurrentUserId(), // In a real app, get the user who made the change
            action: 'update',
            collectionName: 'employees'
          });
          
          console.log(`Audit log created for employee ${change.documentKey._id}`);
        }
      }
    });
    
  } catch (error) {
    console.error('Error in audit system:', error);
    await client.close();
  }
}

// Helper function to extract specific field values from a document
function extractFieldValues(document, fieldPaths) {
  const result = {};
  
  for (const path of fieldPaths) {
    // Handle nested paths like 'personalInfo.address.city'
    const parts = path.split('.');
    let value = document;
    
    for (const part of parts) {
      if (value && typeof value === 'object') {
        value = value[part];
      } else {
        value = undefined;
        break;
      }
    }
    
    result[path] = value;
  }
  
  return result;
}

// Mock function that would normally get the current user from authentication
function getCurrentUserId() {
  return 'admin123';
}

setupAuditLogging();

This audit system:

  1. Monitors employee data changes
  2. Identifies changes to sensitive fields like salary
  3. Records both old and new values for those fields
  4. Creates detailed audit logs for security and compliance purposes

Performance Considerations​

When using full document options, keep these performance factors in mind:

  1. Additional Lookups: The updateLookup option requires an additional database lookup for each change event, which can impact performance for high-volume updates.

  2. Document Size: For large documents, retrieving the full document increases network bandwidth usage.

  3. Timing Considerations: There's a small time gap between the update operation and the lookup for the full document. In rare cases, the document might be modified again or deleted in this window.

  4. Resource Usage: For high-throughput systems, consider whether you truly need the complete document or if just the changed fields would suffice.

Best Practices​

  1. Be Selective: Only use fullDocument when your application truly needs the entire document.

  2. Consider Filtering: Apply filters to your change stream to process only relevant updates.

javascript
// Only watch for updates to specific fields
const changeStream = collection.watch([
  { $match: { 
    'operationType': 'update',
    'updateDescription.updatedFields.email': { $exists: true }
  }}
], {
  fullDocument: 'updateLookup'
});
  1. Error Handling: Handle cases where the document might not be available (deleted between update and lookup).
javascript
changeStream.on('change', (change) => {
  if (change.operationType === 'update') {
    if (!change.fullDocument) {
      console.log('Document no longer exists:', change.documentKey._id);
      return;
    }
    // Process the document...
  }
});
  1. Use Pre-change State Wisely: In MongoDB 6.0+, use fullDocumentBeforeChange only when you need to compare before/after states, as it adds overhead.

Summary​

MongoDB's full document update features in Change Streams provide powerful capabilities for applications that need to react to data changes with complete context:

  • The fullDocument: 'updateLookup' option retrieves the complete updated document
  • MongoDB 6.0+ adds fullDocumentBeforeChange to also capture the document's previous state
  • These features enable sophisticated use cases like audit logging, synchronization, and detailed change tracking
  • Performance considerations should be evaluated for high-volume applications

By understanding how to leverage full document updates in Change Streams, you can build more intelligent, reactive applications that respond effectively to data changes.

Additional Resources and Exercises​

Resources​

Practice Exercises​

  1. Basic Implementation: Create a simple application that monitors a "products" collection and logs the full document whenever a product price is updated.

  2. Inventory Tracker: Build a system that uses change streams to track inventory levels and sends notifications when a product's stock falls below a threshold. Use the full document to include complete product details in the notification.

  3. Comparison Tool: Implement a utility that shows a "diff" view of document changes, comparing the before and after states (requires MongoDB 6.0+).

  4. Advanced: Create a data synchronization system that keeps a secondary system updated with changes from MongoDB, using the full document option to ensure accuracy.


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)