Express Test Database

Introduction

Testing is a crucial part of building reliable web applications. When developing Express applications, testing database operations can be challenging but is essential for ensuring your application works correctly. This guide will walk you through setting up and using test databases for Express applications, focusing on practical approaches that beginners can understand and implement.

A test database is a separate database instance used specifically for running tests, allowing you to verify your database operations without affecting your development or production data. By the end of this guide, you'll understand how to set up test databases, write tests for database operations, and adopt best practices for database testing.

Why Do We Need Separate Test Databases?

Before diving into implementation, let's understand why separate test databases are important:

Isolation: Tests should not affect each other or your development data
Reproducibility: Tests should start with a known database state
Performance: Tests should run quickly without unnecessary setup/teardown
Safety: Tests should never accidentally modify production data

Setting Up a Test Database Environment

Prerequisites

To follow along with this guide, you'll need:

Node.js and npm installed
Basic knowledge of Express.js
Familiarity with a database system (we'll use MongoDB with Mongoose in our examples)
Jest testing framework

Installing Dependencies

First, let's install the necessary dependencies:

bash
npm install --save-dev jest supertest mongodb-memory-server
npm install express mongoose

jest: JavaScript testing framework
supertest: HTTP assertions library for testing Express APIs
mongodb-memory-server: In-memory MongoDB server for testing
express: Web framework for Node.js
mongoose: MongoDB object modeling tool

Approach 1: Using In-Memory Databases

One of the most convenient approaches for testing is using an in-memory database. This creates a temporary database that exists only during test execution. Let's see how to set this up with MongoDB.

Setting Up MongoDB Memory Server

Create a test setup file called testSetup.js:

javascript
const { MongoMemoryServer } = require('mongodb-memory-server');
const mongoose = require('mongoose');

let mongoServer;

// Connect to the in-memory database before tests run
beforeAll(async () => {
  mongoServer = await MongoMemoryServer.create();
  const mongoUri = mongoServer.getUri();
  
  await mongoose.connect(mongoUri, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  });
});

// Clear all collections after each test
afterEach(async () => {
  const collections = mongoose.connection.collections;
  
  for (const key in collections) {
    const collection = collections[key];
    await collection.deleteMany({});
  }
});

// Disconnect and close the database after all tests finish
afterAll(async () => {
  await mongoose.disconnect();
  await mongoServer.stop();
});

Creating a Simple User Model for Testing

Let's create a simple user model to test:

javascript
// models/user.js
const mongoose = require('mongoose');

const userSchema = new mongoose.Schema({
  username: {
    type: String,
    required: true,
    unique: true
  },
  email: {
    type: String,
    required: true
  },
  createdAt: {
    type: Date,
    default: Date.now
  }
});

module.exports = mongoose.model('User', userSchema);

Creating a User Controller

javascript
// controllers/userController.js
const User = require('../models/user');

exports.createUser = async (req, res) => {
  try {
    const user = new User({
      username: req.body.username,
      email: req.body.email
    });
    
    const savedUser = await user.save();
    res.status(201).json(savedUser);
  } catch (error) {
    res.status(400).json({ message: error.message });
  }
};

exports.getUsers = async (req, res) => {
  try {
    const users = await User.find();
    res.status(200).json(users);
  } catch (error) {
    res.status(500).json({ message: error.message });
  }
};

Creating Express Routes

javascript
// routes/userRoutes.js
const express = require('express');
const router = express.Router();
const userController = require('../controllers/userController');

router.post('/users', userController.createUser);
router.get('/users', userController.getUsers);

module.exports = router;

Setting Up the Express App

javascript
// app.js
const express = require('express');
const userRoutes = require('./routes/userRoutes');

const app = express();

app.use(express.json());
app.use('/api', userRoutes);

module.exports = app;

Writing Database Tests

Now that we have our app set up, let's write tests for our user API:

javascript
// tests/user.test.js
const request = require('supertest');
const app = require('../app');
const User = require('../models/user');

// This will use the test setup defined in testSetup.js
// Make sure Jest is configured to use this setup file

describe('User API', () => {
  test('should create a new user', async () => {
    const userData = {
      username: 'testuser',
      email: '[email protected]'
    };
    
    const response = await request(app)
      .post('/api/users')
      .send(userData)
      .expect(201);
      
    // Check response contains user with correct data
    expect(response.body.username).toBe(userData.username);
    expect(response.body.email).toBe(userData.email);
    
    // Verify user was actually saved to database
    const userInDb = await User.findOne({ username: 'testuser' });
    expect(userInDb).toBeTruthy();
    expect(userInDb.email).toBe(userData.email);
  });
  
  test('should fetch all users', async () => {
    // Create test users first
    await User.create([
      { username: 'user1', email: '[email protected]' },
      { username: 'user2', email: '[email protected]' }
    ]);
    
    const response = await request(app)
      .get('/api/users')
      .expect(200);
      
    expect(response.body.length).toBe(2);
    expect(response.body[0].username).toBe('user1');
    expect(response.body[1].username).toBe('user2');
  });
});

Configuration for Jest

Add this to your package.json:

json
"jest": {
  "testEnvironment": "node",
  "setupFilesAfterEnv": ["./testSetup.js"]
}

Approach 2: Using Separate Test Database Instances

While in-memory databases are convenient, sometimes you need to test against the actual database you'll use in production. Here's how to set up separate test databases:

Creating Database Connection Utility

javascript
// utils/database.js
const mongoose = require('mongoose');

const connectDB = async (url) => {
  await mongoose.connect(url, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  });
};

module.exports = connectDB;

Environment-Based Configuration

Create a configuration file to manage different database environments:

javascript
// config/database.js
module.exports = {
  development: 'mongodb://localhost:27017/my_app_dev',
  test: 'mongodb://localhost:27017/my_app_test',
  production: process.env.MONGODB_URI
};

Updating App Startup

javascript
// server.js
const app = require('./app');
const connectDB = require('./utils/database');
const config = require('./config/database');

const environment = process.env.NODE_ENV || 'development';
const dbUrl = config[environment];

const start = async () => {
  try {
    await connectDB(dbUrl);
    console.log(`Connected to database (${environment} environment)`);
    
    const port = process.env.PORT || 3000;
    app.listen(port, () => {
      console.log(`Server running on port ${port}`);
    });
  } catch (error) {
    console.error('Failed to start server:', error);
    process.exit(1);
  }
};

start();

Database Test Setup and Teardown

Create a file for database test setup:

javascript
// tests/dbHandler.js
const mongoose = require('mongoose');
const config = require('../config/database');

// Connect to the test database before tests run
const setupTestDB = async () => {
  await mongoose.connect(config.test);
};

// Clear all data after each test
const clearDatabase = async () => {
  const collections = mongoose.connection.collections;
  
  for (const key in collections) {
    const collection = collections[key];
    await collection.deleteMany({});
  }
};

// Disconnect after all tests complete
const closeDatabase = async () => {
  await mongoose.connection.dropDatabase();
  await mongoose.connection.close();
};

module.exports = {
  setupTestDB,
  clearDatabase,
  closeDatabase
};

Writing Tests with the Test Database

javascript
// tests/user.test.js
const request = require('supertest');
const app = require('../app');
const User = require('../models/user');
const { setupTestDB, clearDatabase, closeDatabase } = require('./dbHandler');

beforeAll(async () => {
  await setupTestDB();
});

afterEach(async () => {
  await clearDatabase();
});

afterAll(async () => {
  await closeDatabase();
});

describe('User API with test database', () => {
  test('should create a new user', async () => {
    const userData = {
      username: 'testuser',
      email: '[email protected]'
    };
    
    const response = await request(app)
      .post('/api/users')
      .send(userData)
      .expect(201);
      
    expect(response.body.username).toBe(userData.username);
    
    // Verify in database
    const user = await User.findOne({ username: userData.username });
    expect(user).toBeTruthy();
  });
});

Advanced Testing Techniques

Testing Database Transactions

For operations that require transactions (atomicity), you need to ensure your tests properly verify transaction behavior:

javascript
// models/account.js
const mongoose = require('mongoose');

const accountSchema = new mongoose.Schema({
  owner: String,
  balance: {
    type: Number,
    min: 0
  }
});

module.exports = mongoose.model('Account', accountSchema);

javascript
// services/transferService.js
const mongoose = require('mongoose');
const Account = require('../models/account');

exports.transferFunds = async (fromAccountId, toAccountId, amount) => {
  const session = await mongoose.startSession();
  session.startTransaction();
  
  try {
    // Withdraw from source account
    const fromAccount = await Account.findById(fromAccountId).session(session);
    if (!fromAccount || fromAccount.balance < amount) {
      throw new Error('Insufficient funds');
    }
    
    fromAccount.balance -= amount;
    await fromAccount.save({ session });
    
    // Deposit to destination account
    const toAccount = await Account.findById(toAccountId).session(session);
    if (!toAccount) {
      throw new Error('Destination account not found');
    }
    
    toAccount.balance += amount;
    await toAccount.save({ session });
    
    // Commit the transaction
    await session.commitTransaction();
    return { success: true };
    
  } catch (error) {
    // Abort transaction on error
    await session.abortTransaction();
    throw error;
  } finally {
    session.endSession();
  }
};

Testing this transaction:

javascript
// tests/transfer.test.js
const mongoose = require('mongoose');
const Account = require('../models/account');
const { transferFunds } = require('../services/transferService');

describe('Fund Transfer Tests', () => {
  let sourceAccountId;
  let destAccountId;
  
  beforeEach(async () => {
    // Create test accounts
    const sourceAccount = await Account.create({
      owner: 'Source User',
      balance: 1000
    });
    
    const destAccount = await Account.create({
      owner: 'Destination User',
      balance: 500
    });
    
    sourceAccountId = sourceAccount._id;
    destAccountId = destAccount._id;
  });
  
  test('should transfer funds between accounts', async () => {
    // Perform transfer
    await transferFunds(sourceAccountId, destAccountId, 300);
    
    // Check balances after transfer
    const sourceAccount = await Account.findById(sourceAccountId);
    const destAccount = await Account.findById(destAccountId);
    
    expect(sourceAccount.balance).toBe(700);
    expect(destAccount.balance).toBe(800);
  });
  
  test('should fail if source has insufficient funds', async () => {
    expect.assertions(1);
    
    try {
      await transferFunds(sourceAccountId, destAccountId, 1500);
    } catch (error) {
      expect(error.message).toBe('Insufficient funds');
    }
    
    // Verify no changes were made to either account
    const sourceAccount = await Account.findById(sourceAccountId);
    const destAccount = await Account.findById(destAccountId);
    
    expect(sourceAccount.balance).toBe(1000);
    expect(destAccount.balance).toBe(500);
  });
});

Testing Database Errors and Edge Cases

It's important to test how your application handles database errors:

javascript
// tests/errorHandling.test.js
const mongoose = require('mongoose');
const User = require('../models/user');

describe('Database Error Handling', () => {
  test('should handle duplicate key errors', async () => {
    // Create a user
    await User.create({
      username: 'uniqueuser',
      email: '[email protected]'
    });
    
    // Try to create another user with the same username (which has a unique constraint)
    try {
      await User.create({
        username: 'uniqueuser',
        email: '[email protected]'
      });
      // If the above doesn't throw, the test should fail
      expect(true).toBe(false);
    } catch (error) {
      expect(error.name).toBe('MongoServerError');
      expect(error.code).toBe(11000); // Duplicate key error code
    }
  });
  
  test('should handle validation errors', async () => {
    try {
      // Missing required field (email)
      await User.create({
        username: 'testuser'
      });
      // If the above doesn't throw, the test should fail
      expect(true).toBe(false);
    } catch (error) {
      expect(error.name).toBe('ValidationError');
      expect(error.errors.email).toBeDefined();
    }
  });
});

Best Practices for Database Testing

Ensure test isolation: Each test should run independently without affecting others.
Clean up test data: Always clean up your test database before or after tests.
Use realistic test data: Test with data that resembles what your application will encounter in production.
Test performance: Include tests for database operation performance if it's critical for your application.
Mock external dependencies: When testing database operations, mock any external services.
Test database migrations: If your application uses database migrations, write tests for them too.
Test error handling: Ensure your application handles database errors gracefully.

Common Pitfalls and Solutions

Test Speed

Problem: Database tests are often slow, especially with real databases.

Solution:

Use in-memory databases for most tests
Batch related tests that can share setup
Only test essential database operations

javascript
// Faster testing with batched tests
describe('User operations', () => {
  // Shared setup runs once for this group
  beforeAll(async () => {
    await User.create([
      { username: 'user1', email: '[email protected]' },
      { username: 'user2', email: '[email protected]' }
    ]);
  });
  
  test('should find user by username', async () => {
    const user = await User.findOne({ username: 'user1' });
    expect(user.email).toBe('[email protected]');
  });
  
  test('should find user by email', async () => {
    const user = await User.findOne({ email: '[email protected]' });
    expect(user.username).toBe('user2');
  });
});

Database Connections Not Closing

Problem: Tests finish but the process hangs because database connections aren't properly closed.

Solution: Ensure proper cleanup in the afterAll hook:

javascript
afterAll(async () => {
  // Force close all connections
  await mongoose.disconnect();
  if (mongoServer) {
    await mongoServer.stop();
  }
});

Race Conditions

Problem: Intermittent test failures due to race conditions with database operations.

Solution: Use async/await properly and avoid mixing promises with callbacks:

javascript
// Bad - potential race condition
test('should update user', () => {
  User.findOne({ username: 'user1' })
    .then(user => {
      user.email = '[email protected]';
      return user.save();
    })
    .then(updatedUser => {
      expect(updatedUser.email).toBe('[email protected]');
    });
});

// Good - proper async/await
test('should update user', async () => {
  const user = await User.findOne({ username: 'user1' });
  user.email = '[email protected]';
  const updatedUser = await user.save();
  expect(updatedUser.email).toBe('[email protected]');
});

Summary

In this guide, we've covered how to set up and use test databases for Express applications. We've explored:

Why separate test databases are important
Setting up in-memory MongoDB for testing
Configuring separate test database instances
Writing tests for CRUD operations, transactions, and error handling
Best practices and common pitfalls in database testing

Testing database operations is essential for building robust Express applications. By following the approaches outlined in this guide, you can ensure your database interactions work correctly in various scenarios, giving you confidence in your application's reliability.

Additional Resources and Exercises

Exercises

Basic Database Testing: Create a simple Express API for a blog with posts and comments. Write tests for creating, reading, updating, and deleting posts and comments.
Transaction Testing: Implement a banking system with accounts and transactions. Write tests to ensure money transfers maintain data integrity.
Performance Testing: Create tests that measure the performance of database queries. Optimize the queries and observe the performance improvements.
Migration Testing: Create database migrations for schema changes and write tests to verify they work correctly.
Error Handling: Write tests for various database error scenarios (connection failures, validation errors, etc.) and ensure your application handles them properly.

By completing these exercises, you'll strengthen your understanding of database testing in Express applications and be well-prepared to implement robust testing strategies in your projects.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

Introduction​

Why Do We Need Separate Test Databases?​

Setting Up a Test Database Environment​

Prerequisites​

Installing Dependencies​

Approach 1: Using In-Memory Databases​

Setting Up MongoDB Memory Server​

Creating a Simple User Model for Testing​

Creating a User Controller​

Creating Express Routes​

Setting Up the Express App​

Writing Database Tests​

Configuration for Jest​

Approach 2: Using Separate Test Database Instances​

Creating Database Connection Utility​

Environment-Based Configuration​

Updating App Startup​

Database Test Setup and Teardown​

Writing Tests with the Test Database​

Advanced Testing Techniques​

Testing Database Transactions​

Testing Database Errors and Edge Cases​

Best Practices for Database Testing​

Common Pitfalls and Solutions​

Test Speed​

Database Connections Not Closing​

Race Conditions​

Summary​

Additional Resources and Exercises​

Further Reading​

Exercises​