Express Continuous Integration
Introduction
Continuous Integration (CI) is a development practice where developers frequently merge their code changes into a central repository, after which automated builds and tests are run. For Express.js applications, implementing CI helps ensure that your application remains stable as new features are added and bugs are fixed.
In this guide, we'll explore how to set up continuous integration for your Express.js projects. We'll cover:
- Understanding CI fundamentals for Express applications
- Setting up CI pipelines with popular tools
- Automating tests in your CI workflow
- Best practices for Express.js CI
What is Continuous Integration?
Continuous Integration is the practice of merging all developer working copies to a shared mainline several times a day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.
For Express.js applications, CI offers several benefits:
- Automatically detecting errors and bugs early
- Ensuring code quality with consistent testing
- Simplifying the deployment process
- Allowing for more frequent releases
Setting Up Your Express Project for CI
Before implementing CI, your Express project should have:
- A consistent project structure
- Automated tests (unit, integration, end-to-end)
- A version control system (typically Git)
Project Structure Example
my-express-app/
├── node_modules/
├── public/
├── src/
│ ├── controllers/
│ ├── models/
│ ├── routes/
│ └── app.js
├── test/
│ ├── unit/
│ └── integration/
├── .env
├── .gitignore
├── package.json
└── README.md
Setting Up Testing
For effective CI, you need automated tests. Here's a simple example using Mocha and Chai:
First, install the dependencies:
npm install --save-dev mocha chai supertest
Then create a test file for an API endpoint:
// test/integration/users.test.js
const request = require('supertest');
const { expect } = require('chai');
const app = require('../../src/app');
describe('User API', () => {
it('should return users when GET /api/users is called', async () => {
const res = await request(app)
.get('/api/users')
.expect(200);
expect(res.body).to.be.an('array');
});
});
Update your package.json with test scripts:
{
"scripts": {
"start": "node src/app.js",
"test": "mocha test/**/*.test.js --exit",
"test:unit": "mocha test/unit/**/*.test.js --exit",
"test:integration": "mocha test/integration/**/*.test.js --exit"
}
}
Popular CI Tools for Express Applications
GitHub Actions
GitHub Actions is a popular choice for CI/CD, especially if your code is hosted on GitHub.
Create a workflow file at .github/workflows/ci.yml:
name: Express CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build-and-test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x, 16.x, 18.x]
steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
- run: npm run lint --if-present
- run: npm test
This workflow:
- Triggers on pushes to main branch or pull requests
- Tests across multiple Node.js versions
- Runs linting and test scripts
Travis CI
For Travis CI, create a .travis.yml file in your project root:
language: node_js
node_js:
- 14
- 16
- 18
cache: npm
before_script:
- npm install
script:
- npm run lint --if-present
- npm test
CircleCI
For CircleCI, create a .circleci/config.yml file:
version: 2.1
jobs:
build:
docker:
- image: cimg/node:16.13
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- v1-dependencies-
- run: npm install
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package.json" }}
- run: npm run lint --if-present
- run: npm test
Setting Up a Database for CI Testing
Many Express applications use databases, which requires special handling in CI environments.
Using In-memory MongoDB for Tests
// test/setup.js
const { MongoMemoryServer } = require('mongodb-memory-server');
const mongoose = require('mongoose');
let mongoServer;
// Set up the database before tests
before(async () => {
mongoServer = await MongoMemoryServer.create();
const mongoUri = mongoServer.getUri();
await mongoose.connect(mongoUri);
});
// Clean up after tests
after(async () => {
await mongoose.disconnect();
await mongoServer.stop();
});
// Clear collections between tests
afterEach(async () => {
const collections = mongoose.connection.collections;
for (const key in collections) {
await collections[key].deleteMany({});
}
});
Then in your test file:
// test/integration/users.test.js
require('../setup'); // Include the setup file
const User = require('../../src/models/user');
describe('User API', () => {
beforeEach(async () => {
// Create test data
await User.create({
name: 'Test User',
email: '[email protected]'
});
});
it('should return users', async () => {
// Test implementation
});
});
Environment Variables in CI
Express apps often use environment variables for configuration. Here's how to handle them in CI:
Setting Environment Variables
GitHub Actions
jobs:
build:
env:
NODE_ENV: test
PORT: 3000
JWT_SECRET: ${{ secrets.JWT_SECRET }}
Travis CI
env:
global:
- NODE_ENV=test
- PORT=3000
For sensitive data, use each CI platform's encrypted secrets feature.
Real-World CI Pipeline for an Express Application
Let's look at a comprehensive CI workflow for an Express application:
name: Express CI/CD Pipeline
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: '16.x'
cache: 'npm'
- run: npm ci
- run: npm run lint
test:
needs: lint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x, 16.x, 18.x]
steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
- run: npm test
build:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: '16.x'
cache: 'npm'
- run: npm ci
- run: npm run build --if-present
- name: Archive build artifact
uses: actions/upload-artifact@v3
with:
name: build-artifact
path: dist
deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v3
with:
name: build-artifact
path: dist
- name: Deploy to staging
run: |
echo "Deploying to staging environment"
# Add deployment steps here
This workflow:
- Runs linting checks
- Proceeds to testing if linting passes
- Builds the application after tests pass
- Deploys only when code is pushed to the main branch
Best Practices for Express CI
1. Include Different Types of Tests
// Unit test example
// test/unit/utils.test.js
const { expect } = require('chai');
const { formatDate } = require('../../src/utils/dateFormatter');
describe('Date Formatter', () => {
it('should format date correctly', () => {
const date = new Date('2023-01-01');
expect(formatDate(date)).to.equal('01/01/2023');
});
});
// Integration test example
// test/integration/auth.test.js
const request = require('supertest');
const { expect } = require('chai');
const app = require('../../src/app');
describe('Auth API', () => {
it('should return JWT token on successful login', async () => {
await request(app)
.post('/api/login')
.send({ email: '[email protected]', password: 'password123' })
.expect(200)
.expect(res => {
expect(res.body).to.have.property('token');
});
});
});
2. Fail Fast
Configure your CI to exit early if any step fails:
# GitHub Actions example
jobs:
build:
steps:
- run: npm run lint || exit 1
- run: npm test
3. Cache Dependencies
# GitHub Actions
steps:
- uses: actions/setup-node@v3
with:
node-version: '16.x'
cache: 'npm'
# CircleCI
steps:
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- v1-dependencies-
4. Parallel Testing
Split your tests to run in parallel when possible:
jobs:
test:
strategy:
matrix:
shard: [1, 2, 3]
steps:
- run: npm test -- --shard=${{ matrix.shard }}/3
Monitoring and Notifications
Set up notifications to alert your team about build status:
Slack Notifications in GitHub Actions
jobs:
notify:
needs: [test, build]
runs-on: ubuntu-latest
steps:
- uses: rtCamp/action-slack-notify@v2
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
SLACK_CHANNEL: ci-notifications
SLACK_COLOR: ${{ job.status }}
SLACK_TITLE: Build Result
SLACK_MESSAGE: 'Build ${{ github.run_number }} completed'
Common CI Issues and Solutions
1. Flaky Tests
Problem: Tests that sometimes pass and sometimes fail with no code changes.
Solution:
// Add retry logic for flaky tests
describe('Occasionally flaky feature', function() {
this.retries(3); // Retry up to 3 times
it('should perform reliably', async () => {
// Test implementation
});
});
2. Long-Running Tests
Problem: Tests taking too long and slowing down CI.
Solution: Identify and optimize slow tests, or split test runs:
{
"scripts": {
"test:fast": "mocha test/unit/**/*.test.js --exit",
"test:slow": "mocha test/integration/**/*.test.js --exit"
}
}
Then in CI:
jobs:
fast-tests:
runs-on: ubuntu-latest
steps:
- run: npm run test:fast
slow-tests:
runs-on: ubuntu-latest
steps:
- run: npm run test:slow
Summary
Continuous Integration is an essential practice for modern Express.js application development. By automatically building and testing your code on every change, you can:
- Catch bugs early in the development process
- Ensure consistent code quality
- Enable faster release cycles
- Build confidence in your codebase
In this guide, we covered setting up CI pipelines with popular tools like GitHub Actions, Travis CI, and CircleCI. We explored best practices for testing Express applications, handling databases, and configuring environment variables in CI environments.
Remember that CI is not just about tools but about adopting a development culture that values frequent integration and automated testing.
Additional Resources
- GitHub Actions Documentation
- Express.js Testing Best Practices
- Mocha Testing Framework
- Chai Assertion Library
- SuperTest for HTTP Testing
Exercises
- Set up a basic GitHub Actions workflow for an existing Express.js project.
- Add unit tests for at least one utility function in your Express application.
- Create an integration test for one of your API endpoints.
- Configure your CI pipeline to run both unit and integration tests.
- Add a notification system to your CI pipeline that alerts you when builds fail.
- Extend your CI pipeline to deploy your application to a staging environment when tests pass.
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)