Docker Compose Dependencies

Introduction

When working with multi-container applications using Docker Compose, understanding how to manage dependencies between services is crucial. In the real world, applications often consist of multiple interconnected components that rely on each other to function properly. For example, a web application might need a database to be available before it can start, or a cache service might depend on another initialization service.

Docker Compose provides several mechanisms to define and manage these dependencies, ensuring your services start in the correct order and handle connection issues gracefully. In this guide, we'll explore the different ways to manage service dependencies in Docker Compose.

Basic Service Dependencies with depends_on

The most straightforward way to define dependencies between services is using the depends_on option in your docker-compose.yml file.

Simple Example

yaml

version: '3'
services:
  web:
    build: ./web
    depends_on:
      - db
      - redis
  db:
    image: postgres
  redis:
    image: redis

In this example:

• The web service depends on both the db and redis services
• Docker Compose will start the db and redis services before starting the web service

How depends_on Works

When you run docker compose up, Docker Compose respects the dependencies defined with depends_on and follows these steps:

1. Creates networks (if not already created)
2. Creates volumes (if not already created)
3. Starts all services in dependency order

If we visualize this for our example:

Limitations of Basic depends_on

It's important to understand that basic depends_on only controls the order of service startup, not the readiness of those services. For example:

• When the db service starts, it doesn't mean the PostgreSQL database inside is ready to accept connections
• The web service might start before the database is fully initialized, causing connection errors

Handling Service Readiness

There are several approaches to ensure a service is fully ready before its dependents try to connect:

1. Health Checks in Docker Compose

Docker Compose version 2.12+ supports health checks:

yaml

version: '3'
services:
  web:
    build: ./web
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
  db:
    image: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
  redis:
    image: redis
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 5s
      retries: 5

In this configuration:

• The web service will only start after both db and redis services are healthy
• The health checks define how to determine if a service is ready

2. Wait Scripts in Entrypoint

Another common approach is to include a "wait script" in your service's entrypoint:

yaml

version: '3'
services:
  web:
    build: ./web
    depends_on:
      - db
    command: ["./wait-for-it.sh", "db:5432", "--", "python", "app.py"]
  db:
    image: postgres

Here's a simple example of what a wait-for-it.sh script might look like:

bash

#!/bin/bash
# wait-for-it.sh

set -e

host="$1"
shift
cmd="$@"

until nc -z "$host" 2>/dev/null; do
  echo "Waiting for $host to be available..."
  sleep 1
done

echo "$host is available, executing command"
exec $cmd

Make sure to include this script in your Docker image and make it executable.

3. Application-Level Retry Logic

A robust application should include retry logic when connecting to dependencies:

python

# Example Python code for database connection with retry
import time
import psycopg2

max_retries = 30
retry_count = 0

while retry_count < max_retries:
    try:
        conn = psycopg2.connect(
            host="db",
            database="postgres",
            user="postgres",
            password="postgres"
        )
        print("Database connection successful")
        break
    except psycopg2.OperationalError:
        retry_count += 1
        print(f"Connection attempt {retry_count} failed, retrying...")
        time.sleep(1)

if retry_count == max_retries:
    print("Could not connect to database after maximum retries")
    exit(1)

# Continue with application startup

Advanced Dependency Configurations

Conditional Startup with Condition Options

Docker Compose offers various conditions for the depends_on option:

yaml

version: '3'
services:
  web:
    build: ./web
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
  db:
    image: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
  redis:
    image: redis

Available conditions include:

• service_started: Wait until the dependency service has been started (default)
• service_healthy: Wait until the dependency service is healthy (passes health checks)
• service_completed_successfully: Wait until the dependency service has completed successfully

Controlling Startup Order in Complex Applications

For complex applications with many services, you can visualize the dependency hierarchy:

Here's how to implement this in Docker Compose:

yaml

version: '3'
services:
  database:
    image: postgres:13
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
  
  cache:
    image: redis:6
    depends_on:
      database:
        condition: service_healthy
  
  auth-service:
    build: ./auth-service
    depends_on:
      database:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 10s
      timeout: 5s
      retries: 3
  
  api-gateway:
    build: ./api-gateway
    depends_on:
      cache:
        condition: service_started
      auth-service:
        condition: service_healthy
  
  web-frontend:
    build: ./web-frontend
    depends_on:
      api-gateway:
        condition: service_started

Practical Example: Multi-Tier Web Application

Let's look at a complete example of a multi-tier web application with proper dependency management:

yaml

version: '3'

services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: app_user
      POSTGRES_PASSWORD: app_password
      POSTGRES_DB: app_database
    volumes:
      - postgres-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app_user -d app_database"]
      interval: 5s
      timeout: 5s
      retries: 5
  
  redis:
    image: redis:6
    volumes:
      - redis-data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 5s
      retries: 5
  
  backend:
    build: ./backend
    environment:
      DATABASE_URL: postgresql://app_user:app_password@postgres:5432/app_database
      REDIS_URL: redis://redis:6379
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/api/health"]
      interval: 10s
      timeout: 5s
      retries: 3
  
  frontend:
    build: ./frontend
    ports:
      - "80:80"
    depends_on:
      backend:
        condition: service_healthy

volumes:
  postgres-data:
  redis-data:

In this example:

1. postgres and redis services have health checks to ensure they're fully initialized
2. The backend service depends on both postgres and redis being healthy
3. The frontend service depends on the backend service being healthy
4. Data persistence is provided through named volumes

Environment Variable Dependencies

Services can also depend on each other through environment variables:

yaml

version: '3'
services:
  db:
    image: postgres
    environment:
      POSTGRES_PASSWORD: example
  
  web:
    build: ./web
    depends_on:
      - db
    environment:
      DATABASE_URL: postgresql://postgres:example@db:5432/postgres

Handling Circular Dependencies

Avoid circular dependencies in your Docker Compose configuration:

This circular dependency (A → B → C → A) will cause Docker Compose to fail with an error message about a circular reference. Instead, redesign your services to break the circular dependency.

Best Practices for Managing Dependencies

1. Use health checks for critical services that need time to initialize
2. Implement retry logic in your application code
3. Keep dependencies simple and avoid deep dependency chains
4. Document the dependency relationships with diagrams or comments
5. Use environment variables for configuration rather than hardcoding connection details
6. Consider using init containers for complex initialization sequences
7. Avoid circular dependencies by redesigning your architecture

Troubleshooting Dependency Issues

Common Problems and Solutions

Problem	Solution
Service fails because dependency isn't ready	Add health checks and use condition: service_healthy
Circular dependency error	Redesign your services to break the circular dependency
Services start in wrong order	Check your depends_on configuration
Health check keeps failing	Check the health check command and adjust retries/intervals

Debug with Docker Compose Events

Use the docker compose events command to see the sequence of service startup:

bash

docker compose events

Example output:

2023-05-15 10:23:45.123 Create db
2023-05-15 10:23:46.234 Start db
2023-05-15 10:23:50.345 Health db healthy
2023-05-15 10:23:51.456 Create web
2023-05-15 10:23:52.567 Start web

Summary

Managing dependencies in Docker Compose is essential for building reliable multi-container applications. The key takeaways are:

• Use depends_on to define the startup order of your services
• Implement health checks to ensure services are fully initialized
• Consider application-level retry logic for robust connections
• Choose the appropriate condition type based on your requirements
• Follow best practices to avoid common pitfalls

By properly managing dependencies in Docker Compose, you'll create more reliable and resilient container-based applications that start correctly every time.

Additional Resources

• Docker Compose Documentation
• Docker Compose Healthcheck Docs
• Docker Compose Depends On Reference

Exercises

1. Basic Exercise: Create a Docker Compose file with a web service that depends on a database service.
2. Intermediate Exercise: Add health checks to ensure the database is ready before the web service starts.
3. Advanced Exercise: Create a complete multi-tier application with at least 4 services and appropriate dependencies and health checks.