Docker Compose Environment

Introduction

Environment variables are a crucial component of containerized applications, providing a way to configure your services without modifying code. In this tutorial, we'll explore how Docker Compose handles environment variables, allowing you to create flexible, configurable, and secure container deployments.

Docker Compose provides several methods for working with environment variables, each with specific use cases and advantages. By the end of this guide, you'll understand how to effectively use environment variables to configure your Docker Compose applications.

Understanding Environment Variables in Docker Compose

Environment variables in Docker Compose serve several important purposes:

1. Configuration: They allow you to configure your application without changing code
2. Secrets Management: They provide a way to pass sensitive information to containers
3. Environment-Specific Settings: They help you handle differences between development, testing, and production environments
4. Service Interaction: They enable communication between services

Ways to Define Environment Variables

Docker Compose offers multiple approaches to define environment variables. Let's explore each method:

1. Using the environment Key

The most direct way to set environment variables is through the environment key in your docker-compose.yml file. You can specify variables in two formats:

Array syntax:

yaml

services:
  web:
    image: nginx
    environment:
      - DEBUG=true
      - DB_HOST=database
      - DB_PORT=5432

Object syntax:

yaml

services:
  web:
    image: nginx
    environment:
      DEBUG: "true"
      DB_HOST: database
      DB_PORT: 5432

Both approaches achieve the same result, setting environment variables inside the container. The array syntax is useful when you want to reference existing environment variables from your host machine.

2. Using the env_file Option

When you have many environment variables, it's cleaner to place them in a separate file. The env_file option allows you to specify one or more .env files:

yaml

services:
  web:
    image: nginx
    env_file:
      - ./common.env
      - ./web.env
      - ./debug.env

Each .env file should contain simple variable assignments:

# ./web.env
DB_HOST=database
DB_PORT=5432
DEBUG=true

This approach helps keep your Docker Compose file cleaner and allows you to reuse environment configurations across multiple services.

3. Using Variable Substitution

Docker Compose supports variable substitution, allowing you to use existing environment variables from the shell where you run docker-compose:

yaml

services:
  web:
    image: nginx
    ports:
      - "${PORT:-8080}:80"
    environment:
      - API_KEY=${API_KEY}
      - DEBUG=${DEBUG:-false}

The syntax ${VARIABLE:-default} provides a default value if the variable isn't set in your shell.

To use this approach:

1. Set variables in your shell:

   bash

   export API_KEY=your_secret_key
   export DEBUG=true

2. Then run Docker Compose:

   bash

   docker-compose up -d

4. Using a .env File for Docker Compose

Docker Compose automatically looks for a file named .env in the same directory as your docker-compose.yml. Variables defined in this file are used for variable substitution in your Docker Compose file but aren't automatically passed to containers.

Example .env file:

PORT=8080
DEBUG=true
API_KEY=your_secret_key
POSTGRES_PASSWORD=secretpassword

Your docker-compose.yml can then use these variables:

yaml

services:
  web:
    image: nginx
    ports:
      - "${PORT}:80"
    environment:
      - DEBUG=${DEBUG}
      - API_KEY=${API_KEY}
  
  db:
    image: postgres
    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}

This approach keeps sensitive information out of your Docker Compose file while still making it available for configuration.

Practical Example: Multi-Environment Configuration

Let's create a practical example with a web application and a database, configured for different environments.

Project Structure

project/
├── docker-compose.yml
├── .env
├── .env.development
├── .env.production
└── web/
    └── Dockerfile

Base Docker Compose File

yaml

version: '3.8'

services:
  web:
    build: ./web
    ports:
      - "${WEB_PORT:-3000}:3000"
    environment:
      - NODE_ENV=${NODE_ENV:-development}
      - DB_HOST=db
      - DB_PORT=${DB_PORT:-5432}
      - DB_NAME=${DB_NAME:-myapp}
      - DB_USER=${DB_USER:-postgres}
      - DB_PASSWORD=${DB_PASSWORD:-postgres}
    depends_on:
      - db
  
  db:
    image: postgres:14
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=${DB_NAME:-myapp}
      - POSTGRES_USER=${DB_USER:-postgres}
      - POSTGRES_PASSWORD=${DB_PASSWORD:-postgres}

volumes:
  db_data:

Environment Files

.env.development

NODE_ENV=development
WEB_PORT=3000
DB_PORT=5432
DB_NAME=myapp_dev
DB_USER=dev_user
DB_PASSWORD=dev_password

.env.production

NODE_ENV=production
WEB_PORT=80
DB_PORT=5432
DB_NAME=myapp_prod
DB_USER=prod_user
DB_PASSWORD=prod_secure_password

Running with Different Environments

To start the app in development mode:

bash

# Load the development environment file
cp .env.development .env
docker-compose up -d

To start the app in production mode:

bash

# Load the production environment file
cp .env.production .env
docker-compose up -d

Handling Sensitive Information

Environment variables are commonly used for sensitive information, but there are some security considerations:

1. Don't commit .env files: Add .env and similar files to your .gitignore
2. Consider secrets management tools: For production, consider Docker Swarm secrets or Kubernetes secrets
3. Limit variable scope: Only pass necessary variables to each service

Environment Variables Inside Containers

Once set, environment variables are available to your application inside the container. Here's how to access them in different languages:

Node.js:

javascript

const dbHost = process.env.DB_HOST;
const dbPort = process.env.DB_PORT;

console.log(`Connecting to database at ${dbHost}:${dbPort}`);

Python:

python

import os

db_host = os.environ.get('DB_HOST')
db_port = os.environ.get('DB_PORT')

print(f"Connecting to database at {db_host}:{db_port}")

Java:

java

String dbHost = System.getenv("DB_HOST");
String dbPort = System.getenv("DB_PORT");

System.out.println("Connecting to database at " + dbHost + ":" + dbPort);

Advanced Docker Compose Environment Techniques

1. Extending Configurations with Multiple Compose Files

Docker Compose allows you to combine multiple configuration files. This is useful for separating environment-specific settings:

bash

# Base configuration + development overrides
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

# Base configuration + production overrides
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up

docker-compose.yml (base configuration):

yaml

version: '3.8'

services:
  web:
    build: ./web
    depends_on:
      - db
  
  db:
    image: postgres:14
    volumes:
      - db_data:/var/lib/postgresql/data

volumes:
  db_data:

docker-compose.dev.yml (development overrides):

yaml

version: '3.8'

services:
  web:
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=development
      - DEBUG=true
    volumes:
      - ./web:/app
  
  db:
    environment:
      - POSTGRES_DB=myapp_dev
      - POSTGRES_USER=dev_user
      - POSTGRES_PASSWORD=dev_password
    ports:
      - "5432:5432"

docker-compose.prod.yml (production overrides):

yaml

version: '3.8'

services:
  web:
    ports:
      - "80:3000"
    environment:
      - NODE_ENV=production
      - DEBUG=false
  
  db:
    environment:
      - POSTGRES_DB=myapp_prod
      - POSTGRES_USER=${PROD_DB_USER}
      - POSTGRES_PASSWORD=${PROD_DB_PASSWORD}

2. Using Environment Variables with Docker Compose Profiles

Profiles allow you to selectively start services based on the environment:

yaml

version: '3.8'

services:
  web:
    build: ./web
    environment:
      - NODE_ENV=${NODE_ENV:-development}
  
  db:
    image: postgres:14
    environment:
      - POSTGRES_DB=${DB_NAME:-myapp}
  
  # Only used in development
  adminer:
    image: adminer
    ports:
      - "8080:8080"
    profiles:
      - dev
    depends_on:
      - db
      
  # Only used in production
  redis:
    image: redis:alpine
    profiles:
      - prod

To start with specific profiles:

bash

# Start with development profile (includes adminer)
docker-compose --profile dev up -d

# Start with production profile (includes redis)
docker-compose --profile prod up -d

Common Pitfalls and Solutions

Problem: Environment Variables Not Being Passed to Containers

Symptoms: Your application can't access expected environment variables.

Solutions:

• Double-check your environment section in docker-compose.yml
• Verify your .env file is in the same directory as your Docker Compose file
• Remember that the .env file variables need to be explicitly passed to containers

Problem: Variable Substitution Not Working

Symptoms: You see literal ${VARIABLE} strings in your container configuration.

Solutions:

• Make sure the variables are defined in your shell or in the .env file
• Check for typos in variable names
• Use the docker-compose config command to verify the processed configuration

Problem: Special Characters in Environment Variables

Symptoms: Variables with special characters aren't parsed correctly.

Solutions:

• Use quotes around values in your .env file
• For very complex values, consider using Docker secrets instead

Summary

Environment variables in Docker Compose provide a powerful way to configure your containerized applications for different environments and use cases. By leveraging the various methods for defining and passing environment variables, you can create more flexible, secure, and maintainable container deployments.

Key takeaways:

1. Use the environment key for direct variable definition
2. Use env_file for organizing variables in separate files
3. Use variable substitution with ${VARIABLE} syntax for dynamic configuration
4. Keep sensitive information in .env files and exclude them from version control
5. Combine multiple Docker Compose files for environment-specific configurations

Additional Resources

• Docker Compose Variable Substitution
• Docker Compose Environment File
• Docker Secrets

Exercises

1. Create a Docker Compose configuration for a three-tier application (web, API, database) using environment variables for configuration.
2. Set up different environment files for development, testing, and production environments.
3. Practice using Docker Compose profiles to selectively start services based on the environment.
4. Implement a secure method for handling database credentials using environment variables.