Docker Compose Profiles
Introduction
When working with Docker Compose, you might have encountered situations where you need different sets of services for various environments or use cases. For example, you may want to run only essential services for development but include monitoring tools for testing. This is where Docker Compose Profiles come in handy.
Profiles provide a way to selectively enable services when running Docker Compose commands, allowing you to maintain a single docker-compose.yml
file while still having the flexibility to start different service combinations based on your current needs.
What are Docker Compose Profiles?
Profiles are a feature introduced in Docker Compose version 1.28 that allows you to assign labels to services and then selectively enable those services during runtime. This way, you can organize your services into logical groups and start only the ones you need for a specific task.
How Profiles Work
Services in your Docker Compose file can be assigned to one or more profiles. If a service doesn't have any profile assigned, it will always start by default. Services assigned to profiles will only start when their profile is explicitly enabled.
Setting Up Profiles
Let's learn how to define and use profiles in your Docker Compose configuration.
Basic Syntax
To assign a service to a profile, add the profiles
attribute to the service definition:
services:
webapp:
image: nginx
# This service will always start by default as it has no profiles
database:
image: postgres
profiles:
- dev
- prod
# This service will only start when the "dev" or "prod" profile is active
monitoring:
image: grafana/grafana
profiles: ["monitoring", "prod"]
# This service will only start when "monitoring" or "prod" profile is active
In this example:
webapp
will always start by defaultdatabase
will only start when either thedev
orprod
profile is activatedmonitoring
will only start when either themonitoring
orprod
profile is activated
Activating Profiles
To activate profiles when running Docker Compose commands, use the --profile
flag or set the COMPOSE_PROFILES
environment variable:
Using the command line flag:
# Start services in the "dev" profile
docker compose --profile dev up
# Start services in multiple profiles
docker compose --profile dev --profile monitoring up
Using the environment variable:
# Set the active profiles
export COMPOSE_PROFILES=dev,monitoring
# Run docker-compose normally
docker compose up
Practical Examples
Let's explore some real-world scenarios where Docker Compose profiles are particularly useful.
Example 1: Development vs. Production Environment
Consider a web application with different service requirements for development and production:
services:
webapp:
image: mywebapp:latest
# This service will always start
database:
image: postgres:13
# This service will always start
dev-tools:
image: mydevtools:latest
profiles: ["dev"]
# Development-only tools
minifier:
image: cssjs-minifier:latest
profiles: ["build"]
# Used only during the build process
monitoring:
image: prometheus:latest
profiles: ["prod", "monitoring"]
# Used in production and monitoring environments
alerting:
image: alertmanager:latest
profiles: ["prod", "monitoring"]
# Used in production and monitoring environments
To start your development environment:
docker compose --profile dev up
This will start webapp
, database
, and dev-tools
.
For production:
docker compose --profile prod up
This will start webapp
, database
, monitoring
, and alerting
.
Example 2: Microservices Architecture
Profiles are particularly valuable in microservices architectures where you may have dozens of services but only need to work with a subset:
services:
# Core services
api-gateway:
image: api-gateway:latest
# Always starts
user-service:
image: user-service:latest
# Always starts
# Optional services
email-service:
image: email-service:latest
profiles: ["notification", "full"]
sms-service:
image: sms-service:latest
profiles: ["notification", "full"]
payment-service:
image: payment-service:latest
profiles: ["payment", "full"]
analytics:
image: analytics:latest
profiles: ["analytics", "full"]
# Development tools
adminer:
image: adminer:latest
profiles: ["debug", "tools"]
redis-commander:
image: redis-commander:latest
profiles: ["debug", "tools"]
When working on the notification system:
docker compose --profile notification up
This starts api-gateway
, user-service
, email-service
, and sms-service
.
For debugging with tools:
docker compose --profile debug up
This starts api-gateway
, user-service
, adminer
, and redis-commander
.
Visualizing Service Dependencies with Profiles
Let's visualize how profiles organize services using a diagram:
Advanced Profile Patterns
1. Overriding Services with Profiles
You can use profiles to override configuration for different environments:
services:
database:
image: postgres:13
profiles: ["prod"]
volumes:
- pg_data:/var/lib/postgresql/data
deploy:
resources:
limits:
memory: 2G
database-dev:
image: postgres:13
profiles: ["dev"]
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
ports:
- "5432:5432"
volumes:
pg_data:
In this example, we have two PostgreSQL service definitions with different configurations. For development, we expose the ports and simplify authentication, while the production version includes volume mounting and resource limits.
2. Utility Profiles
You can create utility profiles for one-off tasks:
services:
webapp:
image: mywebapp:latest
depends_on:
- database
database:
image: postgres:13
database-backup:
image: postgres:13
profiles: ["backup"]
volumes:
- ./backups:/backups
- pg_data:/var/lib/postgresql/data:ro
command: pg_dump -U postgres -d mydb -f /backups/backup.sql
database-restore:
image: postgres:13
profiles: ["restore"]
volumes:
- ./backups:/backups
- pg_data:/var/lib/postgresql/data
command: psql -U postgres -d mydb -f /backups/backup.sql
volumes:
pg_data:
To run a backup:
docker compose --profile backup up
3. Combining Multiple Profiles
You can combine multiple profiles to create custom environments:
# Start core services with monitoring and tools
docker compose --profile monitoring --profile tools up
Best Practices
-
Default Services: Keep critical services without profiles so they always start.
-
Logical Grouping: Group related services under the same profile name.
-
Documentation: Add comments in your Docker Compose file to explain which profiles are available and what they do.
-
Consistent Naming: Use consistent profile naming conventions across projects.
-
Avoid Over-Profiling: Don't create too many profiles that make your file difficult to manage.
Common Issues and Solutions
Issue 1: Services Not Starting
If a service isn't starting when expected, verify:
- The profile name is spelled correctly
- The service is assigned to the profile you're activating
- The service doesn't have dependencies that are in non-activated profiles
Issue 2: Environment Variable Not Working
If using COMPOSE_PROFILES
environment variable and services aren't starting:
- Make sure there are no spaces between profile names in the variable
- Verify the environment variable is exported correctly
- Try using the command-line flag instead to isolate the issue
Summary
Docker Compose Profiles provide a powerful way to manage service groupings within a single Compose file. By using profiles, you can:
- Maintain a single
docker-compose.yml
file for all environments - Selectively start services based on your current needs
- Organize services into logical groups
- Create specialized environments for development, testing, and production
This feature significantly improves the flexibility and maintainability of your Docker Compose configurations, especially for complex applications with multiple services.
Additional Resources
To learn more about Docker Compose Profiles, check out:
- The official Docker documentation on Compose Profiles
- Docker Compose CLI reference for profile-related commands
- Docker Compose file specification version 3.9+
Exercises
-
Basic Profile Setup: Create a Docker Compose file with at least three services, assigning different profiles to each. Practice starting different combinations using profile flags.
-
Development Environment: Set up a development environment with debug tools in one profile and production services in another.
-
Microservices Practice: Create a microservices-based Compose file with at least five services organized into logical profiles.
-
Profile Dependencies: Create a scenario where some services depend on others in different profiles, and find the minimum set of profiles needed to start a specific service.
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)