Docker Swarm
Introduction
Docker Swarm is Docker's native clustering and orchestration solution that allows you to create and manage a cluster of Docker nodes as a single virtual system. It provides tools to deploy, scale, and manage containerized applications across multiple hosts, making it easier to build and maintain highly available applications.
In this tutorial, you'll learn the fundamentals of Docker Swarm, its architecture, how to set it up, and how to deploy and manage services across your swarm.
What is Docker Swarm?
Docker Swarm transforms a group of Docker hosts into a single, virtual Docker host. It uses the standard Docker API, meaning any tool that already communicates with Docker can seamlessly work with a swarm to deploy applications.
Some key features of Docker Swarm include:
- Cluster management integrated with Docker Engine: Use the Docker CLI to create a swarm of Docker Engines
- Decentralized design: You can build an entire swarm from a single disk image
- Declarative service model: Docker Engine uses a declarative approach to define the desired state of services
- Scaling: Easily scale up or down the number of tasks for each service
- Desired state reconciliation: The swarm manager automatically reconciles the cluster to the desired state
- Multi-host networking: Overlay networks enable secure communication between containers across hosts
- Service discovery: Swarm mode includes an internal DNS component for service discovery
- Load balancing: Automatically distribute service containers across the cluster
- Secure by default: Communications are encrypted with automatic key rotation
Docker Swarm Architecture
Docker Swarm operates based on a manager-worker architecture:
Manager Nodes
Manager nodes handle cluster management tasks:
- Maintaining cluster state
- Scheduling services
- Serving the Swarm API
For high availability, you should implement multiple managers in your swarm (3-5 is recommended), with an odd number to maintain quorum.
Worker Nodes
Worker nodes execute the tasks (containers) distributed by manager nodes. They have no authority to make decisions about which containers to run.
Getting Started with Docker Swarm
Prerequisites
Before you begin, ensure you have:
- Docker Engine installed on all hosts you plan to include in your swarm
- Network connectivity between all hosts
- Open ports:
- TCP port 2377 for cluster management communications
- TCP and UDP port 7946 for communication among nodes
- UDP port 4789 for overlay network traffic
Initializing a Swarm
To create a new swarm, run the following command on the node you want to designate as the first manager:
docker swarm init --advertise-addr <MANAGER-IP>
This command initializes a new swarm and makes the current node a manager. The output will look similar to:
Swarm initialized: current node (dxn1zf6l61qsb1josjja83ngz) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-49nj1cmql0jkz5s954yi3oex3nedyz0fb0xx14ie39trti4wxv-8vxv8rssmk743ojnwacrr2e7c 192.168.99.100:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
Adding Worker Nodes
To add a worker node to your swarm, run the join command provided by the docker swarm init
output on each worker machine:
docker swarm join --token <TOKEN> <MANAGER-IP>:2377
Adding Manager Nodes
To add additional manager nodes (recommended for high availability), first get the manager join token:
docker swarm join-token manager
Then run the provided command on the new manager node:
docker swarm join --token <MANAGER-TOKEN> <MANAGER-IP>:2377
Viewing Nodes in Your Swarm
To see a list of all nodes in your swarm:
docker node ls
Example output:
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
dxn1zf6l61qsb1josjja83ngz * manager1 Ready Active Leader 19.03.13
vz1qwopzy9n01fdj6hbmg1ptt worker1 Ready Active 19.03.13
ida9ia9uilbxdozb6yj6lkxjn worker2 Ready Active 19.03.13
Deploying Services in Docker Swarm
Now that we have a swarm up and running, let's deploy services to it.
Creating a Service
A service is the definition of tasks to execute on the swarm. To create a service:
docker service create --name web-server --replicas 3 --publish 80:80 nginx
This command creates a service named web-server
with 3 replicas of the Nginx container, publishing port 80.
Listing Services
To see the services running in your swarm:
docker service ls
Example output:
ID NAME MODE REPLICAS IMAGE PORTS
i0z2jp2zy9z0 web-server replicated 3/3 nginx:latest *:80->80/tcp
Inspecting a Service
To get detailed information about a service:
docker service inspect --pretty web-server
Viewing Service Tasks
To see which nodes are running the service tasks:
docker service ps web-server
Example output:
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
v6d9mvv8lspp web-server.1 nginx:latest manager1 Running Running 2 minutes ago
7l69f0bnr3ch web-server.2 nginx:latest worker1 Running Running 2 minutes ago
pjmxj4xth8xu web-server.3 nginx:latest worker2 Running Running 2 minutes ago
Scaling a Service
To change the number of replicas for a service:
docker service scale web-server=5
Updating a Service
To update a service (for example, to use a newer image):
docker service update --image nginx:1.19 web-server
You can also update other aspects of the service, such as environment variables, ports, or resource constraints.
Docker Stack: Deploying Multi-Service Applications
For complex applications with multiple services, Docker Compose files can be used with Docker Stack to deploy to a swarm.
Example Docker Compose File
Create a file named docker-compose.yml
:
version: '3.8'
services:
web:
image: nginx:latest
ports:
- "80:80"
deploy:
replicas: 2
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
db:
image: postgres:13
environment:
POSTGRES_PASSWORD: example
volumes:
- db-data:/var/lib/postgresql/data
deploy:
placement:
constraints: [node.role == manager]
restart_policy:
condition: on-failure
volumes:
db-data:
Deploying a Stack
To deploy the stack to your swarm:
docker stack deploy -c docker-compose.yml my-app
Managing Stacks
List the stacks:
docker stack ls
List the services in a stack:
docker stack services my-app
List tasks in a stack:
docker stack ps my-app
Remove a stack:
docker stack rm my-app
Networking in Swarm Mode
Docker Swarm provides several networking capabilities:
Overlay Networks
Overlay networks enable service-to-service communication across nodes in the swarm:
docker network create --driver overlay my-network
When you create a service, you can attach it to this network:
docker service create --name my-service --network my-network nginx
Ingress Network
The ingress network is a special overlay network that facilitates load balancing among service containers. It's created automatically when you initialize a swarm.
Volumes in Swarm Mode
For persistent storage, you can use volumes with your swarm services:
docker service create \
--name db \
--mount type=volume,source=db-data,destination=/var/lib/mysql \
mysql:5.7
Secrets Management
Docker Swarm provides a secure way to manage sensitive data like passwords:
Create a secret:
echo "mypassword" | docker secret create db_password -
Use the secret in a service:
docker service create \
--name db \
--secret db_password \
--env MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db_password \
mysql:5.7
Health Checks and Service Reliability
Docker Swarm constantly monitors the health of your services and automatically restarts containers that fail:
docker service create \
--name web \
--replicas 3 \
--health-cmd "curl -f http://localhost/ || exit 1" \
--health-interval 5s \
--health-retries 3 \
--health-timeout 2s \
nginx
Common Swarm Management Commands
Here's a quick reference for commonly used Docker Swarm commands:
Command | Description |
---|---|
docker swarm init | Initialize a swarm |
docker swarm join | Join a swarm as a worker or manager |
docker swarm leave | Leave the swarm |
docker node ls | List nodes in the swarm |
docker node promote | Promote a node to manager status |
docker node demote | Demote a manager node to worker |
docker service create | Create a new service |
docker service ls | List services |
docker service ps | List the tasks of a service |
docker service scale | Scale a service |
docker service update | Update a service |
docker service rm | Remove a service |
docker stack deploy | Deploy a new stack or update an existing stack |
docker stack ls | List stacks |
docker stack services | List the services in a stack |
docker stack rm | Remove a stack |
Real-world Example: Web Application with Database and Load Balancer
Let's put it all together with a complete example of deploying a web application with a database and load balancer:
version: '3.8'
services:
web:
image: my-webapp:latest
depends_on:
- db
environment:
- DATABASE_URL=postgres://postgres:example@db:5432/mydb
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
networks:
- app-network
db:
image: postgres:13
environment:
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
POSTGRES_DB: mydb
volumes:
- db-data:/var/lib/postgresql/data
deploy:
placement:
constraints: [node.role == manager]
restart_policy:
condition: on-failure
networks:
- app-network
secrets:
- db_password
lb:
image: nginx:latest
ports:
- "80:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- web
deploy:
replicas: 2
placement:
constraints: [node.role == worker]
networks:
- app-network
networks:
app-network:
driver: overlay
volumes:
db-data:
secrets:
db_password:
external: true
To deploy this stack:
docker stack deploy -c docker-compose.yml my-app
Troubleshooting Docker Swarm
Common Issues and Solutions
-
Nodes not joining the swarm:
- Check network connectivity between nodes
- Verify that required ports are open
- Check system time synchronization
-
Service container fails to start:
- Check service logs:
docker service logs <SERVICE-NAME>
- Verify image availability:
docker pull <IMAGE>
- Check resource constraints
- Check service logs:
-
Network connectivity issues between services:
- Ensure services are on the same overlay network
- Verify DNS resolution:
docker exec <CONTAINER-ID> ping <SERVICE-NAME>
- Check for network policies or firewalls
Viewing Logs
To view logs for a service:
docker service logs <SERVICE-NAME>
Inspecting the Swarm State
To get detailed information about the swarm:
docker info
Swarm vs. Kubernetes
While Docker Swarm provides excellent orchestration capabilities, it's worth comparing it to Kubernetes, the other major container orchestration platform:
Feature | Docker Swarm | Kubernetes |
---|---|---|
Learning curve | Lower | Higher |
Setup complexity | Simple | Complex |
Scalability | Good for smaller deployments | Excellent for large-scale deployments |
Auto-scaling | Basic | Advanced |
Service discovery | Built-in DNS-based | Multiple options |
Load balancing | Built-in | More configuration options |
Rolling updates | Supported | Extensive options |
Community | Smaller | Very large |
Corporate backing | Docker, Inc. | Cloud Native Computing Foundation |
Docker Swarm is an excellent choice for:
- Teams new to container orchestration
- Smaller deployments
- Scenarios where simplicity is valued over extensive features
Summary
Docker Swarm provides a straightforward way to orchestrate containers across multiple hosts. In this tutorial, you've learned:
- How to set up a Docker Swarm cluster
- How to deploy and manage services
- How to work with stacks for multi-service applications
- Networking and storage options in Swarm mode
- Best practices for production swarm deployments
With these skills, you can now deploy containerized applications at scale, ensuring high availability and easy management.
Additional Resources
Here are some resources to continue your Docker Swarm learning journey:
- Docker's official documentation on Swarm mode: https://docs.docker.com/engine/swarm/
- Docker Compose file reference: https://docs.docker.com/compose/compose-file/
- Docker Forums for community support: https://forums.docker.com/
Exercises
To reinforce your learning, try these exercises:
- Set up a 3-node Docker Swarm cluster (1 manager, 2 workers)
- Deploy a simple web application with 5 replicas and verify load balancing
- Create a stack with a web application and database, using secrets for password management
- Simulate a node failure and observe how Docker Swarm handles container rescheduling
- Perform a rolling update of your application to a new version without downtime
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)