Skip to main content

Docker Swarm

Introduction

Docker Swarm is Docker's native clustering and orchestration solution that allows you to create and manage a cluster of Docker nodes as a single virtual system. It provides tools to deploy, scale, and manage containerized applications across multiple hosts, making it easier to build and maintain highly available applications.

In this tutorial, you'll learn the fundamentals of Docker Swarm, its architecture, how to set it up, and how to deploy and manage services across your swarm.

What is Docker Swarm?

Docker Swarm transforms a group of Docker hosts into a single, virtual Docker host. It uses the standard Docker API, meaning any tool that already communicates with Docker can seamlessly work with a swarm to deploy applications.

Some key features of Docker Swarm include:

  • Cluster management integrated with Docker Engine: Use the Docker CLI to create a swarm of Docker Engines
  • Decentralized design: You can build an entire swarm from a single disk image
  • Declarative service model: Docker Engine uses a declarative approach to define the desired state of services
  • Scaling: Easily scale up or down the number of tasks for each service
  • Desired state reconciliation: The swarm manager automatically reconciles the cluster to the desired state
  • Multi-host networking: Overlay networks enable secure communication between containers across hosts
  • Service discovery: Swarm mode includes an internal DNS component for service discovery
  • Load balancing: Automatically distribute service containers across the cluster
  • Secure by default: Communications are encrypted with automatic key rotation

Docker Swarm Architecture

Docker Swarm operates based on a manager-worker architecture:

Manager Nodes

Manager nodes handle cluster management tasks:

  • Maintaining cluster state
  • Scheduling services
  • Serving the Swarm API

For high availability, you should implement multiple managers in your swarm (3-5 is recommended), with an odd number to maintain quorum.

Worker Nodes

Worker nodes execute the tasks (containers) distributed by manager nodes. They have no authority to make decisions about which containers to run.

Getting Started with Docker Swarm

Prerequisites

Before you begin, ensure you have:

  • Docker Engine installed on all hosts you plan to include in your swarm
  • Network connectivity between all hosts
  • Open ports:
    • TCP port 2377 for cluster management communications
    • TCP and UDP port 7946 for communication among nodes
    • UDP port 4789 for overlay network traffic

Initializing a Swarm

To create a new swarm, run the following command on the node you want to designate as the first manager:

bash
docker swarm init --advertise-addr <MANAGER-IP>

This command initializes a new swarm and makes the current node a manager. The output will look similar to:

Swarm initialized: current node (dxn1zf6l61qsb1josjja83ngz) is now a manager.

To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-49nj1cmql0jkz5s954yi3oex3nedyz0fb0xx14ie39trti4wxv-8vxv8rssmk743ojnwacrr2e7c 192.168.99.100:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

Adding Worker Nodes

To add a worker node to your swarm, run the join command provided by the docker swarm init output on each worker machine:

bash
docker swarm join --token <TOKEN> <MANAGER-IP>:2377

Adding Manager Nodes

To add additional manager nodes (recommended for high availability), first get the manager join token:

bash
docker swarm join-token manager

Then run the provided command on the new manager node:

bash
docker swarm join --token <MANAGER-TOKEN> <MANAGER-IP>:2377

Viewing Nodes in Your Swarm

To see a list of all nodes in your swarm:

bash
docker node ls

Example output:

ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
dxn1zf6l61qsb1josjja83ngz * manager1 Ready Active Leader 19.03.13
vz1qwopzy9n01fdj6hbmg1ptt worker1 Ready Active 19.03.13
ida9ia9uilbxdozb6yj6lkxjn worker2 Ready Active 19.03.13

Deploying Services in Docker Swarm

Now that we have a swarm up and running, let's deploy services to it.

Creating a Service

A service is the definition of tasks to execute on the swarm. To create a service:

bash
docker service create --name web-server --replicas 3 --publish 80:80 nginx

This command creates a service named web-server with 3 replicas of the Nginx container, publishing port 80.

Listing Services

To see the services running in your swarm:

bash
docker service ls

Example output:

ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
i0z2jp2zy9z0 web-server replicated 3/3 nginx:latest *:80->80/tcp

Inspecting a Service

To get detailed information about a service:

bash
docker service inspect --pretty web-server

Viewing Service Tasks

To see which nodes are running the service tasks:

bash
docker service ps web-server

Example output:

ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
v6d9mvv8lspp web-server.1 nginx:latest manager1 Running Running 2 minutes ago
7l69f0bnr3ch web-server.2 nginx:latest worker1 Running Running 2 minutes ago
pjmxj4xth8xu web-server.3 nginx:latest worker2 Running Running 2 minutes ago

Scaling a Service

To change the number of replicas for a service:

bash
docker service scale web-server=5

Updating a Service

To update a service (for example, to use a newer image):

bash
docker service update --image nginx:1.19 web-server

You can also update other aspects of the service, such as environment variables, ports, or resource constraints.

Docker Stack: Deploying Multi-Service Applications

For complex applications with multiple services, Docker Compose files can be used with Docker Stack to deploy to a swarm.

Example Docker Compose File

Create a file named docker-compose.yml:

yaml
version: '3.8'

services:
web:
image: nginx:latest
ports:
- "80:80"
deploy:
replicas: 2
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure

db:
image: postgres:13
environment:
POSTGRES_PASSWORD: example
volumes:
- db-data:/var/lib/postgresql/data
deploy:
placement:
constraints: [node.role == manager]
restart_policy:
condition: on-failure

volumes:
db-data:

Deploying a Stack

To deploy the stack to your swarm:

bash
docker stack deploy -c docker-compose.yml my-app

Managing Stacks

List the stacks:

bash
docker stack ls

List the services in a stack:

bash
docker stack services my-app

List tasks in a stack:

bash
docker stack ps my-app

Remove a stack:

bash
docker stack rm my-app

Networking in Swarm Mode

Docker Swarm provides several networking capabilities:

Overlay Networks

Overlay networks enable service-to-service communication across nodes in the swarm:

bash
docker network create --driver overlay my-network

When you create a service, you can attach it to this network:

bash
docker service create --name my-service --network my-network nginx

Ingress Network

The ingress network is a special overlay network that facilitates load balancing among service containers. It's created automatically when you initialize a swarm.

Volumes in Swarm Mode

For persistent storage, you can use volumes with your swarm services:

bash
docker service create \
--name db \
--mount type=volume,source=db-data,destination=/var/lib/mysql \
mysql:5.7

Secrets Management

Docker Swarm provides a secure way to manage sensitive data like passwords:

Create a secret:

bash
echo "mypassword" | docker secret create db_password -

Use the secret in a service:

bash
docker service create \
--name db \
--secret db_password \
--env MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db_password \
mysql:5.7

Health Checks and Service Reliability

Docker Swarm constantly monitors the health of your services and automatically restarts containers that fail:

bash
docker service create \
--name web \
--replicas 3 \
--health-cmd "curl -f http://localhost/ || exit 1" \
--health-interval 5s \
--health-retries 3 \
--health-timeout 2s \
nginx

Common Swarm Management Commands

Here's a quick reference for commonly used Docker Swarm commands:

CommandDescription
docker swarm initInitialize a swarm
docker swarm joinJoin a swarm as a worker or manager
docker swarm leaveLeave the swarm
docker node lsList nodes in the swarm
docker node promotePromote a node to manager status
docker node demoteDemote a manager node to worker
docker service createCreate a new service
docker service lsList services
docker service psList the tasks of a service
docker service scaleScale a service
docker service updateUpdate a service
docker service rmRemove a service
docker stack deployDeploy a new stack or update an existing stack
docker stack lsList stacks
docker stack servicesList the services in a stack
docker stack rmRemove a stack

Real-world Example: Web Application with Database and Load Balancer

Let's put it all together with a complete example of deploying a web application with a database and load balancer:

yaml
version: '3.8'

services:
web:
image: my-webapp:latest
depends_on:
- db
environment:
- DATABASE_URL=postgres://postgres:example@db:5432/mydb
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
networks:
- app-network

db:
image: postgres:13
environment:
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
POSTGRES_DB: mydb
volumes:
- db-data:/var/lib/postgresql/data
deploy:
placement:
constraints: [node.role == manager]
restart_policy:
condition: on-failure
networks:
- app-network
secrets:
- db_password

lb:
image: nginx:latest
ports:
- "80:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- web
deploy:
replicas: 2
placement:
constraints: [node.role == worker]
networks:
- app-network

networks:
app-network:
driver: overlay

volumes:
db-data:

secrets:
db_password:
external: true

To deploy this stack:

bash
docker stack deploy -c docker-compose.yml my-app

Troubleshooting Docker Swarm

Common Issues and Solutions

  1. Nodes not joining the swarm:

    • Check network connectivity between nodes
    • Verify that required ports are open
    • Check system time synchronization
  2. Service container fails to start:

    • Check service logs: docker service logs <SERVICE-NAME>
    • Verify image availability: docker pull <IMAGE>
    • Check resource constraints
  3. Network connectivity issues between services:

    • Ensure services are on the same overlay network
    • Verify DNS resolution: docker exec <CONTAINER-ID> ping <SERVICE-NAME>
    • Check for network policies or firewalls

Viewing Logs

To view logs for a service:

bash
docker service logs <SERVICE-NAME>

Inspecting the Swarm State

To get detailed information about the swarm:

bash
docker info

Swarm vs. Kubernetes

While Docker Swarm provides excellent orchestration capabilities, it's worth comparing it to Kubernetes, the other major container orchestration platform:

FeatureDocker SwarmKubernetes
Learning curveLowerHigher
Setup complexitySimpleComplex
ScalabilityGood for smaller deploymentsExcellent for large-scale deployments
Auto-scalingBasicAdvanced
Service discoveryBuilt-in DNS-basedMultiple options
Load balancingBuilt-inMore configuration options
Rolling updatesSupportedExtensive options
CommunitySmallerVery large
Corporate backingDocker, Inc.Cloud Native Computing Foundation

Docker Swarm is an excellent choice for:

  • Teams new to container orchestration
  • Smaller deployments
  • Scenarios where simplicity is valued over extensive features

Summary

Docker Swarm provides a straightforward way to orchestrate containers across multiple hosts. In this tutorial, you've learned:

  • How to set up a Docker Swarm cluster
  • How to deploy and manage services
  • How to work with stacks for multi-service applications
  • Networking and storage options in Swarm mode
  • Best practices for production swarm deployments

With these skills, you can now deploy containerized applications at scale, ensuring high availability and easy management.

Additional Resources

Here are some resources to continue your Docker Swarm learning journey:

Exercises

To reinforce your learning, try these exercises:

  1. Set up a 3-node Docker Swarm cluster (1 manager, 2 workers)
  2. Deploy a simple web application with 5 replicas and verify load balancing
  3. Create a stack with a web application and database, using secrets for password management
  4. Simulate a node failure and observe how Docker Swarm handles container rescheduling
  5. Perform a rolling update of your application to a new version without downtime


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)