AB
The final part of our Docker Compose series covers advanced configurations, production deployment strategies, security best practices, and integrations with container orchestration systems.
Welcome to the final installment of our Docker Compose series! In Part 1, we covered the fundamentals. In Part 2, we explored the docker-compose.yml
file structure. In Part 3, we examined essential commands and operations.
Now, we’ll dive into advanced topics and production considerations for Docker Compose. While Docker Compose was originally designed for development and testing environments, it can be adapted for production use with the right approach and considerations.
Before using Docker Compose in production, consider these factors:
For small to medium applications with moderate traffic, Docker Compose can be a viable production solution. For large, mission-critical applications that require high availability and auto-scaling, consider container orchestration platforms like Kubernetes or Docker Swarm.
Let’s transform a development-focused Docker Compose configuration into a production-ready setup:
A common approach is to maintain separate Compose files:
docker-compose.yml
: Base configurationdocker-compose.override.yml
: Development-specific settings (loaded automatically)docker-compose.prod.yml
: Production-specific overridesHere’s how these files might look for a web application:
docker-compose.yml (base configuration):
version: "3.9"
services:
web:
build: ./web
depends_on:
- api
- db
networks:
- frontend
- backend
api:
build: ./api
depends_on:
- db
networks:
- backend
db:
image: postgres:13
volumes:
- db-data:/var/lib/postgresql/data
networks:
- backend
networks:
frontend:
backend:
volumes:
db-data:
docker-compose.override.yml (development settings):
services:
web:
ports:
- "3000:80"
volumes:
- ./web/src:/app/src
environment:
- DEBUG=true
- API_URL=http://api:8000
api:
ports:
- "8000:8000"
volumes:
- ./api/src:/app/src
environment:
- DEBUG=true
- LOG_LEVEL=debug
- DB_HOST=db
- DB_PASSWORD=devpassword
db:
environment:
- POSTGRES_PASSWORD=devpassword
ports:
- "5432:5432"
docker-compose.prod.yml (production settings):
services:
web:
image: ${REGISTRY}/myapp-web:${TAG}
build:
context: ./web
args:
- NODE_ENV=production
ports:
- "80:80"
- "443:443"
restart: unless-stopped
deploy:
replicas: 2
resources:
limits:
cpus: "0.5"
memory: 512M
environment:
- DEBUG=false
- API_URL=http://api:8000
api:
image: ${REGISTRY}/myapp-api:${TAG}
build:
context: ./api
args:
- ENV=production
restart: unless-stopped
deploy:
replicas: 2
resources:
limits:
cpus: "1"
memory: 1G
environment:
- DEBUG=false
- LOG_LEVEL=info
- DB_HOST=db
- DB_PASSWORD=${DB_PASSWORD}
db:
restart: unless-stopped
deploy:
resources:
limits:
cpus: "2"
memory: 2G
environment:
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
- db-data:/var/lib/postgresql/data
- ./backups:/backups
To start the production configuration:
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
Note the key differences in the production configuration:
Security is critical for production deployments. Here are key areas to address:
Never commit secrets to your repository. Instead, use environment variables:
services:
db:
environment:
- POSTGRES_PASSWORD=${DB_PASSWORD}
For a more robust solution, use Docker’s secrets management:
services:
api:
secrets:
- db_password
environment:
- DB_PASSWORD_FILE=/run/secrets/db_password
secrets:
db_password:
file: ./secrets/db_password.txt # Local development
# external: true # In Swarm mode, reference an existing secret
Isolate services using multiple networks:
services:
web:
networks:
- frontend
- backend
api:
networks:
- backend
db:
networks:
- backend
networks:
frontend:
# External-facing network
backend:
# Internal-only network
internal: true
This ensures the database is not directly accessible from the internet.
postgres:13.4
) rather than using latest
Example implementing these practices:
services:
api:
image: myapp-api:1.2.3
user: "1000:1000" # Non-root user
read_only: true
tmpfs:
- /tmp
volumes:
- type: bind
source: ./data
target: /data
read_only: true # Read-only mount
cap_drop:
- ALL # Drop all capabilities
cap_add:
- NET_BIND_SERVICE # Add only what's needed
For larger production environments, you might need to combine Docker Compose with container orchestration.
Docker Compose files are compatible with Docker Swarm with a few adjustments. To deploy a Compose file to Swarm:
docker stack deploy -c docker-compose.yml -c docker-compose.prod.yml myapp
Swarm-specific features in Compose files include:
services:
web:
deploy:
mode: replicated
replicas: 3
update_config:
parallelism: 1
delay: 10s
order: start-first
restart_policy:
condition: on-failure
max_attempts: 3
placement:
constraints:
- node.role == worker
labels:
- "traefik.enable=true"
- "traefik.http.routers.web.rule=Host(`example.com`)"
Traefik is a popular reverse proxy and load balancer that works well with Docker Compose:
version: "3.9"
services:
traefik:
image: traefik:v2.5
command:
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myresolver.acme.tlschallenge=true"
- "[email protected]"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
ports:
- "80:80"
- "443:443"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./letsencrypt:/letsencrypt"
networks:
- frontend
web:
image: myapp-web:latest
labels:
- "traefik.enable=true"
- "traefik.http.routers.web.rule=Host(`example.com`)"
- "traefik.http.routers.web.entrypoints=websecure"
- "traefik.http.routers.web.tls.certresolver=myresolver"
networks:
- frontend
networks:
frontend:
This configuration:
Create a .env
file for environment-specific values:
# .env.prod
TAG=v1.2.3
REGISTRY=registry.example.com
DB_PASSWORD=secure_password
EXTERNAL_PORT=443
REPLICAS=3
Then reference these variables in your Compose file:
services:
web:
image: ${REGISTRY}/myapp-web:${TAG}
deploy:
replicas: ${REPLICAS:-2}
For complex configurations, you can use YAML extensions to avoid repetition:
x-common-config: &common-config
restart: unless-stopped
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
services:
web:
<<: *common-config
image: myapp-web
api:
<<: *common-config
image: myapp-api
Maintain different environment files:
.env.dev
.env.staging
.env.prod
Then specify which one to use:
# Load production environment
env $(cat .env.prod | xargs) docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
To maximize uptime:
services:
api:
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 1m
timeout: 10s
retries: 3
start_period: 30s
restart: unless-stopped
Design your Compose configuration for scaling:
Example of a scalable web service:
services:
web:
image: myapp-web
deploy:
replicas: 3
environment:
- SESSION_STORE=redis
redis:
image: redis:6-alpine
volumes:
- redis-data:/data
Let’s look at a complete example for deploying a production-ready application with Docker Compose:
version: "3.9"
# Common configurations
x-logging: &logging
logging:
driver: "json-file"
options:
max-size: "20m"
max-file: "5"
x-deploy: &deploy
deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
restart: unless-stopped
services:
# Reverse proxy and load balancer
traefik:
image: traefik:v2.5
<<: *logging
command:
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myresolver.acme.tlschallenge=true"
- "--certificatesresolvers.myresolver.acme.email=${ADMIN_EMAIL}"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
ports:
- "80:80"
- "443:443"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "traefik-certificates:/letsencrypt"
networks:
- frontend
deploy:
resources:
limits:
cpus: "0.5"
memory: 256M
restart_policy:
condition: any
# Frontend web application
web:
image: ${REGISTRY}/ecommerce-web:${TAG}
<<: *logging
<<: *deploy
depends_on:
- api
networks:
- frontend
- backend
labels:
- "traefik.enable=true"
- "traefik.http.routers.web.rule=Host(`${DOMAIN}`)"
- "traefik.http.routers.web.entrypoints=websecure"
- "traefik.http.routers.web.tls.certresolver=myresolver"
environment:
- API_URL=http://api:8000
- CACHE_URL=redis://redis:6379
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost/health"]
interval: 30s
timeout: 5s
retries: 3
# Backend API service
api:
image: ${REGISTRY}/ecommerce-api:${TAG}
<<: *logging
<<: *deploy
depends_on:
- db
- redis
networks:
- backend
labels:
- "traefik.enable=true"
- "traefik.http.routers.api.rule=Host(`api.${DOMAIN}`)"
- "traefik.http.routers.api.entrypoints=websecure"
- "traefik.http.routers.api.tls.certresolver=myresolver"
environment:
- DB_HOST=db
- DB_USER=${DB_USER}
- DB_PASSWORD=${DB_PASSWORD}
- DB_NAME=${DB_NAME}
- REDIS_URL=redis://redis:6379
- JWT_SECRET=${JWT_SECRET}
- LOG_LEVEL=info
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 5s
retries: 3
# Database service
db:
image: postgres:13-alpine
<<: *logging
volumes:
- db-data:/var/lib/postgresql/data
- ./backups:/backups
networks:
- backend
environment:
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASSWORD=${DB_PASSWORD}
- POSTGRES_DB=${DB_NAME}
deploy:
resources:
limits:
cpus: "2"
memory: 2G
restart_policy:
condition: any
placement:
constraints:
- node.labels.db == true # For Swarm deployment
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
interval: 30s
timeout: 5s
retries: 3
# Cache service
redis:
image: redis:6-alpine
<<: *logging
volumes:
- redis-data:/data
networks:
- backend
deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
restart_policy:
condition: any
command: ["redis-server", "--appendonly", "yes"]
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 30s
timeout: 5s
retries: 3
# Monitoring service
prometheus:
image: prom/prometheus:v2.30.0
<<: *logging
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus-data:/prometheus
networks:
- monitoring
- backend
deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
# Dashboard service
grafana:
image: grafana/grafana:8.2.0
<<: *logging
volumes:
- grafana-data:/var/lib/grafana
networks:
- monitoring
- frontend
labels:
- "traefik.enable=true"
- "traefik.http.routers.grafana.rule=Host(`monitoring.${DOMAIN}`)"
- "traefik.http.routers.grafana.entrypoints=websecure"
- "traefik.http.routers.grafana.tls.certresolver=myresolver"
environment:
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
- GF_USERS_ALLOW_SIGN_UP=false
deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
networks:
frontend:
backend:
internal: true
monitoring:
volumes:
db-data:
redis-data:
prometheus-data:
grafana-data:
traefik-certificates:
This comprehensive example includes:
To deploy this production stack:
.env.prod
file with all required variables# Set environment variables
export $(cat .env.prod | xargs)
# Login to registry
docker login $REGISTRY
# Build and push images
docker-compose -f docker-compose.yml -f docker-compose.prod.yml build
docker-compose -f docker-compose.yml -f docker-compose.prod.yml push
# Deploy the stack
docker stack deploy -c docker-compose.yml -c docker-compose.prod.yml ecommerce
Integrate monitoring to keep track of your application’s health:
services:
api:
labels:
- "prometheus.scrape=true"
- "prometheus.port=8000"
- "prometheus.path=/metrics"
Implement regular backups for stateful services:
services:
db-backup:
image: postgres:13-alpine
volumes:
- ./backups:/backups
networks:
- backend
environment:
- PGPASSWORD=${DB_PASSWORD}
command: |
sh -c 'pg_dump -h db -U ${DB_USER} ${DB_NAME} | gzip > /backups/backup_$(date +%Y%m%d_%H%M%S).sql.gz'
deploy:
restart_policy:
condition: none
Schedule this backup service to run periodically using a cron job or external scheduler.
For zero-downtime updates in a Swarm environment:
services:
web:
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
order: start-first
failure_action: rollback
This ensures:
Based on our exploration of Docker Compose in production, here’s a summary of best practices:
It depends on your scale and requirements:
Create a separate service for migrations that runs before your application starts:
services:
migrate:
image: ${REGISTRY}/api:${TAG}
command: ["./migrate.sh"]
depends_on:
- db
For simple blue-green deployments:
For proper secrets management:
Docker Compose is a versatile tool that can be adapted for production use with the right approach and considerations. While it may not replace full container orchestration platforms for large-scale applications, it offers a simpler alternative for small to medium deployments.
By following the best practices outlined in this series, you can create robust, secure, and maintainable Docker Compose configurations that work reliably in production environments.
Remember:
With these guidelines in mind, Docker Compose can be an effective part of your production deployment strategy.
Go back to Part 3: Docker Compose Commands and Operations or return to Part 1: Introduction and Fundamentals