All lessons
DevOps & cloudIntermediate13 min readIn my stack

Docker

Package an app with everything it needs, then run it the same way everywhere.

  • Images
  • Containers
  • Compose
  • Volumes
  • Multi-stage builds
Step 1/4
“It works on my machine!” — but the server has a different PHP version, and everything breaks.

In 30 seconds

Docker packs your app, its runtime and its libraries into an image: a sealed, versioned box. Running that image gives you a container, an isolated process that behaves the same on your laptop, in CI and on the server. Docker Compose then describes a whole system (app, database, cache, workers, web server) in one file you can start with one command.

Why I use it

  • I use it because "works on my machine" stops being an excuse: the server runs exactly the image that CI built and tested.
  • I use Compose to describe a full system in one readable file, so a new developer gets the app, database, Redis and workers running in minutes.
  • I use it because upgrades become boring: change one image tag, rebuild, and roll back by pointing at the previous tag if something breaks.
  • It keeps servers clean. Different apps can need different PHP or Node versions without fighting over what is installed on the host.

How it works

01Images, containers and how they differ from VMs

An image is a read-only template: a filesystem snapshot plus metadata such as which command to run. A container is a running instance of an image, with a thin writable layer on top. Think of the image as a recipe card and the container as the dish you cooked from it; you can cook many dishes from one card. A virtual machine emulates a whole computer with its own kernel, so it boots an entire operating system and usually takes gigabytes and tens of seconds. A container is just a normal Linux process that the kernel isolates with namespaces (what it can see: files, network, process list) and limits with cgroups (how much CPU and memory it can use). It shares the host kernel, so it starts in milliseconds and adds little overhead. The trade-off: isolation is weaker than a VM, which is why clouds often run containers inside VMs.

  • Image = immutable template, identified by name:tag and a content digest.
  • Container = image + writable layer + an isolated process.
  • Containers share the host kernel; VMs bring their own.

02Layers and build caching

Every instruction in a Dockerfile (FROM, COPY, RUN) creates a layer, and an image is a stack of layers merged by an overlay filesystem. Layers are cached by content: if an instruction and everything before it are unchanged, Docker reuses the cached layer instead of running it again. The moment one layer changes, every layer after it is rebuilt. That rule decides how fast your builds are. Copy the dependency manifest first (package.json and the lockfile, or composer.json and composer.lock), install dependencies, and only then copy the rest of the source. Now editing a component does not trigger a full npm install. A .dockerignore file keeps node_modules, .git, logs and .env files out of the build context, which makes builds faster and stops secrets leaking into images.

bash
# .dockerignore
node_modules
.next
.git
*.log
.env*
Dockerfile
compose*.yaml

03A production Dockerfile: multi-stage, small, non-root

A multi-stage build uses one stage with compilers and dev dependencies to build the app, then copies only the result into a clean, small runtime stage. The final image has no source maps you did not intend, no build tools and far fewer packages to patch. Use slim or Alpine base images and pin a major version rather than latest. Run as an unprivileged user: if someone exploits the app, they land as a normal user, not root. The example below builds a Next.js app with output: "standalone" set in next.config, which produces a minimal server.js plus only the node_modules it needs. The same pattern works for Laravel: a Composer stage, a Node stage for assets, and a final PHP-FPM stage.

dockerfile
FROM node:24-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
 
FROM node:24-alpine AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
 
FROM node:24-alpine AS run
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build --chown=node:node /app/.next/standalone ./
COPY --from=build --chown=node:node /app/.next/static ./.next/static
COPY --from=build --chown=node:node /app/public ./public
USER node
CMD ["node", "server.js"]

04Volumes and networks

A container's writable layer disappears when the container is removed, so anything you want to keep (database files, uploads) must live in a volume. Named volumes are managed by Docker and are the right choice for databases. Bind mounts map a host folder into the container and are ideal in development, when you want code changes to appear instantly. On the network side, containers on the same user-defined network find each other by service name through Docker's built-in DNS: the app connects to db:5432, not localhost. Inside a container, localhost means that container itself, which is the most common beginner confusion. Only publish ports that the outside world truly needs, usually just 80 and 443 on the reverse proxy.

  • Named volume: persistent data owned by Docker (databases).
  • Bind mount: a host folder, great for live code in development.
  • Published ports skip ufw rules on many setups; bind private ports to 127.0.0.1 or do not publish them at all.

05Docker Compose for a full application

Compose describes several containers, their networks and volumes in one compose.yaml, and docker compose up -d starts everything. Below is a typical web app: nginx in front as the only public entry point, the app container, a queue worker built from the same image with a different command, PostgreSQL with a persistent volume and a health check, and Redis for cache and queues. depends_on with condition: service_healthy makes the app wait until the database actually accepts connections, not merely until its container has started. Note that the database and Redis publish no ports at all; only nginx is reachable from outside. Secrets come from an env file that is not committed to Git.

yaml
services:
  nginx:
    image: nginx:stable-alpine
    ports: ["80:80", "443:443"]
    depends_on: [app]
  app:
    build: .
    env_file: .env
    depends_on:
      db: { condition: service_healthy }
  worker:
    build: .
    command: php artisan queue:work --tries=3
  db:
    image: postgres:17
    volumes: [db-data:/var/lib/postgresql/data]
    healthcheck: { test: ["CMD", "pg_isready", "-U", "postgres"], interval: 5s }
  redis:
    image: redis:7-alpine
volumes: { db-data: {} }

06Development vs production Compose

Development and production want different things. In development you want bind-mounted source code, hot reload, debug tools and maybe a mail catcher. In production you want built images with the code baked in, restart: unless-stopped, resource limits, log rotation, TLS and monitoring. Do not maintain two divergent files. Keep a base compose.yaml with what is shared, and layer override files on top: compose.override.yaml is loaded automatically in development, while production is started explicitly with the base plus a production file. You can add more layers, such as a TLS file or an observability file that adds Prometheus, Grafana and Loki. Validate the merged result with docker compose config before deploying.

bash
# development: compose.yaml + compose.override.yaml (auto-loaded)
docker compose up -d
 
# production: explicit files, no override
docker compose -f compose.yaml -f compose.prod.yaml config --quiet
docker compose -f compose.yaml -f compose.prod.yaml up -d --build
 
# useful day to day
docker compose ps
docker compose logs -f --tail=100 app
docker compose exec app sh

Use it when

  • Any app with more than one moving part (app, database, cache, workers) that new developers need to run quickly.
  • When servers must run different runtime versions side by side without conflicts.
  • When you want CI to build one artifact that is promoted unchanged from testing to production.
  • As the first step before any orchestrator: Kubernetes, ECS and Cloud Run all run container images.

Skip it when

  • A single static site or a small PHP site on shared hosting, where Docker adds moving parts and no benefit.
  • When the team has no one to own image updates and security patches; stale images rot quietly.
  • Desktop GUI apps or workloads that need a different kernel; use a VM instead.

Alternatives

  • Plain VPS setup

    nginx, PHP-FPM or Node with systemd/pm2 installed directly. Simpler for one or two apps, harder to reproduce.

  • Podman

    Daemonless, rootless-by-default container engine, largely compatible with Docker commands and Compose files.

  • Virtual machines

    Stronger isolation and a full OS per workload; heavier and slower to start.

  • Platform as a service

    Heroku-style platforms build and run your app for you; less control, less operations work.

Best practices

  1. Use multi-stage builds and slim base images; pin major versions instead of latest.
  2. Order Dockerfile steps from least to most frequently changing so the cache works for you.
  3. Run containers as a non-root user and never bake secrets into images; pass them at runtime.
  4. Expose only the reverse proxy; keep databases and Redis on the internal network.
  5. Add health checks and restart policies so failures are detected and recovered automatically.
  6. Configure log rotation (the default json-file driver grows forever) and prune old images regularly.
  7. Rebuild images on a schedule to pick up base-image security patches.

Common mistakes

  • Storing database data in the container's writable layer and losing it on docker compose down -v or a rebuild.
  • Publishing the database port to 0.0.0.0 and assuming the host firewall still blocks it.
  • Copying the whole project before installing dependencies, so every code change reinstalls everything.
  • Using localhost inside a container to reach another container instead of the service name.
  • Shipping .env files or .git folders inside the image because there was no .dockerignore.

Deep dive

07What a container really is: the runtime chain and PID 1

When you type docker run, the CLI talks to the Docker daemon (dockerd), which hands the job to containerd, which uses a low-level OCI runtime, usually runc, to create the process. runc asks the kernel for new namespaces, attaches the process to a cgroup, mounts the image layers through overlayfs, and then gets out of the way. The image and runtime formats are open OCI standards, which is why Podman, Kubernetes and Docker can all run the same image. One detail bites people in production: the first process in a container is PID 1. The kernel gives PID 1 no default signal handlers, so a process that never installed a SIGTERM handler simply ignores it, and PID 1 is also expected to reap zombie child processes. A tiny init such as tini, enabled in Compose with init: true, forwards signals and reaps zombies for you.

yaml
services:
  app:
    image: registry.example.com/app:1.8.0
    init: true               # tini as PID 1: forwards signals, reaps zombies
    stop_grace_period: 30s   # time between SIGTERM and SIGKILL (default 10s)
    stop_signal: SIGTERM

08Hardening a container at runtime

A non-root user is the first step, not the last. By default a container still gets a set of Linux capabilities, a writable root filesystem and the ability to gain privileges through setuid binaries. Drop all capabilities and add back only what is needed (a web server binding to port 80 needs NET_BIND_SERVICE; most Node and PHP apps behind a proxy need nothing). Make the root filesystem read-only and give the app a small tmpfs for scratch files, so an attacker cannot drop tools onto disk. Set no-new-privileges so setuid tricks fail. Docker's default seccomp profile already blocks dozens of dangerous system calls; never disable it with seccomp=unconfined. Above all, never mount /var/run/docker.sock into an app container: access to the socket is equivalent to root on the host. For an extra layer, rootless mode runs the daemon itself as a normal user.

yaml
services:
  app:
    user: "1000:1000"
    read_only: true
    tmpfs: [/tmp]
    cap_drop: [ALL]
    security_opt: ["no-new-privileges:true"]
    deploy:
      resources:
        limits: { cpus: "1.0", memory: 512M }
    pids_limit: 200

09Docker, iptables and the firewall surprise

When you publish a port with ports: "5432:5432", Docker writes NAT rules into iptables so that traffic to the host's port is forwarded to the container. Those rules are evaluated in the PREROUTING and FORWARD paths, before the INPUT chain where ufw keeps its rules. The result: ufw status says port 5432 is denied, yet the database answers from the internet. There are three safe answers. Do not publish internal services at all; containers on the same network do not need it. If a host tool must reach a container, bind to loopback with 127.0.0.1:5432:5432. If you really must filter published ports, put your rules in the DOCKER-USER chain, which Docker evaluates before its own rules and never overwrites. After any change, verify from outside with a port scan rather than trusting the firewall's own report.

yaml
services:
  db:
    image: postgres:17
    # reachable only from the host itself, never from the internet
    ports: ["127.0.0.1:5432:5432"]
  redis:
    image: redis:7-alpine
    # no ports at all: only containers on the same network can connect
 
# from another machine: nmap -Pn -p 5432,6379 203.0.113.10

10Faster and safer builds with BuildKit

BuildKit, the default builder in current Docker, adds features that plain layer caching cannot. A cache mount keeps the package manager's download cache between builds without putting it into any layer, so a lockfile change re-installs from a warm cache instead of the network. A secret mount exposes a token (for a private npm registry or Composer auth) to a single RUN step; it never appears in the image history, unlike ARG or ENV values, which anyone can read with docker history. In CI, runners start empty, so export the cache to your registry and import it on the next run. For reproducibility, pin base images by digest as well as tag, and let automation such as Renovate or Dependabot open pull requests when the digest changes. BuildKit can also attach an SBOM and provenance attestation, which security scanners and auditors increasingly expect.

dockerfile
# syntax=docker/dockerfile:1
FROM node:24-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci
 
# build: docker build --secret id=npmrc,src=$HOME/.npmrc .
# CI:    docker buildx build --cache-from type=registry,ref=registry.example.com/app:cache \
#          --cache-to type=registry,ref=registry.example.com/app:cache,mode=max --push .

Real-world scenarios

01

The server disk is full and Docker is the cause

Problem

Monitoring warns that the root disk is at 95%. The app still runs, but the database will stop accepting writes when the disk fills, and nobody has cleaned the server in months.

How to solve it

Start with docker system df to see what is large: images, build cache, volumes or containers. Remove old images with docker image prune -a --filter until=168h and the build cache with docker builder prune; never prune volumes blindly, because that is where database data lives. Then look for container logs under /var/lib/docker/containers, since the default json-file driver never rotates. Set max-size and max-file in /etc/docker/daemon.json, restart the daemon, and recreate containers, because log options apply only to new containers. Finally add a weekly prune job and a disk-usage alert at 80% so this never becomes urgent again.

02

A container keeps restarting with exit code 137

Problem

After a release, the app container restarts every few minutes. Logs end abruptly with no stack trace, and docker ps shows a growing restart count.

How to solve it

Exit code 137 means the process received SIGKILL, and the usual sender is the out-of-memory killer. Confirm with docker inspect --format '{{.State.OOMKilled}}' app and watch usage with docker stats. Then decide whether the limit or the app is wrong. For Node, set --max-old-space-size below the container limit, otherwise the heap can grow past it before garbage collection runs. Look for the leak by comparing memory over time, often an unbounded in-memory cache or a growing array of listeners. Raise the limit only if real usage justifies it, and add a memory alert so the next regression shows up before restarts do.

03

The database is reachable from the internet despite ufw

Problem

A routine scan shows PostgreSQL answering on the public IP. ufw clearly denies port 5432, and the team cannot understand how traffic gets through.

How to solve it

Treat it as an incident first. Rotate the database password and check the database logs for unknown logins. Then fix the cause: the Compose file publishes 5432:5432, and Docker's iptables rules forward that traffic before ufw sees it. Remove the ports entry so only containers on the internal network can connect, or bind it to 127.0.0.1 if a host tool needs it. Recreate the service, then verify from another machine with nmap -Pn -p 5432 203.0.113.10. Add the same external scan to a monthly checklist, and review every Compose file for published ports other than 80 and 443.

Cheat sheet

See what uses disk
docker system df -v
Remove images older than a week
docker image prune -a --filter until=168h
Rotate container logs (daemon.json)
{ "log-driver": "json-file", "log-opts": { "max-size": "10m", "max-file": "3" } }
Check a health check's result
docker inspect --format '{{json .State.Health}}' app
Was it killed for memory?
docker inspect --format '{{.State.OOMKilled}} {{.State.ExitCode}}' app
Live CPU and memory per container
docker stats --no-stream
Rebuild and restart one service only
docker compose up -d --no-deps --build app
Validate merged Compose files
docker compose -f compose.yaml -f compose.prod.yaml config --quiet
Scan an image for known CVEs
trivy image registry.example.com/app:1.8.0
Show who built which layer
docker history --no-trunc registry.example.com/app:1.8.0

In my real projects

  • The multi-tenant POS SaaS runs on a dedicated server with Docker Compose: separate dev, prod, TLS and observability override files.
  • That stack puts nginx as a reverse proxy in front of the app, with PostgreSQL, Redis and BullMQ workers as their own services, behind a CDN.
  • Laravel projects use Sail (Docker) for local development, so every developer gets the same PHP, database and Redis versions.

Questions & answers

Q1What is the difference between an image and a container?

An image is an immutable, layered template: a filesystem plus metadata like the default command and exposed ports. It is identified by a tag and, more precisely, a content digest. A container is a running instance of that image: Docker adds a thin writable layer on top and starts an isolated process with its own namespaces and cgroup limits. You can run many containers from one image, and removing a container throws away its writable layer, which is why persistent data belongs in volumes. A good mental model: the image is the class, the container is the object.

Q2How is Docker different from a virtual machine?

A virtual machine runs a complete guest operating system with its own kernel on top of a hypervisor, so it is heavy: gigabytes of disk, its own boot process, and strong hardware-level isolation. A container is an ordinary process on the host that the Linux kernel isolates with namespaces and limits with cgroups. It shares the host kernel, so it starts in milliseconds and has almost no overhead. The cost is weaker isolation: a kernel vulnerability affects every container on the host. In practice both are combined: cloud providers run your containers inside VMs, getting density from containers and isolation from the VM boundary.

Q3How do you make Docker images smaller and builds faster?

First, multi-stage builds: compile in a stage with all the tooling, then copy only the build output into a slim runtime image. Second, choose a small base such as Alpine or a slim Debian variant, and install only what the app needs at runtime. Third, order instructions for caching: copy the lockfile, install dependencies, then copy the source, so code edits do not reinstall packages. Fourth, a strict .dockerignore keeps node_modules, .git and build output out of the context. Finally, BuildKit cache mounts and a registry-backed cache in CI let repeated builds reuse downloads between runs.

Q4What is the difference between CMD and ENTRYPOINT?

ENTRYPOINT defines the executable the container always runs; CMD provides default arguments, or the default command when there is no ENTRYPOINT. Arguments passed to docker run replace CMD but are appended to ENTRYPOINT. A common pattern is an entrypoint script that prepares the environment, for example waiting for the database or caching config, and then runs exec "$@" to hand over to CMD. Use the JSON array form, like CMD ["node", "server.js"], so your process runs as PID 1 and receives stop signals directly. The shell form wraps it in sh, which can swallow SIGTERM and break graceful shutdown.

Q5Why can my app container not connect to the database on localhost?

Because inside a container, localhost is that container's own network namespace, not the host and not other containers. With Compose, all services join a shared network and Docker provides DNS, so the app should connect to the service name, for example host db and port 5432. Make sure both services are on the same network, that the database is actually ready (use a health check with depends_on condition: service_healthy), and that it listens on all interfaces inside its container. You do not need to publish the database port for this; publishing is only for traffic coming from outside Docker.

Q6Is Docker Compose good enough for production?

For a single server, yes, and it is often the right call. Compose gives you declarative services, health checks, restart policies, volumes and networks, and it is easy for a small team to understand. What it does not give you is multi-host scheduling, automatic failover across machines or built-in rolling updates. So it fits well when one well-sized server, with good backups and monitoring, meets your availability target. When you need several nodes, zero-downtime rollouts across them and automatic rescheduling after a node dies, look at Kubernetes, ECS or a lighter tool like Kamal or Swarm.

Senior-level questions

S1How does graceful shutdown work for containers, and why do some apps lose requests on every deploy?

On docker stop or a Compose recreate, Docker sends the stop signal, SIGTERM by default, to PID 1, waits for the grace period (10 seconds by default), then sends SIGKILL. Requests are lost when that signal never reaches the app or the app ignores it. The shell form of CMD runs the app under sh, which does not forward signals; a process running as PID 1 without its own handler ignores SIGTERM entirely. The fix is the exec form of CMD, an init such as tini, and an app that on SIGTERM stops accepting connections, finishes in-flight requests and closes database pools. The grace period must be longer than the slowest request or job. Behind a proxy, also make sure the proxy stops routing to the old container before it is stopped, or retries idempotent requests.

S2Alpine, Debian slim or distroless: how do you choose a base image?

Alpine is tiny and has few packages to patch, but it uses musl instead of glibc. Most code does not care, yet native modules, some Python wheels, DNS resolution edge cases and performance-sensitive code can behave differently or need compiling from source, which slows builds. Debian or Ubuntu slim images are larger but use glibc, so prebuilt binaries just work and debugging tools are easy to add. Distroless images contain only the runtime and your app, no shell or package manager, which shrinks the attack surface and the CVE list but makes interactive debugging harder; you rely on logs, metrics and ephemeral debug containers instead. My default is slim for compatibility, Alpine when I have tested the stack on it, and distroless for mature services with good observability.

S3How do you secure the container supply chain, from base image to production?

Think of three stages. At build time: minimal, pinned base images, multi-stage builds so compilers never ship, no secrets in layers (BuildKit secret mounts instead), and an SBOM generated with the image. In the registry: scan every image for known vulnerabilities with Trivy, Grype or Docker Scout, fail the pipeline on critical findings that have a fix, and sign images, for example with cosign, so you can prove an image came from your CI. At runtime: deploy by digest rather than mutable tag, run as non-root with dropped capabilities and a read-only filesystem, and rebuild on a schedule so base-image patches arrive even when your code does not change. The trade-off is noise: scanners report many unfixable or irrelevant CVEs, so define a clear policy for what blocks a release.

S4Can you deploy with Docker Compose on one server without downtime?

Not out of the box: docker compose up recreates a changed container by stopping the old one and starting the new one, so there is a gap of seconds while the app boots. There are several ways around it. Run two replicas of the app behind nginx and update them one at a time, waiting for each health check before moving on. Or use a blue-green pair of services, start the idle colour with the new image, check its health endpoint, then switch the nginx upstream and reload, which is graceful. Tools like Kamal or the docker-rollout plugin automate exactly this pattern. Whatever you choose, the app needs graceful shutdown, a real readiness endpoint, and migrations that are compatible with both versions, because for a short time old and new containers serve traffic together.

Glossary

Namespace
A kernel feature that gives a process its own view of files, network, users or process IDs, isolating it from others.
cgroup
A kernel feature that limits and measures how much CPU, memory and I/O a group of processes may use.
OCI
The Open Container Initiative: open standards for image formats and runtimes, so different tools can run the same images.
containerd
The container runtime daemon that pulls images and manages container lifecycles; Docker and Kubernetes both use it.
runc
The low-level tool that actually creates a container process with namespaces and cgroups, following the OCI runtime spec.
Overlay filesystem
A filesystem that stacks read-only image layers and one writable layer so they appear as a single directory tree.
Image digest
A sha256 hash of the image content; unlike a tag, it can never point to different content later.
BuildKit
Docker's modern build engine, adding parallel stages, cache and secret mounts, remote caches and attestations.
PID 1
The first process in a container; it gets no default signal handling and must reap zombie child processes.
Rootless mode
Running the Docker daemon and containers as a normal user, so a container escape does not give root on the host.

Have a product in mind?Let's build it.

Tell me about your idea, your product or your team. I usually reply within one working day.