Build a Realistic Multi-Container Application

Each tier is a service; the proxy is the only one exposed to the host.

The compose file below encodes the whole architecture. Notice what is not there: the API, worker, database, and Redis have no ports: section at all. Two user-defined networks split the topology, frontend carries proxy-to-API traffic and backend carries API/worker-to-data traffic. Nginx is attached only to frontend, so even if the proxy were compromised, an attacker could not reach PostgreSQL directly from it.

# compose.yaml
services:
  proxy:
    image: nginx:1.27-alpine
    volumes:
      - ./proxy/nginx.conf:/etc/nginx/conf.d/default.conf:ro
    ports:
      - "8080:80"
    depends_on:
      api:
        condition: service_started
    networks: [frontend]

  api:
    build: ./api
    env_file: .env
    secrets:
      - db_password
    depends_on:
      db:
        condition: service_healthy
      queue:
        condition: service_started
    networks: [frontend, backend]

  worker:
    build: ./worker
    env_file: .env
    secrets:
      - db_password
    depends_on:
      db:
        condition: service_healthy
      queue:
        condition: service_started
    networks: [backend]

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: appuser
      POSTGRES_DB: appdb
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 10
    networks: [backend]

  queue:
    image: redis:7-alpine
    networks: [backend]

networks:
  frontend:
  backend:

volumes:
  pgdata:

secrets:
  db_password:
    file: ./secrets/db_password

The API and worker are real code you build locally. Placeholder image names would leave the stack unrunnable, so each custom service gets a small Python application and a Dockerfile. Compose builds both images on first up.

# api/Dockerfile
FROM python:3.12-alpine
RUN pip install --no-cache-dir "redis" "psycopg[binary]"
COPY app.py /srv/app.py
EXPOSE 8000
CMD ["python", "/srv/app.py"]

# api/app.py - stdlib HTTP API backed by Redis and PostgreSQL
import json, os, socket, time
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse

import redis, psycopg

def read_secret(name):
    with open(f"/run/secrets/{name}") as f:
        return f.read().strip()

DB = {
    "host": os.environ.get("APP_DB_HOST", "db"),
    "user": os.environ.get("APP_DB_USER", "appuser"),
    "dbname": os.environ.get("APP_DB_NAME", "appdb"),
    "password": read_secret("db_password"),
}
_redis_url = urlparse(os.environ.get("REDIS_URL", "redis://queue:6379/0"))
QUEUE = os.environ.get("QUEUE_NAME", "jobs")
rq = redis.Redis(host=_redis_url.hostname, port=_redis_url.port or 6379, decode_responses=True)

def init_db():
    for _ in range(30):
        try:
            with psycopg.connect(**DB) as conn:
                conn.execute(
                    "CREATE TABLE IF NOT EXISTS results ("
                    "job_id BIGINT PRIMARY KEY, task TEXT NOT NULL, "
                    "duration_ms INTEGER NOT NULL, created_at TIMESTAMPTZ DEFAULT now())"
                )
            return
        except Exception:
            time.sleep(2)
    raise SystemExit("database never became ready")

class Handler(BaseHTTPRequestHandler):
    def log_message(self, fmt, *args):
        fwd = self.headers.get("X-Forwarded-For", "").split(",")[0].strip()
        print(f"{fwd or self.client_address[0]} - {fmt % args}", flush=True)

    def _json(self, code, obj):
        body = json.dumps(obj).encode()
        self.send_response(code)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def do_GET(self):
        if self.path == "/api/whoami":
            self._json(200, {"served_by": socket.gethostname()})
        elif self.path == "/api/health":
            self._json(200, {"status": "ok"})
        else:
            self._json(404, {"error": "not found"})

    def do_POST(self):
        if self.path != "/api/jobs":
            return self._json(404, {"error": "not found"})
        length = int(self.headers.get("Content-Length", 0))
        payload = json.loads(self.rfile.read(length) or b"{}")
        job = {"id": int(payload.get("id", 0)), "task": payload.get("task", "")}
        rq.rpush(QUEUE, json.dumps(job))
        self._json(200, {"queued": True, "job_id": job["id"]})

if __name__ == "__main__":
    init_db()
    HTTPServer(("0.0.0.0", 8000), Handler).serve_forever()

# worker/Dockerfile
FROM python:3.12-alpine
RUN pip install --no-cache-dir "redis" "psycopg[binary]"
COPY worker.py /srv/worker.py
CMD ["python", "/srv/worker.py"]

# worker/worker.py - pops jobs from Redis, writes results to PostgreSQL
import json, os, time
from urllib.parse import urlparse

import redis, psycopg

def read_secret(name):
    with open(f"/run/secrets/{name}") as f:
        return f.read().strip()

DB = {
    "host": os.environ.get("APP_DB_HOST", "db"),
    "user": os.environ.get("APP_DB_USER", "appuser"),
    "dbname": os.environ.get("APP_DB_NAME", "appdb"),
    "password": read_secret("db_password"),
}
_redis_url = urlparse(os.environ.get("REDIS_URL", "redis://queue:6379/0"))
QUEUE = os.environ.get("QUEUE_NAME", "jobs")
rq = redis.Redis(host=_redis_url.hostname, port=_redis_url.port or 6379, decode_responses=True)

print(f"worker up, watching queue {QUEUE!r}", flush=True)
while True:
    item = rq.blpop(QUEUE, timeout=5)
    if not item:
        continue
    job = json.loads(item[1])
    started = time.time()
    time.sleep(0.3)  # stand-in for real work
    duration_ms = int((time.time() - started) * 1000)
    with psycopg.connect(**DB) as conn:
        conn.execute(
            "INSERT INTO results (job_id, task, duration_ms) VALUES (%s, %s, %s) "
            "ON CONFLICT (job_id) DO UPDATE SET task = EXCLUDED.task, duration_ms = EXCLUDED.duration_ms",
            (job["id"], job["task"], duration_ms),
        )
    print(f'job {job["id"]} dequeued from "{QUEUE}", processed in {duration_ms}ms, result written to {DB["dbname"]}', flush=True)

Expected output:

$ docker compose config --quiet && echo "stack definition is valid"
stack definition is valid

Three details deserve attention. First, depends_on with condition: service_healthy makes the API and worker wait until PostgreSQL actually accepts connections, plain depends_on only waits for the container to start, which is a classic source of “connection refused” crashes on first boot. Second, the database reads its password from POSTGRES_PASSWORD_FILE rather than a plain environment variable, so the secret never appears in docker inspect output. Third, pgdata is a named volume: the database files survive docker compose down and container replacement.

Always run config before up. It parses the YAML, resolves variable interpolation and env_file references, and prints the final merged model. A typo like a missing network or a malformed healthcheck fails here with a line number instead of halfway through a deployment. Skim the rendered output to confirm each service landed on the networks you intended.

Common mistake: editing the file and running docker compose up without re-checking config. Compose caches nothing dangerous, but a YAML indentation error only surfaces at parse time, validate after every edit.

What this teaches: The stack definition is the architecture as data; every service, network, and secret declared in one place.

Nginx is the single public entry point. Its job is to accept client connections and forward them to the API tier by service name. Docker's embedded DNS resolves api to the IPs of all healthy API containers, and Nginx re-resolves it per request when you proxy to a variable, but for this lab the simple form below is enough, because step 05 scales the API in the same command that starts the proxy, so Nginx normally resolves all three replicas at startup. There is a startup race: if the proxy resolves before every replica registers with Docker DNS, some replicas get no traffic until you run docker compose restart proxy. Know the limitation: Nginx resolves upstream hostnames once at startup and does not track later topology changes. If you scale the API after the proxy is already up, new replicas receive no traffic until you run docker compose restart proxy.

# proxy/nginx.conf
upstream api_backend {
    server api:8000;
}

server {
    listen 80;
    server_name _;

    location /api/ {
        proxy_pass http://api_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location /healthz {
        return 200 "proxy ok\n";
    }
}

Expected output:

$ docker compose exec proxy nginx -t
nginx: configuration file /etc/nginx/nginx.conf test is successful

The X-Forwarded-For header matters more than it looks: without it, every request arriving at the API appears to come from the proxy's IP, which breaks rate limiting, audit logging, and any IP-based access control downstream. The /healthz endpoint gives you a way to tell “proxy is up” apart from “API is up” when debugging.

How to verify: after the stack is up, curl http://localhost:8080/healthz should return proxy ok even if the API containers are stopped. If it does not, the problem is in the proxy layer, not the app.

What this teaches: The reverse proxy is the only public face; routing rules decide which internal service answers which path.

Anything that varies between your laptop, CI, and production belongs in the environment, not in the image. Create a .env file (Compose reads it automatically) with the non-sensitive configuration:

# .env - non-secret config only; never commit real secrets
APP_ENV=production
APP_DB_HOST=db
APP_DB_USER=appuser
APP_DB_NAME=appdb
REDIS_URL=redis://queue:6379/0
QUEUE_NAME=jobs
LOG_LEVEL=info

Expected output:

$ docker compose exec api env | grep APP_
APP_ENV=production
APP_DB_HOST=db

Notice the hostnames: db and queue are Compose service names, resolved by Docker's internal DNS on the backend network. You never hardcode container IPs, they change on every restart.

Add .env to your .gitignore. A common pattern is to commit a .env.example with dummy values so teammates (and CI) can see which variables are required without seeing real values.

How to verify: run docker compose config again and confirm the values from .env appear in the rendered environment: section of the API and worker services. If they do not, the file is misnamed, in the wrong directory, or the service is missing the env_file: key.

What this teaches: Environment-driven config keeps the image portable; the same image behaves differently per environment.

Environment variables are convenient but leaky: they show up in docker inspect, in crash dumps, and sometimes in logs. For the database password, use Compose's file-based secrets instead. Compose mounts each secret as a read-only file under /run/secrets/ inside the container, and nothing about it appears in the container's environment or metadata.

On plain Compose (this lab), the secret is a local file that the secrets: block in compose.yaml points at. Do not use docker secret create here: it only works on a Swarm manager, and on one it would create a real Swarm secret as a side effect.

Create the secret file directly:

Because compose.yaml declares db_password: { file: ./secrets/db_password }, the API, worker, and db containers each get /run/secrets/db_password mounted at startup. The API reads that file at boot and builds its connection string; PostgreSQL does the same via POSTGRES_PASSWORD_FILE.

How to verify: once the stack is running, docker compose exec api ls -l /run/secrets/ should list db_password, and docker compose exec api env | grep -i pass should return nothing. If the password shows up in env, you accidentally passed it as a variable, remove it and use the file.

Common mistake: a trailing newline in the secret file. echo "pw" > secrets/db_password appends \n, and PostgreSQL will then reject the password the API reads. Use printf (no newline) or echo -n.

Expected output:

$ docker compose exec api ls -l /run/secrets/
-r-------- 1 root root 32 Sep 29 10:31 db_password

What this teaches: Runtime secrets arrive as files, not env vars; the application reads them from the mount path.

Start the whole stack detached, and scale the stateless API to three replicas in the same command. Compose creates the networks and volume first, waits for the database healthcheck to pass, then starts the API replicas, worker, and proxy.

Confirm the full picture with ps. You should see one proxy with a published port, three API containers, one worker, one database reporting healthy, and one Redis container:

Scaling works here because the API is stateless and the scale happens in the same command that starts the proxy, so Nginx resolves all three container IPs at startup. The worker and database are deliberately not scaled: two workers would race on the same queue (fine for some workloads, but requires idempotent job handling), and scaling PostgreSQL is a replication problem, not a Compose flag.

Common mistake: trying to scale a service that publishes a fixed host port. --scale proxy=3 fails with “port already allocated” because all three would claim 8080. Only services without host port mappings scale freely.

Expected output:

$ docker compose up -d --scale api=3
[+] Running 7/7
 Container multiapp-api-1  Started
 Container multiapp-api-2  Started
 Container multiapp-api-3  Started

What this teaches: Scaling stateless services is safe because replicas share nothing; the proxy balances across them.

Before trusting the stack, watch each tier do its job. Stream logs across all services (or filter to one):

Enqueue a test job through the API and watch the worker pick it up. The API pushes onto the Redis queue; the worker blocks on BLPOP, processes the job, and writes the result row to PostgreSQL:

How to verify: if the API says queued: true but the worker log stays silent, the two sides are pointed at different queues or different Redis instances, compare QUEUE_NAME and REDIS_URL in the rendered config for both services.

Expected output:

$ docker compose logs worker --tail=1
multiapp-worker-1  | job 42 dequeued from "jobs", processed in 301ms, result written to appdb

What this teaches: Logs plus the queue depth are the observability pair; they show what happened and what is waiting.

The whole point of the named volume is that data outlives containers. Prove it: write a row, stop the stack, bring it back, and read the row again.

Re-run the SELECT count(*), the row is still there because pgdata survived. Only docker compose down -v deletes named volumes, and that flag is how you reset the lab to a blank slate. Treat -v like rm -rf for your data tier: never run it against anything you care about.

Expected output:

$ docker compose exec db psql -U appuser -d appdb -c "SELECT count(*) FROM results;"
 count 
-------
     1
(1 row)

What this teaches: Proving persistence before teardown is the discipline; the volume surviving down is the test.