Onyx Docker Compose setup with Traefik reverse proxy

This post shows how to set up Onyx, the open-source AI platform for workplace search and chat, using Docker Compose with Traefik as the reverse proxy. Traefik routes /api (and /openapi.json) to the API server and everything else to the web frontend.

Directory setup

First create the data directories. Vespa requires its data directory to be owned by UID 1000, while the Onyx model servers run as UID 1001 and need write access to their HuggingFace cache directories — otherwise the model download fails with OSError: couldn't find them in the cached files:

init.sh

#!/bin/sh
mkdir -p vespa_data model_cache_huggingface indexing_huggingface_model_cache
chown -R 1000:1000 vespa_data
chown -R 1001:1001 model_cache_huggingface indexing_huggingface_model_cache

Setup-Onyx.sh

mkdir -p onyx && cd onyx
./init.sh

Configuration

Create a .env file next to docker-compose.yml:

.env

# Domain Settings
DOMAIN=onyx.mydomain.com
WEB_DOMAIN=https://onyx.mydomain.com

# Database Credentials
POSTGRES_USER=postgres
POSTGRES_PASSWORD=INSERT_YOUR_PASSWORD_HERE
POSTGRES_DB=onyx

# Auth Settings (disabled / basic / oidc)
AUTH_TYPE=disabled
USER_AUTH_SECRET=INSERT_RANDOM_SECRET_HERE

# Traefik Settings
TRAEFIK_ENTRYPOINT=websecure
TRAEFIK_CERTRESOLVER=cloudflare

Generate a random USER_AUTH_SECRET with openssl rand -hex 32.

docker-compose.yml

docker-compose.yml

services:
  db:
    image: postgres:15-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - ./postgres_data:/var/lib/postgresql/data

  index:
    image: vespaengine/vespa:8
    restart: unless-stopped
    environment:
      VESPA_CLI_TELEMETRY: "false"
    volumes:
      - ./vespa_data:/opt/vespa/var

  api_server:
    image: onyxdotapp/onyx-backend:latest
    restart: unless-stopped
    command: >
      /bin/sh -c "alembic upgrade head &&
      echo 'Starting Onyx Api Server' &&
      uvicorn onyx.main:app --host 0.0.0.0 --port 8080"
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_HOST=db
      - POSTGRES_PORT=5432
      - POSTGRES_DB=${POSTGRES_DB}
      - VESPA_HOST=index
      - VESPA_PORT=8081
      - ONYX_DISABLE_VESPA=false
      - ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false
      - REDIS_HOST=cache
      - FILE_STORE_BACKEND=postgres
      - MODEL_SERVER_HOST=inference_model_server
      - INDEXING_MODEL_SERVER_HOST=indexing_model_server
      - WEB_DOMAIN=${WEB_DOMAIN}
      - AUTH_TYPE=${AUTH_TYPE}
      - USER_AUTH_SECRET=${USER_AUTH_SECRET}
      - CODE_INTERPRETER_BASE_URL=http://code-interpreter:8000
    depends_on:
      - db
      - index
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.onyx-api.rule=Host(`${DOMAIN}`) && (PathPrefix(`/api`) || PathPrefix(`/openapi.json`))"
      - "traefik.http.routers.onyx-api.entrypoints=${TRAEFIK_ENTRYPOINT}"
      - "traefik.http.routers.onyx-api.tls.certresolver=${TRAEFIK_CERTRESOLVER}"
      - "traefik.http.routers.onyx-api.middlewares=onyx-strip-api"
      - "traefik.http.middlewares.onyx-strip-api.stripprefix.prefixes=/api"
      - "traefik.http.services.onyx-api.loadbalancer.server.port=8080"

  background:
    image: onyxdotapp/onyx-backend:latest
    restart: unless-stopped
    command: /bin/bash /app/scripts/supervisord_entrypoint.sh
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_HOST=db
      - POSTGRES_PORT=5432
      - POSTGRES_DB=${POSTGRES_DB}
      - VESPA_HOST=index
      - VESPA_PORT=8081
      - ONYX_DISABLE_VESPA=false
      - ENABLE_OPENSEARCH_INDEXING_FOR_ONYX=false
      - REDIS_HOST=cache
      - FILE_STORE_BACKEND=postgres
      - MODEL_SERVER_HOST=inference_model_server
      - INDEXING_MODEL_SERVER_HOST=indexing_model_server
      - WEB_DOMAIN=${WEB_DOMAIN}
      - USER_AUTH_SECRET=${USER_AUTH_SECRET}
      - SANDBOX_BACKEND=docker
      - SANDBOX_PROXY_HOST=sandbox-proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      - db
      - index

  cache:
    image: redis:7-alpine
    restart: unless-stopped

  inference_model_server:
    image: onyxdotapp/onyx-model-server:latest
    restart: unless-stopped
    volumes:
      - ./model_cache_huggingface:/app/.cache/huggingface/

  indexing_model_server:
    image: onyxdotapp/onyx-model-server:latest
    restart: unless-stopped
    environment:
      - INDEXING_ONLY=True
    volumes:
      - ./indexing_huggingface_model_cache:/app/.cache/huggingface/

  code-interpreter:
    image: onyxdotapp/code-interpreter:latest
    command: ["bash", "./entrypoint.sh", "code-interpreter-api"]
    restart: unless-stopped
    healthcheck:
      test:
        [
          "CMD",
          "python",
          "-c",
          "import json,urllib.request; r = json.load(urllib.request.urlopen('http://localhost:8000/health', timeout=5)); raise SystemExit(0 if r['status'] == 'ok' else 1)",
        ]
      interval: 30s
      timeout: 10s
      retries: 3
      # First run pulls the executor image and does not serve until it lands.
      start_period: 120s
    # docker-out-of-docker execution mode
    user: root
    volumes:
      - ${DOCKER_SOCK_PATH:-/var/run/docker.sock}:/var/run/docker.sock

  web_server:
    image: onyxdotapp/onyx-web-server:latest
    restart: unless-stopped
    environment:
      - INTERNAL_URL=http://api_server:8080
    depends_on:
      - api_server
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.onyx-web.rule=Host(`${DOMAIN}`)"
      - "traefik.http.routers.onyx-web.entrypoints=${TRAEFIK_ENTRYPOINT}"
      - "traefik.http.routers.onyx-web.tls.certresolver=${TRAEFIK_CERTRESOLVER}"
      - "traefik.http.services.onyx-web.loadbalancer.server.port=3000"

Notes on the configuration

No explicit networks. All services run on the compose default network. As long as Traefik joins that network, it can reach api_server and web_server and no extra networks: section or traefik.docker.network label is needed. If your Traefik runs in a separate compose project, either start both files together (docker compose -f docker-compose.yml -f docker-compose.traefik.yml up -d) or point it at the same project with -p onyx.

SANDBOX_BACKEND=docker on background. Onyx’s Craft/sandbox feature runs a periodic cleanup_idle_sandboxes Celery task regardless of whether you use it. SANDBOX_BACKEND defaults to kubernetes, which crashes on a plain Docker host with RuntimeError: Failed to load Kubernetes configuration. The docker backend in turn requires a non-empty SANDBOX_PROXY_HOST and the Docker socket mounted into the container — hence the three lines above. If you don’t use Craft, the manager initializes, finds zero running sandboxes, and exits quietly.

code-interpreter. Optional service — it is the sandbox Onyx agents use to execute generated Python/Bash code. Without it the API server periodically logs a harmless Connection refused warning against localhost:8000 (the default CODE_INTERPRETER_BASE_URL). It runs in docker-out-of-docker mode, i.e. it needs the host Docker socket mounted to spawn executor containers. For a rootless Docker setup, set DOCKER_SOCK_PATH=${XDG_RUNTIME_DIR}/docker.sock in .env. If you don’t need code execution, leave the service out — nothing else depends on it.

Path routing. Traefik strips /api before forwarding to the API server (onyx-strip-api middleware), so https://onyx.mydomain.com/api/... reaches api_server as /.... All other paths go to web_server.

Start it

Start-Onyx.sh

docker compose up -d

The first start takes a few minutes while api_server runs its database migrations and the model servers download their HuggingFace models. Watch progress with docker compose logs -f api_server.

Open https://onyx.mydomain.com — on first run you’ll be asked to create an admin account, then you land on the chat UI.

You can setup autostart using Create a systemd service for your docker-compose project in 10 seconds, TL;DR:

Setup-Autostart.sh

curl -fsSL https://techoverflow.net/scripts/create-docker-compose-service.sh | sudo bash /dev/stdin