Running in Docker

Base images

DipDup provides multiple prebuilt images for different environments hosted on Docker Hub. Choose the one according to your needs from the table below.

defaultpytezosslim
base imagepython:3.10-slimpython:3.10-slimpython:3.10-alpine
platformsamd64, arm64amd64, arm64amd64, arm64
latest tag66-pytezos6-slim
image size352M481M136M
dipdup init command
git and poetry included
PyTezos included

Default DipDup image is suitable for development and testing. It includes some development tools to make package management easier. If unsure, use this image.

-slim image

This image is based on Alpine Linux and has a much smaller size than the default one. As a tradeoff, it doesn't include codegen functionality (unlikely to be useful in production).

-pytezos image

The only difference with the default image is the pre-installed PyTezos library, the same as pip install dipdup -E pytezos. DipDup doesn't provide any further PyPoetry integration. Having some patience you can build a trading robot or something like that using this image. I don't know if anyone is using it. If you're the one on them, please let us know!

Nightly builds (ghcr.io)

In addition to Docker Hub we also publish images on GitHub Packages. Builds are triggered on push to any branch for developers' convenience. Do not use this registry in production!

# Slim image for `aux/arm64` branch
FROM ghcr.io/dipdup-net/dipdup:aux-arm64-slim

Writing Dockerfile

Start with creating .dockerignore for your project if it's missing.

# Ignore all
*

# Add build files
!Makefile
!pyproject.toml
!poetry.lock
!requirements**
!README.md

# Add code
!src

# Add configs
!*.yml

# Ignore caches
**/.mypy_cache
**/.pytest_cache
**/__pycache__

A typical Dockerfile looks like this:

FROM dipdup/dipdup:6
# FROM dipdup/dipdup:6-pytezos
# FROM dipdup/dipdup:6-slim

# Optional: install additional dependencies using poetry
# COPY pyproject.toml poetry.lock .
# RUN install_dependencies

# Optional: install additional dependencies using pip
# COPY requirements.txt .
# RUN install_dependencies requirements.txt

COPY . .

Note that Poetry integration is not available in the slim image.

Deploying with docker-compose

Make sure you have docker run and docker-compose installed.

Example docker-compose.yml file:

version: "3.8"

services:
  dipdup:
    build: .
    depends_on:
      - db
    command: ["-c", "dipdup.yml", "-c", "dipdup.prod.yml", "run"]
    restart: always
    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-changeme}
      - ADMIN_SECRET=${ADMIN_SECRET:-changeme}
    ports:
      - 127.0.0.1:9000:9000

  db:
    image: postgres:14
    ports:
      - 127.0.0.1:5432:5432
    volumes:
      - db:/var/lib/postgresql/data
    restart: always
    environment:
      - POSTGRES_USER=dipdup
      - POSTGRES_DB=dipdup
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-changeme}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dipdup"]
      interval: 10s
      timeout: 5s
      retries: 5

  hasura:
    image: hasura/graphql-engine:v2.11.2
    ports:
      - 127.0.0.1:8080:8080
    depends_on:
      - db
    restart: always
    environment:
      - HASURA_GRAPHQL_DATABASE_URL=postgres://dipdup:${POSTGRES_PASSWORD:-changeme}@db:5432/dipdup
      - HASURA_GRAPHQL_ENABLE_CONSOLE=true
      - HASURA_GRAPHQL_DEV_MODE=true
      - HASURA_GRAPHQL_ENABLED_LOG_TYPES=startup, http-log, webhook-log, websocket-log, query-log
      - HASURA_GRAPHQL_ADMIN_SECRET=${HASURA_SECRET:-changeme}
      - HASURA_GRAPHQL_UNAUTHORIZED_ROLE=user
      - HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true

volumes:
  db:

Environment variables are expanded in the DipDup config file; Postgres password and Hasura secret are forwarded in this example.

Create a separate dipdup.<environment>.yml file for this stack:

database:
  kind: postgres
  host: db
  port: 5432
  user: ${POSTGRES_USER:-dipdup}
  password: ${POSTGRES_PASSWORD:-changeme}
  database: ${POSTGRES_DB:-dipdup}

hasura:
  url: http://hasura:8080
  admin_secret: ${HASURA_SECRET:-changeme}
  allow_aggregations: false
  camel_case: true

sentry:
  dsn: ${SENTRY_DSN:-""}
  environment: ${SENTRY_ENVIRONMENT:-prod}

prometheus:
  host: 0.0.0.0

Note the hostnames (resolved in the docker network) and environment variables (expanded by DipDup).

Build and run the containers:

docker-compose up -d --build

We recommend lazydocker for monitoring your application.

Deploying with Docker Swarm

🚧 UNDER CONSTRUCTION

This page or paragraph is yet to be written. Come back later.

Example stack:

version: "3.8"

services:
  dipdup:
    image: ${DOCKER_REGISTRY:-ghcr.io}/dipdup-net/dipdup:${TAG:-master}
    depends_on:
      - db
      - hasura
    command: ["-c", "dipdup.yml", "-c", "dipdup.prod.yml", "run"]
    environment:
      - "POSTGRES_USER=dipdup"
      - "POSTGRES_PASSWORD=changeme"
      - "POSTGRES_DB=dipdup"
      - "HASURA_SECRET=changeme"
    networks:
      - dipdup-private
      - prometheus-private
    deploy:
      mode: replicated
      replicas: ${INDEXER_ENABLED:-1}
      labels:
        - prometheus-job=${SERVICE}
        - prometheus-port=8000
      placement: &placement
        constraints:
          - node.labels.${SERVICE} == true
    logging: &logging
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "10"
        tag: "\{\{.Name\}\}.\{\{.ImageID\}\}"

  db:
    image: postgres:14
    volumes:
      - db:/var/lib/postgresql/data
    environment: 
      - POSTGRES_USER=dipdup
      - POSTGRES_DB=dipdup
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-changeme}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - dipdup-private
    deploy:
      mode: replicated
      replicas: 1
      placement: *placement
    logging: *logging

  hasura:
    image: hasura/graphql-engine:v2.11.2
    depends_on:
      - db
    environment:
      - HASURA_GRAPHQL_DATABASE_URL=postgres://dipdup:${POSTGRES_PASSWORD:-changeme}@db:5432/dipdup
      - HASURA_GRAPHQL_ENABLE_CONSOLE=true
      - HASURA_GRAPHQL_DEV_MODE=false
      - HASURA_GRAPHQL_ENABLED_LOG_TYPES=startup, http-log, websocket-log, query-log
      - HASURA_GRAPHQL_LOG_LEVEL=warn
      - HASURA_GRAPHQL_ENABLE_TELEMETRY=false
      - HASURA_GRAPHQL_ADMIN_SECRET=${HASURA_SECRET}
      - HASURA_GRAPHQL_UNAUTHORIZED_ROLE=user
      - HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true
    networks:
      - dipdup-private
      - traefik-public
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - traefik.enable=true
        - traefik.http.services.${SERVICE}.loadbalancer.server.port=8080
        - "traefik.http.routers.${SERVICE}.rule=Host(`${HOST}`) && (PathPrefix(`/v1/graphql`) || PathPrefix(`/api/rest`))"
        - traefik.http.routers.${SERVICE}.entrypoints=http,${INGRESS:-ingress}
        - "traefik.http.routers.${SERVICE}-console.rule=Host(`${SERVICE}.${SWARM_ROOT_DOMAIN}`)"
        - traefik.http.routers.${SERVICE}-console.entrypoints=https
        - traefik.http.middlewares.${SERVICE}-console.headers.customrequestheaders.X-Hasura-Admin-Secret=${HASURA_SECRET}
        - traefik.http.routers.${SERVICE}-console.middlewares=authelia@docker,${SERVICE}-console
      placement: *placement
    logging: *logging


volumes:
  db:

networks:
  dipdup-private:
  traefik-public:
    external: true
  prometheus-private:
    external: true