Docker Compose

This guide explains how to launch Tabby using docker-compose.

CUDA

For CUDA support in Tabby, install the NVIDIA Container Toolkit. After installation, you can start Tabby with the following docker-compose.yml:

docker-compose.yml

version: '3.5'

services:
  tabby:
    restart: always
    image: tabbyml/tabby
    command: serve --model StarCoder-1B --device cuda
    volumes:
      - "$HOME/.tabby:/data"
    ports:
      - 8080:8080
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

CPU

docker-compose.yml

version: '3.5'

services:
  tabby:
    restart: always
    image: tabbyml/tabby
    entrypoint: /opt/tabby/bin/tabby-cpu
    command: serve --model StarCoder-1B
    volumes:
      - "$HOME/.tabby:/data"
    ports:
      - 8080:8080