Choose a profile#
| Profile | You supply | Guide |
|---|---|---|
| Existing AI endpoints | Reachable transcription/chat API roots, credentials and accepted model IDs | Install with existing endpoints |
| Self-hosted AI | NVIDIA host, model access/cache and sufficient compute for Whisper and GPT-OSS | Install self-hosted AI |
Both recipes include the API, jobs, one-shot migrations, PostgreSQL, RustFS, bucket initialization, management app and an HTTPS proxy. A client is installed or hosted separately. Product images require registry pull access; product source access is unnecessary.
Core infrastructure#
- Linux host with Docker Engine and Compose v2 supporting GPU reservations for the GPU profile. Docker Desktop can preview the docs, but is not the reference GPU deployment target.
- DNS hostname, HTTPS certificate strategy and network access from client devices. The included proxy needs ports 80/443 free and inbound 443 for its ACME TLS challenge.
- Persistent storage for PostgreSQL, recordings, upload drafts, certificates and caches; capacity and retention depend on your workload.
-
GHCR credentials authorized for the private Dicto images and, when selected,
hippolabs-org/llm-server. Accept model terms and obtain separate model access credentials where required. - New credentials for the database, auth secret, storage and bootstrap account. Decide who will back up and recover the installation before rollout.
GPU profile#
Install NVIDIA drivers and Container Toolkit compatible with your pinned inference
images. Check nvidia-smi on the host and GPU visibility inside an approved CUDA
container before troubleshooting model serving. The bundle assigns devices 0/1
to Whisper and device 2 to GPT-OSS, preserving the selected maintained allocations.
It therefore needs those devices or an explicit allocation edit.
The maintained host has 96 GB-class GPUs and additional workloads. Those allocations are not universal minimum requirements. Model weights, context size, parallelism, KV cache and concurrency determine capacity. Start from the model guides, check model access and available VRAM/RAM/cache disk, and validate on your own hardware. Model weights are downloaded separately and are not inside the setup archive.
Versions and validation#
The reference snapshot records source provenance and exact product image digests. It also states what has been checked and which GPU/end-to-end checks need an operator run. A valid Compose file is not a completed installation.