AltFreeStack
ESEN
Artificial Intelligencedocker-readypermissive-license100% FOSS🟢 Lightweight · ~256MB

vLLM: Open Source alternative to OpenAI API in 2026

vLLM is a high-performance inference engine for serving large-scale language models with maximum throughput, exposing an OpenAI-compatible API for production.

Alternative to OpenAI API
License Apache-2.093.7k starsWebsite GitHub Repository

Audit Snapshot

View notes →

  • License — Matches GitHub

  • GitHub — Active

  • Latest release — v0.31.0 · 8 days ago

  • Docker Compose — Partial

  • Production — Not production-tested by AltFreeStack

vLLM vs. OpenAI API

Pros

  • Purpose-built for serving LLMs in production at scale

Things to consider

  • Requires a GPU with enough VRAM for the chosen model

Quick Docker installation guide

Copy this docker-compose.yml, adjust the example passwords and run docker compose up -d on your server. See the full step-by-step guide →

💡 Official SaaS or self-hosted with AltFreeStack?

  • On the official cloud: a recurring monthly fee and usage limits on OpenAI API.
  • On your own VPS: 100% free software on your own server, with no artificial row, project or user limits.
docker-compose.yml
version: "3.9"
services:
  vllm:
    image: vllm/vllm-openai:v0.30.0
    restart: unless-stopped
    ports:
      - "8000:8000"
    command: ["--model", "mistralai/Mistral-7B-Instruct-v0.2"]
    volumes:
      - vllm_cache:/root/.cache/huggingface
volumes:
  vllm_cache:

🚀 Deploy in 1 command

Recommended for beginners

Copy this single command and paste it into your freshly created VPS's console (Vultr, DigitalOcean...). It downloads the script, prints it to your screen so you can see exactly what it's about to do (never a blind direct pipe), and only then installs Docker if needed, writes the files with secrets already generated, and starts everything — no nano, no pasting code blocks.

curl -sSL ".../api/deploy?stack=vllm" -o deploy.sh && cat deploy.sh && bash deploy.sh
See what this script does →

This docker-compose corresponds to an actively maintained project — last commit today.

This docker-compose not working for you? Report it on GitHub →

Technical profile

License
Apache-2.0
Category
Artificial Intelligence
Replaces
OpenAI API
Tech stack
PythonCUDA

Under the hood

Database

None / File-based

Language

Python

Platforms

Web

Key features

  • Much higher throughput thanks to PagedAttention
  • OpenAI-compatible API
  • Supports dozens of model architectures
💾 Backups in 3 steps (Day 2)

As long as the ./data folder (the "volume" mapped into the container) exists on your server, your data survives any restart or container update — that's the only thing you need to back up.

1. Create a compressed backup on the server

tar -czvf backup-$(date +%F).tar.gz ./data

2. Download it to your computer

From your computer's terminal (not your server's):

scp root@YOUR_IP:~/app/backup-*.tar.gz ./

3. Restore in case of emergency

tar -xzvf backup-file.tar.gz
🌐 Connect your domain with free SSL

Step 1 — Set up the DNS

In your domain registrar's dashboard (Cloudflare, Namecheap, GoDaddy...), create a record:

Type
A
Name/Host
@ (root domain) or a subdomain, e.g. app
Content/Value
Your server's public IP

Wait 5 to 15 minutes for the record to propagate across the internet.

Step 2 — Automatic reverse proxy with Caddy

One single file, no Nginx config or manual certificates — Caddy issues and renews Let's Encrypt HTTPS on its own.

Create a file called Caddyfile:

yourdomain.com {
    reverse_proxy localhost:8000
}

Start Caddy with this command (pulls the image and runs it in the background):

docker run -d -p 80:80 -p 443:443 --name caddy --restart always -v $PWD/Caddyfile:/etc/caddy/Caddyfile -v caddy_data:/data caddy