vLLM Recipes

This repo intends to host community maintained common recipes to run vLLM answering the question: How do I run model X on hardware Y for task Z?

Guides

Arcee AI

• Trinity-Large-Thinking

DeepSeek

• DeepSeek-OCR
• DeepSeek-V3.1
• DeepSeek-V3, DeepSeek-R1
• DeepSeek-V3.2-Exp
• DeepSeek-V3.2

Ernie

• Ernie4.5
• Ernie4.5-VL

GLM

• GLM-5.1, GLM-5
• Glyph
• GLM-4.7-Flash, GLM-4.7, GLM-4.6, GLM-4.5, GLM-4.5-Air
• GLM-4.5V, GLM-4.6V

Google

• Gemma 4
• TranslateGemma

inclusionAI

• Ring-1T-FP8

InternVL

• InternVL3.5

InternLM

• Intern-S1

Jina AI

• Jina-reranker-m0

Llama

• Llama4-Scout
• Llama3.3-70B
• Llama3.1

Microsoft

• Phi-4

MiniMax

• MiniMax-M2.7, MiniMax-M2.5, MiniMax-M2.1, MiniMax-M2

Xiaomi MiMo

• MiMo-V2-Flash

Mistral AI

• Ministral-3-Instruct
• Ministral-3-Reasoning
• Mistral-Large-3-Instruct

Moonshotai

• Kimi-K2.5
• Kimi-K2
• Kimi-K2-Think
• Kimi-Linear

NVIDIA

• Nemotron-3-Nano-30B-A3B
• Nemotron-Nano-12B-v2-VL

OpenAI

• gpt-oss

PaddlePaddle

• PaddleOCR-VL

Qwen

• Qwen3
• Qwen3-ASR
• Qwen3-VL
• Qwen3-Next
• Qwen3.5
• Qwen3-Coder-480B-A35B
• Qwen2.5-VL
• Qwen3Guard-Gen

Seed

• Seed-OSS-36B

StepFun

• Step-3.5-Flash

Tencent-Hunyuan

• HunyuanOCR

How recipes are maintained

Each recipe is a structured YAML file at models//.yaml, kept in the repository of the project that knows the model best. This repository is the renderer: it pulls those files in, merges them, and builds recipes.vllm.ai and its JSON API.

flowchart LR
  vllm["vllm-project/vllm<br/>recipes/models/"] --> sync
  omni["vllm-project/vllm-omni<br/>recipes/models/"] --> sync
  ascend["vllm-ascend<br/>overlays/"] -.-> sync
  self["this repo<br/>taxonomy · strategies · kv_store"] --> assemble
  sync["sync-sources.mjs<br/>pinned by sources.lock.json"] --> assemble["assemble-content.mjs<br/>→ .content/&lt;channel&gt;/"]
  assemble --> build["build-recipes-api.mjs + next build"] --> site["recipes.vllm.ai<br/>+ JSON API"]

| Content | Where it lives | Status | | --- | --- | --- | | Model recipes served by vLLM | vllm-project/vllm → recipes/models/ | Owned by vLLM; a merge there reaches recipes.vllm.ai in about 30 minutes. Copies stay in this repo's models/ until they are synced to vLLM and removed; they aren't rendered | | Omni recipes (image, video, audio generation) | vllm-project/vllm-omni → recipes/models/ | Owned by vLLM-Omni, reaching recipes.vllm.ai the same way. Copies stay in this repo's models/ until they are removed; they aren't rendered | | Ascend NPU content | vllm-project/vllm-ascend → recipes/overlays/ | Owned by vLLM Ascend; merged into the vLLM recipes, whose source rejects Huawei hardware | | Schema, hardware profiles, serving strategies, KV-store deployments, platforms | This repo (taxonomy.yaml, strategies/, kv_store/, platforms.yaml) | Owned here | | Command synthesis, site, JSON API | This repo (src/, scripts/) | Owned here |

How the pieces fit:

• sources.yaml lists every source repository for a channel (today only latest). For each source it gives the repo, ref, and directory, a role, and the scope this repo grants it:

• sources.lock.json pins every remote source to a commit, so each build is reproducible.
• Updates are automatic. bump-sources.yml runs every 20 minutes:

If validation fails, the lock stays on the last good commit, so the site keeps serving the previous content, and a sources-bump-failed issue is opened. There is one issue per failure, not one per run, and it closes itself on the next successful run.

• Provenance. Every recipe page links "Edit recipe" to the repository that owns the file. Every recipe JSON carries a source field (repo, commit, path) for tracing a command back to its YAML.

Contributing

Edit or add a recipe in the repository that owns it (see the table above), following CONTRIBUTING.md for the full schema, VRAM formula, and validation steps. Changes to the schema, hardware profiles, strategies, or the site go to this repository.

To preview a recipe change from a local checkout of a source repository, create a gitignored sources.local.yaml in this repo:

overrides:
  vllm: ~/vllm/recipes
  vllm-omni: ~/vllm-omni/recipes

Quick loop:

pnpm install
pnpm dev                              # http://localhost:3000
node scripts/sync-sources.mjs         # fetch the source commits pinned in sources.lock.json
node scripts/build-recipes-api.mjs    # assembles sources, validates every YAML, rebuilds the JSON API
node scripts/validate-source.mjs --source vllm --dir ~/vllm/recipes   # validate a source checkout (what its CI runs)
pnpm content:watch                    # re-assemble when files are added to or removed from a local source

Legacy MkDocs guides

The top-level Markdown directories (DeepSeek/, Qwen/, etc.) are the historical MkDocs site, kept as a reference during the YAML migration. To preview them:

uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
uv run mkdocs serve --dev-addr 127.0.0.1:8001

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.