Intel Arc Pro B70 Inference Cookbook
Pinned vLLM XPU and llama.cpp SYCL recipes, failure notes, and measurement records for Intel Arc Pro B60/B70. Start with what worked, what did not, and what to test next. For a first local run, use START.md.
Pick one path
| Your goal | Start here | Status | |---|---|---| | Choose a model for a job | Model comparison | 94-task quality screen of Bolt 74B-A9B (two cards), Qwen3.8-27B and Qwen3.6-35B-A3B (one card each) with power and heat; one sample per task, Qwen3.6 partial. | | Serve on one B70 | Qwen3.8-27B EXL3 route | Primary route: trellis 4bpw + MTP3 + fp8 KV on the exl3xpu engine; the only measured route at full 262,144 context and clean on the exact-answer battery. The GPTQ/W4A16 route is legacy — that class fails the canary on this model. | | Run GGUF or vision | Muse-Glimmer llama.cpp route | Measured on its own engine and workload; no cross-engine speed ranking without matched timing. | | Run MoE or long context | Qwen3.6-35B-A3B recipe | Historical MTP speed record is stack-specific; a later matched comparison did not reproduce it. | | Serve on two B70s | Independent-card setup | Default: one server per card. FP8 TP2 is a research route, not an automatic single-request win. | | Diagnose or reproduce a number | Image and patch matrix → family recipe → benchmark catalog | Match image, patch, model, context, cache, power, and timing definition. |
Read the decision guide before tuning. It separates measured improvements, failed attempts, unpublished lab screens, and stop/go gates. A passing smoke canary does not establish model quality. No public performance claim from a different model, engine, host, or result generation transfers to this setup.
Reproduce and operate
- Check your card and host with
scripts/doctor.py, then use the quick start for a pinned single-card run. - Check the image and ordered patch matrix for the exact family before launching. Do not mix family patch stacks.
- Connect a client using client setup; use the reliability guide if the server hangs.
- Confirm a recipe with the reproduction procedure and benchmark format. The numeric authority is
data/benchmarks.v1.json;docs/BENCHMARK-CATALOG.mdis generated.
Quantize a model yourself: how the Bolt 74B AutoRound W4A16 quant was made and checked on two B70s, with every command, fix and script.
Request or add a recipe
Need a model or configuration not yet covered? Submit a recipe request with your target Hugging Face model and hardware setup.
To contribute a recipe directly to the B70 Launcher catalog, follow the recipe format specification to edit recipes.json, verify with python3 -m unittest tests.test_recipes, and open a pull request.
For contributors and agents
Use data/system-map.v1.json to select an intent. docs/AGENT-SYSTEM.md explains the authority layers; docs/ADDING-A-RECIPE.md explains publication. Keep historical evidence intact. Validate changes with python3 scripts/check-repo.py.
Code: MIT. Measurement reports and prose: CC BY 4.0. See LICENSE.