LlamaForge
English · 简体中文
The real llama.cpp server, run from a browser tab.
Pick a model, see if it fits your GPU, load it, chat. Every llama-server flag is still there when you want it.
Auto-Forge measures flags on your own GPUs and keeps the fastest: a Qwen3.8-27B went from 26.8 to 126 tokens/s on code here.
Already on Ollama? Your downloaded models run on upstream llama.cpp in place, nothing re-downloaded.
🎉 100 stars. Thank you! Who got it here · what's next
irm https://raw.githubusercontent.com/dadwritestech/LlamaForge/master/install.ps1 | iex # Windows, no admincurl -fsSL https://raw.githubusercontent.com/dadwritestech/LlamaForge/master/install.sh | sh # Linux / macOS
LlamaForge runs no models itself. It installs and drives llama.cpp's own llama-server router and
edits its models.ini for you. Not affiliated with ggml-org. If you'd rather have polish than
control, use LM Studio, Ollama or Jan.
Why LlamaForge
New model architectures land in llama.cpp first. Desktop apps pass them on when they next update their bundled engine. LlamaForge runs the official llama.cpp release itself (or your own build or fork), so a new model is one Update click away, and it puts a UI over every server flag instead of a curated subset.
Discover opens on what llama.cpp just learned to run, checked against the engine you have.
| | LlamaForge | LM Studio | Ollama |
|---|---|---|---|
| Open source | ✅ MIT | ❌ proprietary app | ✅ MIT |
| Engine | official upstream llama.cpp builds, any version, or your own fork | LM Studio's bundled llama.cpp / MLX runtimes | Ollama's own engine on ggml |
| Per-model settings | every flag your llama-server --help lists (200+ on current builds) | many, curated | Modelfile parameters |
| Any GGUF from Hugging Face, rated for your VRAM before download | ✅ (a rough estimate) | ✅ | pulls GGUFs, no fit rating |
| OpenAI + Anthropic-compatible API | ✅, plus one-click Claude Code / Codex / pi.dev config | ✅ | ✅ |
| Backend dependencies | none (Python stdlib) | – | – |
| Native desktop app | ❌ runs in your browser | ✅ | ✅ |
| Maturity | early preview | mature | mature |
As of October 2026, to the best of our knowledge. Spot something wrong? A PR to fix this table is very welcome.
What's in it
- Models: every model on your machine in one list, with live VRAM/util/temp per GPU. Load, unload, or open Chat from the row. Already have Ollama? The scan finds the models it downloaded and runs those same files on llama.cpp: nothing is copied.
models.ini with one-click Undo. Here it took a Qwen3.8-27B from 26.8 tokens/s on code (mostly on the slower GPU) to 126.0, and to 2.4x the best plain setup. Share turns a result into an image card and a markdown table to post, with no paths, keys or hosts. How it works.
!Auto-Forge: from 27 to 126 tokens a second
- Turn on Multi-model in Setup to keep a main model and workers loaded at once. A planner places each one on the GPUs that fit it, using footprints measured on your machine. A model can also be pinned to its own llama.cpp (or ik_llama.cpp) build.
- Chat: llama.cpp's own chat UI (markdown, reasoning, images) inside the dashboard, on its own port, with the API key added for you.
- Voice: text to speech on your GPU through llama.cpp's own
llama-tts: Qwen3-TTS in 10 languages, or the tiny English Pocket TTS that runs faster than real time even on a CPU, optionally in a voice you record or upload (only clone voices you may use). - Embers: small local agents that keep watch on a topic, keep their own wiki and write you a brief on a schedule. A wiki item only counts if it quotes its source verbatim. Forge builds one by interviewing you; Model Scout needs no setup. Embers have no browser or tools, and nothing leaves the machine unless you turn on push notifications.
- Discover: Hugging Face GGUF search that opens on what's new this week. Every quant gets a rough fit rating for your VRAM before you download (FITS / TIGHT / CPU OFFLOAD). Downloads resume after interruption, register themselves, and end in Load & Chat. A link like
http://127.0.0.1:8090/#discover?repo=owner/name&file=x.ggufopens it on one quant. - Will it run?: pick a repo and quant, get the fit and a rough speed estimate.
- Can I run it? without installing: a table of popular models vs graphics cards, worked out from the real GGUF headers with the same footprint math.
- Build / Update: one-click official llama.cpp builds with rollback, or build from source with flags detected for your GPU. Also drives ik_llama.cpp and, on Windows, vLLM in WSL2.
- Stats: per-model tokens, speed and run counts from the router's own metrics, plus live GPU memory and system RAM. Share my rig turns it into a postable card. Per-client stats aren't possible, because clients talk to the router directly.
- Context: Markdown context docs composed into profiles, injected into requests or written into
CLAUDE.md/AGENTS.md. - Recipes: share a profile as readable JSON; others import it in one paste and LlamaForge downloads the model if missing. There's a community gallery.
Use it from other apps
Anything that speaks the OpenAI API (Open WebUI, SillyTavern, Continue, Cline, Aider, the OpenAI SDKs) works against
http://127.0.0.1:8080/v1. The router always runs with an API key, and Client Config on any model gives you the
base URL, key and model id ready to paste, plus filled-in settings for Open WebUI (Docker address included),
Continue's config.yaml, Cline and an aider command.
Embedding and reranker GGUFs are recognized from their header, so they're tagged embed or rerank and
started with --embeddings or --reranking. Their Client Config shows the /v1/embeddings or /v1/rerank call and
the Open WebUI Documents settings for RAG.
- Anthropic-compatible
POST /v1/messageson the panel, with streaming and tool use. - OpenAI speech-compatible
POST /v1/audio/speechon the panel (WAV or PCM), backed byllama-tts. - Connect an agent writes the config for Claude Code, Codex and pi.dev (any file it touches is backed up first).
- Load/unload endpoints let an agent swap models on demand.
- MCP server (stdio,
backend/mcp_server.py): Claude Code, Codex or any MCP client can see what's loaded,
pi_run). One-line setup under
Setup -> MCP server, e.g. claude mcp add --scope user llamaforge -- python /backend/mcp_server.py .
Screenshots
| Models: each GPU drawn to scale, with the open model booked in | Stats: memory and live speed per model | |---|---| | !Models | !Stats |
| Discover: new in llama.cpp, then trending GGUFs | Build / Update | |---|---| | !Discover | !Build / Update |
Install
The one-liners above need no git, admin or compiler. Re-run them any time to update.
The installer finds Python 3.10+ (on Windows it drops a private, SHA-256-pinned copy of
python.org's embeddable Python if you have none), downloads the latest release, adds a
Start menu / app-menu entry (~/Applications/LlamaForge.app on macOS, plus a llamaforge
command on Linux/macOS) and opens the dashboard. Updating keeps your settings, models and
engines. Uninstall from Apps & Features on Windows, or llamaforge uninstall; it asks
before touching your settings or models.
From source (git clone)
git clone https://github.com/dadwritestech/LlamaForge
cd LlamaForge
powershell -ExecutionPolicy Bypass -File bootstrap.ps1 # Windows
./bootstrap.sh # Linux / macOSThe bootstrap script checks for Python and Git (asking before installing anything),
writes config.json and opens the dashboard.
Daily use: open LlamaForge from the Start menu / app menu, or run llamaforge.
It starts the router and the dashboard and opens your browser.
- Dashboard: http://127.0.0.1:8090
- Chat: http://127.0.0.1:8091 (also the dashboard's Chat tab)
- API for your other apps: http://127.0.0.1:8080/v1
llamaforge stop (or stop.ps1 / stop.sh) shuts down the dashboard, the router and
the models it spawned. It stops only processes LlamaForge started; any other
llama-server you run is left alone.
Want it running whenever you log in (say, an always-on box behind Open WebUI)?
llamaforge service on on Linux (a systemd user unit) or macOS (a launch agent);
llamaforge service off removes it. It's off unless you turn it on.
Requirements: Windows 10/11, Linux, or macOS on Apple Silicon. Python 3.10+ (stdlib only; the Windows installer brings its own). NVIDIA (CUDA), AMD/Intel (Vulkan) or Apple (Metal) GPUs run models when present; CPU-only works. The VRAM preview, fit ratings and GPU stats read NVIDIA and Apple memory; AMD/Intel cards show their total and used memory as llama.cpp's own device list reports them (a Vulkan, ROCm or SYCL build; on Linux, AMD sysfs before one is installed), without utilization or temperature. Not yet tested on real AMD/Intel hardware. Building from source also needs Git, CMake, Ninja and a C++ compiler, which the Setup tab can install where a package manager allows.
How it works
LlamaForge contains no llama.cpp code. A pure-stdlib Python backend drives llama.cpp's own
router API, edits models.ini, and fetches official builds (or shells out to git / cmake).
The knob list is parsed live from llama-server --help, so it tracks whatever build you run.
LlamaForge doesn't pin context size, GPU layers or the multi-GPU split. llama.cpp's --fit
(on by default) sizes all three to your free VRAM at load, and moves MoE experts to CPU when
needed. Pinning any of them turns fit off, so they stay unset unless you set them yourself.
Security: the dashboard only listens on 127.0.0.1. The router is keyed even when
local, so a web page you visit can't drive it. LAN access is opt-in from Setup and always
needs a key. Details and how to report a vulnerability privately: SECURITY.md.
A GPU box with no screen: start it with ./run.sh over SSH and it prints the tunnel to
run from your own computer (ssh -N -L 8090:127.0.0.1:8090 you@box), then open
http://127.0.0.1:8090 there. The dashboard never has to listen on the network.
Docs
Everything is in the in-app Help tab and at dadwritestech.github.io/LlamaForge: configuration, keyboard shortcuts, themes & colorblind-safe mode, Auto-Forge, vLLM, troubleshooting, and what's new. ROADMAP.md has what's shipped and planned; it's an early preview, so priorities follow feedback.
Star history
100 stars in about three months, for a solo, heavily AI-assisted side project. Thank you.
Bug reports, ideas and code from @Vergillll, @Alihkhawaher, @BenjaminBilbro, @BrethofAI, @CaspervanWetten, @FLOATING-POINT, @Katchyuser, @jay23606, @mikinko and @tmill328 shaped what's here. If you've filed an issue, you're part of this.
Credits & license
LlamaForge is MIT-licensed (LICENSE). It builds and drives llama.cpp - MIT, (c) The ggml authors - see NOTICE and LICENSE.llama.cpp.txt. The hard part is theirs; please star and support the upstream project.
pi_run drives pi, Mario Zechner's open-source coding agent (MIT).
LlamaForge doesn't ship it: Setup → Install pi fetches the published npm package into
LlamaForge's own agents/ folder using your Node.js (or install it yourself with
npm install -g @earendil-works/pi-coding-agent).