Open source. Our own quantised builds of GLM and Qwen. Built on ExLlamaV3, with speculative decoding, for 100 GB-class models in 128 GB.
| Model | Pack | Prefill | Speculative decode |
|---|---|---|---|
| Qwen3.8-Flash-Next | 95 GB | 1494 at 8K | 47.5 to 73.2 |
| GLM-5.3-Flash | 99.73 GB | 652 at 8K | 27.4 to 36.8 |
| MiMo-V2.6-Flash (1.5) | 105.8 GB | 760 at 8K | 28.9 to 37.0 |
Tokens per second on one Ryzen AI Max+ 395 with 128 GB. Prefill at the context shown in each cell, speculative decode across chat, prose and code prompts. Method, sources and full tables: doc/benchmarks.md.
Long context: up to 500K tokens
| Model | 128K | 256K | 500K |
|---|---|---|---|
| GLM-5.3-Flash | 608 / 30.1 | 577 / 30.2 | 541 / 29.2 |
| Qwen3.8-Flash-Next | 1399 / 46.8 | 1339 / 47.0 | 1240 / 43.4 |
Prefill / speculative decode, tokens per second, Kyojin 1.5.3 on one Ryzen AI Max+ 395 with 128 GB. One real document read as a growing prompt (500K = 499,712 tokens); prefill as reported by the server. Both models recalled two facts hidden at 40 % and 85 % of a ~500K-token prompt (needle test, 15/10).
500K needs -c 524288 at launch. Qwen also needs --cache-bits 8: past 262K it uses the 8-bit cache.
Made with our models
One prompt each, written by the model running on Kyojin and published as it came out. Open the gallery.
 Voxel pagoda GLM-5.3-Flash |  Voxel pagoda Qwen3.8-Flash-Next |  Voxel pagoda MiMo-V2.6-Flash |
Uncensored mode. Each model has an optional mode that makes it refuse far less. The engine applies it at load and the weights stay untouched: ready-made packs for GLM-5.3-Flash and MiMo-V2.6-Flash, and a preset inside the Qwen3.8-Flash-Next pack that is off by default. How it works: doc/refusal_hook.md.
These are first versions. Speed, context length and model support keep improving.
Contributors
Thank you to everyone who tests Kyojin on their own machine and writes precise reports.
- @felladrin: pull requests for the GLM start-up, the GPU target lookup, the bench tool tests, the troubleshooting page,
--sessionsin the Qwen server andtools/bench.sh --json; bug reports, an independent benchmark and issue triage. - @MangetsuC: the GLM image path without solver searches (first image of a new size in seconds) and the dense GEMM tune primer.
- @nobert: the Prometheus
/metricsendpoint, merged for GLM and MiMo. - @morrisfamily: benchmark report on a BOSGAME Beyond-Max.
- @ChrisC381: reported the GLM difference between paired jobs.
- @dturini12: reported streaming without
finish_reason; fixed and confirmed. - @mostlygeek: proposal for a container setup.
Quickstart
Install ROCm torch and build once (doc/install.md), then:
git clone https://github.com/Yamz-Labs/kyojin && cd kyojin
hf download yamz-labs/Qwen3.8-Flash-Next-EXL3-Yamz --local-dir ./qwen-pack
python tools/qwen/serve.py --model ./qwen-pack --port 8000 -c 131072The server speaks the OpenAI API on port 8000. GLM and MiMo commands: doc/models.md.
Read more
- Install and first launch: requirements, build, per-shell setup, first-launch tuning, system settings for full speed
- Models: Qwen, GLM and MiMo weights, serve commands, options
- Benchmarks: how the figures are measured, where each comes from, full tables, share yours with
tools/bench.sh - Troubleshooting, refusal hook, development, credits and licence
MIT licence. Built on ExLlamaV3 by turboderp, with the first AMD ports by vcruz305 and sdougbrown: full credits. News: @YamzLabs on X.