Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw)

2 hours ago

Qwen 3.8- 27B on iPhone 18 Apple Neural Engine. Added draft model , dflash-2 at ~ 2.8 bpw ( mixed 2-4 vector quantization with small drop in acceptance rate) Base is @PrismML Bonsai-2, 1.585 bpw requantized to 1.5 bpw ( VQ 6 bits per 4 weights) Decoding speed is 15-60 tps.

00:00

You finished the article!

Want to go deeper or explore more stories?

Full Article on Hacker News More Tech News

Reading Mode

Adjust typography and theme to your preference.

Typography

Text Scale

View Original

Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw)

Reading Mode

Related Stories

Chrome and Edge are removing XSLT: what breaks and what does not

Microsoft and CMU Team Up to Train Early Career Software Engineers

Pattern Matching Beats Narrative Reasoning

AI Can Write the Code. Who Verifies It?

OpenCode Hacked: User credentials leaked

Related Stories