# AI Girlfriend

Language / 语言 / 言語：

🇨🇳 中文 · 🇬🇧 English · 🇯🇵 日本語

100% Local · Fully Private · Zero API Dependencies

All conversations, voice, images, and character animations are generated on your own machine. No cloud servers, no third-party APIs, no risk of data leakage. Your AI girlfriend belongs to you, and only you.

🗳️ Fourth girlfriend voting in progress - please vote on Issues · Config tutorial: BV16XTV6fEoH · qq: 580322386

⚠️ Default scripts are for NVIDIA GPUs. AMD GPU users: see the AMD_GPU/ folder.

An uncensored AI girlfriend harem project powered by OpenClaw + QQ Bot + Telegram Bot + llama.cpp + GPT-SoVITS + ComfyUI + Sakura Desktop Pet + Live2D - running entirely on your own machine.

Characters: Supports hot-swappable AI girlfriends with isolated memories per character.

Shiki Natsume (四季夏目)

From Starry Moonlit Café & the Butterfly of Death. Tall, aloof, cool exterior with a hidden warmth. A natural quietly-dominant type - she takes the lead, teases you gently, and guards you fiercely. Speaks little, but every word hits.

ATRI (亚托莉)

From ATRI -My Dear Moments-. Petite, innocent, endlessly curious - a bright-eyed girl who wears her heart on her sleeve. Runs toward the future with a smile, dragging you along. The polar opposite of Natsume: bubbly and expressive where Natsume is reserved, emotionally transparent where Natsume is guarded, playful where Natsume is composed. If Natsume is the cool winter night, ATRI is the warm summer sun.

Yono Sakura (夜乃桜)

From Dimension W Lovers!!. Former student council president and the academy's strongest anti-kaiju combatant. Silver-white hair with pink tips, pale blue eyes - cool-headed, restrained, and fiercely responsible. She's not good at smooth words or easy smiles; her care is direct and clumsy, like a command: rest, eat, don't push yourself. In desktop pet form, she's learning that she doesn't have to bear everything alone - that protecting someone's ordinary everyday life from this side of the screen is enough. A quiet guardian: silent but watchful, loyal but stubborn, the senpai who stays by your side without being asked.

✨ Why Choose This Project?

| | Cloud AI Girlfriend | This Project | |-|-|-| | 🛡️ Privacy | Chat logs, voice, and images all stored on vendor servers | Everything stays local - zero data leaves your machine | | 💰 Cost | Monthly subscriptions / per-token billing adds up | Free, one-time setup, runs forever (bring your own hardware) | | 🌐 Network | Needs internet; dead if servers go down | Works offline - flip off your WiFi and keep chatting | | 🎛️ Control | Prompts/templates controlled by vendor, can change anytime | You control all models, parameters, and character settings | | 🔞 Content | Heavy censorship, accounts get banned | No censorship - talk about whatever you want | | 🎨 Extensibility | Locked into vendor models and features | Mix and match - swap LLMs, image models, voice models freely |

📌 Prerequisites

⚠️ First step: Run quick_setup.ps1 to configure paths and language.

> This wizard will:

1. Let you choose the default Agent language (Chinese / Japanese / English) - copies the corresponding AGENTS_*.md to DEFAULT_AGENT.md

2. Auto-detect your installed tools (ComfyUI, GPT-SoVITS, llama.cpp, embedding models)

3. Prompt you for any paths it can't find

4. Generate config.yaml with all paths, ready for download-models.ps1

>

> powershell -ExecutionPolicy Bypass -File quick_setup.ps1

> Or use the new auto-download script (clones ComfyUI and GPT-SoVITS from git automatically):

>

> # Windows

> This script:

- Detects your OS and GPU environment

- Clones ComfyUI from https://github.com/comfyanonymous/ComfyUI

- Clones GPT-SoVITS from https://github.com/RVC-Boss/GPT-SoVITS

- Updates config.yaml with the new paths

> After setup completes, proceed with download-models.ps1 → setup-llama.ps1 → start.ps1.

🎬 Demo

Multi-Channel Chat

👆 QQ Bot: text chat + TTS voice + ComfyUI image generation + character memory

Live2D Desktop Pet

👆 Shiki Natsume Live2D: real-time character animation with emotion-driven motions, lip-sync, and speech bubbles. Controlled via local HTTP bridge.

⭐ ATRI - Second AI Girlfriend

Personality opposite of Natsume, hot-swappable with isolated memory.

!ATRI Live2D

👆 ATRI Live2D: silver hair, ruby-red eyes, barefoot in a white dress - innocent and expressive.

!ATRI ComfyUI

👆 ATRI ComfyUI: AI image generation - seaside sunset, flowing white dress, warm golden-hour lighting.

⭐ Yono Sakura - Third AI Girlfriend

Cool-headed guardian senpai, student council president and academy's strongest combatant - now your desktop companion.

!Sakura Desktop Pet

👆 Yono Sakura Desktop Pet: silver-pink gradient hair, pale blue eyes, school uniform - reactive portrait expressions, proactive care reminders, and real-time TTS voice via GPT-SoVITS.

🌐 Web Chat Frontend

!Web Chat Demo

👆 Web Chat: browser-based chat interface at http://127.0.0.1:19270 - an alternative to QQ/Telegram bots. Connects directly to local daemon proxy → llama.cpp server. 8 GB VRAM can run fully without stopping.

🎙️ TTS Voice

🔊 Listen (click to play, ATRI Japanese):

🎧 tts_atori.mp3 (46KB, plays in browser)

🎨 ComfyUI Image Workshop

!ComfyUI Workshop

👆 Artemis Studio - ComfyUI Workshop: Visual AI image generation console - freely choose character/outfit/scene/art style, one-click generation. Runs in parallel with llama (12GB+ VRAM).

| Feature | Description | |-|-| | 🎭 Dynamic Characters | Auto-loads from skills/harem/, displays persona + tags + greeting per character | | 🔄 Character Hot-Swap | One-click switch from sidebar dropdown, memories and chat context preserved per character | | 🃏 Card Import | Drag-drop or select SillyTavern PNG/JSON character cards, auto-parses metadata and persona | | 🤖 Model Selector | Choose local llama / DeepSeek / Grok from Settings dropdown, routes through daemon proxy | | 💬 Real LLM Chat | Streaming replies via daemon /api/chat → llama.cpp /v1/chat/completions, no fake fallbacks | | 📱 Responsive | Mobile sidebar collapse, adaptive bubble layout, works on desktop and tablet | | 💾 Local Storage | Multi-session chat history, settings, and character state persisted in browser localStorage | | 🎛️ Artemis Studio | Built-in TTS + ComfyUI placeholder panel (voice/image generation controlled via agent subprocesses) |

Hardware

| Component | Model | |-|-| | GPU | NVIDIA GeForce RTX 5070 Laptop (8 GB VRAM) | | CPU | Intel Core i9-14900HX (24 cores, 32 threads) | | RAM | 32 GB DDR5 | | OS | Windows 11 |

🔮 Cosmos World Foundation Model

🚀 Not a "future" plan anymore — it works today: Qwen-Drive-1.0-4B

📖 Full design: imagination.md | Bridge ref: skills/cosmos/BRIDGE_REFERENCE.md

NVIDIA Cosmos (community FP8 quant archived at skills/cosmos/) is a World Foundation Model that generates physics-consistent scene videos and understands spatial relationships.

Why Cosmos?

The four core capabilities (LLM + TTS + ComfyUI + Live2D) are currently disconnected - the LLM doesn't know what Live2D is doing, ComfyUI doesn't sense conversational emotion. Cosmos fills the physical common-sense layer:

Qwen3.6-35B (Language Mind) ←→ Cosmos 3 Nano / Qwen-Drive-1.0 (Physical Mind)
   Language + Emotion             Spatial + Scene Generation

Dual Compact Architecture

| Component | Model | Params | VRAM | |-----------|-------|--------|------| | 🧠 Language Mind | Qwen3.6-35B-A3B (MoE) | 35B total / 3B active | ~8 GB | | 🌍 Physical Mind | Cosmos 3 Nano FP8 | 15.75B | ~16 GB |

Hardware Roadmap

| Year | GPU | Cosmos Status | |------|-----|---------------| | 2026 | RTX 5070 (8-12GB) | ❌ Archived, detection ready | | 2027-01 | RTX 5070 Ti Super (24 GB) | ✅ Already done |

Current Status

• ✅ Repo archived at skills/cosmos/
• ✅ Bridge design imagination.md + cosmos_check.py ready
• ✅ Qwen ↔ Cosmos dual-mind architecture designed
• ✅ 2027-01: RTX 5070 Ti Super (24 GB) — already running

Features

• 🔄 Multi-Character Hot-Swap - One-click switch between AI girlfriends (Natsume ⇄ ATRI ⇄ Sakura); SOUL/IDENTITY/TTS weights/Live2D model all switch automatically, memories isolated per character
• 🃏 SillyTavern Character Card Import - Auto-detect and import PNG/JSON character cards; agent auto-switches persona on import
• 💬 Chat Log Import - Import SillyTavern JSONL conversation logs into memory/role_play//; agent restores context on role switch
• 🎤 TTS Voice Synthesis - Local GPT-SoVITS inference, Japanese voice (emotion-matched per dialogue), 3 character voice models (Natsume / ATRI / Sakura)
• 🎤 ASR Speech Recognition - Local Faster-Whisper small model (~1.5GB VRAM), coexists with llama; 99-language support
• 🎨 AI Image Generation - Local ComfyUI inference, SDXL/Illustrious models, 3 character prompt templates
• 🖥️ Sakura Desktop Pet - PySide6 desktop companion with proactive care, screen observation & local LLM awareness; supports 3 characters
• 🎭 Live2D Character Model - Real-time Live2D rendering with emotion-driven expressions & speech bubbles (Natsume / ATRI L2D; Sakura portrait mode)
• 🧠 Smart VRAM Tiering - Auto-detects GPU VRAM and picks the right strategy: ≥12GB keeps everything online (llama + skills); 8GB hot-swaps llama for GPU-heavy tasks; <8GB safe mode. Zero manual config
• 🎛️ Artemis Studio Console - Visual TTS + ComfyUI workshop, DIY voice & images anytime regardless of llama status - a true offline creative suite
• 💾 Roleplay Memory - Daily conversation summaries per character in memory/role_play/
• 🧠 Long-term Memory System - Powered by headroom (SmartCrusher + CCR) and mem0 (Qdrant vector database):

See skills/behavior-engine/README.md and AGENTS_roleplay_EN.md#behavior-engine.

💖 Relationship System (Behavior Engine)

A layered decision engine ported from the sister-project girl-agent, giving each character an independent relationship score, conflict state, relationship stage, and hormonal cycle that drive their behavior and reply style.

Core loop: each turn produces a moodDelta (interest/trust/attraction/annoyance/cringe) → accumulated into the score → triggers conflict escalation/cool-down → auto-checks relationship stage transitions → shapes the LLM's reply style.

| Field | Range | Meaning | Effect | |-------|-------|---------|--------| | score.interest | -100~100 | Interest | Reply warmth, initiative | | score.trust | -100~100 | Trust | Sharing, dependence | | score.attraction | -100~100 | Attraction | Heart-racing, body language | | score.annoyance | -100~100 | Annoyance | Cold tone, conflict chance | | score.cringe | -100~100 | Cringe tolerance | Acceptance of cheesy lines |

9 relationship stages: first meet → cold period → warming up → convinced → first date → early dating → stable dating → long-term → dumped

4-level conflict system: level 0 normal → level 1 slight sulk → level 2 in a huff → level 3 severe cold war → level 4 blocked/deleted

Hormonal cycle: a Gaussian cycle model simulates periodic swings in energy, irritability, affection, and libido, influencing reply length and tone.

State file: memory/role_play//relationship.json (independent per character, hot-loaded) Module location: skills/behavior-engine/

Models

All models hosted on HuggingFace: TAOTAO777/ai-girlfriend-natsume

See models.yaml for full details.

| Model | Purpose | Size | Context | |-|-|-|-| | LuffyTheFox Qwen3.6-35B-A3B Genesis Hermes V13 MTP APEX Compact (GGUF) | Chat LLM (primary MoE) | 16.11 GB | 120K | | Qwen3.8-27B-TurboFCFusion (Q4_K_S GGUF) | Chat LLM (dense, tooling) | ~15.8 GB | 100K | | Qwen3.6-27B-Fable-MTP (Q4_K_S GGUF) | Chat LLM (dense, legacy) | 13.5 GB | 150K | | Ternary-Bonsai-2-27B PTQ1_0 (ternary GGUF) | Chat LLM (fits fully in 8 GB VRAM, -ngl 99) | ~5.9 GB | ≤ 75K (KV cache must be q4_0) | | WAI-Nsfw-Illustrious-17 | ComfyUI generation (default, SDXL/Illustrious) | 6.46 GB | | | miaomiaoHarem_29BBETA10 | ComfyUI generation (backup, anima/qwen 29B + qwen VAE) | 5.44 GB | | | oneObsession_anima29BV1 | ComfyUI generation (anima/qwen 29B + qwen VAE) | 5.44 GB | | | qwen-image-2.1 Q6_K | ComfyUI generation (qwen-image GGUF, needs qwen3vl_8b TE + qwen VAE) | 5.47 GB | | | qwen3vl_8b_int8_convrot | ComfyUI text encoder (qwen-image-2.1) | 8.71 GB | | | qwen_image_vae | ComfyUI VAE (shared by all non-WAI models) | 242 MB | | | GPT-SoVITS voice weights | TTS voice synthesis | ~303 MB | | | Sakura SoVITS weights | TTS voice synthesis (Sakura voice) | ~313 MB | | | all-MiniLM-L6-v2 | English/cross-lingual embedding (mem0) | ~80 MB | | | BGE-small-zh-v1.5 | Chinese embedding (mem0) | ~91 MB | | | Cosmos 3 Nano FP8 🔮 | World Foundation Model (community FP8 quant, future HW) | ~16 GB | | | Shiki Natsume Live2D Model | Live2D character rendering | ~180 MB (archive) | |

📁 Embedding models path: embedding/all-MiniLM-L6-v2/ + embedding/bge-small-zh-v1.5/ (HF repo)

One-command Download

# Install huggingface-cli: pip install huggingface_hub
huggingface-cli login
Download all models
huggingface-cli download TAOTAO777/ai-girlfriend-natsume --local-dir ./models
Or download individual components:
huggingface-cli download TAOTAO777/ai-girlfriend-natsume llm/ --local-dir ./models
huggingface-cli download TAOTAO777/ai-girlfriend-natsume comfyui/ --local-dir ./comfyui
huggingface-cli download TAOTAO777/ai-girlfriend-natsume gpt-sovits-weights/ --local-dir ./gpt-sovits-weights
huggingface-cli download TAOTAO777/ai-girlfriend-natsume live2d-model/ --local-dir ./live2d-model
Ternary-Bonsai-2-27B PTQ1_0 (mirror of our repo's llm/ folder, fits 8 GB VRAM):
huggingface-cli download TAOTAO777/ai-girlfriend-natsume llm/Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir ./models

🇨🇳 Users in China: use hf-mirror.com - no VPN needed:

set HF_ENDPOINT=https://hf-mirror.com then run hf download as usual.

Local Configuration

1. Run quick_setup.ps1 - interactive wizard that generates config.yaml with your local paths
2. (Alternative) Copy config.example.yaml → config.yaml and edit manually
3. Place downloaded model files according to models.yaml, then update config.yaml paths

⚠️ Disclaimer: All models are community open-source. This project only provides mirror distribution, non-profit. Copyright belongs to original authors.

Local LLM Performance

Running Qwen3.6-35B-A3B Genesis Hermes V13 MTP APEX Compact (MoE, 16.11 GiB, 34.66B params, 8/256 experts) via llama.cpp with speculative MTP (Multi-Token Prediction) decoding.

Launch Command (one source of truth)

🚀 You don't hand-type llama-server args anymore. Every launch entry point

(start.ps1, shiki_daemon.py, restart_llama_degraded.ps1) reads the launch

parameters from config.yaml via skills/shared/llama_config.py, which

auto-matches the active model filename against model_profiles and builds the

full llama-server command. Models are auto-detected and parameters separate

by profile — nothing is hardcoded.

> See "Switching models" below for the one-liner.

⚙️ **The baked-in launch commands are semi-hardcoded — treat them as a

starting point, not gospel.** The profile parameters in config.yaml /

llama_config.py were tuned for the reference machine. Before trusting them

on your own hardware, read LLAMA_TUNING.md (handwritten

field notes: when to use -ngl 99 vs partial -ngl N vs --cpu-moe, MTP

draft tuning, KV cache sizing, batch/ubatch, threads, context window) and

decide the final llama-server command **based on that guide plus your

machine's GPU/RAM/CPU configuration**. In short: pick the offload tier that

matches your VRAM vs model size (partial -ngl N is fine for dense models

that don't fully fit — it's a static split, not dynamic swapping), tune

--spec-draft-n-max × --spec-draft-p-min until acceptance looks good, and

size context/KV cache to your RAM. See the Qwen3.8-27B (Dense, Tooling Model)

section below for the reference flags and live metrics.

Why the root chat_template.jinja exists

The project root ships a fixed Jinja chat template (froggeric/Qwen-Fixed-Chat-Templates, pinned at v22.3 in chat_template.jinja) that overrides the template baked into the GGUFs. The official Qwen 3.5/3.6/3.8 templates contain engine restrictions, Python-specific Jinja logic, and regressions that break local inference and agent workflows — the most visible one is overthinking: the official Qwen 3.8 template hardcodes xhigh reasoning depth by default, which can exhaust the token budget on thinking before the model ever answers.

One file covers all Qwen 3.5 / 3.6 / 3.8 sizes, so it works unchanged for both local models. Launch plumbing: config.yaml → llama_chat_template: chat_template.jinja (relative to the project root), and llama_config.py resolves it to --chat-template-file — nothing hardcoded.

llama-server.exe ... --jinja --reasoning-preserve \
  --chat-template-file "D:\AI_Girlfriend\chat_template.jinja"

Switching models (restart_llama_degraded.ps1 -SwitchTo)

Switch between the two active models with a one-liner — the script kills the current llama-server, rewrites config.yaml (llama_model / llama_model_name / llama_model_id), re-resolves the profile, restarts, and waits for /health:

cd D:\AI_Girlfriend
27B dense (Qwen3.8-27B) — primary tooling model
.\skills\shared\restart_llama_degraded.ps1 -SwitchTo qwen3.8-27b
35B MoE (Hermes Genesis V13) — primary roleplay model
.\skills\shared\restart_llama_degraded.ps1 -SwitchTo qwen3.6-35b

-SwitchTo accepts the key in config.yaml → llama_model_map (e.g. qwen3.8-27b / qwen3.6-35b), or a substring (e.g. -SwitchTo 27b). Use -ForceBatch 1024 to lower batch size if you hit VRAM limits.

See LLAMA_TUNING.md for more llama tuning details.

Served on http://127.0.0.1:9931 (llama.cpp moved from 8080 — 8080 is now the Strata engine port, see the flash-next section below). Note: every argument pair in the PowerShell

array must be comma-separated — a missing comma silently glues two tokens together.

> 💡 rea not specified — defaults to medium reasoning depth via the chat

template (no injected thinking tokens, preserving KV-cache parity). This is the

optimal setting for tooling/agent tasks where you want fast, direct responses.

Qwen3.8-Flash-Next — Strata engine (NOT llama.cpp) 🔥

Qwen3.8-Flash-Next (125B MoE) is the only model served by the Strata engine instead of llama.cpp. Port layout since 2026-10: llama.cpp = 9931, Strata = 8080. Never load flash-next into llama.cpp (RAM/VRAM clash).

Setup — clone the original repo (anywhere outside this project; do not vendor it into skills/):

# 1) Clone the original repo, e.g. next to the project
git clone https://github.com/Niko1221/Strata
cd Strata
2) Run the start script once: it asks model/quant/context questions,
   downloads everything, and generates a per-quant launcher bat (e.g. run-iq3_s.bat)
START-HERE.bat
3) From now on, start the LLM with the generated bat:
.\run-iq3_s.bat

• Strata auto-downloads the chosen quant to Strata-data\models\IQ3_S\ (relative to the clone, when IQ3_S was selected).
• Uncensored build (required here): download from

• Health check: http://127.0.0.1:8080/health → "service": "strata". OpenAI-compatible endpoint: http://127.0.0.1:8080/v1.
• OpenClaw: use the strata/qwen3.8-flash-next provider (see openclaw.example.json). Full manual: skills/strata/SKILL.md.
• Measured on this host (2026-10-31 & 11-02 sessions, RTX 5090 Laptop 24 GB, IQ3_S, --max-context 131072 --kv int8 --spec 4 --mtp):

Ternary-Bonsai-2-27B PTQ1_0 — fits fully in 8 GB VRAM 🔥

Ternary-Bonsai-2-27B PTQ1_0 — hosted in this project's HF repo: TAOTAO777/ai-girlfriend-natsume → llm/Ternary-Bonsai-2-27B-PTQ1_0.gguf (same llm/ folder as the other two LLM models). Original source: base model prism-ml/Ternary-Bonsai-2-27B-gguf; PTQ1_0 ternary quant build by BoldingBuilds.

5.9 GB on disk (~5.5 GiB). Verified running on the reference 8 GB VRAM laptop with -ngl 99: the whole model fits on the card — no RAM layer split needed. Measured: decode ~35 t/s+, prefill ~300 t/s. Tuning notes: LLAMA_TUNING.md.

🔴 Two NON-NEGOTIABLE constraints for 8 GB VRAM:

> 1. KV cache MUST be Q4: -ctk q4_0 -ctv q4_0. Anything higher (f16 / f32 KV) will blow the VRAM budget immediately.

2. Context window MUST be -c ≤ 75000. With Q4 KV, weights (~5.5 GiB) + KV cache + compute buffers stay inside 8 GB only up to ~75K tokens. Anything larger does not fit on an 8 GB card.

Silicon Rider Bench (Agent Benchmark)

Silicon Rider Bench is an agent benchmark that simulates a food-delivery rider working a virtual city: navigate, accept orders, pick up food, deliver on time, and manage battery — scoring total profit over a simulated 24-hour day. Same seed (622539) used across all runs for apples-to-apples comparison.

Models under test (all --seed 622539):

• deepseek-v4-flash (0731) — remote, unlimited-context baseline. Cloud-class agent ability (~Claude 4.6–4.8 tier in this benchmark).
• Hermes3.6-35B-A3B-Uncensored-Genesis-V9-MTP-APEX-Compact.gguf (current) — RTX 5070 Laptop, 8 GB VRAM, 32 GB DDR5 RAM

Results (Seed 622539, Level 1, 24 game-hours)

| Metric | deepseek-v4-flash
(unlimited ctx) | Hermes 35B MoE
(25 ctx) | Hermes 35B MoE
(100 ctx) ✅ | |-|-|-|-| | Profit ¥ | 619.6 | 411.3 | 524.6 | | Orders completed | 33 | 30 | 28 | | On-time rate | 81.8% | 56.7% | 75.0% | | Route efficiency | 1.34 | 1.77 | 1.68 | | API violation rate | 1.3% | 2.3% | 2.2% | | Profit / order ¥ | 18.77 | 13.71 | 18.74 | | Overtime penalty ¥ | 2.75 | 107.9 | 44.7 | | Total tokens | 24.39M | 1.35M | 4.08M | | Token efficiency ¥/M | 25.4 | 304.6 | 128.6 |

Key Takeaways

• Context length is the #1 lever: raising CONTEXT_HISTORY_LIMIT 25 → 100 lifted on-time rate 56.7% → 75% and slashed overtime penalty ¥107.9 → ¥44.7, pushing profit ¥411 → ¥525 (the model finally retains order deadlines + routes across turns).
• Local 35B MoE ≈ 85% of cloud flash: at 100 ctx the local quantized 35B hits ¥524.6 = 84.6% of dsv4-flash's ¥619.6, with on-time rate (75% vs 81.8%) and per-order profit (¥18.74 vs ¥18.77) essentially tied.
• 6× cheaper: flash burned 24.39M tokens (unlimited ctx); local 100-ctx used only 4.08M for 5/6 of the profit → 5× better token efficiency, at zero API cost.
• Remaining gap: route efficiency (1.68 vs 1.34) — the 35B-A3B's 3B active params still underperform flash on multi-leg optimal route planning.

🧪 Full logs & reports in docs/silicon-rider-bench-622539/ (COMPARISON-622539.md + per-run summaries).

Long Context Stability

Qwen3.6 MoE uses SSM (Gated Delta Net) hybrid attention with --kv-unified.

⚠️ Known Limitation: Cross-turn prompt cache reuse is not supported (SSM architecture limitation). Each request triggers full context re-processing. Longer conversations = higher first-token latency (~55s for 59k tokens).

Mitigations:

• Periodic /reset (Natsume writes roleplay summaries to memory/role_play/ before resetting)
• Restore context from summaries on startup, keeping actual token count in 5K-20K range
• config-patch.json sets OpenClaw contextWindow to 262144 to match model capacity

Qwen3.8-27B (Dense, Tooling Model)

Primary tooling/assistant dense model. Runs via llama.cpp with built-in MTP speculative decoding (no separate draft GGUF needed). Auto-detected via config.yaml → model_profiles (qwen3.8-27b-mtp).

Launch Command (8 GB VRAM, Q4_K_S)

Switch to it, or launch manually:

# Preferred: auto-switch + auto-params (see "Switching models" above)
.\skills\shared\restart_llama_degraded.ps1 -SwitchTo qwen3.8-27b

Reference hardware: i9-14900HX + RTX 5070 Laptop (8 GB) + 64 GB RAM. The

model weights are split via a partial offload -ngl 14 (first 14 layers on

GPU, rest in RAM with --no-mmap); KV cache uses --cache-ram 2000; the

MTP draft context is offloaded fully to the GPU (--spec-draft-ngl 99) so

speculative decoding stays fast on an 8 GB card. Q4_K_S quantization keeps

the model at ~15.8 GB — the sweet spot for dense 27B on consumer hardware.

rea not specified — defaults to medium reasoning depth via the chat

template. Flag-by-flag notes, key parameters, and live-log metrics follow below.

💡 27B dense on 8 GB VRAM — key parameters explained:

> - -ngl 14 — 14 layers offloaded to GPU (static split; rest in system RAM). For an 8 GB card with a ~15.8 GB Q4_K_S model, this is the sweet spot that fits without OOM while still getting meaningful GPU acceleration. Adjust up/down based on your actual VRAM.

- -ctk q4_0 -ctv q4_0 — KV cache quantized to 4-bit, halving VRAM usage for the context window. Essential for large context with limited VRAM.

- --cache-ram 2000 — 2 GB RAM budget for the KV cache on the CPU side.

- -c 100000 — 100K token context window (the model's effective limit at this quantization).

- --spec-draft-n-max 3 — MTP speculative decoding drafts up to 3 tokens ahead; Qwen3.8 ships its own MTP head.

- --spec-draft-p-min 0.88 — Only accept draft tokens with ≥88% confidence, keeping the acceptance rate high.

- --spec-draft-ngl 99 — Offload the entire draft context to GPU for faster speculative decoding.

- Quantization: Q4_K_S — ~15.8 GB model size, excellent quality/VRAM balance for dense 27B on consumer hardware. This is a dense (non-MoE) model, so all 27B parameters are active at inference (vs MoE which activates a subset).

- rea not specified — defaults to medium reasoning depth via the chat template (no injected thinking tokens, preserving KV-cache parity).

Flag notes (dense 27B profile, 8 GB VRAM optimum, Q4_K_S):

| Flag | Value | Why | |-|-|-| | -m | Q4_K_S model path | Q4_K_S quantization — ~15.8 GB, excellent quality/VRAM balance for dense 27B on consumer hardware | | -c | 100000 | 100K context window (n_ctx_slot = 100096) | | -ngl | 14 | Partial GPU offload — first 14 layers on GPU, rest in RAM; measured optimum for Q4_K_S on 8 GB VRAM (no dynamic swapping, safe to raise until KV/MTP headroom disappears) | | -ctk / -ctv | q4_0 | KV cache quantized to q4_0 to halve VRAM | | --cache-ram | 2000 | 2 GB RAM budget for KV cache on CPU side | | --batch-size / --ubatch-size | 2048 / 1024 | Prefill batch sized for 8 GB VRAM headroom (2:1 rule) | | --spec-type | draft-mtp | Enable built-in MTP speculative decoding | | --spec-draft-n-max | 3 | Draft up to 3 tokens per step (Qwen3.8's built-in MTP head) | | --spec-draft-p-min | 0.88 | Only accept drafts ≥0.88 token probability for high acceptance rate | | --spec-draft-ngl | 99 | Offload the whole MTP draft context to GPU for fast speculative decoding | | --no-mmap | — | Let llama.cpp manage RAM-side memory (clean CPU/GPU split) | | --reasoning-preserve | — | Preserve thinking blocks for KV reuse | | rea | not specified | Defaults to medium reasoning depth via chat template — optimal for tooling/agent tasks (fast, direct responses) |

Key Metrics (27B Dense, Q4_K_S — from live llama-server log)

| Metric | Value | Notes | |-|-|-| | Model Load Time | ~1s | --no-mmap (~15.8 GB, Q4_K_S) | | Prefill Speed | ~163 ~ 174 t/s | First prompt 19.3k tokens @ 163.6 t/s; scales down with prompt length | | Token Generation | ~4 ~ 5 tok/s | Steady decode (MTP active, -ngl 14) | | MTP draft acceptance | ~93 ~ 97% | e.g. 0.93599 (541/578), 0.96859 (185/191); mean accepted run length 2.5 ~ 5.2 | | Context Limit | 100K (n_ctx_slot = 100096) | --kv-unified + --cache-ram 2000 | | MTP retention (--spec-draft-p-min) | 0.88 | Draft tokens below 0.88 confidence are rejected | | GPU layers (-ngl) | 14 | Static split; log line n_gpu_layers already set by user to 14, abort is a harmless notice (auto-fit skipped), not an error |

📈 MTP explained: with --spec-draft-n-max 5 + --spec-draft-p-min 0.84,

llama.cpp asks the MTP head to propose up to 5 next tokens, then keeps each only if

its probability is ≥0.84. In practice ~90–100% of drafted tokens are accepted

(mean accepted run length ≈ 3.2–5.3), so effective throughput is roughly 3–5× a

single speculative token per forward pass while the 8 GB card stays within its

VRAM cap.

💡 MoE vs Dense: The 35B MoE activates only ~3B parameters per token (8/256 experts) and fits GPU well (48 tok/s). The 27B dense activates all 27B, exceeding 8 GB VRAM, so it splits to CPU/RAM via -ngl 14 and decodes ~4–5 tok/s with MTP. Use the 27B dense when you want full 27B activation for tooling / agent tasks; use the 35B MoE for fast roleplay. The Q4_K_S quant (~15.8 GB) is the sweet spot for dense 27B on consumer hardware — excellent quality while fitting on 8 GB VRAM with partial offload.

VRAM Tiering Strategy

The system auto-detects GPU VRAM and selects the optimal run mode, no manual config:

┌────────────────────────────────────┬────────────┬────────────┬────────────┬────────────┐
│ VRAM Tier                          │ TTS        │ ComfyUI    │ llama      │ ASR        │
├────────────────────────────────────┼────────────┼────────────┼────────────┼────────────┤
│ Tier 0: <8GB                       │ Stop llama │ Stop llama │ Killed     │ Killed     │
│ Tier 1: 8-12GB (current)           │ Stop llama │ Stop llama │ Killed     │ No kill    │
│ Tier 2: ≥12GB                      │ No kill    │ No kill    │ Always on  │ No kill    │
└────────────────────────────────────┴────────────┴────────────┴────────────┴────────────┘

Current setup (8GB VRAM):

8 GB Total VRAM
├── llama-server resident: ~5.8 GB (model 4.6G + KV cache 1.2G)
├── Free: ~2.2 GB
│
├── TTS inference: stop llama → ~8 GB free → resume llama (~70s)
├── ComfyUI generation: stop llama → ~8 GB free → resume llama (~120s)
├── Artemis Studio (TTS/ComfyUI workshop): standalone - works regardless of llama
└── ASR / Live2D / Embedding: always online, unaffected by VRAM tiering

Directory Structure

<PROJECT_DIR>/                            # OpenClaw workspace root
├── start.ps1                             # 🚀 One-click launch: llama + headroom + Live2D + Gateway
├── artemis_headroom_proxy.py             # Headroom proxy (19251): mem0 injection + SmartCrusher + routing
├── shiki_daemon.py                       # Daemon (19260/19270): WebChat backend + auto-inject provider
├── quick_setup.ps1                       # 🛠 Interactive path config wizard
├── config.yaml                           # Generated config
├── download-models.ps1                   # One-click model download (Windows)
├── download-models.sh                    # One-click model download (Linux/macOS)
├── setup-llama.ps1                       # Auto-detect HW + configure llama.cpp (Win)
├── setup-llama.sh                        # Auto-detect HW + configure llama.cpp (Linux/macOS)
├── setup-openclaw.ps1                    # One-click OpenClaw install + deploy (Win)
├── setup-openclaw.sh                     # One-click OpenClaw install + deploy (Linux/macOS)
├── setup-all.ps1                         # 🚀 All-in-One mega script (Windows)
├── setup-all.sh                          # 🚀 All-in-One mega script (Linux/macOS)
├── config-qqbot.json                     # QQ Bot config patch
├── config-telegram.json                  # Telegram Bot config patch
├── config-patch.json                     # OpenClaw LLM config patch
├── AGENTS.md                             # Agent behavior rules
├── SOUL.md                               # Character personality
├── IDENTITY.md                           # Character identity
├── USER.md                               # User info
├── HEARTBEAT.md                          # Heartbeat config
├── TOOLS.md                              # Tool quick reference
├── models.yaml                           # Model catalog + download links
├── LLAMA_TUNING.md                       # ⚙️ Handwritten llama.cpp tuning field notes (read before trusting baked-in launch args)
├── imagination.md                        # 🔮 Cosmos WFM integration vision (future)
├── README.md                             # This file
├── .gitignore
├── live2d/                               # Live2D character model (Cubism 4 Core)
│   ├── index.html                        # Default (Shiki Natsume)
│   ├── index_atri.html                   # ATRI variant
│   ├── index_upper.html                  # Natsume upper-body variant
│   ├── index_atri_upper.html             # ATRI upper-body variant
│   ├── live2dcubismcore.min.js           # Cubism Core 4 (207 KB)
│   ├── plid-v5-bundle.js                 # pixi-live2d-display v0.5.0 bundle
│   ├── live2d-bridge.mjs                 # HTTP (19200) + WebSocket (19201) bridge
│   ├── switch_model.ps1                  # Model switcher (natsume / atri)
│   ├── pixi.min.js, pixi-shim.js         # PIXI.js v7 rendering
│   ├── model/shiki_natsume/              # Natsume model (14 textures, 42 motions, 41 sounds)
│   └── model/atri/                       # ATRI model (2 textures, 620 voice mp3, 8 motions)
├── ren_pro_jp/                           # Ren'Py dialog engine (planned)
├── memory/                               # [.gitignore] Runtime memory
│   └── role_play/                        # Roleplay conversation logs
├── media/                                # [.gitignore] Generated media
│   ├── audio/                            # TTS voice output
│   ├── images/                           # ComfyUI image output
│   └── *.gif                             # README demo GIFs
├── docs/
│   ├── telegram-setup.md                 # Telegram Bot setup guide
│   └── qqbot-setup.md                    # QQ Bot setup guide
└── skills/
    ├── live2d/                           # Live2D control skill
    │   ├── SKILL.md                      # Motion/expression reference + API guide
    │   ├── scripts/start-live2d.ps1      # Live2D launcher
    │   └── media/                        # Shared media output
    ├── tts/
    │   ├── SKILL.md                      # TTS invocation guide
    │   ├── run_tts.ps1                   # TTS launcher script
    │   ├── tts_call.py                   # GPT-SoVITS inference
    │   └── ref_wavs/                     # Reference audio clips
    ├── comfyui/
    │   ├── SKILL.md                      # ComfyUI invocation guide
    │   ├── run_comfyui.ps1               # ComfyUI launcher script
    │   ├── comfyui_call.py               # ComfyUI inference
    │   ├── prompt_template.md            # Character prompt template
    │   └── custom_prompt.txt             # Custom extra prompt
    ├── asr/                              # Speech recognition skill
    │   ├── run_asr.ps1                   # Faster-Whisper launcher (~1.5GB VRAM)
    │   └── asr_call.py                   # Whisper small model inference
    ├── shared/                           # Shared infrastructure
    │   ├── embedding_server.py           # OpenAI-compatible embedding API (9999, dual model)
    │   ├── mem0_bridge.py                # mem0 Qdrant → OpenClaw memory bridge
    │   ├── start_embedding_server.ps1    # Auto-start embedding server
    │   ├── vram.py                       # VRAM tier auto-detection
    │   ├── VRAM_LEVELS.md                # VRAM tier documentation
    │   ├── llama_lifecycle.py            # Llama start/stop management
    │   └── llama_utils.py                # Llama utility functions
    ├── sakura/                           # Sakura Desktop Pet (PySide6 GUI)
    │   ├── SKILL.md                      # Sakura skill documentation
    │   ├── main.py                       # Application entry point
    │   ├── install.bat                   # Windows dependency installer
    │   ├── start.bat                     # Windows launcher
    │   └── app/                          # Source code
    ├── cosmos/                           # 🔮 NVIDIA Cosmos WFM (future hardware)
    │   ├── BRIDGE_REFERENCE.md           # Cosmos ↔ AI Girlfriend bridge design
    │   ├── cosmos_check.py               # Hardware VRAM detection script
    │   ├── cookbooks/                    # Official tutorial examples
    │   └── README.md                     # Upstream documentation
    ├── llama-management.md               # VRAM management architecture doc
    ├── llama-watchdog.ps1                # Llama health check
    ├── cleanup_orphans.ps1               # Orphan process cleanup
    ├── behavior-engine/                  # 💖 Relationship system (behavior engine)
    │   ├── engine.py                     # State load/save/update/reset
    │   ├── hormones.py                   # Hormonal cycle (Gaussian model)
    │   ├── conflict.py                   # 4-level conflict system
    │   ├── stages.py                     # 9 relationship stages
    │   ├── behavior_tick.py              # Behavior decision layer
    │   ├── online_tick.py                # Online/sleep simulation
    │   ├── daily_life.py                 # Daily schedule
    │   ├── README.md                     # Design doc
    │   └── SKILL.md                      # Usage guide
    └── character_importer/               # SillyTavern character card auto-import

🤖 Claude Code + AgentRQ-Style Task Board (NEW)

Artemis now supports Claude Code as a parallel agent runtime alongside OpenClaw. Claude Code connects via MCP to access all Artemis capabilities - with a built-in AgentRQ-compatible task queue for human-agent collaboration.

How it works

┌─────────────────────────────────────────────────────────┐
│  Task Board (http://127.0.0.1:19280)                    │
│  Create task → assignee: agent → notstarted             │
└───────────────────────┬─────────────────────────────────┘
                        │ SQLite (.claude/task_queue.db)
                        ▼
┌─────────────────────────────────────────────────────────┐
│  Claude Code (terminal)                                 │
│  CLAUDE.md → getNextTask() → ongoing → execute          │
│  Artemis tools → TTS / ComfyUI / Live2D / memory        │
│  reply() → updateTaskStatus(completed)                  │
└─────────────────────────────────────────────────────────┘

AgentRQ-Style Task Loop

Claude Code automatically runs a task loop on startup:

1. getWorkspace() - check workspace status
2. getNextTask() - dequeue next pending task
3. updateTaskStatus(taskId, "ongoing") - claim it
4. Execute using Artemis tools (TTS, ComfyUI, etc.)
5. reply(taskId, "Done!") - report result
6. updateTaskStatus(taskId, "completed") - mark done
7. Loop back to getNextTask()

Launch

# Prerequisites: npm install -g @anthropic-ai/claude-code
Start Shiki Daemon first (.\shiki.cmd), then:
Full AgentRQ workflow (Task Board + Claude Code)
.\claude-code.ps1
Task Board only (browser UI, no Claude)
.\claude-code.ps1 -BoardOnly
Stop the task board
.\claude-code.ps1 -KillBoard

Then open http://127.0.0.1:19280 - create tasks, watch Claude Code pick them up.

MCP Tools (15 total)

| Category | Tool | Description | |-|-|-| | 🎤 TTS | tts_generate | Voice synthesis (character/lang/mood) | | 🎨 Image | comfyui_generate | AI image generation (prompt, checkpoint) | | 🎤 ASR | asr_transcribe | Speech-to-text (wav/mp3/ogg/flac, Whisper small, ~1.5GB VRAM) | | 🎭 Live2D | live2d_emotion | Motion + speech bubble | | 🔄 Char | switch_character / list_characters | Character management | | 🧠 Memory | memory_search / memory_add | Vector memory (mem0 Qdrant) | | 📊 Status | get_status | Service health check | | 📋 Task | getWorkspace / getNextTask / createTask | Task queue ops | | 📋 Task | updateTaskStatus / reply / getTaskMessages | Task lifecycle |

Artemis Task Board vs AgentRQ

| Feature | Artemis Task Board | AgentRQ (self-hosted) | |-|-|-| | Runtime | 1 Python script + SQLite | Go+Vue+Docker+Google OAuth | | MCP tools | 6 task + 9 Artemis (15 total) | Same set (8 tools) | | Setup | Zero config | Docker + .env + OAuth |

Files

| File | Purpose | |-|-| | .mcp.json | MCP server config for Claude Code | | .claude/CLAUDE.md | Persona + task loop instructions | | .claude/artemis_mcp_server.py | MCP server (15 tools, JSON-RPC stdio) | | .claude/task_board_api.py | Task board HTTP API (port 19280) | | .claude/task_board.html | Task board browser UI | | .claude/task_queue.db | SQLite task database (auto-created) | | .claude/settings.local.json | Pre-approved MCP tools | | claude-code.ps1 / .sh | Launcher scripts |

🐋 DeepSeek Harness (dsh) — Third Agent Runtime (NEW)

Alongside OpenClaw and Claude Code, Artemis now supports DeepSeek Harness (dsh, community nickname "大肥鱼") as a third agent runtime - an everything-is-a-plugin harness by DeepSeek AI. Roleplay is the hot use case in its community, so the core clone and six RP plugins live under benchmark_projects/ (git-ignored).

# One-shot setup (Node 24+ required, global npm install + verification + plugin inventory)
powershell -ExecutionPolicy Bypass -File setup-dsh.ps1
Launch web UI (or run without installing: npx @deepseek-ai/dsh web)
dsh web

Point dsh at the same local inference backends (llama.cpp :9931 / Strata :8080, OpenAI-compatible) - no extra model copy. Ecosystem survey: benchmark_projects/2026-11-07-dsh-roleplay-report.md.

Skills Overview

| Skill | Type | Llama Kill? | Mechanism | |-|-|-|-| | Embedding | Background process | ❌ No | all-MiniLM-L6-v2 + BGE-small-zh-v1.5 dual models (CPU, port 9999) - OpenClaw memory search + mem0 bridge | | Live2D | HTTP exec | ❌ No | Direct HTTP calls to localhost:19200 bridge | | Web Chat | Browser | ❌ No | Local daemon proxy to llama :9931, port 19270, real-time chat | | Claude Code | Terminal (MCP) | ❌ No | Parallel agent runtime via .claude/artemis_mcp_server.py, uses llama :9931 directly | | DeepSeek Harness (dsh) | Terminal / Web (plugins) | ❌ No | Third agent runtime (npm @deepseek-ai/dsh, see setup-dsh.ps1); reuses llama :9931 / Strata :8080; RP plugins in benchmark_projects/ | | TTS | sessions_spawn | 🔶 VRAM-tiered | ≥12GB: no kill; 8GB: stop llama → GPT-SoVITS → restart llama | | ComfyUI | sessions_spawn | 🔶 VRAM-tiered | ≥12GB: no kill; 8GB: stop llama → image gen → restart llama | | ASR | sessions_spawn | ❌ No | Faster-Whisper small (~1.5GB VRAM, coexists with llama) | | Strata | External engine | 🔶 RAM/VRAM | Qwen3.8-Flash-Next only, serves :8080 (llama.cpp on :9931); don't run both big models at once | | Sakura | Shared llama-client | ❌ No | Detects llama down → waits → auto-resumes | | Artemis Studio | Desktop console | ❌ No | TTS/ComfyUI visual workshop, standalone - works regardless of llama status |

Environment Dependencies

| Component | Version / Source | Purpose | |-|-|-| | OpenClaw | latest | AI Agent Gateway | | QQ Bot | OpenClaw qqbot channel | QQ message relay | | Telegram Bot | OpenClaw telegram channel | Telegram message relay | | llama.cpp | b9222 | Local LLM inference server | | GPT-SoVITS v2 | v2pro-20250604 | TTS voice synthesis | | ComfyUI | aki-v3 | Image generation engine | | Sakura Desktop Pet | v0.9.6-dev | Desktop companion GUI | | pixi-live2d-display | v0.5.0 (bundled) | Live2D WebGL renderer | | Live2

... (README truncated for length)