Lynkr

An LLM Gateway which optimises your token usage.

84% fewer tokens on JSON tool results. 53% fewer tokens on tool-heavy requests. Sub-300ms semantic cache hits. Zero code changes.

Numbers from the bundled benchmark against LiteLLM on identical free local backends — run it yourself: node benchmark-tier-routing.js. It doubles as a 19-scenario routing regression harness (currently 12/12 correctness checks), and MODE=routing runs a routing-only head-to-head that judges both proxies on the same acceptable-tier sets — including LiteLLM's Auto Router v2. How it works →

Agentic benchmark (Terminal-Bench core, 80 tasks, Oct 2026): Lynkr 42/80 for $1.54 vs openrouter/auto 44/80 for $4.72 on the same VM and evening — tied on accuracy within single-run noise (Lynkr's three runs: 39, 39, 42) at ~30% of the cost; vLLM Semantic Router on the same models/hosts: 25/80 (34 agent timeouts from unbounded upstream stalls and per-turn CPU classification). Full comparison, fairness notes and what it changed in Lynkr →

Third-party benchmark: on RouterArena (ICLR 2026, 8,400 queries) Lynkr's routing scores 67.65 arena / 68.41% accuracy at $0.29 per 1K queries with 92.38 robustness — above GPT-5's built-in router and NotDiamond at a fraction of their cost. Methodology & caveats →

🚀 New: Wrap Mode for AI Coding Tools

Use Lynkr's routing with your AI coding assistant — maximize your subscription value:

npm install -g lynkr
Claude Code Pro/Max
lynkr wrap claude

Wrapping gives you:

• ✅ Tier routing (send simple tasks to free Ollama, complex to your subscription/API)
• ✅ Sticky sessions: one routing decision per conversation via content fingerprinting, with automatic escalation when the task outgrows the model
• ✅ TOON/RTK compression (84% token reduction on large JSON tool outputs)
• ✅ Semantic caching (sub-300ms cache hits, 0 tokens billed)
• ✅ Your subscription is used only where it counts — simple requests route to your own local/API models, so subscription capacity goes to the work that needs it
• ✅ Works with OAuth (Claude, Copilot, Cursor) or API keys (Aider, Codex)

Quick Start (2 Minutes)

1. Install Lynkr

npm install -g lynkr

2. Configure Lynkr

The fastest path is the interactive wizard:

lynkr init

It asks four questions — usage mode (Claude Pro/Max via wrap, or direct API keys), tier picks for SIMPLE/MEDIUM/COMPLEX/REASONING across the 14 supported providers, credentials for what you chose, and a few routing-intelligence knobs — then writes a fully-populated .env with sensible production defaults for everything else (caching, compression, policy budgets, MCP sandbox, rate limiting).

Useful flags:

lynkr init --force                # overwrite an existing .env

See docs/init.md for the full wizard reference.

If you'd rather configure by hand, the manual options below still work — copy .env.example to .env and edit it directly:

Option A: Free & Local (Ollama) - Recommended for Testing

# Install Ollama first: https://ollama.com
ollama pull qwen2.5-coder:latest

Then start Lynkr:

lynkr start

3. Connect Your Tool

Cursor IDE

• Settings → Models → Override Base URL
• Set to: http://localhost:8081/v1
• API Key: any-value

Edit ~/.codex/config.toml:

model_provider = "lynkr"

[model_providers.lynkr]
base_url = "http://localhost:8081/v1"
wire_api = "responses"

✅ Done! Your AI tool now uses your chosen provider.

Common Startup Errors

Error: unable to determine transport target for "pino-pretty"

Problem: You're running an older version (< 9.3.0).

Solution: Update to the latest version:

npm install -g lynkr@latest

If you must use an older version, set NODE_ENV=production before starting.

Warning: Missing tier configuration: TIER_SIMPLE, TIER_MEDIUM...

This is just a warning - you can ignore it. Tier routing is optional.

To remove the warning, add to .env:

TIER_SIMPLE=ollama:qwen2.5-coder:latest
TIER_MEDIUM=ollama:qwen2.5-coder:latest
TIER_COMPLEX=ollama:qwen2.5-coder:latest
TIER_REASONING=ollama:qwen2.5-coder:latest

Warning: FALLBACK_PROVIDER='databricks' is enabled but missing credentials

Solution: Add to .env:

FALLBACK_ENABLED=false

Error: connect ECONNREFUSED ::1:11434 (Ollama)

Problem: Ollama is not running.

Solution:

ollama serve

Keep this terminal open, and start Lynkr in a new terminal.

Error: Connection refused or 404 Not Found

Problem: Lynkr is not running or wrong port.

Solution: Check Lynkr is running on the correct port:

curl http://localhost:8081/

Should return: {"service":"Lynkr","version":"9.x.x","status":"running"}

Why Lynkr?

AI coding tools lock you into one provider and send every token raw. Lynkr breaks both locks.

Claude Code / Cursor / Codex / Cline / Continue
                    ↓
                  Lynkr
          ┌─────────────────────┐
          │  Strip unused tools  │  ← 53% fewer tokens on tool calls
          │  Compress JSON blobs │  ← 84% on large tool results
          │  Semantic cache      │  ← <300ms hits, 0 tokens billed
          │  Route by complexity │  ← cheap model for simple, cloud for hard
          │  Learn from outcomes │  ← kNN + bandit + auto-calibration
          └─────────────────────┘
                    ↓
    Ollama | Bedrock | Azure | Moonshot | OpenRouter | OpenAI

What you get:

• ✅ 53% fewer tokens on tool-heavy requests (Claude Code, Cursor sessions)
• ✅ 84% compression on large JSON tool results (grep, file reads, test output)
• ✅ Semantic cache serves repeated queries in under 300ms with 0 tokens billed
• ✅ Automatic tier routing — simple questions go to cheap models, complex ones escalate; sessions stick to one model until the task genuinely outgrows it
• ✅ A closed learning loop — every outcome trains a kNN router and bandit, and tier thresholds re-calibrate nightly from your own traffic
• ✅ Route through your company's infrastructure (Databricks, Azure, Bedrock)
• ✅ Zero code changes — just change one environment variable

Supported Providers

| Provider | Type | Example Models | Cost | |----------|------|---------------|------| | Ollama | Local | qwen2.5-coder, deepseek-coder, llama3 | Free | | llama.cpp | Local | Any GGUF model | Free | | LM Studio | Local | Local models with GUI | Free | | OpenRouter | Cloud | GPT-4o, Claude 3.5, Llama 3, Gemini | $ | | OrcaRouter | Cloud | Chat, reasoning, image, video (one endpoint, live catalog) | $ (zero-markup) | | AWS Bedrock | Cloud | Claude, Llama, Mistral, Titan | $$ | | Databricks | Cloud | Claude Sonnet 4.5, Opus 4.6 | $$$ | | Azure OpenAI | Cloud | GPT-4o, o1, o3 | $$$ | | Azure Anthropic | Cloud | Claude Sonnet, Opus | $$$ | | OpenAI | Cloud | GPT-4o, o3-mini | $$$ | | Atlas Cloud | Cloud | Qwen, DeepSeek, and other OpenAI-compatible models | $-$$$ | | DeepSeek | Cloud | DeepSeek R1, Reasoner | $ | | Z.ai | Cloud | GLM-4.7, GLM-4.5-Air | $ | | Moonshot AI | Cloud | Kimi K2.6, Kimi K3 | $ | | Baidu Qianfan | Cloud | ERNIE 4.5 Turbo, ERNIE X1.1 | $ (unverified — not yet probed against a live key) | | Fireworks AI | Cloud | Kimi K2, GLM-5, DeepSeek V3 (serverless) | $ (unverified — not yet probed against a live key) |

4 local providers for 100% offline, free usage. 14+ cloud providers for scale.

_{Use Lynkr with Atlas Cloud's OpenAI-compatible API and route across its live model catalog with one optional provider configuration. Explore Atlas Cloud.}

Advanced: Tier Routing (Save Even More)

Route different request types to different models automatically:

# .env file
MODEL_PROVIDER=ollama
FALLBACK_ENABLED=false
Use small/fast models for simple tasks
TIER_SIMPLE=ollama:qwen2.5:3b
Use medium models for normal coding
TIER_MEDIUM=ollama:qwen2.5:7b
Use powerful models for complex architecture
TIER_COMPLEX=ollama:deepseek-r1:14b
TIER_REASONING=ollama:deepseek-r1:14b
Optional: Limits (remove for unlimited) for long conversations
POLICY_MAX_STEPS=50
POLICY_MAX_TOOL_CALLS=100

Lynkr analyzes each request and routes it to the appropriate tier. Simple questions use fast models. Complex refactoring uses powerful models. The scorer combines anchor-embedding classification (WS7, payload-invariant) with an LLM difficulty classifier (Phase 6, added 2026-07-19) that catches topic-vs-difficulty confounding — list the exports from this file correctly routes to MEDIUM instead of getting boosted to REASONING by "technical vocabulary" alone.

Result: 70-90% of requests use cheaper/faster models. Only hard problems hit expensive models. 15× reduction in expensive-tier over-routing vs the anchor-only baseline on the 381-prompt eval set (0.6% vs ~15%).

Tier configuration is strictly authoritative — bandit exploration is constrained to the models you've listed in TIER_*, and multi-turn conversations score with a recency-weighted sliding window so context isn't lost on short follow-ups. Conversations get a content-fingerprint session id (clients like Claude Code send none), the decision pins for the session, and a guarded escape ladder (risk keywords, force phrases, score drift, context overflow) re-escalates the moment a task outgrows its model. Full pipeline: docs/routing-intelligence.md · intent scorer: docs/intent-window-routing.md · verify any change: docs/benchmarking.md.

Common Issues & Fixes

| Issue | Solution | |-------|----------| | "Service temporarily overloaded" | Ollama model too large for RAM. Use smaller model or increase --max-old-space-size | | "Route not found: HEAD /" | Ignore - harmless health check from Claude Code | | "Hallucinated tool calls" | Normal - Lynkr automatically filters invalid tools | | "Safe Command DSL blocked" | Add POLICY_SAFE_COMMANDS_ENABLED=false to .env | | "spawn graphify ENOENT" | Optional feature. Set CODE_GRAPH_ENABLED=false in .env (see Advanced Features section for installation) | | Slow first request (20+ sec) | Ollama loading model into memory. Add OLLAMA_KEEP_ALIVE=30m in Ollama config | | No response after N turns | Remove POLICY_MAX_STEPS and POLICY_MAX_TOOL_CALLS from .env (unlimited by default in v9.3.0+) |

Advanced Features

Real-time SSE streaming (all tiers)

Responses stream token-by-token instead of arriving all at once — including through the tier router. Two mechanisms, both on by default:

• Native passthrough: when the upstream already speaks Anthropic SSE

• Cross-format transform: OpenAI-format upstreams (OpenAI, Azure OpenAI,

The LYNKR_VISIBLE_ROUTING badge streams too (injected as the first content block), and telemetry is recorded on stream close with real token counts. Fallback safety: if an upstream fails before the first byte, the request falls back to the buffered path automatically.

# Ollama thinking models (MiniMax): streaming skips the <think>-leak repair.
If you see raw <think> text in responses, buffer that provider instead:
LYNKR_OLLAMA_BUFFER_RESPONSES=true   # default true; false = stream Ollama

Token Optimization (60-80% savings)

# Enable all optimizations
PROMPT_CACHE_ENABLED=true
SEMANTIC_CACHE_ENABLED=true

Always-on (no config): RTK tool-result compression (test/git/grep/lint/build/JSON output), MCP tool dedup (drops built-in WebSearch/WebFetch when an Exa/Tavily MCP tool is present), and request bypass (Claude CLI Warmup / title-extraction calls are answered locally, never hitting a provider).

Optional terse-output mode to cut output tokens:

CAVEMAN_ENABLED=true        # off by default — nudges the model to be concise
CAVEMAN_LEVEL=lite          # lite | full | ultra

Built-in dashboard

Open http://localhost:8081/dashboard while Lynkr is running. No setup, reads the local telemetry store:

• Spend & savings — actual cost vs what the same traffic would have cost

• Tier mix — daily request breakdown across SIMPLE/MEDIUM/COMPLEX/REASONING
• Routing accuracy — over-/under-provisioned request counts, a self-audit

• Request logs — filterable by provider, tier, and errors, with latency,

• Provider health — configured providers, credential warnings, circuit

• Insights — analyzer findings ranked by past overspend (auditable

• Explore — a Metric × Dimension (× Stack) pivot over routing telemetry

• Session drill-down — per-model cost mix and an input-context-per-call

• Cache receipts — projected switch/hold economics from cache-aware

JSON APIs behind it (/dashboard/api/overview|usage|routing|logs|recommendations|analytics|sessions/:id|statusline) if you want the raw numbers.

Status line for Claude Code

One line after every turn — routed tier, served model, today's spend, and cache re-read share — with zero token cost (out-of-band, never enters model context). Add to ~/.claude/settings.json:

"statusLine": { "type": "command", "command": "lynkr-statusline" }

◆ Opus 4.8 · MEDIUM → minimax-m3:cloud (ollama) · today $0.06 · cache 67%

Always prints exactly one line and always exits 0 — a down proxy can never break the harness. Set LYNKR_PORT if Lynkr isn't on 8081.

Cost tracking & model pricing

# Per-1M-token USD prices, JSON keyed by model name
MODEL_PRICE_OVERRIDES={"my-model":{"input":0.5,"output":1.5}}

Memory System (Titans-inspired)

MEMORY_ENABLED=true
MEMORY_TTL=3600000  # 1 hour

Load Shedding & Resilience

LOAD_SHEDDING_ENABLED=true
LOAD_SHEDDING_HEAP_THRESHOLD=0.85

Admin Hot-Reload (no restart needed)

curl -X POST http://localhost:8081/v1/admin/reload

Code Intelligence (Optional - Graphify)

Graphify provides AST-based code analysis for smarter routing decisions.

Installation (Rust required):

# Install Rust if not already installed
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

Build and install graphify
git clone https://github.com/safishamsi/graphify
cd graphify
cargo build --release
sudo cp target/release/graphify /usr/local/bin/
Verify installation
graphify --version

Enable in .env:

CODE_GRAPH_ENABLED=true
CODE_GRAPH_WORKSPACE=/path/to/your/project  # Optional, defaults to cwd

Features:

• AST-based complexity scoring
• Structural code analysis (19 languages supported)
• Enhanced routing decisions based on code structure

Installation Methods

NPM (recommended)

npm install -g lynkr

One-line installer

curl -fsSL https://raw.githubusercontent.com/Fast-Editor/Lynkr/main/install.sh | bash

Homebrew (macOS / Linux)

brew tap fast-editor/lynkr
brew install lynkr
lynkr --version

Docker

git clone https://github.com/Fast-Editor/Lynkr.git
cd Lynkr
docker-compose up -d

From source

git clone https://github.com/Fast-Editor/Lynkr.git
cd Lynkr
npm install
cp .env.example .env
npm start

Documentation

| Guide | Description | |-------|-------------| | Installation | All installation methods | | Provider Setup | Configuration for all 12+ providers | | Claude Code | Claude Code CLI integration | | Cursor IDE | Cursor setup + troubleshooting | | Codex CLI | Codex configuration | | Tier Routing | Smart model routing by complexity | | Token Optimization | 60-80% cost reduction | | Troubleshooting | Common issues and solutions | | API Reference | REST API endpoints | | Production | Enterprise deployment |

Benchmark Results

Head-to-head against LiteLLM on the same backends (Ollama minimax-m2.5, Moonshot, Azure OpenAI), 9 scenarios across 4 feature categories. Apples-to-apples comparison is Lynkr vs LiteLLM billed tokens on the same scenario. Run with node benchmark-tier-routing.js.

_Runs: token benchmarks — June 5, 2026 (Lynkr v9.3.2 · LiteLLM v1.87.1). Tier routing head-to-head — re-run July 19, 2026 (Lynkr working tree w/ Phase-6 classifier + config B · LiteLLM v1.94.0.dev1 Auto Router v2). macOS, Apple Silicon._

Token reduction (vs LiteLLM, same model & prompt)

| Mechanism | Lynkr | LiteLLM | Result | |---|---|---|---| | TOON compression (60-item grep JSON) | 427 tokens · $0.009 | 3,458 tokens · $0.018 | 87.6% fewer tokens, 50% cheaper |

Lynkr binary-compresses large JSON tool results (TOON) in-process, with no added latency.

Semantic cache

| | Tokens billed | Response time | |---|---|---| | First call (cold) | 2,857 | 1,891ms | | Second call — paraphrased, cache hit | 0 (served from cache) | 171ms (11× faster) |

Near-identical prompts return cached responses in 171ms. Zero model tokens billed on a cache hit.

Tier routing — vs LiteLLM Auto Router v2 (re-run July 19, 2026)

LiteLLM v1.94 shipped a native complexity router (auto_router/complexity_router) with the same four tier names Lynkr uses. Head-to-head on the same backends with identical prompts, both proxies live, both judged against the same acceptable-tier sets (11 routing scenarios, MODE=routing node benchmark-tier-routing.js, LiteLLM config in litellm-autorouter-v2.yaml):

| Router | Routing-correct | Notes | |---|---|---| | Lynkr (Phase-6 classifier + config B) | 11/11 ✅ | anchor embedding + local LLM classifier reconcile + FORCE_REASONING patterns + risk-remap; ~500ms warm classifier call, cached | | LiteLLM v2 — heuristic (default) | 4/11 | every miss under-routed — banking security analysis, whole-pipeline refactor, prod auth-file fix, autonomous agentic loop all sent to a 7B local model | | LiteLLM v2 — LLM classifier | 6–8/11 (non-deterministic) | paid GPT-5.2 call + ~2–3s on every request; fails outright with local classifier models |

The 7 requests LiteLLM v2 misroutes to a 7B ollama model (should be COMPLEX or REASONING):

| What you typed | Lynkr sends to | LiteLLM sends to | |---|---|---| | "Analyse security trade-offs of JWT vs httpOnly cookies for a banking app" | Claude Opus (top) | ollama 7B ❌ | | "Refactor the entire ingestion pipeline and give me the plan" | z.ai GLM-5.2 (mid) | ollama 7B ❌ | | "Fix the null-check bug in src/auth/middleware.ts" | Claude Opus (risk-remap) | ollama 7B ❌ | | "Figure out why this test is flaky. You have full autonomy — iterate until 10 runs pass" | Claude Opus (autonomous) | ollama 7B ❌ | | "please summarize the exports of this file quickly" | ollama minimax (correct — MEDIUM) | ollama minimax — but for the wrong reason (LiteLLM under-routed to SIMPLE) |

Lynkr layers the following, in decreasing priority, to catch every miss LiteLLM leaks:

• FORCE_REASONING regex (ultrathink, prove, security audit, from first principles) → deterministic top-tier
• Risk classifier (auth/middleware/credentials paths) → REASONING under config B
• Agentic detector (AUTONOMOUS workflows) → REASONING minimum
• Anchor-embedding classifier over 4 difficulty classes + FRONTIER_MIN_SIM floor
• Local LLM classifier (qwen2.5:3b) reconciles borderline anchor calls
• Envelope invariance — scores cleaned user text only, ignoring tool schemas / history / system-reminders

_Fairness notes: the 11 scenarios derive from Lynkr's own regression suite, so Lynkr has home-field advantage — the transferable finding is the direction of LiteLLM's failures (systematic under-routing to cheap on hard prompts), not the exact scores. Lynkr's top tiers used Claude Opus 4.8 / z.ai GLM-5.2; LiteLLM was given the identical tier targets._

Cost projection (100,000 requests/month, same backend)

Direct cost (raw meter):

| | Monthly cost | vs LiteLLM | |---|---|---| | LiteLLM (TOON tool-heavy scenario) | ~$818 | baseline | | Lynkr (TOON tool-heavy) | ~$409 | ~50% cheaper via token optimization |

Effective cost per correct-tier answer (July 19 head-to-head, all 11 routing scenarios):

| | Direct $/mo | Effective $/mo if you re-issue misroutes at correct tier | |---|---|---| | Lynkr | ~$5.50 (spends where it should, doesn't where it shouldn't) | ~$5.50 | | LiteLLM v2 heuristic | $0 (under-routes everything to ollama) | ~$89 (7 misroutes need re-issue on Moonshot/COMPLEX) |

The story isn't "Lynkr always spends less." It's Lynkr spends where it should and doesn't where it shouldn't. LiteLLM's zero-dollar cost on complex prompts is bought with wrong-tier answers to hard work — a 7B model answering "prove this rate limiter is fair under concurrent refill" is silent quality collapse, not a savings.

→ Full benchmark report with methodology

Cost Comparison

| Scenario | Direct Anthropic | Lynkr + Ollama | Lynkr + OpenRouter | |----------|-----------------|----------------|-------------------| | Daily coding (8h) | $10-30/day | $0 (free) | $2-8/day | | Monthly (heavy use) | $300-900 | $0 | $60-240 |

With tier routing + token optimization: additional 50-87% savings on cloud providers depending on workload.

Why Lynkr vs Alternatives

| Feature | Lynkr | LiteLLM | OpenRouter | PortKey | |---------|-------|---------|-----------|---------| | Setup | npm install -g lynkr | Python + Docker + Postgres | Account signup | Docker stack | | Claude Code native | ✅ Drop-in | ⚠️ Requires config | ❌ | ⚠️ Partial | | Cursor native | ✅ Drop-in | ⚠️ Partial | ❌ | ⚠️ Partial | | Local models | Ollama, llama.cpp, LM Studio | Ollama only | ❌ | ❌ | | Automatic tier routing | ✅ embedding intent + 13-dimension scorer, verified cascade | ⚠️ Auto Router v2 (v1.94): heuristic under-routes, LLM classifier billed per request | ❌ | ❌ Manual metadata | | TOON JSON compression | ✅ up to 87.6% | ❌ | ❌ | ❌ | | Upstream SSE streaming | ✅ native passthrough + cross-format transform | ⚠️ passthrough only | ✅ | ⚠️ | | Semantic cache | ✅ 171ms hits, 0 tokens | ❌ | ❌ | ✅ Prompt cache only | | Savings & routing dashboard | ✅ spend, savings vs flagship, tier mix, routing accuracy, request logs, pivot explorer, session drill-down | ⚠️ spend UI only | ⚠️ usage page | ✅ observability suite (no routing accuracy) | | Cost recommendations | ✅ evidence-backed findings (dead tool schemas, side-traffic, cache weak spots, Wilson-proven downsizes) ranked by past overspend | ❌ | ❌ | ⚠️ alerts, no findings | | Long-term memory | ✅ SQLite, per-session | ❌ | ❌ | ❌ | | MCP integration | ✅ | ❌ | ❌ | ❌ | | Self-hosted | ✅ Node.js only | ✅ Python stack | ❌ SaaS | ✅ Docker | | Dependencies | Node.js 20+ | Python, Prisma, PostgreSQL | None | Docker, Python |

Lynkr's edge: Purpose-built for AI coding tools. Compresses tokens before they reach the model — not just after. Zero-config for Claude Code, Cursor, and Codex. Installs in one command.

Community

• GitHub Discussions — Ask questions
• Report Issues — Bug reports
• NPM Package — Official releases
• DeepWiki — AI-powered docs

License

Apache 2.0 — See LICENSE.

Built by Vishal Veera Reddy for developers who want control over their AI tools.