FlashRSI: blue-white electric arcs trace a lightning logo and the large white wordmark on a deep black background.

FlashRSI: A Unified Framework for Agent Recursive Self-Improvement

One click to reproduce, compare, and develop RSI methods across agents and tasks.

FlashRSI provides one experiment interface for 13 RSI algorithms, 30 benchmarks, and 7 harness integrations: MiniAgent, Claude Code, Codex, Pi, Hermes, OpenClaw, and OpenCode. Each run records the resources that changed, trajectories that produced the score, and the Snapshot selected at the end.

English · 简体中文

Installation / Features / Quick start / Design Logic / Supporting Methods / Supporting Benchmarks

Benchmark, RSI algorithm, and harness reels settle on a combination, followed by illustrative Training, Held-out, Near-domain, and OOD-domain curves.

Features

🧩 Configurable Experiment Framework: Combines an RSI method, Target harness, Worker, benchmark,

and independent model settings in one YAML/CLI experiment. Resource scopes and execution roles can change without writing a new runner.

🧪 30 Benchmarks and 13 RSI Methods: Covers question answering, reasoning, mathematics, coding,

web research, documents, tool use, and interactive tasks, with **34 pinned Hugging Face prepared configurations** for 29 suites, plus runnable adaptations of OPRO, GEPA, EvoPrompt, ACE, SkillOpt, Evo-Harness, AHE, ADAS, DarwinX, Meta-Harness, ModularRSI, RRSI, and Arbor.

🧠 Native Harnesses and Reproducible Evidence: MiniAgent, Claude Code, Codex, Pi, Hermes, OpenClaw, and OpenCode map prompts,

skills, tools, configuration, hooks, plugins, memory, and runtime resources into versioned Snapshots. Each run keeps immutable candidates, redacted configuration, Target/Worker trajectories, metrics, usage, and task artifacts together.

🚀 Simple Workflow and Extensible Docs: Use flashrsi run for evolution and flashrsi evaluate

for saved Snapshots, with method examples, a searchable documentation site, and GitHub Pages support. OpenCode is available through the pure-Python Snapshot runtime; it does not require Bun, Node, npm, or the OpenCode CLI.

See the documentation home for the guide, integrations, and development notes.

Installation

To install the framework and all non-Harbor benchmark integrations, use the single all extra:

git clone https://github.com/RUC-NLPIR/FlashRSI.git
cd FlashRSI
uv venv --python 3.12
source .venv/bin/activate
uv sync --extra all

or using pip:

python -m pip install -e ".[all]"

Harbor is kept as a separate extra. Install it together with the complete environment when you need Harbor tasks:

uv sync --extra all --extra harbor

or using pip:

python -m pip install -e ".[all,harbor]"

For Docker-backed tasks, install Docker Desktop or Docker Engine.

Quick start

Run a small RSI experiment with explicit algorithm, benchmark, and target harness choices:

export TARGET_API_KEY="..." export WORKER_API_KEY="..."

flashrsi run \ --algorithm skillopt \ --benchmark searchqa \ --agent-harness miniagent \ --model deepseek-v4-pro \ --worker-model deepseek-v4.1-flash \ --set agent.llm.api_key="$TARGET_API_KEY" \ --set algorithm.worker.llm.api_key="$WORKER_API_KEY"

The same experiment can be started entirely from the checked-in YAML file:

flashrsi run --config examples/quickstart.yaml

Run options

The CLI keeps the common choices short and accepts every nested configuration field through a dotted override. --algorithm chooses the RSI search method, while --agent-harness chooses the harness whose resources are evolved. Use flashrsi run --help for the complete parser help.

| Option | Purpose | | --- | --- | | --config PATH | Load an existing YAML experiment, including its data, split, and role settings. Start with examples/quickstart.yaml. | | --algorithm NAME, --algo NAME | Select the RSI method that proposes and evaluates candidate edits. See Supporting Methods. | | --benchmark NAME | Select the task suite and scoring protocol used to evaluate candidates. See Supporting Benchmarks. | | --agent-harness NAME, --agent NAME | Select the Target harness to evolve and execute. See Supporting Harnesses. | | --model NAME | Set the Target model used to solve benchmark tasks. | | --worker-model NAME | Set the independent Worker model used to propose edits; Agent Workers can be selected through --set algorithm.worker.type=agent. | | --output-dir PATH | Choose the parent directory for run records, Snapshots, trajectories, and artifacts. | | --set PATH=VALUE | Override any nested setting, such as --set algorithm.max_steps=3 or --set algorithm.worker.name=codex. | | --check-only | Check dependencies and data preparation without calling models or creating an experiment result. | | --prepare-only | Write a redacted configuration directory without checking runtime readiness. |

After the run completes, the following artifacts are produced:

runs/<algorithm>-<agent>-<benchmark>-<timestamp>/
├── config.yaml           # Redacted configuration
├── snapshots/            # Candidate resources and lineage
├── trajectories/         # Target rollouts, Worker calls, and decisions
├── artifacts/harbor/     # Collected files and native logs for Harbor tasks
├── result.json           # Selected Snapshot and final test result
├── end-summary.json      # Machine-readable summary
└── end-summary.md        # Human-readable summary

Evaluate a Snapshot

Use evaluate to test how an evolved harness performs on another benchmark without running evolution. Any JSON file under a run's snapshots/ directory can be used, including an intermediate candidate:

flashrsi evaluate \
  --config examples/quickstart.yaml \
  --snapshot <run-dir>/snapshots/<snapshot-id>.json

Design Logic

FlashRSI represents an agent harness as a small, tagged file system. A Snapshot contains the files that define the agent's behavior, while credentials, downloaded data, temporary logs, and task state remain outside the Snapshot. Every file is assigned a role such as prompt, skill, tool, runtime, or config; an algorithm declares which roles it may observe and edit.

AgentSnapshot (tagged files)
        │
        ├── selected scope ──► RSI algorithm edits and versions a candidate
        │
        └── complete candidate ──► Harness executes ──► Benchmark scores trajectories

This boundary lets algorithms share the same checkout, execution, and scoring operations while keeping their search policies independent. The harness always executes a complete candidate, so editing one skill still preserves the instructions, tools, and runtime that make that skill effective.

Harness mapping strategies

| Harness type | How FlashRSI maps it | Current example | | --- | --- | --- | | Closed or interface-first | Map the stable interfaces exposed by the product to files. The Snapshot contains the resources that can be edited; proprietary internals stay in the native harness. | Claude Code: instructions, skills, agents, hooks, MCP/LSP registrations, and settings. | | Open source with a large runtime | Reduce the execution surface against a pinned baseline, document supported behavior and migration limits, then align that implementation with the Snapshot file layout. | Codex: a Python semantic port of supported paths from a pinned Rust baseline; selected resources execute from the Snapshot in an isolated process. | | Clean open source harness | Map the source tree directly and tag each editable file by role. The native entry point continues to execute the selected Snapshot. | MiniAgent: prompts, skills, tools, configuration, and declared runtime code. |

See Agent integration, Codex integration, and the usage guide for implementation details.

Supporting Methods

FlashRSI implements 13 RSI methods:

| Method | Scope | Search organization | Feedback used | | --- | --- | --- | --- | | OPRO | Instruction search | Scored history · reflection · population | Train scores, outputs | | GEPA | Instruction search | Candidate pool and reflection | Scores, traces | | EvoPrompt | Instruction search | GA / DE population | Dev scores | | SkillOpt | Skill evolution | Fast / slow updates | Train traces, dev scores | | Evo-Harness | Skill evolution | Task-stream accumulation | Failure traces | | ACE | Context evolution | Playbook accumulation | Task feedback, reflections | | AHE | Iterative harness refinement | Successive edits | Traces, diagnoses, edit audits | | Meta-Harness | Iterative harness refinement | File-based history | Code, scores, traces | | RRSI | Iterative harness refinement | History-guided revisions | Traces, scores, token costs | | ADAS | Archive search | Design archive | Design accounts, dev scores | | DarwinX | Branch evolution | Branches and recombination | Repeated outcomes, traces | | ModularRSI | Branch evolution | Independent branches; integration | Success / failure traces | | Arbor | Branch evolution | Hypothesis tree and insight propagation | Train traces, dev scores |

Arbor is newly integrated; the original diagram and experiment results below cover the twelve-method study. A small Arbor × Codex × WorkBench smoke is documented in the Arbor validation notes; it does not establish a performance gain.

For SkillOpt's fast/slow updates, selection rules, and budget settings, see the SkillOpt guide.

Instruction-search guides cover OPRO, GEPA, and EvoPrompt, including their different feedback, selection rules, and budget units.

For experience accumulation, see ACE for playbook curation and offline/online selection, or Evo-Harness for native skill compilation from a task stream.

Directory-editing guides cover AHE, ADAS, and Meta-Harness, with their editing bases, evidence, and final selection rules.

Overall results

The table below reports source-task performance and frozen transfer performance for the twelve methods evaluated on Claude Code and Mini-Agent. It covers four source benchmarks, five near-domain transfers, and two far-OOD transfers. Scores are percentages; each result cell shows the final score followed by its change from the corresponding initial harness in percentage points, and bold marks the highest final score within each benchmark and harness, including ties.

Method	Target harness	Source _In-domain				Near-domain _{Frozen transfer}					Far-OOD _{Frozen transfer}
Method	Target harness	SearchQA _In-domain	HotpotQA _In-domain	Terminal-Bench 2.0 _In-domain	AppWorld _In-domain	SearchQA → NQ _Near-domain	SearchQA → TriviaQA _Near-domain	HotpotQA → MuSiQue _Near-domain	Terminal-Bench 2.0 → SWE-bench _Near-domain	AppWorld → BFCL Multi-Turn _Near-domain	SearchQA → OfficeQA Pro _Far-OOD	HotpotQA → DeepSearchQA _Far-OOD
Initial harness	Claude Code	73.93	57.32	63.33	64.88	56.67	68.67	67.50	76.67	65.00	44.19	66.00
Initial harness	Mini-Agent	76.43	58.54	66.67	65.48	59.33	74.33	65.83	80.00	70.00	27.91	52.00
SkillOpt	Claude Code	83.14 _{+9.21 pp}	60.98 _{+3.66 pp}	70.00 _{+6.67 pp}	84.52 _{+19.64 pp}	69.00 _{+12.33 pp}	82.00 _{+13.33 pp}	76.67 _{+9.17 pp}	80.00 _{+3.33 pp}	68.75 _{+3.75 pp}	46.51 _{+2.32 pp}	58.00 _{-8.00 pp}
SkillOpt	Mini-Agent	82.93 _{+6.50 pp}	59.76 _{+1.22 pp}	66.67 _{0.00 pp}	91.67 _{+26.19 pp}	62.00 _{+2.67 pp}	84.33 _{+10.00 pp}	74.17 _{+8.34 pp}	86.67 _{+6.67 pp}	68.75 _{-1.25 pp}	37.21 _{+9.30 pp}	46.00 _{-6.00 pp}
OPRO	Claude Code	81.43 _{+7.50 pp}	62.20 _{+4.88 pp}	63.33 _{0.00 pp}	66.07 _{+1.19 pp}	69.67 _{+13.00 pp}	80.33 _{+11.66 pp}	73.33 _{+5.83 pp}	80.00 _{+3.33 pp}	66.25 _{+1.25 pp}	51.16 _{+6.97 pp}	60.00 _{-6.00 pp}
OPRO	Mini-Agent	80.71 _{+4.28 pp}	60.98 _{+2.44 pp}	66.67 _{0.00 pp}	59.52 _{-5.96 pp}	64.67 _{+5.34 pp}	81.33 _{+7.00 pp}	78.33 _{+12.50 pp}	80.00 _{0.00 pp}	68.75 _{-1.25 pp}	41.86 _{+13.95 pp}	44.00 _{-8.00 pp}
AHE	Claude Code	79.36 _{+5.43 pp}	60.98 _{+3.66 pp}	70.00 _{+6.67 pp}	91.67 _{+26.79 pp}	66.00 _{+9.33 pp}	81.67 _{+13.00 pp}	74.17 _{+6.67 pp}	80.00 _{+3.33 pp}	71.25 _{+6.25 pp}	46.51 _{+2.32 pp}	60.00 _{-6.00 pp}
AHE	Mini-Agent	78.57 _{+2.14 pp}	57.32 _{-1.22 pp}	60.00 _{-6.67 pp}	71.43 _{+5.95 pp}	66.33 _{+7.00 pp}	79.00 _{+4.67 pp}	73.33 _{+7.50 pp}	83.33 _{+3.33 pp}	72.50 _{+2.50 pp}	34.88 _{+6.97 pp}	46.00 _{-6.00 pp}
ACE	Claude Code	83.93 _{+10.00 pp}	63.41 _{+6.09 pp}	66.67 _{+3.34 pp}	91.07 _{+26.19 pp}	64.00 _{+7.33 pp}	85.67 _{+17.00 pp}	72.50 _{+5.00 pp}	73.33 _{-3.34 pp}	73.75 _{+8.75 pp}	41.86 _{-2.33 pp}	54.00 _{-12.00 pp}
ACE	Mini-Agent	84.71 _{+8.28 pp}	63.41 _{+4.87 pp}	70.00 _{+3.33 pp}	91.67 _{+26.19 pp}	62.67 _{+3.34 pp}	85.33 _{+11.00 pp}	75.83 _{+10.00 pp}	70.00 _{-10.00 pp}	67.50 _{-2.50 pp}	30.23 _{+2.32 pp}	54.00 _{+2.00 pp}
EvoPrompt	Claude Code	74.07 _{+0.14 pp}	58.54 _{+1.22 pp}	63.33 _{0.00 pp}	61.90 _{-2.98 pp}	59.67 _{+3.00 pp}	69.00 _{+0.33 pp}	70.00 _{+2.50 pp}	83.33 _{+6.66 pp}	66.25 _{+1.25 pp}	44.19 _{0.00 pp}	64.00 _{-2.00 pp}
EvoPrompt	Mini-Agent	76.71 _{+0.28 pp}	59.76 _{+1.22 pp}	76.67 _{+10.00 pp}	64.29 _{-1.19 pp}	58.67 _{-0.66 pp}	74.00 _{-0.33 pp}	67.50 _{+1.67 pp}	86.67 _{+6.67 pp}	63.75 _{-6.25 pp}	34.88 _{+6.97 pp}	40.00 _{-12.00 pp}
GEPA	Claude Code	82.43 _{+8.50 pp}	62.20 _{+4.88 pp}	66.67 _{+3.34 pp}	67.86 _{+2.98 pp}	65.33 _{+8.66 pp}	83.67 _{+15.00 pp}	78.33 _{+10.83 pp}	76.67 _{0.00 pp}	70.00 _{+5.00 pp}	41.86 _{-2.33 pp}	52.00 _{-14.00 pp}
GEPA	Mini-Agent	84.64 _{+8.21 pp}	59.76 _{+1.22 pp}	70.00 _{+3.33 pp}	75.00 _{+9.52 pp}	61.67 _{+2.34 pp}	86.33 _{+12.00 pp}	75.83 _{+10.00 pp}	70.00 _{-10.00 pp}	62.50 _{-7.50 pp}	51.16 _{+23.25 pp}	34.00 _{-18.00 pp}
ADAS	Claude Code	81.36 _{+7.43 pp}	60.98 _{+3.66 pp}	73.33 _{+10.00 pp}	89.88 _{+25.00 pp}	63.67 _{+7.00 pp}	83.33 _{+14.66 pp}	77.50 _{+10.00 pp}	73.33 _{-3.34 pp}	70.00 _{+5.00 pp}	51.16 _{+6.97 pp}	54.00 _{-12.00 pp}
ADAS	Mini-Agent	84.86 _{+8.43 pp}	60.98 _{+2.44 pp}	66.67 _{0.00 pp}	79.76 _{+14.28 pp}	62.33 _{+3.00 pp}	84.33 _{+10.00 pp}	74.17 _{+8.34 pp}	73.33 _{-6.67 pp}	70.00 _{0.00 pp}	32.56 _{+4.65 pp}	34.00 _{-18.00 pp}
DarwinX	Claude Code	80.50 _{+6.57 pp}	60.98 _{+3.66 pp}	63.33 _{0.00 pp}	70.83 _{+5.95 pp}	65.67 _{+9.00 pp}	82.00 _{+13.33 pp}	77.50 _{+10.00 pp}	66.67 _{-10.00 pp}	72.50 _{+7.50 pp}	48.84 _{+4.65 pp}	60.00 _{-6.00 pp}
DarwinX	Mini-Agent	79.29 _{+2.86 pp}	60.98 _{+2.44 pp}	66.67 _{0.00 pp}	69.05 _{+3.57 pp}	64.67 _{+5.34 pp}	83.00 _{+8.67 pp}	75.83 _{+10.00 pp}	83.33 _{+3.33 pp}	67.50 _{-2.50 pp}	41.86 _{+13.95 pp}	34.00 _{-18.00 pp}
Meta-Harness	Claude Code	85.07 _{+11.14 pp}	62.20 _{+4.88 pp}	70.00 _{+6.67 pp}	90.48 _{+25.60 pp}	64.00 _{+7.33 pp}	86.33 _{+17.66 pp}	80.00 _{+12.50 pp}	80.00 _{+3.33 pp}	66.25 _{+1.25 pp}	53.49 _{+9.30 pp}	68.00 _{+2.00 pp}
Meta-Harness	Mini-Agent	79.14 _{+2.71 pp}	59.76 _{+1.22 pp}	73.33 _{+6.66 pp}	88.10 _{+22.62 pp}	65.67 _{+6.34 pp}	81.00 _{+6.67 pp}	72.50 _{+6.67 pp}	76.67 _{-3.33 pp}	73.75 _{+3.75 pp}	41.86 _{+13.95 pp}	38.00 _{-14.00 pp}
ModularRSI	Claude Code	81.07 _{+7.14 pp}	59.76 _{+2.44 pp}	60.00 _{-3.33 pp}	56.55 _{-8.33 pp}	63.00 _{+6.33 pp}	84.00 _{+15.33 pp}	72.50 _{+5.00 pp}	80.00 _{+3.33 pp}	65.00 _{0.00 pp}	44.19 _{0.00 pp}	52.00 _{-14.00 pp}
ModularRSI	Mini-Agent	78.64 _{+2.21 pp}	59.76 _{+1.22 pp}	70.00 _{+3.33 pp}	69.05 _{+3.57 pp}	67.00 _{+7.67 pp}	81.00 _{+6.67 pp}	65.83 _{0.00 pp}	73.33 _{-6.67 pp}	75.00 _{+5.00 pp}	53.49 _{+25.58 pp}	40.00 _{-12.00 pp}
RRSI	Claude Code	82.79 _{+8.86 pp}	60.98 _{+3.66 pp}	76.67 _{+13.34 pp}	92.86 _{+27.98 pp}	64.00 _{+7.33 pp}	83.67 _{+15.00 pp}	79.17 _{+11.67 pp}	80.00 _{+3.33 pp}	72.50 _{+7.50 pp}	53.49 _{+9.30 pp}	58.00 _{-8.00 pp}
RRSI	Mini-Agent	81.64 _{+5.21 pp}	59.76 _{+1.22 pp}	76.67 _{+10.00 pp}	84.52 _{+19.04 pp}	68.67 _{+9.34 pp}	83.00 _{+8.67 pp}	76.67 _{+10.84 pp}	76.67 _{-3.33 pp}	66.25 _{-3.75 pp}	53.49 _{+25.58 pp}	36.00 _{-16.00 pp}
Evo-Harness	Claude Code	75.36 _{+1.43 pp}	60.98 _{+3.66 pp}	66.67 _{+3.34 pp}	81.55 _{+16.67 pp}	58.67 _{+2.00 pp}	71.33 _{+2.66 pp}	77.50 _{+10.00 pp}	80.00 _{+3.33 pp}	65.00 _{0.00 pp}	34.88 _{-9.31 pp}	60.00 _{-6.00 pp}
Evo-Harness	Mini-Agent	80.64 _{+4.21 pp}	59.76 _{+1.22 pp}	66.67 _{0.00 pp}	77.38 _{+11.90 pp}	65.33 _{+6.00 pp}	80.00 _{+5.67 pp}	75.00 _{+9.17 pp}	76.67 _{-3.33 pp}	72.50 _{+2.50 pp}	48.84 _{+20.93 pp}	58.00 _{+6.00 pp}

Supporting Harnesses

FlashRSI currently supports seven harness integrations; additional integrations are planned.

| Harness | Editable resources | Integration details | Status | | --- | --- | --- | --- | | MiniAgent | Instructions, skills, tools, runtime code, configuration | Native Python entry point; direct file mapping and role tags. | Available | | Claude Code | Instructions, skills, agents, hooks, MCP/LSP, settings | Closed SDK integration; stable exposed interfaces are mapped into the Snapshot. | Available | | Codex | Instructions, skills, agents, hooks, plugins, memory, rules, configuration | Snapshot-local Python port based on rust-v0.147.0, with documented coverage limits. | Available | | Pi | Pure-Python Snapshot runtime, instructions, skills, prompt templates, and settings | Snapshot-local runtime/pi_native follows Pi v1.4.0's headless loop and Pi-shaped JSON events; no Node/npm dependency. | Available | | Hermes | Python Snapshot runtime, SOUL instructions, skills, tool settings | Supported headless Hermes v0.22.5 path with upstream Python helpers and local tools; no installed Hermes CLI required. | Available | | OpenClaw | Pure-Python Snapshot runtime, bootstrap files, skills, tools, policy, session and memory settings | Snapshot-local runtime/openclaw_native follows the pinned embedded turn path across Chat Completions, Responses and Anthropic Messages; no Node/npm/Gateway dependency. | Available | | OpenCode | Pure-Python Snapshot runtime, instructions, tools, permissions, sessions and settings | Snapshot-local runtime/opencode_native follows the pinned v1.19.3 embedded turn path across Chat Completions, Responses and Anthropic Messages; no Bun/Node/CLI dependency. | Available |

Supporting Benchmarks

FlashRSI supports 30 benchmark suites across question answering, reasoning, mathematics, coding, web research, documents, tool use, and interactive tasks. The table below summarizes each suite's task, and primary metric. 34 pinned Hugging Face configurations are available for 29 suites, while native data sources remain supported.

| Domain | Benchmark | Task | Primary metric | | --- | --- | --- | --- | | QA | SearchQA | Search-snippet short-answer QA | Exact Match | | QA | HotpotQA | Multi-hop Wikipedia QA | Joint F1 | | QA | MuSiQue | Compositional multi-hop QA | Answer F1 | | QA | TriviaQA | Wikipedia trivia QA | Exact Match | | QA | Natural Questions | Short-answer and yes/no QA | Exact Match | | QA | 2WikiMultiHopQA | Multi-hop QA with context | Exact Match | | Reasoning | BBEH | BIG-Bench Extra Hard tasks | Accuracy | | Reasoning | SuperGPQA | Graduate-level multiple choice | Accuracy | | Reasoning | HLE | Text-only Humanity's Last Exam | Accuracy | | Math | MATH | Competition mathematics | Accuracy | | Math | AIME | AIME 2025 / 2026 | Accuracy | | Math | LiveMathBench | Repeated-sampling mathematics | mG-Pass@k | | Coding | SWE-bench Verified | Repository issue repair | Resolved rate | | Coding | SWE-bench Pro V2 | Repository repair | Resolved rate | | Coding | DeepSWE | DeepSWE repository tasks | Reward | | Coding | Terminal-Bench 2.0 / 2.1 | Linux terminal tasks | Reward | | Web research | BrowseComp | Broad web research | Accuracy | | Web research | DeepSearchQA | Single- and set-answer search | F1 | | Web research | WideSearch | Tabular broad search | Score | | Documents | OfficeQA-Pro | Document research | Accuracy | | Documents | Harvey LAB | Legal document work | All-pass rate | | Documents | JobBench | Professional file delivery | Reward | | Documents | SpreadsheetBench 2 | Spreadsheet debugging and modeling | Reward | | Tool use | AppWorld | Application and API automation | Reward | | Tool use | BFCL V4 | Multi-turn function calling | Reward | | Tool use | WorkBench | Workplace database tasks | Correctness | | Tool use | TravelPlanner | Constraint-aware itineraries | Final pass rate | | Interactive | τ²-bench | Airline, retail, telecom dialogue | Reward | | Interactive | τ³ Banking | Multi-turn banking support | Reward | | Interactive | ALFWorld | Text-only embodied tasks | Success rate |

Frequently Asked Questions

How do I add a new benchmark?

Implement a Benchmark (or NativeBenchmark) that yields Task objects and grades Trajectory objects, register it in flashrsi/benchmarks/__init__.py, add a configuration example, and document its source, revision, split policy, metric, and environment requirements. Use the benchmark index and the benchmark interface as the checklist.

How do I add a new algorithm?

Subclass Algorithm, declare the observation and editing scope, and keep proposal, selection, and memory logic inside evolve. Use the shared Snapshot checkout, rollout, and scoring operations; do not move algorithm policy into the runner. Start with the existing algorithm implementations and the usage guide.

How do I add a new harness?

Declare editable resources in a components.yaml, implement an Agent adapter whose _run calls the native entry point, and return a Trajectory. Map the harness according to its source model: interface files for a closed product, a minimal faithful runtime for a large open system, or direct tagged files for a clean open harness. See Agent integration.

What is a Snapshot?

A Snapshot is an immutable, executable version of the harness resources selected for an experiment. It contains the files that define behavior and records parentage and changes; credentials, task data, logs, and other runtime state stay outside it. Any Snapshot can be checked out, evaluated, compared, or promoted without changing the original version.

The persisted Snapshot is one JSON file, usually written as runs//snapshots/.json. Its top-level fields include format_version, agent_type, component_rules, id, root_id, parent_id, and files. Each entry in files stores a normalized relative path, the component role, an executable flag, and the file's exact bytes encoded as canonical base64. The Snapshot ID is a SHA-256 hash of the canonical file contents and version metadata, so changing one declared resource produces a new content-addressed version.

When a candidate is evaluated, FlashRSI materializes those bytes into a temporary checkout and the native harness runs from that complete tree. An algorithm's read/write scope is a temporary permission view; it is not serialized into the Snapshot ID or JSON. Saving uses a temporary file, fsync, and an atomic link, so an existing Snapshot file is never silently overwritten.

Citation

Please cite our paper if FlashRSI helps your research:

@misc{jin2026flashrsi,
  title  = {FlashRSI: A Unified Framework for Agent Self-Improvement},
  author = {Jiajie Jin and Yuze Li and Yuyang Hu and Guanting Dong and
            Xiaoxi Li and Xinyu Yang and Jiejun Tan and Zhicheng Dou and Yutao Zhu},
  year   = {2026},
}

License

FlashRSI is released under the Apache License 2.0. Individual benchmark datasets, model APIs, and upstream harnesses may have additional terms; follow the license and usage conditions of each source.

Gaoling School of Artificial Intelligence · Renmin University of China

RUC-NLPIR/FlashRSI: A Unified Framework for Efficient RSI Research