NeMo Gym

Requirements • Quick Start • Environment Tutorials • Available Environments • Documentation & Resources • Community & Support • Citations

NeMo Gym is a library for evaluating and improving models and agents using environments. NeMo Gym provides infrastructure to develop environments, scalably run evaluation and training, and a collection of popular benchmarks and training environments.

An environment is the complete system an agent interacts with to complete a task. It consists of a dataset (tasks to solve), an agent harness (how the model interacts with the world), a verifier (task completion scoring), and state (per-task execution context).

🎯 When to Use NeMo Gym

• You need to evaluate models or agents in stateful environments (e.g. code execution, tool calling, sandboxes)
• You want reproducible evaluation across teams using shared environments and verifiers
• You need to use environments at scale — multiple repeats per task, or thousands of concurrent requests for training
• You want to seamlessly transition between evaluation, agent optimization, and training

🏆 What NeMo Gym Provides

• Modular, extensible interfaces for agents, environments, tasks, and verifiers
• Environment hub of popular benchmarks and training environments
• Use your own agents or choose from built-in harnesses
• Scale to thousands of concurrent environments
• Train with the RL framework of your choice
• Optional OpenTelemetry tracing across the agent, model, and resources servers
• Battle-tested in production Nemotron training

🌎 Ecosystem

NeMo Gym is a component of NVIDIA NeMo, a GPU-accelerated platform for training generative AI models and optimizing AI agents. NeMo Gym is integrated with the broader agentic ecosystem - see the Ecosystem page for more details.

Environment Libraries: Seamlessly combine environments and benchmarks from other libraries alongside NeMo Gym environments. Examples: Aviary • Harbor • OpenEnv • Reasoning Gym • Verifiers

Training Framework Libraries: Use environments for SFT and RL training. NeMo RL • Unsloth • VeRL

Agent Harnesses: Agent harnesses for evaluation and training available out of the box. Examples: OpenHands • Mini SWE Agent • LangGraph

[!IMPORTANT]

NeMo Gym is currently in early development. You should expect evolving APIs, incomplete documentation, and occasional bugs. We welcome contributions and feedback - for any changes, please open an issue first to kick off discussion!

📣 News

• [09/03/2026] Release v0.6.0:

📋 Requirements

NeMo Gym is designed to run on standard development machines:

| Hardware Requirements | Software Requirements | | --------------------- | --------------------- | | GPU: Not required for NeMo Gym library operation
• GPU may be needed for specific resources servers or model inference (see individual server documentation) | Operating System:
• Linux (Ubuntu 20.04+, or equivalent)
• macOS (11.0+ for x86_64, 12.0+ for Apple Silicon)
• Windows (via WSL2) | | CPU: Any modern x86_64 or ARM64 processor (e.g., Intel, AMD, Apple Silicon) | Python: 3.13.14 or higher | | RAM: Minimum 8 GB (16 GB+ recommended for larger environments) | Git: For cloning the repository | | Storage: Minimum 5 GB free disk space for installation and basic usage | Internet Connection: Required for downloading dependencies and API access |

Additional Requirements

• API Keys: OpenAI API key with available credits (for the quickstart examples)

• Ray: Automatically installed as a dependency (no separate setup required)

🚀 Quick Start

Requires Python 3.13.14+ on x86_64 or ARM64 (Linux, macOS, Windows via WSL2). No GPU required. See the Getting Started docs for a more comprehensive walkthrough.

Install NeMo Gym:

Requires uv and Python 3.13.14+.

git clone [email protected]:NVIDIA-NeMo/Gym.git
cd Gym
uv venv --python 3.13.14 && source .venv/bin/activate
uv sync

Configure your model:

This quickstart uses OpenAI. NeMo Gym supports local and hosted inference — see Configure Model for vLLM, Fireworks, OpenRouter, and others.

Create env.yaml in the project root:

policy_base_url: https://api.openai.com/v1
policy_api_key: <your-openai-api-key>
policy_model_name: gpt-4.1-2025-04-14

Run Evaluation

Run your agent on a set of tasks and score the results. This example uses a simple tool calling agent simple_agent with the mcqa (multiple-choice Q&A) environment and its included example data.

1. Start servers

NeMo Gym uses local servers to coordinate your model, agent, and task verification. Start them first:

gym env start \
    --resources-server mcqa \
    --model-type openai_model

You should see three server instances starting:

[1] mcqa (resources_servers/mcqa)
[2] mcqa_simple_agent (responses_api_agents/simple_agent)
[3] policy_model (responses_api_models/openai_model)

2. Evaluate your agent

In a new terminal, run your agent on a single task to verify everything works:

source .venv/bin/activate
gym eval run --no-serve \
    --agent mcqa_simple_agent \
    --input resources_servers/mcqa/data/example.jsonl \
    --output results/mcqa_rollouts.jsonl \
    --limit 5 \
    --num-repeats 1

You should see a progress bar followed by aggregate metrics:

Collecting rollouts: 100%|██████| 5/5 [01:22<00:00, 16.44s/it]
Key metrics for mcqa_simple_agent:
{
    "mean/reward": 0.8,
    "pass@1[avg-of-1]/accuracy": 80.0,
    "pass@1/accuracy": 80.0
}
Finished rollout collection! View results at:
Fully materialized inputs: results/mcqa_rollouts_materialized_inputs.jsonl
Rollouts: results/mcqa_rollouts.jsonl
Aggregate metrics: results/mcqa_rollouts_aggregate_metrics.json

For per-task pass rates, see the gym eval profile command.

Using the NeMo-Gym Container with VLM or Audio/Video Benchmarks

The NeMo-Gym container omits packages with bundled codec libraries (opencv-python-headless, torchvision, torchaudio) to avoid shipping royalty-bearing binaries. If you are running VLM or audio/video benchmarks inside the container, restore them first:

bash docker/install_codec_deps.sh

This installs the packages at the same versions used during the container build. It is safe to run multiple times.

Next Steps

• Browse Environments — Browse available environments for evaluation and training.
• Agents — Explore available agent harnesses and learn how to integrate your own.
• Training — Improve your agent or model with RL or fine-tuning.
• Build Custom Environments — Create your own evaluation or training environments.

🧭 Environment Tutorials

Learn how to build custom environments through hands-on tutorials. Here are popular starting points:

| Name | Demonstrates | | ---- | ------------ | | Single Step | Basic single-step tool calling | | Multi Step | Multi-step tool calling | | Session State | Session state management (in-memory) | | Multi Reward | Multiple reward components for evaluation and multi-objective RL (e.g. GDPO) |

See all environment tutorials for additional patterns and advanced topics.

📦 Available Environments

Environments for training and evaluation.

Each resources server includes example data, configuration files, and tests. See each server's README for details.

The Dataset column links to publicly available datasets (e.g., on HuggingFace). A - means the train/validation data has not been publicly released yet, or that it is procedurally generated using a provided script. If no data is released yet, new data can be generated, or the environment can be used as a reference. Each server includes 5 example tasks in data/example.jsonl.

| Environment | Domain | Description | Value | Train | Validation | License | Config | Dataset | | --------------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ----- | ---------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Aalcr | other | - | - | - | - | - | aalcr.yaml | - | | Abstention | rlhf | Train models to abstain when unsure using three-tier reward on HotPotQA with LLM judge | Improve calibration by rewarding abstention over incorrect answers | ✓ | ✓ | Creative Commons Attribution-ShareAlike 4.0 International | abstention.yaml | - | | Agentif | instruction_following | AgentIF instruction-following benchmark (707 agentic scenarios) scored with CSR/ISR using an LLM judge for llm/llm_conditional_check constraints and code exec for code constraints. | Improve instruction-following in agentic scenarios across unconditional, conditional, and example-driven constraint dimensions. | - | ✓ | - | agentif.yaml | - | | Anyswe Agent | coding | SWE-bench run by a NeMo Fabric-selected harness inside each task environment. | Evaluate software-engineering agents with a harness-neutral Fabric lifecycle. | - | - | - | anyswe_nemo_fabric.yaml | - | | Anyswe Agent | coding | SWE-bench run by Claude Code natively inside the task container. | Eval software engineering capabilities on SWE-bench with any Gym agent. | - | - | - | anyswe_claude_code.yaml | - | | Anyswe Agent | coding | SWE-bench run by Hermes Agent natively inside the task container. | Eval software engineering capabilities on SWE-bench with any Gym agent. | - | - | - | anyswe_hermes.yaml | - | | Anyswe Agent | coding | SWE-bench run by OpenClaw natively inside the task container. | Eval software engineering capabilities on SWE-bench with any Gym agent. | - | - | - | anyswe_openclaw.yaml | - | | Anyswe Agent | coding | SWE-bench run by OpenCode inside the task container. | Eval software engineering capabilities on SWE-bench with any Gym agent. | - | - | - | anyswe_opencode.yaml | - | | Anyswe Agent | coding | SWE-bench run by Pi inside the task container. | Eval software engineering capabilities on SWE-bench with any Gym agent. | - | - | - | anyswe_pi.yaml | - | | Anyswe Agent | coding | SWE-bench run by the Cline CLI natively inside the task container. | Eval software engineering capabilities on SWE-bench with any Gym agent. | - | - | - | anyswe_cline.yaml | - | | Anyterminal Agent | coding | Terminal Bench run by a NeMo Fabric harness inside the task sandbox. | Evaluate terminal-task agents through a harness-neutral Fabric lifecycle. | - | - | - | anyterminal_nemo_fabric.yaml | - | | Anyterminal Agent | coding | Terminal Bench run by claude-code natively inside the task container. | Evaluate terminal-task capabilities on Terminal Bench with any Gym agent. | - | - | - | anyterminal_claude_code.yaml | - | | Anyterminal Agent | coding | Terminal Bench run by OpenClaw natively inside the task container. | Evaluate terminal-task capabilities on Terminal Bench with any Gym agent. | - | - | - | anyterminal_openclaw.yaml | - | | Anyterminal Agent | coding | Terminal Bench run by Terminus-2 natively inside the task container. | Evaluate terminal-task capabilities on Terminal Bench with any Gym agent. | - | - | - | anyterminal_terminus_2.yaml | - | | Anyterminal Agent | coding | Terminal Bench run by the Hermes agent inside the task container. | Evaluate terminal-task capabilities on Terminal Bench with any Gym agent. | - | - | - | anyterminal_hermes.yaml | - | | Arc Agi | knowledge | Solve puzzles designed to test intelligence. See https://arcprize.org/arc-agi. | Improve puzzle-solving capabilities. | - | ✓ | - | arc_agi.yaml | - | | Arena | rlhf | LMArena proxy v2 chat evaluation benchmark | Measure general chat quality via win rate against baseline responses | - | ✓ | - | lmarena_v2.yaml | - | | Arena | rlhf | LMArena proxy v3 chat evaluation benchmark | Measure general chat quality via win rate against baseline responses | - | ✓ | - | lmarena_v3.yaml | - | | Arena Judge | | - | - | - | - | - | arena_judge.yaml | - | | Asr With Pc | other | ASR with WER scoring (standard, case-sensitive, punctuation+capitalization) | Improve transcription quality with structural detail | - | - | - | asr_with_pc.yaml | - | | Bbq | safety | BBQ comparative QA with Answer and Explanation Quality checks | Evidence-grounded and fairness-safe comparative reasoning | - | - | - | bbq.yaml | - | | Bigcodebench | coding | Verifies model-generated Python solutions against the BigCodeBench unittest suite. | Improve practical, library-rich Python coding capabilities. | - | - | - | bigcodebench.yaml | - | | Bird Sql | coding | Text-to-SQL with execution-based evaluation on BIRD dev (1534 SQLite tasks). Binary reward from unordered result-set equality. | Improve text-to-SQL capabilities on BIRD's realistic dev split using execution-based binary reward without an LLM judge. | - | - | - | bird_sql.yaml | - | | Blackjack | games | Blackjack. Model hits or stands. Reward +1 win, 0 draw, -1 loss/bust. | Example gymnasium-style multi-step environment | - | - | - | blackjack.yaml | - | | Browsecomp Advanced Harness | agent | Model uses search tools to satisfy a user query. | Measure agentic search capability | - | - | - | browsecomp_advanced_harness.yaml | - | | Bunsenbench Chemistry Mcq | knowledge | Public BunsenBench chemistry multiple-choice benchmark verifier | Measure chemistry MCQ reasoning with source and taxonomy breakdowns | - | - | - | bunsenbench_chemistry_mcq.yaml | - | | Calendar | agent | Multi-turn calendar scheduling dataset. User states events and constraints in natural language; model schedules events to satisfy all constraints. | Improve multi-turn instruction following capabilities | ✓ | ✓ | Apache 2.0 | calendar.yaml | Nemotron-RL-agent-calendar_scheduling | | Calendar | agent | Multi-turn calendar scheduling dataset. User states events and constraints in natural language; model schedules events to satisfy all constraints. | Improve multi-turn instruction following capabilities | ✓ | ✓ | Creative Commons Attribution 4.0 International | calendar_v2.yaml | Nemotron-RL-Instruction-Following-Calendar-v2 | | Circle Click | other | Click on circles in images | Improve visual grounding and spatial reasoning | - | - | - | circle_click.yaml | - | | Circle Count | other | Count circles of a given color in images | Improve visual counting and color recognition | - | - | - | circle_count.yaml | - | | Citation If | instruction_following | Citation instruction-following reward checker | Train models to follow citation format instructions in search-grounded synthesis tasks | - | - | - | citation_if.yaml | - | | Code Fim | coding | Code Fill-in-the-Middle judged by HumanEval-Infilling test suite (single_line, multi_line, random_span, random_span_light) | Improve Python code-infilling capabilities (prefix + completion + suffix) | - | - | - | code_fim.yaml | -

... (README truncated for length)