gaoypeng/3dcodeverse: Official Codebase of 3DCodeVerse

Exploring 3D Worlds as Code

Yipeng Gao^1†, Ziyao Zeng², Jinfa Huang³, Yijiang Li⁴
Haolin Xiong¹, Wang Qin¹, Xinlei Yu¹, Youheng Yao², Jie Zhang⁵, Kun-Yu Lin⁶, Rong Liu¹, Yinyuan Zhao⁷, Yajie Zhao¹
Meiqi Guo⁸, Jiebo Luo³, Lei Shu⁸, Yunhao Ge¹, Xinyu Hu⁹, Laurent Itti¹

¹University of Southern California · ²Yale University · ³University of Rochester
⁴University of California, San Diego · ⁵Columbia University · ⁶Nanyang Technological University
⁷University of California, Santa Cruz · ⁸Google DeepMind · ⁹Philo Labs
^*Equal contribution · ^†Project lead

Awesome Showcase

Films rebuilt entirely in code, with full source and agent logs on Hugging Face.

| Code | Reference | Result | | :-- | :-: | :-: | | link |
OpenAI on X |
Opus 5.5 by Code |

Overview

https://github.com/user-attachments/assets/3b0b5e5f-eed0-4ffd-8ab7-f3c709e15fad

3DCodeVerse Harness

An LLM planner defines the target asset, and a DAG of sub-agents codes and assembles a 3D program. Every round, the harness collects three kinds of feedback:

1. Execution — build and runtime logs.
2. Measurement — dimensions, contacts, collisions, kinematics and placement, reported per part

1. Vision — a rubric-guided VLM judge on labelled multi-view renders.

3DCodeVerse-2M Dataset

News

• [10/15/2026] Our paper is now available on arXiv.

Results

Fine-tuning on 3DCodeVerse-2M makes open models execute. The table reports execution rate (%) on 805 fixed tasks:

| Model | 3DCodeBench | Blender | CadQuery | OpenSCAD | GLSL | Three.js | | :-- | :-: | :-: | :-: | :-: | :-: | :-: | | Qwen3.5-9B | 0.0 | 1.9 | 10.5 | 50.0 | 33.0 | 72.5 | |   + SFT | 84.9 | 90.3 | 98.5 | 86.0 | 56.0 | 74.5 | | Qwen3.8-27B | 51.9 | 74.8 | 65.5 | 80.0 | 57.0 | 72.5 | |   + SFT + DPO | 92.9 | 93.2 | 98.5 | 90.0 | 71.5 | 92.5 |

Quick Start

The harness ships as 3dcodeverse, a plugin for Claude Code, Codex and Antigravity: the host agent writes the code with its own model, and the plugin's skills and 3dcode instruments build, measure, render, gate and score it (a fresh-context subagent judges). It needs Python 3.13, Node 20.6+ and Git; Blender 5.2 LTS for the blender, urdf_blender and scene_blender languages. Full guide: harness/README.md and harness/docs/INSTALL.md.

git clone https://github.com/gaoypeng/3dcodeverse && cd 3dcodeverse
bash harness/setup.sh                                  # the 3dcode instruments + runtime_js + headless Chrome, then doctor
claude plugin marketplace add . && claude plugin install 3dcodeverse@3dcodeverse   # Claude Code
codex plugin marketplace add . && codex plugin add 3dcodeverse@3dcodeverse         # Codex
agy plugin install harness                                                         # Antigravity

Then ask your agent for a 3D thing, e.g. "Make a mid-century wooden dining chair in Blender bpy."

Repository Structure

| Directory | Description | Documentation | | :-- | :-- | :-- | | harness/ | The 3dcodeverse agent plugin: skills, judge subagent, and the 3dcode build / measure / render / gate / score instruments. | Plugin guide | | flywheel/ | Reader for runs recorded by the v0.0.x CLI harness: gallery, cost, dataset export, pick. | Flywheel guide | | toolkits/ | General data CLI and per-source corpus curation. | Data toolkit guide | | finetune/ | LLaMA-Factory training-data preparation, supervised fine-tuning, and preference optimization for open models. | Training guide | | eval/ | Evaluation of standalone LLMs / VLMs on 3D coding. | Evaluation guide |

Citation

If you find 3DCodeVerse useful, please cite:

@article{gao20263dcodeverse,
  title={3DCodeVerse: Exploring 3D Worlds as Code},
  author={Gao, Yipeng and Zeng, Ziyao and Huang, Jinfa and Li, Yijiang and Xiong, Haolin and Qin, Wang and Yu, Xinlei and Yao, Youheng and Zhang, Jie and Lin, Kun-Yu and others},
  journal={arXiv preprint arXiv:2610.22341},
  year={2026}
}

License

The code is licensed under Apache-2.0. The published datasets carry their own terms (CC-BY-NC-SA, non-commercial); see each dataset card.