Profile
Back to NewsBack
GitHub Trending 8 min
Reader Mode
Junchao-cs/SolarWM: Open data and scalable training for long-horizon video world models.

Junchao-cs/SolarWM: Open data and scalable training for long-horizon video world models.

15 hours ago

SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

Junchao Huang1,2   Guian Fang3   Shengju Qian4   Xianghao Kong5   Zhuoran Zhao5,6
Wei Huang7   Yihua Du6   Zixin Zhang6   Justin Cui8   Yuchao Gu7   Yukang Chen7   Xinting Hu
Tianyu He9   Shaoshuai Shi   Zhuotao Tian2   Xin Wang   Mike Zheng Shou3   Li Jiang1,2
1CUHK-SZ   2SLAI   3NUS   4CUHK   5HKUST   6HKUST-GZ   7NVIDIA   8UCLA   9MSRA

Project page Paper Hugging Face dataset ModelScope dataset Dataset Access Form Model weights Apache 2.0 license

SolarWM teaser: one framework for diverse interactive worlds

We present SolarWM, a fully open, end-to-end framework for building interactive video world models, from data preparation to scalable training and long-horizon inference.

  • Four models. One codebase. A unified framework supports **four
5B–33B models** across Wan2.2, LTX-2.5, and MiniMax-H3, with shared data, training, and inference interfaces while preserving each backbone's native representation and objective.
  • A simple three-stage training recipe. Bidirectional adaptation,
teacher-forced autoregressive initialization, and distribution matching distillation form a shared route across heterogeneous video backbones, without specialized ODE or consistency-distillation initialization. The resulting causal models support *few-step interaction over arbitrarily long horizons** through sliding-window inference.
  • 1.43M clips · 25 TB. Fully open. We release the **complete dataset
and data-processing pipeline**, unifying 14 datasets into a frame-aligned format with metric camera geometry, captions, quality metadata, and provenance. Source processing is decoupled from training-mixture design, allowing flexible data selection and mixing without reprocessing the source datasets.

News

  • September 13, 2026: We release SolarWM-H3, the first MiniMax-H3-based world model for streaming, few-step interaction. Training and inference code, weights, and pre-encoded training data are now available for all three stages: Stage0.5, Stage1, and Stage2. Get started with the H3 guide.
  • September 3, 2026 — We open-source the training and inference code, the
complete dataset, the data pipeline, model weights for all SolarWM-5B training stages, and bidirectional weights for SolarWM-14B, SolarWM-LTX, and SolarWM-H3.

Training progression

The staged workflow is **Stage0.5 FM → Stage1 TF-AnyFlow → Stage2 DMD via SGF**, turning a bidirectional video model into a camera-controlled few-step autoregressive model.

  • Stage0.5 learns full-clip bidirectional flow matching and establishes the
base video, text, and camera-conditioned representation.
  • Stage1 combines teacher forcing with the AnyFlow loss in one training
stage. Clean history conditions noisy target chunks while the model learns both denoising and finite-step flow maps. This removes the need for a separate ODE or consistency-distillation initialization before Stage2/DMD.
  • Stage2 performs DMD via self-gradient forcing (SGF), training the causal
student on its own autoregressive rollout with a frozen teacher and a trainable critic.

| Backend | Stage0.5 (Bid-Cam) | Stage1 (TF-AnyFlow) | Stage2 (SGF) | Runtime interfaces | |---|:---:|:---:|:---:|---| | Wan2.2-5B | ✓ | ✓ | ✓ | train, infer, preencode | | Wan2.2-14B | ✓ | Coming soon | Coming soon | train, infer, preencode | | LTX-2.5 | ✓ | Coming soon | Coming soon | train, infer, preencode | | MiniMax-H3 | ✓ | ✓ | ✓ | train, infer, preencode |

Install

Wan, LTX, and MiniMax-H3 require separate runtime environments. Activate the environment for the selected backbone, then install the shared SolarWM source:

python -m pip install -e .
solarwm environment probe

Model weights are available from the SolarWM model collection. See the Wan2.2 TI2V-5B guide for Stage0.5, Stage1, and Stage2 commands. The MiniMax-H3 guide provides launch commands for Stage0.5, Stage1, and Stage2.

Data

The public SolarWM-Data release contains release controls, licenses, recipe and test indexes, small format examples, and the SolarWM-Data-Annotation/ package. It does not include the full releases-v1/raw-wds/ or releases-v1/latent-wds/ payloads.

For a released recipe that uses preencoded data, download its matching latent generation. That is sufficient for training and does not require raw-WDS. Raw-WDS is needed only when you want the full processed video corpus, online encoding, your own latent generation, or another workflow whose index points to raw data.

  1. Use preencoded latents. Each latent generation is published in a
separate repository. See the latent-WDS release list for available downloads and generations that are still being uploaded.
  1. Rebuild raw-WDS from annotations. SolarWM-Data-Annotation/ is an
annotation-only release with no videos. It contains the released camera trajectories, captions, metadata, source identities, and reconstruction tools. Follow its README to download the original videos from their source publishers and process them into the expected raw-wds/ layout.
  1. Request prepared raw-WDS. Submit the
Dataset Access Form. Approved applicants receive download instructions by email.
  1. Download the standalone test set. The
SolarWM standalone test set v1 contains the current 1,300-example, 13-source evaluation set in 60 WebDataset tar files (approximately 77.4 GB), without the training payload.

See the dataset access guide for download commands and the payload required by each training, validation, and inference example.

Unified commands

# Validate and render the exact resolved configuration.
solarwm config resolve \
  --config configs/examples/minimax_h3/stage0p5-158f-lora384-sp2.yaml \
  --set distributed.world_size=8 \
  --set train.global_batch_size=4 \
  --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \
  --set data.index_root=/path/to/SolarWM-Data/releases-v1 \
  --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \
  --set data.silence_latents_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/h3_silence_153_158_170.safetensors \
  --set data.encoder_contract_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/encoder_contract.json \
  --set runtime.output_dir=/path/to/output/h3-stage0p5-158f

Train H3 Stage0.5 with preencoded latents.

torchrun --standalone --nproc-per-node=8 \ -m solarwm train \ --config configs/examples/minimax_h3/stage0p5-158f-lora384-sp2.yaml \ --set distributed.world_size=8 \ --set train.global_batch_size=4 \ --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \ --set data.index_root=/path/to/SolarWM-Data/releases-v1 \ --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \ --set data.silence_latents_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/h3_silence_153_158_170.safetensors \ --set data.encoder_contract_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/encoder_contract.json \ --set runtime.output_dir=/path/to/output/h3-stage0p5-158f

Run H3 Stage2 inference for 158 frames.

torchrun --standalone --nproc-per-node=8 -m solarwm infer \ --config configs/examples/minimax_h3/infer-stage2-158f-sp4.yaml \ --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \ --set checkpoint.resume_from=/path/to/SolarWM-models/SolarWM-h3-33B-sgf-stage2-158f \ --set checkpoint.weight_source=ema \ --set data.index_root=/path/to/SolarWM-Data/releases-v1 \ --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \ --set data.silence_latents_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/h3_silence_153_158_170.safetensors \ --set data.encoder_contract_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/encoder_contract.json \ --set runtime.output_dir=/path/to/output/h3-stage2-infer

Preencode raw-WDS for the H3 158f recipe.

torchrun --standalone --nproc-per-node=8 \ -m solarwm preencode \ --config configs/examples/minimax_h3/preencode-158f.yaml \ --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \ --set data.index_root=/path/to/SolarWM-Data/releases-v1 \ --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \ --set preencode.output_root=/path/to/output/preencoded/minimax-h3-158f-768p-nomind-v1 \ --set runtime.output_dir=/path/to/output/h3-preencode

The quickstart covers H3 setup, training, and inference. For Stage1/Stage2 training and full-length inference, see the H3 guide. Other models have their own backend guides.

Every launch writes resolved-config.json and launch-manifest.json before model allocation. Config overrides are explicit and included in the resolved configuration identity.

Local and bucket data

Index rows always contain POSIX shard keys relative to the release directory:

{"sample_id":"...","shard":"raw-wds/abot/shards/kept-high-000001.tar"}

Only the runtime root changes:

# Locally mounted storage
data:
  index_root: /path/to/SolarWM-Data/releases-v1
  transport:
    kind: local
    root: /path/to/SolarWM-Data/releases-v1

Object-store streaming uses the same release-relative rows. Supply the

release root from the distribution channel or deployment environment.

data: index_root: /path/to/SolarWM-Data/releases-v1 transport: kind: gcs root: ${SOLAR_RELEASE_ROOT} cache_dir: /path/to/solar-cache cache_max_gib: 256

Documentation

Acknowledgements

We gratefully thank the teams behind Wan2.2, LTX-2.5, and MiniMax-H3 for releasing the code and pretrained models that make the SolarWM backbone family possible. We also thank the authors of all datasets and open-source projects listed in NOTICE.

Citation

**If you use SolarWM-Data, the data engine, or the released models in your research, please cite our paper.**

Paper: https://arxiv.org/abs/2609.02886

@misc{huang2026solarwmopendatascalable,
      title={SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models}, 
      author={Junchao Huang and Guian Fang and Shengju Qian and Xianghao Kong and Zhuoran Zhao and Wei Huang and Yihua Du and Zixin Zhang and Justin Cui and Yuchao Gu and Yukang Chen and Xinting Hu and Tianyu He and Shaoshuai Shi and Zhuotao Tian and Xin Wang and Mike Zheng Shou and Li Jiang},
      year={2026},
      eprint={2609.02886},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.02886}, 
}

License and attribution

SolarWM is licensed under Apache-2.0. See LICENSE and NOTICE. Model weights and backbone packages may carry their own licenses; review the license and model card in the corresponding release package before use or redistribution. In particular, LTX-2.5 derivatives are subject to the LTX-2.x Community License, and the MiniMax H3 Community License contains territory restrictions. Those packages are not relicensed under the SolarWM code license.

Chat with me