News
- 🎥🎥🎥 [2026-11-22] Video inference now uses YOLO26m for person detection and Pi4 for camera pose and intrinsics estimation by default.
- 🎉🎉🎉 [2026-10-31] Our Hugging Face demo is now live!
- 🚀🚀🚀 [2026-10-30] Updated inference acceleration: approximately 10× faster SAM feature extraction
How to Use FlowHMR
- 💻 CLI — Process a video or a folder of videos from the terminal. Inference guide · Model checkpoints
- 🌐 Web (local) — Run the web app on your machine to select a person in a video and preview their 3D motion. Web demo guide
- 🤗 Hugging Face — Upload a video and preview its motion in your browser without a local installation. Open the hosted demo
- 🧩 Windows extensions — Thanks to @toyxyz for toyxyz_RGBMocap and @xiaoerofficial021-collab for FlowHMR-Windows.
Model Zoo
| Model | Description | Params | Tracking Success ↑ | Download |
|---|---|:---:|:---:|:---:|
| flowhmr_base | Base model | 0.46B | 78.79% | 🤗 HF |
| flowhmr_latest | flowhmr_base + GRPO with PHC+ physics reward (recommended) | 0.46B | 82.47% | 🤗 HF |
Tracking Success: the rate at which the PHC+ controller successfully tracks the predicted motion in simulation, measured on Wild-4K.
Getting Started
Follow the guide for each step, or paste the prompt into your coding agent.
Installation
FlowHMR supports two setups:
- V2M: inference, web demo, and base model training in a single environment
- Full pipeline: V2M plus the PHC+ physics simulator for GRPO post-training and
/dev/shm.
The default video setup uses YOLO26m and Pi4 and requires access to SAM 3D Body and the SMPL-H body model. VGGT-Omega requires separate access when selected.
V2M:
Install FlowHMR from https://github.com/flowhmr/flowhmr following docs/install.md with the V2M setup.
Full pipeline:
Install FlowHMR from https://github.com/flowhmr/flowhmr following docs/install.md with the full pipeline setup.
Web Demo
Upload a video in the browser and reconstruct its 3D motion; when several people are detected, pick the one to reconstruct. Requires the V2M setup.
Launch the FlowHMR web demo following docs/web_demo.md.
Inference
Reconstruct motion from a video on the command line. The pipeline runs person detection
(YOLO26m or YOLOX-L), camera estimation (Pi4, VGGT-Omega, SimpleVO, or a fixed pose), body feature extraction (SAM 3D Body), and
FlowHMR, and saves the motion as .npz (plus .fbx if FBX export is installed).
New runs use YOLO26m for person detection and Pi4 to estimate camera poses and intrinsics.
Use --detector yolox for YOLOX-L or --camera vggt for VGGT-Omega.
If camera estimation runs out of GPU memory, select SimpleVO (--camera simplevo) to estimate camera rotation on the CPU.
Run FlowHMR on my video following docs/inference.md.
Data
- Training data is not bundled. Bring your own in the
- Wild-4K is the 4182-clip in-the-wild evaluation set behind Tracking Success.
Training
Training has two stages: pretraining the base model, then GRPO post-training with a physics reward. Prepare training data first (see Data and Prepare data).
Pretraining (V2M setup) trains the flow-matching base model, flowhmr_base.
Train the FlowHMR base model following docs/training.md.
GRPO post-training (full pipeline setup) fine-tunes flowhmr_base with the PHC+
simulator as the reward: each generated motion is scored by how well the PHC+ controller
tracks it. This produces flowhmr_latest.
Run FlowHMR GRPO post-training following docs/training.md.
Acknowledgements
We thank the authors of these projects for releasing their code and models:
- HY-Motion 1.0: MMDiT flow-matching backbone
- MixGRPO: GRPO post-training for flow matching
- PHC: PHC+ controller for physics reward and tracking evaluation
- GVHMR: world-grounded human motion recovery
- VGGT-Omega: camera estimation
- SAM 3D Body: body features
- YOLOX: person detection
- Kimodo and toyxyz_RGBMocap: motion and contact correction
Citation
If you find FlowHMR useful in your research, please cite:
@article{wang2026flowhmr,
title={FlowHMR: Physically Plausible Motion Capture from Video},
author={Wang, Zhanke and Zhao, Chengfeng and Shuai, Qing and Lin, Jingzhong and Li, Heng and Ling, Zeyu and Wen, Yuxin and Li, Jing and Kang, Di and Guo, Chunchao and Bao, Linchao},
journal={arXiv preprint arXiv:2610.03691},
year={2026}
}
License
The FlowHMR source code is licensed under the Apache License 2.0. Third-party libraries and components are subject to their respective licenses. For model weights, please refer to the license specified in the model card.