A Comprehensive Framework for Visual SLAM Baselines and Datasets
Alejandro Fontan · Tobias Fischer · Nicolas Marticorena
Somayeh Hussaini · Ted Vanderfeen · Beverley Gorry · Javier Civera · Michael Milford
!
Introduction
VSLAM-LAB is designed to simplify the development, evaluation, and application of Visual SLAM (VSLAM) systems. This framework enables users to compile and configure VSLAM systems, download and process datasets, and design, run, and evaluate experiments — all from a single command line!
Why Use VSLAM-LAB?
- Unified Framework: Streamlines the management of VSLAM systems and datasets.
- Ease of Use: Run experiments with minimal configuration and single command executions.
- Broad Compatibility: Supports a wide range of VSLAM systems and datasets.
- Reproducible Results: Standardized methods for evaluating and analyzing results.
Getting Started
To ensure all dependencies are installed in a reproducible manner, we use the package management tool pixi. If you haven't installed pixi yet, please run the following command in your terminal:
curl -fsSL https://pixi.sh/install.sh | bashIf you already have pixi remember to update: pixi self-update
Clone the repository and navigate to the project directory:
git clone https://github.com/VSLAM-LAB/VSLAM-LAB.git && cd VSLAM-LABQuick Demo
You can now execute any baseline on any sequence from any dataset within VSLAM-LAB using the following command:pixi run demo <baseline> <dataset> <sequence> <mode>pixi run demo mast3rslam eth table_3 mono
pixi run demo droidslam rgbdtum rgbd_dataset_freiburg1_xyz rgbd
pixi run demo orbslam2 kitti 04 stereo
pixi run demo pycuvslam euroc MH_01_easy stereo-vipixi run set-benchmark-path /media/${USER}/data
pixi run set-evaluation-path /media/${USER}/dataConfigure your own experiments
With VSLAM-LAB, you can easily design and configure experiments using a YAML file and run them with a single command. To run the experiment demo, execute the following command:pixi run vslamlab configs/exp_vslamlab.yaml # resumes: runs already done are kept
pixi run vslamlab configs/exp_vslamlab.yaml --overwrite # deletes the previous results first
VSLAMLAB_HEADLESS=1 pixi run vslamlab configs/exp_vslamlab.yaml # unattended: figures are saved, no plot windows openExperiments in VSLAM-LAB are sequences of entries in a YAML file (see example configs/exp_vslamlab.yaml):
exp_demo_droidslam:
Config: config_vslamlab.yaml # YAML file containing the sequences to be run
NumRuns: 1 # Maximum number of executions per sequence
Parameters: {verbose: 1, rgb_idx: [0,250]} # Vector with parameters that will be input to the baseline executable
Module: droidslam # droidslam/monogs/orbslam2/mast3rslam/dpvo/...
exp_demo_orbslam2:
Config: config_vslamlab.yaml
NumRuns: 1
Parameters: {verbose: 1, rgb_idx: [0,250]}
Module: orbslam2
exp_demo_colmap:
Config: config_vslamlab.yaml
NumRuns: 1
Parameters: {verbose: 1, rgb_idx: [0,250], matcher_type: exhaustive, matching_type: aliked_lightglue} # baseline-specific keys: see Baselines/colmap/README.md
Module: colmap
Parameters also accepts four optional flags to control which RGB frames of a sequence are fed to the baseline. They are independent and stack in this order — rgb_idx slices the frame range first, rgb_step then keeps 1 frame every n of what remains, rgb_max truncates that result to at most that many frames, and rgb_vpr finally downsamples to at most that many frames by visual dissimilarity (VPR) rather than a fixed stride, so visually redundant frames are dropped preferentially. If none are set, every frame of the sequence is used.
| Parameter | Effect |
|:----------|:-------|
| rgb_idx: [start, end] | Keep only frames start..end (inclusive) |
| rgb_step: n | Keep 1 frame every n frames of what remains |
| rgb_max: n | Truncate the result to at most n frames |
| rgb_vpr: n | Downsample the result to at most n frames, chosen by VPR dissimilarity - computes the sequence's VPR distance matrix via pixi run vpr first if it isn't already cached |
exp_vslamlab:
Config: config_vslamlab.yaml
NumRuns: 1
Parameters: {verbose: 1, rgb_idx: [0, 250], rgb_step: 3, rgb_max: 50, rgb_vpr: 30}
Module: droidslameth:
- table_3
rgbdtum:
- 'rgbd_dataset_freiburg1_xyz'
VSLAM-LAB Pipeline Commands
In addition to running the full automated pipeline, VSLAM-LAB provides modular commands to interact directly with datasets and baselines. For a comprehensive list of all available commands consult Wiki: Command-line Interfacepixi run install-baseline <baseline> # Example: pixi run install-baseline droidslam
pixi run download-sequence <dataset> <sequence> # Example: pixi run download-sequence eth table_3
pixi run run-exp <exp_yaml> # Example: pixi run run-exp configs/exp_vslamlab.yaml
pixi run evaluate <exp_yaml> # Example: pixi run evaluate configs/exp_vslamlab.yaml
pixi run compare <exp_yaml> # Example: pixi run compare configs/exp_vslamlab.yaml
pixi run rerun-vis <dataset> <sequence> # Example: pixi run rerun-vis eth table_3 (Rerun viewer of a sequence)Run a VSLAM system online from an iPhone
The ARStream app streams RGB, intrinsics and ARKit's pose from an iPhone over USB or Wi-Fi.record-iphone turns a recording into a sequence of the iphone dataset (see the Tools table); stream-iphone relays the live stream over ZeroMQ so any number of subscribers (a baseline's execute-mono-stream, the viewer, your own Python/C++ code) consume it at once:
pixi run stream-iphone --run orbslam3-dev # relay + ORB-SLAM3-DEV (or orbslam2-dev) online in one go: Ctrl-C stops the stream, the baseline saves ../VSLAM-LAB-Evaluation/stream_orbslam3-dev/00000_KeyFrameTrajectory.csv
pixi run stream-iphone --run orbslam3-dev --view # ... plus the Rerun viewer as another subscriber (--duration 30, --verbose 0, --exp-folder <dir> are also available)
pixi run stream-iphone --run orbslam2 --mode rgbd # LiDAR phone: RGB-D online with ARKit's depth (orbslam2/orbslam3 and their -dev variants), results in ../VSLAM-LAB-Evaluation/stream_orbslam2_rgbd
pixi run stream-iphone # relay only: phone over USB (or --wifi / --host <ip>) -> tcp://*:5556, writes ../VSLAM-LAB-Benchmark/IPHONE/stream/calibration.yaml from the first frame
pixi run view-iphone --sub # the Rerun viewer as a subscriber
pixi run execute-mono-stream stream_url:tcp://localhost:5556 calibration_yaml:<abs path> exp_folder:<abs dir> settings_yaml:vslamlab_orbslam3-dev_settings.yaml # a baseline's stream entry point by hand (runs inside Baselines/ORB-SLAM3-DEV)KeyFrameTrajectory.csv as usual. A baseline joins --run by providing an execute--stream entry point taking stream_url: instead of sequence_path:/rgb_csv: (see BaselineVSLAMLAB.build_stream_command). Writing a subscriber: arstream.Subscriber (Python) or arstream/arstream.hpp (C++, conda package libarstream), see the ARStream README.
Add a new VSLAM Dataset
Expand the evaluation suite by integrating custom datasets. If you're using Claude Code, run the add-dataset skill (/add-dataset ) to walk through the full workflow end to end. Otherwise, follow the instructions in Wiki: Integrate a new VSLAM Dataset.
Add a new VSLAM Baseline
Incorporate new algorithms into the framework. Follow the guide in Wiki: Integrate a new VSLAM Baseline. Benchmark your method against state-of-the-art baselines across all supported datasets.
For a reference implementation, see the VGGT-SLAM integration in commit 259f7ae.
License
VSLAM-LAB is released under a LICENSE.txt. For a list of code dependencies which are not property of the authors of VSLAM-LAB, please check docs/Dependencies.md.Citation
If you're using VSLAM-LAB in your research, please cite:@INPROCEEDINGS{fontan2025vslam,
author={Fontan, Alejandro and Fischer, Tobias and Civera, Javier and Milford, Michael},
booktitle={2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
title={VSLAM-LAB: A Comprehensive Framework for Visual SLAM Methods and Datasets},
year={2025},
volume={},
number={},
pages={9013-9020},
keywords={Visualization;Simultaneous localization and mapping;Robot vision systems;Standardization;Benchmark testing;Programming;Reproducibility of results;Intelligent robots;Faces;Software development management},
doi={10.1109/IROS60139.2025.11247218}
}VSLAM-LAB Supported Baselines and Datasets
For full per-dataset details — license, access requirements, and publication/BibTeX citation — seeDatasets/extra-files/dataset_table.xlsx.
| Baselines | System | Modes | License | Label | Conda Pkg | Camera Models |
|:----------------------------------------------------------------------------|:------:|:------:|:----------------:|:---------------------------------------------------------------------------:|:------------:|:------------:|
| AllFeature-VSLAM | VSLAM | mono rgbd | GPLv3 | allfeature | ✅ | pinhole radtan4 radtan5 |
| VGGT-SLAM 2.0 | VSLAM | mono | BSD-2 | vggtslam | ✅ | pinhole |
| MASt3R-SLAM | VSLAM | mono | CC BY-NC-SA 4.0 | mast3rslam | ✅ | pinhole radtan4 radtan5 |
| DPVO | VSLAM | mono | License | dpvo | ✅ | pinhole radtan4 radtan5 |
| DROID-SLAM | VSLAM |mono rgbd stereo| BSD-3 | droidslam | ✅ | pinhole radtan4 radtan5 |
| ORB-SLAM2 | VSLAM |mono rgbd stereo| GPLv3| orbslam2 | ✅ | pinhole radtan4 radtan5 |
| MonoGS | VSLAM | mono rgbd | License | monogs-dev | ➖ | pinhole radtan4 radtan5 |
| ---------- | ------- | ------- | ---------- | -------- | --- | ---------- |
| PyCuVSLAM | VSLAM |rgbd stereo(-vi) | NVIDIA | pycuvslam | ➖ | pinhole radtan4 radtan5 equid4 |
| ORB-SLAM3 | VSLAM | mono(-vi) rgbd(-vi) stereo(-vi) | GPLv3 | orbslam3 | ✅ | pinhole radtan4 radtan5 equid4 |
| OKVIS2 | VSLAM | mono-vi | BSD-3 | okvis2 | ✅ | pinhole radtan4 radtan5 radtan8 equid4 |
| ---------- | ------- | ------- | ---------- | -------- | --- | ---------- |
| COLMAP matching_type: SIFT · ALIKED · LoMa features with brute-force or learned matchers · mapper_type: colmap · glomap · dense: 1 with mesher: delaunay · poisson · advancing_front — reference | SfM | mono | BSD | colmap | ✅ | pinhole radtan4 radtan5 radtan8 equid4 unknown |
| Depth Anything 3 | Feed-forward | mono | Apache-2.0 (giant/nested weights CC BY-NC 4.0) | da3 | ➖ | pinhole |
| DA3-Streaming | VSLAM | mono | Apache-2.0 | da3-streaming | ➖ | pinhole |
| MapAnything module: VGGT · VGGT-Ω · π³ · DUSt3R · MASt3R · MUSt3R · Pow3R (mono, each under its own license) | Feed-forward | mono rgbd | Apache-2.0 (default weights CC BY-NC 4.0) | mapanything | ➖ | pinhole |
[!NOTE]
-devvariants:orbslam2-dev,orbslam3-dev,droidslam-dev,dpvo-dev,mast3rslam-dev,vggtslam-dev,allfeature-dev,okvis2-devandmonogs-devclone the baseline's source intoBaselines/and build it locally instead of installing the conda package, with the same modes and camera models - use them to modify a baseline.-DEV/ monogs-devexists only in this form.
| Tools | Features | Label | Modes | Camera Models |
|:--------------------------------------------------------------------------------------------------------------------------------|:---------:|:-----------:|:----------:|:----------:|
| Stray Scanner App | 📸🤳 | strayscanner | mono rgbd | pinhole |
| Pupil Labs Neon | 📸🥽 | pupil-labs | mono(-vi) | pinhole |
| Local video files | 📸 | videos | mono | pinhole unknown |
| YouTube videos
add one with the add-youtube-video skill | 📸 | youtube | mono | pinhole radtan4 unknown |
| iPhone (ARStream app)
record one with pixi run record-iphone [ (USB or --wifi); ARKit's VIO pose as groundtruth | 📸🤳 | iphone | mono | pinhole |
| iPhone with LiDAR (ARStream app) pixi run record-iphone --lidar [; ARKit's sceneDepth (256×192, confidence-masked, upsampled to the RGB size) as depth_0 | 📸🤳 | iphone-lidar | mono rgbd | pinhole |
Real / Synthetic : 📸 / 💻
Indoor / Outdoor / Underwater / Intracorporeal : 🏠 / 🏞️ / 🌊 / 🫀
Handheld / Headmounted / Vehicle / UAV / Robot : 🤳 / 🥽 / 🚗 / 🚁 / 🤖
Section dividers only (not per-row features) — Construction Site / Planetary-analog Terrain : 🏗️ / 🪐