Michelangelo AI
Michelangelo AI is an open-source platform designed to streamline the development, deployment, and monitoring of machine learning models at scale. It offers a comprehensive suite of tools and services that facilitate the entire machine learning lifecycle, from data management to model serving.
:warning: Beta Notice This project is currently in beta. APIs and features may evolve, and breaking changes may occur as we continue to improve and stabilize the platform.
Open Source Initiative
As part of our commitment to the ML community, we are open-sourcing an end-to-end lifecycle management system grounded in extensive operational expertise. Our goals are to:
- Drive standardization and interoperability across the ML ecosystem,
- Enable easy adoption of scalable ML solutions in new production use cases,
- Foster innovation and trust through collaboration with partner teams, and
- Cultivate a vibrant and responsible ML culture that empowers the community to build with confidence and speed.
Features
- Feature Management: Efficiently handle large datasets with built-in support for data ingestion, transformation, and storage.
- Model Training: Train models using various algorithms, including support for distributed training across multiple nodes. The
michelangelo.lib.trainer.torch.pytorch_lightningpackage provides a Ray Train wrapper around PyTorch Lightning, with pluggable experiment tracking (Comet, MLflow). See the MovieLens NCF example for an end-to-end walkthrough. - Model Evaluation: Assess model performance with a range of metrics and visualization tools.
- Model Deployment: Seamlessly deploy models to production environments with support for both batch and real-time inference.
- Monitoring and Logging: Continuously monitor model performance and log predictions to ensure reliability and accuracy.
Installation
Follow the Sandbox Setup guide to get a fully functional local environment running.
Usage
Here's a quick example of how to define and run an ML pipeline:
# Clone the repo and install dependencies
git clone https://github.com/michelangelo-ai/michelangelo.git
cd michelangelo/python
poetry install
source .venv/bin/activate
Build the local-only images the sandbox needs
cd .. && bash scripts/kuberay/build-kuberay-images.sh
Spin up a local sandbox cluster
ma sandbox create
Run the demo pipeline to verify everything works
ma sandbox demo pipelineTo define your own pipeline, use the @task and @workflow decorators:
import michelangelo.uniflow.core as uniflow
@uniflow.task()
def train(learning_rate: float = 0.01) -> str:
# your training logic here
return "model_path"
@uniflow.workflow()
def my_pipeline(learning_rate: float = 0.01):
model = train(learning_rate=learning_rate)
See the Sandbox Setup guide for prerequisites (including docker buildx) and platform-specific notes before running the above.
For a full walkthrough, see the Getting Started with ML Pipelines guide.
Build and Test
See the User Guides in the documentation for instructions on running tests and working with the development environment.
Consuming and Using the Containers
See the Sandbox Setup guide for instructions on running and importing container images into your local cluster.
Contributing
We welcome contributions to Michelangelo AI! If you're interested in contributing, please read our Contributing Guidelines to get started.
License
This project is licensed under the Apache 2.0 License.
Acknowledgments
Thank you to the Michelangelo AI Open Source team for getting this project off the ground, and thank you in advance to our contributors.