Open Source MLOps Tools
MLOps brings machine learning development and operations together to make models easier to build, deploy, monitor, and maintain. It addresses challenges such as inconsistent experiments, fragile data and model pipelines, unreliable releases, and changes in model performance over time. Practices and tools in this area help teams make workflows repeatable and manage models throughout their production lifecycle.
Open source MLOps tools include workflow orchestrators, experiment and model tracking systems, deployment and serving frameworks, and tools for monitoring data quality and model behavior. When choosing, consider project maturity, license, maintenance activity, infrastructure requirements, and integration with your existing data and machine learning stack. These tools are useful to data scientists, machine learning engineers, platform teams, and organizations operating models in production.
8 repositories · updated September 12, 2026
Awesome Automated AI/ML: Your Curated Guide to AI/ML Automation Tools
Awesome Automated AI/ML is a comprehensive, curated list featuring over 300 tools for automating various aspects of AI and Machine Learning. It covers everything from hyperparameter optimization to autonomous AI agents, offering a dynamic resource for ML engineers, AI researchers, and product builders.
LangTest: A Comprehensive Library for Safe & Effective Language Models
LangTest is an open-source Python library dedicated to ensuring the safety and effectiveness of language models. It offers a comprehensive framework for testing model quality, covering robustness, bias, fairness, and accuracy across various NLP tasks and LLM providers. With LangTest, developers can generate and execute over 60 distinct test types with just one line of code, promoting responsible AI development.
Evidently: Open-Source ML and LLM Observability Framework
Evidently is an open-source Python library designed for evaluating, testing, and monitoring machine learning and large language model systems. It provides over 100 built-in metrics for various tasks, from data drift detection to LLM judges, supporting both tabular and text data. This framework helps ensure the quality and performance of AI-powered systems throughout their lifecycle.
Dagster: An Orchestration Platform for Data Assets
Dagster is a powerful open-source orchestration platform designed for the development, production, and observation of data assets. It provides a unified programming model for building and managing data pipelines, making it easier to define, test, and deploy complex data workflows. This platform supports various data engineering, analytics, and machine learning operations.
Giskard-OSS: Open-Source Evaluation & Testing Library for LLM Agents
Giskard-OSS is an open-source Python library designed for evaluating and testing AI systems, particularly LLM-based applications and traditional ML models. It automatically detects performance, bias, and security issues, offering comprehensive tools for ensuring the reliability and safety of AI. The library includes a powerful RAG Evaluation Toolkit (RAGET) for in-depth assessment of Retrieval Augmented Generation applications.
handit.ai: Your AI Teammate for Reliable Production AI
handit.ai is an open-source AI teammate designed to ensure the reliability of your AI applications in production. It automatically detects issues like hallucinations and schema breaks, generates and tests fixes, and ships them as pull requests. This tool eliminates 2 AM debugging sessions, making AI truly dependable.
Flyte: Scalable Workflow Orchestration for Data and ML
Flyte is an open-source, scalable, and flexible workflow orchestration platform that seamlessly unifies data, machine learning, and analytics stacks. It leverages Kubernetes as its underlying platform, enabling the construction of robust and reproducible production-grade pipelines.
Plexe: Build Machine Learning Models from Natural Language Prompts
Plexe is an innovative Python library that empowers developers to build machine learning models using natural language descriptions. It automates the entire model creation process, from intent to deployment, through an intelligent multi-agent architecture. This allows for rapid development and experimentation, making ML accessible and efficient.