evidentlyai

evidently: Evaluate and Monitor ML and LLM Systems

Evidently is an open-source Python framework for evaluating, testing, and monitoring ML and LLM-powered systems, from tabular data to generative AI. Use it for offline reports and CI checks, or run its UI to track metrics over time.

ActiveJupyter NotebookApache-2.0PythonMachine LearningLLM
Stars
8k
Forks
954
Stars, last 30 days
+70
Last commit
1 month ago
Contributors
88

Our take

Activerecent releases

Health score

57/100

How it is scored
Activity
50
Community
84
Issues
46
Pull requests
46

+70 stars in the last 30 days

  • More contributors than 75% of the projects we track

Evidently is an actively maintained, Apache-2.0-licensed Python framework with ongoing releases and a broad contributor base, though recent issue and pull request closure activity has been limited.

Good fit if

  • You need a Python-based way to evaluate and monitor both traditional ML and LLM systems.
  • Your workflow benefits from combining reports, pass/fail tests, and a monitoring UI.
  • You want built-in evaluations with the option to define custom metrics and export results.

Look elsewhere if

  • You need an evaluation or monitoring tool that does not require Python 3.10 or newer.
  • Your requirement is for a fully managed service with no self-hosting, since the open-source UI is self-hosted and Evidently Cloud is a separate option.
  • Your team needs high recent issue-resolution throughput: the measured period shows few issues closed and pull requests merged.
All health signals
Last commit2026-09-10 (1 month ago)
Commits, last 90 days3
Releases, last 12 months7 (latest v0.7.23, 2026-09-11)
Contributors88 (top contributor: 24% of commits)
Issues closed, last 90 days1
Pull requests merged, last 90 days2
Project age5 years

Checked on 2026-10-22 with the GitHub API.

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

8,068

Forks

976

Change since Nov 17, 2026: +2 stars·-1 forks

Overview

Evidently is an open-source Python library for evaluating, testing, and monitoring ML and LLM-powered systems. It helps teams assess data and model quality, investigate issues during experiments, and check system behavior in production, with support for tabular and text data.

Its modular approach spans one-off reports and pass/fail test suites through to a self-hosted monitoring UI. Teams can use built-in metrics or add custom ones, and export evaluation results for integration with other tools.

Key Features

  • Provides more than 100 built-in metrics for data quality, distribution drift, predictive ML, text, and LLM evaluations.
  • Builds interactive reports from presets or individually selected metrics, with JSON, dictionary, and HTML output options.
  • Turns reports into test suites with configurable pass/fail conditions for regression checks, CI/CD, and data validation.
  • Adds row-level text descriptors, including sentiment, length, and content checks.
  • Supports evaluations for generative tasks, including LLM-as-a-judge options and RAG-related quality checks.
  • Offers a monitoring UI for viewing metrics and test results over time, with a self-hosted option.
  • Allows custom metrics through its Python interface and exports data for integration with other tools.

Use Cases

  • Data scientists comparing reference and current datasets to detect distribution shifts during experiments or after deployment.
  • ML engineers adding data and model quality checks to regression testing or CI/CD pipelines.
  • LLM teams evaluating generated answers, such as checking text properties, retrieval relevance, or response quality.
  • Operations teams tracking evaluation metrics and test results over time in the self-hosted monitoring UI.

What you need

Detected in the repository

  • Python >=3.10 (from pyproject.toml)
  • A test suite and automated checks on GitHub Actions

License in plain words

Apache-2.0permissive

  • Commercial use: yes
  • Modify and redistribute: yes
  • You must keep: the license, the NOTICE file and a note of your changes
  • Share your changes: no
  • Includes an explicit patent grant from the contributors:

A summary, not legal advice: the LICENSE file is what applies.

Getting Started

Install from PyPI:

pip install evidently

See the README and documentation for examples and setup details.

Alternatives

  • opik: Opik focuses on LLM tracing, experiments, and production monitoring, while Evidently also covers tabular data quality and drift.
  • weave: Weave centers on tracing and comparing LLM application runs, while Evidently also evaluates and monitors tabular data.
  • phoenix: Phoenix focuses on self-hosted LLM tracing and troubleshooting, while Evidently also supports tabular data quality and drift.
  • langwatch: LangWatch adds agent simulation, prompt management, and an AI gateway, while Evidently also targets tabular data evaluation and drift.
ProjectLanguageLicenseStarsStatus
evidentlyJupyter NotebookApache-2.08kActive
opikPythonApache-2.022.8kActive
weavePythonApache-2.01.1kActive
phoenixPythonOther11.7kActive
langwatchTypeScriptApache-2.04.9kActive

Considerations

  • The project requires Python 3.10 or newer. The supplied requirements do not specify a GPU or database dependency.
  • The project is actively releasing, but measured recent activity includes few commits, issues closed, and pull requests merged. Its contributors are not concentrated around one person.
  • The self-hosted monitoring UI is available, while some additional capabilities are offered through Evidently Cloud. Check the project documentation to confirm which deployment model and features suit your needs.
  • The Apache-2.0 license permits broad use, subject to its license terms and notices.

Found this useful?

Share it with someone who would like evidently.

Comparisons