LogoLogo
HomeBlogGitHub
v0.1.57
v0.1.57
  • What is Evidently?
  • Installation
  • Get Started Tutorial
  • Reports
    • Data Drift
    • Data Quality
    • Numerical Target Drift
    • Categorical Target Drift
    • Regression Performance
    • Classification Performance
    • Probabilistic Classification Performance
  • Tests
  • Examples
  • Integrations
    • Evidently and Grafana
    • Evidently and Airflow
    • Evidently and MLflow
  • Features
    • Dashboards
      • Input data
      • Column mapping
      • Generate dashboards
      • CLI
      • Colab and other environments
    • Profiling
      • Input data
      • Column mapping
      • Generate profiles
      • CLI
    • Monitoring
  • User Guide
    • Customization
      • Select Widgets
      • Custom Widgets and Tabs
      • Options for Data / Target drift
      • Options for Quality Metrics
      • Options for Statistical Tests
      • Options for Color Schema
    • Recipes
  • SUPPORT
    • Contact
    • F.A.Q.
    • Telemetry
    • Changelog
  • GitHub Page
  • Website
Powered by GitBook
On this page
  • Quick Start
  • How it works
  • 1. Interactive visual reports
  • Data Drift and Quality
  • Categorical and Numerical Target Drift
  • Classification Performance
  • Regression Performance
  • 2. Data and ML model profiling
  • 3. Real-time ML monitoring
  • Overview
  • Community and support

What is Evidently?

NextInstallation

Last updated 2 years ago

Evidently is an open-source Python library for data scientists and ML engineers. It helps evaluate, test and monitor the performance of ML models from validation to production.

You can think of it as an evaluation layer that fits into the existing ML stack.

Quick Start

Walk through a basic implementation to understand key Evidently features in under 10 minutes:

Explore the examples of the Evidently reports on different datasets and code tutorials:

How it works

Evidently has a modular approach with 3 interfaces on top of the shared Analyzer functionality.

  1. Interactive visual reports

  2. Data and model profiling

  3. Real-time ML monitoring

Evidently generates interactive HTML reports from pandas.DataFrame or csv files. You can use them for visual model evaluation, debugging and documentation.

Each report covers a certain aspect of the model performance. You can display reports as Dashboard objects in Jupyter notebook or Colab or export as an HTML file.

Evidently currently works with tabular data. 7 reports are available. You can combine, customize the reports or contribute your own report.

Data Drift and Quality

Categorical and Numerical Target Drift

Classification Performance

Regression Performance

Evidently also generates JSON Profiles. You can use them to integrate the data or model evaluation step into the ML pipeline.

For example, you can use it to perform scheduled batch checks of model health or log JSON profiles for further analysis. You can also build a conditional workflow based on the result of the check, e.g. to trigger alert, retraining, or generate a visual report.

Each Evidently dashboard has a corresponding JSON profile that returns the summary of metrics and statistical test results.

You can explore integrations with other tools:

Evidently also has Monitors that collect data and model metrics from a deployed ML service. You can use them to build live monitoring dashboards. Evidently helps configure the monitoring on top of the streaming data and emits the metrics. You can log and use the metrics elsewhere.

Overview

Here is a quick visual summary on how Evidently works. You can track and explore different facets of the ML model quality via reports, profiles or monitoring interface and flexibly fit it into your existing stack.

Community and support

: detects changes in feature distribution. : provides the feature overview.

Detect changes in or target and feature behavior.

Analyzes the performance and errors of a or model. Works both for binary and multi-class.

Analyzes the performance and errors of a model. Time series version coming soon.

There is a lightweight that comes with pre-built dashboards.

Evidently is in active development, and we are happy to receive and incorporate feedback. If you have any questions, ideas or want to hang out and chat about doing ML in production, !

Get Started Tutorial
Examples
1. Interactive visual reports
Data Drift
Data Quality
Numerical
Categorical
Classification
Probabilistic Classification
Regression
2. Data and ML model profiling
Evidently and MLflow
Evidently and Airflow
3. Real-time ML monitoring
integration with Prometheus and Grafana
join our Discord community
Data Drift
Data Quality
Categorical target drift
Numerical target drift
Classification Performance
Probabilistic Classification Performance
Regression Performance
Time Series