LogoLogo
HomeBlogGitHub
latest
latest
  • New DOCS
  • What is Evidently?
  • Get Started
    • Evidently Cloud
      • Quickstart - LLM tracing
      • Quickstart - LLM evaluations
      • Quickstart - Data and ML checks
      • Quickstart - No-code evaluations
    • Evidently OSS
      • OSS Quickstart - LLM evals
      • OSS Quickstart - Data and ML monitoring
  • Presets
    • All Presets
    • Data Drift
    • Data Quality
    • Target Drift
    • Regression Performance
    • Classification Performance
    • NoTargetPerformance
    • Text Evals
    • Recommender System
  • Tutorials and Examples
    • All Tutorials
    • Tutorial - Tracing
    • Tutorial - Reports and Tests
    • Tutorial - Data & ML Monitoring
    • Tutorial - LLM Evaluation
    • Self-host ML Monitoring
    • LLM as a judge
    • LLM Regression Testing
  • Setup
    • Installation
    • Evidently Cloud
    • Self-hosting
  • User Guide
    • 📂Projects
      • Projects overview
      • Manage Projects
    • 📶Tracing
      • Tracing overview
      • Set up tracing
    • 🔢Input data
      • Input data overview
      • Column mapping
      • Data for Classification
      • Data for Recommendations
      • Load data to pandas
    • 🚦Tests and Reports
      • Reports and Tests Overview
      • Get a Report
      • Run a Test Suite
      • Evaluate Text Data
      • Output formats
      • Generate multiple Tests or Metrics
      • Run Evidently on Spark
    • 📊Evaluations
      • Evaluations overview
      • Generate snapshots
      • Run no code evals
    • 🔎Monitoring
      • Monitoring overview
      • Batch monitoring
      • Collector service
      • Scheduled evaluations
      • Send alerts
    • 📈Dashboard
      • Dashboard overview
      • Pre-built Tabs
      • Panel types
      • Adding Panels
    • 📚Datasets
      • Datasets overview
      • Work with Datasets
    • 🛠️Customization
      • Data drift parameters
      • Embeddings drift parameters
      • Feature importance in data drift
      • Text evals with LLM-as-judge
      • Text evals with HuggingFace
      • Add a custom text descriptor
      • Add a custom drift method
      • Add a custom Metric or Test
      • Customize JSON output
      • Show raw data in Reports
      • Add text comments to Reports
      • Change color schema
    • How-to guides
  • Reference
    • All tests
    • All metrics
      • Ranking metrics
    • Data drift algorithm
    • API Reference
      • evidently.calculations
        • evidently.calculations.stattests
      • evidently.metrics
        • evidently.metrics.classification_performance
        • evidently.metrics.data_drift
        • evidently.metrics.data_integrity
        • evidently.metrics.data_quality
        • evidently.metrics.regression_performance
      • evidently.metric_preset
      • evidently.options
      • evidently.pipeline
      • evidently.renderers
      • evidently.report
      • evidently.suite
      • evidently.test_preset
      • evidently.test_suite
      • evidently.tests
      • evidently.utils
  • Integrations
    • Integrations
      • Evidently integrations
      • Notebook environments
      • Evidently and Airflow
      • Evidently and MLflow
      • Evidently and DVCLive
      • Evidently and Metaflow
  • SUPPORT
    • Migration
    • Contact
    • F.A.Q.
    • Telemetry
    • Changelog
  • GitHub Page
  • Website
Powered by GitBook
On this page
  • class OptionsProvider()
  • Submodules
  • color_scheme module
  • class ColorOptions(primary_color: str = '#ed0400', secondary_color: str = '#4d4d4d', current_data_color: Optional[str] = None, reference_data_color: Optional[str] = None, color_sequence: Sequence[str] = ('#ed0400', '#0a5f38', '#6c3461', '#71aa34', '#d8dcd6', '#6b8ba4'), fill_color: str = 'LightGreen', zero_line_color: str = 'green', non_visible_color: str = 'white', underestimation_color: str = '#6574f7', overestimation_color: str = '#ee5540', majority_color: str = '#1acc98', vertical_lines: str = 'green', heatmap: str = 'RdBu_r')
  • data_drift module
  • class DataDriftOptions(confidence: Optional[Union[float, Dict[str, float]]] = None, threshold: Optional[Union[float, Dict[str, float]]] = None, drift_share: float = 0.5, nbinsx: Union[int, Dict[str, int]] = 10, xbins: Optional[Dict[str, int]] = None, feature_stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest, Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]]] = None, all_features_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, cat_features_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_features_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, per_feature_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]]] = None, all_features_threshold: Optional[float] = None, cat_features_threshold: Optional[float] = None, num_features_threshold: Optional[float] = None, per_feature_threshold: Optional[Dict[str, float]] = None, cat_target_threshold: Optional[float] = None, num_target_threshold: Optional[float] = None, cat_target_stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None, num_target_stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]] = None)
  • quality_metrics module
  • class QualityMetricsOptions(conf_interval_n_sigmas: int = 1, classification_threshold: float = 0.5, cut_quantile: Union[NoneType, Tuple[str, float], Dict[str, Tuple[str, float]]] = None)
  1. Reference
  2. API Reference

evidently.options

class OptionsProvider()

Bases: object

Methods:

add(options)

get(options_type: Type[TypeParam])

Submodules

color_scheme module

class ColorOptions(primary_color: str = '#ed0400', secondary_color: str = '#4d4d4d', current_data_color: Optional[str] = None, reference_data_color: Optional[str] = None, color_sequence: Sequence[str] = ('#ed0400', '#0a5f38', '#6c3461', '#71aa34', '#d8dcd6', '#6b8ba4'), fill_color: str = 'LightGreen', zero_line_color: str = 'green', non_visible_color: str = 'white', underestimation_color: str = '#6574f7', overestimation_color: str = '#ee5540', majority_color: str = '#1acc98', vertical_lines: str = 'green', heatmap: str = 'RdBu_r')

Bases: object

Collection of colors for data visualization

  • primary_color - basic color for data visualization.

    Uses by default for all bars and lines for widgets with one dataset and as a default for current data.

  • secondary_color - basic color for second data visualization if we have two data sets, for example, reference data.

  • current_data_color - color for all current data, by default primary color is used

  • reference_data_color - color for reference data, by default secondary color is used

  • color_sequence - set of colors for drawing a number of lines in one graph, in for data quality, for example

  • fill_color - fill color for areas in line graphs

  • zero_line_color - color for base, zero line in line graphs

  • non_visible_color - color for technical, not visible dots or points for better scalability

  • underestimation_color - color for underestimation line in regression

  • overestimation_color - color for overestimation line in regression

  • majority_color - color for majority line in regression

  • lines - color for vertical lines

  • heatmap_colors - colors for heatmap

Attributes:

color_sequence : Sequence[str] = ('#ed0400', '#0a5f38', '#6c3461', '#71aa34', '#d8dcd6', '#6b8ba4')

current_data_color : Optional[str] = None

fill_color : str = 'LightGreen'

heatmap : str = 'RdBu_r'

majority_color : str = '#1acc98'

non_visible_color : str = 'white'

overestimation_color : str = '#ee5540'

primary_color : str = '#ed0400'

reference_data_color : Optional[str] = None

secondary_color : str = '#4d4d4d'

underestimation_color : str = '#6574f7'

vertical_lines : str = 'green'

zero_line_color : str = 'green'

Methods:

get_current_data_color()

get_reference_data_color()

data_drift module

Bases: object

Configuration for Data Drift calculations.

  • Parameters

    • confidence – Defines the confidence level for statistical tests. Applies to all features (if passed as float) or certain features (if passed as dictionary). (Deprecated) Use threshold to define confidence level for statistical tests as more universal solution.

    • threshold – Defines thresholds for statistical tests. Applies to all features (if passed as float) or certain features (if passed as dictionary).

    • drift_share – Sets the share of drifting features as a condition for Dataset Drift in the Data Drift report.

    • nbinsx – Defines the number of bins in a histogram. Applies to all features (if passed as int) or certain features (if passed as dictionary).

    • xbins – Defines the boundaries for the size of a specific bin in a histogram.

    • feature_stattest_func – Defines a custom statistical test for drift detection in the Data Drift report. Applies to all features (if passed as a function) or individual features (if a dict). (Deprecated) Use all_features_stattest or per_feature_stattest.

    • all_features_stattest – Defines a custom statistical test for drift detection in the Data Drift report for all features.

    • cat_features_stattest – Defines a custom statistical test for drift detection in the Data Drift report for categorical features only.

    • num_features_stattest – Defines a custom statistical test for drift detection in the Data Drift report for numerical features only.

    • per_feature_stattest – Defines a custom statistical test for drift detection in the Data Drift report per feature.

    • cat_target_stattest_func – Defines a custom statistical test to detect target drift in category target.

    • num_target_stattest_func – Defines a custom statistical test to detect target drift in numeric target.

Attributes:

all_features_threshold : Optional[float] = None

cat_features_threshold : Optional[float] = None

cat_target_threshold : Optional[float] = None

confidence : Optional[Union[float, Dict[str, float]]] = None

drift_share : float = 0.5

nbinsx : Union[int, Dict[str, int]] = 10

num_features_threshold : Optional[float] = None

num_target_threshold : Optional[float] = None

per_feature_threshold : Optional[Dict[str, float]] = None

threshold : Optional[Union[float, Dict[str, float]]] = None

xbins : Optional[Dict[str, int]] = None

Methods:

as_dict()

get_feature_stattest_func(feature_name: str, feature_type: str)

get_nbinsx(feature_name: str)

get_threshold(feature_name: str, feature_type: str)

quality_metrics module

class QualityMetricsOptions(conf_interval_n_sigmas: int = 1, classification_threshold: float = 0.5, cut_quantile: Union[NoneType, Tuple[str, float], Dict[str, Tuple[str, float]]] = None)

Bases: object

Attributes:

classification_threshold : float = 0.5

conf_interval_n_sigmas : int = 1

cut_quantile : Union[None, Tuple[str, float], Dict[str, Tuple[str, float]]] = None

Methods:

as_dict()

get_cut_quantile(feature_name: str)

Previousevidently.metric_presetNextevidently.pipeline

Last updated 2 months ago

class DataDriftOptions(confidence: Optional[Union[float, Dict[str, float]]] = None, threshold: Optional[Union[float, Dict[str, float]]] = None, drift_share: float = 0.5, nbinsx: Union[int, Dict[str, int]] = 10, xbins: Optional[Dict[str, int]] = None, feature_stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], , Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]]]] = None, all_features_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None, cat_features_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None, num_features_stattest: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None, per_feature_stattest: Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]]] = None, all_features_threshold: Optional[float] = None, cat_features_threshold: Optional[float] = None, num_features_threshold: Optional[float] = None, per_feature_threshold: Optional[Dict[str, float]] = None, cat_target_threshold: Optional[float] = None, num_target_threshold: Optional[float] = None, cat_target_stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None, num_target_stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None)

all_features_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None

cat_features_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None

cat_target_stattest_func : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None

feature_stattest_func : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], , Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]]]] = None

num_features_stattest : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None

num_target_stattest_func : Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]] = None

per_feature_stattest : Optional[Dict[str, Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], ]]] = None

StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest
StatTest