# Embeddings drift parameters {% hint style="info" %} **You are looking at the old Evidently documentation**: this API is available with versions 0.6.7 or lower. Check the newer docs version [here](https://docs.evidentlyai.com/introduction). {% endhint %} **Pre-requisites**: * You know how to generate Reports or Test Suites with default parameters. * You know how to pass custom parameters for Reports or Test Suites. * You know how to use Column Mapping to map embeddings in the input data. ## Code example You can refer to an example How-to-notebook showing how to pass parameters for different embeddings drift detection methods: {% embed url="" %} ## Default When you calculate embeddings drift, Evidently automatically applies the default drift detection method (“model”). In Reports: ```python report = Report(metrics=[ EmbeddingsDriftMetric('small_subset') ]) ``` In Test Suites: ```python tests = TestSuite(tests=[ TestEmbeddingsDrift(embeddings_name='small_subset') ]) ``` It works the same inside presets, like `DataDriftPreset`. ## Embedding parameters - Metrics and Tests You can override the defaults by passing a custom `drift_method` parameter to the relevant Metric or Test. You can define the embeddings drift detection method, the threshold, or both. Pass the `drift_method` parameter: ```python from evidently.metrics.data_drift.embedding_drift_methods import model report = Report(metrics = [ EmbeddingsDriftMetric('small_subset', drift_method = model() ) ]) ``` ## Embedding parameters - Presets When you use `NoTargetPerformanceTestPreset`, `DataDriftTestPreset` or `DataDriftPreset` you can specify which subsets of columns with embeddings to include using `embeddings`, and the drift detection method using `embeddings_drift_method`. By default, the Presets will include all columns mapped as containing embeddings in `column_mapping`. To exclude columns with embeddings: ```python embeddings = [] ``` To specify which sets of columns to include (with the default drift detection method): ```python embeddings = [‘set1’, ‘set2’] ``` To specify which sets of columns to include, and specify the method: ```python embeddings = [‘set1’, ‘set2’] embeddings_drift_method = {‘set1’: model(), ‘set2’: ratio())} ``` ## Embedding drift detection methods Currently 4 embeddings drift detection methods are available. | Embeddings drift detection method | Description and default | | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |

drift\_method=model
(Default)

| | | `drift_method=ratio` | | | `drift_method=distance` | | | `drift_method=mmd` | | If you specify an embedding drift detection method but do not pass additional parameters, defaults will apply. You can also specify parameters for any chosen method. Since the methods are different, each has a different set of parameters. Note that you should pass the parameters **directly to the chosen drift detection method**, not to the Metric. ### Model-based (“model”) ```python report = Report(metrics = [ EmbeddingsDriftMetric('small_subset', drift_method = model( threshold = 0.55, bootstrap = None, quantile_probability = 0.05, pca_components = None, ) ) ]) ``` | Parameter | Description | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `threshold` |

Sets the threshold for drift detection (ROC AUC). Drift is detected when drift\_score > threshold.
Applies when bootstrap != True.

Default: 0.55.

| | `bootstrap` (optional) |

Boolean parameter (True/False) to determine whether to apply statistical hypothesis testing.
If applied, the ROC AUC of the classifier is compared to the ROC AUC of the random classifier at a set percentile. The calculation is repeated 1000 times with randomly assigned target class probabilities. This produces a distribution of random roc\_auc scores with a mean of 0,5. We then take the 95th percentile (default) of this distribution and compare it to the ROC-AUC score of the classifier. If the classifier score is higher, data drift is detected.

Default: True if <= 1000 objects, False if > 1000 objects.

| | `quantile_probability` (optional) |

Sets the percentile of the possible ROC AUC values of the random classifier to compare against.
This applies when bootstrap is True.

Default: 0.95

| | `pca_components` (optional) |

The number of PCA components. If specified, dimensionality reduction will be applied to project data to n-dimensional space based on the number of pca\_components.

Default: None.

| ### Maximum mean discrepancy (“mmd”) ```python report = Report(metrics = [ EmbeddingsDriftMetric('small_subset', drift_method = mmd( threshold = 0.015, bootstrap = None, quantile_probability = 0.05, pca_components = None, ) ) ]) ``` | Parameter | Description | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `threshold` |

Sets the threshold value of MMD for drift detection. Drift is detected when drift\_score > threshold.
Applies when bootstrap != True.

Default: 0.015 (MMD).

| | `bootstrap` (optional) |

Boolean parameter (True/False) to determine whether to apply statistical hypothesis testing.

If applied, the value of MMD between reference and current (mmd\_0) is tested against possible MMD values in reference. We randomly split the reference data into two parts and compute MMD values (mmd\_i) between them. The calculation is repeated 100 times. This produces a distribution of MMD values obtained for a reference dataset. We then take the 95th percentile (default) of this distribution and compare it to the MMD between reference and current datasets. If the mmd\_0 > mmd\_95, data drift is detected.

Default: True if <= 1000 objects, False if > 1000 objects.

| | `quantile_probability` (optional) |

Sets the percentile of the possible MMD values in reference to compare against.
Applies when bootstrap == True.

Default: 0.95.

| | `pca_components` (optional) |

The number of PCA components. If specified, dimensionality reduction will be applied to project data to n-dimensional space based on the number of pca\_components.

Default: None.

| ### Share of drifted embedding components (“ratio”) ```python report = Report(metrics = [ EmbeddingsDriftMetric('small_subset', drift_method = ratio( component_stattest = 'wasserstein', component_stattest_threshold = 0.1, threshold = 0.2, pca_components = None, ) ) ]) ``` | Parameter | Description | | ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `component_stattest` (optional) |

Sets the tabular drift detection method (any of the tabular drift detection methods for numerical features available in Evidently).

Default: Wasserstein

| | `component_stattest_threshold` (optional) |

Sets the threshold for drift detection for individual embedding components. Drift is detected when drift\_score > component\_stattest\_threshold in case of distance/divergence metrics where the threshold is the metric value or drift\_score < component\_stattest\_threshold in case of statistical tests where the threshold is the p-value.

Default: 0.1 (relevant for Wasserstein).

| | `threshold` (optional) |

Sets the threshold (share of drifted embedding components) for drift detection for the overall dataset.

Default: 0.2

| | `pca_components` (optional) |

The number of PCA components. If specified, dimensionality reduction will be applied to project data to n-dimensional space based on the number of pca\_components.

Default: None.

| ### Distance-based methods (“distance”) ```python report = Report(metrics = [ EmbeddingsDriftMetric('small_subset', drift_method = distance( dist = 'euclidean', #"euclidean", "cosine", "cityblock" or "chebyshev" threshold = 0.2, pca_components = None, bootstrap = None, quantile_probability = 0.05 ) ) ]) ``` | Parameter | Description | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |

dist (optional)

Available:
euclidean
cosine
cityblock (manhattan distance)
chebyshev

|

Sets the distance metric for drift detection.

Default: Euclidean distance

| | `threshold` (optional) |

Sets the threshold for drift detection. Drift is detected when drift\_score > threshold.
Applies when bootstrap != True

Default: 0.2 (relevant for euclidean distance)

| | `bootstrap` (optional) |

Boolean parameter (True/False) to determine whether to apply statistical hypothesis testing.

If applied, the distance between reference and current is tested against possible distance values in reference. We randomly split the reference data into two parts and compute the distance between them. The calculation is repeated 100 times. This produces a distribution of distance values obtained for a reference dataset. We then take the 95th percentile (default) of this distribution and compare it to the distance between reference and current datasets. If the distance between the reference and current is higher than the 95th percentile of the distance obtained for the reference dataset, the drift is detected.

Default: True if <= 1000 objects, False if > 1000 objects.

| | `quantile_probability` (optional) |

Sets the percentile of the possible distance values in reference to compare against.
Applies when bootstrap == True.

Default: 0.95.

| | `pca_components` (optional) |

The number of PCA components. If specified, dimensionality reduction will be applied to project data to n-dimensional space based on the number of pca\_components.

Default: None.

| --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs-old.evidentlyai.com/user-guide/customization/embeddings-drift-parameters.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.