HomeBlogGitHub

evidently.calculations.stattests

Available statistical tests. For detailed information about statistical tests see module documentation.

Submodules

anderson_darling_stattest module

Anderson-Darling test of two samples.

Name: “anderson”

Import:

>>> from evidently.calculations.stattests import anderson_darling_test

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

>>> from evidently.options import DataDriftOptions
>>> from evidently.calculations.stattests import anderson_darling_test
>>> options = DataDriftOptions(all_features_stattest=anderson_darling_test)

Using by name:

chisquare_stattest module

Chisquare test of two samples.

Name: “chisquare”

Import:

Properties:

  • only for categorical features

  • returns p-value

Example

Using by object:

Using by name:

cramer_von_mises_stattest module

Cramer-Von-mises test of two samples.

Name: “cramer_von_mises”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

class CramerVonMisesResult(statistic, pvalue)

Bases: object

energy_distance module

Energy-distance test of two samples.

Name: “ed”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

epps_singleton_stattest module

Epps-Singleton test of two samples.

Name: “es”

Import:

Properties:

  • only for numerical features

  • returns p-value

  • default threshold 0.05

Example

Using by object:

Using by name:

fisher_exact_stattest module

Fisher’s exact test of two samples.

Name: “fisher_exact”

Import:

Properties:

  • only for categorical features

  • returns p-value

Example

Using by object:

Using by name:

g_stattest module

G-test of two samples.

Name: “g_test”

Import:

Properties:

  • only for categorical features

  • returns p-value

Example

Using by object:

Using by name:

hellinger_distance module

Hellinger distance of two samples.

Name: “hellinger”

Import:

Properties:

  • only for categorical and numerical features

  • returns distance

Example

Using by object:

Using by name:

jensenshannon module

Jensen-Shannon distance of two samples.

Name: “jensenshannon”

Import:

Properties:

  • only for categorical and numerical features

  • returns distance

Example

Using by object:

Using by name:

kl_div module

Kullback-Leibler divergence of two samples.

Name: “kl_div”

Import:

Properties:

  • only for categorical and numerical features

  • returns divergence

Example

Using by object:

Using by name:

ks_stattest module

Kolmogorov-Smirnov test of two samples.

Name: “ks”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

mann_whitney_urank_stattest module

Mann-Whitney U-rank test of two samples.

Name: “mannw”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

psi module

PSI of two samples.

Name: “psi”

Import:

Properties:

  • only for categorical and numerical features

  • returns PSI value

Example

Using by object:

Using by name:

registry module

class StatTest(name: str, display_name: str, func: Callable[[pandas.core.series.Series, pandas.core.series.Series, str, float], Tuple[float, bool]], allowed_feature_types: List[str], default_threshold: float = 0.05)

Bases: object

Attributes:

allowed_feature_types : List[str]

default_threshold : float = 0.05

display_name : str

func : Callable[[Series, Series, str, float], Tuple[float, bool]]

name : str

exception StatTestInvalidFeatureTypeError(stattest_name: str, feature_type: str)

Bases: ValueError

exception StatTestNotFoundError(stattest_name: str)

Bases: ValueError

class StatTestResult(drift_score: float, drifted: bool, actual_threshold: float)

Bases: object

Attributes:

actual_threshold : float

drift_score : float

drifted : bool

get_stattest(reference_data: Series, current_data: Series, feature_type: str, stattest_func: Optional[Union[str, Callable[[Series, Series, str, float], Tuple[float, bool]], StatTest]])

register_stattest(stat_test: StatTest)

t_test module

T test of two samples.

Name: “t_test”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

tvd_stattest module

Total variation distance of two samples.

Name: “TVD”

Import:

Properties:

  • only for numerical features

  • returns distance

Example

Using by object:

Using by name:

utils module

generate_fisher2x2_contingency_table(reference_data: Series, current_data: Series)

Generate 2x2 contingency matrix for fisher exact test :param reference_data: reference data :param current_data: current data

  • Raises

    ValueError – if reference_data and current_data are not of equal length

  • Returns

    contingency_matrix for binary data

  • Return type

    contingency_matrix

get_binned_data(reference_data: Series, current_data: Series, feature_type: str, n: int, feel_zeroes: bool = True)

Split variable into n buckets based on reference quantiles :param reference_data: reference data :param current_data: current data :param feature_type: feature type :param n: number of quantiles

  • Returns

    % of records in each bucket for reference current_percents: % of records in each bucket for current

  • Return type

    reference_percents

get_unique_not_nan_values_list_from_series(current_data: Series, reference_data: Series)

Get unique values from current and reference series, drop NaNs

permutation_test(reference_data, current_data, observed, test_statistic_func, iterations=100)

Perform a two-sided permutation test :param reference_data: reference data :param current_data: current data :param observed: observed value :param test_statistic_func: the test statistic function :param iterations: number of times to permute

  • Returns

    two-sided p_value

  • Return type

    p_value

wasserstein_distance_norm module

Wasserstein distance of two samples.

Name: “wasserstein”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

z_stattest module

Mann-Whitney U-rank test of two samples.

Name: “mannw”

Import:

Properties:

  • only for numerical features

  • returns p-value

Example

Using by object:

Using by name:

proportions_diff_z_stat_ind(ref: DataFrame, curr: DataFrame)

proportions_diff_z_test(z_stat, alternative='two-sided')

Previousevidently.calculationsNextevidently.metrics

Last updated