Evidently and Airflow

Run model evaluation or data drift analysis as a part of Airflow DAG.

Apache Airflow is an open-source workflow management tool.

You can use this integration to generate JSON profiles or HTML reports and as a step in the Airflow DAG.

An integration example is available as a Docker container:

evidently/examples/integrations/airflow_drift_detection at main · evidentlyai/evidentlyGitHub

Follow the readme to install and modify the example.

It contains two specific DAGs that match common batch monitoring needs.

1. Generate model performance reports as a batch job

You can generate an Evidently report (e.g. a data drift report) every time the new data arrives. You can then store it in your file system.

Here is a DAG example:

evidently/evidently_drift_dashboard.py at main · evidentlyai/evidentlyGitHub

2. Generate visual performance reports on defined conditions

You might not always need to generate visual reports every time.

For example, you can run checks on the model performance and only generate the reports if a certain condition is satisfied. For example, if you detect drift or performance drop. Otherwise, you can simply log the results.

Here is a DAG example:

evidently/evidently_conditional_drift_dashboard_generation.py at main · evidentlyai/evidentlyGitHub

It works as the following:

• Run a data drift check by generating an Evidently JSON profile

• If the drift is not detected, log the JSON output

• If the drift is detected, generate and store a visual HTML report