A Hands-On Intro to Drift Detection

About this webinar

Although powerful, modern machine learning models can be sensitive. Seemingly subtle changes in data distribution can destroy the performance of otherwise state-of-the art models, which can be especially problematic when ML models are deployed in production. In this webinar, we will give a hands-on overview to drift detection; the discipline focused on detecting such changes. We will start by building an understanding of the ways in which drift can occur, and why it pays to detect it. We’ll then explore the anatomy of a drift detector, and learn how they can be used to detect drift in a principled manner.

We’ll work through a real-world example using Alibi Detect, an open-source Python library offering powerful algorithms for adversarial, outlier and drift detection. You’ll learn how to set up drift detectors, and deduce what type of drift is occurring. Since data can take on many forms, such as image, text or tabular data, you’ll explore how to use existing ML models to preprocess your data into a form suitable for drift detectors. Then, to gain further insights into the causes of drift, we will employ advanced detectors which are able to perform fine-grained attribution to instances and features. To assess whether model performance has been affected by drift, we’ll experiment with using model uncertainty-based detectors. Finally, we’ll use a novel context-aware drift detector. This takes in context (or conditioning) variables, allowing you to test for drift depending on context that is permitted to change. We’ll discuss how this functionality can be crucial in many real-life drift detection scenarios.

Speakers

Ed Shee

Head of Developer Relations

Ashley Scillitoe

Data Science Research Engineer

What you'll learn

  • The importance of drift detection
  • How data drift can occur
  • How to use Alibi to set up drift detectors and deduce what type of drift is occurring
  • How these approaches can be applied and create value in real-world situations

Watch the video