MLOps Glossary

A/B Testing

A/B Testing:

Running 2 (or more) models in parallel with traffic splitting equally whilst collecting data on model performance to compare which model is best. This is traditionally applied in areas such as online advertising. Note that traditional A/B testing may not include any underlying ML models per se, but rather refers to different variants of some product, e.g. different views of a website to test which increases user retention.