Bias vs Fairness vs Explainability in AI


Photo by Emily Morter on Unsplash


  • Black Box — You have no access or information about the underlying model. The inputs and outputs of the model are all you can use to generate an explanation.
  • White Box — You have access to the underlying model so it’s easier to provide information about exactly why a certain prediction was made.
  • Model View — Overall, what features are more important than others to the model?
  • Instance View — For a particular prediction, what factors contributed?
Photo by Piret Ilver on Unsplash


  • The sensitive features might actually be critical to the model. Imagine you’re trying to predict the height a child will be when they are fully grown. Removing sensitive attributes like age and sex will make your predictions useless.
  • Fairness is not necessarily about being agnostic. Sometimes it’s important to include sensitive features in order to favor those who might be discriminated against in other features. An example of this is university admissions, where raw grades alone may not be the best way to find the brightest pupils. Those who had access to fewer resources or a lower quality of education might have had better scores otherwise.
  • Sensitive features might be hidden in other attributes. It is often possible to determine the values for sensitive features using a combination of non-sensitive ones. For example, an applicant’s full name might allow a machine learning model to infer their race, nationality, or gender.



Want to find out more about Seldon? Fill out the form here and we’ll be in touch.