A big part of making ML systems trustworthy is ensuring that they do not create or perpetuate biases by making predictions for or against people by age, race, or other key attributes. With ML systems, it is possible to have an imbalance in the training data across different groups such as age or income bracket. An example could be training a medical diagnostic system on only an older population and then trying to apply the predictions to a different age group. That’s just one example of bias and there are many more. The goal is to have ML systems that treat people fairly and that do not favor one group over another in ways that are either unethical, illegal, or both.
The goal is to have ML systems that treat people fairly and that do not favor one group over another in ways that are either unethical, illegal, or both.
There is a class of tools and techniques emerging that are focused on solving this particular type of problem. They look to see if there are biases being introduced into the data acquisition, data preparation, data sampling, and data splitting stages. They can also compare the training data to the eventual production data and look for opportunities for bias to creep in. Additionally, they can provide insight into how various features play into the end prediction, and whether those weightings are unbalanced or biased. Finally, it is important that these tools watch the data and the models overtime to ensure that bias does not happen as the input data changes.
This will likely be an exciting space for the foreseeable future as these problems are addressed at a system and platform level.