Romila Pradhan, a data scientist at Purdue University in Indiana, is using counterfactuals to make automated decision making more transparent. Organizations now use machine-learning models to choose who gets credit, jobs, parole, even housing (and who doesn’t). Regulators have started to require organizations to explain the outcome of many of these decisions to those affected by them. But reconstructing the steps made by a complex algorithm is hard.
Pradhan thinks counterfactuals can help. Let’s say a bank’s machine-learning model rejects your loan application and you want to know why. One way to answer that question is with counterfactuals. Given that the application was rejected in the actual world, would it have been rejected in a fictional world in which your credit history was different? What about if you had a different zip code, job, income, and so on? Building the ability to answer such questions into future loan approval programs, Pradhan says, would give banks a way to offer customers reasons rather than just a yes or no.
Counterfactuals are important because it’s how people think about different outcomes, says Pradhan: “They are a good way to capture explanations.”
They can also help companies predict people’s behavior. Because counterfactuals make it possible to infer what might happen in a particular situation, not just on average, tech platforms can use it to pigeonhole people with more precision than ever.
The same logic that can disentangle the effects of dirty water or lending decisions can be used to hone the impact of Spotify playlists, Instagram notifications, and ad targeting. If we play this song, will that user listen for longer? If we show this picture, will that person keep scrolling? “Companies want to understand how to give recommendations to specific users rather than the average user,” says Gilligan-Lee.