Biases aren’t useless: Let’s cut AI some slack on these
Summary
Heuristics or mental shortcuts help humans survive data overloads. Instead of aiming for zero-bias artificial intelligence (AI), let’s aim to minimize the harms caused by biases. Trade-offs need to be embraced.
We often think of bias as a flaw that needs to be purged. It’s not. It’s a cognitive shortcut that we need to fine-tune. We are constantly bombarded by shapes, colours, sounds and smells. Even though the human sensory system can collect this information at a rate of approximately 10 million bits per second, our conscious mind can only process 50 bits of information per second. That we are such a successful species, even though we can only process 0.0005% of all the data we collect, is a testament to our brain’s ability to quickly sort the important from the trivial.
We do this using heuristics, simple rules of thumb that help us decide which information to prioritize based on context, experience and human evolutionary adaptations. This is how we can tell, just from its colour and appearance, that we should not put something in our mouth. It’s how we know a dangerous neighbourhood the moment we enter it, even if we have never been there before. When we hear something rustling in the dark, that fight-or-flight response we instantly feel is because, at some time in humanity’s past, that sound indicated the presence of a predator.
Also Read: Artificial intelligence is evolving just as our minds do while growing up
We use heuristics like these to make all sorts of decisions. It is how we zero in on relevant facts when cognitive limitations, incomplete information and time constraints deny us the luxury of processing all the information we need.
We use different heuristics, depending on the circumstances. For instance, when we meet new people, we invoke the representativeness heuristic that allows us to evaluate them based on their similarity to certain stereotypes we have already formed. Every time we take an instant liking to someone, it has more to do with how certain superficial traits they possess align with our preconceived notions than any real knowledge of who they are.
These heuristics keep us going, helping us navigate life in a complex world. But they sometimes backfire, and when that happens, we call them biases.
The ‘availability bias,’ for instance, over-weights emotionally significant events that are vivid and recent, even though it is more than reasonable to presume that recently processed data is representative of similar information previously obtained. When we are accused of ‘confirmation bias,’ it means we have favoured information that supports our existing beliefs, even though this is a useful heuristic to keep us from constantly questioning what we already know.
Biases can also result in unfair outcomes. Recruiters have been known to overlook deserving candidates, and even supposedly impartial judges have allowed their subconscious prejudices to affect their decisions. To address this, we put in place automated decision-making systems wherever we could with the aim of ensuring that critical decisions were made objectively, based solely on the available data. This, we believed, would reduce the harms resulting from bias.
Also Read: Confidently wrong: Why AI is so exasperatingly human-like
We soon discovered that the algorithms we were using had biases of their own. The conclusions they drew were unexpectedly skewed by gender and ethnicity, even though there was nothing explicit in the data to support it. Automated résumé screeners systematically down-ranked the CVs of women applicants because historic hiring patterns suggested that men were better candidates. Since these biases were implicit in their training data, they got encoded into the heuristics these algorithms developed for the tasks they were given. What we thought was objective logic concealed an entirely different set of inequities.
Today, algorithmic bias is considered one of the most significant risks of artificial intelligence (AI). Now that we know they could conceal biases that are hard to detect, we treat AI decisions with suspicion. As a result, most AI governance efforts are focused on eliminating every last sign of possible bias.
This approach presumes that the ideal state is radical neutrality—complete freedom from bias of all sorts. This, however, misunderstands both the nature of bias and the process of cognition.
Also Read: Jaspreet Bindra: Being AI in the age of humans
Biases aren’t bugs in our system; they are tools we have developed to help us quickly arrive at the conclusions we need to make, given the constraints of real-time decision-making. If we don’t invoke them, we would simply be incapable of processing the nearly infinite amount of information available in the time we have to decide. These are the trade-offs we have to make to avoid being paralysed by the amount of data we would otherwise have to consider.
If we think of biases as a set of heuristics that are essential to discovering useful signals from the fire-hose of information, it becomes easier for us to reconcile ourselves to the trade-offs involved in deploying them. Rather than looking to build bias-free algorithms, we need to better understand how these biases work so that we can mitigate the harms that might result from their use.
One way to do this might be to develop tools that can detect these harms on the margin—early warning systems designed to indicate when the decisions of an AI system have started to deviate from the norm in unreasonable ways. This will allow us to redesign our systems before the unintended consequences of these heuristics result in widespread harm.
Biases are an essential element of human intelligence. If we need these cognitive tools to extract meaning from a universe too vast to completely comprehend, we should embrace the trade-offs necessary for their use in the age of AI.
The author is a partner at Trilegal and the author of ‘The Third Way: India’s Revolutionary Approach to Data Governance’. His X handle is @matthan.
