Hyderabad: Researchers from the Indian Institute of Technology Hyderabad (IIT-H) have developed a method by which the inner workings of Artificial Intelligence (AI) models can be understood in terms of causal attributes.
Modern AI models, also known as Deep learning (DL), can arrive at decisions by in a more human-like manner, but how they arrive at those decisions is unknown, said the IIT-H in a statement, adding that it makes DL less useful when needed to understand that aspect.
Artificial Neural Networks (ANN) are AI models and programs that mimic the working of the human brain so that machines can learn to make decisions in a more human-like manner. Modern ANNs, often also called Deep Learning (DL), have increased tremendously in complexity such that machines can train themselves to process and learn from data that has been supplied to them as input, and almost match human performance in many tasks.
However, how they arrive at decisions is unknown, making them less useful when the reason to understand that is necessary. The practical implications of the lack of transparency in DL models are that end-users can lose their trust over the system, said a statement from IIT-H on Monday. “There is thus a need for methods that can access the underbelly of the AI programs and unravel their structure and functions," it added.
Researchers from IIT-H, led by Dr. Vineeth N. Balasubramanian, associate professor from the department of computer science and engineering, approached the problem with ANN architectures using causal inference, with what is known in the field as a ‘Structural Causal Model.’
Explaining this area of work, Dr. Balasubramanian said, “Thanks to our students’ efforts and hard work, we have proposed a new method to compute the Average Causal Effect of an input neuron on an output neuron. It is important to understand which input parameter is ‘causally’ responsible for a given output; for example in the field of medicine, how does one know which patient attribute was causally responsible for the heart attack? Our (IIT Hyderabad researchers’) method provides a tool to analyze such causal effects."
Transparency and understandability of the workings of DL models are gaining importance as discussions around the ethics of Artificial intelligence grows, added Dr. Balasubramanian on the importance of his team’s work on ‘explainable machine learning.’ “This makes sense given that the European Union General Data Protection Regulation (GDPR) regulation requires that an explanation must be provided if a machine learning model is used for any decisions made on its citizens, on any domain, be it banking, security or health.," said the statement from IIT-H.
It added the work on AI or DL was recently published in the proceedings of the 36th International Conference on Machine Learning, considered worldwide to be one of the highest-rated conferences in the area of Artificial Intelligence and Machine Learning.
The code developed by the IIT Hyderabad researchers to understand the workings of DL models, is available for free. The research paper is also available here
