Open AI's popular chatbot ChatGPT may have aced the Wharton Master of Business Administration (MBA) examination, however, a recent report has stated that it failed the Union Public Service Commission (UPSC) exam held in India to train and recruit civil servants.
The chatbot that was launched in November 2022, has created ripples through the academic, creative, and technological industry, to the extent where it has now raised questions of eliminating people's jobs. A popular belief goes that ChatGPT has all the answers.
However, the UPSC exam proved otherwise.
While the chatbot easily cleared some of the most difficult exams around the world, including the Master of Business Administration (MBA) programme at the University of Pennsylvania’s Wharton School to the US medical exam and further the US bar exam for law, the chatbot failed to clear even the prelims to the UPSC examination.
For the uninitiated, the Union Public Service Commission conducts various examinations to recruit eligible candidates for the Indian Government.
The latest attempt for the chatbot was made possible by Analytics India Magazine (AIM). When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?", it knew that it was going to be difficult, the magazine noted. The chatbot had to say that it cannot give a “definite answer" on whether it can pass the exam or not.
ChatGPT fails UPSC examination
AIM asked ChatGPT all 100 questions from Question Paper 1 (Set A) from the UPSC Prelims 2022. ChatGPT could only answer 54 of them correctly which was a surprise for the magazine company. The cutoff for the general category candidates was 87.54 which suggests that ChatGPT couldn’t clear the UPSC examination.
Notably, ChatGPT's knowledge is limited till September 2021. Therefore it failed to answer current affairs questions for India. But that is not it!
The believed-to-be-know-it-all chatbot failed to answer Geography, History, and Economy questions from the UPSC paper as well. Further in some multiple-choice questions, the chatbot created an extra option that wasn’t provided in the original question and labelled it “None of the above".
The UPSC examination is considered one of the toughest exams in the world. About 11-12 lakh candidates appear for the exam every year but only 5 per cent of them make it to the mains stage.
ChatGPT aces Wharton MBA exam
Professor Christian Terwiesch released a paper this week, according to Fortune, which maps ChatGPT's performance on the Operations Management paper. According to Terwiesch, the chatbot "does an amazing job at basic operations management and process analysis questions including those that are based on case studies."
The professor also noted that ChatGPT “would have received a B to B- grade on
