Only 2.4% in math: Is ChatGPT turning dumb?
SummaryOver the last two-three months, many ChatGPT users have been complaining of a drop in its performance. A new research paper that has gone viral confirms this trend, which may make companies more wary of using ChatGPT. Mint explores this issue.
Why is ChatGPT in the news?
Recently, researchers Lingjiao Chen and James Zou from Stanford University, and Matei Zaharia from UC Berkeley tested GPT-3.5 and GPT-4 for solving math problems, answering sensitive and dangerous questions, generating code and for visual reasoning. The conclusion: the “performance and behaviour" of both these large language models (LLMs) “can vary greatly over time". The March version of GPT-4 identified prime numbers with 97.6% accuracy. In the June version, accuracy collapsed to 2.4%. Both made “more formatting mistakes in code generation in June than in March".