BBC Study Reveals AI Assistants Struggle With News Content Accuracy

The BBC has published a new study revealing that publicly available AI assistants struggle to provide accurate responses to questions from experts about news published on its website.

According to the study shared by the BBC, the researchers considered popular generative AI chatbots such as ChatGPT, Perplexity, Copilot, and Gemini. The study was performed over a month—providing access to BBC’s website—and it showed that 51% of the answers given by the AI tools had significant issues.

Journalists and experts considered the accuracy, impartiality, and analysis of how the AI technologies represented the content provided. Other findings in the study showed that 13% of the quotes provided by the AI models were either altered from the original source or made up, and that 19% of the results considering BBC’s content had factual errors or inaccuracies—related to numbers, dates, or statements.

“We’re excited about the future of AI and the value it can bring audiences,” said Pete Archer, Programme Director for Generative AI at the BBC, in a public announcement. “But AI is also bringing significant challenges for audiences. People may think they can trust what they’re reading from these AI assistants, but this research shows they can produce responses to questions about key news events that are distorted, factually incorrect, or misleading.”

The experts noted that the information provided by ChatGPT and Copilot was not up-to-date. As an example, they said that both chatbots stated that Rishi Sunak and Nicola Sturgeon were still performing as Prime Minister and First Minister respectively after they both left.

The BBC has also complained about other technologies providing inaccurate information, like Apple’s Intelligence summarized news alerts providing false statements. The tech giant recently announced a new update to the AI news summary feature after the journal filed a complaint.