Why You Shouldnt Trust News Briefings From AI Chatbots

You should definitely think twice before using an AI chatbot to provide quick news summaries. A new report from the BBC finds that popular chatbots make major errors in their summaries.

Google Gemini has the highest rate of problematic summaries

The test included ChatGPT , Google Gemini , Microsoft Copilot , and Perplexity AI . To start, the BBC asked each chatbot 100 news questions, requiring them to use BBC News sources whenever possible.

Experts from the BBC then assessed the quality of these summaries. Of the summaries, 51% contained some type of error, which could be factual inaccuracies, misquotations or outdated information.

Of those, 19% contained some kind of factual error, such as incorrect dates. And 13% of quotes attributed to the BBC in summaries were altered from the original or did not exist in the articles provided to the chatbots.

When analyzed by chatbot, Google's Gemini was the worst offender with over 60% of summaries containing problematic information. Microsoft Copilot came in second with 50% while ChatGPT and Perplexity had around 40% of problematic responses.

In its conclusion, the BBC said many of the problems were not just misinformation:

The study also found that the range of errors made by AI assistants extends beyond factual inaccuracies. The AI ​​assistants we tested struggled to distinguish between opinion and fact, editorialized, and often failed to provide the necessary context. Even if every statement in a response were accurate, these types of problems could lead to misleading or biased responses.

Many people have never tried using AI chatbots to summarize news because they don’t trust the technology to be reliable. But the research results are still surprising, with a large number of questionable results. AI still has a long way to go before it becomes a reliable way to learn more about the news.

AI features are still a work in progress

AI technology, especially chatbots, continues to improve rapidly. But as the BBC’s research shows, expecting accurate information from news sources is an extremely problematic area.

The BBC has complained about another AI-powered feature: Apple Intelligence’s News Briefing. In December 2024, a news brief from the channel incorrectly stated that Luigi Mangione shot himself. He was the man accused of killing healthcare company CEO Brian Thompson.

In response to complaints from the BBC and others, Apple temporarily disabled summary features for news and entertainment apps starting with iOS 18.3.

So when you want to follow the news, keep it simple: Skip the AI's summary and read it yourself.