AI Search Engine Challenges: Citation Issues and Inaccurate Answers

A recent study published by the Columbia Journalism Review found that search engines and artificial intelligence chatbots often provide incorrect answers and invent article citations. This phenomenon, especially in AI-focused tools, heightens concerns about the accuracy and reliability of the information made available to users. Why does this matter? AI-powered search tools have increased their use of web content to deliver quick answers to users. However, this often means users do not click through to the original sites, harming traffic across various channels. Another independent study indicates that click-through rates for AI searches and chatbots are significantly lower than those for Google Search. When citations are fabricated, an already unfavorable situation gets considerably worse, undermining users' trust in the results.

Analysis of the main errors

More than half of the answers from the Gemini and Grok 3 search engines referenced fabricated or broken URLs. These links take users to error pages, causing frustration and misinformation. The statistics are alarming: the chatbots analyzed gave incorrect answers to more than 60% of the queries submitted. Grok 3 had the highest error rate, with 94% incorrect answers, while Gemini was completely accurate only once in ten attempts. Perplexity, on the other hand, had the lowest error rate, answering 37% of queries incorrectly. The study's authors, Klaudia Jaźwińska and Aisvarya Chandrasekar, observed that "multiple chatbots appear to ignore the preferences of the Robot Exclusion Protocol," concluding that the research reinforces the findings of previous studies on chatbots such as ChatGPT, which highlight confident presentations of incorrect information and inconsistent information retrieval practices.

---

Read also: What is multimodal content?

Final thoughts on the use of AI in Search

The study analyzed 1,600 queries to compare the ability of AI tools (such as ChatGPT search, Perplexity, DeepSeek search, Microsoft CoPilot, and others) to identify article titles, original publishers, publication dates, and URLs. The findings raise concerns about the lack of transparency and user agency, amplifying problems associated with bias in information access systems. This criticism echoes that of experts such as Chirag Shah and Emily M. Bender, who point to unfounded and potentially toxic answers that may go unchecked by a typical user.