How LLMs Handle Contradictory Information From Multiple Sources

When sources disagree, a language model has to resolve the conflict somehow. Researchers group these conflicts into three kinds: a retrieved source contradicting what the model learned in training, retrieved sources contradicting each other, and the model’s own learned knowledge being inconsistent. Audits of AI search tools show the practical result: answers delivered with confidence even when they are wrong. For publishers, the lesson is to make your own position on a disputed point explicit, sourced and dated, so it can be represented accurately when a system has to choose.

Three kinds of conflict

A survey by Xu and colleagues, Knowledge Conflicts for LLMs, first posted in March 2024, organizes the research around three categories, which it calls context-memory, inter-context and intra-memory conflict. The survey describes these conflicts as able to “significantly impact the trustworthiness and performance of LLMs, especially in real-world applications where noise and misinformation are common.”

In plain terms:

• Context-memory conflict: a page the system retrieves at answer time says one thing, and what the model learned in training says another. One case is a recently changed fact: the training data has the old value, the current page has the new one.
• Inter-context conflict: two or more retrieved sources disagree with each other. Two sites quote different prices, dates or figures for the same thing.
• Intra-memory conflict: the model’s own learned knowledge is inconsistent, so it may answer the same question differently depending on how it is asked.

The survey reviews how models behave under each kind of conflict and what methods have been proposed to make them more robust. It does not describe how any specific commercial product resolves conflicts, and Google’s public statements on AI Overviews, quoted below, do not describe it either.

What Google says about its approach

Google describes AI Overviews as anchored to search results. In a May 2024 post on AI Overview errors, Liz Reid of Google wrote that AI Overviews “are built to only show information that is backed up by top web results.” The same post explains that errors usually come from “misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available,” and describes limits on satire, humor and user-generated content in responses that could offer misleading advice.

Read together, these statements suggest that when top results disagree, the overview depends on which results rank and how the system interprets them. Google has not published a rule for how AI Overviews handle a split among sources, such as whether they present both sides, pick the majority or defer to a particular kind of source. That last point is a reading of Google’s silence, not a statement Google has made.

What audits show

Independent audits show how conflicts and uncertainty surface in practice.

The Tow Center for Digital Journalism tested eight AI search tools in early 2025 and found they answered more than 60% of its queries incorrectly overall. It also observed that “most of the tools we tested presented inaccurate answers with alarming confidence, rarely using qualifying phrases such as ‘it appears,’ ‘it’s possible,’ ‘might,’ etc.” Copilot was the exception in one respect: it declined more questions than it answered.

A larger study coordinated by the European Broadcasting Union and the BBC, published as a News Integrity toolkit in October 2025, evaluated more than 3,000 responses from ChatGPT, Copilot, Perplexity and Gemini across 22 public broadcasters and found at least one significant issue in 45% of them.

Neither audit isolates conflict between sources as the cause of errors. Both show that when an answer is uncertain or wrong, the reader is often not told.

What this means for publishers

If your page sits on one side of a disagreement, a summary may represent you accurately, merge you with the other side, or drop you. You cannot control which, but you can make accurate representation easier.

1. State your position explicitly. If your figure differs from a commonly cited one, say so directly: “Our measurement found X; other published figures range from Y to Z because of differences in method.” A clear statement of the disagreement is harder to merge away than a number on its own.
2. Explain why sources differ. Different dates, samples or definitions can explain conflicting numbers. Naming the difference helps readers and gives a summary the context to report it.
3. Date facts that change. Context-memory conflicts can come from stale training data. A page that says “as of October 2026” and shows when it was updated makes the current value easier to identify.
4. Cite primary sources. When a claim traces to an official document or original study, link to it. A sourced claim gives a reader, and possibly a summary, something to verify. No platform documents a rule that favors it, but it costs nothing and helps the reader either way.
5. Correct errors where they start. If the conflicting information comes from an outdated page you control, or from a reference source such as Wikipedia, fix it there through the proper process.
6. Check how engines describe the point. Ask AI tools about the disputed fact, repeat the question over time, and note whether they report the disagreement, choose a side or get it wrong.

Write disputed points so they are hard to flatten: your position stated, the disagreement named, the reason for it explained and the date attached. Models resolve conflicts in ways their makers have not documented in detail, and audits show they rarely flag uncertainty, so the clarity has to come from the source.