AIs disagree on brands in 23% of questions

In short. Across 617 questions from real audits, asked to 4.3 models on average, 23% showed a presence disagreement: at least one model named the brand and at least one other did not. This records a difference observed for the same question. It does not decide whether every answer is true or false, and it does not make the sample representative of every brand.

What the 23% rate measures

The calculation concerns one specific event: whether a brand appears in an answer. For every question, we check whether the brand occurs in the text generated by each model. A question enters the disagreement rate when answers differ on that point: one names the brand and another does not. The 23% rate is therefore the share of the 617 questions that meet that definition.

This deliberately narrow definition prevents a common shortcut. It does not measure answer quality, tone, length, recommendation order, or the accuracy of every claim. A response can name a brand cautiously, inaccurately, or off topic; another can omit it while remaining useful. Those dimensions need separate review.

Why one model gives only a partial measurement

A model is not an interchangeable channel. Asking one system the same question gives one observation under stated conditions: a model, a moment, a response mode, and a wording. When another model does not name the brand, that does not make the first answer false. It shows that observed presence also depends on the measurement condition.

The practical consequence is straightforward. A single capture can be a useful signal, but it is weak evidence for a general conclusion. Teams that want to compare situations should retain the same questions and record the model. Without that context, apparently conflicting results may simply describe different conditions.

Plausible mechanisms are not established causes

Several mechanisms may contribute to different responses: different corpora and update dates, response rules, source selection, question wording, or web-search availability. These are plausible hypotheses. The 617 questions do not isolate a single cause or attribute a gap to a specific supplier decision.

An omission in one answer does not establish general absence; an isolated mention does not establish stable presence either.

Web search and model memory are separate contexts

Source: our 2026 barometer, using data collected from September 2025 to June 2026. The table covers the subset of 63 audits measuring both modes, not the 617 questions in this study or the current edition of the Observatory.

Response settingAverage visibility score (/100)Audits compared
Native, without web search63.463
With web search70.563
Web minus native difference7.163

The difference is 7.1 score points, not a 7.1% brand-presence rate. It is not an average to apply to every brand. Native mode measures a response without external web retrieval enabled, not proven inclusion of a brand in training data.

Why sources should be examined without overreading them

Answers may use different signals and sources. Our source study lists families observed in separate audits. It can guide checks, such as reviewing whether information published by a brand and by third parties is consistent. It does not turn an unresolved URL into evidence that information was invented.

A URL can fail for many reasons: a moved page, a redirect, access controls, a temporary incident, or a transcription error. The sound approach is to retain the URL, check its status, and read the statement in context. The conclusion can then concern a verified item rather than an assumption.

How to document a multi-model audit

To make comparison usable, retain the exact question, language, model, response mode, collection date, full text, and presence result. Add displayed sources when available, but do not treat them as causal proof. This record makes it possible to return to the cases behind an aggregate.

  • Compare questions with the same intent: recommendation, comparison, and a specific need are not interchangeable.
  • Keep the wording stable when the objective is to monitor a variation.
  • Read responses before summarising a rate, especially where models diverge.
  • Separate presence, accuracy, tone, and sources in the interpretation.

Also retain available settings and the sources as displayed at collection time. Another reviewer should be able to understand why two answers received different presence labels. A presence checkbox preserves neither qualifications nor context: keep the full text.

Reading 23% at the right level

This detail protects decision-making. Applying the rate to a whole panel would hide the actual distribution: some brands or question types may concentrate differences, while others remain stable in the collected answers. Only the detailed data can test that distribution.

What this measurement can inform

Our view: a single-model audit is insufficient for conclusions about visibility across models. The 23% of questions with disagreement supports that position, but does not prescribe a universal model count. One favourable screenshot does not validate the whole landscape.

Start with the commercial questions that matter to the brand. Use the exact wording across several models, separate native and web modes, and review divergent cases first. For each case, record what is missing, what is inaccurate, and what the brand can correct with a verifiable source. This is our recommended workflow, not an effect measured by this study.

Tracking change without manufacturing a trend

Models and their outputs change. Repeating a question set can compare observations, provided the question, language, model, and mode are not changed at the same time. When several conditions change, an observed difference becomes hard to interpret. Traceability is therefore more useful than a dashboard reduced to one line.

Tracking does not promise a trajectory. An increase or decrease may be observed; it does not automatically demonstrate the effect of a recent action. To connect a change with a hypothesis, record the changes, retain the answers, and check that the compared scope is consistent.

Limits that should remain visible

The 617 questions came from brands that chose to be audited. The sample is not presented as representative of every brand. The average of 4.3 models describes this collection, not a standard for every study. Finally, disagreement concerns brand presence, not every dimension of an answer.

Conclusion: prefer cases and distributions to slogans

Our priority is to make disagreements verifiable before starting corrections. Keep the case, identify the information to check, then repeat the comparison under documented conditions. That is more useful than an isolated score for deciding what to investigate.

For further context, read the 2026 barometer, the study of sources cited by AIs, the AGS methodology, the Observatory, and the guide to auditing brand visibility.

Questions and answers

Do different AIs give the same answer about a brand?

No. Across 617 questions asked to several models, in 23% of cases at least one model cites the brand and another does not. Presence in AI depends on which model is asked.

Why don't ChatGPT, Gemini and Claude agree about my brand?

Because they have training memories frozen at different dates, some browse the web and others don't, and they don't rely on the same sources (LinkedIn, Wikipedia, reviews, registries).

Is an audit on ChatGPT alone enough?

No: it measures one reality out of four. Your customers also use Gemini, Claude or Perplexity. A reliable audit queries several models and reasons in distribution, with confidence intervals.

How do you measure this disagreement?

For each question asked to at least two models, we detect whether the brand actually appears in each answer, then count the questions where models diverge (one cites, another does not).

About the author

Davy Abderrahman

Founder of

As a specialist in visibility in AI answers (AEO/GEO/LLMO), I help agencies and consultants measure and analyze their clients’ presence on ChatGPT, Claude, Gemini, Perplexity and other AI answer engines.

AEO GEO LLMO AI visibility AI audits
How do your brands appear in AI answers?

Agencies, consultants and in-house teams: compare the answers generated for your own scenarios, languages and selected models. Our infrastructure, your scenarios, your expertise.

Was this article helpful?

- (0 votes)