Why AI engines can recommend different contractors for the same question

The short answer

Ask ChatGPT, Gemini, Perplexity, and Claude the same question about hiring a Phoenix contractor and you can get a different business at the top of each answer. Each engine builds its answer from its own search results. They use different crawlers, different indexes, and different ways of turning one question into searches. A contractor who is well documented on the pages one engine reads can be absent from the pages another engine reads, so being recommended by one engine does not guarantee the next.

Whether they actually disagree is a measurement, not an assumption. Our latest measurement covers window and door questions and two engines, Claude and Gemini; ChatGPT and Perplexity will be added. In that run, on 100%¹ of the 8² questions we compared, Claude and Gemini named the same business most. Agreeing on the top name is only part of the picture: the two engines differed in which businesses they named at all and in which websites they cited, as the sections below show.

How we measure disagreement

In each weekly run we ask the engines we measure the same set of questions a Phoenix homeowner might ask about hiring a contractor, and record which businesses each answer names. For each question, we take each engine's most-named business: the one it named in the most of its answers to that question, with ties kept.

Then we compare. An engine that answered but named no business sits that question out, because silence is not disagreement. A question where fewer than two engines named a business is not compared at all, and the study counts those separately so they are not hidden. What is left is a clean test: among the engines that did name someone, did they point at the same contractor?

So far that test has run on one trade, windows and doors, and one pair of engines. We have not measured roofing or heating and air conditioning yet. Each footnote states how many trades and weeks a figure covers. The per-trade tables are on the engine disagreement study.

Which engines agree with which

Pairs tell you more than totals once several engines are in the comparison. For each pair, we take the share of questions, among those where both named a business, on which they named the same business most. The one pair measured so far, Claude and Gemini, agreed on 100%³ of those questions. The pairs with ChatGPT and Perplexity will be reported when those engines are added.

A pair that agrees often is likely reading overlapping sources. A pair that rarely agrees is working from different material, and a business that wants to be named by both needs to be present in both sets of sources.

Agreeing on the top name is not the same as naming the same list. When we compare the engines with Google, the difference shows. Of the window and door businesses Google shows in Phoenix that we could compare, the share Claude never named was 33%⁴, and for Gemini it was 0%⁵. The two engines also cited different websites most: ecowatch.com⁶ for Claude and energyshieldwindowsanddoors.com⁷ for Gemini. Two engines can agree on who leads and still read, and name, a different slice of the market.

Why the engines disagree

The reasons are in how each engine finds pages, and the engines document much of it themselves.

Different crawlers. OpenAI surfaces websites in ChatGPT's search features with its own crawler, OAI-SearchBot⁸, and says sites that opt out of it are not shown in ChatGPT search answers. Perplexity surfaces and links sites with PerplexityBot⁹. Anthropic indexes pages for Claude's search results with Claude-SearchBot¹⁰. Google's AI features in Search draw on pages Google has indexed. A site that blocks one of these bots, or whose host blocks it without the owner knowing, drops out of that one engine's reading list and stays in the others.

Different search partners. OpenAI's help center says ChatGPT may send search queries to Bing¹¹ to return web results for business workspaces. So part of what ChatGPT reads can come from another company's search index.

Different ways of searching. Google documents that AI Overviews and AI Mode may use query fan-out¹², turning one question into several related searches across subtopics. The searches an engine chooses decide which pages it reads. Two engines given the same question can end up reading different pages because they searched for different things.

Different sources cited. Our citation study measures each engine separately: which website it cites most, and how often it shows sources at all. We cover that in which websites AI assistants cite.

Put those together and disagreement is always possible, even when, as in our first run, two engines land on the same business at the top. Each engine is a separate reader with its own library.

What it does not mean

Disagreement does not mean the answers are random, and it does not mean one engine is right and the others wrong. Each answer reflects what that engine found when it searched. A business that tops one engine and is missing from another is telling you something specific: the sources that engine reads describe it well, and the other engine's sources do not.

It also does not mean you can pick one engine and ignore the others. Your customers do not all use the same assistant, and you cannot choose which one they open.

What this means for an owner

Treat each engine as its own channel. A check in ChatGPT tells you about ChatGPT. To know where you stand, you need the same questions asked of each engine, repeatedly, with the answers compared per engine.

Find out which sources each engine reads for your trade in your city, then make sure you are present and accurate on each. Confirm that none of the search crawlers is blocked at your site or your host. Our guide to how the engines find local businesses covers those mechanics in detail.

Measure over weeks, not in one sitting. A weekly per-engine share of voice shows whether a change you made moved one engine, several, or none. That is the report we produce for clients: which engines name you, how often, and which sources they cite. To see a first version of it for your business, start with how to check whether ChatGPT recommends your business.