Methodology and measurement

Published so you can check our work.

Every company in this market will show you a number. Very few will tell you how it was produced, how often, with what sample size, or what it cannot prove. This page is the whole method, including the parts that are inconvenient for us. If a vendor you are speaking to cannot produce a page like this, that is worth knowing before you sign anything.

On this page

What we measure

A buyer conversation, not a prompt. Each conversation is three turns: the discovery question, the objection question and the decision question. All three are asked of every engine in the set, as a continuous exchange rather than three separate queries, because an engine behaves differently once it has already answered you once.

Unit

One buyer conversation, three turns

Default scope

40 conversations per account

Engines

ChatGPT, Google AI Overviews, Perplexity, Gemini, Claude, Copilot

Samples

5 per engine, per turn, per cycle

Cycle

Monthly by default, weekly on the top twenty at higher scope

Volume

3,600 measured answers per cycle at default scope

We record four things from every answer: whether you were named, in what position, the verbatim sentence used about you, and which sources the engine cited to build it. The verbatim sentence is the part that matters. A mention count cannot tell you that an engine is describing you as the cheap option.

How questions are chosen

Your forty conversations are proposed by us from the vertical question library, then agreed with you in writing before any baseline is measured. We do not pick questions we already know you win, and you can refuse any of them.

The set is versioned and dated. Changing it after the baseline requires written agreement from you, and the version number appears on every report. This is the single easiest thing for a vendor to manipulate, which is why it is the first thing we fix in place.

Sampling and repetition

Generative engines are not deterministic. The same question asked twice can return different companies, so a single query proves nothing. We take five samples per engine per turn and treat the result as a distribution rather than a fact.

Samples run on a fixed schedule rather than when a report is due, from consistent geography and language settings, with no personalisation or signed in account carried between runs. Where an engine supports a region parameter we set it to your market and state which.

Why engines are never blended

The six engines rarely agree. They quote different sources and often recommend different businesses for the same question. Averaging six engines into one visibility score destroys exactly the information you need in order to act.

So we never produce a single blended number. Every row shows six engines separately, and where a board level figure appears it is a count of engines rather than an average of scores.

Named, recommended, cited and displaced are four different things

Most tools collapse these into one word, usually mention. We separate them, because they have different causes and different fixes.

Named

Your brand appears in the answer at all

Recommended

Named with a reason attached that favours you

Cited

A page you own is listed as a source for the answer

Displaced

A competitor is named with the reason that should have been yours

A row where you are named but a competitor is recommended is Contested, not Won. That distinction is the reason the board exists.

The survival metric, defined

Survival is the average number of turns, out of three, in which a brand is still named across the engine set.

Numerator

Sum of turns in which the brand is named, across all engines and samples

Denominator

Total engine and sample pairs measured for that conversation

Range

0.0 to 3.0

Eligibility

A conversation counts only when all three turns completed on at least five of six engines in that cycle

Row level

Survival for one conversation

Board level

Mean survival across all eligible conversations

We believe this is a measurement nobody else currently produces, because computing it requires measuring multi turn conversations rather than single prompts. We say believe rather than know, because we cannot audit the internals of every competitor product, only what they publish.

When a change counts as movement

Every figure carries a confidence interval derived from the sample distribution. A change is only reported as movement when the intervals of two measurements do not overlap. Anything inside the noise is reported as flat, in writing, with the numbers shown.

This has a consequence we accept deliberately. Some months we will tell you nothing happened. If two consecutive months come back flat we will recommend you stop, before you have to ask. We would rather lose a fee than hold one quietly.

Model updates and how we mark them

Engines ship model changes without announcement, and a change can rewrite how a whole category is described overnight. We stamp every measurement cycle against the model releases in that period, and when a drop coincides with one we mark it on the chart rather than smoothing it out.

This means some of your charts will have an explained dip in them. That is the honest shape of this data, and a chart with no dips in it should make you suspicious of whoever produced it.

Verifying that work actually shipped

Nothing enters the work log when it is published. It enters when it is verified, which means we fetched the URL afterwards and confirmed three things: the content is live, the structured data parses, and the crawlers the engines use can reach it.

A page that published behind a robots rule did not publish. A page with structured data that fails to parse did not ship. We have had both happen, which is why the check exists.

What we will not claim

This list is here because it is the part of a methodology page that usually goes missing.

Things this measurement cannot prove

That a board movement caused a sale. AI referral traffic largely arrives untraceable, and anyone claiming a clean attribution chain from an AI answer to revenue is overstating what the data supports.
That your result will match another client's. Categories, competitors and source landscapes differ enormously.
That a specific action caused a specific movement, where more than one action touched the same row in the same window. We mark those as contributing rather than causing.
That an llms.txt file improved anything. We will put one in place because it costs ten minutes, but no major provider uses it in production and Google has said it does not support it. It will never appear on your invoice.
That we measure every engine. We measure six. Grok and others are evaluated but not yet in the set, and we will say so rather than imply full coverage.
That a ranking or a citation can be guaranteed. Anyone offering that in this market is selling something they do not control.

If you are comparing vendors, the thirteen questions worth asking all of them are here, including us.

Movement, with the uncertainty left in

A number without a range is a sales pitch.

The six engines rarely agree. They quote different sources and often recommend different businesses for the same question, which is why we never blend them into one score. An average hides exactly the thing you need in order to act.

Engines answer the same question differently each time, so we sample repeatedly and plot the spread. When this month’s band overlaps last month’s we call it flat and say so. The dip below is real and it stayed in the chart.

Rows won, out of forty

Rows won

Confidence range

Illustrative

August went backwards because an engine changed its model mid month. We stamp every board against the model releases in that period, so a drop gets explained rather than argued about.

Run this method against your own category.

A free AI visibility snapshot, measured exactly as described above and yours to keep either way.