You've accepted that AI search visibility matters. You know, in principle, that you should be tracking whether ChatGPT and Claude mention your brand when a buyer asks for a recommendation. So you open a fresh chat, type in a question from your own category, read the answer, and... now what? One good or bad answer tells you almost nothing. The question that actually matters is how to turn "I asked once and it went okay" into a number you can trust and watch move over time.
This is the method we'd use, whether you're doing it by hand with a spreadsheet or with a tool built for it.
Start with the questions, not the metric
Before you can measure anything, you need a fixed, repeatable set of prompts, because a single ad hoc question tells you about that question, not about your category. Write down every way a real buyer in your space would open a conversation with an AI assistant: "what's the best tool for X", "compare A and B for Y", "I need something that does Z on a tight budget". Aim for at least ten to fifteen of these, covering the different buyer types and budgets you actually sell to, not just the single head-term query that's easiest to write.
Keep the wording close to how buyers actually talk, not how your product team talks. A buyer says "I need to keep an eye on what people say about us on Reddit and X without paying for enterprise software", not "social listening solution evaluation criteria". If you're not sure how buyers phrase it, read your own sales call notes or support tickets before you write the prompt list; the language is usually sitting right there.
This fixed prompt set is what makes the rest of measurement possible. Change the questions every time and you're comparing noise to noise.
Run it across models, not just one
A single AI assistant is not a representative sample. ChatGPT, Claude, Perplexity and Gemini draw on different training data, different retrieval setups and different web search behaviour, so the same prompt can get a completely different answer depending on which one you ask. We've seen the same brand appear in every answer from one model and barely register in another, in the same week, for the same question.
Run your prompt set across at least two or three assistants, and keep the results separate by model rather than averaging them together. A strong showing in Perplexity and a weak one in Claude is a real finding, not noise to smooth over, and it tells you something specific: Perplexity leans on live web search, so it rewards fresh, well-indexed pages; a weaker Claude result might mean the underlying training data about you is thin or stale.
The four numbers, and how to actually calculate them
The AI visibility pillar names four metrics. Here's what running them in practice actually looks like.
- Appearance rate. Out of every prompt-and-model combination you run, count the share where your brand gets a mention at all, anywhere in the answer. If you run 12 prompts across 3 models, that's 36 conversations; if you appear in 20 of them, your appearance rate is 56%. Track it per model as well as overall, because a healthy blended number can hide one model where you're essentially invisible.
- Share of AI voice. For every conversation where any brand in your category gets named, note every competitor that's mentioned alongside you, then calculate what fraction of the total mentions are yours. This is the AI-answer equivalent of share of voice in social listening, and it's the number that tells you whether you're actually gaining ground or just holding steady while the whole category grows.
- Recommendation rate. Being mentioned and being recommended are different outcomes, and conflating them is the most common measurement mistake we see. Read to the end of each answer and record whether you were named as a runner-up, named as the top pick, or the one the AI actively steered the buyer away from. Only the middle one counts as a genuine win.
- Sentiment. For every mention, note in plain words how you were described: unreserved, hedged with a caveat, or actively negative. Write the exact caveat down verbatim ("good for small teams but pricier at scale", say), because that sentence is the raw material your content team needs to go and fix the underlying gap.
Follow the conversation, not just the opening answer
The biggest gap between a quick manual check and real measurement is that most people only ever read the first answer. Real buyers don't stop there. Watching actual AI buyer journeys shows the pattern clearly: a buyer asks a broad opening question, gets a shortlist, then adds a constraint the AI didn't have yet, a budget, a compliance requirement, a specific integration, and the shortlist changes.
If your measurement only covers turn one, you're scoring the easy part of the conversation and missing the part where deals are actually won or lost. Where you can, carry the conversation two or three turns deep with a realistic follow-up ("does it handle GDPR", "what's the cheapest plan that includes X"), and record whether you survive the follow-up as well as the opening question.
Judge winnability before you chase a query
Not every prompt is worth measuring forever. If a query is dominated by a handful of large, well-established competitors across every model you check, a small or newer brand is unlikely to break in soon regardless of what content you publish. That's still useful to know, it tells you to deprioritise that prompt and put your energy into questions where the field is more open, typically the narrower, more specific ones ("best tool for X on a tight budget", "which platform handles Y") rather than the broadest category term.
Set a cadence and stick to it
A one-off measurement round tells you where you stand today. It says nothing about whether you're improving, because model behaviour shifts as vendors update training data, retrieval and web search, sometimes within weeks, with no announcement. Re-run your full prompt set on a fixed schedule, weekly is a sensible starting cadence, and log every run rather than overwriting the last one, so a dip or a rise shows up as a trend instead of a surprise.
Benchmark your named competitors on the exact same schedule. A flat appearance rate for you that coincides with a rising one for a competitor is a different story to a flat rate across the board, and you'll only see the difference if you're tracking both.
Mistakes that quietly wreck the numbers
A few patterns we'd watch for, because they make a measurement look more solid than it is:
- Changing the prompt wording between runs. Even a small rewording can shift which brands an AI reaches for, so a change in your score might just be a change in your question.
- Testing only your own name. If you ask "tell me about Babel42" instead of a genuine buyer question, you're not measuring visibility, you're checking whether the model has heard of you.
- Averaging across models. A blended score across four assistants hides exactly the model-by-model gap that tells you where to focus.
- Stopping at the opening answer. As above, the real signal is often in how the shortlist holds up once a constraint gets added.
Where to go from here
Once you have a fixed prompt set, a repeatable way to score appearance rate, share of AI voice, recommendation rate and sentiment, and a regular cadence to re-run it, you have a genuine baseline rather than a gut feeling. The next question is what to actually do with a weak score, which we cover in our playbook for improving AI search visibility.
Doing this by hand across a dozen prompts and three or four models, every week, is a lot of repetitive manual work, which is exactly why we built Babel42's AI Visibility product to run it automatically: structured AI Buyers work through real, multi-turn shopping journeys across the assistants your buyers use, and the appearance rate, share of AI voice, recommendation rate and sentiment come out the other end as a tracked trend rather than a one-off spreadsheet. The free plan runs one AI Buyer across Perplexity and ChatGPT on a weekly cadence, enough to start building your own baseline before you commit to anything.
