A repeatable system for tracking whether your brand shows up when buyers ask AI (and whether that’s getting better or worse).

In our last post, we walked through a self-audit to see what LLMs actually know about your brand. If you ran it, you probably came away with a spreadsheet full of screenshots and a gut feeling… but gut feelings don’t survive a leadership meeting. “We seem to show up more in Claude than ChatGPT” isn’t a metric, and it won’t tell you whether the content you published last month did anything.

This guide turns the audit into a scorecard: a repeatable measurement system you can run every quarter, defend with data, and use to decide where to invest next.

Why One Audit Isn’t Enough

AI answers are not stable. Ask the same prompt twice and you will get two different lists of vendors. Ask it from a different account and the answer will change again, because of memory, personalization, location, or whether search was switched on. Model updates shift things further.

That variance means a single run is an anecdote. Eight vendors named in one answer and five in the next doesn’t mean anything changed about your brand. It means the system is probabilistic.

A scorecard solves this by sampling: more prompts, multiple runs, consistent conditions, and metrics that measure rates instead of single outcomes.

Step 1: Design Your Prompt Set

Aim to put together a list of 30 to 50 prompts. Fewer than that and one odd result skews everything; many more and the manual work becomes unmanageable.

Build the set as a matrix across 3 dimensions:

Funnel stage:
1. Problem: “how do teams reduce onboarding time?”
2. Solution: “what types of tools handle this?”
3. Vendor: “best platforms for…”
4. Decision-stage: “X vs. Y,” “alternatives to X,” “is X worth it?”
Persona: the different members of your buying committee. A CMO, a RevOps lead, and a procurement manager do not prompt the same way.
Prompt type: category shortlists, use cases, comparisons, integration or stack questions, and branded queries

Three rules make the set useful:

Write prompts the way buyers actually talk. Real prompts are longer and carry context: “I’m a VP of Marketing at a 200-person SaaS company with a small ops team. What should we use to…” Pull language from sales call transcripts, discovery questions, support tickets, and community threads rather than inventing it.

Keep most prompts unbranded. If your company name is in the prompt, you are testing recall, not discovery. Discovery is where deals are won or lost, so most of your set should be prompts where you have to earn the mention.

Freeze the set, then version it. Add prompts over time, but don’t reword old ones. Otherwise you can’t compare this quarter to last.

“I’m a VP of Marketing at a 200-person SaaS company with a small ops team. What should we use to…”

Step 2: Run Every Prompt Multiple Times

Run each prompt three to five times per AI platform. Keep conditions clean and consistent where possible:

A fresh session for every run
Memory and personalization off or use a clean test account
Search on for one pass and off for another, where the platform lets you (this separates what the model learned in training from what it retrieves live)
The same platforms every cycle. Cover ChatGPT, Claude, Perplexity, and Gemini, then weight them by where your buyers actually do their research

Log the date, platform, model version (if shown), and whether search was on. Those details will matter in six months when results shift and you’re trying to work out why.

Step 3: Score What You Find

For each run, capture a handful of fields, then roll them up into rates.

Metric	What It Measures	How to Calculate
Mention Rate	How often you appear at all	Runs with your brand ÷ total runs
Citation Rate	How often your domain is cited as a source	Runs citing your site ÷ runs with citations (retrieval-enabled platforms only)
Prominence	Where you appear	First named, top three, or buried
Share of Voice	Your presence relative to competitors	Your mentions ÷ all tracked-brand mentions
Accuracy	Whether the description is right	0 = wrong, 1 = vague, 2 = accurate
Framing	How you’re positioned	Leader, alternative, budget option, niche, or absent

Two more things worth logging that most teams skip: which third-party sources get cited when you are absent, and what the model says about competitors that it doesn’t say about you. Those two columns turn the scorecard from a report card into a roadmap.

Step 4: Build the One-Page Scorecard

Remember that if leadership can’t read it in two minutes, it won’t get used, so keep the output to one page:

Headline number: overall share of voice, with the change vs. last cycle
Breakdowns: by funnel stage, by persona, and by platform
Top five gaps: prompts where competitors appear and you don’t
Top five sources: the third-party domains most often cited in your category
Accuracy flags: anywhere AI is getting your story wrong

The breakdowns are where the insight lives. A healthy overall number can hide a collapse at the decision stage, or total absence for one persona.

Step 5: Set a Cadence and Know Your Noise Floor

Run your 10 to 15 highest-stakes prompts monthly and the full set quarterly. Rerun after major changes: a repositioning, a big content launch, a new product line.

Before you celebrate or panic over any movement, find your noise floor. Run the full set twice back to back and compare. The difference between those two runs is your baseline variance, and any change smaller than that isn’t a signal. Look for movement that holds across two or more cycles.

Tools vs. Spreadsheets

A spreadsheet is enough to start and it forces you to read the actual answers, which is where most of the learning happens. Once you pass 50 prompts or need multi-platform runs at scale, dedicated AI visibility tracking tools can save real time.

If you evaluate one, ask how it collects data. Tools that call a model’s API and tools that capture what the consumer product displays can return different answers, and neither is necessarily “the truth.” Make sure the method matches how your buyers actually search.

Common Mistakes

Cherry-picking prompts you already win. A flattering scorecard is a useless one.
Counting any mention as a win. Being listed as “also consider” isn’t the same as being the recommendation.
Ignoring wrong answers. An inaccurate description in front of a buyer is worse than absence.
Treating it like a ranking. There’s no position #1 to chase. You’re measuring the probability of being included.
Measuring without acting. A scorecard with no owner and no follow-up is a dashboard nobody opens.

Turning the Scorecard Into Action

Each pattern points to a different fix:

Absent from category prompts: you have an authority problem. Prioritize earned mentions and non-commodity content.
Present but described wrong: you have a consistency problem. Align your site, profiles, and third-party listings.
Strong in some personas, missing in others: you have a coverage problem. Build content for the missing roles and their questions.
Competitors cited from sources you’re not in: you have a distribution problem. Those sources are your targets.

The brands that win in AI search won’t be the ones that ran an audit once. They’ll be the ones that measured, acted, and measured again.

Tois helps B2B companies measure, build, and grow their AI visibility. Start with an AI Visibility Audit and we’ll build your baseline scorecard.

How to Build an AI Visibility Scorecard