How to use AI response patterns to build better content

The last year has had many of us trying to understand how to report on AI visibility and understand what it takes to be seen and cited by AI.

But Rand Fishkin’s latest study on AI response variability has emphasized that LLM outputs aren’t as stable and predictable as search rankings, making this KPI an inconsistent piece of the puzzle.

The study found there’s less than a 1 in 100 chance that ChatGPT or Google AI will return the same list of brands across two responses. They analyzed thousands of prompts across multiple LLMs to highlight just how varied they are.

This has left some of the SEO community questioning the value of rank tracking at scale. But, rank tracking is far from useless. It’s just misapplied.

AI response tracking is an unstable performance KPI in its current state, but it becomes extremely powerful when used as an analysis tool to inform content strategy.

Let’s take a look at why you should still be investing in prompt tracking and how it can be used to inform your content strategy.

Why AI visibility tracking is unstable (for now)

LLMs aren’t deterministic ranking engines. They’re probabilistic language models that can gather and synthesize information from their own training data or live searches. These models use context windows and understanding of intent to serve different answers at any moment.

We’ve seen that responses change based on the prompts, and we know that the same question can be written in so many different ways, which opens the door for your CMO to question why you’re not showing up for a specific prompt when they just saw your brand mentioned or cited.

Tracking visibility remains an area of uncertainty until there’s greater clarity on user prompting. But it’s still valuable.

If prompt response tracking isn’t a stable KPI, then what is it? It’s pattern analysis, something SEOs are very familiar with.

Instead of only focusing on whether or not you are cited or listed, you should be trying to understand:

• How is the prompt response structured?
• What concepts repeatedly appear?
• What key phrases or terms are showing up?
• What level of nuance is typically included?

This requires a mental shift.

Dig deeper: 7 hard truths about measuring AI visibility and GEO performance

Traditional SEO vs. AI pattern analysis

In traditional SEO, we reverse engineer what’s already ranking. With AI search, we can apply the same thinking by reverse engineering the patterns we see in results.