The Death of Keyword Strategies: Our Study Showed 93% of Keywords Failed to Map to AI Searches

This study suggests that over 90% of traditional keywords don't map into AI search. Here's what businesses should do about it.

By Sean Fulford · 13,086 keywords / 301 LLM searches analyzed

For years, keyword research has been the foundation of SEO. Find what people search for, build content around those searches, and capture the demand.

But what happens when the search engine itself starts deciding what to search for?

We wanted to understand just how much traditional keyword research carries over into the world of AI search. More specifically, how closely do the searches generated by LLMs resemble the keywords people search for on Google?

The results were pretty alarming.

Across four datasets, 93.16% of traditional keywords had no close semantic counterpart among our sampled LLM-generated searches, using a cosine similarity threshold of 0.80.

Interestingly, the inverse was not true. A much larger proportion of LLM searches were represented within traditional keyword datasets.

Let's dig into what we found.

How We Conducted the Study

We used DataForSEO to collect traditional Google keyword suggestions and searches generated by AI, grouping both datasets around the same seed term (we pulled every term and prompt for each seed).

We then embedded the searches using OpenAI's text-embedding-3-small model and calculated cosine similarity to measure their semantic relationships.

We examined two things:

1. Cosine distribution overlap: How similarly are traditional keywords and LLM searches distributed in semantic proximity to their shared seed keyword?
2. Cross-dataset semantic coverage: What percentage of searches in one dataset have a close semantic counterpart in the other?

For semantic coverage, we primarily used a cosine similarity threshold of 0.80, with some additional testing at 0.90.

It's worth noting that cosine similarity measures semantic proximity, not necessarily identical search intent. Our results therefore measure representation within the sampled search datasets rather than actual changes in search volume.

Findings for Blue Shoes: The First Eye-Opener

Our first analysis began with a simple niche search for "blue shoes" while we were testing the tool. And we were immediately alarmed.

We pulled traditional Google keywords containing "blue shoes" from DataForSEO and compared them against LLM-generated searches associated with the same seed. The initial finding was pretty staggering.

Only 7.5% of traditional keywords had an LLM search counterpart with a cosine similarity of 0.80 or higher.

In other words, 92.5% of the traditional keywords had no sufficiently close semantic match within the sampled LLM searches.

That's an enormous difference between the traditional keyword universe and what the LLM was actually searching for.

Full corpus: 48 LLM searches × 2,626 keywords. Coverage depends on the threshold and measures embedding similarity, not verified identical search intent.

When we increased the cosine similarity threshold to 0.90, things became even more dramatic: only 0.5% of traditional keywords had a close semantic match.

Ironically, and perhaps a little sadly for traditional keyword research, the inverse was not true. When we compared LLM searches against traditional keywords, 68.8% of LLM-generated searches had a traditional keyword counterpart with cosine similarity of 0.80 or greater.

So what does this mean? It suggests that while most traditional keywords were not represented among the sampled LLM searches, a substantial portion of LLM searches were already represented within the traditional keyword universe.

Put another way, LLM-generated searches may be exploring a much smaller portion of the semantic territory represented by conventional keyword research.

And if that holds across industries, it has some pretty significant implications for how we approach SEO and GEO.

Scaling Up to Baseball Caps to Make Sure We're Not Crazy

Naturally, we wanted to make sure blue shoes wasn't an anomaly. So we repeated the process with "baseball caps." Remarkably, we saw a very similar pattern.

Only 3.6% of traditional keywords had an LLM search counterpart with cosine similarity of 0.80 or greater. Meanwhile, 60.7% of LLM searches had a sufficiently close match in the traditional keyword dataset.

Full corpus: 56 LLM searches × 5,470 keywords.

Once again, more than 90% of traditional keywords were not represented within our sampled LLM searches at the selected similarity threshold. Yet a majority of the LLM searches were represented within the traditional keyword universe. This was beginning to look less like an anomaly and more like a recurring pattern.

What About Service-Based Searches?

Products are one thing, but what about services? To investigate, we looked at searches related to HVAC companies. Even within a service-based category, the pattern appeared to hold.

Only 8.6% of traditional keywords had an LLM search counterpart with cosine similarity of 0.80 or higher. Meanwhile, 34.8% of LLM searches had a sufficiently close counterpart within the traditional keyword dataset.

Full corpus: 92 LLM searches × 784 keywords.

The proportion of LLM searches represented by traditional keywords was noticeably lower than in our previous examples. However, the broader finding remained consistent: very few traditional keywords had close semantic counterparts within the LLM searches we collected. This suggests that the relationship between traditional SEO and LLM-generated searches may vary by industry, even if the broader pattern is similar.

Checking Out Informational Searches: Quantum Physics

Lastly, we wanted to make sure we had a reasonable pulse check across different types of search behavior. So we examined an almost purely informational topic: quantum physics. The results were slightly different, but the same general pattern emerged.

Only 11.2% of traditional quantum physics keywords had a close semantic counterpart within the LLM-generated searches. Meanwhile, 54.3% of LLM searches had a traditional keyword counterpart with cosine similarity of 0.80 or greater.

Full corpus: 105 LLM searches × 4,206 keywords.

Even within an informational category, traditional keyword datasets contained substantially more searches without close counterparts in the LLM sample. At this point, we had observed the same directional relationship across four different datasets.

Getting a Bit More Sciencey: Looking at the Confidence in Our Data

Across our four datasets, we analyzed 13,086 traditional keywords and 301 LLM-generated searches. Despite differences between the categories, the pattern was remarkably consistent.

Across the 13,086 traditional search keywords, 93.16% had no semantically similar counterpart among sampled LLM-generated searches at a cosine similarity threshold of 0.80 (99% CI: 92.57%–93.71%).

That means only 6.84% of traditional keywords had a close semantic match in the sampled LLM search universe. However, this did NOT hold true in the opposite direction.

Across 301 LLM-generated searches, 52.82% had a semantically similar counterpart within traditional keyword datasets at the same threshold (99% CI: 45.43%–60.09%).

In other words, more than half of the sampled LLM-generated searches were already represented in traditional keyword data, while fewer than 7% of traditional keywords were represented in the LLM datasets. That's a pretty dramatic asymmetry.

One important consideration: our LLM datasets were substantially smaller than the traditional keyword datasets. This naturally affects the probability of finding close matches, so these results should be interpreted as preliminary evidence rather than proof that 93% of actual search demand has disappeared. The confidence intervals also assume independent observations — because searches within the same topic may be correlated, the actual uncertainty could be greater. Nevertheless, the consistency of the directional differences across our samples gives us a compelling reason to investigate further.

Caveats and Limitations: What Our Data Doesn't Tell Us

As interesting as these findings are, there are a few important limitations worth addressing before we start declaring the death of traditional keyword research.

First, our LLM search samples were considerably smaller than our traditional keyword datasets — however, this may be a symptom of a smaller search universe in general. Across the four datasets, we examined 301 LLM-generated searches against 13,086 traditional keywords. Naturally, a larger dataset has more opportunities to contain a close semantic match. This likely contributes to the difference between our LLM-to-keyword and keyword-to-LLM coverage percentages. It also means we cannot confidently conclude that 93% of traditional search topics have disappeared. What we can say is that 93.16% of the traditional keywords had no sufficiently close counterpart within the LLM searches we sampled.

Second, cosine similarity is not a perfect representation of search intent. We used OpenAI's text-embedding-3-small model and a cosine similarity threshold of 0.80 to determine whether two searches were semantically close. However, a cosine score of 0.80 doesn't necessarily mean two queries have identical search intents. Likewise, two searches falling below that threshold might still address substantially the same underlying information need. The threshold is a methodological choice, not a universally established boundary between matching and nonmatching searches.

Third, we're examining a limited number of topics. Our initial research covered four categories spanning products, services, and informational searches. The consistency of the results is interesting, but four categories are not enough to establish that this pattern applies universally across search. Larger datasets covering more industries, particularly highly specialized and commercially competitive verticals, would help us understand how broadly these findings apply.

Fourth, our datasets are not necessarily representative of all search behavior. We sourced traditional keyword suggestions and AI searches through DataForSEO, selecting queries associated with shared seed terms. These datasets offer a useful comparison, but they aren't a complete record of everything people search for on Google or everything LLMs search for. Our analysis also measures query representation, not actual search frequency or traffic-weighted demand. A keyword with almost no search volume is treated the same as one searched thousands of times.

Fifth, our confidence intervals come with assumptions. Our reported 99% confidence intervals assume that individual observations are independent. In reality, searches within the same category can be highly correlated, which means our confidence intervals may be narrower than they would be under a model accounting for topic-level clustering. We also need additional independent topic samples to establish how much these results vary across the broader search landscape.

Finally, we're measuring semantic correspondence, not the disappearance of search demand. This is perhaps the most important distinction. Just because a traditional keyword doesn't have a close counterpart among sampled LLM searches doesn't mean people have stopped searching for it, or that the underlying information no longer exists. It means that particular keyword wasn't closely represented within the LLM-generated search sample at our chosen similarity threshold.

With all that said, the findings are still worth paying attention to. Across four different datasets, we observed the same broad pattern: traditional keywords had relatively little coverage within sampled LLM searches, while a much larger share of LLM searches had traditional keyword counterparts. We aren't claiming to have conclusively mapped the entire AI search universe. But we do think we've uncovered an interesting pattern that warrants considerably more investigation, particularly as businesses begin shifting resources from traditional SEO toward GEO.

What Does This Mean for Businesses?

I think the biggest takeaway is that traditional keyword strategy and AI search strategy are not interchangeable. The old assumption that you can feed two birds with one scone by simply applying your existing keyword research to AI search may not hold up.

Our data suggests that a large portion of the traditional keyword universe is not directly represented in the searches LLMs generate. That doesn't mean traditional SEO is dead — far from it. In fact, the finding that more than half of LLM searches had traditional keyword counterparts suggests conventional keyword research still captures a meaningful portion of the AI search landscape. But it does suggest businesses need to think differently about how they identify and prioritize search opportunities.

First, businesses should start examining the actual searches LLMs generate. Rather than assuming traditional keyword research tells the whole story, analyze the queries AI systems use when researching products, services, and informational topics.

Second, content strategies should increasingly consider semantic topics and search intent rather than individual keyword variations. If AI systems are concentrating their searches around a narrower collection of topics, creating separate content for every conceivable keyword variation may provide diminishing value in the AI search environment.

Third, businesses should measure the semantic overlap between their traditional keyword strategies and LLM search behavior. Understanding which existing content topics map into AI-generated searches, and which don't, could help identify where additional investment is worthwhile. Ultimately, this isn't necessarily about producing more content. It may be about producing content that better represents the topics and information AI systems actually retrieve.

Conclusion: The Search Universe May Be Shrinking

For more than two decades, SEO has been built around understanding the enormous number of ways people search. Different keywords, different variations, different modifiers, and different intents. But AI search introduces a fundamentally different dynamic. Instead of a person constructing every search query, an LLM can interpret a request and generate its own searches to retrieve the information it needs. And based on our preliminary findings, those generated searches may represent a substantially smaller portion of the traditional keyword universe than many of us would expect.

Across four datasets, more than 93% of traditional keywords had no close semantic match within our sampled LLM searches. Yet more than half of LLM searches had a close traditional keyword counterpart. To me, that's the most interesting finding. It suggests that AI search may not be introducing an entirely new universe of search topics. Instead, it may be selecting and reformulating a smaller subset of the semantic territory that traditional keyword research already captures.

There is still more work to do. Larger LLM query samples, additional industries, and further validation of semantic similarity thresholds will help determine how consistently this pattern holds. But if these findings continue to replicate, they raise a pretty uncomfortable question for our industry:

What happens to keyword strategies built around thousands of search variations when AI systems only explore a fraction of that territory?

The answer may force us to reconsider not only how we approach GEO, but how we think about search demand itself.