Google’s AI Overviews: 4 Key Performance Insights

Google’s AI Overviews: 4 Key Performance Insights

My goal with this post is to share some initial performance insights about Google’s new AI Overviews. In short, our initial analysis led to the follow insights:

1. ‘Zero Click Search’ keyword performance is not one size fits all, but it’s largely going to lead to traffic declines and impression increases
2. There’s good value in ranking within AIO citations for queries that invite further reading
3. Google may be right about AIOs driving more clicks than organic web results
4. Appearing past position 2 in AIO citations likely has little to no value

A (very) brief summary of AI Overviews

AI Overviews (formerly referred to as Search Generative Experience) are a Google search feature that uses Generative AI to generate text about a query (as opposed to traditional Google search, which uses information retrieval to match your query with the most relevant information found online). 

Liz Reid has taken us on a wild ride with AI Overviews (AIOs) since she was appointed Head of Google Search in March 2024. We’ve seen varying levels of roll out that seem to be tapering off into a more stable position (for now).

Our position at Seer has been consistent: We want to see the data. It’s unfortunate that Google made the decision not to include data about AIO visibility within Google Search Console. As a result, search marketers have been left seeking alternative tracking sources to get a sense of:

• Which keywords trigger AI Overviews?
• When is my website linked in AI Overview citations?
• When is my brand mentioned in AI Overview content?
• What visibility do my competitors have in AIO content and citations?

After a few months of digging we were happy to connect with the good folks at Ziptie.dev to access this data.

We have lots of analysis left to do at scale, but I haven’t been able to help myself from digging into some of the most interesting data points:

• Do AIO citations really drive more traffic than web results like Google says?
• Is there value in ranking past ‘position’ 2 in the AIO link carousel?
• I believe AIOs are ultimately good for users, but are they good for brands?

This post will share our initial insights from data provided by Ziptie and Google Search Console. Our Google Search Console data compares May 17 – June 17, 2024 vs the previous period (April 16 – May 16, 2024).

In order to get ahead of anticipated reply guys, I’ll provide some caveats:

• Our “after” data set includes varying ranges of AIO exposure: both the initial wide-scale roll out, as well as the subsequent roll back. This likely means our findings are inflated.
• This data is anecdotal and not statistically significant. The only decisions that should be made on this data should be hypotheses to test further
• In my evaluation I am seeing inconsistency in the ability to trigger AI Overviews. In other words, there’s no guarantee that AIOs were triggering for the keywords evaluated all 30 days of the ‘after’ data set
• I’ll be anonymizing the keywords in order to protect the privacy of our clients while including the real performance data
• Finally, a tip: In my research it seemed that Google wasn’t providing query level metrics within Search Console, so my performance metrics are coming from the associated page. This means there’s a mix of web and AIO traffic included in the data.

Okay, let’s get into the fun stuff.

Key Insight 1: ‘Zero Click Search’ keyword performance is not one size fits all, but it’s largely going to lead to traffic declines and impression increases

One of the first keyword groups I dug into were queries where AIOs were present and answered the query very well. In other words, the AI Overview seems to give users all of the information they need – so why would they click?

I presumed users will see everything they need in the AIO and not need to click through. I was mostly right, with some exceptions.

One thing I didn’t think about is the comprehensiveness with which users seem to engage with AIOs. Impressions are largely *way* up for queries that trigger AIOs, and I presume that’s a result of less scrolling through all the various SERP features and instead acceptance of the AI Overview (and its citations) as being enough.

Query 1:

• Modified Query: “phone code in (state)”
• Monthly Search Volume (MSV): 60,500
• AIO Citation Position: 1
• Impact: 5% increase in traffic
• Impact: 18% increase in impressions
• Organic Rank: N/A, but ranked in Featured Snippet

Query 2:

• Modified Query: “phone code for (state)”
• MSV: 14,800
• AIO Citation Position: 1
• Impact: 15% decrease in traffic
• Impact: 6% decrease in impressions
• Organic Rank: N/A, but ranked in Featured Snippet

Query 3:

• Modified Query: “current versions of (software)”
• MSV: 1,000
• AIO Citation Position: 1
• Impact: 58% decrease in traffic
• Impact: 224% increase in impressions
• Organic Rank: 1

Query 4:

• Modified Query: “what is (product) made of”
• MSV: 590
• AIO Citation Position: 2
• Impact: 38% decrease in traffic
• Impact: 50% increase in impressions
• Organic Rank: 1

Further questions to answer:

• At a certain level of MSV (i.e. above 50k), does the exposure tend to result in increased traffic just by nature of the potential for clicks?
• Will a decrease in traffic or impressions for ‘zero click searches’ have any demonstrable impact on conversions?
• Was this traffic ever valuable in the first place?
• How can brands best monitor for branded zero click search AIOs, where the measurable impact may be nominal but the brand integrity is at stake

Key Insight 2: There’s good value in ranking within AIO citations for queries that invite further reading

One of the keyword cohorts I was most interested in were keywords that solve users’ questions and problems in a more robust way than defining a concept or identifying a formula. In other words, the AIO answers the query but there’s more to be said on the topic. 

This is where we’ve long prioritized investing in content strategy at Seer, and I wasn’t sure which way things might go. In the initial data set, this cohort of keywords looks promising.

Query 1:

• Modified Query: “where is (insert automobile brand) made”
• MSV: 5,400
• AIO Citation Position: 1
• Impact: 205% increase in traffic
• Organic Rank: N/A, but ranked in Featured Snippet

Query 2:

• Modified Query: “how much to spend on (entity)”
• MSV: 260
• AIO Citation Position: 2
• Impact: 165% increase in traffic (+81)
• Impact: 150% increase in impressions (+11k)
• Organic Rank: 4

Query 3:

• Modified Query: “(obfuscated) (body part) syndrome”
• MSV: 1,000
• AI Overview Position: 2
• Impact: 44% increase in traffic (+820)
• Impact: 41% increase in impressions (+17k)
• Organic Rank: 3

A good way to think about these queries is that they represent questions users have that can’t be satisfied in an instant. If I want to know what time it is in Hawaii, I can be satisfied with an answer instantly. If I want to know what kind of food Hawaii is known for, my curiosity may not be satisfied by a list, even a comprehensive one. 

A definition of a rare syndrome invites further reading more than a definition of an adjective. A question about how or why to do something invites further reading more than a question about the location or status of something.

Further questions to answer:

• In the Venn diagram of how to rank in AIO Citations, Organic Web Results, and Featured Snippets, what commonalities exist and how do we exploit these for our clients advantage?
• What value, if any, will these impression increases offer brands?

Key Insight 3: Google may be right about AIOs driving more clicks than organic web results

An important caveat here is that I think Google’s AI Overviews do a good job of simplifying the highly Frankenstein-ed SERPs of 2024, and performance increases are likely a direct result of that.

Perhaps an even more important caveat: If you’re ranking as the sole citation in a Featured Snippet, and you move to Position 1 of AIOs, it’s possible you’ll lose traffic. 

But, if you were ranking organically and buried under ads, people also ask results, and a featured snippet: an AI Overview can help you skip to the front of the line in a way that really impacts traffic.

Within our dataset we have a client who is in a space highly impacted by AI Overviews (appearing for more than half of their priority queries) and earning great visibility within those AI Overviews (largely position 1-2 within the citations. 

That client saw a 17% increase in clicks and a 20% increase in impressions period-over-period.

Nothing in marketing happens in a vacuum, and it’s a fool’s errand to try to draw absolute correlations and causations. That said, the numbers are what they are. If I were responsible for a brand’s marketing strategy and was considering investing in attempting to rank better in AI Overviews, this anecdotal data would give me some confidence in exploring that strategy further.

Key Insight 4: Appearing past position 2 in AIO citations likely has little to no value

Based on the data we are seeing, users aren’t clicking through the link carousel. Perhaps if a user was looking for a specific resource or familiar brand, we’d see more clicks and impressions. Generally speaking though, this is ‘page 2 of Google SERPs’ territory. 

This could serve as a good understanding of the level of difficulty for optimizing for different queries’ AIOs. If position 1 and 2 are held by extremely high competition, perhaps we should seek another query to test.

That of course begs the question, what elements represent ‘extremely high competition’? In organic web results we may look at things like domain authority, links, how robust is the content, how well does the content answer the query, etc.

With AI Overviews, we don’t yet know what those elements are. 

Where do we go from here?

This is an incredibly exciting time to be working in marketing. The velocity of change is fascinating, and before AI was in the mix we weren’t exactly complaining about digital marketing being a slow industry.

I think this data starts to point us in the direction of the right answer. Of course in this space the answer will always start with “it depends,” but the fast follow should always be to showcase the data the answer depends upon.

From what I am seeing: 

• I do believe that AIO citations drive more traffic than web results.
• I don’t believe that there’s any value in ranking past ‘position’ 2 within the AIO link carousel.
• I think AIOs can be as good for brands as they are for consumers. Especially brands that are good to consumers.

We in the SEO industry have a habit of getting high off our own stench from time to time, and it seems we are falling victim to that habit once more. It makes sense. It’s a lot easier to talk about work than to work. But I firmly believe this is a time to work. Analyze the data. Optimize the content to solve users’ problems. Identify the workflows by which your team can get more done. If this truly is a period of Great Reset, shouldn’t we all be trying to get the head start we always wished we had?

Marketers who want to work must start with the data. Work with a provider like Ziptie.dev to analyze the visibility of AI Overviews in your space and identify if they are a priority to explore. If you find they are a priority to explore, start testing with your content and data. Or, partner with a consultant like us to get started.