Google AI Updates: Personalization, Crawl Limits & Traffic Impact – SEJ News
Google has expanded access to its AI-powered Personal Intelligence feature, now making it available to all free users in the United States, a move that is poised to significantly alter how search results are presented and perceived. The feature, previously limited to paid subscribers of AI Pro and Ultra plans, integrates information from users’ Gmail and Google Photos accounts to personalize responses within AI Mode, the Gemini app, and Chrome.
The expansion, announced this week, allows AI Mode to draw context from email confirmations, travel bookings, and stored photos when responding to user queries. While currently limited to the US and excluding Workspace accounts, the rollout represents a substantial increase in the user base exposed to AI-driven personalization, raising questions about the consistency of search results across different individuals. According to Google, the Gemini app and Chrome rollouts are currently underway.
Alongside the expansion of Personal Intelligence, Google has clarified its approach to web crawling, revealing that the commonly cited 15-megabyte limit for Googlebot crawlers is not a rigid ceiling. Gary Illyes and Martin Splitt of Google Search have indicated that internal teams can override this limit, and that Google Search operates with a more practical 2-megabyte threshold. This flexibility means that pages with heavier content – including extensive scripts, data objects, or embedded media – may be affected, though most pages remain well under the 2MB limit.
The changes come as data emerges highlighting the impact of Google’s AI Overviews on organic search traffic. An analysis by SISTRIX of over 100 million German keywords revealed a 59% decrease in the click-through rate for the top organic position when an AI Overview is displayed, falling from 27% to 11%. SISTRIX estimates this translates to 265 million lost organic clicks per month in Germany alone, representing a 6.6% overall click loss across all keywords. This data mirrors earlier findings in the US, suggesting a consistent pattern of AI Overviews diminishing organic search visibility.
The impact is not evenly distributed. New data from Chartbeat indicates that smaller publishers are experiencing significantly steeper declines in search referral traffic than larger ones. Over the past two years, small publishers have lost 60% of their search referral traffic, compared to 47% for mid-sized publishers and 22% for large publishers. Larger publishers are partially mitigating these losses through increased direct traffic, email marketing, and app referrals. Google Discover referrals have also fallen, decreasing by 15% over the same period.
While ChatGPT referrals have grown by over 200% in the Chartbeat data, they still represent less than 1% of publisher page views, demonstrating that chatbot traffic is currently insufficient to offset the losses from search. Industry observers are noting the implications of these trends. Barry Adams, founder of Polemic Digital, stated on LinkedIn that citations within AI Overviews appear to have little impact on click-through rates, emphasizing the require for publishers to offer content that AI cannot easily replicate, such as breaking news. Steven Waldman, founder of Rebuild Local News and Report for America, described the Chartbeat data as “incredibly important,” highlighting the greater resilience of larger publishers due to stronger brand recognition and direct-to-consumer strategies.
These developments collectively suggest a shift in the dynamics of online information retrieval, where general benchmarks are becoming less reliable. The effectiveness of SEO strategies, the relevance of crawl limits, and the consistency of AI Mode results are all increasingly dependent on specific contexts – vertical, site size, and audience. The data underscores the need for a more nuanced understanding of how these changes are impacting different segments of the online ecosystem.