Google Is Using AI to Censor Independent Websites Like Mine

6 days ago 4

[Note: What follows is a blog-post formatted version of the 34 page letter I submitted to the FTC on May 21, 2025 in response to the agency’s RFI entitled: “Request for Public Comment Regarding Technology Platform Censorship.” I have also added a section at the end on what you can do to save the open web from Google’s AI takeover.]

Introduction:

I am an American citizen and founder of the Florida company Travel Lemming LLC. Travel Lemming runs a website, TravelLemming.com, that provides online travel advice. 

In this letter I will explain how Google censored my travel website – and thousands of small and independent web publishers like me – all so that it can use AI to control the flow of information online. 

In late 2023 and early 2024, Google released an unprecedented series of algorithm updates that utterly decimated thousands of independent websites. 

Travel Lemming lost more than 95% of our Google organic search referral traffic in these updates. 

The shadowban algorithm that hit us was supposedly based on the content on our websites. But we later realized the shadowban really was about the type of website we are (i.e., small and independent). 

While Google gives large publishers an appeal and recovery process, small and independent publishers have no path to appeal our shadowbans. 

This is true even though Google admitted our shadowbans are its fault and not ours.

In fact, last October, Google even flew me and 19 other publishers out to its headquarters for a tour, an admission of wrongdoing, and an apology. 

Though Google apologized, it also said that search has permanently changed with AI and thus our traffic may never return. 

In this letter I will describe how I believe Google has been laying the groundwork for a grand plan to rethink search from the ground-up so as to profit from AI. 

Google isn’t satisfied with its monopoly on the questions we search. 

Google wants to use AI to monopolize the very answers themselves. 

As one Google executive recently explained: “Organizing information is clearly a trillion-dollar opportunity, but a trillion dollars is not cool anymore. What’s cool is a quadrillion dollars.”

Google plans to use AI to consume and replace the open web. 

I believe demolishing independent sites like mine was Google’s first step in clearing ground so it has space to rebuild search from the ground up for an “AI-first” future. 

Google envisions a future where “Google does the Googling for you,” its AI and ads do the answering – and users never need to leave Google. 

Google will just source information from a handful of sources and partner websites that it controls and selects – effectively creating an information cartel. 

If Google can use AI to censor a travel website from the web arbitrarily and without opportunity for appeal – it can do the same to any source of information it wants. 

And American citizens and Internet users everywhere will be worse off for it. 

So while you may not really care about the plight of some random travel website getting censored, everyone should care about the way Google is deploying AI to build a censorship cartel that lets it control the flow of information online. 

What follows is a lengthy summary of my experiences and my opinions as an independent publisher trying to survive in a monopolist’s information economy. 

To start, let me explain how we got to this point where Google has the power to do this: 

Google Promised Publishers a “fundamental fair exchange between Google and the web” 

To lay the context for Google’s censorship, I need to briefly explain the social contract between Google and the open web – and how AI threatens to change it. 

According to Google, its “mission is to organize the world’s information and make it universally accessible and useful.” 

In the early days of the Internet, Google gained popularity by promising to be a “pure search engine” that just provided links to websites with “no distractions”:  

A graphic showing the Google ad from 1999A Google ad from 1999 (Source)


To this day, Google’s official corporate philosophy page still states: “We may be the only people in the world who can say our goal is to have people leave our website as quickly as possible.”

Google created a social contract with the online publishing industry: publishers provide content for Google to crawl and, in exchange, Google sends valuable clicks back to publishers.

As recently as 2020, Google’s official blog recognized the existence of this social contract: 

Google Search has evolved since its early beginnings, but one thing that hasn’t changed is the fundamental fair exchange between Google and the web. Google crawls, indexes and links to websites in search results, and each search result includes a short preview of what to expect at the site. Websites gain free traffic from users interested in what they have to offer, and each user visit is an opportunity to build a long-term relationship and monetize through advertising or subscriptions.”
Google’s The Keyword Blog (June 26, 2020, emphasis added)

To this day, Google’s website still says the company believes search should “help creators succeed online” and that: 

“To support a healthy ecosystem of fresh and useful content in all the world’s languages, we help people, publishers, and businesses of all sizes succeed and be found by others. We do this by sending visitors to websites small and large through our search results, or by connections such as listing business addresses and phone numbers. We don’t charge to be in our search listings, and we also provide free tools and resources to help site owners be successful.”
Google’s “Our approach to search” webpage (accessed May 18, 2025, emphasis added) 

Google Encouraged Americans to “Create Millions of Sustainable Content-First Businesses on the Web”

Google didn’t just promise a fair exchange of content for clicks – Google actively encouraged Americans to start small online businesses creating content. 

In 2018, shortly after I started Travel Lemming, Google told publishers: “the world has an insatiable appetite for great content and publishers like you remain the beating heart of the open web. Sharing in this mission with you, helping to create millions of sustainable content-first businesses on the web, keeps us going.”

Google Promised to Give Small & Independent Publishers a Fair Shot on the Open Web

Importantly, Google promised that its “fundamental fair exchange” of content for clicks would be a marketplace open not just to massive legacy news publications, but also to new, small, and independent websites. 

In 2014, Google representative Matt Cuts told publishers in a video: “The small guys absolutely can outperform the larger guys as long as they do a really good job at it.” 

The way smaller publishers could compete? Quality content! 

Cutts encouraged sites: “don’t stop trying to produce superior content, because over time that’s one of the best ways to rank higher on the web.” 

But what makes content “superior” and worthy of receiving Google clicks? 

Well, Google had things to say about that too … 

Google Promised to Reward Publishers Who Invest Resources Into Content Created “By People, For People” 

Google provides websites with extensive documentation on what kind of content Google says it seeks to reward with search traffic.

Google says “While there’s no guarantee that any particular site will be added to Google’s index, sites that follow the Search Essentials are more likely to show up in Google’s search results.”

At the top of Google’s list of “key best practices” for websites is an instruction to “[c]reate helpful, reliable, people-first content.” 

Google has a whole page that with questions webmasters can ask to self-assess whether they are creating the kind of content Google seeks to reward. Some examples are: 

  • “Does your content clearly demonstrate first-hand expertise and a depth of knowledge (for example, expertise that comes from having actually used a product or service, or visiting a place)?”
  • “Is this content written or reviewed by an expert or enthusiast who demonstrably knows the topic well?”
  • “Does the content provide original information, reporting, research, or analysis?

In short, Google claims it rewards publishers who invest resources and effort into their content – as opposed to spammers, who often use automated or low-effort content production systems. 

In August 2022, Google summed up this guidance in a now-infamous blog post announcing Google was “rolling out a series of improvements to Search to make it easier for people to find helpful content made by, and for, people.” 

A Google's blog post, titled more content by people, for people in SearchGoogle’s blog post calling for “more content by people, for people in Search”

Like Many Small & Independent Publishers, My Website Invested Enormous Resources Into Creating Content “By People, For People” 

While Google’s Gemini AI will create you a blog post with just a click of a button, creating content by people, for people, is — by contrast – an enormously expensive and laborious proposition. 

At Travel Lemming, we ban AI content flat out. Instead, we produce guides written by knowledgeable locals and destination experts. We hire employees and freelance creators in destinations around the United States and the world, and pay them fair wages. 

Our creators actually visit the places we write about. 

Our creators take and publish thousands of original photos. 

Our creators create original videos, podcasts, and multimedia content to share on the site. 

And, because the world is constantly changing, we also invest significant resources into updating our guides and keeping them fresh and accurate over time. 

We also invest considerable resources engaging with our community. We respond to thousands of reader comments, emails, and messages with personalized travel advice. 

We also host free community meetups in 8 cities in the US and Canada. 

We are not a perfect travel guide. If you look hard enough at any website you’ll find imperfections, and we have many. But we really do try to create the kind of content Google claims it rewards. 

And here’s the thing – creating content by people, for people, is incredibly expensive.

Travel Lemming invests tens of thousands of dollars every month on the costs of producing and maintaining our travel guides. 

And that doesn’t even include the opportunity cost of my time as founder. 

For me, founding Travel Lemming meant leaving behind a much more stable and promising legal career.

The path of online content creation is one of enormous risk and uncertainty. There is no guarantee of a paycheck each month. No one gives you health insurance, benefits, or paid vacation. If you fail, you often fail hard. 

At the time I started the blog, I still had nearly six figures in law school debt (debt I only finally paid off in 2023, just months before Google’s shadowban devastated Travel Lemming).

In 2020, pandemic-related restrictions on travel devastated Travel Lemming’s traffic and, with it, my personal finances. 

I was faced with a choice: close the business, or take drastic action. 

Against the advice of nearly everyone close to me, I doubled down on my passion and invested in myself. I sold my one remaining significant financial asset – my house in Denver – and invested the proceeds into Travel Lemming’s content. 

I say none of this to elicit sympathy for me personally. I have done well in life and I am enormously grateful for all I have given. Travel Lemming, despite the challenges and the betrayal by Google, persists in our mission of helping our readers wherever they can still find us.

But I do want to show that creating content “by people, for people” has a very real human cost. 

And that cost is only worth it if there is an open web ecosystem that properly incentivizes talented entrepreneurs to create high quality content for the web.

Google says it supports such an ecosystem, but let’s look at what it really does … 

Meanwhile, Google Monopolized Search, Subjugating Publishers of All Sizes Into a Dependent Relationship 

Google promised publishers a “fair exchange” of content for valuable traffic. But exchanges seldom remain fair when one party wields unfair leverage over the other. 

Over the past decade, Google has come to dominate – and monopolize – the way that Americans search for and retrieve information. 

A federal court recently found Google illegally built and maintained a monopoly over the search market, noting that Google has a 90+% search market share. 

One consequence of Google’s illegal search monopoly is that web publishers have been forced into a dependent relationship with Google. 

The reality is that search is the main driver of traffic to most content websites.  

A 2024 report by SparkToro found that more than 74% of web traffic referrals come from search engines – and 64% come from Google

Search referral traffic is an even more important revenue source for informational websites, such as travel guides like ours. Search is where users seek out the type of long-form guide content we create. Social media may be great for quick visual inspiration, but search is still where most travelers plan the specifics of their trips.  

And, because Google has a monopoly on search, online publishers are fundamentally dependent on Google. 

It’s a monopolist’s web, and we’re just publishing in it.

Which gives Google a lot of power, including the power to effectively hide entire businesses from being found. 

After ChatGPT, Google Executives Announced a Plan to “Reimagine” and “Reinvent” Search for an “AI-First” Future 

In late 2022, just a few months after Google’s call for “more content by people, for people in Search,” ChatGPT took the world by storm.

Though Google had been working on its own AI ambitions for years, the emergence of ChatGPT reportedly caused Google to declare a “code red” internally – dramatically restructuring the company, calling in its former founders, and reworking everything around AI. 

In February 2023, then-Google SVP Prabhakar Raghavan announced that Google was “reinventing what it means to search.” 

At Google’s annual I/O event in May, 2023, CEO Sundar Picahi reiterated the scope of the change, saying: “We are reimagining all of our core products, including search.” 

After the event, The Verge declared in a headline: “The AI takeover of Google Search starts now.” 

That article astutely observed: “The future of Google Search is AI. But not in the way you think.” 

You see, Google wasn’t planning to compete head on in a chatbot war with OpenAI. 

Instead, as The Verge explained, Google decided to leverage its search market share to put “AI front and center in the most valuable real estate on the internet: its existing search results.” 

“Reimagining Search” Means Reworking the Open Web’s Social Contract 

Reimagining search means reworking the social contract that had underpinned the open web ecosystem for decades, something Google CEO Sundar Pichai acknowledged in Alphabet’s 2023 Q2 earnings call in July, 2023.

In that call, Pichai told investors in that call that AI presented “an opportunity to reimagine many of our products, including our most important product, Search” and that Google was “engaging with the broader ecosystem and will continue to prioritize approaches that send valuable traffic and support a healthy, open web.” (emphasis added)

That same month, Google published a blog post for webmasters entitled “A principled approach to evolving choice and control for web content,” saying:

We believe everyone benefits from a vibrant content ecosystem. Key to that is web publishers having choice and control over their content, and opportunities to derive value from participating in the web ecosystem. However, we recognize that existing web publisher controls were developed before new AI and research use cases.” (emphasis added) 

In short, Google was acknowledging that any fair approach to an AI-first web would require re-writing the web’s underlying social contract.

So Google Promised Publishers a “Public Discussion” About the Future of the Web and AI (But Never Gave Us A Chance to Speak)

In that 2023 blog post, Google promised a “public discussion” to “evolve standards and protocols that support the web’s future development.” 

Specifically Google was speaking about robots.txt – the voluntary protocol that publishers use to control which crawlers crawl our sites for which purposes. 

This protocol lets publishers block Google’s crawlers if we don’t want our content to appear in Google’s search results. 

However, robots.txt was designed decades ago, well before the AI age. 

The protocol provides a very “all or nothing” choice, and doesn’t give publishers the type of granular control we need to really be able to control how our content is used by AI systems (and which AI systems). 

Why is granular control so important to publishers? 

Well, because publishers want Google to pay us when it uses our content for its AI features. 

But we have no real leverage to negotiate if Google already has access to our content for its search indexing. 

As the US Department of Justice recently explained its proposed remedies in the search antitrust case, web publishers “have little-to-no bargaining power against Google’s monopoly because Google can “leverage its monopoly power to feed artificial intelligence features” our content, which we have to provide if we want our websites to be visible in online search.

Google now opposes the DOJ’s proposed remedy to give publishers more granular control over how AI companies use our content. 

But back in its July, 2023 blog post, Google seemed to agree robots.txt isn’t enough:

As new technologies emerge, they present opportunities for the web community to evolve standards and protocols that support the web’s future development. One such community-developed web standard, robots.txt, was created nearly 30 years ago and has proven to be a simple and transparent way for web publishers to control how search engines crawl their content. We believe it’s time for the web and AI communities to explore additional machine-readable means for web publisher choice and control for emerging AI and research use cases.” (emphasis added)

After that blog post, Google’s promised “public discussion” about web publisher control never actually happened. 

The blog post led to a now-closed form that only asked for contact info and didn’t even give us a field to write comments. 

I filled my contact info into the form, but there was never really a public discussion apart from a single one-off webinar. 

Months later, in March 2024, I complained about this silent treatment on X

Google representative John Mueller replied (in a post he has since apparently deleted restricted): 

A reply from Google representative John MuellerA restricted X post by Google rep John Mueller from March 26, 2024

A single one-off “webinar” and a few unpublicized discussions at foreign conferences abroad do not, in my opinion, constitute a “public discussion” on an issue of such importance. 

And what about the “Google-Extended” directive Mueller mentions? 

Well, Google announced that in October 2023. But Google-Extended is a robots.txt directive – in other words, precisely what Google’s initial blog post admitted was an insufficient control mechanism for the AI age. 

So why did Google do a 180 on publisher controls in late 2023? 

Well, I can only speculate – but it might have had something to do with my question back to Mueller in the tweet above (which he never answered). 

Because as it turns out … 

Behind Closed Doors, Google Was Secretly Negotiating a Private Contract with Reddit 

Publishers were not the only ones concerned about Google scraping our content to feed its AI systems.

The owners of the social media platform Reddit were very vocal that they didn’t think it was fair for AI companies to scrape and use Reddit’s content for free. 

They even started making noises about Reddit blocking Google’s search crawlers. 

In an April 2023 New York Times article, Reddit CEO Steve Huffman said: “Crawling Reddit, generating value and not returning any of that value to our users is something we have a problem with … It’s a good time for us to tighten things up.” 

Unlike independent publishers, though, Reddit actually had leverage over Google. Reddit was the owner of a trove of historical user generated content that Google wanted for its grand AI plans. 

If Google could secure a deal for Reddit’s content, maybe that would spare Google the expense of negotiating licensing deals with the web’s many disparate publishers and rightsholders. 

We didn’t know it at the time, but even as Google strung publishers along in 2023 with false promises of a public discussion, behind closed doors the search monopolist was negotiating a private contract with Reddit

Although I and others had our suspicions at the time, the existence of the Reddit-Google negotiations remained a secret until their eventual deal was announced in February, 2024. 

(more on that later) 

Meanwhile, as summer 2023 drew to a close, Google was working on algorithm updates that would start laying the groundwork for its “reimagined” AI-first version of search – and for a new social contract for the web that it would unilaterally write. 

A new social contract that, as it would turn out, excluded independent publishers … 

Google Begins Demolishing the Open Web So It Can “Reimagine Search” from the “Ground Up” for an AI-First Future

In summer 2023, we and many other publishers were very apprehensive about AI. But some of us held out hope Google might responsibly guide the open web through the transition to the AI future. 

At Travel Lemming, we continued to invest exclusively in content “written by people, for people” even as pressure grew in the industry to use new AI tools to create cheaper content at scale. 

Importantly, Google still claimed in its webmaster documentation that it wanted to prioritize human-created content in its ranking algorithms. 

Surely, I thought, Google would realize that publishers are the beating heart of the open web ecosystem – and that even an AI-first future requires publishers who create fresh material for AIs to use and learn from. 

In my view, AI is a poor substitute for human publishers. 

AI is inherently derivative. 

AI cannot experience the world. 

AI cannot visit a place.

AI cannot handle a product. 

AI can only take, summarize, and regurgitate what other actual humans have created. 

And, most importantly, Google had spent the prior years making such a big deal about wanting to elevate content “by people, for people.” 

After making such a big deal about the importance of human content, wouldn’t it be hypocritical for Google to suddenly put AI content at the center of search? 

Well, yes, it would be. 

So … 

Google Quietly Erased “Written by People” From Its Guidance for Websites

Google did what any good monopolist does when it gets caught breaking the rules. It just changed the rulebook. 

On September 14, 2023, Google quietly updated its documentation to remove “written by people” from its guidance for websites:

A graphic showing changes from GoogleA before and after screenshot showing Google’s change (Source: Search Engine Land)

The Same Day, Google Unleashed an Algorithmic Wrecking Ball on Independent Web Publishers

To reimagine search from the ground up, Google had to begin by tearing down what was currently there. 

By this summer 2023, Google knew it wanted to insert AI directly into search results. 

But rolling out AI all at once would mean a massive drop in clicks for publishers – and a potential litigation threat for Google. 

So Google started its AI demolition project by razing the parts of the open web least capable of fighting back: independent publishers. 

Google implemented this demolition project under the pretext of search ranking algorithm updates. 

Google frequently adjusts its rankings algorithms to change how its systems rank web content. Announced algorithm updates happen several times a year and are a recurring part of life for publishers on the open web. 

As a publisher focused on human-first content, Travel Lemming had never really worried too much about Google’s updates. 

Indeed, Google’s guidance to webmasters still states that “most sites don’t need to worry about core updates and may not even realize one has happened.” 

And, indeed, Travel Lemming had never once seen a negative effect from a Google update. 

Plus, even when I witnessed other sites getting hit by past updates, the effect typically was only partial (something on the order of a 10-25% drop in traffic). And those webmasters were usually given opportunities to improve their site and to recover ranking, which they often did. 

But this time was different. 

On September 14, 2023 – the same day Google removed “written by people” from its guidance – a massive Google algo was announced. 

Almost immediately, search referral traffic to Travel Lemming plummeted: 

A graphic showing the Travel Lemming's search traffic in Google Search ConsoleA graphic of Travel Lemming’s search referral traffic report in Google Search Console, overlaid with contemporaneous headlines1

The effect on Travel Lemming’s traffic and business was swift and devastating. 

That September update that hit us was nicknamed “The Helpful Content Update.” It was an update to the content system Google had announced a year earlier. 

And it was only the beginning of the carnage … 

Between Fall 2023 and Spring 2024, Google Updates Systematically Demolished Independent Publishers 

Between fall 2023 and spring 2024, Google unleashed a flurry of updates on a scale that hadn’t been seen in nearly a decade, both in terms of their frequency and their overall effect on the web ecosystem. 

By the time this wave of updates finished in May 2024, Travel Lemming’s organic search referrals from Google had been reduced by more than 97%.

At least our site was not alone in our misery. 

Google’s flurry of updates decimated independent sites across the board: product review sites, gaming sites, entertainment websites, fitness websites, and more verticals felt the pain.2

In short, the period from fall 2023 to spring 2024 was an absolute bloodbath for independent websites.

As CNET described it in an article I was interviewed for, “Google’s major search algorithm updates this past year have left many smaller websites with no other choice than to lay off staff. The internet is worse for it.

The travel vertical in particular was hit particularly hard. 

An analysis by Digitaloft found that 78% of travel publishers lost organic traffic during these updates – and 32% lost more than 90% of their organic traffic

Furthermore, the handful of travel publishers who survived were almost all owned by large mainstream media corporations.

Google Initially Claimed It Was Only Shadowbanning Websites of “Little Value” 

When the September, 2023 “HCU” update rolled out, Google provided written guidance to affected site owners. 

Google has since deleted those statements but they are archived here.

Google initially claimed the shadowbanning algorithm that hit my site “automatically identifies content that seems to have little value, low-added value or is otherwise not particularly helpful to people.”

It was hurtful to hear Google thought our site was of “little value” to the web.

As creators, we at Travel Lemming pride ourselves on sharing our personal travel experiences and pouring our soul into making original, helpful guides. Our readers agree, consistently rating our guides an average of 4.7 out of 5 stars. 

But now Google’s algorithms were saying our entire site was of “little value.” 

But Google Promised Siteowners Like Us That We Could Recover

But Google’s guidance was also telling us that recovery was possible if we just “removed” our “unhelpful” on-page content: 

“If you’ve noticed a change in traffic you suspect may be related to this system (such as after a publicly-posted ranking update to the system), then you should self-assess your content and fix or remove any that seems unhelpful. Our help page on how to create helpful, reliable people-first content has questions that you can use to self-assess your content to be successful with the helpful content system.

A natural question some will have is how long will it take for a site to do better, if it removes unhelpful content? Sites identified by this system may find the signal applied to them over a period of months. Our classifier runs continuously, allowing it to monitor newly-launched sites and existing ones. As it determines that the unhelpful content hasn’t returned in the long-term, the classification will no longer apply.”

-Source: archived version of Google’s since-deleted guidance (emphasis added)

Note Google represented that these were content-based updates, and said that making content more “helpful” – or deleting a site’s weakest content entirely – could lead to recovery.

Google’s documentation further stated that the “classifier” that had shadowbanned our sites ran “continuously,” meaning it would be constantly re-calculated in “real-time.” 

Further, Google said the classifier was “weighted.” Had that been true (it wasn’t), it would have meant that at least partial recovery was possible even with just partial content “improvement.” 

In retrospect, the first sign of Google’s lies was that it replaced indie sites with content that clearly violated Google’s own guidelines (but more on that later). 

My Site, And Many Others, Invested Precious Time and Resources Chasing Google’s False Promises

Despite being hurt by Google calling us of “little value” and brutally shadowbanning us from the web, many of us affected site owners actually took Google’s statements to heart. 

Us independent site owners are also small business owners. 

For many of us, our sites aren’t just our livelihoods – they are also our passion projects. 

I consider Travel Lemming my life’s work. I’ve always wanted it to be the most helpful site it can be, and not just because Google tells me so. 

And, while I thought Travel Lemming’s guides were high quality, every site can always improve further. So that’s what we set out to do. 

The entire team at Travel Lemming invested nearly a year of time and more than six-figures of resources turning our content upside down in an effort to “improve it” and meet Google’s guidelines for content guidelines even more. 

We even removed the majority of our content from Google’s index – leaving only the guides we were 100% confident were abundantly “helpful” beyond any doubt. 

Many other sites invested similar resources trying to recover in vain. 

This investment came at great cost to many of us – because we had to invest such significant resources precisely at the time that our websites were completely starved of search referral traffic

But it turns out we were all just chasing Google’s shadows – shadows that kept us all busy while Google worked on its blueprint for reimagining search around AI. 

But, Despite Those Efforts, Recovery Proved Impossible 

Nearly a year went by and, despite massive efforts, we didn’t recover at all.

Neither did anyone else.

In fact, like almost all affected indie sites, we continued to lose rankings further – no matter how much we changed our content. 

Something seemed …. wrong. 

As it turns out, the algorithms that hit ours and other sites were not really based on content at all.

Instead, it eventually became clear that these were really “authority” based algorithms. Which is a fancy way of saying that Google decided to promote larger sites over small ones – and it actually had nothing to do with content all along.

That explained why all our investments in overhauling or deleting content didn’t move the needle.

But it didn’t explain what we indie publishers should do next – other than, as one site owner snarkingly put it to me, “becoming Forbes.” 

Google Does Not Let Small Independent Sites Appeal Their Shadowbans (But Large Media Companies Can)

By mid-2024, we and other affected sites were in a truly terrible position: not only were shadowbanned, but there didn’t seem like there was anything we could even do about it. 

Google offers no way to appeal an algorithmic shadow ban like the classifier that hit our site.

This is why Google usually avoids deploying algorithmic shadowbans against big sites owned by media conglomerates, even when big sites knowingly break Google’s spam policies. 

Instead, when large sites break Google’s rules, Google instead issues them “manual penalties.” 

The big publishers then get access to a structured appeal and reconsideration process.

Google representatives will even help guide these sites through what changes they need to make, until they eventually recover their rankings. 

For example, in March 2024, Google announced it would supposedly crack down on “site reputation abuse” – a practice where large big-brand websites are able to abuse their site’s authority score to pump out low-effort content en masse. 

Some of these large publications are even known for renting out urls on their webpages to third party spammers, a practice known in the spamming community as “Parasite SEO.” 

This spammy practice is only possible because of how heavily Google’s algorithm now favors a site’s “authority” signals (as opposed to actually measuring content helpfulness). 

These large sites were abusing Google’s long-existing policies against low effort content. And they were doing it intentionally. 

But, nonetheless, Google did something unprecedented: it gave these large sites several months advance notice to clean up their act.

Google said: “To allow time for site owners to prepare for this change, this new policy will take effect starting May 5, 2024.”

I initially expected this meant Google would unleash an algorithm to punish these large sites. But no algorithm update ever hit these sites.

Instead, in June 2024, Google explained it would “only [be] doing manual actions on scaled content abuse, not algorithmic.”

Why only manual and not algorithmic? 

Google Search Liaison Danny Sullivan answered this in an August, 2024 interview

“There’s no algorithmic action, I don’t expect there to be any algorithmic action anytime in the near future,” Sullivan told me. He said if and when it becomes algorithmic, Google will announce it. Until then, it is not.

Why is it not algorithmic? “The reason we probably won’t have it any time in the near future is because we wouldn’t be exceedingly careful and, and thoughtful in how we do it. So that’s just taking time and for the moment, the manual actions are the way for us to go,” Sullivan explained. (emphasis added)

Google was admitting that it uses manual actions for sites owned by large media conglomerates because it treats them differently than independent sites. 

Large sites are allowed to knowingly and intentionally abuse Google – and the worst risk they face is a manual action that comes with an opportunity to appeal and work with Google to get the manual action lifted. 

As an example of this, a few months later Google hit the website Forbes for violating Google’s “site reputation abuse” policy. 

This meant Forbes was given access to an appeals and recovery process – and was able to recover from the penalty just a few months later. 

Shortly thereafter, a Forbes representative told The Wall Street Journal that “Forbes continues to partner closely with Google” and that Google “has a careful review and appeals process for site owners” that Forbes was able to use to get the manual action lifted quickly. 

Independent sites like us, however, get no such appeals procedure. 

Our entire sites can be permanently shadowbanned from Google by an algorithm. 

Independent site owners exist on Google’s web in a permanent state of anxiety and insecurity. We can be erased by an algorithm at any time – arbitrarily, brutally, and with zero opportunity for appeal. 

The one thing we could do was use our voice on our blogs and on platforms like X.

So, even though many site owners were scared of retaliation by Google, some of us started speaking out publicly about what was happening (examples here, here, here, here, and here). 

Google Eventually Admitted Fault – And Invited 20 Creators (Including Me) Out to Google’s Headquarters to Apologize

At a certain point the evidence was just too overwhelming to ignore. Too many quality independent sites had been hit too hard for too long.

Google’s algorithms obviously weren’t working the way Google said they should.

Google even admitted it.

In October 2024, Google invited me and 19 other independent web creators out to Google’s headquarters in Mountain View, California.

Google gave us a tour of its (mostly empty) campus.

Travel Lemming founder Nate Hake outside the Google’s HeadquartersMe at Google’s Headquarters during the event

And Google gave us a clear and unequivocal apology.

Google said our sites didn’t deserve our shadowbans, and that it wasn’t our fault.

Personally, I at least felt validated by the apology. I had let the events of the past year shake my sense of self-worth, and it was nice to not be gaslit anymore.

But, as a fellow creator said at the event, “apologies don’t pay the bills.”

We all still had no idea of how we could get our sites un-shadowbanned. 

Unfortunately, Google didn’t really have an idea either.

Google Told Us There Was Nothing We Could Do to Recover, Because Search Had Permanently Changed With AI

We all gave Google a mouthful of our opinions at the event. 

The Commission can review some public accounts of the event from other creator attendees here, here, here, here, here, here, here, here, and here

You can read my own post-event reflections here

And you can also read this Bloomberg article for more context. 

At one point I aggressively pressured Google executive Pandu Nayak for specific guidance on how web creators are supposed to survive in Google’s AI-first future. 

Nayak appeared visibly shaken by my questioning, but he ultimately gave no real answers. He made an offhand comment about how we could try using AI to create content (umm, no thank you!). But beyond that, he just blustered about how AI was changing search.

And, ultimately, that was really the main takeaway from the event:

Googlers clearly told us that, even if our algorithmic shadowbans got lifted, our traffic might never return because search had been fundamentally changed by AI over the previous year.

I’d note that Googlers used passive phrasing to make it seem like the changing search landscape was a result of forces of nature – and not of Google’s very deliberate decisions.

But, regardless, the message was clear – AI is fundamentally changing how Google works, and the old ecosystem of search isn’t ever coming back.

Google did promise to work on improving its algorithms so as to elevate more independent sites like ours.

While I initially held out some hope, most of my fellow creators left pessimistic that Google actually planned to change anything.

Unfortunately, time has proven them right.

As I write this 8 months after the event, Travel Lemming and most other affected sites remain shadowbanned by Google’s algorithms.

Meanwhile, Google continues to press full-steam ahead on its project of rebuilding search from the ground up with AI …

“Rebuilding Search from the Ground Up” – What Did Google Build on the Ground It Demolished? 

Which brings me to the obvious question you may have: if Google cleared out so many independent publishers, what took their place in search results

Well, let’s rewind to spring, 2024, when Google wrapped up its demolition job. 

The last of Google’s ground-clearing updates was announced as complete on April 26, 2024.

The foundation for Google’s “re-imagining” of search had been laid. 

Now it was time for Google to start the process of rebuilding search from the ground up. 

Google Rolled Out AI Overviews – Two Weeks After Purging Indie Publishers

Just two weeks after Google finished clearing out independent publishers Google announced the next phase in its plans at its annual “I/O’ event: 

Google was putting AI content directly into search results with AI Overviews. 

Google's blog post about AI OverviewsGoogle’s blog post announcing the roll out of AI Overviews

Google’s AI Overviews work by scraping content from websites, summarizing it with an LLM, and presenting a summarized answer right on Google. 

In its I/O presentation, Google demo’ed of an AI Overview for a question about a broken record player (video here, starting around minute 51): 

A woman during a Google’s 2024 demo of an AI OverviewGoogle’s 2024 demo of an AI Overview at Google I/O

Google’s on stage AI Overview demo showed the fabric of the web’s social contract fraying in real time:  

The AI Overview pulled, summarized, and slightly reworded content from the source website Audio-Technica

A text similarity analysis showing the similarity of AI Overview demo’s text to a website's contentA text similarity analysis showing the AI Overview demo’s text was lifted from the source website

Notably, the Googler performing the demo was able to “solve” her problem without actually clicking through to the website that created the content that helped her.

This presents an existential problem for publishers – if searchers can find the information we create without ever visiting our websites, how are we supposed to earn revenue to fund content creation? 

Indeed, in the year since, multiple independent studies have found that the presence of AI Overviews in search results reduces click through rates to web publishers by anywhere from 15% to 55%. (Sources: Seer Interactive, Ahrefs, Amisive)

Instead of clicking through to websites, AI Overviews keep searchers clicking around Google – letting Google generate more profit at the expense of the hard work of publishers. 

Google Plans to Use AI to Monetize Not Just Searchers’ Questions, But Also the Answers

A graphic showing Google's AI

As one prominent search expert put it, “Google’s AI Overviews are seemingly designed to shift user clicks away from websites and nudge them towards Google ads.” 

At I/O 2024, Google announced its vision for an AI-first future where “Google will do the Googling for You.”

Just a week later, at Google’s Marketing Live Keynote, revealed plans for “Ads that Answer” – letting Google monetize not just questions but also the answers themselves. 

Google’s plan is working.

Google’s search ad revenue has surged 16% over the last year – and that’s despite competition from ChatGPT, the maturity of the search market, and the fact that Google’s market share has not grown. 

Google Also Favors Its Own Properties in Search Results 

Google cut down independent publishers to create space for AI and ads – but also so it could promote its own web properties. 

Remember how I said that Google’s algorithmic wrecking balls hit the travel sector especially hard?

Well, consider this chart of Google Travel’s search visibility over the past few years: 

A graphic showing the Google Travel search visibility(Source: Lily Ray X post)

You may notice that Google Travel’s graph looks like a near-perfect inverse of Travel Lemming’s search referral graph. 

Google also favors its other properties like Youtube and Maps. 

And Google aggressively injecting links back to its own search results within AI Overviews, to keep searchers from clicking out to third party websites. 

One recent study found that “43% Of AI Overviews Point Back To Google.”

Google consistently favors itself above all other websites.

Though Google does have another favorite recently … 

Reddit Got an AI Licensing Deal With Google – Plus an “Unprecedented” Boost in Search Visibility – Right Before Its IPO

Apart from Google, there is one clear winner from Google’s project of reimagining search: Reddit. 

The same algorithm updates that removed independent publishers from search results also elevated Reddit to a degree never before seen in Google search history:

A graphic showing the search visibility of RedditReddit’s Sistrix visibility graph (Source: Lily Ray, X

In summer 2024, Steve Paine of the search visibility tool Sistrix told Business Insider that Reddit’s growth was “unprecedented.” 

He continued: “There hasn’t been a website that’s grown so much search visibility so quickly in the US in at least the last five years.”

But search visibility was only part of what Google gave Reddit. 

Reddit also got cold hard cash.

On February 22, 2024, Reddit and Google announced a $60 million per year AI licensing deal.

Reddit’s sabre-rattling in 2023 about blocking Google’s crawlers had apparently worked. 

The deal, as well as the search visibility boost, came at an incredibly convenient time for Reddit and its executives. You see, almost exactly one month after the deal announcement, Reddit completed its IPO and went public on the NYSE. 

Reddit priced its IPO at $34 per share, which was “the top of the target range set by Reddit’s investment bankers” according to The Guardian. The shares popped 48% higher just in their first day of trading. 

As of mid-May, 2025, Reddit is trading at over $100 per share – giving it a market cap of almost $20 billion.

Now, to be fair, Google will say that the two things – Reddit’s boost in search, and its AI licensing deal – are completely unrelated and a total coincidence. 

You can judge the facts for yourself.

Collusive or coincidence, one thing is clear: the whole incident shows just how much power Google wields to reshape the entire Internet to its whims all in the span of a few months.  

And AI will give Google even more power to re-write the Internet: 

Google Plans to Use AI Licensing Fees to Let It Control Which Websites Survive In the AI-First Future

How will websites survive when users don’t need to click through? 

Well, many web experts believe that an AI licensing fee model is the logical replacement for the current “clicks for content” web social contract. 

Google apparently agrees. 

Sundar Pichai recently said: “There will be a marketplace in the future, I think. There will be creators who create for AI models and get paid for it. I really think that’s part of the future and people will figure it out.”

But Pichai doesn’t seem to be in a rush to create that marketplace just yet. 

Google paid Reddit $60 million a year for AI licensing fees, but so far refuses to seriously discuss licensing with most other publishers. 

Why? 

Well, Google explained its AI licensing position in a recent letter to the UK government

Google said that publishers have a right to opt out of their content appearing in Google search but that “this does not extend to a right to be paid.” 

Google’s letter goes on to explain that Google is open to “negotiating agreements and partnership deals for a variety of situations, including programmatic access to custom APIs, access to data, digitisation, etc.”

But, Google said: “No single piece of content has value, and as such, pricing becomes a pure bargaining issue.”

In other words: Google is only willing to pay AI licensing fees to parties (like Reddit) with enough leverage to bargain against the monopolist. 

And, conveniently for Google, the more Google rolls out AI, the less leverage publishers have. 

Like a reality TV game show, Google can slowly eliminate publishers from the web – winnowing Google’s final bargaining counterparties to a small handful of contestants Google gets to select. 

And there are indications Google has already chosen which sites will get a chance to survive in its AI-first future … 

Google Is Heavily Biased Towards A Few Large Media Conglomerates 

When Google needed to raze ground to build out its reimagined AI search, independent publishers were an obvious first target for Google. 

After all, small websites typically don’t have the resources to sue, as some larger publications have threatened to do over the years (or, as some like Chegg, have actually done recently).

But Google has, at least so far, treated the mega media companies differently – sparing them from its AI wrecking ball (at least so far).

This, in turn, has led to incredible consolidation of the information ecosystem. In fact, “close to half of all of Google’s traffic is going to just a handful, a few hundred websites.” (Source: SparkToro

Most of those websites are owned by just 16 VC-backed media conglomerates.

This consolidation of the information ecosystem hurts users in many invisible ways. 

Gisele Navarro of the website HouseFresh has extensively documented how some big brands owned by some of these conglomerates often flood Google with questionable content. 

I strongly recommend Commission’s staff review these articles from Gisele:

I personally think many of these large companies are living on borrowed time, and that Google is likely to come for their slots in search results in the next round of its game show. 

I hold out hope some of them will see the light and join independent publishers in speaking out more forcefully against Google. 

But, at least for now, many of these large media conglomerates seem content to be permitted by Google to exist within the monopolist’s walled garden (for now). 

Google Is Planning to Use AI to Consolidate the Flow of Information Online 

Google already has the power to control which sources of information users find and click on when they search.

And, as I’ve shown above, Google is not afraid of abusing its monopoly power to direct users to itself, its partners, or its preferred sources of media.

But what happens when Google controls not just the sources of information – but also the actual information itself?

That is exactly what Google aims to do with AI: to become the singular source of all information, opinions, and advice on the web.

Soon we may not have to worry at all about which websites get clicks from search – because Google won’t be sending clicks to anyone at all.

Unless, of course, you pay Google to show an “ad that answers” users’ questions.

And therein lies the rub …

Google Isn’t Satisfied With Its Search Monopoly – It Wants a Monopoly on Answers

Google executive Noam Shazeer recently let slip Google’s real plans for AI in a podcast interview:

Organizing information is clearly a trillion-dollar opportunity, but a trillion dollars is not cool anymore. What’s cool is a quadrillion dollars.”

Indeed, Google is well on its way to realizing that quadrillion dollar opportunity – silently, and with less fanfare than OpenAI.

Already AI Overviews are used by 1.5 billion users. 

Of course, those 1.5 billion didn’t choose to use AI Overviews – Google just leveraged its monopoly to force AI Overviews on users.

At Google’s 2025 I/O event, Google CEO Sundar Pichai even bragged about how Google is leveraging its search monopoly to force AI Overviews on “more people than any product in the world”:

The Google CEO during the Google’s 2025 I/O event

Because, for Google, taking away user choice is precisely the goal anyway … 

Because Once Google Controls the Answers, It Will Control the People Too

Google envisions a future where “Google does the Googling for you” and its AI and ads supply the answers right there on Google – all sourced from a handful of large media conglomerates forced to license their content to Google via contracts of adhesion. 

For Google, a monopoly on answers is the ultimate goal. 

Answers are so powerful because answers dictate people’s decisions.

So when you control the answers, you actually control the people themselves.

Americans already use Google to find answers to so much more than relatively frivolous questions like where to travel.

Americans use Google to find answers on where to live.

Americans use Google to find answers on what careers to pursue.

Americans use Google to find answers on whether to have kids.

Americans use Google to find answers to what medical procedures to get.

Americans use Google to find answers to what faith to follow.

Americans use Google to find answers on who to trust.

And Americans use Google to find answers on how to vote.

I believe that our sources for those answers should be diffuse, varied, and independent – not monopolized by a single tech company.

Because if one company can monopolize the people’s answers, it can also monopolize the people’s decisions.

What You (Yes, You!) Can Do to Save the Open Web from Google’s AI Takeover3

The open web is under attack by Google and AI – but there is still time to save it, so long as we all act together swiftly.

Here is what you can do right now to save the open web:

  • Share what Google is doing to the open web. You could start by sharing this article. Alternatively, I’ve left other good articles to share in the further reading section below.
  • Switch your default search engine to literally anything except Google. Here are instructions. I use Kagi (paid) on desktop and DuckDuckGo on mobile. Both are 10x better than Google, and better citizens of the open web. But switching to literally any other search engine will help de-centralize the Internet.
  • Visit independent publishers directly. You won’t find many of us on Google anymore, but many of us are still publishing helpful content written by humans — we just need you to find and support us directly. My friend Gisele has a good guide on how to do this. If you find an independent website you like, bookmark it. Subscribe to their newsletter. And go straight to that site for your next question.
  • De-Googlefy your life as much as possible. From Gmail to to Workspace to Maps to Photos to Calendar to Android to Chrome to hundreds of other products, Google’s grip on our digital lives goes way beyond search. And though many of these products are free (and you may even love some of them), they also fuel Google’s monopolistic ways by providing data about you that Google uses to train its AI systems and entrench its control over the web. Switching away from as many Google products as possible will help starve the monopolist of the data it feeds on. Here is a guide with more details.
  • Call Congress to express support the efforts of the DOJ and the FTC to hold Google to account. These agencies have been battling Google in court across multiple administrations. No matter your party, you should support these agencies in their fight to break Google’s hold on the open web. One way to do that is to contact your representatives and express your support for the breakup of Google.
  • If you are a independent publisher, share your story. Talk to your audience wherever they can still find you. Tell them what Google is doing to the open web and how they can help. If you publish a post, send it to me so I can help amplify.
  • If you are a large publisher, now is the time to stop your silence. I know many large publications are scared to speak up against Google. You may think you’ll get an AI licensing deal. You may think Google will spare you from the AI carnage. I doubt it. And, even if your publication is among the privileged few permitted to exist behind Google’s walled garden, Google will still squeeze you for every penny it can. If you work at one of these publications, please push your editors to stop being silent about what Google is doing to the open web with AI. If you need sources or material, reach out to me.

There was a web before Google and there will be a web after Google, so long as we all decide to take back control of our online lives before it’s too late.

Recommendations for the Commission

  • Investigate Google’s algorithm updates in 2023 and 2024 for evidence that they were pretextual efforts to wrongly censor specific categories of web publishers and/or to unfairly promote Google’s AI ambitions
  • Require Google to treat independent websites the same way as large publications, including by for example giving siteowners an explanation for shadowbans and an appeals process
  • Continue to support the Department of Justice’s effort to break Google’s monopoly on Search, so publishers have more leverage in the information economy of the future
  • Investigate Google’s potentially collusive actions with other information market participants like Reddit
  • Require Google to publicly report important statistics necessary for publishers to fairly compete in an AI-first online economy, such as click through rates for AI Overviews and AI Mode
  • Require AI companies like Google to disclose their AI training material
  • Require AI companies like Google to give rightsholders granular consent and control mechanisms
  • Take steps to force AI companies like Google to pay just compensation to rightsholders when they use our content to train their AI models 

Conclusion

If you’ve read this far, I thank you from the bottom of my heart for hearing me out. Believe it or not, this was the abbreviated version of my story. If the Commission wants further information or explanation, I’ve got a lot more to say and would be happy to be contacted.

I realize few people care about the plight of a struggling travel blogger.

But I truly believe that we independent publishers were the canaries in the coal mine.

What Google did to censor us with AI it can do to others – and America and the world will be worse off for it.

I pray that the Commission and the Department of Justice will stop this evil monopolist before it’s too late for us all.

I truly believe you are the last best hope for freedom online.

Further reading: 

Footnotes:

Read Entire Article