ChatGPT Now Uses Google Index – Proved Using Orphan Page Experiment

TL;DR:

1. The paid version of ChatGPT is using Google Search.
2. We prove this using two methods:
   1. “sting” page – ie. a page that is only present in Google’s index and no other search engine
   2. deep analysis of ChatGPT’s search references
3. The free version of ChatGPT likely uses OpenAI’s own web-search engine
4. Bing does not seem to be used anywhere

Why this is a big deal?

1. Impact on SEO

This means the whole SEO / Answer Engine Optimization (AEO) narrative has come full circle from “Google is dead” -> “rank in Google to show up on ChatGPT”!

2. Shady Practice on OpenAI’s part to route large volumes of private data without explicit declaration or consent

There has been no official announcement about the OpenAI-Google partnership as of this writing.

However, as a paying ChatGPT user, now your data is being sent to Google without your consent.

ChatGPT’s docs does mention that results may be sent to Bing or Shopify, but not Google.

(from https://help.openai.com/en/articles/9237897-chatgpt-search)

Background About Me

A bit about me – I’m Abhishek (X @distantgradient). I used to work as a search infrastructure engineer at Google in a past life. Nowadays, I build an AI SEO agent (ACME.BOT), so digging into ChatGPT and its behavior is somewhat routine for me.

Proof #1 that ChatGPT uses Google – The Sting Page

I created a dummy secret web page. By dummy I mean a page about a unique non-existent term. By secret I mean not linked from anywhere on the internet. Then, I got that page manually into Google’s index.

If ChatGPT gains knowledge of this page after it makes it into Google’s index, it means that ChatGPT uses Google’s index.

We first check with ChatGPT before creating and indexing our dummy page:

(Ref. https://chatgpt.com/share/687bdac3-853c-8007-a5d6-2dbb66cae718)

As expected, ChatGPT denies any knowledge of this term.

And, after creating the dummy page and getting it indexed in Google.

Now suddenly ChatGPT starts quoting from this dummy hidden webpage that only Google knows about.

The result for this particular term in Google, Bing and other search engines:

Bing:

I also checked other search engines like DuckDuckGo, Yandex etc. pretty much everything returns no results – as expected.

In my opinion, this is undeniable proof that ChatGPT uses Google search engine.

Ironically, back in 2011 Google had accused Bing of stealing Google’s search results. It was called the “Bing Sting” or “hiybbprqag”. The test used in that “sting operation” was quite similar to this one – get a made up term indexed in Google and see if the other search result reflects it.

Looks like the same test is quite relevant even today!

Free ChatGPT does not Google, nor Bing

When checked with the free version of ChatGPT, our dummy term does not show up at all – suggesting that the free version of ChatGPT likely uses some other search engine.

We also ran some search engine overlap tests to conclude that the results are not from Bing as well and are likely from a different search engine.

OpenAI was rumored to have been working on their own search engine and it is likely that they are using / testing this out on the free version of ChatGPT.

Proof #2 that ChatGPT uses Google – Query Results / Snippet Overlap

The test is simple – extract the query results ChatGPT is using in its result using Chrome’s network tab. Run the same set of queries and gather results for Google and Bing and arrive a similarity score of the search results

We found that the results from ChatGPT are most similar to Google’s search results. 

On the other hand, they are actually quite dissimilar to Bing’s search results.

ChatGPT search’s search domain overlap with Google and Bing.

Metric	Value	What it means
Search-result (domain) overlap	1.0	Every domain that appears in ChatGPT’s search results also appears in Google / Bing.
	0.5	Half of ChatGPT’s result domains are shared with Google / Bing.
	0.0	ChatGPT’s result domains do not appear in Google / Bing at all.

ChatGPT search’s token overlap with Google and Bing:

Snippet-token overlap (measured only on those shared domains)	1.0	ChatGPT’s snippet contains exactly the same tokens as the corresponding Google / Bing snippet.
	0.0	No token overlap between ChatGPT and Google / Bing.

We ran the same test on the free version of ChatGPT and found that its search results are less similar to Google and Bing than Google and Bing are to each other. In other words, Google and Bing align more closely with each other than either aligns with free ChatGPT’s results and snippets.

This strongly suggests that the free version of ChatGPT may be operating on OpenAI’s own experimental web search engine, rather than using either Google’s or Bing’s.