Proof that Google and ChatGPT are NOT reading the same internet

Page note — not spoken on film. Question.Marketing delivers AEO and GEO work in partnership with GEOforge, who produced the study analysed here. Method published. We are not neutral on the company. We are neutral on the numbers.

Hi, everybody. I want to talk to you today about a really important piece that Paris Childress of GEOforge and I talked about on a live on LinkedIn a couple of days ago. We surmised, or we had a bit of an argument around, is social media engagement leading to citation in any of the LLMs?

Now, I have said for a long time that if social media posts are engaged with heavily, that should then lead to rankings in Google. Google would never admit to this, and a lot of SEOs would push heavily back on the fact that that does not seem to necessarily be the case. But we are now entering a world, as we know, where prompts, rather than keywords and key phrases and questions, are important, hence me working on Question.Marketing and AEO, GEO, and working across the different LLMs.

Now, the findings in Paris's report here, which I have got on the screen, are incredible as far as I am concerned. I am a nerd, right? Why not? I like this sort of stuff. But he pulled all of the LinkedIn and YouTube citation data from GEOforge. So they have over 10 million citations in their database, across I think about 850 prompts that they are currently tracking for clients.

Now, what is evident, and I would encourage you to read this article of his at length. I am not going to go through every single piece of it. I will give you my take. But Paris runs the GEO Show. It is published as a podcast as I also talk to people about how you get your businesses listed in AI.

One of the things that is very apparent is, and I have been talking about this for a while now, Google is losing its grip and trying to ring-fence its own data, and one search engine does not mean the dataset for all of the LLMs. That is evidenced by the fact that 47% of the YouTube videos that are cited have less than 1,000 views. So that is not necessarily engagement, but what it is showing you is that Google's AI references YouTube videos far, far more than, for example, ChatGPT. ChatGPT pulls a lot of its citation from LinkedIn, for example.

Now, if you look at 25,000 ChatGPT citations, only one points to YouTube. So clearly, people are going crazy over does Reddit point to this? Which social property or which sites or which authority domains do you need to publish on to be cited? Clearly, making YouTube videos does not get you into ChatGPT, whatever you do. We are also looking at the fact that if you are publishing on LinkedIn, that is working and getting you cited in ChatGPT, so articles, pulsed articles.

Now, what does that say to us? I thought that if you take a post and it gets a lot of engagement, I would have made, or I did make, the assumption that if a post is getting a ton of engagement, we would imagine that it would go further in search, be cited more, and that does not appear to be the case. There does seem to be a threshold, so you do need to break through certain levels. You cannot necessarily be an absolutely brand-new channel and get no engagement and expect to get a post cited, right? But once you got past, for example, 1,000 views on a video on YouTube, the citation rate did not increase dynamically, which you might have expected.

Now, GEOforge measured over 10.2 million citation events across 336,000 unique cited URLs, and that constituted 458,000 prompt runs over 848 prompts. This is a significant piece of data that I think people need to pay attention to. So all the top people that are talking about AEO, GEO, and SEO, and SEO being part of these new disciplines, need to pay attention to the fact that this data shows clearly how citations are driven. And social platforms account for 8.8% of that corpus. So of that data that is out there in terms of citations, we are looking at nearly 10% of it coming just from social media. That may well be a minority at the moment, but it is meaningful and it is not something that B2B brands in particular should be ignoring.

So you do clearly need to be publishing across all social media. More so, you need to be publishing unique knowledge, and Paris does talk a lot about that, needing to have knowledge that is new to the LLMs, right?

Now, this is the bit that showed this part of the report: which AI engines actually cite social media. Now, if you look at ChatGPT, Google and Google, they are not reading the same internet. Let me repeat that for you, and this will be the title of this episode from me. ChatGPT and Google are not reading the same internet. Remember that. That is key.

I have been talking about Google losing its grip. Somebody called me, "Am I on mushrooms or am I on crack?" by saying that maybe in years to come we do not have... Well, we will have Google in some respect, but it is definitely slipping, because if you look at ChatGPT with over a million citations in this piece of data from GEOforge, look at the percentages: 0.004% for YouTube, about 0.2% for LinkedIn, 0.09% for Reddit. Now, I see everybody chatting about Reddit. Reddit must be this, Reddit must be that. Well, not for ChatGPT, clearly. Instagram, zero, right?

Google AI Mode conversely: YouTube about 6.34%, LinkedIn one and a half percent, Reddit 1.8%, Instagram 0.72%. So a clear huge differential between the way things are getting picked up. And then obviously Google AI Overviews: YouTube about 6.63%, LinkedIn one and a half, Reddit 2.1%, and Instagram 0.56%.

Now, I would imagine, and again maybe this is a massive assumption, YouTube videos do not necessarily build as much content because people typically do not make enough video, right? I would imagine that should we be able to put a video in front of Google that answers a very, very specific topic, i.e. a prompt that we want to be listed for clearly in Google, which is not going to work in ChatGPT, but I am going to come on to another experiment in a bit. If that video matches a prompt almost exactly and fills a knowledge gap, does it then become cited more easily? Does it get picked up almost like an exact match keyword phrase, right? Long-tail keyword phrase if you are an SEO, think of it that way. Can we get a prompt straight into Google AI Mode or Google AI Overviews using a video which matches that prompt and fills a knowledge gap? That is one of the questions that we are going to ask and try and answer in this series of performance that we are working on.

So if we see that ChatGPT only cited 42 videos out of over a million citations, that is not going to work, right? We can see that that is basically ChatGPT reading its own index, referencing something completely different to Google AI Mode or Gemini, et cetera, right?

So we looked also at how big is the audience on these cited videos, and it showed that cited videos did not have to have that many views. So if you look at distribution over videos where there were under 100 views, 100 to 1,000, 1,000 to 10,000. Now, the assumption was obviously if you have got a video that is getting hundreds of thousands of views, you may well presume that that would be cited. It is not anything any greater. Although, do we know whether we have referenced that many videos with over 100,000 views on them? Maybe, maybe not. There are probably not that many videos that answer prompts that are over 100,000 views, so that would be another thing to test.

So LinkedIn, and if you are watching this on LinkedIn, different story, right? In the same scenario, less engagement does not matter here, or it does a little bit, right? And the story it is going to tell is 60% of the cited LinkedIn articles that are picked up on ChatGPT had less than 50 comments or likes or engagements on them, and 19 or 20% of them had less than 10. So maybe we could be thinking, right, as long as we can get 10, 15 comments or engagements on a post, it shows the relevance and then it maybe gets picked up a little bit easier. So that is one thing to learn.

So the answer is, does engagement make a post more likely to be cited? And in reality, the answer is no, not really, not necessarily. Does it matter if it matches the question or the prompt more closely and gives some unique knowledge? Yes, I think it does.

There is tons more in this article. Particularly here if we look at LinkedIn posts, they die after a couple of days. We know that. After 48 hours, typically a LinkedIn post is pretty much gone. But we are finding that LinkedIn posts and articles that get picked up, they are typically getting picked up 35 days or more after they were originally published. Now, whether that is a circle on the AI wheel that comes back round and spiders that content and checks that it is still there and has had its engagement and is still living, maybe that is a thing.

So the findings that Paris put together: pick the engine that you want to list on. Now, my advice would be I want you to be able to be listed across all of them. Of course I do, and that is producing content and only publishing it in one way. So for example, if you produced a YouTube video and you posted that on your website and you wrote a blog around it, et cetera, yeah, great. You are doing traditional SEO and it is working for you on Google AI Mode, maybe, if you can get it picked up and spidered and you have structured it correctly and it has got the right prompts in it, et cetera, et cetera. If you then want to get that distributed and referenced and shared, it needs to go onto other platforms and you need content across all of them, right? And you need to work with, I would hope, an expert who knows how to do that, right? You do not have to necessarily get it published with tons of engagement, which is a great sign, right?

It is also evident that you are better off publishing more and more content, rather than just trying to get winners. So the thought here is you are not trying to get an amazing post that gets loads of engagement because that will get you the reach and get you the citations. No, you are better off posting better content, more of it, filling in the gaps, and doing what, for example, Gary Vaynerchuk says, and that is getting a lot of content out there and looking for the winners. So this backs up what he is saying about publishing a lot of content, and get the engagement across as much of it as you can. You do not need to worry that if you are not getting tons of engagement, it is about publishing unique knowledge as much as you can across as many platforms as you can and distributing that.

So we are going to run a test, Paris and I. We are going to publish some very specific content absolutely tailored to some prompts that we know do not have answers behind them at the moment, or exact answers. So one of the things that fascinates me, I was looking at the knowledge gaps in a corpus and saying, right, okay, if certain models do not know the answers, how does it answer? And the answer is it will give you its best guess or its nearest approximation. Now, if the then exact answer appears, does it then get cited? And that is what we want to know next.

So we are going to publish some content. We are going to then have that match a knowledge gap. We are going to publish it as widely as we can for clients. One client in particular, Stitch AI, we are going to publish that as part of this study and we will then report back to you how that does, and where it appears if it appears and if it is cited. So that is the next experiment.

I would love you to keep following me. Please do share and like and comment. I would love to hear from you anything that you are finding out as well. This is an open book, right? And we are all here to learn. So love to hear from any of the audience and speak to you again soon.