Module 6 - Too much data

Why can too much data be dangerous?

Too much data is dangerous because it is easy to end up going down a rabbit hole, looking at so much information that it becomes impossible to work out what is actually affecting what.

Some clients need a full-time data analyst just to work through the reports once they're spending more than two or three hundred pounds a day, tracking where conversions are coming from, which audiences, which keywords are costing what. The trap is well known: you see most conversions coming from one campaign, then start drilling into whether it's mobile, desktop or TV within that campaign, then a particular device type, and you can quite easily end up chasing an assumption that turns out to be wrong.

What happened when a client's TV traffic looked like it wasn't converting?

A campaign that looked like a mobile success story turned out, once the real sales were counted, to be driven mainly by TV.

In this particular case, mobile had far more micro conversions, time on site, people adding to cart, than anything else. But when it came to conversions that actually mattered, real sales, TV won, and by a wide margin. The first instinct might have been to turn TV off, because so few people were adding to cart or spending time on site from it. That would have been a mistake, an admitted one, because people watching an ad on TV don't necessarily click through there and then. What they were actually doing was watching, absorbing the awareness, and then ordering the product on average about four days later. Finding that kind of insight inside a campaign is where the real value in the data lives, but only if you don't stop looking too soon.

How much data is actually enough?

If a campaign is spending less than a quarter of another campaign's budget, you're not comparing apples with apples, and a few days of data, or a few hundred clicks, usually isn't enough to draw a real conclusion.

Google might spin out three or four thousand placements into a campaign against a budget of only fifty dollars a day, and most of those placements won't have had a fair chance to run, or will have had only a minimal number of impressions. You can't reasonably decide a channel or placement type doesn't work if it hasn't even been properly tested. A rough guide is to wait for at least 50 or 100 impressions on a particular channel before drawing any conclusion from it.

Once an account is spending thousands of pounds a day, there's usually enough data to make decisions from the numbers alone. The majority of people watching a course like this are probably spending under £100 a day, in which case getting to a genuine single source of the truth, understanding what really works, can take three, four, five months. If that's a hard thing to hear, it's still the reality, because getting there means testing and learning, which means making assumptions you don't yet have the data to fully support.

How do you test assumptions properly?

The way forward is to test assumptions with small budgets, confirm whether they hold up, and only then scale.

You might see a higher view rate, more time on site and more sales from a particular age bracket, so the next assumption to test could be running a campaign purely on that bracket, or testing certain keywords and competitor audiences that already look stronger than others. It helps to write down why you're making a change before you make it, so you can review it a week or two later and see whether it turned out to be true. That's the only way campaigns genuinely learn: test, learn, reiterate, improve, aiming for one or two percent gains that eventually compound into big leaps.

There's a well known example of this kind of thinking outside marketing: Dave Brailsford, who coached the GB Olympic cycling team to gold, built his whole approach around one percent improvements across every area leading to winning. The improvement from any one test might be more than one percent, but the principle, continual small improvement across all campaigns, is the same one that applies here.

What did the webinar campaign teach about drilling into data too early?

A client's webinar campaign looked like an audience or content problem, but the real fault was the landing page.

Registrations to the webinar were sent from a campaign, and after around 200 clicks the campaign was paused because, although plenty of people were registering, watch time was averaging 20 to 30 minutes against a roughly 42 minute webinar, at which point the pitch and the sale attempt happened, and a lot of people were dropping off before then. It would have been easy to assume the audience was wrong, or that the bids, placements or age groups needed changing. Instead, a second webinar was split tested against the first, and with the same audience, people started reaching the end and paying. The actual fault was the landing page experience, not the audience or the front end targeting at all. Had the decision been made purely from the front end numbers, "no conversions, turn it off", the real, fixable problem would never have been found.

How do you avoid going too far into the data?

Keep drilling into the data where you have enough of it to support a conclusion, but always ask yourself first whether you actually have enough data to make the assumption you're about to make.

Big changes to a live campaign will send it back to the start to relearn, so tinkering, small unnecessary adjustments, is something to avoid. If a test shows something is working, for example TV outperforming mobile for a particular client by four or five times, that's worth building into its own campaign and letting it learn and scale on its own terms, rather than switching other parts of the account off abruptly and forcing a relearn.

This whole process comes down to gut feel built from experience, testing assumptions, seeing whether they hold, and using that to decide where to go next. It's genuinely difficult to teach directly, because everybody who runs ads develops their own style, there's no single paint-by-numbers approach to Google Ads and YouTube. What can be taught is structure: run certain campaigns in certain ways, test properly, get enough data before concluding anything, and use what you learn to keep making small, compounding improvements rather than big, unproven leaps.