Millions of years ago, before even the Internet existed, I was briefly involved in a research project with a large US beverage manufacturer (think dray horses pulling a cartload of beer in barrels). We were in the run-up to game day for the Super Bowl, and the client wanted to understand if a campaign involving billboard and newspaper advertising, combined with a new point-of-sale promotion, would cost-effectively increase net-new sales. We devised a test in which we matched different zip-coded markets by a variety of factors (e.g. economic status), then ran a relatively sophisticated version of an A/B split test to see if the campaign would yield results. Sales did indeed spike in the markets in which we ran the campaign, but mostly in rural zip codes, which helped us conduct targeting.

I was reminded of this finding when I read a recent article by Jesse Frederik and Maurits Martjn titled “The New Dot-Com Bubble is Here and It’s Called Online Advertising.” In the article, the authors make a startling claim: The vast majority of data that we use to show the ROI of demand-centric digital advertising is wrong, and that most digital advertising spend is completely wasted.

The essence of their argument rests on a simple concept called selection bias. Selection bias exists everywhere, but it is particularly evident on social media review sites like Yelp!, where rabid fans and haters are the most likely to leave comments and reviews. This self-selection results in a very skewed perspective, with the silent majority under-represented. In a different guise, the same thing happens with digital advertising, where we hyper-target the people who see our ads. Imagine you have the thankless job of distributing fliers for a local pizza restaurant, and you have to decide where to go to get the best results. Perhaps you should go to the busy traffic intersection or outside the office complex down the street? The answer, of course, is to stand right outside the door of the pizza joint.

In effect, with hyper-targeted digital advertising, we’re often doing the equivalent of standing outside the door of the restaurant. Almost everyone who grabs our flier (i.e. who clicks on the ad) was already heading in to buy a slice with pepperoni. There’s a mountain of data to support this argument, from people with real skin in the game like Yahoo!, Facebook and Google. When they ran sophisticated tests very similar to the ones I ran for the Super Bowl campaign, they found that the net economic benefit of much digital advertising shrank by more than 80%.

There are two key takeaways from these findings.

First, running tests of your digital advertising is essential if you want to understand if all your campaigns are really as effective as you think. Simple, well-designed A/B tests are often enough to discern the real demand uplift.

The second takeaway begins to explain why apparently sane companies paid more than $5.6M for a 30-second spot at the 2020 Super Bowl, according to AdAge: It’s all about the brand. Super Bowl ads (like all advertising) are geared to emotional appeals and generating associations with the audience that are memorable and affirming. They also generate buzz and earned media coverage, to say nothing of signaling to partners and employees their investment in the brand itself. Many B2B companies focus on the demand-benefits of digital advertising, but they don’t account for how advertising primarily lifts awareness, perception and preference for the company.

Leaders of B2B advertising programs need to consider the long-term brand benefits as a primary motivator for the investments they make, and begin to measure reputation as an integral part of their programs. The SiriusDecisions Brand Measurement Framework can help by showing a way to track brand performance over time. You may not be at the Super Bowl, but that doesn’t mean you should discount how your digital advertising programs can help build your brand.