Yesterday Facebook blogged "An Update on Metrics and Reporting" to inform marketers that several of their organic metrics were misreported including organic reach, time spent on Instant Articles, and follower count. This follows Facebook's revelatory September announcement that their video metrics contained a discrepancy and were over-reported.

The industry was outraged. But who’s really at fault here? Tina Moffett and I break it down like Judge Judy.

For The Defense: Facebook

How long did it take TV to standardize measurement? The social media industry is young and a fast-changing work in progress. Lest we forget, Facebook is only 12 years old and started as a platform for people; metrics was the furthest thing from their minds. But, it’s now the biggest social network serving people, brands, and publishers and is constantly having to prove value to all three. We would be surprised if Facebook was not screwing up along the way. To build their platform, Facebook will continue to mess up and correct, mess up and correct. Instead, let's focus on how Facebook's metrics woes will impact their relationship with marketers (and agencies):

  • Facebook’s perceived value will diminish. While its metrics snafus do not affect marketer ad costs, Facebook fails to acknowledge that their miscalculations impact Facebook’s effectiveness in a brand’s overall marketing mix. Marketers believed that Facebook was effective but now, because calculations are misaligned, marketers aren't sure of Facebook’s value and regress to operating in a social media black box circa 2010.
  • Facebook will lose marketers' trust but gain responsibility. Marketer trust will continue to erode as Facebook makes more of these announcements. While that doesn’t mean it will lose marketers (Facebook might have reached “too big to fail" status), the bigger it becomes, the more responsibility the company assumes. To regain trust, Facebook must incorporate data transparency and governance.

For The Prosecution: Marketers (And Agencies)

Marketers already have a tenuous relationship with Facebook and the latest from Facebook will leave them disillusioned over Facebook’s effectiveness in connecting with customers. And, they will feel frustrated because the metrics they do have aren’t comparable and don’t allow them to benchmark progress year-over-year, month-over-month, or campaign to campaign. Marketers will have to retroactively fix data for accurate reporting and "ain't nobody got time for that." But while we feel marketers’ pain, they aren’t off the hook, either. Marketers are equally at fault because:

  • Marketers took Facebook’s metrics at face value. Like lemmings, marketers followed the industry when Facebook set the bar for social measurement. Facebook’s initial metrics were rudimentary (example: only providing reach instead of impression data) yet they became the norm. Now marketers struggle to calculate Facebook’s value because there are limited metrics and a dearth of insights on content performance — and Facebook is the social network industry leader (now that’s a scary thought!).
  • Marketers still lack metric rigor. Marketers must do a better job of asking fundamental questions about their metrics. For example, they need to probe if video view time is inclusive of the whole audience or if it excludes certain audiences. These basic accuracy questions will create clarity and engender confidence. Pressuring social networks to go deeper in improving metrics will benefit social networks and marketers alike.

The Verdict

Facebook is trying to demonstrate a serious focus on social metrics: they started a metrics blog for marketers, created an internal measurement standards committee, named an external metrics committee comprised of marketers and agencies, and sought deeper partnerships with third-party verifiers like Moat and Nielsen. This is a good start. Facebook must also build expertise around marketing and media strategy — a faster path to developing the apples-to-apples metrics across digital properties that marketers crave. 

Marketers (and agencies) must focus on building metrics-driven organizations and audit their measurement practices. These audits should probe data sources, data quality, and calculations. Third-party verification companies can help marketers uncover potential metrics miscalculations or inaccuracies. Finally, marketers must look to the Interactive Advertising Bureau, Advertising Research Foundation, and Marketing Accountability Standards Board for metrics guidance.  

In summary: Both sides can win if they work together to create stronger and deeper metrics and finally answer, “Is my Facebook presence driving real business results?”