Data Quality And Data Science Are Not Polar Opposites
Big data gurus have said that data quality isn’t important for big data. Good enough is good enough. However, business stakeholders still complain about poor data quality. In fact, when Forrester surveyed customer intelligence professionals, the ability to integrate data and manage data quality are the top two factors holding customer intelligence back.
So, do big data gurus have it wrong? Sort of . . .
I had the chance to attend and present at a marketing event put on by MITX last week in Boston that focused on data science for marketing and customer experience. I recommend all data and big data professionals do this. Here is why. How marketers and agencies talk about big data and data science is different than how IT talks about it. This isn’t just a language barrier, it’s a philosophy barrier. Let’s look at this closer:
- Data is totals. When IT talks about data, it’s talking of the physical elements stored in systems. When marketing talks about data, it’s referring to the totals and calculation outputs from analysis.
- Quality is completeness. At the MITX event, Panera Bread was asked, how do they understand customers that pay cash? This lack of data didn’t hinder analysis. Panera looked at customers in their loyalty program and promotions that paid cash to make assumptions about this segment and their behavior. Analytics was the data quality tool that completed the customer picture.
- Data rules are algorithms. When rules are applied to data, these are more aligned to segmentation and status that would be input into personalized customer interaction. Data rules are not about transformation to marketers.
- Quality is about outcomes. The data marketers have is for influence. Marketers have feedback on results from applied algorithms and models in weeks, not months. They need to trust the result.
- Hadoop doesn’t matter. Marketers can get big data in a box. Agencies and marketing technology vendors have figured out how to make analytics and big data packaged and consumable.
What this means is marketing data scientists are able to do a lot to overcome dirty data in the course of customer and behavior analysis. Analysis dampens anomalies and outliers. Analysis can fill gaps by emphasizing pattern recognition over master data. Marketing data scientists work around data quality issues as well as address them through analytic insight.
However, this doesn’t diminish IT’s role in supporting marketing data scientists. IT still needs to support the heavy data integration requirements across marketing channels and third-party data sources. Creating consistency through data integration and quality best practices is critical to ensure integrity in behavioral and attribute linkage. Master data management still plays a strong role to know the customer and managed internal attribution of marketing data to sales and customer service.
In the world of marketing data science, IT may not be completely relieved of its data quality duties, but the game has changed. Lack of or reduced IT data quality services are not always a barrier to big data when in context of relevant quality customer insight. Yet IT still needs to support and certify data quality in the access and integration of data. It isn’t a question of good enough data; it’s about data quality efforts that matter to outcomes.