With all the talk of data out there, who is actually using it and what are they using?  Turns out, most data used in the enterprise today comes from internal applications. That is starting to change, and the trend will accelerate.  But for now, when asked which data types were important to their firm’s overall business strategy, the majority of business intelligence users and planners cited internal sources such as transactional data from corporate apps or other customer data. Only about 1/3 of respondents reported the importance of external sources such as scientific data, partner data or other 3rd party data. Fewer still used unstructured external data such as Twitter feeds or other social media sources. Current data sources are limited.  Yet both business and IT decision-makers recognize the need to improve their use of data:  56% of business and IT decision-makers surveyed by Forrester see improving the use of data and analytics to improve business decisions and outcomes as a top priority. And, that potentially includes expanding the use of external data… if they can find it.   

Where do they go for external data?  What types of data might complement their transactional and other internal data? How can corporate strategists and market research teams identify new sources of information? Where can they find them, and how can they acquire and consume them? Can they be combined with internal data? Are the sources safe? reliable? sustainable?

We’re kicking off research here at Forrester that looks at data providers and enablers in the new data economy. Some are veterans of the data industry like Lexis Nexis  or Dunn & Bradstreet.  But others are new players who have entered the market to help facilitate the exchange and use of data.

For example, Enigma is a new search and discovery platform for public data – note they do not say “open data” but rather any data that is available to the public whether enterprise data, scientific data, academic data or open government data.  They describe the “public data paradox” in which data is out there and available but not accessible.  Public data remains in diverse formats – although that will slowly change with new government mandates for API access – and is not yet indexed and searchable.  As they put it “you are limited to the data you know about” and you can’t see the connections among different data sets.  At Enigma, they have built an infrastructure for acquiring, indexing and searching public data.  It makes it easy to find data on a particular subject, and most importantly find data sets you didn’t even know about. As they say, “a lot of people have been pioneering how we analyze the world with data” but what was missing was how to find that data.  They want to be the "Google" for public data.

Another example on my side of the pond is Data Publica. Like Enigma, they help businesses acquire data, yet with less emphasis on self-service or a “Google” approach.  Data Publica will help identify data sources, will extract and transform the raw data into a usable structure, and will deliver data as a service.  Delivery mechanisms range from dashboards and reports to data sets or streams, either as a one-off purchase or through subscription access.  Customer solutions include regional dashboards used to determine market opportunity and RFP notifications alerting potential bidders of a new request.  Small businesses who might not have the internal resources to watch the websites of multiple government agencies on a daily basis benefit from the RFP aggregation Data Publica provides by pulling data on public RFPs from over 100 sources.  Getting a timely alert that an RFP has been issued can be the key to winning the bid.

DataPublica also assists the other side of the equation, the producers/owners of public data.  In the City of Nantes, DataPublica provided guidance in the launch of the open data initiatives, advising on data structure, as well as the delivery of data through both visualizations and APIs.  Public organizations may produce data, but they face the perennial challenge of bringing it to market.

Given the skills shortages and difficulty in recruiting data expertise, enablers like Data Publica and Enigma are key to the development of the data economy. Stay tuned on more Forrester research on these enablers of the data economy.