We attended the recent Glimpse Conference 2013, where members of New York's tech scene came together at Bloomberg headquarters to talk about social discovery, predictive analytics, and customer engagement.

Our key takeaway from the event: small, real-time data coming from very personal apps like email, calendar, social, and other online services will fuel next-level predictive apps and services. Specifically:

•    Better insight doesn’t require more data; it needs the right data. Amassing large databases of customer profiles, purchase history, and web browser activity only goes so far, and is costing companies millions, if not billions of dollars every year. Mikael Berner from EasilyDo sees a new opportunity in better utilizing data scattered across personal email indices, calendars, social networks, and file and content repositories that directly indicate customers’ plans, interests, and motivations. 

•    Email, calendar, and location data is a goldmine for predictive analytics. Expedia or TripAdvisor can track web activities to recall a user searched for hotels last November and is likely to travel again this year, but a flight confirmation sitting in email or vacation time logged in calendar is a much stronger indicator of travel plans.

•    Small data and big data interact. Real-time data provides immediate context for current activities, while historic logs of big data come into play in predicting what activity will come next. Eric Singley from Yelp mentioned its new “nearby” function that uses real-time data to figure out that it’s raining and you’re interested in going to the movies. Only then does it tap into big data collected from its larger user base to recommend a theater and predict what you’re likely to do after the show.

How do enterprises gain access to these valuable sources of context? It won't be easy but potential providers are emerging:

•    New mobile apps. Apps like Google Now, Cue (recently acquired by Apple), Tipbit, and EasilyDo ask users to volunteer their data from online services in exchange for personal assistance in daily activities. (We have a research report in preparation that will profile emerging vendors in this space.) These or other vendors may become independent intermediaries between users and enterprise systems of engagement.

•    Major mobile platforms. Apple, Google, Microsoft, Amazon, and Facebook already hold digital assets for predictive analytics through a variety of online service offerings like email, social, and file sync. And they hold digital trust with users, in the form of credit cards on file. They’ll follow Google in building proactive assistant apps like Google Now to better engage users, and explore using predictive technologies to play digital matchmaker between their users and a network of partners.

The key will be sharing such personal data between individuals and organizations in a clean and transparent way. Platform and app vendors will provide end-customers with: i) access and control over how their personal data is shared with institutions; ii) override control to share only data highly relevant to the relationship; and iii) the means to deny enterprises direct access to personal data.

How will widespread sharing of personal data impact the CIO and her organization? CIOs will see personal data impacting their:

•    App dev teams. They will seek more seamless access to data stored in consumerized apps. There are simply too many services individuals are using today to integrate on a one-off basis. Ink Mobility, for example, gives developers a single API window to access data from many of the most popular online services like Dropbox, Evernote, and Box.

•    Enterprise architects. They will build flexible IT systems for the marriage of big data and small, real-time data. Today’s servers and infrastructure are ill-prepared for the interplay between high velocity, real-time data and systems of record. Tomorrow’s systems will feature rapid analytics and extreme-scale operations on raw data in an affordable distributed data hub.

•    Infrastructure and operations pros. They will support the digital self, not just apps and device end-points. Imagine simple task automation for onboarding a new employee: detect a new employee, sign her into Yammer, and add her Evernote account to her new team’s shared license. The employee will be treated as a singular IT system.

Michael Yamnitsky is a Researcher at Forrester Research. You can follow him on Twitter @ItsYamnitsky.