Self-Service Predictive Modeling: Vendors Still Have Far to Go
Self-service analytics is one of my core coverage focus areas. It applies not just to business intelligence (BI) but also to advanced analytics.
When, a few months ago, I uttered the immortal phrase “roll over rocket scientists,” I was referring more specifically to the need for pervasive self-service tools for predictive analytics and data mining (PA/DM). Considering that my recently published Forrester Wave on PA/DM Solutions primarily addressed the traditional requirements of “rocket scientist” experts in statistical analysis, I did not put a huge emphasis on data mining features geared to business analysts, subject matter experts, and other “non-technical” information workers.
As I’ve stated in that blogpost and the follow-on podcast, the core problem with today’s PA/DM offerings is that many of them are power tools, not solutions that have been designed for the mass business market. Vendors such as SAS Institute, IBM/SPSS, KXEN, Oracle, Portrait Software, Angoss, FICO, and TIBCO Spotfire provide data mining specialists with feature-rich algorithm-powered solutions for modeling, scoring, regression, and other core PA/DM functions. Their core, traditional user base consists of statisticians, mathematicians, and other highly educated analytics professionals.
No one denies that traditional PA/DM tools are the analytical bedrock of mission-critical applications in diverse industries. But I challenge you to point to a single case study where they are used directly by the CEO, senior executives, or any other casual user, rather than indirectly through being embedded in some custom application.
Forrester regards PA/DM as key to the evolution of business intelligence (BI). However, this technology can’t realize its potential until it breaks out of its golden ghetto. BI vendors know this, and are focusing in 2010 on integrating both technologies into their solution portfolios while providing user-friendly visualization and development tools that submerge the complexities and accelerate time to insight.
Interactive visualization and exploration features are key. This is the core of what I sometimes refer to as “data mining lite,” and it’s also the heart of the new generation of in-memory BI clients. It enables information workers to discover new and actionable insights in information at rest or in motion, while proactively detecting and responding to events. And this functionality is at the heart of many next-generation BI tools, though many cannot begin to approximate the algorithm-driven sophistication of the leading PA/DM power tools covered in the Wave (though, I should note, TIBCO Spotfire’s BI-integrated predictive modeling power tool was in my Wave).
Forrester sees many of the leading PA/DM vendors offering more user-friendly, visual, exploratory tooling for the general business-analyst market. Though it wasn’t my core focus this time around, I had demoed for me, was briefed on, and evaluated several vendors’ self-service offerings under the Wave. Most of these are designed to improve the productivity of traditional PA/DM modelers, and also to support quick adoption and analysts by business analysts.
But PA/DM vendors have a long way to go till their offerings are truly ready for pervasive deployment throughout the information workplace. I think all vendors should continue to simplify their go-to-market messages, packaging, pricing, and architectures. Here are some high-level recommendations that should guide vendors in this regard:
- Make your self-service PA/DM tools an integral component of a BI stack that is architected for visual, user-centric, mashup-style development of predictive models as well as reports, queries, dashboards, charts, and other views.
- Provide a lightweight client-side install that does not depend on licensing or deployment of a heavyweight PA/DM tool/workbench, but can be integrated with and extend an enterprise’s investments in these heavyweight tools.
- Incorporate wizard-driven, guided, visual tools to step non-technical users—in other words, those without training in advanced statistics–through every step of the modeling, scoring, data prep, and other PA/DM processes.
- Offer expert, context-sensitive, task-oriented recommendations to help users solve their specific business problems.
- Provide prebuilt predictive models for the various horizontal business problems (e.g., customer churn analysis, financial risk analysis, etc.) for which information workers require predictive analysis, forecasting, and what-if modeling, as well as prebuilt vertical models for their specific industries.
- Provide a collaborative environment wherein non-technical modelers can collectively develop, validate, publish, share, and extend models among themselves, and also in conjunction with “technical modelers” (statisticians, data mining specialists), with full life-cycle model governance.
- Allow users to fully leverage and extend their company’s investments in BI, data warehousing (DW), extract transform load (ETL), data quality (DQ), and other analytics infrastructure/apps.
- Enable users to embed self-service PA/DM features in all transactional, collaborative applications and business processes, both premise-based and hosted/SaaS, and make these accessible to information workers through Web 2.0, REST, and other ubiquitous browser-oriented environments, and also through Microsoft Excel.
The self-service PA/DM tools on the market—from various of the vendors in my Wave, as well as from MicroStrategy, Microsoft, eThority, and others, meet various of these criteria, but no one vendor nails them all. Enterprise users should use these guidelines when sorting through the growing number of self-service predictive modeling options on the market.