By Boris Evelson

How do you know if your BI application has high, low or no ROI? How do you know that what the business users requested last month and you spent countless of hours and sleepless nights working on is actually being used? How do you know if your BI applications are efficient and effective? I don't have all the answers, but here's what I recommend.

Start with collecting basic data about your BI environment. The data model (hint, it's a classical multidimensional model exercise) should have the following components:

  •  Requests (these should be available from your help desk and project/portfolio management applications), such as
    • User provisioning
    • New applications
    • New data sources
    • Data model changes
    • New/changed metrics
    • New/changed reports
    • New report delivery options
  • Usage (these should be available from your DBMS and BI apps log files or from or by
    • Person
    • Time of day
    • Database
    • BI application
    • Report
    • Index
    • Aggregate
    • KMI/KPM
  • Track additional events like
    • Application usage vs. using application/report just to download or export data
    • Incomplete/cancelled queries


Surround these attributes by dimensions like department, line of business, geography, time, request/event complexity, and others. What you can now get out of such a model are reports on efficiency, effectiveness and all sort of analytics, or BI on BI. I highly recommend that these are a must in any BI Solutions Center or BI Competency Center as part of a standard operating procedure. Here are just some of the examples:

  • Efficiency
    • Ratio of business application users  to IT support staff
    • Ratio of power users (who can mostly fulfill their own BI requests) to casual/average users
    • Searching for a  report (opening a BI app, logging into a portal, clicking on tabs, going through menus and folders) vs. actually executing a report/query
    • Turnaround time for requests
  • Effectiveness
    • Self serviced BI requests vs. requested fulfilled by IT
    • User satisfaction (can be measured by some index that incorporates high usage of reports, shorter time looking for reports vs. longer time using reports, more app usage vs. less exports/downloads, no or few abandoned, cancelled queries, and others)
    • Correlate usage to profitability or market leadership. I don't know of anyone using this successfully but a few clients I talked to are in the planning stages to implement something like this. I offer to publish a case study about whoever implements this first!


Next you can report on and analyze this data for the following and many other purposes

  • Application, infrastructure tuning and optimization (weed out the junk that is not being used, or build an index/aggregate on a query that is running slowly)
  • Future BI investments justification and business cases
  • Incentive compensation for workers utilizing BI application as required or above and beyond the call of duty
  • Governance, compliance and risk management