Research4
Summarize PowerPoint content.
Please use simple grammar and phrases, 150 words.
Thank you
How to Measure BI Success
1
Defining BI Success is Challenging
For BI practitioners, understanding project critical success factors is crucial
Ask
How will I know my project is a success?
What measures should we track to confirm improved {enter project deliverable here, e.g. data access, cycle time improvements, increased sales} that we can directly attribute to {enter the project name here}
This session is about defining value and success to secure active support
2
BI Success and Business Impact
Vendors present this situation as worse than it is
That said, the degree of very successful BI deployments is low (note the mistake
in 2012, the second slightly successful should be very successful)
3
Source: Howson
BIs Contribution to Company Performance
It would be interesting to see what respondents meant by somewhat and slightly
Note: Neither somewhat nor significant cracked 50%
4
Source: Howson
Successful BIs Contribution to Performance
At 34% the percentage of initiatives significantly contributing to company performance is 10% higher than those who describe their solution as very successfully deployed
So A BI initiative can be perceived as being less successful but positively contributing to business performance. This might result from poor execution with good content
5
Source: Howson
BI Success and Business Impact Are Not Synonymous
Success Perceptions
IT = Technical implementation
Business = How the data is used
The gap between vision and reality often disappoints
It may be difficult to comprehend specific actions taken directly resulting from BI
A BI solution with great architecture may not have the intended, positive impact with its business users
6
Source: Howson
Profs Notes
Howsons point about ITs evolution toward a better business orientation mirrors the concept of IT as a service
Since BI is highly client facing this is crucial
Project managers (PMs) can bridge the gap
An effective PM captures success criteria, ensures project success and tracks and shares success stories with all parties
7
Justifying Success Managers Love Numbers
Most projects calculate Net Present Value (NPV) and Return on Investment (ROI) during the project but
rarely calculate it after transition to operation because its not easy to determine the ongoing returns
Where F = future cash flow and R is the companys discount, or hurdle rate
So
If an investment of $10 yields $20 one year hence, you should invest
If an investment of $10 yields $8 one year hence, you should not invest
Companies who are risk adverse tend to set higher discount rates than companies with a high appetite for risk
8
– Initial Investment
Source: Howson
Class Exercise: Lets Calculate NPV Using Excel
9
Download your Excel spreadsheet from Canvas module 4
Note how a difference in initial investment triggered acceptance or rejection for Company A, Project A
Note how a difference in the discount, or hurdle, rate triggered acceptance or rejection for Company B, Project B
BIs Return on Investment (ROI)
BI should be like a phone or the internet. Unfortunately, its not yet, so we have to invest more effort to justify the investment
Not everyone will agree whether revenue increases are specifically related to BI vs. other factors like training or organizational realignments
Cycle time savings tend to be a logical play but be careful: sponsors are never keen on promising staff reductions
Suggestion: Tie anticipated cycle time savings to potential growth increases to maintain status quo (this is cost avoidance)
ROI is a precise number derived from imprecise inputs. Always document and confirm the assumptions that went into your numbers! You will be held to account and assumptions provide a potential escape valve
Howson (2014)
10
ROI (Continued)
Average ROI for BI projects: 300 400% (some as high as 2,000%!)
For every $1 spent on analytics organizations earned an average of $10.66 (Nucleus Research, 2011)
Forrester (2012): if a composite organization of 1,500 employees with $500 million in sales invests in BI, they should see an ROI of 97%
ROI calculations provide a basis for comparison among BI implementations
11
Source: Howson
ROI (Continued)
Guesstimating ROI is better than nothing. Your company will probably require an ROI calculation with its project evaluation
For the revenue component, devise a number (it will be debatable) and obtain support prior to presenting it
Be ready to justify your numbers under heavy questioning!
12
Source: Howson
The Case of Data Mart A
Cost/Benefit Analysis (CBA) completed in Year 0
CBA was refreshed in Year 2
This was a data mart developed to support the small business insurance line of a major primary insurance company
Prof was the program manager for this initiative and then managed it in full operation for three years
13
Data Mart A: User Success Criteria
Captured during the project initiation phase
14
Data Mart A: From the CBAs Benefits Summary
When the Data Mart A program was initiated three years ago, management anticipated a loss ratio reduction of .01% resulting in additional profit of $3 million annually from cycle time savings and increased productivity
The subject matter experts have since removed this benefit because they confirmed there is no way of verifying whether additional profit was attributable to Data Mart A
The revised benefits included in this analysis cover only cycle time improvements and infrastructure and technical support savings
15
Data Mart A: The CBAs Reported Cycle Time Improvements
On Year 2 the subject matter experts confirmed:
Cycle time savings of 10% resulted from Data Mart As single version of the truth and simpler reconciliation procedures. This percentage is expected to increase to 25% starting with Data Mart As third year of operation
Time savings of 75% were also cited from Data Mart As provision of a comprehensive data dictionary. This means if analysis using prior applications took one hour to complete, improved data understanding now speeds the same analysis to finish within fifteen minutes
Therefore, for the applications first two years of operation, 85% of applied analysts time should be saved, with an anticipated increase to 100% starting year three. This means starting with the third year of operation, a request that would have taken one hour to finish in previous applications will now take less than a minute
16
Data Mart A: The CBAs Reported Cycle Time Improvements (Continued)
Despite this positive feedback, we considered our experience in removing the loss ratio benefit and referred to our original subject matter expert feedback captured during initiation that confirmed before Data Mart A, approximately 40% of applied analysts time was spent querying data
For year 1 we derived the cycle time benefit by halving the 40% applied time pre Data Mart A to 20% post Data Mart A and taking the difference
For year 2 onward, we halved the applied time again, to 10% and applied the difference
17
Data Mart A: The CBAs Reported Usage and Salary Impact
To achieve a number of applied resources, we derived an average monthly distinct user count using Data Mart A specific Cognos self service reporting usage statistics. As of May, year 2, an average of 200 distinct users have accessed Data Mart A monthly over a period of six months. We then took a very conservative 40% of that number and applied it for these benefits estimates
A conservative average annual salary, including benefits, of $75,000 was applied in this analysis for all personnel, regardless of organizational role
18
Data Mart A: The CBAs Reported Infrastructure Savings
With the implementation of Data Mart A a series of legacy applications were decommissioned, among them the The Express Database, its Access based ad hoc query tool, multiple data feeds, a legacy data mart and related storage. While the team was unable to procure a precise dollar amount for these savings, for the purposes of this analysis we have conservatively estimated them to be $5,000 annually
19
Data Mart A: The CBAs Reported Support Savings
Technical support is no longer required for the decommissioned legacy applications. Again, the team was unable to achieve a precise dollar amount, so a similar calculation was applied to that used to derive cycle time savings: the removal of the need for 25% of two employees time per year at an annual salary, including benefits, of $75,000
20
Data Mart A: Benefits Summary
21
Data Mart A: Cash Flow and ROI Analysis
22
Data Mart A: Net Cash Flow Analysis
23
Data Mart A: Cumulative Cash Flow Analysis
24
Data Mart A: Net and Discounted Cash Flow Analysis
25
The Case of the Dashboard Program, Project Proposal
Executive Management has requested an executive level dashboard to consolidate and display aggregated data pertaining to key performance indicators for the small business line.
Please Note:
The main savings from this project are in cycle time savings, which the program sponsors confirm equate to approximately two personnel annually.
This will not result in a workforce reduction (italics and color added), instead personnel will be redirected to more productive tasks upon implementation.
26
The Case of the Dashboard Program, Project Proposal
This CBA focuses on all phases with specific target focused on:
Flow, loss and premium information
An Account Executive Dashboard to provide meaningful and actionable metrics to support the field in their daily operations
An inventory management module which will provide account executives with account and policy level flow detail to support the management and growth of their territory
A prospecting module which will assist the account executive in identifying and flagging new prospects. Data from a variety of sources will be used to identify potential prospects
Cognos reporting functionality will be integrated with the tool. Reports will be for use not only by the account executives but by their respective managers to provide higher level “roll-ups” on multiple sales territories
27
Dashboard Program: Dashboard Program Goals and Benefits
Note: This slide refers to that portion of the dashboard program CBA focused on an executive dashboard
28
Dashboard Program ROI Example
29
Dashboard Program Net Cash Flow Analysis
30
Dashboard Program Cumulative Cash Flow Analysis
31
Dashboard Program Net and Discounted Cash Flow Analysis
32
So What Happened?
Data Mart A was originally the primary data store for the companys small business lines
An executive dashboard was built leveraging Data Mart A which wasnt used much because executives did not like hands on access
The account executive dashboard was then built leveraging Data Mart A. Since the CEO mandated its use it was a huge success
Note: Your Prof thought that a CRM system would have better served this need
33
So What Happened (Continued)?
The teams responsibilities grew!
They had been on their own supporting the small business line
They were given an underused corporate data warehouse to manage
Data Mart As goal was revised to become a true data mart, no longer accessing source files directly, it now accessed the corporate data warehouse
As your Prof left they considered leveraging lessons learned to leverage Data Mart A to reinvigorate an existing, little used enterprise data mart
34
Managerial Implications
Emphasize business benefits, not technology
The BI team must have both business and technical knowledge
Procure direct business input for critical success factors
Comfort with financial calculations and budgeting is critical
Project and program management skills are fundamental
ROI is most often described by a combination of the net present value (NPV), internal rate of return (IRR) and payback period
35
Reference List
Howson, C. (2014). Successful business intelligence: Unlock the value of BI and big data. New York. McGraw Hill Education.
ISBN: 9780071809184
36
36
ContactFindings
Business Sponsor
– 25% of 40 people’s time is spent on data reconciliation tasks
– Reasonable goal should be to cut that in half: 12.5%
– If that’s the case, then people should free up enough time for more analysis, resulting in improved pricing and profitability
– Thinks we can lower the loss ratio by .1. Against $3 billion in assets under management, that savings is approximately $3m.
User A
– Chuck currently spends 10% of his time in TED. User B spends approximately 25% of her time.
– Loses cycle time to incorrect data and rework
– Data Mart A could reduce cycle time and support faster decisions to support rate
– Selection will be more accurate
– Thinks an improvement in rate adequacy and a reduction in loss ratio is justifiable
User B
– Doesn’t see the same chasing of reconciliation issues as the others but does see the data quality issues being a problem.
– If a Data Mart A t is provided:
a. Sees increased usage among his five folks
b. Sees faster, better decision making
c. Thinks he can hit the hot spots sooner
User C
– Can’t respond to the question “how does my flow look on the 15th?”
– Every region is doing their own calculations
– Calculation:
a. 60 people on staff, at least 50 spend between 25 – 50% pursuing data.
b. They think that there’s an improvement in the loss ratio of at least .1%.
Projected Annual Cycle Time Savings
Average Annual Salary (incl. benefits)$75,000
* Number of Average Distinct Cognos Users (200 * 40%)$6,000,000
/ 40% (Estimated Analysis Time, Pre Data Mart A, 2009)$2,400,000
* 20% (Estimated Analysis Time, Year 1)$1,200,000$1,200,000Savings, year 1 (Pre Data Mart A % – Post Data Mart A %)
* 10% (Estimated Analysis Time: Year 2+)$600,000$1,800,000Savings, year 2+
Estimated Annual Hardware Savings $5,000(50% applied year 1)
Estimated Annual Support Savings, All Applications
Average Annual Salary (incl. benefits)$75,000
* Number of Support Personnel (estimated at 2)$150,000
* 25% (Conservative estimated time dedication)$37,500
Total Estimated Annual Support Savings$42,500(50% applied year 1)
5 Year Benefits ProjectionYear 0 = 2009Year 1 = 2010Year 2 = 2011Year 3 = 2012Year 4 = 2013Year 5 = 2014Total Over 5 Years
Cycle Time Savings$0$1,200,000$1,800,000$1,800,000$1,800,000$1,800,000$8,400,000
Hardware Savings$0$2,500$5,000$5,000$5,000$5,000$22,500
Support Savings$0$18,750$37,500$37,500$37,500$37,500$168,750
$0$1,221,250$1,842,500$1,842,500$1,842,500$1,842,500$8,591,250
** Note: Phase 1.0 was implemented in June, 2010
Goals: The Dashboard program produces an integrated view of Key Business Metrics in an intuitive dashboard visualization format resulting in a readily available,
consistent and well defined business metrics.
Benefits: The Executive Dashboard provides the following key business benefits: Cycle Time Savings, Integrated view of business drivers, Intuitive Dashboard
Visualization with readily available and actionable business metrics, it is the cycle time savings that are proposed heretic is estimated that two full-time employees spend
their time researching in addition to Agent level metrics and IT research ETL design enhancements to streamline the load process thus making it more manageable to
maintain. The time savings that will result will provide a platform to re-route these personnel to move value added work. Students will explore topics of interest focused on the managerial aspects of the course covered in class and discussed in the Howson book and will submit three research papers. Students should consider these papers management briefings and convey their points succinctly. Reference articles must have been published within the past three years.
Research Paper Specifications
The first research paper covers any topic of interest from class sessions 1 3 and corresponding Howson chapters. (The content is 07 Sep 2020 the PowerPoint, 14 Sep 2020 PowerPoint, and today PowerPoint)
Minimum Required Length: 2 pages
Minimum Required References: 3 references
Reference Citation Format: APA
Required Outline:
Introduction
Findings
Conclusion
Reference List.
Notes:
Title pages and reference lists do not count toward the length requirements. Students must submit two pages of original content.
Students cannot include the Howson book among the three required external references for these research papers. They may add this book as a third reference.
Research papers must consist of industry and academic sources. Use of Wikipedia for research papers is not acceptable.
