Measuring Learning s Impact on the Business. Jeffrey Berk Chief Operating Officer KnowledgeAdvisors

Transcription

1 Measuring Learning s Impact on the Business Jeffrey Berk Chief Operating Officer KnowledgeAdvisors

2 KnowledgeAdvisors Experts in metrics that matter KnowledgeAdvisors is a human capital analytics firm. We are the world s largest provider of learning and talent measurement solutions. February 16, , KnowledgeAdvisors 2

3 KnowledgeAdvisors Clients February 16, , KnowledgeAdvisors 3

4 What holds organizations back? The problem is not technology or approaches but commitment and resources What are your biggest challenges for effective training measurement? Bersin & Associates, High Impact Learning Measurement 343 Organizations, February, 2010 February 16, , KnowledgeAdvisors 4

5 How Much of L&D Should you Spend? 8 to 10% - what is spent by world class organizations 5 to 7% - what is spent by leading edge organizations 3 to 5 % - what should be spent by most organizations 1 to 3 % - what is spent by some organizations 0 to 1% - what is spent by most organizations Source: Josh Bersin, KnowledgeAdvisors Analytics Symposium 2010 February 16, , KnowledgeAdvisors 5

6 Linking to Shareholder Value In 2009 Companies investing in learning analytics beat S&P 500 by 17% Source: Human Capital Measurement and Its Impact on Stock Performance, KnowledgeAdvisors, 2009

7 Methods to Measure Impact on Business Option 1: Use Smart Sheet Benchmark Questions Option 2: Use Human Capital Approach Option 3: Use Advanced Results Evaluation Approach Option 4: Use Business Impact Templates Option 5: Use Actual Results Correlations Option 6: Use Causal Modeling February 16, , KnowledgeAdvisors 7

8 Showing Business Impact High Investment Once you determine Business Impact what analysis is appropriate, select a method: Requires actual results data and advanced analysis techniques Typically employs surveys Smart Sheets Advanced Business Results Human Evaluation Capital Approach Business Impact Templates Actual Results Correlations Causal Modeling Low Investment Simple Complex

9 Option 1: Smile Sheets into Smart Sheets February 16, , KnowledgeAdvisors 9

10 Use Smart Sheet Benchmark Questions Background: Large pharmaceutical sales training Business Issue: New Sales VP, not convinced of trainings value Analytics Solution: Used evaluation process consistently to measure value Analytics Results: Benchmarked sales performance vs. Metrics that Matter sales benchmark and outperformed the benchmark in excess of 20% on job impact. Convinced new VP that learning was a strategic tool February 16, , KnowledgeAdvisors 10

11 Making Decisions With Impact Data Cut or improve programs that aren t providing impact to the business We need our measurement tools and process now more than ever; if we don t have access to this data, how will we be able to make intelligent decisions on what to cut? - KnowledgeAdvisors Client February 16, , KnowledgeAdvisors 11

12 Option 2: Human Capital Approach Uses improved job performance as the overall measure of business impact Easy to use, practical, scalable, replicable Provides a financial ROI relative to the improvement in human capital performance Uses principles of estimation, isolation, and adjustment Assumption is that human capital s value (i.e. salary) is like any other asset that has expected value that can appreciate or depreciate over time Factor Item Amount Estimate Isolate Isolate Given all factors, including this training, estimate how much your job performance related to the course subject matter will improve % Based on your response to the prior question, estimate how much of the improvement will be a direct result of this training % What percent of your total work time requires the knowledge/skills presented in this training? 62.65% Adjust Bias, confidence, conservatism 65%

13 Use Human Capital Approach This report is used by a majority of organizations to provide high level indicators of performance change, linkage to business results and financial and non-financial ROI. February 16, , KnowledgeAdvisors 13

14 Option 3: Business Results Evaluation Approach Use specific business results as a measure of business impact Uses an advanced survey to estimate improvement in results Leverage technology to make practical and scalable Provides a financial ROI relative to the change in the business result that prompted the training Uses principles of estimation, isolation, and adjustment specific to the business result February 16, , KnowledgeAdvisors 14

15 Business Results Evaluation Process February 16, , KnowledgeAdvisors 15

16 Use Advanced Results Evaluation Approach Background: ERP company performing customer education Business Issue: Show value of training to customers in terms of increased productivity and quality Measurement Solution: Collected 12,770 end of program evaluations within 6 months asking participants to forecast impact Analytics Results: 18.2% estimated increase in productivity from the training (adjusted down for self-reported bias) Analyze & monetize this result Post Event Business Results: Productivity Total % improvement in productivity including training 51.0% Training's contribution to improved productivity 57.0% Percent improvement due to training 29.1% Adjustment factor for confidence in estimations 65.0% Adjusted percent improvement due to training 18.9% Analyze & monetize this result Post Event Business Results: Quality Total % improvement in quality, including training 54.0% Training's contribution to improved quality 57.0% Percent improvement due to training 30.8% Adjustment factor for confidence in estimations 65.0% Adjusted percent improvement due to training 20.0% Analyze & monetize this result Post Event Business Results: Customer Satisfaction Total % improvement in customer satisfaction including training 52.0% Training's contribution to improved customer satisfaction 57.0% Percent improvement due to training 29.6% Adjustment factor for confidence in estimations 65.0% Adjusted percent improvement due to training 19.3% February 16, , KnowledgeAdvisors 16

17 Option 4: Business Impact Template Monetize Results Estimate the Impact of Outside Factors Compare to Cost of Training February 16, , KnowledgeAdvisors 17

18 Use Business Impact Templates Background: Large CPG based in Canada, sales training Business Issue: Visible sales program, requested to link between it and sales results Analytics Solution: Used process to evaluate sales change and impact training had on sales Analytics Results: 7% increase in sales, 20% of the increase was due to training, this was a $74,000 increase per sales person when the training was $5000 per person February 16, , KnowledgeAdvisors 18

19 Option 5: Actual Results Correlations This is a method where actual results can be input or imported at an aggregate, periodic frequency. It is then correlated in time periods against training evaluation data for impact indicators. The example below is from sales data and sale training. February 16, , KnowledgeAdvisors 19

20 Option 6: Causal Modeling High level statistical analysis using Structural Equation Modeling Determines the key factors that influence business impact Causation not correlation Worthwhile Investment Courseware Quality Individual Learning The Causal Model can be used to predict future results and determine investments Instructor Effectiveness February 16, , KnowledgeAdvisors 20

21 Actionable Metrics Benchmarks: Internal or External Goals: Actual to Goal Trends: Time based analysis Dashboards: Concise Data February 16, , KnowledgeAdvisors 21

22 Performance Optimization February 16, , KnowledgeAdvisors 22

23 8 7 Culture supports and drives need for standards and common talent development processes and measurement 6 Strong desire to use the data for decision making Maturityi n how data is reported and used to improve end to end talent processes Standardization for a consistent and single point of analytics with integration to and from feeder systems 5 Measurement Sustainability 1 Strong leadership for driving measurement with governance to support it. 7 Data Data Utilization Utilization 6 Technology Technology Standards, Tools Standards have been created for data collection Data integrity exists 2 Roles 1 Leadership 5 Standards, Tools 3 Skills / Behaviors 4 Processes 3 2 Processes Roles and accountabilities for metrics are defined with consistent expectations on what should be measured. 4 3 Broad base of skills in analytics and measurement and strong capabilities in end-to-end measurement methods Established and consistent processes, technologies and communications for measurement 2/16/ KnowledgeAdvisors

24 Please enter your key ideas in the chat Key Ideas?

25 Turning Metrics Into Business Intelligence Analytics Symposium March 9 11,

26 Q & A

27 Contact Information For further information on the information contact: Jeffrey Berk Chief Operating Officer