Evaluation of Human Resources Management & Development Interventions (Calculating ROI)

Transcription

1 The Human Resources Management Association of Jamaica Conference 35 November 20 th, 2015 Evaluation of Human Resources Management & Development Interventions (Calculating ROI) Caroline Hubble, MSOD, CPLP, CRP

2 Objectives Calculate the ROI in programs and projects Describe the ROI in the context of other measures Isolate the effects of programs on improvement in key business measures Determine when to use ROI 2

3 What is ROI? BCR = Program Benefits Program Costs ROI = Net Program Benefits Program Costs X 100 3

4 Try it! BCR = $750,000 $425,000 $750,000 - $425,000 ROI = X 100 $425,000 4

5 Try it! BCR = $750,000 $425,000 = 1.76:1 $750,000 - $425,000 ROI = X 100 $425,000 = 76% 5

6 Human Capital Investments Call for ROI Capital Investments (15%) Non-Capital Investments (85%) Buildings Vehicles Equipment Companies Tools Some Technology Marketing Some Technology Human Resources Staff Support Quality Processes 6

7 The New Definition of Value Value Must: Be balanced, with qualitative and quantitative data Contain financial and non-financial perspectives Reflect strategic and tactical issues Represent different time frames Satisfy all key stakeholders Be consistent in collection and analysis Be grounded in conservative standards Come from credible sources Create a call for action 7

8 Levels of Evaluation Measurement Focus Typical Measures 0. Inputs and Indicators The input into the project in terms of scope, volume, efficiencies, costs Participants, Hours, Costs, Timing 1. Reaction & Perceived Value Reaction to the project or program, including the perceived value Relevance, Importance, Usefulness, Appropriateness, Intent to use, Motivation to take action 2. Learning Learning to use the content and materials, including the confidence to use what was learned Skills, Knowledge, Capacity, Competencies, Confidence, Contacts 3. Application & Implementation Use of content and materials in the work environment, including progress with actual items and implementation Extent of use, Task completion, Frequency of use, Actions completed, Success with use, Barriers to use, Enablers to use 4. Business Impact The consequences of the use of the content and materials expressed as business impact measures Productivity, Revenue, Quality, Time, Efficiency, Customer Satisfaction, Employee Engagement 5. ROI Comparison of monetary benefits from program to program costs Benefit-Cost Ratio (BCR), ROI%, Payback Period 8

9 The ROI Methodology Evaluation Planning Data Collection LEVEL 1: REACTION AND PLANNED ACTION LEVEL 3: APPLICATION Develop/Review Objective of Program Develop Evaluation Plans and Baseline Data Collect Data During Program Implementation Collect Data After Program Implementation LEVEL 2: LEARNING LEVEL 4: BUSINESS IMPACT 9

10 The ROI Methodology Capture Costs of Program Data Analysis Reporting Isolate the Effects of Program Convert Data To Monetary Value Calculate the Return On Investment Generate Impact Study LEVEL 5: ROI Identify Intangibles INTANGIBLE BENEFITS 10

11 Operating Standards 1. Report the complete story 2. Conserve resources 3. Use the most credible sources 4. Choose the most conservative alternatives 5. Isolate the effects of the program 6. No data no improvement 7. Adjust estimates for error 8. Throw out the extreme and unsupported claims 9. Use first year benefits for short-term programs 10.Include fully-loaded costs 11.Report intangible benefits 12.Communicate results to all stakeholders 11

12 Chain of Impact Inputs Reaction Learning Application Isolate the Effects of the Program Impact ROI Intangible Benefits aaaaaaaaaaaaaaaaa aaaaaaaaaaaa 12

13 Criteria for Evaluation to Impact & ROI Life cycle of the program Linkage of program to operational goals and issues Importance of program to strategic objectives Top administrator interest in the evaluation Cost of the program Visibility of the program Size of target audience Investment of time required 13

14 Isolation Methods Method Control Groups Trend Line Analysis Forecasting Methods Participant Estimates Manager Estimates Sr. Management Estimates Expert Input Customer Input 14

15 Classic Control Group Design What is the difference in the change in performance? Experimental Group Pre-Measure Program Post-Measure 35 5 Control Group Pre-Measure Post-Measure Experimental: 35-5 = 30 Control: = 8 P = 30-8 = 22 15

16 Trend Line Analysis 18.5% Pre Program Average 20% Fraud Incident Rate 10% Fraud Policy 14.5% Projected Average 7% Post Program Average 0% J F M A M J J A S O N D J MONTHS 16

17 Estimation Monthly increase in credit card accounts: 175 (fact) Contributing Factors Consensus Impact (%) Average Confidence (%) Skill-Based Pay 32% 83% Incentives 41% 87% Management Reinforcement 14% 62% Market Fluctuations 11% 91% Other 2% 91% 100% 17

18 Example of Estimations Influence Fact % Contrib. Est. Impact Confidence Adjusted Impact Skill-Based Pay % 56 83% Fact: 175 New Credit Card Accounts % Contribution: 32% Est. Impact: 56 Uncertainty 17% Margin of Error: +/

19 Healthcare, Inc. Data Collection Techniques (3) Self Assessment Questionnaire 6 months after program (3) Employee Survey (25% Sample) 6 months after program (4) Complaint and Turnover Records 12 months after program Isolation Techniques Complaints Trend Line Analysis Turnover Forecasting Data Conversion Techniques Complaints Historical costs and input from experts (internal EEO/AA staff) Turnover External studies within industry Program costs Fully-loaded including needs assessment, development, coordination, salaries and benefits of initial meeting, and evaluation Total Costs - $277,987 Benefits Complaints Value of one complaint - $24,343 Prevented over one year Turnover Value of one turnover statistic - $20,887 Prevented over one year 136 ROI = ($360, $2,840,632) - $277,987 X 100 = 1,051% $277,987 19

20 Financial Program Skills Based Pay Systems Business Impact Employee Turnover Staffing Level Customer and Job Satisfaction Product Sales Cross Selling 20

21 Mining Program Employee Retraining Business Impact Performance as a function of transported tons, average distance traveled, and effective time for gravel tons 21

22 Plastics Manufacturing Program Self-Coaching for Managers Business Impact Work Engagement (Utrecht Work Engagement Scale) Controllable Waste Rework 22

23 Technology Program Customer Service Training Business Impact (Target) Customer Complaint Escalation 23

24 What Makes a Good ROI? 1. Set the value at the same level as other investments 15% 2. Set slightly above other investments 25% 3. Set at breakeven 0% 4. Set at client expectations 24

25 Next Steps Assess your readiness for ROI Identify stakeholders and their data needs Determine the purpose of your evaluation practice Identify programs suitable for ROI Develop capability in the ROI Methodology 25

26 Questions & Comments Caroline Hubble

27 Join us for ROI Certification Orlando, Florida November 30 th -December 4th hosted by Jet Blue *This program has been approved for 32 (HR General) recertification credit hours toward PHR, SPHR and GPHR recertification through the HR Certification Institute. 27

28 28