Measuring Change Impact: There s a Gap for That!

Transcription

1 Measuring Change Impact: There s a Gap for That! Holly Burkett, PhD, SPHR Senior Organizational Development Specialist University of California Davis NOTE: Session content and sequence subject to minor revisions & updates. Case scenarios will be provided during the session.

2 Welcome, Introductions Opening Exercise 2

3 Our Focus Together Identify common levels and focus areas used to measure change impact Assess how change measures were applied to select scenarios Prioritize success indicators with an existing change effort or case scenario Tips, tools, and best practices Workplace Application Tapping into the Wisdom of the Crowd

4 Why Measure Change Impact? 4

5 The Case for Measuring Change Fast Facts Accelerated rate of change Increased investment in organizational change initiatives High degree of ineffectiveness in managing change

6 Increased revenue Most Common Change Drivers Cost savings 48% 51% Organizational growth 47% Process changes Customer needs Management changes New products/services Technology changes Org design changes 42% 42% 39% 39% 37% 35% 0% 10% 20% 30% 40% 50% 60% Source: Change Agents: The Role of Organizational Learning in Change Management. ATD Research 2014

7 Types of Change Process changes System changes Job role changes Organizational structure changes Plus. Which type of change is most common to you, your team, and your organization?

8 WHAT to Measure: Key Areas of Impact Individual Performance Knowledge Awareness Ability, proficiency Desire, participation Utilization Compliance Adoption Organizational Performance Project success Benefit realization Progress to plan Improved performance Readiness Integration, adoption Change Management Effectiveness Communication Engagement Sponsorship Training effectiveness Resistance Project Management

9 High Level Change Management Process Assessing needs, gaps Defining objectives Developing strategy Engaging sponsors Building a coalition Identifying project team Planning Implementing Managing communications Executing strategies, actions Assessing and managing risks Monitoring progress Integrating with project management Collecting and analyzing feedback Reinforcing change progress Celebrating success Institutionalizing change Establishing ongoing follow-up and maintenance processes Sustaining Where does measurement fit in?

10 High Level Change Management Process Assessing needs, gaps Defining objectives Designing, developing strategy Engaging sponsors Building a coalition Identifying project team Planning approach Initiating Implementing Managing communications Executing strategies, plans Assessing and managing risks Monitoring progress Integrating with project management Measurement Process Planning Data Collection Data Analysis Reporting Collecting and analyzing feedback Reinforcing change progress Celebrating success Institutionalizing change Establishing ongoing follow-up and maintenance processes Sustaining

11 Self-Assessment Activity

12 Consider.. Initiatives with Excellent Change Management are 6 Times More Likely to Meet Objectives Than Those with Poor Change Management Source: Prosci Research

13 WHAT to Measure: Key Areas of Impact Individual Performance Knowledge Awareness Ability, proficiency Desire, participation Utilization Compliance Adoption Organizational Performance Project success Benefit realization Progress to plan Improved performance Readiness Integration, adoption Change Management Effectiveness Communication Engagement Sponsorship Training effectiveness Resistance Project Management

14 Critique These Objectives As a result of this initiative, participants will: 1. Rate briefing sessions as entertaining and informative. 2. Be able to understand the difference among rebate, adjustment, and fee waiver. 3. Have a better understanding of the legal issues related to hiring and firing department employees. 4. Apply leadership skills. 5. Appreciate the viewpoint of other team members and perform as a great teammate. 6. Fully adopt the new technology.

15 Measuring Impact: Individual Performance Levels Level Measurement Objective 1. Reaction & Planned Action Measures participant satisfaction and captures planned actions 2. Learning Measures changes in knowledge, skills, and attitudes 3. Application (Behavior) Measures changes in on-the-job behavior 4. Business Impact (Results) Measures how changes in individuals on-the-job behavior impacted business outcomes 5. Return-on-investment (ROI) Compares monetary benefits of business outcomes to program costs Includes Intangible Benefits Source: Jack Phillips ROI Institute 15

16 Measuring Impact: Individual Performance Levels Level Sample Objectives 1. Reaction & Planned Action 80% of participants rate program on Likert scale. 2. Learning Participants will demonstrate how to guidelines provided 3. Application (Behavior) Managers will 4. Business Impact (Results) 70% of participating employees 5. Return-on-investment (ROI) Intangible Benefits: Improved attraction/retention, collaboration, communications 16

17 Criteria for Selecting Measurement Levels & Targets The of the program The criticality of the program to strategic objectives of the program The cost of the program The size of the target audience The investment of time Valid interest

18 Hallmarks of Effective Measures

19 Case Scenario Small Group Exercise

20 The Value Chain of Impact

21 Value Chain of Impact Example: Coaching Scenario Business Needs Payoff Worthwhile investment of resources Impact Increased leadership effectiveness, employee engagement Performance Conduct Development Discussions; increase leadership effectiveness Learning Define strengths, development needs Identify performance priorities Measurement Targets % ROI % Cost-Benefit % Labor Efficiency % Intent to Stay Isolate Effects of the Program % Utilization % Proficiency % Performance Improvement % Learning Gains Application Learning Measurement Levels ROI Business Impact How do you know it was the program? Isolate the effects. Reaction Timely, relevant, useful, customized % Relevance % Intent to Use Reaction & Planned Action 21

22 How Do You Compare? How Do You Measure Change Training Effectiveness? YOUR TOP TWO AS CHANGE MANAGEMENT PROFESSIONALS, WE MEASURE 1. Leaders satisfaction with change initiative outcomes 2. We don t measure the effectiveness of change management training 3. Percentage of employees who complete training 4. Key performance indicators 5. Learning test results 6. ROI of change management training

23 How Does Your Organization Measure the Effectiveness of Change Management Training? Leaders' satisfaction with change initiatives We do not measure effectiveness of change management training Performance of employees who completed training Key performance indicators 32% 38% 38% 46% Learning test results ROI of change management training 15% 18% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Source: Change Agents: The Role of Organizational Learning in Change Management. ATD Research 2014

24 Assessing & Optimizing Learning Impact Chunk it Write it down Space it out Learning Research Mix it up Retrieve it Test it Source: Adapted from Pichee (2016)

25 Assessing Performance Impact Coaching Initiative Application Questions / Results with Objectives (90 days) I regularly apply Strongly Agree Neutral Disagree Strongly NA the 5-step coaching Agree Disagree process when providing performance feedback 76% 24% 5% 5% 0% 0% I meet with my direct reports, at least monthly, to review their Development Plan 52% 26% 18% 4% 0% 0%

26 Performance & Impact Data Issues, Sample Questions

27 Data Collection & Analysis Tools, Tips, Considerations

28 Data Collection Common Approaches Surveys Observations Feedback Mechanisms Assignments Interviews Performance Monitoring

29 Surveys Recommended Items Progress with objectives Improvements, accomplishments implemented of program Knowledge/skill enhancement levels Skills used Materials used Perceived benefits to application Management support Suggestions for improvement Improving Response Rates

30 Sample Questions for an ROE Interview To what degree (from 0-100%) were your expectations for change management met? Please comment briefly on any unmet expectations that you rated below 80%. Please comment briefly on any expectations that met, or exceeded, your original goals. Would you come to our department again for similar needs? To what extent (from 0-100%) would you recommend our change management services to others?

31 Data Collection Issues and Considerations Communication Accountability Resources Commitment Timing

32 Common Barriers to Collecting Data in Higher Education

33 Polling Activity Where does data collection break down in your change management efforts?

34 The Impact of Constant

35 Trainee s Perceptions of Performance Barriers Lack of support from immediate manager Lack of support from peers or workgroup Organizational structures are not supportive Conflicting priorities

36 WHAT to Measure: Key Areas of Impact Individual Performance Knowledge Awareness Ability, proficiency Desire, participation Utilization Compliance Adoption Organizational Performance Project success Benefit realization Progress to plan Improved performance Readiness Integration, adoption Change Management Effectiveness Communication Engagement Sponsorship Training effectiveness Resistance Project Management

37 Most Common CM Mistakes Underestimating change impact Failing to hold people accountable Failing to communicate effectively Failing to remain actively visible and involved Failing to define performance targets Failing to follow through How Do You Compare?

38 Measuring CM Effectiveness Common Focus Areas Leadership (sponsorship, coalition support, engagement) Application (consistent, common approach) CM competencies Standardization (integration, governance) Would you come to our department again for similar needs? To what extent (from 0-100%) would you recommend our change management services to others? Small Group Discussion Adoption vs. Utilization Metrics

39 Levels of Outcomes The Logic Model illustrates the relationship between activities, outputs, and outcomes. INPUTS Labor Money Tools Policy Demand We provide CM training, coaching, consultation # of classes taught # coaching sessions Activity / Output so that Clients will apply CM practices to existing change projects # projects with change management plans Immediate Outcome so that CM will be embedded with all project plans % of projects with integrated CM % CM tools consistently applied Intermediate Outcome Improved CM capability across the enterprise % adoption rates % successful change projects Ultimate Outcome Change Management as a Process and a Competency

40 Measuring Change Impact: Review Individual Performance Organizational Performance Change Management (CM) Effectiveness Consider a current change project. Brainstorm 1-2 measures for assessing change impact in each of the 3 focus areas.

41 Recommended Best Practices Define performance targets at the front-end Establish accountability mechanisms Identify, distribute measurement responsibilities Provide initiative-specific follow-up after the change has been implemented Monitor employees post-change performance to assess application Communicate post-change success stories Measure What Matters

42 HOW RESILIENT CHANGE LEADERS DO IT

43 Reporting Results Sample Scorecards

44 What we know about measuring change What we know about driving change impact What we know about our role as change agents Closing Activity

45 Key TAKEAWAYS Planned ACTIONS Q & A

46 Let s Connect... THANK YOU! hjburkett@ucdavis.edu phone: