Redefining Measurement for Continuous Learning

Transcription

1 Redefining Measurement for Continuous Learning How to demonstrate impact when learning goes beyond L&D Todd Tauber Vice President, Learning Research Bersin by Deloitte, Deloitte Consulting LLP April 1, 2014

2 Contents The importance of measuring impact Persistent challenges to measuring impact New complexities in measuring impact Redefining measurement and evaluation 1 IMPACT 2014 THE BUSINESS OF TALENT

3 Todd Tauber Vice President, Learning Research Bersin by Deloitte, Deloitte Consulting LLP

4 The importance of measuring impact

5 Measuring impact is about demonstrating and communicating the value of L&D to the organization If you don t tell people about your success, they probably won t know about it. Donald Trump, The Art of the Deal 4 IMPACT 2014 THE BUSINESS OF TALENT

6 Share of S&P 500 Value Learning is increasingly important Companies are increasingly relying on human capital 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 83% 68% 32% 20% 20% 80% 80% 68% 32% 17% 1970s 1980s 1990s 2000s 2010s Tangible assets Intangible assets Source: Ocean Tomo LLC, Intangible Asset Value, 4/ IMPACT 2014 THE BUSINESS OF TALENT

7 % of payroll Organizations expect a lot from L&D Organizations invest a lot in L&D 4% 3% $1,169 per learner 2% 1% 0% Source: ASTD, 2013 State of the Industry, 10/2013; Bersin by Deloitte, 2014 Corporate Learning Factbook, 2/ IMPACT 2014 THE BUSINESS OF TALENT

8 Three ingredients for high-impact learning and employee satisfaction isn t one of them Alignment Effectiveness Business and organizational impact Efficiency Source: Bersin by Deloitte, Becoming a High-Impact Learning Organization, 8/ IMPACT 2014 THE BUSINESS OF TALENT

9 Learning isn t working for many organizations It s not aligned It s not effective It s not efficient 15% 10% 50% of global HR executives believe their organization is ready with the right people with the right skills in the right places. of the knowledge and skills people learn in training are actually applied on-the-job with positive results of the money spent on employee learning and development is wasted irrelevant, redundant, not absorbed, not applied. Source: Deloitte, Global Human Capital Trends, 3/2014; Chief Learning Officer, Scrap Learning and Manager Engagement, 3/ IMPACT 2014 THE BUSINESS OF TALENT

10 L&D staff / 1,000 learners Learning is essential, but L&D is vulnerable L&D is often one of the first things to get cut when times are tough Bersin by Deloitte, 2014 Corporate Learning Factbook, 2/ IMPACT 2014 THE BUSINESS OF TALENT

11 Would anyone miss you? I told them not to send [the reports] to anyone Guess what? At the end of the month, hardly anyone had asked for their reports, so they clearly weren t getting any insights from them. [But] if we had asked them if they wanted their reports, I m sure they would have said yes. The moral of the story is that overuse or misuse of data analytics is in danger of diverting attention away from the real business issues. former Chief Learning Officer, manufacturing company Source: Henley Business School, Big Data and HR, 7/ IMPACT 2014 THE BUSINESS OF TALENT

12 Persistent challenges to measuring impact

13 Three stubborn hurdles What are the biggest challenges you have in measurement? Where to start? Who / what it s for? What methodologies to use? People and relationships Systems for capturing data Tools for analyzing data Skills Funding Leadership / management support Frameworks Infrastructure Culture Source: Bersin by Deloitte, Becoming a High-Impact Learning Organization, 8/ IMPACT 2014 THE BUSINESS OF TALENT

14 The customer is always right How CEOs measure learning success We should measure this 96% We do measure this 74% 82% 78% 61% 53% 8% 4% 11% 28% 32% 22% Impact ROI Application Cost efficiency Learning Reactions Source: ASTD / ROI Institute, Measuring What Matters: How CEOs View Learning Success, 12/ IMPACT 2014 THE BUSINESS OF TALENT

15 Directional insights, not precise calculations Where people get messed up is in thinking that they have to measure individual training classes to make sure that participants absorbed the information and are applying it to their jobs. I'm not interested in that, and neither is our CFO. He wants to measure whether managers are performing well and whether the organization is productive. Chief Learning Officer, technology company Source: CFO, Measured Response, 6/ IMPACT 2014 THE BUSINESS OF TALENT

16 Value Business metrics, not busy metrics More mature L&D groups dig deeper into talent and business outcomes 100% Business leaders CLOs Instructional designers Job impact (Level 3) Business impact (Level 4) 50% Impact on business metrics ROI Manager satisfaction Learning Scores (Level 2) Costs Utilization Hours delivered Learner satisfaction (Level 1) Enrollments 0% 0% 50% 100% Frequency Source: Bersin by Deloitte, Practical and Proven Approaches to Learning Measurement, 4/ IMPACT 2014 THE BUSINESS OF TALENT

17 New complexities in measuring impact

18 Knowledge / skills Training event Continuous learning happens with and without L&D Learning isn t just programs and courses anymore Continuous learning Traditional training Reinforcement via Coaching & mentoring Peer feedback Job aids / tools Online / mobile courses On-demand content Conferences Professional networks Social network Communities of practice etc. Time 17 IMPACT 2014 THE BUSINESS OF TALENT

19 Continuous learning activities can't be measured as simply as events, courses and programs We learn through interaction, everywhere, all the time Performance support Websites Job rotations Interviews Teaching Collaboration Conversations Mentoring Dialogue Peers Meetings Feedback Manuals Successes Lectures Instructions Simulations Mistakes Virtual worlds Demonstrations Role Playing Debates Webinars Goals Whitepapers Coaching Experiments Books Customer data Presentations Lunch-and-learns Play Communities-of-practice After action reviews Podcasts Wikis Conferences Observation Journaling Micro-blogs Discussion boards e-learning Articles Onboarding Chalk talks Following Visual aids Video Advice Labs Social networking Case studies Role models Search Post-mortems Consulting Job shadowing Stretch goals Tagging 18 IMPACT 2014 THE BUSINESS OF TALENT

20 More digital learning means lots more stuff to track What % of learning hours are delivered via Other things Online self-study Live virtual Live instructor-led Source: Bersin by Deloitte, 2014 Corporate Learning Factbook, 2/ IMPACT 2014 THE BUSINESS OF TALENT

21 Data is fragmenting way beyond the LMS There s a lot more to collect and connect Learning systems Multiple LMSs vilt systems Collaboration platforms Learning portals Business systems ERP Finance Business intelligence CRM / marketing HR / HCM systems ATS / assessment Core HRIS Workforce analytics Talent management systems Other systems Social media Web analytics Mobile apps MOOCs 20 IMPACT 2014 THE BUSINESS OF TALENT

22 Growing demand for better insights Are your business leaders asking for quantitative measures of HR / L&D? In 2009, 21% said yes In % are saying yes (and 29% expect them to soon) Source: KnowledgeAdvisors, The Business Drivers of Talent Analytics, 3/ IMPACT 2014 THE BUSINESS OF TALENT

23 Expectations outpacing capabilities Which functions have strong analytics capabilities? Finance Operations 77% 81% Sales Marketing 58% 56% HR 15% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Bersin by Deloitte, High-Impact Talent Analytics, 10/ IMPACT 2014 THE BUSINESS OF TALENT

24 The more things change, the more they stay the same Same old lack of know-how, systems and support Frameworks Infrastructure Culture For informal learning For on-demand learning For continuous, self-directed learning New data New data sources New systems and tools Changing demands Increasing expectations Different skills and capabilities 23 IMPACT 2014 THE BUSINESS OF TALENT

25 Redefining measurement and evaluation

26 The basics still apply Have a plan 60% of L&D leaders we surveyed said their companies don t have a real plan for measuring learning impact. Build relationships with stakeholders 1.2x companies that regularly engage with stakeholders are more likely to have high-impact measurement capabilities. Dedicate resources and manpower 2% 64% of L&D organizations spend 2% or less of their budgets on measurement. 72% think they should spend more. Source: Bersin by Deloitte, Key Trends in the Measurement of Talent Development, 3/2012; Bersin by Deloitte, Becoming a High-Impact Learning Organization, 8/ IMPACT 2014 THE BUSINESS OF TALENT

27 Rethink yesterday s tools and practices Kirkpatrick and Phillips methods are only the beginning Too simplistic Too hard And too backward Does liking a course or passing a test really translate directly into better performance or business results? There are no clear, consistent definitions and it s rarely simple or easy to isolate learning s contribution ( or L&D s). Finding out after training that people were not satisfied or that results didn t improve doesn t tell you what to change. Source: Bersin by Deloitte, Key Trends in the Measurement of Talent Development, 3/ IMPACT 2014 THE BUSINESS OF TALENT

28 Build (some) new capabilities Utility / usefulness Stakeholder satisfaction Experience API (xapi) Learning records stores Employee engagement Readiness / promotions Information technology Adaptive learning Employee retention Finance and accounting Web analytics Time to proficiency Marketing Social listening Activity / contributions Business intelligence CRM systems Influence and feedback New friends New tools New metrics 27 IMPACT 2014 THE BUSINESS OF TALENT

29 Adopt some new approaches Good enough is good enough Predictive analytics Continuous monitoring 28 IMPACT 2014 THE BUSINESS OF TALENT

30 Find inspiration in finance and marketing Marketers measure and optimize ad campaigns continuously Market conditions Season Consumer confidence Fuel prices Marketing actions TV advertisements Online ads Social media Competitive activity Product releases Pricing Promotions Analyze Optimize Allocate Response Search Online chatter Store visits Purchases Outcomes Unit sales Revenues Margins Market share Share of voice Customer value Source: Adapted from Harvard Business Review, A Better Way to Measure Your Ad Campaign, 3/ IMPACT 2014 THE BUSINESS OF TALENT

31 The payoff Better alignment which (often) leads to better relationships with business leaders More effective learning And more efficient operations which (often) leads to more credibility and opportunities which (often) makes it easier to gain access to resources 30 IMPACT 2014 THE BUSINESS OF TALENT

32 This publication contains general information only and Deloitte is not, by means of this publication, rendering accounting, business, financial, investment, legal, tax, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor. Deloitte shall not be responsible for any loss sustained by any person who relies on this publication. As used in this document, "Deloitte" means Deloitte Consulting LLP, a subsidiary of Deloitte LLP. Please see for a detailed description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting. 36 USC Member of Deloitte Touche Tohmatsu Limited