#amecsummit. amecglobalsummit.org. Amecorg

Transcription

1

2 The Cutting Edge in Evaluation Metrics and Insights that Cut it in the C-suite Distinguished Professor Jim Macnamara PhD, FAMEC, FAMI, CPM University of Technology Sydney; The London School of Economics and Political Science Madelon Engels Head of Strategic Communication, Achmea International, The Netherlands

3 Introduction Basic, quick and easy metrics are necessary Progress tracking measures Low budget projects Considerable focus on basic metrics Talk about a seat at the table in the boardroom What s the ticket to the boardroom look like? What levels of expertise and evidence are expected in the C-suite?

4 What does the C-suite want? Inputs Activities Outputs OUTCOMES IMPACT Research Consultation Writing Media relations Media coverage Events Awareness change Support for a cause High reputation / brand Pre-testing Organising events Web content Membership growth Producing videos HTML coding Etc. Sponsorships Brochures Newsletters Attitude change Profits Cost savings Etc. Reduced violence against women COST CENTRE Behaviour change Improved public health Reduced road toll VALUE ADDING CENTRE

5 What does the C-suite want? Inputs Activities Outputs OUTCOMES IMPACT Research Consultation Writing Media relations Media coverage Events Awareness change Support for a cause Racial tolerance Pre-testing Organising events Web content Membership growth Strategic insights (e.g., identify gaps, opportunities, etc.) Producing videos HTML coding Etc. Sponsorships Brochures Newsletters Attitude change Profits Cost savings Etc. Reduced violence against women VALUE COST CENTRE Behaviour change Improved public health Reduced road toll VALUE ADDING CENTRE

6 High end evaluation four examples Social network analysis (SNA) and social influencer mapping Behavioural insights From behavioural economics / behavioural psychology Net Promoter Score (NPS) Closed loop methodology (Reichheld and Bain & Company) Customer journey mapping Customer decision journey (McKinsey)

7 Social media influencers Popular bloggers Web sites that review products and services Social media users with large numbers of follows Social media sites of respected organizations Online cross-networkers who connect hubs Celebrities

8 Social network analysis

9 Social network analysis Social network analysis (SNA) is the mapping and measuring of relationships and flows between people, groups, organizations, computers, URLs, and other connected information/knowledge entities.

10 Behavioural insights Behavioural economics Recognised economic incentives do not always influence behaviour Behavioural psychology Understanding choice architecture how humans make decisions Identifying drivers of behaviour habitual, instinctive, emotional, social, as well as rational (informs nudge theory ) Book by Richard Thaler and Cass Sunstein: Nudge: Improving Decisions About Health, Wealth, and Happiness Behavioural Insights Team (BIT) Spun off from the UK Government

11 Behavioural insights Case study 5.5 million hospital outpatient appointments were no shows in (NHS England, 2014) Cost British taxpayers 225 million a year (National Audit Office, 2014) 2 x randomized controlled trials (RCTs) were conducted in to test various forms of reminders and the content of reminder messages SMS text messages performed better than other reminder methods such as telephone calls or Then 4 x different SMS text messages were tested with 10,000 patients to identify the most effective wording The nudge testing found: Conformity message ( most people keep their appointments ) increased attendance Characteristics most effective were (1) Personalized language including directly addressing recipients as you (2) Stating the cost to the NHS and to you of each missed appointment (3) Listing and linking the phone number to call to cancel (make it easy) Reduced missed appointments from 11.7% in the control group to 8.3% Estimated savings of up to 40 million a year

12 5 th largest insurance group in Europe Established in 1811 as a mutual (co-operative) by Dutch farmers continues as a mutual today Operates in the Netherlands, Greece, Turkey, Slovakia, and Australia 19.5 billion premiums annually (income); 115 billion assets under management 13 million customers worldwide 14,500 staff Major brands include

13 Net Promoter Score (NPS) Developed by Fred Reichheld in 2003 as a customer satisfaction / voice of the customer measurement tool Trademark owned by Fred Reichheld, Bain & Company, and Satmetrix Reportedly an indicator of future growth Can be applied to other stakeholders including employees Core methodology: 0 10 score based on one question How likely are you to recommend [NAME] based on your experience? 0 6 = detractors; 7 8 passives ; 9 10 promoters Most applications add additional qualitative questions Why did you give the score you did? What improvements should [NAME] make? Achmea International uses an advanced form of NPS Closed loop methodology

14 NPS closed loop methodology First wave survey Transactional (i.e., after an interaction with an operating company / brand) Periodic (e.g., annual) Seek permission to contact participating customers Call back detractors and potentially passives Seek more information; try to resolve issues / problems Second wave survey

15 NPS closed loop methodology results Anytime (Online brand Greece); Interamerican (Greece); Union (Slovakia); Eureka Sigorta (Turkey) 1st survey 2 nd survey (post call back) n = 766

16 NPS closed loop methodology results Anytime (Online brand Greece); Interamerican (Greece); Union (Slovakia) 1st survey 2 nd survey (post call back) n = 766

17 Customer journey mapping Based on McKinsey s customer decision journey Three key steps: Conduct customer experience research to collect data on touchpoints (e.g., Website, retailer, call centre, post sales service, etc.) quantitative and qualitative (surveys; CRM feedback; complaints data; social media comments; Web site data; etc.) Visualize the customer journey showing positive and negative touchpoints Identify and implement improvements Specialist software applications now available to visualize Can be applied to partner, member, patient, student journeys

18 Customer journey mapping case study Interamerican motor vehicle claims

19 Customer journey mapping case study

20 Customer journey mapping case study Identify improvement actions

21 Result Redesign of process

22 Customer decision journey Two-thirds of the touch points during the active-evaluation phase involve consumer-driven activities such as Internet reviews, word-of-mouth, and recommendations from friends and family.

23 An architecture of listening is key to evaluation Evaluation requires listening To employees, customers, partners, stakeholders, communities Listening involves Recognition and acknowledgement Paying attention Understanding Giving consideration to what is learned Responding in an appropriate way Organizational listening requires an architecture of listening Culture Avoiding the politics of listening Policies Structures and processes Resources Technologies Skills Articulation of findings to decision making

24 LISTEN AND SILENT ARE SPELLED WITH THE SAME LETTERS

25 Listening & Leadership is a virtuous cycle Organizational Listening Listening to Self Interpersonal Listening

26 Transforming culture, transforming people

27 Business performance is strongly correlated with creative and inversely correlated with reactive behaviors Source: Leadership Circle Profile

28

29 Conclusions Evaluation needs more than looking backwards at what you have done Evaluation needs to look beyond media metrics channel and intermediary activity Evaluation needs to show OUTCOMES and ideally IMPACT on audiences / publics Evaluation can provide valuable insights that can inform improvement There are social science research methods and tools available, including advanced methods Also needs culture change, leadership, knowledge and skills

30 Cutting Edge Evaluation The Beginning