Impact Analytics in Practice

Transcription

1 Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy s Karen Hudzinski, Manager Market Research, Macy s Peter Kriss, Lead Research Scientist, Medallia

2 Customer Experience at Macy s Macy s My Customer Engagement Program measures customer experience across main touchpoints Test and Learn is not new to Macy s, but the magnitude and speed of this process have increased. 2

3 How We Analyze Impact Today What? Standard monthly analyses Ad hoc projects When? Monthly reporting cadence YoY primary benchmark Who? Key CX stakeholders: Macy s Stores & macys.com, Furniture & Mattress, Wedding & Gift Registry, Macy s Credit Customer Service, Macy s Backstage, Bloomingdale s Omnichannel & Outlet What works well? Collaboration from stakeholders Data as yardstick of initiative success Engagement from all areas of the organization What are the challenges? Limited sample size for smaller programs Multiple data sources within MCY & BLM 3

4 What About Your Organization? Who are the primary owners of improvement initiatives? a) Frontline leaders b) Mid-level operational leaders c) Senior operational leaders of each function d) A centralized team e) Other What level of knowledge does the CX team usually have about specific initiatives? a) No insight into what they are b) Aware but only at high level c) Knows the details well but not involved in execution d) Directly involved in execution 4

5 What About Your Organization? Compare and contrast your organization with others at your table: Who evaluates whether an initiative is successful? What types of success metrics are most critical? What level of analytical rigor is expected? Choose a notable similarity or difference to share with the room 5

6 Our Impact Analytics Journey Today Innovation at Scale Research Science Partner Program Results of Prototype 6

7 A Collaborative Prototype Process Weekly conversations with Peter to discuss: Results and implications of tests run throughout the week Approaches to dig deeper into noteworthy results Different or new initiatives to test Barriers to tests we wanted to run but could not 7

8 Fine Jewelry & Watch Pilot Impact on NPS Purpose to compare group impacted vs. group not impacted over time to measure what that impact is Impact score difference between the impact and comparison groups, taking into account how much they differed before the pilot +##.## + ##.## ##.## NPS improvement of 24% among jewelry/watch customers attributable to the pilot 8

9 Fine Jewelry & Watch Pilot Impact on Spend + ##.## Average jewelry/watch purchase size saw a double-digit percentage increase attributable to the pilot +##.## ##.## 9

10 Top Doors Initiative Impact on NPS + ##.## NPS improvement of 1.8% among all customers who purchased in Top Doors pilot stores attributed to the changes +#.## #.## 10

11 How Action Impact is Valuable Conceptually valuable Different lens, fresh perspective Convenient platform for advanced analyses Practically valuable Embedded into Medallia portal Visual representation Level of protection 11

12 Implications and Learnings Implications for Customer Engagement team Addition to quarterly, seasonal, or annual recaps Quick pulse on pre/post analyses Implications for the organization Test and learn phase is not just a phase Innovation is a necessity Indicate when customers begin to give credit for improvements 12

13 What We Learned About the Impact Analysis Process Iterative and exploratory Opportunity to go deeper 13

14 What We Learned About the Potential Value Making analysis easier is good It quickly and easily answers a lot of the questions that we probably spend way too much time trying to figure out the answers to. Prototype Client X but the key value is supporting decisions It helps us on our own, without assistance from our global analytics team, get a quick and accurate read on the impact to help us make better decisions about what to do next. Prototype Client Z 14

15 Quantifying the impact of initiatives is a strategic imperative (and it s never been easier) 15

16 Q&A 16