Research for Different Levels of Design

Transcription

1 CX NYC 2018 Research for Different Levels of Design Kelly Price, Analyst

2 Agenda The role of research in CX How to get the insights you need to succeed How to adjust your research plan Discussion and Q&A 2

3 Mastering six competencies leads to great CX 3

4 But getting to great starts with research 4

5 What do we mean when we say research? 5

6 Data-driven Analytics, ML, Big Data N Insights derived from quantitative methods using large samples Thick data Depth of insight Source: Chart adapted from Tricia Wang : 6

7 Data-driven Analytics, ML, Big Data N Insights derived from quantitative methods using large samples Human-insights led Thick data Depth of insight Source: Chart adapted from Tricia Wang : 7

8 Data-driven Analytics, ML, Big Data N Insights derived from quantitative methods using large samples Human-insights led Thick data Deep understanding derived from qualitative research conducted with smaller sample sizes Depth of insight Source: Chart adapted from Tricia Wang : 8

9 Good inventors and designers deeply understand their customer. They spend tremendous energy developing that intuition. They study and understand many anecdotes rather than relying only on the averages you'll find in surveys. - Jeff Bezos 2017 Letter to Amazon Shareholders 2018 FORRESTER. REPRODUCTION PROHIBITED. 9

10 Where does this actually come into play? 10

11 Typically people start with methods Website redesign 11

12 but this is jumping the gun Website redesign What problem am I solving? 12

13 Design happens at different levels of scale Source: Peter Merholz and Kristin Skinner s Org Design for Design Orgs 13

14 Asking the right questions starts with identifying the level of design Example Challenges Identify what opportunities exist to improve our customers lives what should we be designing? Reimagine a customer journey (e.g. onboarding) Design or redesign a website or mobile app Evaluate whether our site navigation matches how users model content Align our website design with our company s new brand guidelines Level Big picture Strategy, Structure, Surface Strategy, Structure, Surface Structure Structure, Surface 14

15 Research basically answers four core questions What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? 15

16 Challenges map to specific questions Big Picture Strategy Structure Surface What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? 16

17 Challenges map to specific questions Big Picture Strategy Structure Surface What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? UNCERTAINTY/PATTERNS/INSIGHTS CLARITY/FOCUS 17

18 Once we know where we re starting we can get on the path to the right insights Design Challenge Core research question Research questions Research goals Methods Informed design decisions 18

19 Example: Research on Retirement Savings Core question related to design challenge: What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? 19

20 Example: Research on Retirement Savings Core question related to design challenge: What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? Research goal: Understand customers retirement goals Identify behaviors that prevent customers from achieving their goals Understand what features increase customers retirement contributions Get insight into whether customers are meeting their goals 20

21 Example: Research on Retirement Savings Core question related to design challenge: What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? Research goal: Understand customers retirement goals Identify behaviors that prevent customers from achieving their goals Understand what features increase customers retirement contributions Get insight into whether customers are meeting their goals Specific question(s): What does retirement mean to our customers? What information are people looking for when they interact with us about retirement? Are people able to find the information they need? How are my experience KPIs tracking over time? 21

22 Example: Research on Retirement Savings Core question related to design challenge: What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? Research goal: Understand customers retirement goals Identify behaviors that prevent customers from achieving their goals Understand what features increase customers retirement contributions Get insight into whether customers are meeting their goals Specific question(s): What does retirement mean to our customers? What information are people looking for when they interact with us about retirement? Are people able to find the information they need? How are my experience KPIs tracking over time? Methods: Discovery Definition Evaluation Measurement 22

23 Example Methods What problems or opportunities exist? What s the best way to solve this problem? Are our solutions correct? What s happening with this experience? Discovery Definition Evaluation Measurement Cultural probe Participatory design/co-creation WoZ testing Analytics data Field observation/ethnography Card sorting/tree testing KPI tracking Diary study Benchmarking Contextual inquiry Cognitive walkthroughs Search log review /FAQ review Experience sampling A/B/ multivariate testing Support call logs Cognitive mapping Existing research synthesis Concept mapping Heuristic evaluation VoC feedback Usability bug review Competitive analysis Usability testing Surveys Interviews Eye-tracking Biometrics (EMG, GSR) 23

24 What happens when we jump ahead? 24

25 25

26 The Result? $1.85B Estimated $1.85 billion losses in same store sales 26

27 Common barriers to ideal research $ Budget Buy-in Time Regulations Access to participants 27

28 Example: Overcoming Challenges Design challenge: Improve the journey of researching and applying for a checking account, to drive sales What the research team wanted to do: Diary study Challenges: 3 weeks to do research, < $3K budget, compliance not comfortable with diary tools What research did instead: Mapped using existing data and insights, focused research on filling gaps Remote retrospective interviews over Web Ex with target customers Total cost: < $

29 Example: Overcoming Challenges Design challenge: Improve the journey of researching and applying for a checking account, to drive sales What the research team wanted to do: Diary study Challenges: 3 weeks to do research, < $3K budget, compliance not comfortable with diary tools What research did instead: Key: Be willing to get scrappy, but do Mapped using existing data and insights, focused research on filling gaps so within the appropriate research swim lane Remote retrospective interviews over Web Ex with target customers Total cost: < $

30 Project Tittle: Created by: Date: Version: STUDY BACKGROUND RESEARCH QUESTIONS METHODS What is the context of this research? Why is it being done? What are the 5-10 questions you want to have answered by the end of this project? Why are these questions important? Do you have any existing research or data that provides insight into any of these questions? Identification What methods will be used? How will each method apply to the research questions you ve identified? RESEARCH GOALS Why are we conducting research? What are the 2-4 goals we have for this project? Why are these goals important/justified? PARTICIPANTS Attributes What attributes will you use to identify and recruit? Why are these attributes important? Planning What methods will you manage internally? Will you work with any outside research providers? Where will the research take place? Will you need to arrange transportation, or book space? How much budget do you have to allocate, and how will that be divided across the different methods employed? SCHEDULE What is the timeline for this project? What milestones will you meet along the way, and at what dates? (e.g. recruitment, running methods, analysis) Recruitment How will you locate and schedule participants? Will you work with an outside provider? Incentives Will you offer incentives for participation? If so, how much? How will the incentives be delivered? SUCCESS METRICS What behavioral indicators will you use to provide evidence that research findings are being used? What metrics will you use to provide evidence that research lead to tangible experience improvements? STAKEHOLDERS Business Leaders Who s buy-in are you going to need to bring this research plan to life? How will you frame the value and importance of this research to get them on board? How will you involve them in the research process to continue to prove the value? Subject Matter Experts Who has subject matter expertise for the questions you are answering? What information might they have to help you form stronger hypotheses? Are there places you can pull them into the research process to help design better questions, find better participants etc.? Facilitators Who will help plan, organize, and execute the research? Who can help prove value creation or help link research to success metrics? Implementers Who are the decision makers who will use the research to make changes to the experience? How will you involve them in the process to increase their buy-in? Source: Forrester Research, Inc. Unauthorized reproduction, citation, or distribution prohibited.

31 Discussion & Q&A 31

32 Thank you Kelly Price

33 Appendix 33

34 How do we get from design challenge to We start with a design answer? challenge or a problem 2018 FORRESTER. REPRODUCTION PROHIBITED. 34

35 How do we get from design challenge to design answer? We have a goal we want to achieve 2018 FORRESTER. REPRODUCTION PROHIBITED. 35

36 How do we get from design challenge to design answer? Solving the problem and achieving our goal requires the right human insights 2018 FORRESTER. REPRODUCTION PROHIBITED. 36

37 Questions guide our research approach What is the right design? Is this design right? Discovery Definition Evaluation Measurement 37