18 Appendix A: Survey Methodology

Transcription

1 2017 JUST Capital Ranking Methodology 18 Appendix A: Survey Methodology Copyright 2017 JUST Capital Foundation, Inc. All rights reserved. Appendix A: Survey Methodology

2 TECHNICAL OVERVIEW OF THE AMERISPEAK PANEL NORC S PROBABILITY-BASED RESEARCH PANEL Updated May 9, 2017 Prepared by J. Michael Dennis, Ph.D. Overview Funded and operated by NORC at the University of Chicago, AmeriSpeak is a probability-based panel designed to be representative of the US household population. Randomly selected US households are sampled with a known, non-zero probability of selection from the NORC National Frame, and then contacted by US mail, telephone interviewers, overnight express mailers, and field interviewers (face to face). AmeriSpeak panelists participate in NORC studies or studies conducted by NORC on behalf of NORC s clients. In 2016, the AmeriSpeak Panel expanded to 20,000 households, with a large oversample of young African- American, Hispanic, and Asian adults (age 18 to 30). AmeriSpeak will expand to 25,000 households in 2017 by creating new panels specific to Latino and teen research. Sample Frame In order to provide a nationally representative sample, AmeriSpeak leverages the NORC National Frame, which provides sample coverage for over 97 percent of the U.S. households. The 2010 National Frame used a two-stage probability sample design to select a representative sample of households in the United States. The first stage the sampling unit is a National Frame Area (NFA), which is either an entire metropolitan area (made up of one or more counties) or a county (some counties were combined so that each NFA contains a population of at least 10,000). The largest NFAs with a population of at least 1,543,728 (0.5 percent of the 2010 Census U.S. population) were selected with certainty; these areas have a high-population density, and are dominated by tracts with street-style addresses. These areas contain 56 percent of the population within 8 percent of the geographic area of the United States. The remaining areas were stratified into areas where street-style addresses predominate, and the remaining areas, which are less likely to have street -style addresses. The latter stratum ( rural areas) comprises 81 percent of the geographic area, but only 14 percent of the population. Within the selected NFAs, the second stage sampling unit is a segment, defined either in terms of Census tracts or block groups, containing at least 300 housing units according to the 2010 Census. A stratified probability sample of 1,514 segments was selected with probability proportional to size. For most of the 1,514 segments, the U.S. Postal Service Delivery Sequence File (DSF) provided over 90 percent coverage of the segments in terms of city-style addresses that are geo-codeable. For the 123 segments where the DSF provided insufficient coverage, we enhanced the DSF address list with in-person listing. The National Frame contains almost 3 million households, including over 80,000 rural households added through the in-person listing. The National Frame involves addresses in almost every state. For the remaining states, AmeriSpeak added some address-based sampling (ABS) addresses in 2016 and 2017 from the USPS DSF to assure AmeriSpeak

3 sample representation for all US States and Washington, DC. As of October 2016, 0.9% of AmeriSpeak Panel recruited adults were sourced from the ABS and 99.1% from the National Frame. Proper weights allow the full use of the combined sample. In 2017, a targeted address-based sample was added to AmeriSpeak recruitment in order to develop a new Latino Panel with adequate representation of Spanish-language-dominant Hispanics. Census tracts with high incidence (at least 30%) of Spanish-dominant Hispanics were targeted for this recruitment. Furthermore, within these Census tracts, households that were flagged as Hispanic based on consumer vendor data (that are typically used for direct-mail marketing) were oversampled. This new Latino Panel will have at least 5,000 Hispanic panelists with approximately 30% of those panelists being Spanish-language dominant. Sample Selection for Panel Recruitment The AmeriSpeak Panel sample consists of nationally representative housing units drawn from the 2010 NORC National Sample Frame and less than 1% from address-based sampling. The 2010 NORC National Sample Frame is stratified based on segment (Census tract or Census block group) characteristics such as age and race/ethnicity composition of the segment, and then, a stratified simple random sample of housing units is selected. Specifically, based on Census tract-level data, segments were classified as having a higher concentration of year old adults or not, and a higher concentration of Hispanics, non-hispanic African Americans, and other. Based on these strata definitions, 6 strata (2 based on age times 3 based on race/ethnicity) were used to oversample housing units in segments higher in young adults and/or Hispanics and non-hispanic African-Americans. This is referred to as the initial sample or first stage of panel recruitment. In the second stage of panel recruitment, initially sampled but nonresponding housing units are subsampled for a nonresponse follow-up (NRFU). At this stage, consumer vendor data are matched to housing units, and housing units that are flagged (based on consumer vendor data) as having a young adult or minority (Hispanic and non-hispanic African American) are oversampled for the NRFU. Overall, approximately one in five initially nonresponding housing units are subsampled for NRFU. However, as mentioned previously, selection of housing units for NRFU is a stratified simple random sample based on consumer vendor data. Due to NRFU, these initially nonresponding housing units have a much higher selection probability compared to the housing units that were recruited during the first stage of panel recruitment. Note that a small fraction of initially nonresponding housing units are not eligible for NRFU due to these housing units being classified as hard refusals or having an appointment for a call back from NORC. In summary, there are two reasons why the sampling design for AmeriSpeak Panel recruitment deviates from Equal Probability of Selection Method (EPSEM) sampling: (a) oversampling of housing units in segments with a higher concentration of young adults and minorities results in the sample selection probabilities being higher for housing units in these segments; and (b) the nonresponse follow-up effort results in initially nonresponding housing units having a much higher selection probability. Furthermore, oversampling associated with NRFU results in higher selection probabilities for initially nonresponding housing units that are flagged (based on consumer vendor data) as having a young adult or minority. AmeriSpeak Panel Recruitment Procedures Recruitment is a two-stage process: initial recruitment using less expensive methods and then non-response follow-up using personal interviewers. For the initial recruitment, sample units are invited to join AmeriSpeak online by visiting the panel website AmeriSpeak.org or by telephone (in-bound/outbound supported). English and Spanish language are supported for both online and telephone recruitment. Study invitations are communicated via an over-sized pre-notification postcard, a USPS recruitment package in a 9 x12 envelope (containing a cover letter, a summary of the privacy policy, FAQs, and a study brochure), two follow-up post cards, and also contact by NORC s telephone research center for sample units matched to a telephone number. 2

4 The second-stage non-response follow-up targets a stratified random sub-sample of the non-responders from the initial recruitment. Stratification is based on consumer vendor data and stratification variables from the initial recruitment stage in order to increase sample representation of young adults, non-hispanic African Americans, and Hispanics. Units sampled for the non-response follow-up are sent by Federal Express a new recruitment package with an enhanced incentive offer. NORC field interviewers then make personal, face-toface visits to the respondents homes to encourage participation. NORC field interviewers administer the recruitment survey in-person using CAPI or else encourage the respondents to register at AmeriSpeak.org or call the toll-free AmeriSpeak telephone number to register. Recruiting Non-Internet and Net Averse Households Under certain conditions, AmeriSpeak gives respondents a choice regarding their preferred mode for future participation in AmeriSpeak surveys. For the recruitment, 79% of the recruited panelists were enrolled in AmeriSpeak to receive online surveys, while 21% of the recruited adults agreed to participate in AmeriSpeak telephone mode surveys. For the 2016 recruitment, respondents provided an option of online or telephone modes include: persons without internet access, persons whose only internet access is via a smartphone, and persons with internet access but unwilling to share an address. A recruited household can consist of both web-mode and phone-mode panelists residing in the same household. Impact of Non-Response Follow-up The non-response follow-up (NRFU) reduces non-response bias significantly by improving the representativeness of the AmeriSpeak panel sample with respect to certain demographic segments, including but not limited to rural and/or lower income households, cell-phone only households, persons age 18 to 34, African Americans, Hispanics, and persons without a high school degree or have only a high school degree (no college). Even though NRFU panelists are more reluctant to complete surveys, the addition of NRFU panelists reduced absolute bias on average 35-40% when compared to the initial stage recruits (among examined surveys). Compared to panelists recruited in the initial stage, panelists recruited via the nonresponse follow-up campaign are more politically conservative, are less knowledgeable about science, report less interest in current events and topics in the news (such as climate change and energy resources), and are less likely to read a print newspaper (more likely to read the news online and use social media). They are also more likely to attend church, be against gun control, and more likely to eat at a fast food restaurant than the initial stage recruits. Accordingly, NRFU panelists make the substantive estimates in any AmeriSpeak study more representative and accurate. AmeriSpeak Panel Recruitment Response Rate and Other Sample Metrics Between October 2014 and October 2016, 20,939 households were recruited to the AmeriSpeak Panel. The AAPOR RR3 (response rate) for the panel recruitment during this time frame is 34.3% (weighted to take into account selection probabilities). 1 The estimated cumulative AAPOR RR3 for client surveys is 10% to 20% (varying according to study parameters and taking into account all sources of non-response including panel recruitment, panel household attrition, and survey participation). 2 NORC documented the AAPOR response rate calculation methodology for recruitment. 3 1 The response rate calculation incorporates the selection probabilities of the samples for the initial recruitment and non-response follow-up stages, as calculated by the US Bureau of the Census for the American Community Survey. 2 A properly calculated AAPOR response rate for panel-based research takes into account all sources of nonresponse at each stage of the panel recruitment, management, and survey administration process. A common misapplication of the term response rate in online panel surveys is represent the survey-specific cooperation rate as the survey response rate. 3 See Response Rate Calculation Methodology for Recruitment of a Two-Phase Probability-Based Panel: The Case of AmeriSpeak authored by Robert Montgomery, J. Michael Dennis, Nada Ganesh. The paper is available at amerispeak.norc.org on the research page. 3

5 Key statistics with respect to the recruited households are as follows: 51% recruited via the nonresponse follow-up recruitment using overnight Federal Express mailers and face-to-face methodology (with NORC field staff visiting households); 21% indicated a preference for the telephone mode of data collection for participating in AmeriSpeak studies; 25% of the recruited households are non-internet; 71% are cellphone only or cell-phone mostly; 18% are African-American and 15% Hispanic; and 33% have household income below $30,000 (compared to ACS benchmark of 29%). Mixed-Mode Data Collection Panelists may participate in two to three AmeriSpeak Panel studies per month via online (computer, tablet, or smartphones) or by CATI phone. CATI phone mode respondents represent a population currently underrepresented in web panels that exclude non-internet households or net averse persons. NORC s telephone interviewers administer the phone mode of survey questionnaires using a data collection system supporting both the phone and web modes of data collection, providing an integrated sample management and data collection platform. For panelists using smartphones for web-mode AmeriSpeak surveys, the NORC survey system renders an optimized presentation of the survey questions for these mobile users. For general population client studies, approximately 20% of the completed interviews are completed by the telephone mode. Panel Management Policies NORC maintains strict rules to limit respondent burden and reduce the risk of panel fatigue. On average, AmeriSpeak panel members typically participate in AmeriSpeak web-based or phone-based studies two to three times a month. Because the risk of panel attrition increases with the fielding of poorly constructed survey questionnaires, the AmeriSpeak team works with NORC clients to create surveys that provide an appropriate user experience for AmeriSpeak panelists. AmeriSpeak will not field surveys that in our professional opinion will result in a poor user experience for our panelists and in panel attrition. ABOUT NORC AT THE UNIVERSITY OF CHICAGO As one of the world s foremost independent research institutions, NORC at the University of Chicago delivers objective data and meaningful analysis to help decision-makers and leading organizations make informed choices and identify new opportunities. Since 1941, NORC has applied sophisticated methods and tools, innovative and cost-effective solutions, and the highest standards of scientific integrity and quality to conduct and advance research on critical issues. Today, NORC expands on this tradition by partnering with government, business, and nonprofit clients to create deep insight across a broad range of topics and to disseminate useful knowledge throughout society. Headquartered in downtown Chicago, NORC works in over 40 countries around the world, with additional offices on the University of Chicago campus, the DC metro area, Atlanta, Boston, and San Francisco. ADDITIONAL RESOURCES To learn more about AmeriSpeak or to share an RFP, please contact AmeriSpeak at AmeriSpeak- BD@norc.org. Information about AmeriSpeak capabilities and research papers are available online at AmeriSpeak.NORC.org. 4

6 YouGov Sampling Methodology Sampling and Sample Matching Sample matching is a methodology for selection of representative samples from non-randomly selected pools of respondents. It is ideally suited for Web access panels, but could also be used for other types of surveys, such as phone surveys. Sample matching starts with an enumeration of the target population. For general population studies, the target population is all adults, and can be enumerated through the use of the decennial Census or a high quality survey, such as the American Community Survey. In other contexts, this is known as the sampling frame, though, unlike conventional sampling, the sample is not drawn from the frame. Traditional sampling, then, selects individuals from the sampling frame at random for participation in the study. This may not be feasible or economical as the contact information, especially addresses, is not available for all individuals in the frame and refusals to participate increase the costs of sampling in this way. Sample selection using the matching methodology is a two-stage process. First, a random sample is drawn from the target population. We call this sample the target sample. Details on how the target sample is drawn are provided below, but the essential idea is that this sample is a true probability sample and thus representative of the frame from which it was drawn. Second, for each member of the target sample, we select one or more matching members from our pool of opt-in respondents. This is called the matched sample. Matching is accomplished using a large set of variables that are available in consumer and voter databases for both the target population and the opt-in panel. The purpose of matching is to find an available respondent who is as similar as possible to the selected member of the target sample. The result is a sample of respondents who have the same measured characteristics as the target sample. Under certain conditions, described below, the matched sample will have similar properties to a true random sample. That is, the matched sample mimics the characteristics of the target sample. It is, as far as we can tell, representative of the target population (because it is similar to the target sample). The Distance Function When choosing the matched sample, it is necessary to find the closest matching respondent in the panel of opt-ins to each member of the target sample. Various types of matching could be employed: exact matching, propensity score matching, and proximity matching. Exact matching is impossible if the set of characteristics used for matching is large and, even for a small set of characteristics, requires a very large panel (to find an exact match). Propensity score matching has the disadvantage of requiring estimation of the propensity score. Either a propensity score needs to be estimated for each individual study, so the procedure is automatic, or a single propensity score must be estimated for all studies. If large numbers of variables are used the estimated propensity scores can become unstable and lead to poor samples. 1

7 YouGov employs the proximity matching method. For each variable used for matching, we define a distance function, d(x,y), which describes how close the values x and y are on a particular attribute. The overall distance between a member of the target sample and a member of the panel is a weighted sum of the individual distance functions on each attribute. The weights can be adjusted for each study based upon which variables are thought to be important for that study, though, for the most part, we have not found the matching procedure to be sensitive to small adjustments of the weights. A large weight, on the other hand, forces the algorithm toward an exact match on that dimension. Theoretical Background for Sample Matching To understand better the sample matching methodology, it may be helpful to think of the target sample as a simple random sample (SRS) from the target population. The SRS yields unbiased estimates because the selection mechanism is unrelated to particular characteristics of the population. The efficiency of the SRS can be improved by using stratified sampling in place of simple random sampling. SRS is generally less efficient than stratified sampling because the size of population subgroups varies in the target sample. Stratified random sampling partitions the population into a set of categories that are believed to be more homogeneous than the overall population, called strata. For example, we might divide the population into race, age, and gender categories. The cross-classification of these three attributes divides the overall population into a set of mutually exclusive and exhaustive groups or strata. Then an SRS is drawn from each category and the combined set of respondents constitutes a stratified sample. If the number of respondents selected in each strata is proportional to their frequency in the target population, then the sample is selfrepresenting and requires no additional weighting. The intuition behind sample matching is analogous to stratified sampling: if respondents who are similar on a large number of characteristics tend to be similar on other items for which we lack data, then substituting one for the other should have little impact upon the sample. This intuition can be made rigorous under certain assumptions. Assumption 1: Ignorability. Panel participation is assumed to be ignorable with respect to the variables measured by survey conditional upon the variables used for matching. What this means is that if we examined panel participants and non-participants who have exactly the same values of the matching variables, then on average there would be no difference between how these sets of respondents answered the survey. This does not imply that panel participants and non-participants are identical, but only that the differences are captured by the variables used for matching. Since the set of data used for matching is quite extensive, this is, in most cases, a plausible assumption. 2

8 Assumption 2: Smoothness. The expected value of the survey items given the variables used for matching is a smooth function. Smoothness is a technical term meaning that the function is continuously differentiable with bounded first derivative. In practice, this means that that the expected value function doesn t have any kinks or jumps. Assumption 3: Common Support. The variables used for matching need to have a distribution that covers the same range of values for panelists and non-panelists. More precisely, the probability distribution of the matching variables must be bounded away from zero for panelists on the range of values (known as the support ) taken by the non-panelists. In practice, this excludes attempts to match on variables for which there are no possible matches within the panel. For instance, it would be impossible to match on computer usage because there are no panelists without some experience using computers. Under Assumptions 1-3, it can be shown that if the panel is sufficiently large, then the matched sample provides consistent estimates for survey measurements. The sampling variances will depend upon how close the matches are if the number of variables used for matching is large. In this study, over 150,000 respondents to YouGov s Internet surveys were used for the pool from which to construct the matches for the final sample. Current Sampling Frame and Target Sample YouGov has constructed a sampling frame of U.S. Citizens from the 2012 American Community Survey, including data on age, race, gender, education, marital status, number of children under 18, family income, employment status, citizenship, state, and metropolitan area. The frame was constructed by stratified sampling from the full 2012 ACS sample with selection within strata by weighted sampling with replacement (using the person weights on the public use file). Data on reported 2012 voter registration and turnout from the November 2012 Current Population Survey was matched to this frame using a weighted Euclidean distance metric. Data on religion, church attendance, born again or evangelical status, interest in politics, party identification and ideology were matched from the 2007 Pew U.S. Religious Landscape Survey. Characteristics of target samples vary based on the requirements of the projects. Typical general population target samples are selected by stratification by age, race, gender, education, and voter registration, and by simple random sampling within strata. At the matching stage, the final set of completed interviews are matched to the target frame, using a weighted Euclidean distances metric. 3

9 Weighting The matched cases are weighted to the sampling frame using propensity scores. The matched cases and the frame are combined and a logistic regression is estimated for inclusion in the frame. The propensity score function may include a number of variables, including age, years of education, gender, race/ethnicity, predicted voter registration, interest in politics, born again status, ideological self-placement and inability to place oneself on an ideological scale, and baseline party identification (i.e., the profiled party identification that was collected before the survey was conducted). The propensity scores are then grouped into deciles of the estimated propensity score in the frame and post-stratified according to these deciles. The final weights may then be post-stratified by gender, race, education, and age. Large weights are trimmed and the final weights are normalized to equal sample size. 4

10 2017 JUST Capital Ranking Methodology 19 Appendix B: Qualitative Report Copyright 2017 JUST Capital Foundation, Inc. All rights reserved. Appendix B: Qualitative Report

11 F I N A L R E P O R T JUST Capital Focus Groups Final Report DATE: AUGUST 16, 2017 PRESENTED TO: NATALIE JACKSON ANGELA AKINYEMI PRESENTED BY: NORC

12 NORC JUST Capital Focus Groups Table of Contents Table of Contents... i I. Introduction... 1 II. Session Flow... 1 III. Findings... 3 Driver Categories... 3 Component Categories... 4 New Component Recommendations... 6 Component Categories... 7 IV. Appendix A Lobby Exercise & Discussion Guides... 8 V. Appendix B Participants FINAL REPORT I

13 NORC JUST Capital Focus Groups I. Introduction On April 10-20, 2017, NORC conducted 12 focus groups in six location in order to understand whether any modifications were needed to the existing Drivers and Components framework. Additionally, the research investigated some specific issues raised in the development of the measurement framework. Focus Group Objectives 1) Develop a set of Drivers and Components to be included in the 2017 Ranking Model. a. Identify the specific or more detailed components that should be included under the broad general buckets drivers. b. Validate existing Components. c. Explore whether there are Components we should add that were not included in the 2016 model. 2) Develop an approach that can be conducted year after year to understand modifications that may need to be made to the existing Driver/Component framework. 3) In line with this approach, ask participants about their concerns about corporate behavior broadly. Compare new ideas to the existing list of Components. Potential differences will be explored, and participants will be asked to react to both the concepts represented by the Components, as well as the language. NORC conducted 12 focus groups in six locations: New York, Chicago, Albuquerque, Wichita, San Francisco, and Atlanta. These groups provided representation of a wide variety of Americans. Each group included between eight and fifteen participants. In each location we hosted two groups, alternating between Drivers and Components. NORC developed a moderator protocol that started broadly, asking participants their thoughts about corporate just behavior. High level thinking was promoted by asking participants to imagine a business as a superhero. Themes, stakeholders, and broad categories were noted. The result of these focus groups was an initial set of Drivers (ranging between 5-8) and Components (we anticipated 35-45) for testing quantitatively. II. Session Flow Lobby Exercise While waiting for the groups to start in the lobby, respondents were given a sheet of paper with an activity to do in order to get them in the right mindset and get them thinking about the topics. Participants were instructed to think of three businesses which they had a favorable impression of and to write down five characteristics or things about those businesses that drive their impression of them. Participants were then instructed to do the same for three business of which they had an unfavorable impression. FINAL REPORT 1

14 NORC JUST Capital Focus Groups Introduction NORC explained that the participants had been invited to participate in a focus group about what it means for a business to do the right thing. Participants were informed that the group discussion would be managed in order to stay on schedule. NORC moderators explained that, at times, discussions may have to be cut short and that some participants may be called on in order to capture feedback from all participants. Participants were also instructed to avoid interrupting one another and to be respectful of viewpoints with which they disagreed. Finally, participants were asked to introduce themselves and list one event they were looking forward to in the next few months. Lobby Exercise Discussion Participants were asked to share the businesses they had listed on the lobby exercise and to explain why those businesses were favorable. Participants were then asked to think about businesses with which they had never conducted business, but had a favorable impression of nevertheless. The same exercise was conducted for businesses of which participants had an unfavorable opinion. Introduction of Just The moderators asked the group to describe what doing the right thing related to in a broad sense. After discussing participant ideas, NORC moderators introduced the concept of just. Just was explained as a business operating justly, with a concept of justice and justness. NORC moderators made it clear that just did NOT mean merely or only. NORC moderators noted that participants found it helpful to think of a just business behaving as how a superhero might, in order to avoid focusing on profitable behavior versus just behavior. Participants were instructed to prioritize what they considered to be just behavior, even if they believed that behavior might impact a businesses profit. Driver Focus Group NORC began the Driver focus group discussion themes, stakeholders, and broad categories of just behaviors. These themes were noted and then removed in order to let participants discuss concepts of justness unstructured. NORC wrote down participant ideas and placed them on a wall. After several rounds of idea generation, participants discussed how best to organize their concepts. Finally, the name and language used to describe each Driver was discussed. Components Focus Group NORC conducted a group focused on Components, and the mapping of Components to Drivers. Consistent with the previous approach, participants were asked about their concerns about corporate just behavior. The existing Components were introduced along with the name and language used to describe each Component. Participants were asked if any changes should be made to the existing Components or if there were any Components which had not been covered. Again, themes, stakeholders, and broad categories were noted. Finally, participants were asked to name the categories into which they had organized the Components. Close out FINAL REPORT 2

15 NORC JUST Capital Focus Groups Participants were thanked and informed that their input would be used in JUST Capital s research into America s Most Just Businesses. III. Findings Driver Categories NEW YORK CITY ATLANTA WICHITA Groups: 1a. Employee/Worker rights 1b. Fairness in pay/compensation (could go with rights #1) 2. Social responsibility/impact - we have domestic and international 3a. Customer rights/relations 3b. Product integrity 4. Ethical business practices 5. Environmental responsibility/conscience Groups: 1. Integrity/Business Ethics 2. Environmentally responsible/conscience 3. Work life balance/human resources/employees/people 4. Company values 5. Politics/Political awareness 6. Responsibility Groups: 1. Financial integrity 2. Environmental conciseness 3. Money/Bottom line 4. Overseas affairs/international 5. Good neighbors 6. Customer service/care 7. Transparent/quality control 8. Employee Relations SAN FRANCISCO CHICAGO ALBUQUERQUE Groups (did not name): 1. Workers 2. Products/pollution/pricing/advertising 3. Leaders/taxes/financials/laws 4. Customers/charitable 5. Resources/environmentally resp. 6. Supply chain 7. Profit Groups: 1. Business practices 2. Customers 3. Worker/Employee Groups: 1. Great for Employees 2. Law abiding 3. Social responsibility 4. Respects customers 5. Quality products 6. Environmentally responsible FINAL REPORT 3

16 NORC JUST Capital Focus Groups Component Categories FINAL REPORT 4

17 NORC JUST Capital Focus Groups FINAL REPORT 5

18 NORC JUST Capital Focus Groups New Component Recommendations FINAL REPORT 6

19 NORC JUST Capital Focus Groups Component Categories FINAL REPORT 7

20 NORC JUST Capital Focus Groups IV. Appendix A Lobby Exercise & Discussion Guides LOBBY EXERCISE: Today we re going to be talking about businesses, what people think about them, what their role in society should be, and what they should and should not do. PART 1: 1. Can you think of three businesses that you have a favorable impression of or that you think generally do the right thing? Put their names at the top of each column below. 2. In the column underneath each name, write down five characteristics or things about them that drives your impression of them. 3. Circle any of the businesses that you either follow in the news, go out of your way to purchase products or services from or, if you invest, choose to invest in (either directly or as part of a portfolio). Business 1: Business 2: Business 3: FLIP OVER FOR PART 2 FINAL REPORT 8

21 NORC JUST Capital Focus Groups Part 2: 1. Can you think of three businesses that you have an unfavorable impression of or that you think generally DO NOT do the right thing? Put their names at the top of each column below. 2. In the column underneath each name, write down five characteristics or things about that business that drives your impression of them. 3. Circle any of the businesses if, because of your impression of it, you AVOID buying its products and services or, if you invest, AVOID investing in its stock. Business 1: Business 2: Business 3: FINAL REPORT 9

22 NORC JUST Capital Focus Groups Just Capital Research Components Discussion Guide Focus Group Objectives: 1) Develop a set of Components to be included in the 2017 Ranking Model a. Identify the specific or more detailed components that should be included under the broad general buckets drivers b. Validate existing Components c. Explore Components not represented in the 2016 model 2) Develop an approach that can be conducted year after year to understand modifications that may need to be made to the existing Driver/Component framework 3) Consistent with the previous approach, ask participants about their concerns about corporate behavior broadly. Compare new ideas to the existing list of Components. Potential differences will be explored, and participants will be asked to react to both the concepts represented by the Components, as well as the language. Lobby Exercise: While waiting for the groups to start in the lobby, respondents will be given a sheet of paper with an activity to do in order to get them in the right mindset and get them thinking about the topics we want to engage on. FRONT: We re going to be talking today about businesses, what people think about them, what their role in society should be, and what they should and should not do. 1. Can you think of three businesses that you have a favorable impression of or that you think generally do the right thing? Put their names at the top of each column below. 2. In the column underneath each name, write down five characteristics or things about them that drives your impression of them. 3. Put a star under any of the businesses that you either follow in the news, go out of your way purchase products or services from or, if you invest, choose to invest in (either directly or as part of a portfolio). 4. Where do you get information you can trust about these businesses? Flip your paper over. BACK: 5. Can you think of three businesses that you have an unfavorable impression of or that you think generally do not do the right thing? Put their names at the top of each column below. 6. In the column underneath each name, write down five characteristics or things about that business that drives your impression of them. 7. Put a star under any of the businesses if, because of your impression of it, you avoid buying its products and services or, if you invest, investing in its stock. 8. Where do you get information you can trust about these businesses? Please take your paper into the group with you. FINAL REPORT 10

23 NORC JUST Capital Focus Groups Introduction ~5 Mins o Have 2 component cards face-down on table in front of each participant s seat (so you do NOT have to take time handing out cards) o Group general overview: o Please turn off your cell phones. o There s a tendency in these groups for some people to speak at length when there s an issue they know about or are particularly passionate about. However, we have a lot to get through in the next 90 minutes so we will be moving along at a fairly quick pace. I may have to cut some of you off in order to move us on to another topic. If that happens, please do not take it personally. We are not looking to embarrass you. It s just that we have limited time and a lot of material to cover. o Additionally, there may be some of you who do not speak up much at all. I may call on you to get your opinion. Again, if I do this, it is not to embarrass you. We value your opinion and would like to hear from everyone. o Please do not interrupt the other person when they are talking. o Lastly, and most importantly, there are no right or wrong answers. If you disagree with one another, please feel free to verbalize that, but be respectful of the other person s feelings. Lobby Exercise Discussion ~10 Mins 1) Conversation: discussion of lobby exercise Thinking about the businesses you are favorable toward o Why did you include these businesses? o What is it about these businesses that leads you to feel good about them? (Probe if needed: What characteristics do they have?) o What things do they do that are good or admirable? (Probe if needed: What specific behaviors?) o What about a business that you may not have ever dealt with, for example a business that deals with other businesses or the government (give an example of a business that operates somewhere else in the country which you ve never dealt with but that you have a favorable impression of) Thinking about the businesses you are unfavorable toward o Why did you include these businesses? o Again what are the key overall characteristics of these businesses that lead you to have a negative opinion? o What are the things they do that lead you to feel unfavorably about them? [Probe if necessary: What specific behaviors?) o What are some of the things you wrote down on your list that we haven t mentioned? [Moderator will specifically ask this question to respondents who have been silent, to bring them into the conversation] o What about a business that you may not have ever dealt with, for example a business that deals with other businesses or the government (give an example of a business that operates somewhere else in the country which you ve never dealt with but that you have an unfavorable impression of) How do you know if they are doing the right thing? o Where do you get information about these businesses? o Why do you trust these sources? o Which sources do you NOT trust? FINAL REPORT 11

24 NORC JUST Capital Focus Groups Introduce Just ~ 10 Mins Does the way businesses operate matter? That is, should we care that businesses reflect the values of the American public? 1) Understand how the public would define just What words would you use to describe businesses that generally do the right thing? What do you think about the word just as a description of these businesses and behaviors? How would you define the word just in this context? What about that word fits or doesn t? IF NEEDED: By just we don t mean only, we mean fair, equitable, balanced that kind of thing. So we use this term JUST to describe business that operate this way as in a concept of Justice or Justness. Tonight, we re interested in understanding what YOU think a JUST business might look like. One thing to keep in mind as we talk, is that there are several groups of stakeholders that we might think about in terms of JUST behavior: Consumers, Employees, Investors, Communities, Environment, Business leadership Is it important for a businesses to be just? Why or why not? Are the businesses you are favorable toward just businesses? Are the businesses you are unfavorable toward unjust businesses? Are the characteristics of just and unjust businesses the same as what you have listed? What is the same or different? Components ~60 Mins For the rest of the conversation we are specifically interested in discussing large businesses, such as those that have over 5,000 employees and are among the thousand or so largest in the U.S. I m going to hand out slips of paper that have components of justness, that is these components might be considered something a business does that makes it qualified to be labeled just. What we d like to do is discuss what these components mean and then group these components in to broad, high level categories. When I call on you, I d like you read the component on your first slip, and then we can discuss, as a group, how to categorize it. After a few we ll start putting our slips on the wall in to the categories we determine, but we can always moving things around as we d like. One last item before we start: there s a tendency in these conversations to think about how a business would behave in order to be PROFITABLE. That is NOT what we are looking to discuss here. In this room, for our remaining time, we can have our cake and eat it too. We want to almost think about a business if it were a superhero fighting for truth and justice. How would THAT type of business behave? What does your showcard say? o What does that mean to you? o Does anyone else have a different idea of what that means? After 15 (or so) components have been placed, start talking about fitting the components into categories: So, let s stop for a second and think about the categories that these components fit into. Once all showcards are on the wall, repeat the component exercise above. Then go category by category: Do we like these items categorized together? Are there any that don t belong? Are there any duplicates? What might we call or name this category? What s missing in this category? Are there any items that should be here that aren t? FINAL REPORT 12

25 NORC JUST Capital Focus Groups Close out MODERATOR: Have respondents fill out any additional profiling Conclusion and thanks Provide background info on JUST Capital s purpose and interest in creating a measurement of the most just businesses in America. FINAL REPORT 13

26 NORC JUST Capital Focus Groups Just Capital Research Discussion Guide - Drivers Focus Group Objectives: 1) Develop a set of high level themes (currently referred to as Drivers) to be included in the Ranking Model 2) Develop an approach that can be conducted year after year to understand modifications that may need to be made to the existing Driver/Component framework 3) Consistent with the previous approach, ask participants about their concerns about corporate behavior broadly. Lobby Exercise: While waiting for the groups to start in the lobby, respondents will be given a sheet of paper with an activity to do in order to get them in the right mindset and get them thinking about the topics we want to engage on. FRONT: We re going to be talking today about businesses, what people think about them, what their role in society should be, and what they should and should not do. 1. Can you think of three businesses that you have a favorable impression of or that you think generally do the right thing? Put their names at the top of each column below. 2. In the column underneath each name, write down five characteristics or things about them that drives your impression of them. 3. Put a star under any of the businesses that you either follow in the news, go out of your way purchase products or services from or, if you invest, choose to invest in (either directly or as part of a portfolio). 4. Where do you get information you can trust about these businesses? Flip your paper over. BACK: 5. Can you think of three businesses that you have an unfavorable impression of or that you think generally do not do the right thing? Put their names at the top of each column below. 6. In the column underneath each name, write down five characteristics or things about that business that drives your impression of them. 7. Put a star under any of the businesses if, because of your impression of it, you avoid buying its products and services or, if you invest, investing in its stock. 8. Where do you get information you can trust about these businesses? Please take your paper into the group with you. Introduction ~5 Mins Group general overview: o Please turn off your cell phones. o There s a tendency in these groups for some people to speak at length when there s an issue they know about or are particularly passionate about. However, we have a lot to get through in the next 90 minutes so we will be moving along at a fairly quick pace. I may have to cut some of you off in order to move us on to another topic. If that happens, please do not take FINAL REPORT 14

27 NORC JUST Capital Focus Groups it personally. We are not looking to embarrass you. It s just that we have limited time and a lot of material to cover. o Additionally, there may be some of you who do not speak up much at all. I may call on you to get your opinion. Again, if I do this, it is not to embarrass you. We value your opinion and would like to hear from everyone. o Please do not interrupt the other person when they are talking. o Lastly, and most importantly, there are no right or wrong answers. If you disagree with one another, please feel free to verbalize that, but be respectful of the other person s feelings. Lobby Exercise Discussion ~10 Mins 1) Conversation: discussion of lobby exercise Thinking about the businesses you are favorable toward o Why did you include these businesses? o What is it about these businesses that leads you to feel good about them? (Probe if needed: What characteristics do they have?) o What things do they do that are good or admirable? (Probe if needed: What specific behaviors?) o What about a business that you may not have ever dealt with, for example a business that deals with other businesses or the government (give an example of a business that operates somewhere else in the country which you ve never dealt with but that you have a favorable impression of) does anyone have an example of a business like this? Thinking about the businesses you are unfavorable toward o Why did you include these businesses? o Again what are the key overall characteristics of these businesses that lead you to have a negative opinion? o What are the things they do that lead you to feel unfavorably about them? [Probe if necessary: What specific behaviors?) o What are some of the things you wrote down on your list that we haven t mentioned? [Moderator will specifically ask this question to respondents who have been silent, to bring them into the conversation] o What about a business that you may not have ever dealt with, for example a business that deals with other businesses or the government (give an example of a business that operates somewhere else in the country which you ve never dealt with but that you have an unfavorable impression of) does anyone have an example of a business like this? How do you know if they are doing the right thing? o Where do you get information about these businesses? o Why do you trust these sources? o Which sources do you NOT trust? Introduce Just ~ 10 Mins 2) Understand how the public would define just Is doing the right thing important? What words would you use to describe businesses that generally do the right thing? What do you think about the word just as a description of these businesses and behaviors? How would FINAL REPORT 15

28 NORC JUST Capital Focus Groups you define the word just in this context? What about that word fits or doesn t? IF NEEDED: By just we don t mean only, we mean fair, equitable, balanced that kind of thing. So we use this term JUST to describe business that operate this way as in a concept of Justice or Justness. Tonight, we re interested in understanding what YOU think a JUST business might look like. Is it important for a businesses to be just? Why or why not? Are the businesses you are favorable toward just businesses? Are the businesses you are unfavorable toward unjust businesses? Drivers ~60 mins In thinking about JUSTness, we can first start to think about who might care if a company is JUST. That is, who are the stakeholders in JUSTness? What groups of people might care? [probe until group mentions all or most: Consumers, Employees, Investors, Communities, Environment, Business leadership] [MODERATOR write on white board or half sheets and post on wall as group mentions stakeholders] [MODERATOR READ] For the rest of the conversation we are specifically interested in discussing large businesses, such as those that have over 5,000 employees and are among the thousand or so largest in the U.S. One item before we start: there s a tendency in these conversations to think about how a business would be behave in order to be PROFITABLE. That is NOT what we are looking to discuss here. In this room, for our remaining time, we can have our cake and eat it too. We want to almost think about a business if it were a superhero fighting for truth and justice. How would THAT type of business behave? 1) Brainstorm: create a list of all the issues that go into making a business just or unjust [as participants name items, write on half sheets and tape to wall] Imagine that you were writing a news story about businesses that were doing the right thing. What kinds of things would those be? What would be the most important thing these business could be doing? Imagine you were writing an advertisement about your JUST company what would be the 2-3 high level JUST things you d want to brag about? [Additional prompts] What makes a business just? What kinds of issues would you take into consideration? What characteristics and behaviors does a just business have? What makes a business unjust? What kinds of issues would you take into consideration? What characteristics and behaviors does an unjust businesses have? How about a member of the surrounding community? How about an employee? How about a customer? How about from an environmental perspective? If you are a stockholder is there anything different you would want to see on this list? Why is that important? Final Prompt: Thinking of all of the groups that businesses serve and impact shareholders, consumers, communities, government, and employees what s missing from our lists? FINAL REPORT 16

29 NORC JUST Capital Focus Groups 2) Group Brainstorm items[move half sheets in to groups] Start putting the ideas into categories (Drivers) We have quite a few issue here. Are there any of these that could be grouped together, in a single larger category? [as group directs, move half sheets in to categories on the wall] For issues within each category that remain uncategorized, ask the following questions: Why do you consider this issue important? Does a business s performance on this issue have any impact on your view of the business as just? Why or why not? 3) Name groups If we wanted to come up with names of these broad categories of JUST behaviors, what would they be? [Once groups have been named] Do these names appropriately describe these groups? Are these groups/group names different enough from each other? If this group name was mentioned in a news article, would you think of these items we grouped together? Closing Question and Conclusion ~5 Mins Final Thinking about everything we have discussed is there anything about JUSTness that we ve missed? What would that be? Conclusion and thanks Provide background info on JUST Capital s purpose and interest in creating a measurement of the most just businesses in America. FINAL REPORT 17

30 NORC JUST Capital Focus Groups V. Appendix B Participants FINAL REPORT 18

31 NORC JUST Capital Focus Groups FINAL REPORT 19

32 NORC JUST Capital Focus Groups FINAL REPORT 20

33 NORC JUST Capital Focus Groups FINAL REPORT 21

34 NORC JUST Capital Focus Groups FINAL REPORT 22