ESOMAR 28 Questions to Help

Transcription

1 ESOMAR 28 Questions to Help Research Buyers of Online Sample 5 April, 2016

2 Table of Contents Introduction... 2 Company Profile What experience does your company have with providing online samples for market research?... 2 Sample Sources and Recruitment Please describe and explain the type(s) of online sample sources from which you get respondents How are the different sample sources blended together to ensure validity? Are your sample source(s) used solely for market research? How do you source groups that may be hard-to-reach on the internet? If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners?... 4 Sampling and Project Management What steps do you take to achieve a representative sample of the target population? Do you employ a survey router? Please describe the allocation process within your router What measures do you take to guard against, or mitigate, any bias arising from employing a router? Who in your company sets the parameters of the router? What profiling data is held on respondents? How is it done? Please describe your survey invitation process Please describe the incentives that respondents are offered for taking part in your surveys What information about a project do you need in order to give an accurate estimate of feasibility using your own resources? 16. Do you measure respondent satisfaction? Is this information made available to clients? What information do you provide to debrief your client after the project has finished? Data Quality & Validation Who is responsible for data quality checks? How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How often can the same individual take part in a survey within a specified period? Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Do you have a confirmation of respondent identity procedure? Policies and Compliance Please describe the opt-in for market research processes for all your online sample sources Please provide a link to your Privacy Policy Please describe the measures you take to ensure data protection and data security What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents? Are you certified to any specific quality system? If so, which one(s)? Do you conduct online surveys with children and young people? Page 1

3 Introduction Maru/Matchbox is a customer intelligence professional services firm with a vision to grow our clients business through a unique combination of expertise in Insight Community platforms and sector-specific research consulting. Formerly the Research & Consulting division of Vision Critical, Maru/Matchbox consultants have a deep heritage in both strategic insights consulting and disruptive insights technology. As a Vision Critical value-added partner, Maru/Matchbox brings a unique level of expertise in delivering Insight Communities, community management, and advanced research consulting services to its global client base. As part of MARU Group, Maru/Matchbox will expand its capabilities, methodologies and global reach through access to an advanced insight technology ecosystem. Sector-specific consultants tailor community-based solutions to the unique needs of the industries they serve. We leverage technology to simplify the process of collecting actionable insights, leaving more time to focus on the implications and the distribution of intelligence that drives successful business strategy for our clients. Maru/Matchbox runs numerous full serve global insight communities for our clients; as well as our own national panels. These pertain to Maru/Matchbox s, Market Communities ( Maru/Matchbox MC ) panels and the sample they provide for online research projects. It is also important to note that Maru/Matchbox MC panels are not directly comparable to the large sample providers known as Access Panels. Our panels are highly restricted in access. Most typically those who are using these panels for sample are our own researcher teams, existing community panel clients and a few highly trusted partners. This allows for a significant degree of quality control in terms of research, sampling, and panel member experience, which we believe leads to superior quality in all of these respects. Company Profile 1. What experience does your company have with providing online samples for market research? This answer might help you to form an opinion about the relevant experience of the sample provider. How long has the sample provider been providing this service and do they have, for example, a market research, direct marketing or more technological background? Are the samples solely provided for third party research, or does the company also conduct proprietary work using their panels? Maru/Matchbox, formerly the Research & Consulting division of Vision Critical, has been recruiting and managing online panels to facilitate research since These panels are hosted on Vision Critical s industry leading software platform, Sparq, which offers a panel member-centered experience as well as a full suite of research authoring, analysis, panel health and communication tools. These panels support sample for both qualitative and quantitative online studies on behalf of clients. The panels include our client Insight Community panels, tailored to meet the specific sample needs of the client, as well as a number of large, national panels in Canada and the U.S. to support research sample to the Maru/Matchbox s research clients (Maru/Matchbox MC panels). Maru/Matchbox refers to these panels as restricted access panels, as access to the panel is limited to our own researcher teams, existing custom panel clients and highly trusted partners. This allows a greater degree of both Private & Confidential Page 2

4 research quality control and management of the wider research and engagement program that drives the experience of being a panel member. As a result, we see far lower levels of some of the problem behaviors seen on access panels. Further, our network includes access to select external partners globally to obtain online sample from anywhere in the world including Europe, Asia, Africa, Australia, and Latin America, as well as river sources available through our router system. All of our external partners are carefully vetted for quality and are available upon discussion with our clients. Recent comparison tests of our panels versus other leading industry panels shows Maru/Matchbox MC panels to have the lowest levels of straight-lining plus highest levels of completion rates today. Sample Sources and Recruitment 2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples? The description of the type of source a provider uses for delivering an online sample will provide insight into the quality of the sample. Sample is primarily supplied to our research and fieldwork clients from Maru/Matchbox MC panels. These panels consist of participants who are actively recruited through a variety of online (and sometimes offline) methods. Each recruitment campaign is individually tracked and monitored for quality control. On some occasions, we may supplement samples drawn from our own panel with a select group of external partners. These suppliers are carefully vetted for their quality processes. Clients are always told where sample is sourced externally and also if multiple sources are blended to meet the total requirement. These would most often be in cases where the study is multi-country, including some countries in which we do not currently have restricted access panels. Less often, this may occur with projects which require low incidence samples which we cannot completely fulfill through our own panel. Respondents can also be obtained at times from river sources which can encompass web intercept river sources, routed panel sources and social media river sources. 3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources? The variation in data coming from different sources has been well documented. Overlap between different panel providers can be significant in some cases and de-duplication removes this source of error, and frustration for respondents. Private & Confidential Page 3

5 In order to ensure the validity of blended sample, should it be necessary, we would implement our router to scientifically blend multiple sample sources to ensure the sample composition remained matched over time in terms of sample source and quota balancing. The router allows for replication of samples blends implemented for tracking research and multi-wave/multi-cell research so consistency can be maintained across waves. Our choice of provider depends on an assessment made as to the quality of their panel practices (for example, examining their own answers to the ESOMAR 28) and on our experience in working with them. In addition, different sample providers may be more appropriate depending on the project and sample specifications. We work with a third party industry-leading digital finger-printing solution provider which enables us to identify and remove potential duplicate respondents that may be members of both panels and have answered the study already, eliminating this potential source of error. 4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for? Combining respondents from sources set up primarily for different purposes (like direct marketing for example) may cause undesirable survey effects. Yes. Maru/Matchbox s Voice of Market Panels are solely used for market research purposes. 5. How do you source groups that may be hard-to-reach on the internet? Ensuring the inclusion of hard-to-reach groups on the internet (like ethnic minority groups, young people, seniors etc.) may increase population coverage and improve the quality of the sample provided. We reach out to hard-to-reach groups by using both the appropriate mode and the appropriate messaging. In terms of recruitment mode, we try to maximize our reach into the population by using recruitment sources that other panel companies do not. These include online sources such as direct media buys of target lists, website display ads, affiliate networks, retargeting campaigns, co-registration methods, social media sites and recruits from existing Insight Communities. We also develop strategic partnerships with organizations to provide us access to non-commercially-available online and offline media properties. 6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider? Many providers work with third parties. This means that the quality of the sample is also dependent on the quality of sample providers that the buyer did not select. Transparency is essential in this situation. Overlap between different providers can be significant in some cases and de-duplication removes this source of error, and frustration for respondents. Providers who observe process standards like the ISO standards are required to give you this information. While the primary source for sample is from Maru/Matchbox MC panels, Maru/Matchbox also sources sample for some research projects using a select group of external partners, in markets or verticals not supported by Private & Confidential Page 4

6 Maru/Matchbox MC panels. These suppliers are carefully vetted for their quality processes. Clients are always told where sample is sourced externally and also if multiple sources are blended to meet the total requirement. Private & Confidential Page 5

7 Sampling and Project Management 7. What steps do you take to achieve a representative sample of the target population? The sampling processes (i.e., how individuals are selected or allocated from the sample sources) used are the main factor in sample provision. A systematic approach based on market research fundamentals may increase sample quality. The appropriate sampling process may vary from project to project, however this is a broad overview of the process used to create a balanced sample, according to known population statistics, using a technique of interlocked quota sampling combined with response propensity. This process ensures that the sample achieved may be described as representative of the population which it is designed to model, according to those variables by which it is balanced. 1) The population of interest is defined and a balancing matrix is created according to known profiled information 2) The number of completes required per cell is calculated according to the total sample size required, multiplied by the target % for the cell as defined by the target matrix. 3) The total outgo (invitations) required per cell is calculated based on the known response rate for that cell (response propensity). 4) Available panel members are assigned the cell to which they belong, and then a random selection is drawn of qualifying panel members, per cell, up to the number of invitations required for that cell. 5) Selected panel members are deployed an invitation to complete the study. Each link is unique to the panel member therefore only sampled panel members can participate in the study. Depending on the length of time in field, reminders may also be sent. 8. Do you employ a survey router? A survey router is a software system that allocates willing respondents to surveys for which they are likely to qualify. Respondents will have been directed to the router for different reasons, perhaps after not qualifying for another survey in which they had been directly invited to participate, or maybe as a result of a general invitation from the router itself. There is no consensus at present about whether and how the use of a router affects the responses that individuals give to survey questions. Yes. We employ a survey router, powered by Fulcrum, as a complement to Maru/Matchbox MC panels sample. The benefit of incorporating a routing technology to our sampling system is to maximize the likelihood that any panelist invited to a survey will get to complete a survey. This sample optimization tool enhances the efficiency of sampling as well as panelist experience. Private & Confidential Page 6

8 9. If you use a router: Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys? Biases of varying severity may arise from the prioritization in choices of surveys to present to respondents and the method of allocation. Our survey allocation process is based on a random priority algorithm within our routing system. The algorithm considers routing a panelist to a study they are more likely to qualify for and complete. The priority algorithm is carefully studied so as to not introduce additional issues, such as self-selection bias. 10. If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias? If Person A is allocated to Survey X on the basis of some characteristic, then they may not be allowed to also do Survey Y. The sample for Survey Y is potentially biased by the absence of people like Person A. Extensive research has been done on our routing methodology to observe if any response bias existed between targeted and routed groups within the same sample. We observed no significant differences on survey responses in targeted vs. routed sample. In addition, we ensure a minimum of five or more source surveys are used and employ random allocation to mitigate selection bias within the sample. 11. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers? It may be necessary to try to replicate your project in the future with as many of the parameters as possible set to the same values. How difficult or easy will this be? The parameters of the router were determined through research performed under our Chief Research Officer. Those parameters (number of attempts and length of time in router) stay fixed for all surveys and are based on the best practices determined by the research that was done. Private & Confidential Page 7

9 12. What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with? The usefulness to your project of pre-profiled information will depend on the precise question asked and may also depend on when it was asked. If real time profiling is used, what control do you have over what question is actually asked? Comprehensive and regular profiling of panel members is key to the way in which Maru/Matchbox MC panels are managed. Not only does it allow us to pinpoint sub-samples quickly and accurately for our clients, but it reduces the number of occasions in which panel members are invited to participate in a study but disqualified after a number of questions. It is consistent across all of Maru/Matchbox MC panel sources but may differ with external partner s samples due to differences in methods of partners. There are a number of ways in which profiling information is collected and updated on panel members, the main three of which are described below. However, we retain every data point collected on members in every survey, therefore, in reality, any survey question asked to any member could be used as a reference point for sampling, or as background data. 1) Profiling Questionnaire (PQ) Every panel member completes a basic profiling questionnaire when they register to become a member of the panel, the core variables on this are therefore collected for ALL panel members. 2) Updatable Profiles Members can update their profile information themselves via the member portals. This includes updating not only basic login information ( , password), but also key profile variables that may be subject to change. As above, the specific variables may vary by market but would typically include zip/postal code and associated geo-variables, home ownership, marital status, HH composition, presence of children, HH income, etc. 3) Weekly Mosaic Profiling Surveys All members are invited to a panel-wide survey conducted each week called the Mosaic. The primary purpose of these studies is to supplement the profile information held on panel members. 4) Themed Monthly Profile Screeners In addition to the Mosaic, all members are invited to a monthly screener that focuses on a particular topic, such as health, employment or technology. These screeners are each run once to twice a year to maintain the integrity of profiling data, as well as to update those data points that are more likely to change. Private & Confidential Page 8

10 13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of s. The type of proposition (and associated rewards) could influence the type of people who agree to take part in specific projects and can therefore influence sample quality. The level of detail given about the project may also influence response. Members are invited to take part in a survey purely via . The contains a unique link which means that only those members that are invited may participate in a study and may only do so once. A typical survey invitation will provide the following pieces of information: The approximate survey length Any incentive or reward available for completing the survey (and a link to accompanying legal terms, where appropriate) The date on which the study will be closed. Where the study will be open for a particularly short timeframe, this is also noted. In addition to invitations, members who have opted-in to our Facebook application can also answer surveys via Facebook notifications. Again, this invitation is unique to each individual member. Members also have the option to log in to their account, or Member Page, to complete studies that are open or infield. 14. Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics? The reward or incentive system may impact on the reasons why people participate in a specific project and these effects can cause bias to the sample. The Maru/Matchbox MC panels operate on a per complete incentive model, whereby, for the majority of surveys, members receive a small cash reward if they complete the survey. Incentive amounts do not vary based on sample source or member characteristics, but rather according to survey length. The typical range is $0.50-$3 (US & Canada). In addition, each panel operates a schedule of regular prize draws with cash prizes as a supplement to the cash credits system. Draw entries are awarded for a number of actions, including participation in the various profiling surveys and when members attempt to, but are unable to complete a survey due to full quotas or disqualification. 15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources? The size of any panel or source may not necessarily be an accurate indicator that your specific project can be completed within your desired time frame. Private & Confidential Page 9

11 In order to accurately determine the feasibility of a project, we require the following information: a. The sample specs (who do you need to talk to) and quotas that must be reached b. The incidence of this group of people in the general population (can be estimated, if no prior research has been done) c. The number of completes that the project requires d. The length of the study in question e. The duration of time the study will be in field 16. Do you measure respondent satisfaction? Is this information made available to clients? Respondent satisfaction may be an indicator of willingness to take future surveys. Respondent reactions to your survey from self-reported feedback or from an analysis of suspend points might be very valuable to help understand survey results. We take member satisfaction very seriously and measure/monitor this in a number of different ways: 1) Panel Member Satisfaction Study Firstly, we run a bi-annual Panel Member Satisfaction study. This study asks about overall satisfaction with membership of the panel as well as satisfaction with a number of key aspects of membership. In addition to the general findings, we run a regression analysis which tells us which of the drivers of satisfaction have the most impact on overall satisfaction. This provides us with very actionable insight as to which aspects of membership we could improve to have the biggest affect in driving up satisfaction overall. Since this survey is also made available to our Insight Community panel clients as part of their panel management package, we are also able to develop a large bank of data against which individual panels can compare themselves, Maru/Matchbox MC panels included. 2) Survey Satisfaction/Feedback At the end of each study run on Maru/Matchbox MC panels, we ask respondents a brief, standardized set of questions in order to get their feedback on the survey, immediately after completion. This data is made available to the Project Manager and Researcher responsible for the study, and may be passed onto the client, as appropriate. We also track this over time to examine overall trends in questionnaire preferences and what makes for a positive or negative survey experience. 3) Stop Points Reporting For every survey we run, a Stop Points reports is available. This shows the last question answered before an individual exited from the survey (without completing it). This report helps to identify any areas of the study which may have caused members not to want to continue. 4) Panel Member Support Service Our highly responsive Panel Member Support Service provides responses to panel member s within 24 hours. The qualitative feedback gained via this method also helps us to identify and resolves any issues that relate to member satisfaction whether they be survey-specific, related to overall membership or technology focused. Private & Confidential Page 10

12 17. What information do you provide to debrief your client after the project has finished? One should expect a full sample provider debrief report, including gross sample, start rate, participation rate, dropout rate, the invitation/contact text, a description of the field work process, and so on. Sample providers should be able to list the standard reports and metrics that they make available. Reporting on the sample post-field is available on a per project basis, depending on client requirements. Data Quality & Validation 18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviours, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item nonresponse (e.g. Don t know ) or (d) speeding (too rapid survey completion)? Please describe these procedures. The use of such procedures may increase the reliability and validity of the survey data We have implemented an industry leading validation solution which checks on a per survey level for duplicate survey responses through our Relevant ID system, i.e., the same individual taking the same surveys using more than one registered account. We also employ Geo-IP checking to ensure that members are located where they say they are. In addition, we have the ability to implement real-time speeder checks through our use of Fulcrum to replace current post-survey check, which determines whether an individual has completed the survey in an unacceptably short time and therefore was either deliberately clicking-through or not answering thoughtfully. These criteria are used to exclude these responses from the dataset. 19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources? Over solicitation may have an impact on respondent engagement or on self-selection and non-response bias. We retain a complete history of each panel member s participation in surveys which we use to determine response propensities as well as to implement the burden rules which ensure that no single member is invited to too many studies in any one month (no more than 10), or more than one study within too close a time frame (within 3 days of another invitation). 20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods? Frequency of survey participation may increase the risk of undesirable conditioning effects or other potential biases. We retain a complete history of each panel member s participation in surveys which we use to determine response propensities as well as to implement the burden rules which ensure that no single member is invited to too many studies in any one month (no more than 15), or more than one study within too close a time frame (within 2 days Private & Confidential Page 11

13 of another invitation). We also have the ability to apply specified exclusion rules at the survey category level as well, if requested. 21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data? This type of data per respondent including how the total population is defined and how the sample was selected and drawn, may increase the possibilities for analysis of data quality. We have a full participation history on every panel member (indeed, they are able to access this information themselves via the membership portal), as well as the date they joined (panel tenure), the recruitment sources from which they came (since Jan 2009, even down to the individual online ad or they clicked on). Upon request, any participation data for a specific study but only at an aggregate level can be provided to clients. 22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples, what are the procedures there, if any? Confirmation of identity might increase quality by decreasing multiple entries, fraudulent panelists, etc. The following processes are in place during registration: No individual may sign up more than once using the same account. A digital finger-printing solution is used to detect if the same individual is completing the profiling questionnaire more than once (as defined by a certain level of match in terms of their computer characteristics, in combination with other known demographic variables) enabling us to flag and/or prevent these individuals from being selected to participate in surveys. This system also performs Geo-IP validation preventing anyone from joining the panel outside the country. This system is employed at the individual survey level as well. Internal consistency checking e.g., requiring that the zip code matches the state of residence selected earlier in the registration process. We have implemented the CAPTCHA system across the Profiling Questionnaires for our Maru/Matchbox MC panels. CAPTCHA is a type of challenge-response test used in computing to ensure the response is not generated by a computer. It is a contrived acronym for "Completely Automated Public Turing test to tell Computers and Humans Apart. This CAPTCHA system can also be implemented at the survey level as well if requested. Quality questions included in the Profiling Questionnaire designed to capture bot traffic and reviewed weekly to also identify inattentive panelists who are not giving considered responses. Internal consistency checking e.g. requiring that zip code matches to the state of residence selected earlier in the registration process As a post-registration measure, a match 4 process is implemented which looks for panel members that match on a minimum of three or more basic details and determines if they are a multiple registration on the panel. Any additional membership accounts identified are deactivated. Private & Confidential Page 12

14 A tighter definition of what constitutes an active member, which is defined as a member that has participated in at least one survey or updated his/her profile, or registered to join the panel within the last 6 months (the proposed ISO standard suggests 12 months). Review of open ends on the Omni studies that run 2-3 times a week to identify suspicious open ends which are then reported for review and action. Proactively fielding of a trap questionnaire to the entire panel twice a year to flag fraudulent/inattentive members. Any panelists flagged as repeat offenders to failing any survey level validation checks are also flagged in our database and purged after three violations. B2B panelists are identified through our themed screener procedures, but we do not actively validate in any off-line manner the validity of their B2B screenings. In Canada, we have a separate B2B panel operated in partnership with a Loyalty program who has identified through their databases, the business targeting of the panel. Policies and Compliance 23. Please describe the opt-in for market research processes for all your online sample sources. The opt-in process indicates the respondents relationship with the sample source provider. The market generally makes a distinction between single and double opt-in. Double opt-in refers to the process by which a check is made to confirm that the person joining a panel or database wishes to be a member and understands what to expect (in advance of participating in an actual survey for a paying client). Maru/Matchbox s Voice of Market Panels operate on a double opt-in process. Firstly, prospective members are asked to complete a Profiling Questionnaire which collects a range of key demographic, attitudinal and social variables, as well as name, address and login information. The prospective members are also shown and must agree to our Terms and Conditions, as a requirement of membership. Once the profiling questionnaire is completed, the prospective member is sent a confirmation . To become a panel member, they are required to click on the link in this confirmation to double opt-in and confirm their membership. 24. Please provide a link to your Privacy Policy. How is your Privacy Policy provided to your respondents? Not complying with local and international privacy laws might mean the sample provider is operating illegally. An example privacy policy is given in the ESOMAR Guideline for Online Research. Each panel has a privacy policy in place, accessible from the footer of the panel portal pages and each page of our surveys. It is also highlighted to prospective members prior to joining the panel. The privacy policies can be viewed here: Springboard America: Private & Confidential Page 13

15 Angus Reid Forum - English Angus Reid Forum - French Please describe the measures you take to ensure data protection and data security. The sample provider usually stores sensitive and confidential information on panelists and clients in databases. These data need to be properly secured and backed-up, as does any confidential information provided by the client. The sample provider should be able to provide you with the latest date at which their security has been evaluated by a credible third-party. Maru/Matchbox works hard to protect our customer s data, comply with regulations and make it easy for our customers to access information about how we operate. Please refer to the following legal documents: Data Protection Schedule 2016 Data Center & Data Hosting 26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents? There are no foolproof methods for protecting audio, video, still images or concept descriptions in online surveys. In today s social media world, clients should be aware that the combination of technology solutions and respondent confidentiality agreements are speed bumps that mitigate but cannot guarantee that a client s stimuli will not be shared or described in social media. While the Maru/Matchbox MC panels are diligent in reducing the risk of members sharing sensitive materials, it is not completely avoidable. Members are informed when they re being presented with confidential materials and we can have them agree to the terms of the survey (and remind them of the Terms and Conditions under which they joined). This can mitigate the risk of a breach, but we will also inform our clients that even strict protection protocols cannot safeguard all risks all of the time. 27. Are you certified to any specific quality system? If so, which one(s)? A Being certified may require the supplier to perform tasks in a pre-determined manner and document procedures that should be followed. While we are investigating third party options, we maintain a strict internal quality control process, with regular reviews of process and procedures related to our ongoing commitment to being the quality leaders in the online market research space. Private & Confidential Page 14

16 28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with? The ICC/ESOMAR International Code requires special permissions for interviewing children. These are described in the ESOMAR Online Research Guideline. In the USA researchers must adhere to the requirements of the Children s Online Privacy Act (COPPA). Further information on legislation and codes of practice can be found in Section 6 of ESOMAR s Guideline for Online Research. From time-to-time interviews are conducted with children and young people via our panels. The precise process varies according to the appropriate regulations by market. For example, in the US, young people aged between 14 and 17 (under the age of majority) are able to join the panel themselves, directly, without parental permission, as this is legally and ethically permissible. However, for those ages, per market, for which children and young people are only eligible for interview with parental permission, the following process is implemented: Screening & general permission: All panel members are screened for having children, plus the age and gender of their children. Those parent(s) with children are then asked whether they would be willing for us to invite them to participate in research in which the survey would be completed by their child / children. Those that provide this permission form a sub-panel from which we can choose who to invite for individual studies, based on the data we have on their household and children. The child or parent may choose not to participate at any point in the process. They may also request that we do not invite them to this type of research in the future. Further, our panels comply with all regional, national and local laws with respect to privacy, data protection and children, in the US and Canada, including COPPA and EU Safe Harbour. We are also compliant with the ESOMAR International Code on Market and Social Research, the Market Research Society s Code of Conduct, MRIA, and CASRO guidelines. Private & Confidential Page 15