Our Sample Your Research. ESOMAR 28 Questions. ensuring your presence

Transcription

1 ESOMAR 28 Questions ensuring your presence

2 ensuring your presence Company Pro le 1 ESOMAR 28 QUESTIONS TO HELP RESEARCH BUYERS OF ONLINE SAMPLES What experience does your company have with providing online samples for market research? (Context: This answer might help you to form an opinion about the relevant experience of the sample provider. How long has the sample provider been providing this service and do they have for example a market research, direct marketing or more technological background? Are the samples solely provided for third party research, or does the company also conduct proprietary work using their panels?) Our company was established in May, 2011 and since its inception, we are mainly working on building proprietary panels in most of the major English speaking countries. Till now, we have been successful in building a decent double opt in panel size in India, US, UK, New Zealand, Canada, Australia, Singapore, Malaysia, Hong Kong and South Africa and are growing. We started our operations in December, 2011 and till now we have become preferred online sample supplier for most of our clients. In a short span of time IRB has become one of the fastest growing online panel companies from India. Being a new organization, we have gained faith and con dence in our clients and they appreciate us for the quality of our panel data and service standards. We understand that the objective of research can be achieved only if we use high quality data. IRB continues to endeavour delivering high data quality Sample Sources and Recruitment 2 Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples? (Context: The description of the types of sources a provider uses for delivering an online sample will provide insight into the quality of the sample.) The sources of our online sample are described in the below points: Proprietary panel- The primary source of our online sample is the databases of our own double opt-in proprietary panel which is very well managed from our team of experts in order to provide quality data to our clients. We run our proprietary panel at where members come and register with us for market research surveys. We recruit panel members from different sources to keep a good mix and high quality panel members. We bring audiences to our website through search engines, social media sites, web publishers, campaigns, recruitment af liates and ask them to join OpinionBureau survey community and share their opinion in different market research surveys.

3 As a result we get more than unique visitors every month globally and more than members join our panel every month. OSBT (Online Survey Bureau Traf c, also river sample) - We also manage another well-de ned sample source which is known as OSBT (popularly known as river sample). The main source of OSBT sample is a network of media and social af liates who help us in getting online traf c directly to the survey based on demographic criteria. Their traf c is managed through our in-built software which helps us routing the right respondent to the right survey. Respondents coming through OSBT are not our registered panel members. However, we have all the quality measures in place to check the relevance of those respondents. 3 What do you consider to be the primary advantage of your sample over other sample sources in the marketplace? (Context: The variation in data coming from different sources has been well documented. Overlap between different panel providers can be signi cant in some cases and de-duplication removes this source of error, and frustration for respondents.) The primary advantage of our sample over other sample sources can be summed up in two main points. They are Extensive targeting and pro ling- This is one of our major strong holds. We have distributed our respondents under various categories and sub categories based on the answers they provide at the time they get registered with us. It helps us to identify suitable respondents as per the research need of our clients. Quality - Maintaining optimum panel quality is our foremost objective. We have imposed various quality checks at our end to ensure that the best quality data is delivered to our clients. Below is the glimpse of some of our quality check parameters: a. Veri cation of respondents mailing address and IP address. b. IP address veri cation and geo IP check. c. CAPCHA for all respondents to ensure they are human d. Making sure all active members are double opt-in. e. Scheduled cleaning of the panel from fraudulent, speeders and straighliners. More details of our quality check parameters can be seen at our website -

4 4 Are your sample source(s) used solely for market research? If not, what other purposes are they used for? (Context: Combining respondents from sources set up primarily for different purposes (like direct marketing for example) may cause undesirable survey effects.) Yes, our proprietary panel is solely used for Market research purpose only. 5 How do you source groups that may be hard-to-reach on the internet? (Context: Ensuring the inclusion of hard-to-reach groups on the internet (like ethnic minority groups, young people, seniors etc.) may increase population coverage and improve the quality of the sample provided.) We have distributed our panel in 2 main broad categories: Consumer and IT/B2B The IT/B2B audiences have always been a tough to reach audiences. We pay a very special attention in keeping their pro le information up to date in order to maintain a healthy conversion rate. We also make sure that our IT/B2B respondents remain active and highly responsive, plus we also take care that they only get a certain number of surveys to answer per day in order to avoid panellist fatigue. Secondly, we also have an extensive partner network speci cally focussing IT/B2B related surveys and maintain a high panel quality. 6 If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider? (Context: Many providers work with third parties. This means that the quality of the sample is also dependent on the quality of sample providers that the buyer did not select. Transparency is essential in this situation. Overlap between different providers can be signi cant in some cases and de-duplication removes this source of error, and frustration for respondents. Providers who observe process standards like the ISO standards are required to give you this information.) We do not use a lot of outside sample, however, on the occasion when IRB supplement sample we make sure that sample provider is following the quality guidelines set by market research association(s). We do enquiry about panel recruitment methodology, answer to ESOMAR 26 or 28 questions, quality measurements from the partners and a partner is added in our preferred list only they successfully run few projects without any quality complains. We outsource sample only if we have approval from the client. In some cases we bid the project based on sample partner s feasibility which we notify to the client before booking the project.

5 Sampling and Project Management 7 What steps do you take to achieve a representative sample of the target population? (Context: The sampling processes (i.e. how individuals are selected or allocated from the sample sources) used are the main factor in sample provision. A systematic approach based on market research fundamentals may increase sample quality.) Achieve representative sample if the target is our primary goal. We have internally developed a very strong sampling management application which is capable of setting demographic quotas on both completes and clicks level. We always send sample in batches and monitor it closely so that representative can be balanced on overall target. We are capable of pausing and opening the quota which helps us identifying the slow moving quotas and pushing sample to it so that we can maintain representative. We always try to maintain representative as much as possible even it s not a hard requirement by the client. And, if this is a hard requirement by the client, we maintain it strictly. We also try to keep a good mix of all sample sources to attain representative and give high quality data to our client. 8 Do you employ a survey router? (Context: A survey router is a software system that allocates willing respondents to surveys for which they are likely to qualify. Respondents will have been directed to the router for different reasons, perhaps after not qualifying for another survey in which they had been directly invited to participate, or maybe as a result of a general invitation from the router itself. There is no consensus at present about whether and how the use of a router affects the responses that individuals give to survey questions.) Yes, IRB has developed routing technology for panel members and OSBT (Social Media Sample) members. We would like to optimize user experience for our members so we give them a chance to participate in other survey if they disqualify one survey. We also do not want to disguise our members so we always give them opt-in option before routing them from one survey to other survey and we provide the new survey details e.g. length of interview, rewards and generic survey topic. Our routing technology has also been built in such a way that members route to only those surveys where their demographic criteria match with that particular survey. This stops routing member to wrong surveys and improves user experience. Routing technology also helps us optimize our sample and lling up the surveys faster.

6 9 If you use a router: Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys? (Context: Biases of varying severity may arise from the prioritisation in choices of surveys to present to respondents and the method of allocation.) The survey is allocated to the respondents based on their demographic criteria e.g. age, gender, HHI, education, ethnicity/race and region and demographic quota requirement. And survey is allocated to the respondents on random basis and there is prioritization in place in our routing technology. 10 If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias? (Context: If Person A is allocated to Survey X on the basis of some characteristic then they may not be allowed to also do Survey Y. The sample for Survey Y is potentially biased by the absence of people like Person A.) We understand that clients may have some hesitation using routing technology in a particular survey so we have developed a functionality where they can choose if they do not want to use routing technology or set the quota for routed completes in order to compare the results with the methodology. We can also supply the breakup of completes coming from router and other methods. 11 If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers? (Context: It may be necessary to try to replicate your project in the future with as many of the parameters as possible set to the same values. How dif cult or easy will this be?) Routing has been setup at system level by development team and project managers have exibility so activate and de-activate the router at survey and survey quota level. The client has full exibility opt-in and opt-out routing system on project level. They may opt in invite only project and no routing at all or may request our optimized methodology to get maximum feasibility and reach of the target audience. 12 What pro ling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant pro ling data is held, how are low incidence projects dealt with? (Context: The usefulness to your project of pre-pro led information will depend on the precise question asked and may also depend on when it was asked. If real time pro ling is used, what control do you have over what question is actually asked?)

7 We collect following information when member register to our panel: First Name Last Name ID Date of Birth Gender Household Income Highest Level of Education Employment Status State/Province Physical Address Zip Code/Postcode Phone Number (Optional) We also ask our members to keep their information updated and to encourage them to do so we always send them to their pro ling page after completing a survey. We also do an extensive pro ling of our panel members. Based on the information that the respondents provide at the time of registration and other pro le information which they updater after joining, we have distributed our panel based on the below categories: 1) Basic demographic 3)Family 5) Health 7) Leisure and Activity 9) Household appliances 11) Travel 13) Media 2) Education 4) Occupation 6) Finance 8) Food and Beverage 10) Automotive 12) Games We also have a system to use pre-screening in the surveys so if someone has not updated their all pro le information, we can target them though pre-screener and send the targeted sample to client s survey. We also understand client s sampling methodology and maintain their preference. Our project managers communicate with clients on project level and understand their requirement in order to achieve high quality results and maintain consistency.

8 13 Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to speci c surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of s. (Context: The type of proposition (and associated rewards) could in uence the type of people who agree to take part in speci c projects and can therefore in uence sample quality. The level of detail given about the project may also in uence response.) We send survey invitation to our members based on their demographic information and targeting criteria of the survey. We keep subject line standard for al, typically, A New Survey Opportunity from OpinionBureau and basic information about the survey e.g. survey number, reward, length of interview, survey category (e.g. Automotive, Finance, Travel, Technology etc.). We use standard HTM template for all surveys and do not provide any leading information in the survey template. Project Managers do not have right to make any modi cation in the invitation template as it has been setup from backend by panel team. To summarize, invitation contains following information: a. Standard Subject line b. Survey Number c. Survey Category d. Incentive information e. Length of the Interview f. Link of the survey g. Help link h. Link to unsubscribe i.our website address where our members can check the privacy policy and terms and conditions. j.our physical address At times, client wants to include special information in survey to get the attention from panelists particularly in follow-up surveys, diary surveys, recruit and re-contact surveys, etc. but in this case panel team upload a new template for that particular survey and project managers get this approved by the clients before using it. Apart from invitation (and router), panelists can also login into our website and select surveys from current surveys portal and participate in. In survey portal also they see basic information for each survey e.g. survey number, reward, length of interview, survey category and can choose best survey of their interest. This activity helps us to maintain panelist experience and increases panel engagement and retention rate. Our client has full exibility to opt-in completes exclusively from invitation, router portal or combination of any two channels or all.

9 14 Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics? (Context: The reward or incentive system may impact on the reasons why people participate in a speci c project and these effects can cause bias to the sample.) We mainly follow a uniform incentive pattern for most of our consumer jobs. But, we also take the following key factors in consideration while nalizing the incentive plan. a. Length of interview b. Incidence rate c. Tough to reach target groups d. Type of the survey We have different incentive grid for consumer and b2b surveys. Base incentive for B2B surveys is higher than that of consumer surveys. We give multiple of options of reward redemptions to our members to keep them engaged and interested in our panel and surveys. We offer following reward options to our members: a. Cash via Paypal b. Amazon Coupons c. Donation Coupons d. Local Shopping coupons 15 What information about a project do you need in order to give an accurate estimate of feasibility using your own resources? (Context: The size of any panel or source may not necessarily be an accurate indicator that your speci c project can be completed or completed within your desired time frame.) We expect following basic information from the clients in order to provide the best estimation:

10 a. Country and region information b. Demographic target of the audience c. Any particular target group d. Length of interview e. Incidence of the project on genpop or given target group f. Total interviews needed and quota breakup if there is any g. Field time of the survey h. Spread-out the total completes in given eld time or on asap basis I. If there is any daily or weekly quota requirement j. If PII is collected k. If there is any special note on targeting l. If there is de-dupe in the survey from past surveys 16 Do you measure respondent satisfaction? Is this information made available to clients? (Context: Respondent satisfaction may be an indicator of willingness to take future surveys. Respondent reactions to your survey from self-reported feedback or from an analysis of suspend points might be very valuable to help understand survey results.) We do not run any satisfaction survey in the among panel members but do have helpdesk support facility for our members and panelists keep sending their feedback and quires which helps us to improve user experience and satisfaction level of our members. We closely monitor their feedbacks and answer their quires in order to keep them happy. 17 What information do you provide to debrief your client after the project has nished? (Context: One should expect a full sample provider debrief report, including gross sample, start rate, participation rate, drop-out rate, the invitation/contact text, a description of the eld work process, and so on. Sample providers should be able to list the standard reports and metrics that they make available.) We mainly provide the below mentioned information to our clients once a project is nished in eld. a. Actual Incidence rate b. Actual conversion rate c. Median length of interview d. Daily progress report, if requested (however we keep our client updated with this information on a daily basis during the course of the eld) e. Also, we send our clients to ll a form to measure their level of satisfaction with our services.

11 Data Quality and Validation 18 Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviours, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. Don t Know ) or (d) speeding (too rapid survey completion)? Please describe these procedures. (Context: The use of such procedures may increase the reliability and validity of the survey data.) Yes, we do apply validation techniques in order to maintain the quality of our panel. Below are some techniques which we use to detect fraudulent respondents. a. Filtration process at the time of registration. b. Geo IP check c. Time stamp (1/3rd of the length of interview) d. Duplicate respondent detection We also maintain individual level data of our panel members and their survey history. This helps us identifying the panelists and their behaviour in surveys and removing them from panel in a certain time period if they are found fraud or cheater. We consider quali ed completes only if this has been approved by client. So, we expect to receive nal number of completes and any rejections on quality so that we can reward right panelists who completed the survey and can remove those who were rejected on quality checks. We maintain the list of rejected ids on survey level and if we see repeated cases rejections for any member, we set their status as fraud in our database so that they are not contacted in future surveys. 19 How often can the same individual be contacted to take part in a survey within a speci ed period whether they respond to the contact or not? How does this vary across your sample sources? (Context: Over solicitation may have an impact on respondent engagement or on self-selection and nonresponse bias.) We do not send more than 5 surveys to our members in a week and also ensure that same panelists do not receive more than 1 survey invitation in 24 hours.

12 20 How often can the same individual take part in a survey within a speci ed period? How does this vary across your sample sources? How do you manage this within categories and/or time periods? (Context: Frequency of survey participation may increase the risk of undesirable conditioning effects or other potential biases.) Since we maintain survey level member s history, we can exclude the respondents from a particular survey based on his survey history. We exclude the respondents based on the exclusion criteria set by the client. It can be category exclusion or time period exclusion. When project managers set up a survey he has exibility to category participating for given time period or any survey level participation. We can also exclude completes or screen outs, OQs or drop offs or all. 21 Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data? (Context: This type of data per respondent including how the total population is de ned and how the sample was selected and drawn, may increase the possibilities for analysis of data quality.) Yes, we maintain all the information about recent participation history, date of entry, source, etc., of our members and can share this with client if requested except contact information. 22 Do you have a con rmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any? (Context: Con rmation of identity can increase quality by decreasing multiple entries, fraudulent panellists etc.) At the time of registration itself we ensure that same members are not joining our panel multiple times. We have developed our technology in such a way that same respondents can be identi ed using standard quality checks and algorithm of their answers at registration page. We also maintain their answers on survey level and answer behaviour at pre-screening questions and check if there are any variances in their answers. Using this method we are able to identify the duplicate or fraud respondents in order to block them from surveys. We have also deployed all quality checks at survey level so that same respondent cannot enter to same survey irrespective of sources.

13 This means if a respondent has participated through invitation, he will not be routed to the same survey or he will see the same survey in survey portal. He would be even rejected if he is coming from OSBT or any partners. We do supply B2B sample and we have similar quality checks in place for all our panelists since we do not have a separate panel for B2B panelists. We identify them using our pro ling. Policies and Compliance 23 Please describe the opt-in for market research processes for all your online sample sources. (Context: The opt-in process indicates the respondents relationship with the sample source provider. The market generally makes a distinction between single and double opt-in. Double opt-in refers to the process by which a check is made to con rm that the person joining a panel or database wishes to be a member and understands what to expect (in advance of participating in an actual survey for a paying client).) We are a double opt-in panel company. Our members need to double opt in in order to become active members of our panel. They need to ll an online registration form at our website after which they become single opt in. Then, we send them a con rmation with a link. After clicking on that link they get registered as double opt in. Only double opt in panel members receive survey invitations from us. 24 Please provide a link to your Privacy Policy. How is your Privacy Policy provided to your respondents? (Context: Not complying with local and international privacy laws might mean the sample provider is operating illegally. An example privacy policy is given in the ESOMAR Guideline for Online Research.) We have a strict privacy policy in place which is based on the laws of the countries where we have our panel. It states that the participation is voluntary, we gather data only for research purposes, and we don t share any personal identi able information of our panel members with our clients or any third party. We also strictly adhere to the industry standards in the eld of online research. In addition we are a certi ed member of CASRO and comply with CASRO s code of standards and ethic for survey research. Our privacy policy is also certi ed by TRUSTe. Our privacy policy can viewed at below URL.

14 25 Please describe the measures you take to ensure data protection and data security. (Context: The sample provider usually stores sensitive and con dential information on panellists and clients in databases. These data need to be properly secured and backed-up, as does any con dential information provided by the client. The sample provider should be able to provide you with the latest date at which their security has been evaluated by a credible third-party) We keep the data of our panel members in a very much secured server which can only be accessed by our authorized team members. Also, we do not share any personal identi able information of our panel members with our clients or any other third party. Our website and server is also secured by SSL certi cate of VeriSign. SSL certi cate can be checked at below URL: le=fdf/splash.fdf&dn= =en 26 What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents? (Context: There are no fool proof methods for protecting audio, video, still images or concept descriptions in online surveys. In today s social media world, clients should be aware that the combination of technology solutions and respondent con dentiality agreements are speed bumps that mitigate but cannot guarantee that a client s stimuli will not be shared or described in social media.) Whenever a new member comes to join our panel we present our Terms & Conditions and Privacy Policy which includes con dentiality and non-disclosure agreements. Respondents can register to our panel only if they agree to our policies. Members are ensured that their sensitive information will never be disclosed. And if we need to collect their sensitive information in a particular survey, they need to give their consent. We also ensure that their information will never be published. In such surveys we also ask client to add non-disclosure statement in the survey and sign an NDA (non-disclosure agreement) with us. 27 Are you certi ed to any speci c quality system? If so, which one(s)? (Context: Being certi ed may require the supplier to perform tasks in a pre-determined manner and document procedures that should be followed.) We comply with industry standards and also are active member of CASRO.

15 28 Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with? (Context: The ICC/ESOMAR International Code requires special permissions for interviewing children. These are described in the ESOMAR Online Research Guideline. In the USA researchers must adhere to the requirements of the Children s Online Privacy Act (COPPA). Further information on legislation and codes of practice can be found in Section 6 of ESOMAR s Guideline for Online Research.) We allow teens (14-19 years) to participate in the surveys directly. But, if we have to contact less than 14 years old kid, we contact them through their parents on their consent. Sometimes we contact teens also via parents.

16 Contact Us Internet Research Bureau Pvt. Ltd. Plot No. 2A, 2nd Floor, Corner market Malviya Nagar, New delhi , India Alternate Mailing Contact IRB at Ph. No.: Fax No.: URL: