2014 Survey of Environmental Workers Section 1 Methodology

Transcription

1 2014 Survey of Environmental Workers Section 1 Methodology April 1 P2015 a g e

2 Introduction This document reports the results from the 2014 Survey of Environmental Workers. The primary purpose of the survey was to validate the current sector and subsector model. It gathered information regarding the comprehensiveness of the model, and it also captured other areas of environmental work. In addition, the survey included a section about ECO Canada, the degree of awareness amongst respondents, the degree of penetration of its products and services, and the perceived strengths and weaknesses of ECO. The survey was collected online. ECO staff contacted most of the respondents individually and invited them to participate in the survey. A minority of respondents were invited by their employers or colleagues. The survey was in the field between August 6 and October 31, 2014 and collected the opinions of 1,246 environmental workers across Canada in either English or French. French responses were translated and analysed together with the rest of the responses. This report consists of four stand-alone sections: 1. Methodology. 2. ECO Canada awareness. 3. Results on LMI results. 4. Workplan for NOS development. For the sake of clarity, this introduction is repeated at the beginning of each section. In this section This part of the report corresponds to Section 1 Methodology. It covers the following topics: 1. Description of methods. 2. Validity issues 2 P a g e

3 Contents Introduction... 2 In this section... 2 List of Tables... 3 List of Figures... 3 Objectives of Section Topic 1 Description of Methods... 5 Sampling... 5 Collection Streams... 7 Key Findings Topic 2 Validity issues Per collection stream Sample Size per subsector Key Findings Summary and Conclusions Summary Conclusions List of Tables Table 1 Participation per Provincial Environmental Industry Association... 7 Table 2 Participation per province for EP members contacted through mass ing... 8 Table 3 Participation per Province for people recruited telephonically... 9 Table 4 Margin of error (at 95% confidence interval) per subsector List of Figures Figure 1 The participation funnel... 6 Figure 2 Engagement rates for EP members contacted through mass ing P a g e

4 Objectives of Section 1 The objective of this section is to analyse and summarize the methodology, addressing in particular, 1. The methodology that was followed so that it can be replicated in the future. 2. Validity issues of the methodology as they affect the results. 4 P a g e

5 Topic 1 Description of Methods Sampling A total 1,292 partial or total completions were obtained, 1,246 self-identified themselves as environmental workers. They were asked questions about their career stage, some characteristics of their current job, and their awareness and use of ECO Canada products and services. In addition, 896 of them declared being workers with 5 or more years of experience who spent 50% of more of their time using their environmental skills. These 896 workers also evaluated the current sector and subsector model. Figure 1 presents a graphic representation of the participation to the survey. 5 P a g e

6 Figure 1 The participation funnel 1,292 People opened the survey 1,284 People answered at least one question 1,246 Self-identified environmental workers They answered questions about career stage, their current jobs, awareness and use of ECO products and services 1,064 Workers who spend >50% in environmental job duties 896 Environmental workers with 5+ YOE and 50%+ time spent in environment They answered all the questions as above, plus they evaluated the sector/subsector model. 6 P a g e

7 Since this was a targeted survey, the figures above are not reflective of the current Canadian environmental labour force. Collection Streams To obtain the 1,246 responses, three different collection methodologies were implemented: 1. Invitation through Provincial environmental associations 8 associations invited, rendering 35 completions. 2. Invitation to ECO Canada s Environmental Professionals (EP) database 1,341 invitations were sent, receiving 154 (11%) responses. 3. Personal invitation. People invited through telephonic recruitment 7,202 phone calls were made to invite people to the survey. 1,594 accepted to participate, with 1,057 actual responses, rendering a completion rate of 66%. Provincial environmental industry associations The Provincial environmental industry associations were informed of the survey and were asked to inform their associates for them to invite their environmental workers to the survey. This approach contributed with 35 responses with mixed results depending on the Province. Table 1 reports the results at the provincial level. Table 1 Participation per Provincial Environmental Industry Association Association Result Responses British Columbia Environmental Invited. Agreed to participate. Added survey 2 Industry Association information in their newsletter Environmental Services Association Invited. Agreed to participate. Added survey 7 of Alberta information in their September newsletter Saskatchewan Environmental Invited. Never heard back from them. - Industry and Managers Association Manitoba Environmental Industries Invited. Agreed to participate. Sent survey 11 Association invitation in mass and featured survey info in their Information Bulletin Ontario Environmental Industry Invited. Agreed to participate. Featured survey 1 Association info in 2 editions of their newsfeed. Réseaux Environnement Invited. Agreed to participate. Survey info 2 featured in their newsletter Environmental Services Association Invited. Agreed to participate. Sent survey 12 of Nova Scotia invitation in mass . Newfoundland and Labrador Invited. Never heard back from them. - 7 P a g e

8 Association Result Responses Environmental Industry Association Other Provinces Without industry associations or with inactive - associations Total 35 EP population The EP population was asked to answer the survey through a personalized mass method, with a reminder to people who opened and clicked on the link of the first communication. In total, 1,142 people were contacted. Of those, 154 (13%) answered the survey. The engagement rates can be seen in Figure 2 and the provincial distribution and success rate is in Table 2. Table 2 Participation per province for EP members contacted through mass ing Total EP* Responses Response rate** Alberta % British Columbia % Manitoba % New Brunswick % Newfoundland % Northwest Territories % Nova Scotia % Nunavut 6 0 0% Ontario % Saskatchewan % Yukon 5 0 0% Not able to classify Canada % *Only 1,142 members were contacted through mass ing. For the rest, either they have opted out to be contacted through , or were contacted personally through telephonic recruitment. **Approximate figures. The response rate assumes that all EPs received the invitation. However, in some cases the internal database contained inaccurate records preventing some EPs from receiving the invitation. 8 P a g e

9 Figure 2 Engagement rates for EP members contacted through mass ing Telephone recruitment The telephone interviews happened from August 6 to October 17. A total of 7,202 phone calls were made by 3 part-time assistants in hours combined. In total, 1,594 people agreed to answered the survey, and 1,057 (66%) of them actually did. The provincial distribution can be seen in Table 3. Province Phone calls* Table 3 Participation per Province for people recruited telephonically People accepting to participate in the survey Phone calls per agreed person Participated** Participated to people accepting ratio ON % AB % BC % QC % SK % MB % NS % NB % NL % YT % NU % NT % PE % Canada % 9 P a g e

10 *The provincial figures add to 7,184 phone calls made in Canada. The rest (18) were made to the US. Phone calls included multiple calls to attempt to reach respondents, often resulting in a No contact. Few people directly declined to participate. **A total of 1,009 respondents reported the province in which the office they reported to was located. The recruitment process followed these general guidelines: 1. Canadian companies and offices were extracted from the 19 th edition of the Canadian Environmental Resource Guide. The guide contains information of environmentally related government and private agencies, educational and research facilities, foundations, law firms, manufacturers and service providers. 2. For each company its environmental staff was sought. The search was done through online sources such as LinkedIn, membership to environmental organizations or associations, or the companies websites. 3. Determining whether or not a person was an environmental worker was done mainly through ECO Canada s Interactive Assessment tool The potential environmental workers were called and invited to answer the survey. Key Findings A total 1,246 completions were obtained o 1,057 people were invited over the phone, agreed to answer the survey and completed at least one part of it. o Additional 154 cases were obtained after inviting Environmental Professionals (EP) via . o Further, responses from 35 workers invited through the Provincial Environmental Industry Associations were obtained. 896 workers evaluated the core part of the survey: the current sector and subsector model. Those were survey participants with 5 or more years of experience who spent 50% or more of their time using their environmental skills. Topic 2 Validity issues Per collection stream While the response rate for people invited over the phone was high, the survey presented several issues: 1 ECO Canada 2012 Careers in Environmental Professional Services 10 P a g e

11 Most of the environmental industry associations agreed to collaborate with ECO but only 35 responses were obtained through this stream. Moreover, the responses were highly concentrated in Nova Scotia and Manitoba (not reflective of the national distribution of environmental workers). EPs were invited via , with 141 (out of 1,341) responses. The response rate, 11%, was lower than in previous similar surveys (15%), a result that might be attributed to fatigue from EPs regarding ECO communications. The response rate per province also presented variations, with NL and SK above 20% and AB and BC below 9%. The telephone recruitment followed some loose quotas to maintain the % of responses per province as close as possible to the actual distribution of environmental employees (as defined by the Profile of Canadian Environmental Employment). To keep that distribution different time allocations were needed. While in BC, ON or QC more than 5 phone calls were needed to have a person agree to answer the survey, in AB fewer than 3 phone calls were needed. This methodology might have introduced a bias due to different ease of contacting people. Having different collection streams may have affected the global results regarding the awareness and level of use of ECO products and services. However, there is no reason to believe that the different collection streams might have affected the evaluation of sectors and subsectors. Sample Size per subsector Survey respondents selected the subsector they felt was the best fit to their current job. Acknowledging that this strategy might produce imbalances, several strategies were implemented to obtain an equilibrated sample size per subsector. The implemented strategies to obtain an equilibrated sample size were: To establish a minimum quota of 40 respondents per subsector. To source participants whose job titles were directly associated with a particular subsector (according to previous LMI research). To source participants whose job titles were not associated with overrepresented subsectors. Nonetheless, some subsectors obtained a larger-than-expected sample size, whereas others were underrepresented. In particular, the results obtained for Energy, Research & Development, and Environmental Health and Safety are not as solid as originally intended. The original objective was to obtain a sample size that was big enough for the average of the evaluation of the characterizations of the sectors and subsectors to have small confidence intervals (calculated assuming a simple random survey). Margins of error over 0.5 can potentially make confidence intervals to lie in three different whole numbers. Therefore, in the cases of Policy & Legislation, Air Quality, Waste Management, Environmental Health and Safety, Research & Development, and Energy the results are 11 P a g e

12 not as solid as originally intended (Table 4 contains the complete list of margins of error). In particular, the results for the latter 3 have to be analysed with extra precaution. Table 4 Margin of error (at 95% confidence interval) per subsector Subsector Std dev n Margin of error (95% confidence interval)* Energy Research & Development Env Health and Safety Waste Management Air Quality Policy & Legislation Sustainability Education & Training Communications & Public Awareness Natural Resource Management Fisheries & Wildlife Water Quality Site Assessment and Reclamation Environmental Management *Based on average rating and a 10 point scale. Key Findings Of the three collection streams, the telephone recruitment was successful in engaging a large number of participants in the survey. Contacting the provincial environmental associations and cold ing EPs were methods that produced lower productivity levels. The survey allowed participants to select the environmental subsector they felt was the best fit to their current job. This created different sample sizes for different subsectors. Despite having strategies to equilibrate the sample size per subsector, the sample sizes for Energy, Research & Development, and Environmental Health and Safety were smaller than intended. Results on characterizations of these subsectors may be tentative and will require detailed analysis. Summary and Conclusions 12 P a g e

13 Summary A total of 1,246 responses were obtained through three collection streams: Contacting the provincial environmental associations They contacted employers for them to contact environmental workers. This was the least successful collection stream, with 35 completions. Contacting Environmental Professionals (EPs) They were contacted via with a link to the survey. 11% of the membership responded, generating 154 completions. Contacting Environmental workers over the phone They were contacted after sourcing their information through directories and LinkedIn. Through this stream 1,057 completions were obtained. Not all of the participants evaluated the current sector and subsector model. The current sector and subsector model was presented only to workers with 5 or more years of environmental experience who spent 50% or more of their time using their environmental skills. Workers that were presented with the sector and subsector model selected the environmental subsector they felt was the best fit to their current job. This strategy created different sample sizes for different subsectors. Despite having strategies to equilibrate the sample size per subsector, the sample sizes for Energy, Research & Development, and Environmental Health and Safety were smaller than intended. Conclusions The survey collected information from a large number of environmental workers. Even though three different collection streams were implemented, the survey had to be in the field for three months to have a robust sample size. In the end the sample size was large enough to produce solid general results. The sample size is also robust enough to produce results at a regional level. Originally, at least 40 completes were planned per environmental subsector. A lot of time and resources were spent in this target but for some subsectors the effort was unsuccessful. Further research is necessary to find out why. Also, the results corresponding to those subsectors might be affected by the presence of outliers. 13 P a g e