EMSs and Performance Change: What Happens?

Transcription

1 EMSs and Performance Change: What Happens? Daniel (David) Edwards, Jr. Deborah Amaral Richard (Pete) Andrews University of North Carolina at Chapel Hill MSWG Moving to Higher Environmental Performance Using EMS Workshop, Orlando, Florida, June 3, /29/2002 1

2 Overview Project Objectives and Design Environmental Performance Changes Economic Performance Changes Compliance Changes Explaining Differences Among Facilities Performance Summary and Conclusions 5/29/2002 2

3 Project Objectives and Design 5/29/2002 3

4 Objectives of Study What effects does EMS have on environmental performance, economic performance, regulatory compliance? For what kinds of facilities, and what kinds of EMSs? Who are these facilities? large or small, public or private What are their prior experiences? management practices, compliance histories What is the character of the EMS? outside involvement, transparency, ISO certification 5/29/2002 4

5 Project Design Longitudinal Design: facility-level data, three phases Baseline: EMS Design: Performance Updates: 3 Years 1 Year 2 Years Baseline: pre-ems characteristics, performance, management practices. EMS Design: characteristics and process of each facility s EMS. Post-EMS Updates (2x): changes in performance and in evolution of EMS itself. 5/29/2002 ndems@unc.edu 5

6 How Many Facilities? 91 facilities provided baseline data 61 facilities provided EMS design data 34 facilities provided first update data Second update in progress (19 facilities to date, more expected) 5/29/2002 6

7 Sample for this Analysis The 34 facilities that have reported first update data to NDEMS Facilities in various stages of EMS development and implementation Median number of years, last baseline year reported to first update: 2.5 years 25 facilities reported environmental performance data 34 facilities reported economic performance data 32 facilities reported compliance data 9 states represented 20 sectors represented 5/29/2002 ndems@unc.edu 7

8 Highlights of Findings Improved environmental performance 56% improved at least half of their environmental indicators 64% improved at least half of their environmental indicators related to EMS objectives Improved economic performance 85% perceived benefits of EMS adoption and implementation 45% improved at least half of their economic indicators 18% reported positive net quantifiable benefits Improved compliance: 71% eliminated violations 5/29/2002 8

9 Environmental Performance Changes 5/29/2002 9

10 Environmental Performance Indicators (EPIs) NDEMS facilities reported six broad categories of EPIs used: Air Emissions Wastewater Natural Resource Use Production Elements (e.g. byproducts, material use) Waste Generation and Disposal Sustainability Practices 5/29/

11 Evaluating Changes in EPIs (1) Identified best baseline EPI value: if update value is better, performance has improved Identified worst baseline value: if update value is worse, performance has deteriorated If the update value is neither better than the best, nor worse than the worst, performance is similar 5/29/

12 Evaluating Changes in EPIs (2) Two scores for each facility, based on proportion of EPIs with improved performance: One score for all EPIs One score for only EPIs related to facility s EMS Objectives and Targets 5/29/2002 ndems@unc.edu 12

13 EPIs: Results Almost all facilities had some improved indicators More than half of the facilities (56%) -- at least half of all indicators improved Almost 2/3 of the facilities (64%) -- at least half the indicators related to objectives and targets improved fewer indicators in this group deteriorated but this is a much smaller group of indicators 5/29/2002 ndems@unc.edu 13

14 Environmental Performance: ALL EPIs ID Score n Facility /29/2002 ndems@unc.edu Proportion of Indicators in Performance Category Worse Same Better

15 EPI Performance: Objective & Target Related ID Score N Facility /29/2002 ndems@unc.edu Proportion of OBJECTIVE AND TARGET Associated Indicators in PerformanceCategory Worse Same Better

16 EPI Performance: Case Study Facility: Privately held auto supplier (rubber products). < 300 employees. Experience with a number of management systems during the baseline period. No non-compliances during baseline. EMS: Influenced by market pressures. ISO certified. Template for other facilities within the organization. Developed internally by all levels of employees and with input from government and local citizens. Environmental indicators: Eight indicators in four categories (sustainable practices, air releases, waste generation/disposal and natural resource use). Half objectives and targets focused. Performance: 75% of all EPIs improved, + 100% O&T EPIs 5/29/2002 ndems@unc.edu 16

17 EPI Performance: Case Results HAP emissions recycling INDICATOR non-hap emissions hazardous waste generation electrical energy used natural gas use industrial waste generation water/sewer use PERFORMANCE Improved Improved Improved Improved Improved Improved Similar Similar O&T FOCUS Yes Yes Yes Yes No No No No 5/29/

18 Economic Performance Changes 5/29/

19 Economic Performance : Benefits and Costs of EMS Adoption Economic Impact Indicators (EIIs) were evaluated and a performance index developed Quantified costs reported by each facility were subtracted from quantified benefits Benefits that were perceived but not quantified were also evaluated 5/29/2002 ndems@unc.edu 19

20 Economic Impact Indicators (EII) 20 facilities reported baseline values for EIIs that could be compared to Update values. Nearly half (45%) showed improvement in at least half of their economic indicators More than half (55%) exhibited some improved EIIs, but a few reported worse economic performance Most EII improvements represented changes in economic analogues to EPIs (e.g. cost of waste disposal, cost of materials, cost of natural resources, recycling dollars generated) 5/29/2002 ndems@unc.edu 20

21 Economic Performance : EIIs ID Score n Facility Proportion of TOTAL Indicators in Performance Category Worse Same Better 5/29/2002 ndems@unc.edu 21

22 Quantified Benefits and Costs 88% of facilities reported quantifiable costs associated with EMS design and/or implementation. 25% reported quantifiable benefits. EMS Benefits n Median Net EMS design benefits 30 : ($54,000) Net EMS implementation benefits 32 : $ 0 Net benefits 30 : ($40,000) Six NDEMS facilities (18%) reported positive quantified net benefits. The median net benefit reported was $129,000. 5/29/2002 ndems@unc.edu 22

23 Net Quantified Benefits ID Net (3) 34 (13) 14 (19) 5 (21) 6 (22) 18 (24) 3 (30) 12 (33) 30 (35) 10 (45) 28 (63) 31 (77) 8 (81) 29 (86) 13 (105) 27 (109) 22 (117) 24 (129) 26 (170) 32 (180) 15 (232) 25 (260) 4 (273) 7 (628) USD $1, (50) (150) (250) (350) (450) (550) (650) Net Benefits Facility 5/29/2002 ndems@unc.edu 23

24 Quantified Benefits: Case Study Facility: Privately held metal finisher, ~100 employees; some experience with management systems during the baseline period. Several instances of non-compliance during baseline. EMS: Influenced mainly by marketing, but seeking regulatory benefits. Intends ISO certification. Developed with input from government, consultants and by employees at all levels. Design Costs: $10, staff time, consulting fees Implementation Costs: $70,000 staff time, materials, equipment Total Costs: $80,000 5/29/

25 Quantified Benefits: Case Study Design Benefit: $0 none reported. Implementation Benefit: $203,000 reduced raw materials, waste disposal, fines, monitoring, natural resource use Net Benefit: $203,000 - $80,000 = $123,000 How were net benefits achieved? Eliminated Chemical Use new equipment: $8,000 saved/yr. Material Substitution decreased rejects: $128,000 saved/yr. Decrease wastewater added sensors: $4,500 saved/yr. Decreased waste volume added sensors: $12,000 saved/yr. 5/29/2002 ndems@unc.edu 25

26 Benefits Perceived, Not Quantified 29 facilities (85%) also reported perceived but unquantified economic benefits associated with EMS design and implementation Six broad categories of benefits identified (including 27 more specific types) 5/29/

27 Types of Non-quantified Benefits Improved Regulatory Performance Reduced Liability Benefit Increased Management Efficiency Increased Operational Efficiency Improved Supply Chain Relations Improved Community Relations Percent Reporting Benefit (n=29) 93% 76% 59% 55% 21% 14% 5/30/02 27

28 Non-quantified Benefits: Examples Design: Increase in management commitment, and in employee awareness and sense of responsibility; process changes, disposal costs, electrical consumption; maintaining customers Implementation: More involvement by personnel external to the EHS function; decrease in hazardous materials, chemical purchases, waste disposal, power requirements, permit fees, direct labor handling; better utilization of resources for all applications; air permit requirement eliminated; maintained market share 5/29/

29 Regulatory Compliance Changes 5/29/

30 Regulatory Compliance 32 facilities reported information on regulatory compliance during the baseline and update period. 57% had no violations either in the baseline or the update 71% of the 14 facilities that had a violation during the baseline had none during the update 11% of the 18 facilities that had no violations during the baseline received at least one NOV during the update Only 1 facility had a major violation during the baseline and update periods; one additional facility had a major violation during the update. All other violations were classified as minor. 5/29/2002 ndems@unc.edu 30

31 Violation Activity: Baseline-Update 6 2 UPDATE Violations BASELINE No Violations 16 UPDATE No Violations 10 BASELINE Violations 26 5/29/2002 ndems@unc.edu 31

32 Summary : Performance Changes Environmental performance: Almost all improved on some indicators, and more than half improved a majority of their indicators Economic performance: More than half showed improvement in some economic impact indicators; quantified costs > benefits for most but not all; most also noted important unquantified benefits Compliance: Significant reduction in number of facilities experiencing violations Variability: Differences in outcomes among facilities 5/29/2002 ndems@unc.edu 32

33 Explaining Differences Among Facilities Performance 5/29/

34 Explaining Differences: Demographics The following demographic characteristics were associated with improved performance: SIZE Large facilities (>300 employees) higher EPI More than 50 but less than 1,000 employees higher EII OWNERSHIP Publicly traded and government facilities higher EPI Publicly traded and privately held facilities higher EII 5/29/

35 Management System Experience Facilities with some baseline experience with management systems (e.g. TQM, ISO 9000) scored higher on both EPI and EII indexes Prior System Experience Yes n=18 Median EPI Score n=13 Median EII Score No n= n= /29/2002 ndems@unc.edu 35

36 Other Prior EM Experience Facilities that used other EM techniques during the baseline period (e.g. P2 Plans) had higher scores on EPIs related to EMS objectives and targets. P2 Plan Median EPI OT Score Yes No n=13 n= /29/

37 Explaining Differences: Compliance History Of the independent variables examined, a compliance history free of violations, fines or non-compliances was positively associated with the greatest number of variables that measured subsequent performance. Facilities with these outstanding compliance histories: Scored higher on the overall EPI index Scored higher on the Objective- and Target-Related EPIs Had higher net benefits 5/29/2002 ndems@unc.edu 37

38 Explaining Differences: Compliance n= n=8 EPI Score EPI OT Score 0 Compliant Non-Compliant 5/29/2002 ndems@unc.edu 38

39 Explaining Differences: Compliance History cont. These results do NOT mean that EMSs are unlikely to produce significant improvement by more problematic facilities Few facilities with major violations included in study Results show compliance improvement Only first update data, non-compliant facilities may simply take longer to improve 5/29/

40 Explaining Differences: EMS Design EMS design and implementation is a flexible process that allows for variation in process and content. Examples include variations in: ISO certification intentions Third party audits Outside Party involvement Use of consultants in EMS design EMS transparency These variations might also contribute to differing levels of environmental and economic performance. 5/29/2002 ndems@unc.edu 40

41 ISO Certification and EMS Audits 69% of the facilities were certified or intended to certify to ISO % of the facilities audit their EMS, and 72% of those that do audit use third-party auditors No association with performance results 5/29/

42 Outside Party Involvement 85% of the facilities involved external individuals or groups in the design of their EMS 53% used consultants 41% involved government institutions or organizations 12% involved environmental organizations Facilities developing EMSs without using consultants had median EPI-OT scores nearly twice as high as those using consultants Involvement by other outside parties was not associated with performance results 5/29/

43 EMS Transparency Public disclosure of the results of the EMS process may increase public trust and regulator confidence. 75% of the 34 facilities planned to release objectives and targets to interested parties 39% intended to disclose the results of EMS audits 24% intended to disclose both objective and target lists and EMS audit results Facilities intending to disclose O&T lists and audit results had higher median EPI-OT scores than non-disclosers 5/29/

44 Outside Parties, Transparency EMS DESIGN VARIABLE Developed EMS Internally n 9 Median EPI score Median EPI OT score Used Consultants Disclose O&T and Audit Results No Disclosure /29/2002 ndems@unc.edu 44

45 Summary and Conclusions 5/29/

46 Summary Analysis of first update data for 34 NDEMS pilot facilities Overall, improvements in environmental and economic performance and regulatory compliance are evident In general, facilities with the most prior system management experience and the best compliance histories had the best performance scores EMS transparency was associated with higher performance scores, and use of a consultant during EMS design was associated with lower scores 5/29/2002 ndems@unc.edu 46

47 Strongest Findings Improved environmental performance : 92% had some improved indicators Environmental indicators focused on EMS Objectives and Targets were more successful: 64% improved at least half Compliance improved: 71% eliminated violations reported during the baseline period Previous management system experience: these facilities had better environmental performance scores Positive compliance histories: these facilities had better environmental and economic indicator scores 5/29/2002 ndems@unc.edu 47

48 Limitations Small sample (34 facilities in 20 sectors) Short time period (first update data) Cooperating facilities bias Volunteers: pride and confidence in their performance Facilities recruited by states and EPA Varied government incentives for participation (e.g technical and financial assistance, regulatory flexibility) 5/29/

49 Continuing Research Second update data: summer 2002 Final report & databases publicly available: Fall/Winter Follow-on study in progress: Larger sample of facilities Relationships between external incentives, facility motivations, and EMS outcomes and subsequent performance 5/29/

50 Special Thanks We are deeply grateful to the participating facilities and state managers who have provided data for these analyses We are also grateful for the sponsorship of the Office of Wastewater Management and other units of the U.S. Environmental Protection Agency 5/29/