NYC Education Evaluation Report

Transcription

1 NYC Education Evaluation Report February 11, 2016 STORM Quarterly NPDES Permit Requirements Each permittee shall measure the understanding and adoption of the targeted behaviors for at least one target audience in at least one subject area. No later than February 2, 2016, Permittees shall use the resulting measurements to direct education and outreach resources most effectively, as well as to evaluate changes in adoption of the targeted behaviors. Permittees may meet this requirement individually or as a member of a regional group. Audience: Residents Targeted behavior: Yard care techniques protective of water quality Goals for Today Two Program Models Context: GROSS Grant (NS, SS) NEP Grant (SS) North Sound Executive Summary South Sound Executive Summary Evaluation: Key Findings & Recommendations Videos Context NPDES Permit Requirement Phase I S5C.10.c Phase II S5.C.1.c Two Program Models Evaluation (GROSS Grant) North Sound Strategy 0 0 Target Audience: Urban & Non urban single family residents of properties less than 1 acre Focus: Lawn & Yard Strategy: Lecture workshop series Master Gardeners 1

2 South Sound Strategy Target Audience: SE Olympia Neighborhoods, single family residential homeowners of properties less than one acre, and Use fast release or weed & feed fertilizers Maintained own lawn Focus: Lawns only Strategy: Lawn Coaches/Home Visits Demonstration Events Incentives Evaluation Report, 2015 North Sound Program South Sound Program Comparison Recommendations Evaluation Goals 1. Meet NPDES Permit reporting requirements 2. Assess effectiveness of each program (statistically valid manner) Difference in knowledge & understanding of practices Difference in adoption of practices 3. Compare programs effectiveness (qualitatively) 4. Develop recommendations on program elements Demographic factors, motivators for & barriers to change Social Diffusion Improve or streamline programs 5. Fulfill grant requirements North Sound Executive Summary Program Goals Reduce all pollutant runoff from lawns & planting beds Improve yard health & resiliency Evaluation Activities Participant Households (P) 1. Baseline survey, before receiving education 2. Immediate post outreach surveys, directly after 3. Medium term post outreach surveys, conducted 3 12 months after education Non Participating Households (NP) Baseline survey After education (3 12 months) Participant Interviews & Program Staff Surveys (staff, presenters, volunteers) narrative reports 2014 North Sound Partners Arlington Bothell Brier Edmonds Everett Granite Falls Lynnwood Marysville Mill Creek Monroe Mountlake Terrace Mukilteo Snohomish Snohomish Conservation District WSU Snohomish County Master Gardeners Snohomish County (lead) Participant Average 7 series Attendance offered Total across all per 7 Goal: Series: attendees/ people from series households 2

3 Program History : Develop cost effective NYC program able to be integrated countywide to residents in urban and non urban areas. Modelled after King County s 3 part, lecture style workshop series Adapted Seattle s publications Involve local partners: Master Gardeners, Conservation District Target urban and non urban residents Limit marketing to 2 mailers per series Achieved Moderate Change, (Room for Improvement Remains) Key Outcome: Practices that Protect WQ Achieved Little Change After the program, at least 70% of participants were using key practices Post Outreach Use: High (>69% preferred practices; <26% harmful practices) Moderate (40 69% preferred practices; 26 60% harmful practices); Low (<40% preferred practices; >60% harmful practices) Behavior change: H (20 percentage points); M (10 19); L (<10%) Key Outcome: Where Program is Working Effectively Key Outcome: Participant Demographics More than twice as likely to have lived in their homes 3 years or less Slightly younger More likely to have college degree Place more importance on: Using yard as source of fruit and vegetables Having attractive yard than green or weed free lawn 3

4 Key Outcome: Understanding of Natural & Conventional Practices P & NP had similar baseline levels of understanding of key concepts. Some knowledge Substantial room for improvement South Sound Executive Summary Program Goals Reduce nutrient and pesticide runoff. Improve lawn and soil health. Key Outcomes: Social Diffusion 2014 South Sound Partners Most Useful Topics Learned Program History 2009 Olympia commissioned two research studies Olympia piloted its natural lawn care education model. Home visits Demonstration workshops Incentives 4

5 Program Delivery Model Soil test Lawn coach consultations Demonstration workshops: Lawn and soil health Fertilizer & Lime application Aerating, top dressing, and over seeding Mowing and watering Slow release fertilizer & dolomite lime $30 rebate lawn aeration Key Outcomes Behavior Change Key Outcomes Participant Engagement Key Outcomes Behavior Change Strong participant engagement Desire to receive information postprogram (prompts) Less than 5% dropout rate. Key Outcomes Participant Knowledge Key Outcomes Social Diffusion Social diffusion more than tripled the program s reach from a base of 190 households. 5

6 Most Useful Topics Learned Lawn Care cross over topic between programs Key Findings & Recommendations Qualitative Comparison between Programs Programs differed substantially Lawn care is cross over topic Key Findings 1. Both program models: resulted in significant & substantial behavior change in many practices Had varied results in behavior change and participant use of key practices after the programs (most moderate) 2. SS model: Incentive + outdoor demos = higher use post program (short term) Outdoor demo (w/o incentive) = strong, but varied behavior change 3. Cost: SS ($550/household) more than twice NS ($250/household) 4. Jurisdictions would benefit from testing a hybrid program Large lectures Small outdoor demonstrations With or without incentives Two Program Models Detail Findings, Examples 6

7 Examples of Detail Findings Recommendations: Program Model Core program model hybrid Lecture workshops Outdoor demonstrations 1. SS Participants signed an agreement to use these products 2. SS Program gave participant free slow release fertilizer 3. SS Participants received more intensive education including hands on demonstration Taught by Yard Care Professionals proven in: Using NYC practices Presenting practices Recommendations (pgs ) Logistics Guides Use in conjunction with these recommendations Download from STORM website, Library section, 2014 GROSS Grant page Program Model (continued) Personalized assistance at programs (WSU Master Gardeners) Take home materials support care practices Publications core printed materials Resources Seasonal s Recommendations (pgs ) Benefit from Key Regional Programs & Resources Program Model: Add ons Videos Update curriculum as needed Personalized assistance at home Incentives (if shown to produce long term behavior change) 7

8 Recommendations: Strategies for Teaching Specific Practices To meet NPDES permit requirements, programs should ensure they address the following topic areas that directly reduce polluted runoff: Avoiding weed and feed use. Choosing and properly applying slow release fertilizer. Controlling weeds, pests, and diseases using leasttoxic methods. Applying mulch to planting beds. Aerating and top dressing with compost. Storage and use of garden products. Example: Strategies for Fertilizer Choices In lectures, videos, and a webpage, show participants how to identify and choose slow release fertilizer: How to read the guaranteed analysis (NPK numbers). Words that signal the fertilizer contains slow release nitrogen. Benefits and drawbacks between types of slow release nitrogen. Offer a coupon* with a discount on slow release fertilizer redeemable at stores that have agreed to promote this product. *Inform participants how to identify slow release fertilizer and which stores carry the product. Coordinate with stores to regularly stock and promote slowrelease fertilizer. List participating stores and fertilizer information on program webpages. (Review successes and challenges of previous point of purchase programs.) If additional research shows that providing a one time incentive of free slow release fertilizer creates lasting behavior change, seek funding to provide free fertilizer (based on soil test). High High Moderate (costly; takes a lot of coordination) Low (costly) Then address relevant topic areas that reduce the need to use fertilizers and pesticides: Building healthy soil through soil testing, applying lime, and preparing soil with compost. Using Right Plant, Right Place principles and proper planting techniques. Mulch mowing to feed the soil. Using proper watering techniques for plant health and water conservation. Recommendations: Program Evaluation All future programs should include: Baseline survey (7 10 questions) Pledge form Questionnaire at end of workshop or event Post outreach survey (7 10 questions) months after education Recommendations: Strategies for Teaching Specific Practices Outdoor demonstration Indoor display Tools and assistance direct help Information resource Messaging key points to convey Videos visual lessons Incentives rewards or reduce costs Recommended priority level (impact, cost effectiveness) High; Moderate; Low Recommendations: Additional Research Evaluate Long Term Effects of SS Incentives (2016 or 2017) Evaluate Long Term Effects of NS Participants (2016 or 2017) Research the Effectiveness of Incentive Structures (separate effects of the incentives from more intensive education) Compile & Summarize Information on Regional NYC Programs 8

9 Natural Lawn Care Videos Videos show viewers how and why to practice natural lawn care Featuring Ladd Smith and Gina Jarasitis Videos follow SPU s 5 steps to natural lawn care Contacts Susan McCleary City of Olympia smccleary@ci.olympia.wa.us Peggy Campbell Snohomish County peggy.campbell@snoco.org How to Video Series Introduction and overview of NLC Mowing Soil testing Fertilizer and lime Aerating, top dressing with compost, and over seeding Marketing Social Media Facebook Twitter Regional Websites (Thurston Talks) Olympia Utility Insert 9