Welcome. Results-Based Accountability Training And Grant Process Overview

Transcription

1 1 Welcome Results-Based Accountability Training And Grant Process Overview

2 2 Today s Agenda Hello! Nice to meet you! New role here to serve you, and our internal United Way teams Results-Based training Collaborative lunch break Putting it all together creating outcomes and performance measurements for your collective impact work group Wrap up/next steps

3 Why Results-Based 3 Accountability Training Purposeful data gathering matters More transparency, more information Informs donors and funders of community impact Helps everyone move the needle/turn the curve Opportunity to practice new skills TODAY Dan Duncan, Clear Impact

4 Dan Duncan Senior Consultant, Clear Impact 4 Faculty member, Asset-Based Community Development Institute About In his current role, Dan brings a wealth of Results-Based Accountability (RBA), Asset-Based Community Development (ABCD), and Collective Impact experience to Clear Impact clients across the country. He uses his experience to help clients and communities achieve even greater community-level Outcomes. Education dan@clearimpact.com After receiving a Masters of Social Work (MSW) from Arizona State University, Dan started his professional career as the Founder and Executive Director of the Community Food Bank and the Executive Director of Information and Referral Services, both in Tucson, Arizona. Prior to joining Clear Impact he spent 30 years a United Way leader.

5 5 What is Results-Based Accountability?

6 Assumptions for Creating Real Community Level Change 6 Understand how complex the problems and lives of those in our communities are It takes a wide variety of strategies and activities to achieve community impact. It is not just about better programs. We must change policies, institutions and structures and most importantly engage the community. Communities have an abundance of resources. The issue is that they have not been identified and engaged

7 Simple, Complicated and Complex Problems 7 Source: Getting to Maybe: How the World Is Changed Frances Westley, Brenda Zimmerman, Michael Patton

8 Assumptions for Creating Real Community Level Change 8 To achieve real impact requires the community and its residents to be involved as producers and coproducers Organizations do not collaborate People do based on common purpose, relationships and trust Should not be rushed It takes time to build trust and relationships We need to conduct our work through a racial equity lens People, places and results matter

9 Values of RBA 9 Simple Common Sense Plain Language Minimal Paper Useful Disciplined method of thinking and taking action to help organizations get from talk to action quickly

10 Where Did RBA Come From? A Brief History 10 From Outcomes to Budgets Published Trying Hard Is Not Good Enough Published First International RBA Conference First RBA Conference Clear Impact Founded

11 Where Has RBA/OBA Been Used? 11 Worldwide 13 Countries United States 50 States Organizations National Governments, Regional Governments, Local Governments, Foundations, Non-Profits, NGOs, and More. Active RBA/OBA Efforts Not Currently Used

12 Results-Based Accountability In a Nutshell Kinds of Accountability Population and Performance Kinds of Performance Measures How Much, How Well, Better Off Jan Feb Mar April May Jun 5 Core Questions to Turn the Curve Baseline, Story Behind the Curve, Partners, What Works, Action Plan Used by communities to improve quality of life and by organizations to improve program performance 5 core questions based on the 7 population and performance questions developed by Mark Friedman.

13 2 Kinds of Accountability 13 Whole Population Population Accountability The well-being of Whole Populations Communities, Cities, Counties, States, Nations Client Population Performance Accountability The well-being of Client Populations Programs, Agencies, Service Systems

14 From Ends to Means 14 Whole Population Ends Results and Indicators Means Client Population Programs and Performance Measures Customer Result = Ends Service Delivery = Means

15 3 - Performance Measures 15 How much did we do? Who are our customers and what services do we provide for them? How well did we do it? How well do we provide those services? Is anyone better off? What is the desired impact of those services on our customers ( customer results )?

16 5 - Turn the Curve Thinking Questions How are we doing? 1 5. What is our action plan to turn the curve? 2. What is the story behind the curve? What works to turn the curve? Turn the Curve? 3. Who are the partners who have a role to play in turning the curve? 4 3

17 Why Turn the Curve Thinking? TALK Action!

18 18 RBA in Acton Gas Company Reducing Truck Accidents through RBA

19 19

20 20

21 21

22 Gas Company Reducing Truck Accidents # Quantity How much did we do? Quality How well did we do it? # of Trucks Effect Effort # of drivers # of trainers # of training sessions # of cones # %/# # % #/% of drivers trained #/% of trucks with cones #/% of trucks serviced # truck accidents % of truck accidents Is anyone better off?

23 23 Common Language Language Discipline

24 The Language Trap Too many terms. Too few definitions. Too little discipline 24 Target Benchmark Outcome Indicator Modifiers Measurable Targeted Urgent Incremental Core Performance Priority Strategic Qualitative Systemic Result Goal Measure Objective Term: Measurable Urgent Systemic Indicators Lewis Carroll Center for Language Disorders

25 RBA Definitions 25 Population Accountability R Result A condition of well-being for children, adults, families or communities. Children Born Healthy Children Ready for School Safe Communities Clean Environment Prosperous Economy Indicator A measure which helps quantify the achievement of a result. Rate of low-birth weight babies Percent ready at K entry Crime rate Air quality index Unemployment rate Performance Accountability Program A program, agency, strategy, solution, or service system Department of Public Health Foundation Nonprofit / Community-based Organization Promise Neighborhood Partner Early Childhood Education Service System Performance Measure A measure of how well a program, agency or service system is working. Three Types 1. How much did we do? 2. How well did we do it? 3. Is anyone better off?

26 Common Language - What Are These? 26 Result, Indicator, Program, or Performance Measure? Result Indicator Perf. Measure Result Safe Community Crime Rate Average Police Dept. Response Time People Have Living Wage Jobs and Income 5 Indicator Adult Literacy Rate Program Indicator Perf. Measure Social Services Department % of People with Living Wage Jobs and Income % of Participants in Job Training Who Get Living Wage Jobs

27 27 Population Accountability Results & Indicators

28 Population Accountability 28 Whole Population Population Accountability The well-being of Whole Populations Communities, Cities, Counties, States, Nations Example Conditions of Well Being: Client Population Residents with Good Jobs Children Ready for School A Safe and Clean Neighborhood

29 Whole Populations 29 Think about your neighborhood, town, city, county or state and complete the following sentences (no more than two words) We want children to be We want families that are We want communities that are

30 Results for Children, Families, and Communities A working list 30 Healthy Children and Adults Stable Families Children Ready for School Prosperous Economy Safe and Supportive Communities Clean and Sustainable Environment Children Succeed in School Inclusive and Diverse Communities

31 Developing Results a population (or subpopulation) + in a geographic area + a condition of well being Complete the sentence: We want geographic area] who are [a population in a Use simple, plain language. Avoid referencing data or improvement. Avoid referencing services. Think of services as a means to an end.

32 32 Exercise: Visioning the Future Creating your North Star How would our community different years from now if you are successful?

33 I Indicator A measure that helps quantify the achievement of a result* Result: All children in our community succeed in school Indicators: *Can be Approximate High School graduation rate % of students performing at grade level or above Overall attendance rate

34 Sources of Indicator Data 34 Populations Level Indicators are macro level data collected from community level data sources: Census State Department of Education Local School Districts Providence Health Department City Health Department Surveys Other Sources? Indicators are not collected from agency/program /client data

35 Sample Indicators Under age 5 mortality per 1,000 live births City s unemployment rate % of school aged population enrolled in schools Transportation fatalities per 100,000 population % of city population living in poverty

36 United Way Larimer Results & Indicators 36 Community Results Indicators Goal Area 1: School Readiness % of local children who enter school ready to learn Goal Area 2: Kids on Track Youth Success School attendance rate % of students reading at grade-level by the end of third grade Goal Area 3: Transitions to Independence # of students graduating from high school Goal Area 4: Working Families Thrive # of local people increasing their income

37 37 Performance Accountability Strategy & Program Level

38 RBA Levels 38 Population Level - Community Level Outcomes Turn the Curve to identify system level strategies Results Population Level Indicators Strategy level Backbone Collective Impact Turn the Curve to implement system level strategies Strategies System Performance Measures for the implementation of strategies Aggregated common program Performance Measures Activity Level Program work Turn the Curve to implement effective activities/programs Activities/Programs Program/Activity Performance Measures ProSPER

39 Performance Accountability Whole Population Performance Accountability Client Population The well-being of Client Populations Programs, Agencies, Service Systems How much did we do? How well did we do it? Is anyone better off?

40 All performance measures that have ever existed for any program in the history of the universe involve answering two sets of interlocking questions Mark Friedman

41 Performance Measures Quantity Quality How much did we do? How well did we do it? # %

42 Performance Measures Effect Effort How hard did we try? Is anyone better off?

43 Performance Measurement Matrix Quantity How much did we do? Quality How well did we do it? # of Customers Served Customer Satisfaction Effect Effort # Activities How productive? # Skills / Knowledge # Attitude / Opinion # Behavior # Circumstance/Condition # %/# # % Is anyone better off? Retention Rates Following Protocols Are we doing things right? % Skills / Knowledge % Attitude / Opinion % Behavior % Circumstance/Condition Are we doing the right things?

44 Education Quantity Quality How much did we do? How well did we do it? Effect Effort Number of students Number of high school graduates # % # % Student-teacher ratio Percent of high school graduates Is anyone better off?

45 Fire Department Quantity How much did we do? Quality How well did we do it? Effect Effort Number of responses # of fires kept to room of origin # %/# # % Response Time % of fires kept to room of origin Is anyone better off?

46 Not All Performance Measures Are Created Equal Quantity Quality How much did we do? How well did we do it? Effect Effort Least Important # % # % Also Very Important Most Important Is anyone better off?

47 The Matter of Control Quantity Quality How much did we do? How well did we do it? Effect Effort Most Control # % # % Is anyone better off? Least Control Partnerships

48 Hayward Promise Neighborhood's Sample 48 Quantity Quality How much did we do? How well did we do it? Effort # of patients treated % of patients treated in less than 1 hour # of students Student-teacher ratio Number of calls Response Time Is anyone better off? Effect # children fully immunized % children fully immunized # students that graduate on time % students that graduate on time Number of fire related deaths Rate of fire deaths per 100,000

49 Hayward Promise Neighborhood's Sample 49 Quantity Quality How much were we able do? How well were we able to do it? Effort # Clients/customers served % Common measures workload ratio, staff turnover rate, % staff fully trained # Activities (by type of activity) % Activity-specific measures % timely intakes, % accreditation standards met How are people better off? Effect #,% Skills / Knowledge (e.g. cognitive, social, physical) #,% Attitude (e.g. toward language, parenting) #,% Circumstances (e.g. child care, transportation) #,% Behavior (e.g. reading to child at home)

50 The Matter of Use The first purpose of performance measurement is to improve performance Avoid the performance measurement equals punishment trap Create a healthy organizational environment. Start small. Build bottom-up and top-down simultaneously.

51 Setting Performance Measure Targets 51 For continuous improvement it is important to set targets for each performance measure and apply the story behind the curve thinking to the difference between the target and the actual performance

52 52 Turn the Curve: Using Data for Continuous Improvement

53 Turn the Curve Thinking How are we doing? 1 5. What is our action plan to turn the curve? 2. What is the story behind the curve? What works to turn the curve? Turn the Curve? 3. Who are the partners who have a role to play in turning the curve? 4 3

54 The Matter of Baselines 54 OK? Turning the Curve History Forecast Baselines have two parts: history and forecast

55 Story Behind the Curve 55 Identify Root Causes Disaggregated Data Demographic Factors Cyclical Factors Force Field Analysis Consider the Restricting and Contributing Factors Bullet Point Prioritize the top 3-5

56 Partners Who Has a Role to Play in Improving Progress? 56 Funders Government Entities Community Partners Businesses Education Neighbors and Families

57 What Works What would work to Turn the Curve? 57 Evidence-based What can we apply in our community? Promising Practices Our best guess about what will work here in our community Off the Wall Suggest off the wall and outrageous ideas as well as researched best practices. Low Cost/No Cost What can we start without a grant? What is the role of clients and community residents? Multi-Faceted Does the proposed solution address multiple root causes or just one?

58 Action Plan What do we propose to do to Turn the Curve? 58 S M T W T F S S M T W T F S First Task Second Task Third Task Fourth Task Fifth Task Sixth Task Seventh Task Eighth Task Be specific: Who, What, by When?

59 Population vs. Performance 59

60 The Linkage between Population and Performance 60 Whole Population Population Accountability The well-being of Whole Populations Communities, Cities, Counties, States, Nations R Client Population Performance Accountability The well-being of Client Populations Programs, Agencies, Service Systems

61 Population Accountability R Self-Sufficient Families 61 Percent of parents earning a living wage Contributory relationship Job Training Program # persons receiving training # who get living wage jobs Unit cost per person trained % who get living wage jobs Alignment of measures Appropriate responsibilities Performance Accountability

62 Population Accountability Two R Lessons to turn a population curve: It takes many aligned programs & activities Self-Sufficient Families Percent of parents earning a living wage You cannot prove causality of any one program to turn a population curve 62 Contributory relationship Alignment of measures Appropriate responsibilities Performance Accountability

63 63 Disaggregation of Data Results-Based Accountability? (or the unintended consequences of stopping at all )

64 Equity & Inclusion Disaggregating Data 64 To apply an equity lens data must be disaggregated by age, gender, race, class, time in Vaughan, location, etc. to develop effective strategies that ensures no one is left behind.

65 Equity & Inclusion: The Foundation of Effective Collective Impact 65 Not a separate principle or an add-on Requirements Front-end commitment to furthering equity Disaggregation of data Story behind the data Engage community members about lived experiences

66 Data Through the Aggregated Lens 66

67 Data Through the Disaggregated Lens 67 White Asian All children Black Hispanic

68 Emergency Room - PM # Quantity How much did we do? Quality How well did we do it? Effect Effort # of Patients Avg. emergency room # %/# # % wait time. # of patents stabilized % of patients stabilized Is anyone better off?

69 Break 69

70 70 Putting RBA into Action - Group Exercises Selecting Performance Measures Prioritizing Performance Measures Conducting Turn the Curve Thinking

71 Too Many Canadian Geese 71

72 72

73 73 Selecting Performance Measures Program????

74 Prioritizing Performance Measures 74 Headline PMs Secondary PMs Data Development Agenda 3 to 5 Headline PMs Meets the Public Square Test Everything else that s any good Used later in the Story Behind the Curve New Data Data in need of repair (quality/timeliness)

75 Criteria for Prioritizing PMs 75 Rate each candidate PM as High, Medium and Low. Communication Power Does the PM communicate to a broad range of audiences? Proxy Power Sometimes referred to as Importance Power. Does the PM say something of central importance about the Result? Does the PM bring along the data herd? Data Power Quality data available on a timely basis?

76 Choosing PMs Worksheet 76 Result Candidate PMs Communication Power Proxy Power Data Power Measure 1 H M L H M L H M L Measure 2 Measure 3 Measure 4 H H H Measure 5 H H L Measure 6 Measure 7 Measure 8 Data Development Agenda

77 77 Exercise: Turn the Curve Performance Accountability

78 78 Applying RBA to Your Work UW Grant Process Community Impact Grant Outcomes Table Identifying Performance Measures Turn the Curve Thinking for Greater Impact

79 RBA Levels 79 Population Level - Community Level Outcomes Turn the Curve to identify system level strategies Results Population Level Indicators Strategy level Backbone Collective Impact Turn the Curve to implement system level strategies Strategies System Performance Measures for the implementation of strategies Aggregated common program Performance Measures Activity Level Program work Turn the Curve to implement effective activities/programs Activities/Programs Program/Activity Performance Measures ProSPER

80 80 COMMUNITY IMPACT GRANT OUTCOMES TABLE OVERALL COMMUNITY RESULT: RESEARCH- BASED STRATEGY PERFORMANCE MEASURES (BETTER OFF) PERFORMANCE MEASURE TARGETS ACTIVITIES (ACTIONS) RESPONSIBLE PARTNER / ORGANIZATION PERFORMANCE MEASURES (HOW MUCH, HOW WELL, BETTER OFF) PERFORMANCE MEASURES TARGETS MEASUREMENT TOOLS Fiscal Year

81 Key RBA Steps to Complete the Community Impact Grant Outcome Chart Identify the Community Result Area 2. Identify the Research Based Strategy 3. Identify headline Better Off performance measures and targets for the Strategy (How are people better off?) As new data are available conduct Turn the Curve Thinking to identify new action steps and partners. 4. Partners identify the headline performance measures (How much, how well and better off), targets and measurement tools for each of their actions/programs. Partners as new data are available conduct Turn the Curve Thinking to identify program improvement strategies

82 Performance Measurement Matrix Quantity How much did we do? Quality How well did we do it? # of Customers Served Customer Satisfaction Effect Effort # Activities How productive? # Skills / Knowledge # Attitude / Opinion # Behavior # Circumstance/Condition # %/# # % Is anyone better off? Retention Rates Following Protocols Are we doing things right? % Skills / Knowledge % Attitude / Opinion % Behavior % Circumstance/Condition Are we doing the right things?

83 83 COMMUNITY IMPACT GRANT OUTCOMES TABLE OVERALL COMMUNITY RESULT: All children in Larimer County are ready for kindergarten RESEARCH- BASED STRATEGY PERFORMANCE MEASURES (BETTER OFF) PERFORMANCE MEASURE TARGETS ACTIVITIES (ACTIONS) RESPONSIBLE PARTNER / ORGANIZATION PERFORMANCE MEASURES (HOW MUCH, HOW WELL, BETTER OFF) PERFORMANCE MEASURES TARGETS MEASUREMENT TOOLS Children have parents who read to them regularly % of low-income parents who report reading regularly to their child(ren) 50% increase of parents who report reading regularly reading to their chid(ren) Parents-As- Teachers home visits ABC Example Agency How much - # of home visits How much - # of parents visited How well - #/% of parents who reported their learned something of value /80% Case logs Case logs End of home visit survey Better off - #/% of parents who reported they are now reading to their child(ren) regularly 50/66% 3-month follow-up survey Fiscal Year

84 84 COMMUNITY IMPACT GRANT OUTCOMES TABLE OVERALL COMMUNITY RESULT: RESEARCH- BASED STRATEGY PERFORMANCE MEASURES (BETTER OFF) PERFORMANCE MEASURE TARGETS ACTIVITIES (ACTIONS) RESPONSIBLE PARTNER / ORGANIZATION PERFORMANCE MEASURES (HOW MUCH, HOW WELL, BETTER OFF) PERFORMANCE MEASURES TARGETS MEASUREMENT TOOLS Fiscal Year

85 85 Selecting Performance Measures

86 Criteria for Prioritizing PMs 86 (Headline PMs) Rate each candidate PM as High, Medium and Low. Communication Power Does the PM communicate to a broad range of audiences? /PM Proxy Power Sometimes referred to as Importance Power. Does the PM say something of central importance about the Result? Does the PM bring along the data herd? Data Power Quality data available on a timely basis?

87 Turn the Curve Thinking How are we doing? 1 5. What is our action plan to turn the curve? 2. What is the story behind the curve? What works to turn the curve? Turn the Curve? 3. Who are the partners who have a role to play in turning the curve? 4 3

88 When Prioritizing Strategies & Actions 88 Key Criteria Leverage Feasibility Specificity Values

89 89 Turn the Curve: Performance Measures Research Based Strategies to Identify Activities

90 90 Turn the Curve: Performance Measures Program/Action continuous improvement

91 Resources - RBA

92 92 Next Steps Existing Collaborative Work Group Meetings to finalize applications and outcome tables August 7: School Readiness August 8: Kids on Track August 8: Transitions to Independence August 8: Working Families Thrive August 31: Application/Outcomes table deadline TBD fall: Community Impact Committee meeting presentations December 31: Grant notifications

93 Share a word or short phrase you want to remember and share about RBA 93

94 What is one word that describes this workshop 94

95 95 Thank you! 95