Performance Measurement and Social Indicators

Transcription

1 Performance Measurement and Social Indicators Maria P. Aristigueta Professor and Director School of Public Policy and Administration

2 Measurement Models as impetus for accountability to the funding source and public. Measurement Models Social Indicators called benchmarks or milestones in State government. Also frequently used in the non-profit sector. Performance Measures regular measurement of the results (outcomes) and efficiency of services or programs.

3 Measures Driving Reforms Ideally, these improvements, to include social indicators, will alter policymaking and policy management by facilitating the emergence of outcome based accountability systems, systems-wide coordination and integration efforts, performance-based competitive service models, and public sector privatization and democratization schemes (Corbett 1997, xix).

4 Definitions Performance measurement is the monitoring on a regular basis of the results and efficiency of services or programs. Social indicators are descriptions of conditions that are intended to inform public opinion and policy making (Duncan, 1974). Social indicators measure the wellbeing of society, while agency performance measure, measure the input, output or outcome of a specific program.

5 Principles of Performance Measurement Result-oriented Selective Reliable Useful Shared

6 Criteria for Performance Measures Relevance Reliability consistency of data collection Validity accurate representation of what is intended to be measured. Coverage Cost Effectiveness

7 Limitations of Performance Measures Do not tell why the outcome occurred this requires program evaluation Some outcomes cannot be measured directly. The information by itself is not sufficient for decisions need political judgments, leadership.

8 Performance Measures Should Focus on outcome Capture data that is accurate, verifiable, and consistent over time Yield information that helps make realitybased decisions Be reported regularly Be logically and directly related to an agency s goals, strategies and functionality Be worth the cost of collecting and analyzing

9 Types of Performance Measures Outcome Measures observable changes in desired skills, attitudes, knowledge, behavior, functioning, etc. Efficiency Measures measures resource costs in currency, employee time, or equipment used per unit produced or service output Output Measures counts the goods and services produced by an agency Input Measures show the resources used to produce services Social Indicators using societal information for public decision-making

10 Examples of Performance Measures Outcome Measures (intermediate and end outcomes) Evidence of increased learning by students. Recovering owed child support payments from absent parents People completing employment training programs where program participants are volunteering (intermediate outcome). Efficiency Measures Average cost per client served Average cost per inspection Cost per mile maintained- Asphalt Average wait time per customer (measure of quality service delivery)

11 Examples of Performance Measures" (continued) Output Measures Number of clients served Number of criminal cases proceeded Number of inspections conducted Number of lane miles maintained per year Number of customers served per day Input Measures amount of resources used (expressed as funds or number of employee-years, or both)

12 Intermediate Outcomes Intermediate outcomes are expected to lead to the ends desired but are not themselves ends. Service quality as intermediate outcome how well a service was delivered NOT what results occurred after the service

13 Desirable Prerequisites High-level support for performance measurement Reasonable program stability At least some computerized data-processing capability Agreement on use of measures Adequate time is allocated for data collection

14 Outcome Indicator Identifies a specific numerical measurement that indicates progress towards achieving an outcome. It is not usually an outcome. Percent of, ratio of, proportion of

15 Social Indicators Further Defined The quantification of societal phenomena for public decision making. Time series measurements allowing for the identification of long-term trends, periodic changes, and fluctuations in rate of change in conditions affecting well being of individuals and communities.

16 Examples of Social Indicators Perhaps, best known are the Indicators of Children s well-being (KIDS COUNT) in the 50 states funded through the Casey Foundation. Other social indicator reports in the United States include the Bureau of the Census, Social Indicators; the National Center for Education Statistics, The Condition of Education; the National Center for Health Statistics, Health United States In addition, States maintain their own indicators on state conditions including children, families, environment, crime, perceptions of community and government

17 Examples of Social Indicators Minnesota Milestones Delaware Compares Oregon Benchmarks Kids Count in all 50 States

18 Uses for Indicators State governments are using social indicators as basic tools of governance for state and local planning initiatives pre-policy to help define a problem or provide a perspective accountability efforts to share program initiatives to address the problem For example, Delaware s pre and post natal care programs to address low-birth babies

19 Other Uses Tools for policy development Program design and administration Resource allocation Intergovernmental relations Alignment with program performance measures Community projects to assist in provision of services to the public

20 Findings for Use of State " Indicators by Non-Profits Policy decision-making Internal management Accountability to the public A source for community projects Professional knowledge/growth Awareness

21 Criteria to Develop Indicators Indicators should assess well-being across a broad array of outcomes, behavior, and processes; Be easily and readily understood by the public and the policy makers; Assess positive and negative aspects of wellbeing; Meet standards for validity and reliability; And be reflective of social goals.

22 Misuse of Social Indicators Data that is available sometimes becomes the indicator rather than what is needed. Explanation: Data on indicators are expensive and sometimes difficult to obtain. Unevenness in coverage of quantifiable societal conditions sometimes occurs do to personal or political area of interest. Explanation: The process for using social indicators selection, collection, mode of presentation, choice of baseline or comparative data is subjective.

23 Misuse Agencies may be held accountable for social indicator. Factors may include societal trends (increase in births to single parents); nation s demographics (increase in social service needs among elderly population). Data presented without explanation. Possibilities for unwarranted conclusions or alternate interpretations.

24 Benefit of Social Indicators Social reporting de-mystifies politics. Provides a common ground on which everyone can learn something on the status of the communities well-being. Useful in discussions of the limitations of government. Stimulate debate for best solutions to social problems.

25 Basic Types of Breakouts " for Outcome Data* Workload or customer characteristic Organizational unit or project Geographical location Type and amount of services *Note: may be used for social indicators and performance measures.

26 Common Problems in Communicating Performance Data Failing to provide context and interpretation for performance data Failing to anticipate different legislative information needs Providing too much or too little detail

27 Making the Data Decision:" A Balancing Act Too many measures reported overwhelm the process Performance measures should be layered Let them know what data you have without necessarily including it in report Too few measures will not provide the information needed to assess performance

28 Types of Data to Report Make sure the measures report the bottom line. Are things getting better, worse, or remaining the same?

29 Models Logic Model -- Diagram cause and effect Inputs or resources Outputs Expected Outcomes Balanced Scorecard Goals and associated performance measures 4 quadrants Customer Financial performance Internal business process Innovation and learning Strategy maps Links performance measures to goals

30 Communicating in a " Legislative Environment Make performance data: Stimulating Informative Self-contained Provide human element when possible in discussion

31 Sources T. Poister in Joseph S. Wholey, Harry P. Hatry, and Kathryn E. Newcomer, Handbook of Practical Program Evaluation, chapter 5, (Jossey-Bass, 2010) Harry Hatry, Performance Measurement: Getting Results (Washington, D.C.: Urban Institute, 1999). M. Aristigueta, Indicators of Living Conditions in Marc Holzer, Public Productivity Handbook, Dekker)