Strengthening Cost Analyses of Prevention Programs. November 18, 2013 D. Max Crowley

Transcription

1 Strengthening Cost Analyses of Prevention Programs November 18, 2013 D. Max Crowley

2 Outline The Disappointment of Research on Prevention s Cost Methods & Rationale for Ingredients-Based Costing Process of Costing Prevention Programs Improving Research on the Cost of Prevention

3 Estimating the Costs of Prevention Many cost analyses will focus exclusively on budgetary records and fail to capture the full opportunity cost of a program (Belfield & Levin, 2013; Crowley, Hill, Kuklinski, & Jones, 2013; Foster, Porter, Ayers, Kaplan, Sandler, 2007) Three areas in particular that are often missed Costs of local programming infrastructure Volunteer and In-kind costs Participant costs

4 Prevention Programming Infrastructure Cost analyses of prevention programs tend to focus on the most proximal resources needed to implement programs Swisher, Scherer & Yin, 2004; Miller & Hendrie, 2009 These estimates often fail to account for infrastructure and local capacity that are essential to support programs (Crowley, Hill, Kuklinski, & Jones, 2013) Lack of local knowledge and skills about prevention requires resources be set aside for capacity building (Elliot & Mihalic, 2004; Crowley et al., 2013) These resources need to be accounted for within cost analyses (Kuklinski, Briney, Hawkins, & Catalano, 2012; Crowley, Jones, Greenberg, Feinberg, & Spoth, 2012)

5 Prevention Programming Infrastructure Cost analyses of prevention programs tend to focus on the most proximal resources needed to implement programs Swisher, Scherer & Yin, 2004; Miller & Hendrie, 2009 These estimates often fail to account for infrastructure and local capacity that are essential to support programs (Crowley, Hill, Kuklinski, & Jones, 2013) Lack of local knowledge and skills about prevention requires resources be set aside for capacity building (Elliot & Mihalic, 2004; Crowley et al., 2013) These resources need to be accounted for within cost analyses (Kuklinski, Briney, Hawkins, & Catalano, 2012; Crowley, Jones, Greenberg, Feinberg, & Spoth, 2012)

6 Prevention Programming Infrastructure Cost analyses of prevention programs tend to focus on the most proximal resources needed to implement programs Swisher, Scherer & Yin, 2004; Miller & Hendrie, 2009 These estimates often fail to account for infrastructure and local capacity that are essential to support programs (Crowley, Hill, Kuklinski, & Jones, 2013) Lack of local knowledge and skills about prevention requires resources be set aside for capacity building (Elliot & Mihalic, 2004; Crowley et al., 2013) These resources need to be accounted for within cost analyses (Kuklinski, Briney, Hawkins, & Catalano, 2012; Crowley, Jones, Greenberg, Feinberg, & Spoth, 2012)

7 Prevention Programming Infrastructure Cost analyses of prevention programs tend to focus on the most proximal resources needed to implement programs Swisher, Scherer & Yin, 2004; Miller & Hendrie, 2009 These estimates often fail to account for infrastructure and local capacity that are essential to support programs (Crowley, Hill, Kuklinski, & Jones, 2013) Lack of local knowledge and skills about prevention requires resources be set aside for capacity building (Elliot & Mihalic, 2004; Crowley et al., 2013) These resources need to be accounted for within cost analyses (Kuklinski, Briney, Hawkins, & Catalano, 2012; Crowley, Jones, Greenberg, Feinberg, & Spoth, 2012)

8 Prevention Programming Infrastructure Adoption Capacity (Lutenbacher, Cooper & Faccia, 2002; Ringwalt et al., 2005; 2009) Program Specific Training General Knowledge Training Implementation Capacity (Durlak & DuPre, 2008; Flay et al., 2005) Program Delivery Quality Assurance Systems Sustainability Capacity (Feinberg, Bontempo, & Greenberg, 2008; Gomez, Greenberg & Feinberg, 2005) Fundraising Maintaining Fidelity Integration and Expansion

9 Prevention Programming Infrastructure Adoption Capacity (Lutenbacher, Cooper & Faccia, 2002; Ringwalt et al., 2005; 2009) Program Specific Training General Knowledge Training Implementation Capacity (Durlak & DuPre, 2008; Flay et al., 2005) Program Delivery Quality Assurance Systems Sustainability Capacity (Feinberg, Bontempo, & Greenberg, 2008; Gomez, Greenberg & Feinberg, 2005) Fundraising Maintaining Fidelity Integration and Expansion

10 Prevention Programming Infrastructure Adoption Capacity (Lutenbacher, Cooper & Faccia, 2002; Ringwalt et al., 2005; 2009) Program Specific Training General Knowledge Training Implementation Capacity (Durlak & DuPre, 2008; Flay et al., 2005) Program Delivery Quality Assurance Systems Sustainability Capacity (Feinberg, Bontempo, & Greenberg, 2008; Gomez, Greenberg & Feinberg, 2005) Fundraising Maintaining Fidelity Integration and Expansion

11 Prevention Infrastructure Costs Crowley, Jones, Greenberg, Feinberg & Spoth, Resource Consumption of a Diffusion Model for Prevention Programs. Journal of Adolescent Health, (50)

12 Volunteer and In-Kind Donation Prevention outcomes often require substantial community buy-in to be successful Elliott & Mihalic, 2004 Use volunteer and in-kind donation to solidify community partnerships and shore up resource gaps Feinberg, Bontempo, & Greenberg, 2008) Not generally reflected in budgets and can be easily missed in retrospective analyses Crowley, Jones, Greenberg, Feinberg, Spoth, 2012 While often small during program trials, the cost can be substantial when programs delivered at scale Capturing the costs necessitates prospective analyses that can capture these non-budgetary costs

13 Volunteer and In-Kind Donation Prevention outcomes often require substantial community buy-in to be successful Elliott & Mihalic, 2004 Prevention efforts use volunteer and in-kind donation to solidify community partnerships and shore up resource gaps Feinberg, Bontempo, & Greenberg, 2008 Not generally reflected in budgets and can be easily missed in retrospective analyses Crowley, Jones, Greenberg, Feinberg, Spoth, 2012 While often small during program trials, the cost can be substantial when programs delivered at scale Capturing the costs necessitates prospective analyses that can capture these non-budgetary costs

14 Volunteer and In-Kind Donation Prevention outcomes often require substantial community buy-in to be successful Elliott & Mihalic, 2004 Prevention efforts use volunteer and in-kind donation to solidify community partnerships and shore up resource gaps Feinberg, Bontempo, & Greenberg, 2008 Not generally reflected in budgets and can be easily missed in retrospective analyses Crowley, Jones, Greenberg, Feinberg, Spoth, 2012 While often small during program trials, the cost can be substantial when programs delivered at scale Capturing the costs necessitates prospective analyses that can capture these non-budgetary costs

15 Volunteer and In-Kind Donation Prevention outcomes often require substantial community buy-in to be successful Elliott & Mihalic, 2004 Prevention efforts use volunteer and in-kind donation to solidify community partnerships and shore up resource gaps Feinberg, Bontempo, & Greenberg, 2008 Not generally reflected in budgets and can be easily missed in retrospective analyses Crowley, Jones, Greenberg, Feinberg, Spoth, 2012 While often small during program trials, the cost can be substantial when programs delivered at scale Capturing the costs necessitates prospective analyses that can capture these non-budgetary costs

16 Volunteer and In-Kind Donation Prevention outcomes often require substantial community buy-in to be successful Elliott & Mihalic, 2004 Prevention efforts use volunteer and in-kind donation to solidify community partnerships and shore up resource gaps Feinberg, Bontempo, & Greenberg, 2008 Not generally reflected in budgets and can be easily missed in retrospective analyses Crowley, Jones, Greenberg, Feinberg, Spoth, 2012 While often small during program trials, the cost can be substantial when programs delivered at scale Capturing these costs necessitates prospective analyses that can account for non-budgetary resource consumption

17 Participant Costs Participant costs represent the resources lost by the child, parent of family participating in the program Often includes up-front cost of time to receive program Also can be costs from downstream program effects (loss of means tested programs) Measurement of participant costs remains underdeveloped Numerous solutions to develop more robust estimates

18 Participant Costs Participant costs represent the resources lost by the child, parent of family participating in the program Often includes up-front cost of time to receive program Also can be costs from downstream program effects (loss of means tested programs) Measurement of participant costs remains underdeveloped Numerous solutions to develop more robust estimates

19 Participant Costs Participant costs represent the resources lost by the child, parent of family participating in the program Often includes up-front cost of time to receive program Costs also result from costs from downstream program effects (e.g., loss of means tested programs; Currie, 2004; Kitsman et al., 2000) Measurement of participant costs remains underdeveloped Numerous solutions to develop more robust estimates

20 Participant Costs Participant costs represent the resources lost by the child, parent of family participating in the program Often includes up-front cost of time to receive program Costs also result from costs from downstream program effects (e.g., loss of means tested programs; Currie, 2004; Kitsman et al., 2000) Measurement of participant costs remains underdeveloped Gold, Siegal, Russell & Weinstein, 1996 Numerous solutions to develop more robust estimates

21 Participant Costs Participant costs represent the resources lost by the child, parent of family participating in the program Often includes up-front cost of time to receive program Costs also result from costs from downstream program effects (e.g., loss of means tested programs; Currie, 2004; Kitsman et al., 2000) Measurement of participant costs remains underdeveloped Gold, Siegal, Russell & Weinstein, 1996 Numerous solutions to develop more robust estimates

22 Geographic Information Systems (GIS) CDC Behavioral Risk Factor Surveillance System (BRFSS)

23 Geographic Information Systems (GIS) CDC Behavioral Risk Factor Surveillance System (BRFSS) Crime Mapping in DC

24 Geographic Information Systems (GIS) CDC Behavioral Risk Factor Surveillance System (BRFSS) Crime Mapping in DC Program Delivery Site Mapping Participant Time Costs from Travel

25 Participant Costs Weekly participant costs captured through ehealth data architecture Move from a descriptive process to an area of empirical study (Basu, 2011; Crowley, 2013)

26 Participant Costs Weekly participant costs captured through ehealth data architecture Move from descriptive analysis to an area of empirical study (Basu, 2011; Crowley, et al. 2013)

27 Best Practices for Cost Analyses of Prevention Programs 1. Conduct Prospective Cost Analyses

28 Best Practices for Cost Analyses of Prevention Programs 1. Conduct Prospective Cost Analyses 2. Employ an Ingredients-Based Approach

29 Best Practices for Cost Analyses of Prevention Programs 1. Conduct Prospective Cost Analyses 2. Employ an Ingredients-Based Approach 3. Estimate Full Opportunity Cost of Prevention

30 Best Practices for Cost Analyses of Prevention Programs 1. Conduct Prospective Cost Analyses 2. Employ an Ingredients-Based Approach 3. Estimate Full Opportunity Cost of Prevention 4. Model Uncertainty in Costs

31 Possible Areas for Standardization Cost analyses are the foundation for economic analyses and require greater research attention Best practices exist and when implemented consistently can greatly improve the quality of economic analyses The limited attention cost analyses have received in the past leave substantial room for standards development Essential Cost Categories Appropriate Handling of Infrastructure & Participant Costs Minimum-Level of Sensitivity Analyses Consistent Reporting for Transparency and Comparability

32 Possible Areas for Standardization Cost analyses are the foundation for economic analyses and require greater research attention Best practices exist and when implemented consistently can greatly improve the quality of economic analyses The limited attention cost analyses have received in the past leave substantial room for standards development Essential Cost Categories Appropriate Handling of Infrastructure & Participant Costs Minimum-Level of Sensitivity Analyses Consistent Reporting for Transparency and Comparability

33 Possible Areas for Standardization Cost analyses are the foundation for economic analyses and require greater research attention Best practices exist and when implemented consistently can greatly improve the quality of economic analyses The limited attention cost analyses have received in the past leave substantial room for standards development Essential Cost Categories Appropriate Handling of Infrastructure & Participant Costs Minimum-Level of Sensitivity Analyses Consistent Reporting for Transparency and Comparability

34 Possible Areas for Standardization Cost analyses are the foundation for economic analyses and require greater research attention Best practices exist and when implemented consistently can greatly improve the quality of economic analyses The limited attention cost analyses have received in the past leave substantial room for standards development Essential Cost Categories Appropriate Handling of Infrastructure & Participant Costs Minimum-Level of Sensitivity Analyses Consistent Reporting for Transparency and Comparability

35 Possible Areas for Standardization Cost analyses are the foundation for economic analyses and require greater research attention Best practices exist and when implemented consistently can greatly improve the quality of economic analyses The limited attention cost analyses have received in the past leave substantial room for standards development 1. Essential Cost Categories 2. Appropriate Handling of Infrastructure & Participant Costs 3. Minimum-Level of Sensitivity Analyses 4. Consistent Reporting for Transparency and Utility

36 References Basu, Anirban. Economics of Individualization in Comparative Effectiveness Research and a Basis for a Patient-Centered Health Care. Journal of Health Economics 30, no. 3 (May 2011): doi: /j.jhealeco Belfield, CR, and H Levin. Guiding the Development and Use of Cost-Effectiveness Analysis in Education. Center for Benefit-Cost Studies of Education: Columbia University, August Crowley, D Max, Laura G. Hill, Margaret R. Kuklinski, and Damon Jones. Research Priorities for Economic Analyses of Prevention: Current Issues & Future Directions. Prevention Science (2013). Crowley, D. Max, Damon E. Jones, Mark T. Greenberg, Mark E. Feinberg, and Richard Spoth. Resource Consumption of a Diffusion Model for Prevention Programs: The PROSPER Delivery System. Journal of Adolescent Health 50, no. 3 (March 2012): doi: /j.jadohealth Durlak, Joseph A., and Emily P. DuPre. Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. American Journal of Community Psychology 41 (March 6, 2008): doi: /s Elliott, Delbert S., and Sharon Mihalic. Issues in Disseminating and Replicating Effective Prevention Programs. Prevention Science 5 (March 2004): doi: /b:prev Feinberg, M. E., Bontempo, D. E., & Greenberg, M. T. (2008). Predictors and level of sustainability of community prevention coalitions. American journal of preventive medicine, 34(6), Foster, E. Michael, M. M. Porter, T. S. Ayers, D. L. Kaplan, and I. Sandler. Estimating the Costs of Preventive Interventions. Evaluation Review 31, no. 3 (June 2007): doi: / x Gold, M., Spiegal, J. Russell, L., & Weinstein, M. (1996) Cost-Effectiveness in Health and Medicine. New York: Oxford University Press. Gomez, B. J., Greenberg, M. T., & Feinberg, M. E. (2005). Sustainability of community coalitions: An evaluation of communities that care. Prevention Science, 6(3), Kuklinski, Margaret R., John S. Briney, J. David Hawkins, and Richard F. Catalano. Cost-Benefit Analysis of Communities That Care Outcomes at Eighth Grade. Prevention Science (2012). doi: /s Miller, and D Hendrie. Substance Abuse Prevention Dollars and Cents: A Cost-Benefit Analysis. Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration, Swisher, John D., Jennifer Scherer, and Robert K. Yin. Cost-Benefit Estimates in Prevention Research. The Journal of Primary Prevention 25, no. 2 (October 2004): doi: /b:jopp c0.