REQUEST FOR PROPOSALS For the Development of an Accreditation Evaluation Plan for the National Public Health Accreditation Program February 16, 2012

Transcription

1 PUBLIC HEALTH ACCREDITATION BOARD REQUEST FOR PROPOSALS For the Development of an Accreditation Evaluation Plan for the National Public Health Accreditation Program February 16, 2012 I. BACKGROUND Quality improvement is a key aspect of the voluntary national accreditation program recently launched by PHAB, the Public Health Accreditation Board. PHAB hopes that, through participating in the accreditation process, public health agencies will move down a path of continuous quality improvement, thus improving the delivery and quality of services provided to their constituents. This will, ultimately, improve community health. While quality improvement is an essential part of the accreditation program administered by PHAB, it is also a key aspect of PHAB s internal operations. While the standards, measures and processes developed by PHAB have been refined in response to feedback from a Beta test, PHAB recognizes that it too needs to embrace quality improvement to ensure that it is positioned to best serve the various stakeholders associated with the accreditation movement. As a result, a logic model (Attachment A) was developed to guide the evaluation of the accreditation program administered PHAB. The logic model is broken into7 broad categories: resources, inputs, strategies, outputs, short term outcomes, intermediate outcomes and long term outcomes. While this logic model focuses on evaluating PHAB operations, it does so while taking into consideration the greater context under which PHAB operates, and the ultimate mission of PHAB. Thus, the various stakeholders associated with accreditation are also integrated into certain aspects of the model. II. PROJECT DESCRIPTION PHAB is looking for a partner to consider the elements of the logic model, Attachment A, and develop a long-range evaluation plan for PHAB s Accreditation Program. The plan should encompass the utilization of the resource identified in the Logic Model as well as the incorporating the framework to include the strategies, inputs and outputs identified by PHAB s Research and Evaluation Committee. The plan should address all these elements and include a February 17, 2012 Page 1 of 4

2 recommended implementation strategy that would get at some of, or all, the defined outcomes in the Logic Model. The plan should also address general time lines for each of the evaluation components. The final evaluation plan should be detailed enough to guide various aspects of PHAB s evaluation over time, while building in enough flexibility to address the developmental nature of national public health department accreditation. Attachment B provides additional background information that should be helpful in the development of this proposal. III. SCOPE OF WORK To develop a detailed plan that takes into account the work done by PHAB s Research and Evaluation Committee and the PHAB Board through the Logic Model provided in Attachment A. The plan should provide a roadmap for a future contract in implementing an internal programmatic evaluation. The final deliverable is an evaluation plan for the future; no evaluation work is expected under this RFP. The plan will guide the development of future Requests for Proposals and contracts to conduct the detailed evaluation work. IV. RESOURCES PHAB has designated funding in the amount of $25,000 for the plan development phase of the evaluation project. The budget includes personnel costs, travel and meetings, and other deliverables. V. STAFFING PHAB will provide a dedicated program manager and other staff as needed to be available during this development stage. VI. TIMELINE PHAB expects to award the contract by March 16, 2012 with an expected completion date of May 16, VII. REQUEST FOR PROPOSAL In order to be considered for this project, applicants must be able to demonstrate significant experience in developing an evaluation plan. Ideally, the candidate would have experience in implementing the same type of plan. Bidders are encouraged to propose additional activities that would expand upon or improve the tasks and are within the overall scope and budget, or, if necessary, to suggest those that must be eliminated in order to remain within the budget. February 17, 2012 Page 2 of 4

3 In order to be considered for this project, proposals must include: 1. The name, and contact information, and brief bio of the principal and other significant contributors to the effort. 2. The qualifications of the organization to perform the contract. This will include a description of the organization's experience with similar efforts and available resources. 3. Clear delineation of plans, budgets and deliverables included in the proposal. 4. Work plan, including detailed/line item budget and timeline. 5. Summary of the project teams qualifications to perform the required scope of work. 6. One to two examples of the applicant s work in developing and evaluation plan. 7. Two references who are familiar with the applicant s ability to perform the scope of work and who would be willing to be contacted regarding the proposal. Proposals should not exceed 15 pages, including budgets and timelines but not including attachments (e.g., curriculum vitae and examples of work). Narratives should be written in at least a 12 point font with margins of no less than one inch and 1.5 line spacing. Description of organizational capacity, previous similar evaluation projects, resumes, and any other additional background information may be included as attachments, and can be excluded from the 15 page limit. Proposals must be delivered electronically, PDF only and should be ed to mpaepcke@phaboard.org no later than 5:00 pm Eastern time, March 16, Hard copies and facsimiles will not be accepted. VIII. REVIEW AND SCORING OF PROPOSALS Proposals will be reviewed on how well the plan addresses the Logic Model and address some of the research agenda questions from Appendix B. IX. OTHER CONSIDERATIONS A presentation for the PHAB Board of Directors may be required once the final evaluation plan has been submitted. PHAB will cover the costs of attending a meeting if that requirement is requested. X. CONTACT INFORMATION Direct questions and proposals to: Mark Paepcke Public Health Accreditation Board 1600 Duke Street, Suite 440 Alexandria, VA February 17, 2012 Page 3 of 4

4 ext Attachment A PHAB Evaluation Logic Model Attachment B Background Information for the Development of a Long-Term Evaluation Plan February 17, 2012 Page 4 of 4

5 Attachment A: Public Health Agency Accreditation System Implementation Approved August 2010 Inputs Strategies Outputs Short-Term Outcomes Intermediate Outcomes Long-Term Outcomes PHAB Resources: Organizational structure Board, committees and work groups Staffing and expertise Information system Standards, measures and guidance Assessment process Site visitors External Resources: Funders and partner organizations Funding Incentives Technical Assistance Public Health Agencies: Interest, buy-in and commitment to seek accreditation Appropriate stability, resources and level of readiness to apply Previous quality improvement and assessment experience PHAB Strategies: Market program Implement program - Train agencies - Review application and documentation - Conduct Site visit - Determine accreditation status - Write and share report Develop database Evaluate program and improve quality Promote research Stakeholder and Partner Strategies: Promote national accreditation Encourage agencies to seek accreditation Support agencies through TA before, during and after process Public Health Agency Strategies: Participate in training and TA opportunities Submit application Conduct selfassessment Host site visit Review findings Share results Develop and implement improvement plan Implement QI Participate in reaccreditation process PHAB: Accreditation program: marketed, implemented, evaluated and improved Database developed Stakeholders and Partners: Promotion and support efforts provided Research conducted Public Health Agencies: Agencies are accredited Report/results received and acted on QI efforts are in place Plans for reaccreditation underway Strong, credible and sustainable accreditation program in place Increased science base for public health Increased ability to communicate work and results Increased support for accreditation Increased use of benchmarks for evaluating performance Increased organizational accountability Increased knowledge of organizational strengths and weaknesses Improved identification and use of best practices Increased consistency in practice Improved quality of services Increased interagency and intersectoral collaboration Increased visibility of public health agencies PH agencies more effectively and efficiently use resources Strengthened organizational capacity and workforce Improved responsiveness to community priorities Strengthened public health agencies and systems Improved conditions in which people can be healthy Improved community health indicators Increased public investment in public health Increased public recognition of public health role and value Legend Accrediting Agency Individual Public Health Agencies Stakeholders and Partners Public Health Field

6 APPENDIX B BACKGROUND INFORMATION FOR THE DEVELOPMENT OF A LONG-TERM EVALUATION PLAN FOR PHAB Resources Given the relative youth of PHAB s accreditation program (it was officially launched in September 2011), early evaluation efforts will focus mainly on assuring that PHAB has sufficient capacity to effectively carry out the accreditation program. These efforts examine the resources, inputs and strategies associated with accreditation. Some resources that PHAB has identified are: Public health practice partner organizations Funders, public health agencies, policy makers The general public A basic foundation that supports accreditation, as well as perceived momentum toward accreditation. While early evaluation may consider all these resources, resources of particular interest to PHAB are those that may ultimately determine the success or failure of the accreditation movementtribal, state and local public health agencies and policy makers including elected and appointed officials.

7 Accreditation is a voluntary process, and PHAB must assure that public health agencies are willing to assume the costs and risks inherent in the accreditation process. Perception and value are key constructs agencies must perceive both that accreditation is a worthwhile endeavor, and that the benefits outweigh the costs. Resources of particular interest to PHAB related to local public health agencies fall into two main areas, context and capacity. Evaluators may wish to consider items such as: The interest of agencies to pursue accreditation Agency perception of accreditation Agency readiness for accreditation Agency financial resources to apply for/maintain accreditation Previous agency experience with quality improvement. For a more detailed list of sample questions that may help evaluate these items, see Appendix B. Resources associated with policy makers relate primarily to their knowledge and perception of accreditation. In order to assure a supportive political environment exists for accreditation, they, too must perceive that accreditation is a worthwhile endeavor, and that the benefits outweigh the costs. Resources of particular interest to PHAB here are: Policymaker knowledge of accreditation Policymaker perception of its worth Policymaker willingness to support accreditation efforts. For a more detailed list of sample questions that may help evaluate these items, see Appendix B.

8 Inputs PHAB recognizes that, as the accreditation process is implemented and, later, modified, it must continue to be mindful of the necessity to embrace a sustainable business model. A key aspect of this is that PHAB must have adequate infrastructure and support to effectively carry out the accreditation process. In addition, there are other resources that will work in synergy with the accreditation process. Some inputs that PHAB has identified are: The staff and financial contributions of funding agencies The staff and financial contributions of partner agencies Things such as data and existing public health standards A knowledgeable public health community in accreditation and Quality Improvement. Resources for quality improvement PHAB s funders and partners provide financial support to help PHAB operate- this is particularly important during the early phases of accreditation, when any revenues from accreditation fees may not be sufficient to support a large portion of the overall cost of PHAB operations. Funders and partners also provide contributions through the expertise of staff members assigned to assist PHAB- these individuals provide PHAB with technical support and advice on how to best execute PHAB operations. Other key inputs identified by PHAB are resources that will help inform or facilitate PHAB s efforts. These include data that will promote research and evaluation focused on accreditation, existing public health standards that may help agencies prepare for accreditation, or may provide PHAB with models to modify standards and measures, and a knowledgeable public health community. For a list of potential questions that may help evaluate these items, see Appendix B.

9 Strategies PHAB is in the initial phase of executing several activities that it has identified as being key determinants of a successful accreditation program. While these activities are in various stages of execution, they are ongoing processes and many of them will require evaluation to determine if and how they should be modified. Evaluation of these processes relates primarily to the internal operations of PHAB itself, the development and marketing of the accreditation process, and evaluation of PHAB. Some strategies that PHAB has identified are: Develop and implement evaluation strategy Build sufficient PHAB infrastructure to support national voluntary accreditation Establish management & committee structures to support PHAB work Develop PHAB accreditation standards Develop the 7 steps of the accreditation process Pilot accreditation standards and steps in the accreditation process Actively market and promote PHAB accreditation Conduct training on PHAB processes Establish incentives/funding, in conjunction with partners PHAB recognizes the need to evaluate many domains of accreditation. A key aspect of early evaluation efforts will be evaluating PHAB operations ensure that PHAB has sufficient internal capacity to effectively manage and administer accreditation activities. This internal capacity encompasses both the PHAB business office, and PHAB s governing body and the various committees associated with PHAB.

10 While the strategies to develop PHAB standards, measures, training materials on accreditation, and the overall accreditation process have already undergone Beta testing and evaluation, PHAB plans to engage in continuous quality improvement around these areas. As a result, PHAB recognizes the need to continuously evaluate these areas to assure they remain relevant to the public health practice community, and continue to promote effective public health practice. These areas can be viewed in the context of the seven steps of the accreditation developed by PHAB: 1. Pre application, where an applicant prepares for accreditation, assesses its readiness, completes online orientation, and informs PHAB of its intent to apply 2. Application, including completing application training 3. Document selection and submission, where an applicant selects documentation to support compliance with each measure, and uploads and submits it via e-phab. 4. The site visit, where specially trained site visitors examine the applicant in the context of PHAB s standards/measures, and develop a site visit report. 5. The accreditation decision, where PHAB s Accreditation Committee reviews the site report and determines the accreditation status of the applicant. 6. The reports process, where accredited agencies submit annual reports for the five year duration of the accreditation decision. 7. Reaccreditation Effective marketing is a critical aspect of demonstrating the value of accreditation to the general public, the practice community, and policy makers. PHAB seeks, through marketing, to educate the public health community about accreditation- its cost, its value, and the process itself. In addition, PHAB is seeking to determine the optimal types of incentives and sources and amount

11 of funding that will move public health agencies toward seeking accreditation. For a list of potential questions that may help evaluate items associated with strategies, see Appendix B. Outputs The strategies and activities associated with accreditation produce a number of specific outputs. In many cases, evaluation of these outputs focuses on PHAB internal operations, and many aspects will focus on short term manifestations of PHAB s initial strategies. Outputs identified by PHAB include: PHAB infrastructure PHAB staff and internal processes PHAB board and committees PHAB standards PHAB products PHAB Process 7 steps Site visitors Beta Test QI/evaluation processes Marketing/promotion campaigns While evaluation up to this point has focused on assuring that PHAB has put in place the building blocks necessary to support initial operations, evaluation activities can now examine the results produced by accreditation. However, they may still mainly focus on issues such as capacity. Evaluation can now use data gathered from the accreditation process itself to provide a deeper understanding of the areas covered under the resources, inputs, and strategies sections.

12 While many of the outputs in this section are addressed in previous sections, they are now examined from a different perspective. For example, while the accreditation standards may have been examined for adequacy at earlier stages, including the Beta test, it is now possible to evaluate using the final version of the standards and using data gathered from agencies actually engaging in the refined accreditation process. While the site visitation process may have been evaluated earlier, it was done using a limited amount of visitors and a small amount of agencies. Now, evaluation can focus on the actual products and 7 steps of the accreditation process. Questions that may potentially help evaluate these items are contained in Appendix B. Short Term Outcomes PHAB s operations are all intended to produce outputs that effect both proximal and distal outcomes. The evaluation of short term outcomes mainly focuses on the basic products of PHAB, rather than system wide impact of PHAB activities. PHAB has identified the following short term outcomes: Strong/sustainable PHAB infrastructure Effective and efficient PHAB operations Trained/experienced site visitors Public health agencies accredited Public health agencies respond to incentives These short term outcomes are all things that are directly impacted by PHAB activities. The heavy focus on PHAB s internal operations seen in earlier phases is still present- most of the outcomes (infrastructure, operations, incentives and site visitors) relate directly to PHAB.

13 However, as accreditation progresses to this stage it is possible to examine outcomes, and PHAB s internal operations can be examined using more and better data that provides a wider perspective. Evaluation can now focus on the actual products of accreditation- accredited public health agencies- and the effect of accreditation on these agencies. While it may not be possible to yet examine in depth the changes brought about in public health agencies by accreditation, or the impact of accreditation on health, it is possible explore initial questions related to the effect of accreditation on agency operations, and the ability of agencies to maintain accreditation. For a list of potential questions that may help evaluate these items, see Appendix B. Intermediate Outcomes The intermediate outcomes developed by PHAB reflect the continued shift from evaluating internal operations to evaluating the impact of accreditation on public health agencies. PHAB has identified the following intermediate outcomes related PHAB operations: Accreditation is the norm Ongoing revision of PHAB standards/measures/processes Strategy to use CQI/evaluation results to improve PHAB operations and processes PHAB initial research agenda is implemented Incentives funded These outcomes focus mostly on PHAB s effectiveness in promoting accreditation, the degree to which PHAB has embraced and implemented quality improvement activities, and the ability of PHAB to engage the research community in activities that will provide data to support and

14 improve accreditation. Appendix B contains a detailed list of potential questions that may help evaluate these items. PHAB has identified the following intermediate outcomes related to public health agencies: Buy in Increased understanding of agency strengths and weaknesses Agencies adopt policies that are in agreement with/fulfill PHAB standards/measures Agencies adopt PHAB pre requisites Agencies see value in accreditation Agencies see value in PHAB Agencies are more responsive to community priorities Agencies use resources better Agencies increase organizational capacity Agencies increase inter-organizational collaboration PHAs increase inter-sectoral collaboration PHAs increase adoption of best practices/promising practices PHAs increase quality of services Evaluation of public health agencies can now focus, to a greater degree, on the extent to which accreditation has assisted public health agencies in improving their internal operations. It is now possible to get a more detailed view of the impact of accreditation on specific agency operations. For a list of potential questions that may help evaluate these items, see Appendix B.

15 Long Term Outcomes The long term outcomes identified by PHAB shift the evaluation focus from the public health agency to the impact of agency operations on the public, and changes in the perception of public health. Long term outcomes identified by PHAB include: Increased visibility for public health agencies Increased demonstration of accountability by public health agency to decision makers Increased demonstration of accountability by public health agency to the public Stronger public health agencies Improvements in the conditions/environment necessary for health Improved health indicators Increased recognition of the value of public health by decision makers Increased recognition of the value of public health by the public Increased investments in public health agencies These outcomes may be the ultimate determinants of the success or failure of accreditation. Accreditation seeks to facilitate the creation of a stronger public health system. This ideally will lead not just to stronger agencies, with increased resources, but also to improved community health. The core mission of public health agencies is to improve and maintain health. Unless PHAB can demonstrate the value of accreditation in terms that directly relate to increased capacity to improve community health, agencies and funders may ultimately judge that accreditation is not worth the cost in terms of money and effort. Thus, while some of these areas may be difficult to measure and not measurable for some time, it is critical that questions related to these areas be answered.

16 Developed by the Evaluation Subcommittee of the PHAB Research and Evaluation Committee Approved by the PHAB Board of Directors December 2011