Last Updated: December 18, 2015 Welcome Baby Impact Evaluation Request for Proposals FREQUENTLY ASKED QUESTIONS Information Webinar Please note that some responses may have been refined to more clearly address the question posed by webinar participants. 1) Do WB participants need a high/moderate/low risk score? As the Welcome Baby Home Visiting System is currently implemented, only women who score high low/moderate risk on the Modified Bridges for Newborns Screening Tool (Modified Bridges) and reside in a Best Start Community are eligible to receive the Welcome Baby Program. However, First 5 LA expects that both mothers who score high and score low/moderate on the Modified Bridges will be included in the intervention group for the impact evaluation. This is a deviation from the standard implementation of the program, but it is to allow the impact evaluation to address research question number 3 (i.e., Is Welcome Baby equally beneficially for women of varying risk levels and their children?). 2) I m assuming Select Home Visiting services wouldn t be offered to mothers in hospitals where the evaluation is being conducted, correct? It is likely that Select Home Visiting Services would continue to be offered to mothers in the hospitals where the evaluation is being conducted. Not all mothers delivering at the selected impact evaluation sites will be eligible for or consent to participating in the impact evaluation. First 5 LA will continue to offer home visiting services as typically implemented (see pages 6-7 of the RFP) in these circumstances. 3) Is it correct that the evaluators would not have access to the Children s Data Network (CDN)? I m wondering if the evaluators would be able to use any data elements from the CDN. The evaluators will work in partnership with the CDN to link the evaluation data with other sources of administrative data. The CDN staff will conduct the linkages because of their data sharing agreements with the agencies that oversee those data sources. The CDN will not be able to release identifiable data to the evaluator. 4) It sounds like the evaluator has to ensure that there are sufficient identifiers available from the data it collects so that the CDN staff can match those data to its CDN data. Yes. That is correct. During the planning period, the selected contractor will have the opportunity to meet with the CDN team to ensure the appropriate identifiers are collected and that the appropriate participant consents/permissions are in place to allow for data matching.
5) Can you provide a little more information about the data files that you would like to receive? For example, how would they be used? Would any sort of Public Use File be created after the WB evaluation is over? As noted in section X. Collection and Evaluation of Data and Information of the Sample Agreement included in this RFP as Appendix I: COMMISSION may internally use research findings and results generated from the data and information for planning purposes prior to CONTRACTOR s publication of the findings and results. Neither COMMISSION nor CONTRACTOR shall disseminate the data and information beyond its internal staff without the other party s consent. Within thirty (30) calendar days of the expiration or termination of this Contract, CONTRACTOR shall deliver a copy of all collected data and information to the designated COMMISSION staff in hard copy, electronic format, or in such other format as requested by the designated COMMISSION staff. The parties shall determine the timing, format and manner of the dissemination of the data and information and any report of CONTRACTOR s results, conclusions or recommendations. First 5 LA s use of the data files may include, but is not limited to, conducting analysis for program improvement, reporting to the First 5 LA Board of Commissioners, or disseminating information about the Welcome Baby program s effectiveness to stakeholder groups. 6) Would baseline data collection need to be done in addiction to accessing data that are collected during the hospital visit when the baby is born? Yes. Our expectation is that the selected contractor will collect baseline data on outcomes relevant to the Welcome Baby program. 7) A few questions about the partner organizations: Can you please explain the role of LABBN? Also, do the WB programs and hospitals help with recruitment? Finally, who are the other third-party evaluators? Los Angeles Best Babies Network (LABBN) leads the Family Strengthening Oversight Entity. The Oversight Entity oversees and supports the standardization of the Welcome Baby program to ensure adherence to program fidelity. Their major functions include supporting implementation of the Welcome Baby program with quality and fidelity to the model; provision of technical assistance support for the data management information system; facilitation of cross-site peer learning and information exchanges; and support for ongoing communication and messaging efforts. LABBN has extensive knowledge of the Welcome Baby program that will be helpful to the evaluation team and will also help the evaluation team liaise with the Welcome Baby sites. We have not yet determined whether the Welcome Baby sites will assist with evaluation participant recruitment and their scope of assistance. This is something that could be negotiated between the selected contractor, the individual Welcome Baby sites, First 5 LA, and LABBN after the contractor has been selected. However, applicants should assume in their proposals and budgets that the Welcome Baby sites will not assist with recruiting participants into the evaluation.
There are two evaluations currently underway in the Welcome Baby evaluation portfolio. The first is an implementation and outcomes evaluation of Welcome Baby at the current Welcome Baby sites. This evaluation was launched in April 2015 and is being conducted by RAND. The second is a psychometric study of the Modified Bridges for Newborns Screening Tool. This evaluation was launched in August 2015 and is being conducted by a separate team at RAND. 8) Are the caps on hourly rates, in particular for consultants? There are no caps on hourly rates. It is expected that the applicants budgets will be clearly justified in the budget narrative and will be appropriate to the work proposed. 9) Can you please clarify how long the contract will be? The RFP says 3.5 years, but it looks like it will be 4 years? The total project term will be for 3.5 years. The initial contract terms is expected to be 15 months. Contract renewal and annual budgets are subject to approval by First 5 LA s Board of Commissioners. 10) How many of the site already have Welcome Baby, and how many will be new sites? Thirteen sites are currently implementing Welcome Baby, one site will begin implementing Welcome Baby pending First 5 LA Board of Commissioner s approval, and another site is currently in contract negotiations (See RFP Appendix E for additional details). First 5 LA does not have plans at this point for adding additional Welcome Baby sites beyond those listed in Appendix E of RFP. The sites for the Welcome Baby Impact Evaluation will be determined jointly by the selected contractor, Los Angeles Best Babies Network, and the First 5 LA team. Site selection considerations include size and number of births at each site, demographics of participants at each site, site fidelity to the Welcome Baby model, and each site s willingness to participate in an RCT. 11) The Implementation and Outcome Evaluation of Welcome Baby (at the current 14 hospital sites) that was launched in April 2015 -- When would the successful contractor have access to preliminary findings? The Welcome Baby Implementation and Outcomes Evaluation addresses a number of questions and will use multiple rounds of staggered data collection. Exact deadlines for findings have yet to be determined, but it is expected that data will be available to First 5 LA during fiscal year 16-17. Relevant information resulting from the Welcome Baby Implementation and Outcomes evaluation will be discussed with the selected contractor as needed to inform the Welcome Baby Impact Evaluation. 12) Does the budget we are to prepare include the costs of design, implementation, and evaluation, or is First5LA going to pay for the implementation separately? The budget for the impact evaluation should include costs for all of the tasks necessary to complete the impact evaluation (e.g., designing final evaluation plan, data collection, data
analysis). Information about the Scope of Work for this evaluation can be found in Section V of the RFP (pages 14-18). 13) Will you email a copy of the PPT presentation to webinar participants? The video recording of the Welcome Baby Impact Evaluation RFP Information Session and a copy of the PowerPoint slides is available on the First 5 LA website at: http://www.first5la.org/index.php?r=site/article&id=3641 14) Do WB participants need to receive a high, moderate, or low score on the modified bridges screening tool? As the program is implemented, all women delivering at a Welcome Baby hospital are offered the Welcome Baby hospital visit. During the hospital visit, the Modified Bridges is administered. If a woman scores high low/moderate risk on the Modified Bridges and resides in a Best Start Community, she will be offered the Welcome Baby home visiting program. (Please see pages 6-7 and Appendix B of the RFP for additional information about how the Modified Bridges and place of residence are used to determine level of services.) However, First 5 LA expects that both mothers who score high and score low/moderate on the Modified Bridges will be included in the intervention group for the impact evaluation. This is a deviation from the standard implementation of the program, but it is to allow the impact evaluation to address research question number 3 (i.e., Is Welcome Baby equally beneficially for women of varying risk levels and their children?). Questions Received After the Welcome Baby Impact Evaluation RFP Information Webinar 1) Does the agency require price by deliverable based on the table on pages 17-18 of the RFP? Yes, applicants should create a budget organized by task using the template provided as Appendix M. The Table on pages 17-18 of the RFP provides a recap of tasks and likely deliverables for this study. If the applicant proposes different or additional tasks, those tasks and associated prices should be included in the proposed budget. 2) Is payment to be fixed price per deliverable? Yes. 3) Is the contract to be a fixed price contract? Yes. ADDED ON DECEMBER 3rd, 2015: 1. Appendix E outlines the number of Best Start births and non-best Start births at the participating hospitals. a. Is it possible to share the number of Best Start families enrolled in WB at the participating hospitals?
BS Families enrolled in WB Hospital Name FY 14-15 Antelope Valley Hospital 867 California Hospital Medical 353 Centinela Memorial Medical 73 Citrus Valley Medical - Queen of the Valley Campus 248 Miller Childrens Hospital 448 Northridge Hospital Medical 58 Providence Holy Cross Medical 315 Providence LCM Medical - San Pedro 54 St. Francis Medical 569 St. Mary Medical 498 Torrance Medical 77 Valley Presbyterian Hospital 408 White Memorial Medical 532 b. Is it possible to share the proportion of Best Start families enrolled in WB prenatally at the participating hospitals? Prenatal enrollment % of BSC clients in FY 14-15 Antelope Valley Hospital 30.63% California Hospital Medical 29.26% Centinela Hospital Medical 25.95% Citrus Valley Medical - Queen of the Valley Campus 27.38% Miller Children s Hospital 19.73% Northridge Hospital 61.42% Providence Holy Cross Medical 40.42% Providence LCM Medical - San Pedro/ Torrance Memorial 22.22% St Mary Medical 22.19% St. Francis Medical 30.73% Valley Presbyterian Hospital 41.72% White Memorial Medical 46.38% 2. There are four primary goals to the evaluation. The second goal is to determine whether the impact of WB program varies across maternal risk levels. Does First 5 LA have different risk levels in mind that they want tested here? First 5 LA measures maternal risk using the Modified Bridges Screening Tool. The Modified Bridges Screening Tool categorizes women as high risk with a score greater than or equal to 50 or low/moderate risk with a score of less than 50. Under normal circumstances, women would be triaged into different types of services ranging from information and referrals to more intensive home visiting services depending on risk and whether or not the mother lives in a Best Start community. For the purpose of this evaluation, First 5 LA is interested in determining the impact
of Welcome Baby across low/moderate and high risk levels so it will be important for applicants to take this into consideration when designing this evaluation, as it will have implications for recruitment and analysis. 3. The third evaluation goal is to identify variation in the impact of WB program across demographic subgroups. On pg. 13, it states that First 5 LA is especially interested in variations in program impact related to participant ethnicity. Can First 5 LA share the enrollment rates by participant ethnicity at the different sites? Race enrollment of BSC clients in FY 14-15 Hospital Name Alaska Native/ American Indian Asian Black/ African American Caucasian Hispanic/ Latino Middle Eastern Multiracial Other Race Pacific Islander Unknown Antelope Valley Hospital 0.35% 1.27% 19.72% 15.34% 55.71% 0.35% 6.00% 1.15% 0.12% California Hospital Medical Centinela Memorial Medical Citrus Valley Medical - Queen of the Valley Campus 0.57% 0.28% 24.36% 0.28% 72.52% 0.57% 0.28% 0.57% 0.28% 52.05% 1.37% 41.10% 4.11% 1.37% 0.81% 1.61% 95.16% 2.42% 0.28% 0.00% 0.00% Miller Children s Hospital 0.22% 4.69% 18.08% 2.90% 70.09% 2.46% 0.67% 0.89% Northridge Hospital Medical Providence Holy Cross Medical Providence LCM Medical - San Pedro 5.17% 1.72% 3.45% 86.21% 1.72% 1.90% 2.22% 2.22% 91.75% 1.59% 5.56% 1.85% 1.85% 87.04% 3.70% 1.72% 0.32% 0.00% St. Francis Medical 0.18% 19.89% 1.06% 76.94% 1.23% 0.70% 0.00% St. Mary Medical 2.41% 7.23% 1.81% 87.15% 0.20% 1.20% Torrance Medical 3.90% 10.39% 3.90% 77.92% 2.60% 1.30% Valley Presbyterian Hospital 1.72% 2.70% 1.47% 91.18% 0.49% 2.21% 0.25% White Memorial Medical 0.19% 0.19% 99.06% 0.56% 0.00% ADDED ON DECEMBER 15 th, 2015 1. Will the revised start date of mid-april mean that the first budget period is 14.5 rather than 15 months In order to align with the First 5 fiscal year? a. If this is the case, would the final budget period be longer to bring the project to 3.5 years? Yes. The revised start date will shorten the initial budget period slightly to align with the First 5 LA s fiscal year, but the revised start date will not result in a longer final budget period. 2. The timeline section of the RFP indicates that the contract period ends in 2020 (see quote below). On the other hand, the Terms of Project section indicates that the project is 3.5 years, thus running through the Fall of 2019. Can you please clarify?
The language under the Terms of Project section are accurate. First 5 LA anticipates that the project will be completed in the fall quarter of 2019. 3. Should the columns in the budget template labeled Staff A, Staff B, etc. be treated as labor categories for multiple staff or individual staff members? Can we add additional columns? The columns in the budget template should be treated as individual staff members. Additional columns can be added to reflect the staff working on the project. 4. Are subcontractor s subject to First 5 LA approval before the Contractor can enter into an agreement with them? Yes. As noted in section XXIX. Assignment and Delegation of Sample Contract included in the RFP as Appendix X: CONTRACTOR may not assign any of its rights or delegate any of its duties under this Contract without COMMISSION s prior written consent, which consent may be withheld in COMMISSION s sole and absolute discretion. If COMMISSION consents to CONTRACTOR s subcontracting of all or a portion of this Contract, CONTRACTOR shall submit to COMMISSION all proposed subcontractors and/or a copy of the subcontract or memorandum of understanding between CONTRACTOR and the subcontractor if required by the designated COMMISSION staff for COMMISSION s prior review and approval. CONTRACTOR shall provide copies of executed subcontracts if requested by the designated COMMISSION staff. 5. Can you clarify the difference between the 2 deliverables in Task C? Are we to budget for preparing each IRB submission under IRB packet(s), including consent forms and budget for answering IRB-specific questions and handling post submission follow up work under IRB approval letter(s)? Applicants should prepare their budgets in a manner that allows them to complete the work required. First 5 LA requires that the selected contractor submit the IRB packet(s), consent forms, and IRB approval letter(s), but it is up to the contractor to divide the specific activities resulting in IRB submission and IRB approval in whichever way they see fit. Applicants should keep in mind that the tasks and deliverables included on page 17 of the RFP are likely deliverables for the Welcome Baby Impact Study. The information in this table is preliminary and subject to change. If applicants design a study that calls for different tasks and deliverables, those should be included in the proposal along with the rationale supporting those decisions. 6. Can you clarify the difference among the 4 deliverables in Task D? Are there 4 separate drafts of the protocols and instruments submitted to First 5 LA? We understand that we submit an initial draft, a second draft incorporating feedback from First 5 LA, and a third draft incorporating revisions from the piloting process. How do the two versions of the protocol and instruments differ between the Data collection protocol instruments, including translated versions deliverable and Post-pilot memo & revised protocol and instruments based on piloting process deliverable?
Applicants should keep in mind that the tasks and deliverables included on page 17 of the RFP are likely deliverables for the Welcome Baby Impact Study. The information in this table is preliminary and subject to change. If applicants design a study that calls for different tasks and deliverables, those should be included in the proposal with the rationale supporting those decisions. The deliverable included under Task D as Data collection protocol instruments, including translated versions was meant to indicate the inclusion of translations in the agreed upon languages. First 5 LA generally expects a draft deliverable for review and a final deliverable for all deliverables. This particular task will also require a memo explaining the changes made and lessons learned from the piloting process.