NEMO WIB WIA Title I-B Substate Monitoring Plan

NEMO WIB WIA Title I-B Substate Monitoring Plan PRE-MONITORING ACTIVITIES Monitoring Scheduling All program operator subcontractors shall be monitored by staff of the NEMO Workforce Investment Board for each program they operate, at least annually. The Monitoring schedule and sample size will be in accordance with DWD Issuance 15-2010 Sub-State Monitoring policy. The Subcontractor shall be notified of the intended dates of scheduled monitoring at least two weeks prior to the date of the visit whenever possible. The NEMO WIB reserves the right to conduct unscheduled monitoring in conjunction with or in response to investigations of misfeasance and/or malfeasance or previous monitoring findings. Programmatic and Operational Monitoring will be conducted that effectively measures compliance with WIA regulations and DWD Policies including all of the NGCC policies. Monitoring will include a comprehensive examination of compliance issues cited in prior State and Region reviews and the determination where corrective measures were taken to address and resolve those issues. Sample Size A random sampling of files from every funding stream will be selected to test eligibility. WIA Adult and Dislocated Worker records/enrollments will be combined for the purpose of sampling NGCC service delivery. A sampling of core-only enrollments will be conducted. In addition, a combined sampling of records/files of Adult and Dislocated Workers participants enrolled in any of the following services: Classroom Training On-the-Job Training Work Experience/Internship Support Services/Needs-related payments Any other service that results in direct payment being made to, or on behalf of, a participant The following sample sizes are required, at a minimum, depending on the universe to be reviewed. These guidelines are applicable for every sample to be reviewed i.e., a sampling will be selected from the universe sample size listed below for Classroom Training, On the-job training, Work Experience/Internship, Support Services/Needs-Related payments and any other service that results in direct payment being made to, or on behalf of, a participant. Universal Sampling Size From To # of Files to review 1 200 51 201 300 55 301 400 58 401 500 60 501 1,000 63 1,001 2,000 65 2,001 10,000 67 Universe Sampling Size This table is based on random sampling for attributes when the expected rate of occurrence is not over (20) percent with a confidence level of ninety (90) percent, plus or minus eight (8) percent. Rvsd 10-25-12 1

Pre-Monitoring Desk Top Review Prior to the beginning of the monitoring visit, monitoring staff may review records by desk top. A random sampling of items may be reviewed during the monitoring. Those items may include but not be limited to: 1. program operator subcontractors contract scope of work; 2. program operator subcontractors planned performance levels (as applicable); 3. case management system information relating actual performance; 4. contract progress reports to determine actual expenditures; 5. Toolbox record data ON-SITE ACTIVITIES Entrance Conference Upon arrival at the program operator subcontractor's primary office, the lead person of the monitoring team shall hold an Entrance Conference with the WIA Title I-B Operator and/or Designated One-Stop Operator of the Career Center being monitored or his/her representative. The lead monitor shall outline the monitoring activities to be conducted and the WIB personnel will be responsible for each activity. The schedule for visiting work-sites and/or training sites will be planned. Monitors will answer any questions that the program operator subcontractor and/or one-stop operator staff may have relating to the monitoring. One-Stop Operator Staff Interview A consortium of one-stop partners for each full service career center requested to be designated and certified as the one-stop operator for that area. In doing so, they agreed to perform general tasks and responsibilities at the career center. The NEMO Workforce Investment Board has designated the Functional Leaders as the One-Stop Operator. As the One-Stop Operator the Functional Leader shall be interviewed at least once during each program year. The interview shall use a standardized form, which may be customized to meet special provisions of the individual Once-Stop Operator consortiums. A standardized form will be used so that Functional Leaders can conduct a Career Center self evaluation. The questionnaire is designed to allow Functional leaders to create and evaluate progress with NGCC process and procedures, evaluate minimum standards and the effectiveness of the processes. The WIB will use their responses for improvements and to identify skills/service gaps. Monitoring process will include questionnaires and procedures for interviewing customers, employers, trainers and Career Center staff. However at a minimum, the interview will inquire about the following processes and systems: 1. General management of the career center; including facility and maintenance responsibilities 2. Coordination of services 3. Continuous improvement 4. LMI system and data 5. Availability of resources 6. Performance and outcome measures, including customer satisfaction 7. Record keeping and reporting, including participant tracking 8. Determination Appropriateness of Adults and Dislocated Workers in Need of Intensive and Training Services 9. Next Generation Career Center (NGCC) services and model system 10. Orientation to services to include participant rights under complaint and grievance procedures 11. Implementation of NGCC minimum standards 12. NGCC Membership/Assessment 13. Labor Exchange 14. NGCC Products and Service Usage to include workshops WIA Adult and Dislocated Worker Rvsd 10-25-12 2

Classroom training On the Job Training Short Term pre-vocational services Support Services 15. WIA Youth 16. EO Observation WIA Title I-B Staff Interview WIA Title I-B program staff shall be interviewed at least once during each program year. The interview shall use a standardized form which may be customized to meet special contract provisions of individual sub-contractors. At a minimum, the interview will inquire about the following processes and systems: 1. Participant and employer outreach, including compliance with the EEO provisions of the WIA; 2. Intake and eligibility determination, including compliance with the EEO provisions of WIA; 3. Method of determining an individuals advancement from staff assisted core to intensive to training services; 4. Assessment and Individual Employment Plan (IEP)/Individual Service Strategy (ISS) development; 5. Participant tracking; 6. Time limits warning system (if applicable); 7. Case management and linkages; 8. Job development and placement; 9. Complaint and grievance response system; and 10. Exit and Follow-up. Personnel Systems The NEMO Workforce Investment Board monitoring staff may review one or more personnel records of individuals paid through contracts with the NEMO WIB. The purpose of this review shall be to ascertain if program operator subcontractor staff: 1. Possess the education and experience noted in proposals submitted to the WIB; 2. Expend the time on contract related activities charged against the contracts being monitored; 3. Have documentation of capacity building efforts on their behalf; and 4. Have signed complaint and grievance procedures on file. Confidential information, such as disciplinary action and service reports are not considered relevant to normal monitoring and should be stored in a closed envelop within the personnel file. EEO and ADA Compliance NEMO WIB monitoring staff shall survey the program operator subcontractor s hiring practices, promotion practices, facilities, and outreach and intake procedures to ascertain if the program operator subcontractor is in compliance with the Equal Employment Opportunities and the Americans with Disabilities Acts. Financial Procedures Review At least once during each program year, the WIB financial monitoring staff shall review each program operator's financial procedures and systems to ensure fiscal integrity and compliance with Section 184(a)(4) of the WIA and Title 29 Part 95 (Institutions of Higher Education, Hospitals, and Other Non- Profit Organizations) or Title 29 Part 97 (State and Local Governments). In addition, funds intended to support adult, dislocated worker, youth, Next Generation Career Centers (NGCC), stand-alone summer youth programs or any other special initiatives will be monitored in accordance with contractual scopes of Rvsd 10-25-12 3

work. The monitoring shall be completed using a standardized monitoring instrument, which may be customized to meet special contract provisions of individual subcontractors. This review shall include a sampling, covering at least one month s activities, and shall consist of the following procedures: 1. tracing reports to books (contract progress reports to original books); 2. a computation of excess cash and review cash for interest earned on advances; 3. a review of trial balances and bank reconciliations; 4. an examination to assure that FDIC coverage and collateral security are adequate for cash on hand; 5. an examination of inventory records; 6. a review of bonding policy; 7. a check on internal controls and separation of duties; 8. an examination to determine if program income exists, is being accounted for and reported in accordance with applicable requirements; 9. a review and interview to ensure that staff is following the proper guide for determining costs applicable to WIA (OMB Circular) and WIA procurement guidelines; 10. a review of the cost allocation system; 11. a regular examination of expenditures to test transactions against cost categories, and cost limitations, and; and 12. to a review and certify certification that the procurement process is in compliance with the WIA procurement system guidelines; 13. to a determine determination if stand-in costs exist and are being accounted for; and 14. a review of records to ensure proper source documents exist. The monitoring will result in a written monitoring report, subject to review and approval by higher authority, other than the monitor. Any findings will result in required corrective action plans submitted to the Administrative Entity within 30 days after the program operator receives the written monitoring report from the Performance and Outcomes/Monitoring and Continuous Improvement Committee, and correction satisfactorily implemented within one year of the finding. Subsequent monitoring will investigate any prior findings and their resolution. Participant File Review At a minimum, participant records are to be reviewed for the following: Documentation of participant eligibility and/or priority for the program and services received Orientation to services Orientation of the participant to his/her rights under complaint and grievance procedures Justification for the provision of Intensive or Training services Method of assessment Employment planning Individual Training Accounts Appropriateness and accuracy of participant payments (support service and needs related) Appropriate date entry and posting of outcomes, including attainment of a degree or certificate and any supplemental employment data NEMO WIB monitors shall review the eligibility of a random sampling of participants. In addition to the eligibility determination, at least 10% of the files that are reviewed will be monitored for compliance with non-eligible related requirements and for the quality of training and services provided. Files will be monitored using a standardized Participant File Review Data form. The method for file review shall be as follows: 1. Eligibility Only. A. The Toolbox form (WIA Registration/Attestation form) shall be reviewed to determine: 1. Program Activity and/or Title of Funding, 2. Accuracy of Eligibility related entries. Rvsd 10-25-12 4

3. Completeness and consistency with documentation; and, 4. Validity of the information entered into Toolbox. 2. Total File Review: A. The Toolbox form (or WIA Registration/Attestation form) shall be further reviewed to determine: 1. the participants educational status and highest grade completed, 2. the participants reading and math levels, 3. appropriateness and accuracy of other entries data entered, and 4. documentation of any barriers to employment (if applicable) B. The Toolbox form (WIA Registration form) will be further reviewed for: 1. accuracy and completeness, 2. consistency of dates with dates on pay and/or attendance records, and 3. consistency of class/worksite information with relevant file documentation C. When applicable, the (WIA registration form) exit form will be reviewed for: 1. accuracy and completeness, 2. appropriateness of exit code, 3. consistency of exit date with dates of attendance and/or receipt of pay, and 4. timeliness and accuracy of follow-up information D. Complaint and grievance receipts will be reviewed for the participants signature and the date signed, E. I-9 forms will be reviewed for accuracy, and inclusion of documentation and appropriate signatures, as applicable, F. Individual Employment Plans/Individual Service Strategies will be reviewed for: 1. completeness, relative to the length of enrollment, 2. appropriateness of plan of action, relevant to goals and assessment, 3. evidence of regular updating, 4. incorporation of case management, and 5. signature of both participant and career consultant center staff G. IEP/ISS updates (service notes) will be reviewed for: 1. frequency of contact, 2. relevance to the IEP/ISS, and 3. documentation of participant progress 4. documentation of orientations 5. documentation of need of services, i.e., support services H. Participant time and attendance records and OJT invoices will be reviewed for: 1. accuracy of calculations, 2. reasonableness of entries, 3. evidence of on-site monitoring of OJT employers for contract compliance prior to initial payment of invoices, 4. evidence of payment, and 5. appropriate signatures I. Orientations will be reviewed for completeness and appropriate signatures; J. When applicable, OJT contracts will be reviewed for: 1. completeness, including training plan and assurances, 2. compliance with state and federal guidelines, 3. appropriateness of training, Rvsd 10-25-12 5

4. reasonableness of training time, 5. specificity of description of skills, and 6. appropriate signatures on both the contract and training plan K. When applicable, worksite agreements will be reviewed for inclusion of a participant training plan and appropriate signatures. L. When applicable, request for payment of training dollars will be reviewed for: 1. accuracy and completeness of information completed by referring office, 2. accurate computation of total cost, and 3. appropriate signatures and dates Subsidized Worksite-Training Site Reviews During each monitoring visit, a representative sampling of active or recently active work and/or training sites will be visited. The sampling shall, whenever possible, represent all of the activities in which participants have been served. Employers, supervisors and/or instructors will be interviewed using the appropriate form. The monitor will also use these forms to review health and safety standards at the work/training site and to make such other comments as may be relevant to the site review. WIB staff will monitor a minimum of 10% of OJT employers who have contracted with the program operator subcontractor. OJT employer's payroll and training system records will be monitored to ensure on-going integrity and compliance with DWD standards. Participant Interviews During each work and/or training site review, a sample of participants will be interviewed as well. A sampling of participants should be interviewed at each site where one or more participants are present. Scheduling of such interviews should, whenever possible, take into consideration the needs and convenience of the worksite, employer and/or educational institution. Monitoring and Oversight MJDF/CUSTOMIZED TRAINING 1. Monitoring of the MJDF OJT Program is part of the contracting agency s oversight responsibility. The duty may be assigned to the Administrative Entity or contracted (with proper procurement) to another entity or individual. Monitoring and oversight must include participant training, PARTICIPANT ELIGIBILITY, and corresponding employer payroll records. Monitoring of MJDF OJT contracts should also be used to effectively ensure that the employer is providing the training specified by the contract. The AE representative or subcontractor will monitor the project s payroll documentation no later than halfway through the contract period. If this duty is performed by the subcontract, the AE must ensure that the subcontractor fulfills this requirement. There are distinct differences between the Workforce Investment Area s (WIA s) implementation of the MCTP and the Workforce Investment Area s implementation of the WIA program. The local WIA s are allowed to develop stricter requirements under the WIA program. However, the WIA is under contract with DWD to implement the MCTP, and is therefore required to follow the guidelines set forth by DWD. Because the MCTP is an economic development program with the primary purpose of providing a service for Missouri employers, it is important that documentation and monitoring requirements be Rvsd 10-25-12 6

kept to a minimum. Therefore, for all MJDF OJT projects, the AE s may not impose stricter requirements than the MJDF OJT guidelines outlined, without written approval from DWD. 2. MJDF Monitoring of Payroll Documentation For MCTP projects funded through MJDF, the AE will monitor OJT payroll documentation by completing the requirements in sections A, B, or C below, depending on the applicable situation. A. If the AE operates OJT internally and writes the OJT contract directly with the employer, the AE must monitor payroll documentation by completing the following requirements. (1) Randomly select a sample of OJT participants whose payroll documentation is to be monitored. (For projects with 49 or less trainees, 20% must be sampled. For 50 or more trainees, 10% must be sampled.) For each participant sampled, the following documentation must be reviewed through an on-site visit: (a) Time and attendance records used to prepare the employer s payroll; (b) A copy of the payroll register, pay record, or check stub which shows hours worked and rate of pay for each OJT participant SAMPLED (if the employer cannot provide any of these items, the attached OJT invoice may be used as acceptable documentation.); and (c) A copy of the personnel action which show official hire date, if the employer uses such a document. (2) Complete the appropriate monitoring documentation. When payroll documentation is reviewed on-site, documentation of the monitoring shall include: (a) A formal report stating employer name, location of records, date examined, monitor name(s), and if any discrepancies are noted, the corrective action required; (b) Work paper showing evidence of items checked (a copy of the forms submitted with a checkmark by each item is sufficient), which should be initialed and dated by the monitor; and (c) Evidence that any discrepancies noted was sufficiently explained and/or repaid, as appropriate. B. If an AE contracts with a broker to operate and/or monitor OJT, the AE must: (1) Require the broker to implement all the requirements outlined in sections A (1) and (2) above; (2) Monitor OJT brokers at least annually as required by the financial manual, including a review of the broker s implementation of a system required under sections A (1) and 2 above; and (3) Prepare a written report on all monitoring activities and require explanation and/or repayment for any deficiencies noted. (The report should include correspondence to and from the broker and the employer regarding the monitoring.) C. For OJT participants whose payroll records are located outside the WIA s boundaries, the AE or broker should conduct the following review instead of the procedures outlined in section A or B above. The out of area review shall consist of ALL of the following: Rvsd 10-25-12 7

(1) The following documentation must be submitted to the AE or broker for a sample of the OJT participants (20% sample for 49 or less trainees, 10% sample for 50 or more trainees): (a) A copy of the time sheet or time card showing hours worked; (b) Copies of the payroll register, pay record, or check stub, which shows computation of the gross wages and net wages paid. (NOTE: Computation must agree to hours worked as shown on the required time sheet or time card described above. If the employer cannot provide any of these items, the attached OJT invoice may be used as acceptable documentation); and (c) A copy of the personnel action which shows official hire date, if the employer uses such a document. 3. Conduct a desk audit of all documentation submitted by the employer, including normal mathematical computations and comparison of endorsements, if any, to participant s file. Monitoring documentation should include evidence that nay discrepancies noted were either sufficiently explained and/or repaid, if appropriate. 4. An AE representative or broker should conduct a face to face interview with a randomly selected sample of the MJDF OJT participants to verify the hourly wage, approximately how many hours were worked per pay period, the start date at the company and confirm that the participant is being paid in accordance with the contract. 5. The AE representative or broker conducting the interview in (3) above shall prepare a written report noting the time, place, and information used to verify the OJT reimbursements. IN REVIEWING PAYROLL DOCUMENTATION AS STATED IN ABOVE SECTIONS A, B, OR C, SHOULD THE AE DISCOVER DISCREPANCIES AT A WORKSITE BETWEEN AN EMPLOYER S PAYROLL RECORDS AND OJT INVOICES SUBMITTED, THE AE SHOULD CONTACT DWD IMMEDIATELY. Project Start Dates OJT Contracts will be written in accordance with the project start date approved by DWD. No participant activity is authorized prior to the project start date. (NOTE: The project start date is shown on the General Project Summary. The date of the employer approval date letter is NOT the project start date or approval date.) Exit Conference Upon completion of the on-site monitoring activities, the lead monitor and/or monitoring team will conduct an Exit Conference for the WIA Title I-B Operator and/or Designated One-Stop Operator of the Career Center being monitored or his/her representative. A copy of work papers including administrative reviews, file review forms and worksite/training site review forms will be given to the appropriate entity. At the discretion of the lead monitor, information, which may identify specific individuals who have been interviewed and who have made accusations of mismanagement, misfeasance or malfeasance, may be withheld pending an in-depth investigation of the accusations. The Exit Conference shall inform the agency of the areas which were covered during the monitoring, monitoring findings and (if formulated) recommendations relating to the findings. If disallowable costs have been identified, the program operator subcontractor will be made aware of them during the Exit Conference. POST-MONITORING ACTIVITIES Rvsd 10-25-12 8

Monitoring Report The monitoring team shall produce a written monitoring report for each program operator subcontractor, which will outline at a minimum: the activities which were monitored; the systems which were monitored; individual participant files which were found to contain errors; Adequacy of assessment Planning of Activities and services Coordination with One-Stop system partners to meet the comprehensive needs of customers Customer outcomes a summary of the outcomes from the worksites and training sites which were visited; summary of centers progress towards integrating the NGCC model; monitoring findings; and recommended corrective action As this report will be considered a public document, names of participants will not appear in the report. Participants will be identified by file numbers assigned by the monitors during the monitoring or by their worksheet or training site. The monitoring report shall be reviewed and approved by the Executive Director of the NEMO Workforce Investment Board. The Executive Director will prepare a cover letter to the program operator subcontractor, which will indicate whether or not the program operator subcontractor is to respond to the monitoring report. Any findings will result in required corrective action plans submitted to the Administrative Entity within 30 days after the program operator subcontractor receives the written monitoring report. Copies of the monitoring report and the cover letter will be sent simultaneously to the program operator subcontractor and to members of the Performance and Outcomes/Monitoring and Continuous Improvement Committee. If the subcontractor s responses are unacceptable the WIB will respond within 30 days. The subcontractor will have 10 days to respond to this second request. Monitoring reports will be provided to the WIB and CLEOS on an annual basis. Response to Monitoring Reports When program operator subcontractor responses are received by the Administrative Entity, they will be reviewed and a memorandum to the Performance and Outcomes/Monitoring and Continuous Improvement Committee will be prepared outlining any questions, comments or further recommendations which NEMO Workforce Investment Board staff may have. The Executive Director will review the memorandum and sign it and forward it and the program operator subcontractor's responses to the Performance and Outcomes/Monitoring and Continuous Improvement Committee. Performance and Outcomes/Monitoring and Continuous Improvement Committee Action The Performance and Outcomes/Monitoring and Continuous Improvement Committee will review the monitoring report, the program operator subcontractors response and the Administrative Entity memorandum and may: Accept the program operator subcontractors response Reject all or part of the response and/or require resubmission of the response Require additional corrective action, or Recommend the institution of sanctions against the program operator subcontractor to the NEMO WIB. When, in the opinion of the Performance and Outcomes/Monitoring and Continuous Improvement Committee, all the monitoring findings are satisfied, the Committee shall request that a summary of the monitoring report be prepared by the NEMO WIB staff for presentation to the NEMO WIB. A Rvsd 10-25-12 9

recommendation that further monitoring activities be suspended for the program year in question, pending receipt of any additional information which might call for reactivation of monitoring, will also be submitted to the WIB. NEMO WIB Action The NEMO WIB may accept or reject the summary monitoring report, or may request additional information regarding all or part of the report. When the NEMO WIB is satisfied that the process has been satisfactorily completed, they will either accept the original, amended or substitute summary report, and move suspension of monitoring. If the Performance and Outcomes/Monitoring and Continuous Improvement Committee recommends sanction to the NEMO WIB, the WIB shall take up the recommendation at the next scheduled WIB meeting or the Chairperson may call a special meeting of the WIB to consider the recommendation. One-Stop Committee Action The One-Stop Committee shall also receive a summary of the final monitoring reports. The Committee may want to consider this information when reviewing eligibility for and recommending the selection of Title I Program operator subcontractors, One-Stop Operators and operators of other programs assigned to the Board, with the full WIB and CEO's. Rvsd 10-25-12 10