1
PROGRAM YEAR 29-21 REPORT ON PROGRAMMATIC MONITORING OF STATEWIDE WORKFORCE INTRODUCTION Florida s Strategic Plan for workforce development was collaboratively developed by Workforce Florida, Inc. (WFI) and the Agency for Workforce Innovation (AWI). The plan communicates the State s goals, objectives and strategies for the workforce development system. Monitoring and Oversight is a critical part of realizing this vision and accomplishing the State s workforce development goals. Systematic on-going monitoring is the primary vehicle for identifying the status of workforce program operations and practices, and plays a crucial role in ensuring a return on investment, system accountability, and continuous improvement. To meet federal and State oversight requirements, AWI in consultation with WFI, annually develops and implements a process for monitoring Regional Workforce Development Boards (RWB) and for follow-up of findings which require corrective action. The State s monitoring and oversight system must ensure that program outcomes and standards meet the objectives of federal and State laws. As such, local boards must be monitored annually for compliance with applicable laws and regulations in accordance with the State s monitoring system. As in the past, the State continues to provide regular oversight and monitoring of its statewide workforce programs and projects. MONITORING OVERVIEW Oversight, monitoring and reporting are basic tenets of federal and State grant programs. For the past four years, annual programmatic monitoring of workforce programs has been done by AWI staff in the Office of Workforce Services, One-Stop and Program Support. In program year 29, the American Recovery and Reinvestment Act (ARRA) was enacted, providing states with additional funding to provide expanded services to individuals affected by the downturn in the economy. As a result, the scope of program monitoring changed. Each of the 24 RWBs are now monitored on-site. Additionally, the methodology for selecting participant samples also changed which increased the number of participant case files reviewed. This report primarily covers Program Year (PY) 29-1. However, the results of the monitoring reviews over the past three years are also included for comparison purposes. For PY 29-21 (July 29 through June 21), the activities and services of all 24 RWBs and a number of special projects and contracts, including the summer youth program, were reviewed. This report describes how program monitoring is organized, highlights programs reviewed, and concludes with monitoring results. This summary report also focuses on some of the monitoring issues observed. 2
PURPOSE AND SCOPE The monitoring reviews are intended to provide an assessment of the 24 RWBs compliance with respective laws, regulations, State plans, and contract or agreement terms. Program operations and processes are also reviewed to identify factors that may impact overall service delivery and quality. The purpose and objectives of the State s programmatic monitoring reviews include, but are not limited to, the following: ensure the integrity of federal programs by reviewing performance, assessing compliance with applicable laws, regulations and policies, and identifying successful methods and practices that serve to enhance the workforce system as a whole through continuous monitoring and improvement; ensure programs, processes, and systems are aligned with State and local workforce objectives and priorities; increase program accountability, efficiency and effectiveness by providing a system of early detection of fraud and abuse; ensure contractual compliance and adherence to program goals and objectives; identify and correct any actual or potential problems, deficiencies or inaccuracies that may lead to noncompliance; and provide technical assistance and support to program staff, when needed, to help prevent problems before they occur and promote the RWBs ability to meet higher standards. MONITORING STRATEGY The strategy for programmatic monitoring includes three basic dimensions: a compliance and process dimension (assessing compliance with rules, regulations, and standard operating policies and procedures); a technical assistance dimension that helps prevent problems before they occur; and a program dimension which highlights any notable observations and/or best practices that may have been implemented by the regions. MONITORED Overview of the major programs that were monitored: Workforce Investment Act (WIA): The WIA is designed to consolidate, coordinate, and improve employment, training, literacy, and vocational rehabilitation programs to empower individuals to obtain the services and skills they want and need to overcome barriers to employment. The WIA program provides services to three categories of individuals: Adults, Dislocated Workers, and Youth. o Adults: The Adult Program provides employment and training services to help individuals 18 years of age and older find and qualify for meaningful employment. Employment and training services include job search, referral and placement services, skill evaluation and assistance, job training, etc. o Dislocated Workers: The Dislocated Worker Program is designed to provide employment and training services to individuals who have been terminated or laid off from their job and are eligible for or have exhausted their employment benefits. o Youth: Individuals 14 to 21 years of age who are low income and face one or more barriers to employment. 3
Wagner-Peyser (WP) - including Veterans and Migrant and Seasonal Farm Workers: Wagner-Peyser is a labor exchange program that brings together individuals who are seeking employment and employers who are seeking employees. The State administered labor exchange system must have the capacity to assist job seekers to find employment, to assist employers in filling jobs, to facilitate the match between job seekers and employers, and to participate in a system for clearing labor between the states, including the use of standardized classification systems to meet the work test requirement of the State s Unemployment Compensation (UC) system. Welfare Transition (WT)/Temporary Assistance to Needy Families (TANF): The WT/TANF block grant program provides time-limited cash assistance to needy families in exchange for participation in work activities designed to move the family towards selfsufficiency. Flexibility in the legislation allows TANF funds to be utilized creatively to serve families that meet TANF purposes. Food Stamp Employment and Training (FSET) Program: The FSET Program emphasizes work, selfsufficiency, and personal responsibility. The program strives to meet the needs of participants in gaining skills, training, work, and experience that will increase the program participant s ability to obtain total self-sufficiency. Trade Adjustment Assistance (TAA): The TAA program helps trade-affected workers who have lost their jobs as a result of increased imports or shifts in production out of the United States. Certified individuals may be eligible to receive one or more program benefits and services depending on what is needed to return them to employment. Special Projects Grants/Contracts: Special projects included the Displaced Homemaker Program (DHP), WIRED, Tropical Storms, Banner Centers, Youth, PASSPORT, etc. Summer Youth Employment Program (SYEP): Additional ARRA funds were provided to states to offer and create expanded summer employment opportunities for youth. Each RWB offered training and employment opportunities for youth in varying disciplines. AUTHORITY The reviews are conducted under the authority of federal and State laws and regulations including, but not limited to, the following: Workforce Investment Act (WIA) of 1998 {184(a)(4) and 184(d)(2)(a)} 2 CFR: 66.3, 667.4, 667.41, and 667.5 Wagner-Peyser Act of 1933 (as amended) and 2 CFR 652 Personal Responsibility and Work Opportunity Reconciliation Act of 1996 Florida Statutes, Chapters 414 and 445 Food Stamp Act of 1977 (as amended); 7 CFR, Parts 273.7 and 273.24 Temporary Assistance to Needy Families (TANF) Regulations; 45 CFR 26 Work Verification Plan Florida Administrative Code (65A-4) Trade Act of 1974 (as amended) OMB circulars and other applicable authoritative guidance Contract and Agreement Terms. 4
METHODOLOGY Program Monitors verify and validate data through a number of sources. The methodology for our monitoring reviews includes sample testing of participant records for program compliance requirements. A statistically valid sample of files to be reviewed is electronically generated from the State s Management Information System (MIS) by AWI s Performance, Reporting and Analysis, Technical Assistance and Oversight Unit. The monitoring process is accomplished through a combination of desk and on-site reviews. The desk reviews are used to collect and analyze existing information (reports, system data, policies and procedures, local plans, contracts, etc.) that is needed to support the on-site reviews. The on-site analysis process allows the Monitors to verify program data collected during the desk review by reviewing hard copy (paper) case files kept at the local One-Stop Career Centers. The hard copy files are checked to determine whether adequate documentation is maintained. These files are also matched against MIS records to ensure that data has been correctly entered in the systems. Reporting data is also reviewed and validated by checking the accuracy of a sample of computerized records and comparing keyed entries made by the RWB against original source documents. Additional on site reviews may be conducted on an as needed basis dependant on the issues identified and follow-up requirements. MONITORING PARAMETERS After conducting a monitoring review, a written report is issued to appropriate parties which addresses the results of the monitoring review, as well as recommendations and suggestions on how to address any identified issue. The results of the monitoring review are presented in the report as findings, systemic issues, observations, and notable program practices. Programmatic findings are instances where noncompliance with requirements contained in federal or State law, regulation, administrative code, guidance or other document is found. Systemic issues are elements found within program operations that may lead to a finding or other reporting/performance issue if not corrected. Observations are informative statements and are made to identify processes that, when improved, can result in positive program outcomes. Observations are also made to recognize and promote notable program practices and processes in the regions. Notable program practices are informative statements that highlight and recognize program improvements that promote positive program practices and outcomes. 5
OVERVIEW OF STEPS INVOLVED IN THE ON-SITE MONITORING PROCESS MONITORING COMPONENTS Planning/Desk Review On-Site Visit Field Work Updates/Technical Assistance Exit Conference/Wrap-up and Analysis Final Report Corrective Action Plan Closeout DESCRIPTION Conduct desk review of all relevant information and documentation (i.e., State plans, reports, policies & procedures, contracts, MIS records, prior reports, etc.). This is done to verify that RWBs are administering programs in accordance with federal requirements and their own local operating procedures. Conduct entrance conference to discuss oversight responsibilities, purpose and scope of review, establish timelines for field work, daily briefings, site visits, contacts, etc. Review participant hard files and test against data entered in State s information systems and applicable laws, regulations and other directives. Conduct daily updates to keep RWBs informed of the review s progress. Also provide technical assistance to assist regions in establishing updated processes. Exit conference (official overarching summary of the review including a written summary of issues identified). Prepare and issue written report to RWB based on results from the monitoring visit. If a finding/deficiency is noted in the report, the RWB must develop a corrective action plan (CAP). All issues must be resolved. Issue closeout letter to RWB after all findings have been resolved. All issues must be resolved before report is closed out. TECHNICAL ASSISTANCE-WORKING TOGETHER TO BRIDGE THE GAP As previously mentioned in this report, one of the strategies for continuous improvement is the provision of training and technical assistance while the Monitors are on-site. While specific training is limited, technical assistance is often provided. AWI s philosophy is that programmatic monitoring should not only include a compliance dimension, but should also include a technical assistance dimension that helps promote the RWB s ability to meet higher standards and prevent problems before they occur. During the course of the on-site reviews, technical assistance is provided when and where necessary. In some instances, it is provided to local program leads and in other instances, it is provided directly to case managers, upon request by the RWB. Types of technical assistance provided included, but were not limited to, the following: o MIS input and retrieval and what type(s) of data should be entered; o documentation requirements for completing initial assessments, Individual Responsibility Plans (IRP), recording of participation hours, and sanctions; o participant eligibility criteria and sources of documentation that should be obtained; and o program operations and processes to determine if there are factors that impact overall service delivery and quality and how they can be improved; etc. 6
MONITORING COMPLIANCE ISSUES This report is not a comprehensive assessment of all areas of the State s monitoring system (e.g., fiscal, performance, audit, etc.). Any deficiencies cited in a fiscal review or audit would be detailed separately and the RWB would need to comply with those authoritative requirements. The overall results of the review of workforce service delivery programs, practices and systems operated in each of the regions indicated that all RWBs appeared generally to be in compliance with established federal and State laws, policies, and procedures. However, some deficiencies in operational, system practices, and case file documentation requirements were observed. FAST FACTS A few selected facts and figures for PY 29-1 include the following: 24 RWB programmatic monitoring reviews were conducted. 24 summer youth programs were monitored. 342 findings were identified as a result of the reviews. 299 systemic issues were observed. 278 observations were noted. The total number of findings increased by 115.1 percent over due to a number of factors as outlined on page 21 of this report. 7
SUMMARY OF ISSUES NOTED BY PROGRAM Number of Number of Systemic Issues Number of Observations Welfare Transition/TANF 143 65 112 Wagner-Peyser 115 83 12 Food Stamp Employment and Training Program 46 11 5 Workforce Investment Act - Adult and Dislocated Worker 15 81 16 Trade Adjustment Assistance 1 8 73 One-Stop Credentialing 8 12 Workforce Investment Act - Older and Younger Youth 5 51 3 TOTAL 342 299 278 Issues Noted by Program PY 29-21 Number of Number of Systemic Issues Number of Observations 16 143 14 12 112 115 1 8 65 83 81 73 6 46 5 51 4 2 Welfare Transition/TANF 12 Wagner-Peyser 11 Food Stamp Employment and Training Program 15 16 Workforce Investment Act - Adult and Dislocated Worker 1 8 8 12 5 Trade Adjustment Assistance One Stop Credentialing 3 Workforce Investment Act - Older and Younger Youth 8
DETAILED FINDINGS BY PROGRAM AND YEAR Below is a summary, by major program, of the types of findings cited during the monitoring reviews and the number of Regions to which the findings were applicable. The chart includes monitoring results for the past three-year monitoring cycles. It should be noted that is the first year each region was monitored on site. WELFARE TRANSITION/TANF PROGRAM WELFARE TRANSITION /TANF ASSESSMENTS / INDIVIDUAL RESPONSIBILIIY PLAN (IRP) Documentation of Assessment: Initial assessments not completed within 3 days of the participant becoming eligible for cash assistance. Initial assessments did not include the required components (skills, work history, and employability). 24 22 47 JOB PARTICIPATION RATES (JPR) DEFERRALS PRE-PENALTIES and SANCTIONS Signed copies of the initial or updated IRP not retained in the participant case files. Documentation of Participation: Documentation to support JPR hours reported to Health and Human Services not retained in participant case files. Documentation of Deferral for Medical Reasons: Documentation not signed by a licensed physician and retained in the participant's case file to verify entering a medical deferral in the One-Stop Service Tracking (OSST) System. The medical deferral entered in the OSST system was entered before the medical documentation was received. Documentation of Sanction Process: Oral attempt not made to contact participants regarding the pre-penalty within ten days of the date of failure. The Failure to Demonstrate Satisfactory Compliance, Form AWI WTP 2292, not mailed when sanction was requested as a result of two failures within a 3-day period. 1 1 24 4 7 18 36 18 28 9
WELFARE TRANSITION / TANF PROGRAM Continued WELFARE TRANSITION / TANF Continued TRANSITIONAL CASES: DIVERSION Transitional Cases: Transitional services not appropriately provided based on transitional eligibility requirements. Up-Front Diversion: The Up-Front Diversion Screening form not completed and signed by appropriate parties prior to being approved for up-front diversion payment/service. 8 6 15 1 1 RECORD RETENTION Record Retention: Participant records not appropriately retained on file. 1 2 2 SPECIAL PROJECTS RELOCATION ASSISTANCE Special Projects: Participants not determined eligible before being enrolled in TANF funded projects, and no written Local Operating Procedures (LOP) developed for operating local projects. Relocation Process: Participants provided relocation assistance were missing transfer letters, 9-day program follow-ups, and did not meet eligibility requirements. 1 1 7 WELFARE TRANSITION/TANF TOTAL 83 67 143 16 14 12 1 8 6 4 2 Welfare Transition / TANF Trend 143 83 67 1
WORKFORCE INVESTMENT ACT PROGRAM WIA ADULT/ DISLOCATED WORKER PROGRAM ELIGIBILITY REQUIREMENTS Documentation of US Citizenship: The region did not comply with the requirement to maintain documentation of a participant's citizenship status. Veteran Status: Documentation of participant's veteran status not maintained. 1 4 3 Age: No documentation that participant met adult age requirement. 1 2 Dislocated Worker Program eligibility: No documentation of a participant's Dislocated Worker program eligibility status. Low-income Determination: The region did not to comply with the requirement to maintain documentation to support the participant's low-income determination. 1 2 3 Selective Service Registration: The region did not comply with the requirement to maintain documentation of an eligible male s selective service registration. Customized Training: Files did not contain executed agreements with employers. Participant Case File Retention: Participant case files were missing and could not be located. 3 1 1 1 WIA ADULT/ DISLOCATED WORKER TOTAL 6 2 15 WIA Adult and Dislocated Worker Trend 2 15 15 1 5 6 2 11
WORKFORCE INVESTMENT ACT PROGRAM Continued WIA YOUTH PROGRAM ELIGIBILITY Signature of Parent or Legal Guardian: The WIA applications for youth ages 14-17 were not signed by a parent or guardian as required by federal and State laws. 1 1 1 Documentation of Selected Barrier: Case files did not include documentation of a federal or approved local RWB barrier when the first WIA service was provided. 1 1 1 Participant Case File Retention: Participant case files were missing and could not be located. 1 Documentation of Low-Income Calculation: Files were missing documentation showing the family s total earnings that were used for determining low-income. 2 2 WIA YOUTH TOTAL 4 2 5 5 WIA Youth Trend 5 4 4 3 2 2 1 12
TRADE ADJUSTMENT ACT PROGRAM TRADE ADJUSTMENT ACT Trade Adjustment Act Deobligation Form: Region was not in compliance with federal requirement to complete and retain a copy of the TAA Deobligation form in the case file for participants that quit the TAA training program. Enrollment in WIA: TAA participants were enrolled in WIA to receive Core and Intensive services prior to enrollment in the TAA training program. 1 2 5 2 Six TAA Program Criteria: Region was not in compliance with federal requirement to determine that participants met all program criteria; and a copy of the assessment tool was not retained in the case file. 3 Documentation of Job Seeker status: Region was not in compliance with requirement to maintain documentation that a worker was a job seeker. 1 TRADE ADJUSTMENT ACT TOTAL 1 3 1 Trade Adjustment Act Trend 1 8 6 4 2 1 3 1 13
WAGNER PEYSER PROGRAM WAGNER PEYSER Job Placement Job Order: Job orders showed the job seeker's hire date was prior to the job seeker s job development contact/referral date to the employer. Also, the individual was not pre-selected for the job prior to referral. Job Placement-JobSeekers: The job seeker's hire date was prior to the job seeker s job development contact/referral date to the employer. Job Development Job Order: The RWB s process did not comply with federal law that requires a job development job order may only be written to take credit for a placement of a specific job seeker who has been hired by an employer as a result of a job development contact/referral for a specific job with the employer. Job Development Job Seeker: The RWB s process did not comply with federal law for processing job developments for job seekers. The job development service is given if there is no suitable opening in the labor exchange system by staff contacting an employer on behalf of the job seeker. 5 11 7 1 8 8 17 19 13 2 7 2 Staff referral: The state requires staff to refer only qualified individuals to job orders. The RWB referred individuals who did not qualify for the job. 6 Veteran Services: The veteran did not receive services when registered by staff as required by state policy. 1 1 H-2A Required I-9: The region did not comply with federal requirements that H-2A job order staff complete an I-9 and a 516 INS form prior to referral of a job seeker to an H-2A job. The original I-9s and a copy of the 516 INS forms are to be retained in the issuing office and are valid for a three-year period. 2 8 14
WAGNER PEYSER PROGRAM Continued WAGNER PEYSER Continued I-9/516 INS: The region did not comply with federal requirements that state employment agency staff complete an I-9 and a 516 INS form prior to referral of a job seeker when requested by the employer. The original I-9s and a copy of the 516 INS forms are to be retained in the issuing office and are valid for a three-year period. O*NET Code: The state requires that a correct O*NET code be listed on a job order. The region did not use appropriate codes. Priority Re-employment Planning Program (PREP) Assessments: The region did not comply with the State's requirement that a personal assessment be conducted on PREP program participants. Migrant and Seasonal Farm Worker (MSFW) Application: The MSFW application was missing federally required information such as type of work preferred, other experience or training, crop code, etc. Agriculture Job Orders: Federally required information such as period of time, days/hours, specific method of working, specific rate if paid by piece, etc., was missing from the job order. Re-employment and Eligibility Assessments (REA): The region did not comply with federal law that requires REA participants be given an assessment to develop an Employability Development Plan. Counseling/Group Counseling: A counseling service was not documented as required by state guidance. Employability Development Plan (EDP): State guidance requires certain criteria be included on an EDP. The region did not properly complete the EDP for the REA Program. REA Orientation: Orientation was not provided to REA participants as required. 2 7 1 6 8 3 8 11 2 2 4 1 2 13 7 1 15
WAGNER PEYSER PROGRAM Continued WAGNER PEYSER Continued Recording PREP Services: Services were not being entered on the OSMIS work log as required for federal reporting. Federal Bonding: The state requires that credit for a bonding service can only be taken when staff actually writes a bond for an individual. No bonding form was on file to support the bonding service taken. A copy of the form must be kept on file for five years. Job Order Equal Employment Opportunity (EEO) Compliance: The job orders listed a minimum age or required the applicant to have his/her own tools, which was not in compliance with federal discrimination laws regarding age and job advertisement. 1 3 4 4 11 Personal resume assistance was not provided to participant as required. 1 WAGNER PEYSER TOTAL 37 68 115 Wagner Peyser Trend 12 115 1 8 6 4 2 37 68 16
WAGNER PEYSER PROGRAM Continued CREDENTIALING WAGNER PEYSER Tier I Certification: The RWB did not comply with the requirement that frontline staff complete Tier 1 certification within six months of their hire date. Continuing Education: The RWB did not comply with the requirement that front-line staff who complete Tier 1 certification must have 15 hours of continuing education annually. 1 5 6 2 2 WAGNER PEYSER CREDENTIALING TOTAL 1 7 8 Wagner Peyser Credential Trend 8 7 6 5 4 3 7 8 2 1 1 17
WAGNER PEYSER PROGRAM Continued WAGNER PEYSER COMPLAINT RESOLUTION Wagner-Peyser Complaint Resolution System: Federal regulations require a specific process be followed when a One-Stop Center receives a complaint. The RWB did not follow the required process. 1 1 WAGNER PEYSER COMPLAINT RESOLUTION TOTAL 1 1 FOOD STAMP EMPLOYMENT AND TRAINING PROGRAM FOOD STAMP EMPLOYMENT AND TRAINING Conciliation Procedures not Initiated: Conciliation procedures were not initiated when the participant did not comply with program requirements. Sanction Warranted but Not Requested: Sanctions were not initiated when conciliation procedures were exhausted. 7 4 7 2 3 4 Missing Case Files: Participant case files could not be located. 1 2 Assignment of Hours: Hours assigned to participants were not equal to the household allotment of food stamps. 3 17 Food Stamp Reimbursement (FSR): FSRs exceeded the maximum rate; documentation was missing; or individual was not eligible. 7 Program Participation: Participants not placed in a countable activity by the 31 st day after referral. 3 2 9 FOOD STAMP EMPLOYMENT AND TRAINING TOTAL 16 9 46 The FSET program became a voluntary program effective July 1, 29. However, participant case files that were managed under both the mandatory and voluntary programs were reviewed during the review period. 18
Food Stamp Employment and Training Trend 5 45 4 35 3 25 2 15 1 5 46 16 9 SUMMER YOUTH EMPLOYMENT AND TRAINING PROGRAM SUMMER YOUTH EMPLOYMENT AND TRAINING Insufficient Case file Documentation: Lack of sufficient documentation to support program participant eligibility (selected barriers, selective service registration for older youth, etc.). Some income information calculated incorrectly or only supported by partial documentation. Insufficient monitoring of worksites. Pre and post-assessment information not sufficiently documented. N/A N/A Various findings resolved while Monitors were on site. Other related findings in all programs were observed during the course of the monitoring visits; however, those issues were resolved either while the Monitors were on-site or during the ten-day follow-up response period. Therefore, those findings were identified as general comments in the reports and are not reflected in the above tables. 19
SUMMARY OF FINDINGS BY PROGRAM AND YEAR The following chart compares the total number of findings, by year, based on results from the monitoring reviews. PROGRAM WELFARE TRANSITION 83 67 143 WAGNER PEYSER 37 68 115 FOOD STAMP EMPLOYMENT AND TRAINING 16 9 46 WIA ADULT / DISLOCATED WORKER 6 2 15 WIA YOUTH 4 2 5 TRADE ADJUSTMENT ACT 1 3 1 ONE STOP CREDENTIALING 1 7 8 TOTAL FINDINGS 149 159 342 by Program 16 14 12 1 8 6 4 2 143 83 67 68 37 115 WT / TANF WP FSET WIA Adult and Dislocated Worker 16 46 9 6 15 2 4 2 5 1 3 1 1 7 8 WIA Youth TAA ONE STOP 4 3 2 1 by Year 149 159 342 2
MONITORING TRENDS The compliance tables above represent monitoring program results for the past three years. As noted, the number of findings increased during in almost every program and category. However, there was a slight decrease in the number of findings between and in some programs. In, the total number of findings increased by 6.7 percent, compared to a 115.1 percent increase in 21. Although the rate increased significantly, the increase is attributed to a number of factors. The three-year monitoring trend analysis should be evaluated in varying degrees, meaning every finding should not be compared based simply on the same sample selection criteria and review results from year to year. Because the methodology for monitoring includes random sampling of participant case file records for program compliance requirements, the number of participant files selected and the number of instances the problem occurred for a specific element would not be the same from year to year. For example, out of 2 random job orders reviewed, there may have been ten that had a job placement. In another year, out of 2 random job orders selected, only one job placement may have been applicable to the 2 participant case files sampled. Additionally, the methodology for selecting the sample size may vary (increase) from year to year, monitoring tools are updated annually with new data elements, and different findings occur from year to year. Also, a desktop review may have been conducted in one year and an on-site review may have been conducted the following year. The use of alternating on-site and desktop review approaches along with other risk factors further limits the ability to draw sound conclusions. Although these factors do not diminish the evaluation of the findings and recommendations on an annual basis, they should, however, be considered when assessing or analyzing multi-year results. The monitoring reviews and reports showed some improvement in certain areas and a slight decrease in others. For example, a monitoring report for showed that tremendous strides had been made in an RWB s effort to improve the initial assessment and Individual Responsibility Plan (IRP) processes in the Welfare Transition (WT) Program, and that the remaining issues could be corrected with refresher training. Another report noted that an issue in the WT Program was an isolated incident and training was not recommended. Although a finding was cited, the number of instances drastically reduced. CONCLUSION As noted in the above tables and charts, we found a number of instances of noncompliance with statutory, regulatory, and/or policy requirements in several program areas, but none that would substantially or materially affect program operations. In addition to the findings observed, we identified other conditions that may become compliance issues if not addressed. We further identified practices and processes in program operations that could be improved. We also noted that some improvements in program processes had been made which reduced the number of instances the problems occurred. The reviews also found opportunities to strengthen the RWBs compliance efforts. Since no actions were observed that would materially affect program operations or result in questionable and/or disallowed costs, no Performance Improvement Plans (PIP) were recommended as a result of program monitoring. However, some training may be necessary based on the degree of problems consistently observed in the regions. 21