2011 2013 Audit Plan Marlene Hartinger, Chief of Audit Services Auditors: Nancy McDaniel Johnny Alexander Julie Ratcliff Sarah Myers Approved October 21, 2011
Table of Contents Overview... 1 Audit Budget Full Staff... 2 2011 2013 Audit Plan with Full Staff... 3 Audit Risk Assessment Methodology... 4 Review Previously Identified Risks... 7 Interviews... 8 Risk Categorization... 8 Risk Areas Not Included in Audit Plan... 9
Overview The descriptions and broad objectives in this plan are starting points for audits that we anticipate will be assigned in the 2011 2013 biennium. The topics are based on riskassessment discussions with ODOT staff and stakeholders as well as our judgment of the most effective deployment of audit resources. Once an audit is underway, it is possible that the scope and objectives will change during the survey phase as auditors seek updated information about risks and prepare detailed work plans. Audit priorities were discussed and identified during the May 5, 2011 Audit Committee meeting and are reflected in the revised audit plans and topics referred to management. 1
Audit Budget Full Staff The full staff audit budget was based on five auditors; we received approval to fill our two open auditor vacancies. To estimate the annual audit hours available it was assumed that each auditor spends 15 percent of their available time on administrative tasks and will take all accrued vacation and sick leave, 40 hours of CPE. We have factored in estimated furlough days as we anticipate that furloughs will continue in fiscal years 2012 and 2013. Together these nonaudit hours represent 35 percent of an auditor s time. This methodology also factors in anticipated long term leave and reduction of hours for an auditor and a delay in hiring a principal and senior auditor. After this calculation 5,600 and 6,400 hours are available for audit work in 2012 and 2013, respectively. Audit Plan Topics FYE 2012 FYE 2013 Carry Over from 2009-2011 Construction QA 700 OWIN Follow-Up 480 OWIN Payment Process 640 A&E Contract Administration 1,280 Change Orders Follow-Up 480 Information Classification 270 370 OTIA III Close-Out 480 Construction QA Follow-Up 480 Alternative Contracting 960 Document Management 960 Capping Report: Statement of Work Writing 200 Delegated Authority Follow-Up 480 External Reporting 640 Total Planned and Carry Over 3,850 4,890 Other Audit Work Management Requests 1,250 1,250 Risk Assessment 100 400 Routine Follow-Up 100 100 SPOTS Review 200 200 Oregon Peer Review 100 Total Other Audit Work 1,750 1,950 Total Estimated Hours 5,600 6,840 Available Hours 5,600 6,400 2
2011 2013 Audit Plan with Full Staff Audit Area Construction QA (Carry-Over from 2010) OWIN Payment Process Oregon Wireless Interoperability Network Site Acquisition Follow-Up Information Classification Description Construction Quality Assurance is carry-over work from 2011 and part two of report 10-03, Construction Quality Assurance: Structure Provides Checks and Balances but Improvements Would Strengthen Construction Oversight. The overall objective of the assignment is to assess whether ODOT's processes are adequate to ensure that the quality of materials incorporated in projects is verified before they are installed and paid for. This part will test a sample of completed projects to assess the effectiveness of ODOT's Construction Quality Assurance Program. OWIN management is designing a payment process specific to the CM/GC nature of the OWIN project and will not be using the MPB payment process. Using a payment sample, an audit of OWIN payment practices will: Review the payment processes to assess internal controls; Test compliance with payment policies and procedures; and Identify areas of improvement. Audit report 10-05, Oregon Wireless Interoperability Network: Controls Needed in Partnership and Site Acquisition Processes, found that controls over site acquisition and partnership processes were inadequate. When the final program budget is determined, we will perform a follow-up audit to assess progress at implementing audit recommendations. Oregon Statute and Department of Administrative Services Policy require all state agencies to classify information assets and protect them accordingly. Executive staff, management, and Audit Services identified personally identifiable information, information access, and inability to find information as risk areas. An audit of ODOT s Security Fabric, the implementation of statute and DAS policy, could address these information-related risks and potentially highlight other vulnerabilities. An audit of ODOT s information classification would include: Review of Security Fabric implementation; Estimated Hours 700 640 480 640 3
Change Orders Follow-Up OTIA III Close- Out A&E Contract Administration Assessment of compliance with state statute and DAS policy; Review controls and retreivability of all levels of information; and Identify areas of improvement. Report 08-04, System to Track and Analyze Change Orders Needs Oversight, found that while the efforts made to improve change order tracking and reporting are commendable, further improvements in data reliability and report sharing are necessary to make this information useful and meaningful to decision makers. We will perform a follow-up audit to assess progress at implementing audit recommendations. With the OTIA III program winding down and lack of finalized close-out program, audit staff is concerned with the adequacy of the final control over program costs. An audit of program close-out would include an assessment of close-out methodology, schedule and progress. The A&E contract administration was identified as a top risk by executive staff, upper management, audit staff, and stakeholders. Concerns involved: Contracting Methods Procurement Compliance Contract Quality Negotiation Documentation Deliverables Receipt Contract Costs Payment Appropriateness Performance Monitoring Using a sample selected from central and regional procurement, an audit of this area would include the following: Review payments and deliverables; Review consultant evaluations; Review regional and central contract administration guidance; Evaluate statewide consistency; Review best practices for contract administration; and Identify areas of improvement. 480 480 1,280 4
Construction Quality Assurance Follow-Up Alternative Contracting Document Management Capping Report: Statement of Work Writing Audit report 10-03, Construction Quality Assurance: Structure Provides Checks and Balances but Improvements Would Strengthen Construction Oversight, identified areas of improvement. This follow-up will review recommendation implementation status. With the expansion of alternative contracting on large scale projects and current project challenges, ODOT management and audit staff identified alternative contracting as a risk area. ODOT has used alternative contracting in the OTIA III program with mixed success and these contracting methods are being used for a local agency project. To audit ODOT s implementation of alternative CMGC contracting, we will: Review best practices and ODOT guidance; Evaluate the CMGC procurements and oversight processes of Willamette River Bridge, Transportation building remodel, Oregon Wireless Interoperability Network and ODOT s participation in the Sellwood Bridge. Identify lessons learned and areas of improvement. The Pioneer Mountain Eddyville project may be considered for a design-build lessons learned review. Coupled with information classification is the risk that these assets are not sufficiently protected or controlled in a redundant recovery method. An audit of document management would involve an enterprise assessment of information: Security Storage Costs Retreivability Long Term Accessibility Audit Services will recap the recommendations and implementation status of statement of work related findings in IGAs, construction, and A&E contracting. 480 960 1,280 200 5
Delegated Authority Follow-Up External Reporting Audit report 08-07, ODOT s IT Procurement Structure Lacks Effective Oversight, found that overall delegations of authority throughout ODOT do not comply with the Public Contracting Code. Further, report 10-02, Follow-Up Audit: Intergovernmental Agreements Lack of Authority Makes Agency-wide Change Unlikely at this Time, found that OPO completed an internal review of current delegations and determined that OPO did not possess the necessary oversight authority agency-wide for IGA standardization to be completely successful. The follow-up audit will review ODOT s efforts to reorganize, clarify, and improve the agencies delegation process. Externally reported data is manually compiled. This manual process can be inconsistent, prone to error, and time consuming. An audit of external reports would include: Identifying legislative reports; Assessing report preparation and reliability; Reviewing data sources; and Identifying areas of improvement. 480 640 6
Audit Risk Assessment Methodology The purpose of the biennial risk assessment is to create an audit plan based on an assessment of risks that have the potential to interfere with ODOT s ability to achieve its mission. To prepare the biennial audit plan we: Reviewed previously identified risks to identify specific and current risks; Interviewed ODOT management and staff; Interviewed external stakeholders; Reviewed results from the ODOT s enterprise risk management (ERM) approach; Identified specific audit areas and objectives; Specified audit resources; and Categorized and prioritized auditable risk areas. To identify relevant auditable risks we reviewed previously identified risks, interviewed 49 ODOT staff and stakeholders, and reviewed ERM results. Based on this information we prioritized risk areas to create an audit plan for the 2011-2013 biennium. Review Previously Identified Risks In reviewing past audit plans coverage, we found that we had not adequately addressed some previously identified risks. As a result, we reviewed these audit topics to determine if they were current risks and, if so, attempted to specifically define the risk area. These topic areas were: Bridge Contract Oversight Financial Data Integrity Human Resources Information Security OTIA III Procurement Technical Center Decentralization Revenue Safety Of the 10 previously identified risks, eight were still considered high risk areas. Due to additional research, we were able to specifically identify risk areas and potential audit objectives to ensure audit coverage of these areas in the future. Reoccurring risks, and specific areas, are listed below: Contract Oversight A&E Contract Administration and Construction Documentation Consistency Human Resources Succession Planning, Recruitment & Retention, Classification, and Discipline Information Security Balance Security and Productivity, ODOT s Security Fabric, and Protection of Personally Identifiable Information 7
Data Integrity Information Technology Business Planning and External Manual Reporting OTIA III Close-out Process Procurement Statement of Work Writing Revenue Funding Uncertainty Safety Employee Safety and Emergency Preparedness Interviews We interviewed 49 ODOT staff and stakeholders. ODOT staff included 11 executive staff, 17 managers, and 17 staff members. All division executives or administrators participated in the risk assessment either through interviews or written responses to questions. Further, all ODOT administrative functions and modes provided valuable input throughout the risk assessment process. Besides ODOT staff, we interviewed two Federal Highway Administration employees, a Department of Justice attorney, a Legislative Fiscal Office analyst, and an American Council of Engineering Companies representative. A gap in our stakeholder coverage was that we were unable to speak with a member of the Oregon Transportation Commission. Risk Categorization We tabulated the risks from the interviews, ERM, and audit brainstorming session into a list and grouped them by topic and risk levels of high and medium. We judged risk level as the likelihood that an event would occur and the severity of the effect if it did. We included as many of the high risks on our 2011-2013 Audit Plan as we have the resources to assess. The remaining topics were reported to the Audit Committee with the expectation that they will be communicated to management and that management will address the risks without audit assistance. 8
Risk Areas Not Included in Audit Plan Through the ERM process executive staff prioritized initiatives that were also identified as part of the audit risk assessment and these areas were not included in the audit plan. These executive priorities are: 1. Succession Planning (including recruitment, retention, and classification); 2. State Data Center; and 3. Funding. Other risk areas not included in either of our proposed audit plans, and not covered by executive staff, are presented in the table below. Potential audit approaches are described. In the absence of an audit, Audit Services expects that the Audit Committee will communicate the risk areas to management to address the risks. Risk Area Transportation Applications Development Process Information Technology Business Planning Description Information technology application processes were identified across the board as relevant risks to ODOT achieving its mission. Indentified concerns included: Lengthy and time intensive process; Result in less than optimal systems; and Antiquated software and systems. An audit of the TAD process will include a process review to: Identify length of process and potential bottlenecks; Evaluate customer satisfaction; Evaluate consistency; and Identify areas of improvement. The lack of an enterprise approach to information technology planning was identified as a top risk by executive staff, upper management, and audit staff. Concerns involved: Governance Resource Deployment Business Continuity To audit information technology business planning we will: Review governance, project prioritization, and resource deployment; and Identify areas of improvement. Potential Management Action Management is considering hiring a consultant to address the risk. Management is considering hiring a consultant to address the risk. 9
A&E Procurement Process Emergency Preparedness Business Continuity Employee Safety Human Resources Structure Review Construction Documentation Consistency The A&E contract procurement process and subsequent contract administration were identified as top risks by executive staff, upper management, audit staff, and stakeholders. Concerns involved: Contracting Methods Procurement Compliance Contract Quality Negotiation Documentation Deliverables Receipt Contract Costs Payment Appropriateness Performance Monitoring Using a sample selected from central and regional procurement, an audit would include the following: Test compliance with ORS 125-248; Review documentation standards; Review process timelines; Identify potential bottlenecks; Evaluate statewide consistency; and Identify areas of improvement. Emergency preparedness has received increased emphasis across the agency due to tsunami threats and potential earthquakes. An audit of emergency preparedness would assess ODOT s ability to continue operations in an emergency. Employee safety was raised as a risk area due to three recent workplace deaths. These deaths end a 10-year period of no workplace deaths. An audit of employee safety will include a review of safety training and culture. Human resources have been identified as a high risk area for a number of years. Due to turnover in management and staff, resources may not be deployed in the most effective manner. An audit of human resources will review organizational structure and staffing. Construction documentation was identified as an auditable risk by executive staff because documentation standards are lacking and projects may be documented in different manners. To audit construction documentation consistency, we will select a sample of construction contracts and: Review documentation standards and requirements; Management is considering hiring a consultant to address the risk. This risk is an agency priority. This risk is an agency priority. Management is considering hiring a consultant to address the risk. 10
Access Management Bulk Fuel Local Agency Certification Procurement Responsibility Local Agency Certification Administration and Oversight Compare existing documentation to standards; Evaluate statewide consistency; and Identify areas of improvement. Access Management is a program with high public visibility and safety impacts to the traveling public, and economic impacts to private businesses. As a result of SB 1024, access management processes at ODOT must be clarified, simplified, and based on objective standards. An area that the legislation did not address but that was suggested by the stakeholder committee is a process review for considering grants of access for public roads and streets. Controls over bulk fuel tanks is an issue that Fleet Management has consistently raised over the past few years and was passed on to Audit Services through the ERM process. An audit of bulk fuel controls would involve a sample of tanks and: Review of existing controls; Possibility of undetected theft; Assessment and costs of additional controls; and Identification of areas of improvement. ERM and ODOT management expressed concern over the procurement part of the Local Agency Certification Program. Concerns raised included: Stewardship Responsibilities Accountability Continuous Review Roles and Responsibilities An audit of the procurement responsibilities in the Local Agency Certification will include: Review of certified agency procurement process; Assessment of stewardship responsibilities; and Evaluation of roles and responsibilities. Now that more local agencies are seeking certification, management and staff are concerned with continued oversight of certified agencies. An audit of continued oversight of certified agencies would include: Assessment of continued ODOT oversight; Comparison to WSDOT local program oversight; and Evaluation of project close-out. A bill is being worked on that addresses this risk area. 11
12