Internal Audit Follow-Up Report. Toll Operations Contract Management TxDOT Internal Audit Division

Transcription

1 Internal Audit Follow-Up Report Toll Operations Contract Management TxDOT Internal Audit Division

2 Objective Assess the status of corrective actions for high-risk Management Action Plans (MAPs) previously communicated in the Toll Operations Contract Management audit issued August Summary Results Testing consisted of an evaluation of 14 MAPs to determine if corrective actions were implemented as agreed. MAP Status Comments 14 Closed Corrective actions have been completed. Toll Operations Division has developed corrective actions to demonstrate completion of the Management Action Plans. However, issues relating to the contract, including certain performance required by the vendor, had not been fully resolved by the date of this engagement. Scope The scope included a review of submitted documentation regarding Toll Operations Division s (TOD) contract management and oversight activities and the implementation of a contractual performance measure process. The engagement was performed by Alberto Calderon and Lindsay Bibeau (Engagement Lead). The engagement was conducted during the period from May 23, 2016 to July 15, Methodology The methodology used to complete the objective of the engagement work included: Reviewed prior audit report findings and Management Action Plans (MAPs) Conducted interviews with key personnel in Toll Operations Division (TOD), such as, the division director, the project manager, and quality assurance/quality control personnel to understand the actions taken to mitigate the risks previously identified Examined Project Management Professional (PMP) certifications for both TOD and those employees from a third-party consultant provider supporting TOD operations Reviewed the Project Management Institute Project Management Body of Knowledge (PMBOK) guidelines that were used to help ensure TOD has implemented Standard Operating Procedures (SOPs) for the TOD projects in accordance with these procedures Reviewed listing and content of newly created SOPs, including Project Management Office (PMO) and Key Performance Indicators (KPI) that will assist TOD in monitoring and reviewing operational activities and KPIs Reviewed monitoring reports, templates, and project matrices detailing planned activities used to verify TOD is tracking projects and receiving accurate KPI reports used for performance measurements Reviewed TOD s Strategic Plan, organizational chart, and other reports from an additional third party consultant to document TOD s efforts for process improvements August

3 Background This report is prepared for the Texas Transportation Commission and for the Administration and Management of TxDOT. The report presents the results of the Toll Operations Contract Management Follow-Up engagement which was conducted as part of the Fiscal Year 2016 Audit Plan. In June 2013, TxDOT entered into a contract with a vendor to modernize the customer service center system. This modernization included converting existing toll accounts to a new system, merging accounts to simplify account maintenance for customer, implementing a new interactive voice response system, and updating other customer service capabilities to improve the customers experience. The contract included milestones, which were to be completed by the vendor to enable transition from the previous toll system. The contract further established key performance indicators which were to be used by TOD to help monitor vendor performance as established by the contract. We conducted this follow-up engagement in accordance with Generally Accepted Government Auditing Standards and in conformance with the International Standards for the Professional Practice of Internal Auditing. Those standards require that we plan and perform the engagement to obtain sufficient, appropriate evidence to provide a reasonable basis for our conclusions based on our engagement objective. Recommendations to mitigate risks identified were previously provided to management during the original engagement to assist in the formulation of the management action plans referenced in this report. The Internal Audit Division uses the Committee of Sponsoring Organizations of the Treadway Commission (COSO) Internal Control Integrated Framework version August

4 Detailed MAP Follow-Up Status MAP Status: Closed Corrective actions have been completed. Original Audit Finding Number 1: Contract Management/Oversight Framework MAP Owner: Rick Nelson, Director, Toll Operations Division MAP 1.1: Project Management Processes TxDOT will implement Project Management Institute (PMI) PMBOK guideline usage by TxDOT, consultants, and vendor to ensure a well-designed process is implemented and becomes standard operating procedure for contract and project management. Original Completion Date: October 15, 2015 Actual Completion Date: May 19, 2016 MAP Owner: Roland Silva, Quality Assurance/Quality Control, Toll Operations Division MAP 1.3: Standard Operating Procedure Development Phase 2 TOD will continue its development of TOD staff SOPs which will address each TOD SME s specific oversight and monitoring activities of specific operations processes. Original Completion Date: November 15, 2015 Actual Completion Date: June 6, 2016 MAP Owner: Brian Smallwood, Director, Operational Excellence, Toll Operations Division MAP 1.4: Verification & Validation Processes TOD will expand its verification/validation activities to include process audits by TOD operational area SMEs which will be subject to review and audit by TOD QA/QC Team. Activities identified on MAP 1.2 monitoring SOPs will be audited periodically according to a TOD QC Audit Schedule and subject to regular spot checks. Results and any recommendations will be reported to the Director of Operational Excellence and Toll Operations Director when needed and/or on a quarterly basis. In the interim, TOD will continue its monitoring of identified key processes and coordinate appropriate actions with vendor through existing regularly scheduled Issues Review, Release Planning and QA/QC meetings. Original Completion Date: October 15, 2015 Actual Completion Date: June 30, 2016 August

5 MAP Owner: Rick Nelson, Director, Toll Operations Division MAP 1.5: Independent 3 rd party review of operations TOD will work with an independent consultant to provide an independent information technology assessment of contract management and review of operations to provide specific areas of focus to improve both. Consultant will also provide an independent validation of vendor contract and performance by implementing tools using Tableau and Crystal Reports. These tools will be owned and used by TOD to monitor contract performance after the initial assessment. Original Completion Date: December 15, 2015 MAP Owner: Rick Nelson, Director, Toll Operations Division MAP 1.6: Additional PMO process Third party PMP certified consultant implementation of a project matrix detailing approved projects, project owners, resources assigned, percentage of employees time assigned to project, project approved schedule by budget assigned and expended by month. This matrix will include all planned and approved activities for the next 18 months and project and/or contract managers will report monthly at TOD staff meetings on these projects. Original Completion Date: October 15, 2015 Actual Completion Date: March 15, 2016 MAP Owner: Linda Sexton, Deputy Director, Toll Operations Division MAP 1.7: Atkins PMO Certified Managers TOD will direct their third party consultant in writing to provide the PMP certified project manager(s) required by contract and to adhere to PMBOK guidelines. Original Completion Date: September 15, 2015 MAP Owners: Rick Nelson, Director, Toll Operations Division Linda Sexton, Deputy Director, Toll Operations Division MAP 1.8: TOD Organization TOD will evaluate its present organizational structure and provided a recommendation to the TXDOT CFO of any needed changes required to address unsatisfactory ratings noted in this Internal Audit Report. Original Completion Date: November 15, 2015 August

6 MAP Owners: Rick Nelson, Director, Toll Operations Division Linda Sexton, Deputy Director, Toll Operations Division MAP 1.9: TxDOT PMO Certified Managers TOD will require all identified TOD project managers to obtain PMP certification and use PMBOK guidelines in managing projects. Phase 1 certification of three TxDOT project managers will be completed by December 15, Phase 2 will provide a plan to certify all remaining TxDOT TOD project managers. Original Completion Date: February 15, 2016 Actual Completion Date: May 05, 2016 MAP Owners: Rick Nelson, Director, Toll Operations Division Teresa Lemons, Funds Management Section Director, Toll Operations Division MAP 1.10: Procure Project Management Software Develop action plan to procure and implement third party software for Project Management designed around the PMBOK guidelines to provide a dashboard/score card using both qualitative and quantitative information on all approved tasks and projects. Said software shall provide a transparent dashboard/score card and shall be accessible 24x7 to senior management of TOD and any other users approved by TxDOT CFO. Original Completion Date: January 15, 2016 Actual Completion Date: May 15, 2016 MAP Owner: Teresa Lemons, Funds Management Section Director, Toll Operations Division MAP 1.11: TOD Budget Officer Recruit and hire an individual who is a Certified Public Accountant to fill the Budget Officer position to provide Generally Accepted Accounting Principles compliance guidance to TOD. Original Completion Date: September 15, 2015 August

7 Original Audit Finding Number 2: Performance Measure Achievement MAP Owner: Brian Smallwood, Director, Operational Excellence, Toll Operations Division MAP 2.1: Complete Implementation Milestones TOD will continue to coordinate with the vendor to identify and prioritize the resolution of incomplete implementation milestones. o TOD is meeting with the vendor twice weekly to identify issues impeding implementation and to prioritize the evaluating of solution releases through Issues and Release Planning meetings. Original Completion Date: October 15, 2015 MAP Owner: Brian Smallwood, Director, Operational Excellence, Toll Operations Division MAP 2.2: Complete Key Performance Indicator (KPI) Reports TOD will continue to evaluate KPI reports available and drive this report review effort, including validation and approval of vendor performance reporting. TOD will deliver a formal report Validation and Release process document to the vendor to facilitate the validation of the KPI reports. TOD currently meets with the vendor s Technical Reporting team daily to resolve KPI report issues and prioritize their report fixes or enhancement necessary for accurate and complete KPI reporting. A meeting will also be scheduled targeted at gaining a more comprehensive understanding of Vector 4G data and database relationships to facilitate report and data validation. Original Completion Date: November 15, 2015 MAP Owners: Rick Nelson, Director, Toll Operations Division Linda Sexton, Deputy Director, Toll Operations Division MAP 2.4: System Monitoring Improvements TOD will implement third party software to automate lane audits and monitor system performance. Original Completion Date: December 15, 2015 August

8 MAP Owners: Linda Sexton, Deputy Director, Toll Operations Division Brian Smallwood, Director, Operational Excellence, Toll Operations Division MAP 2.5: Interim Performance Reporting TOD will calculate performance measure achievement by using statistics provided by the vendor and independently evaluate the data to assess damages. This will be done until such time as required reports are approved by TxDOT through Operational Observation Period and available for use. Original Completion Date: November 15, 2015 Actual Completion Date: May 15, 2016 Closing Comments The results of this MAP Follow-Up engagement were discussed with Toll Operations Division on August 9, The Internal Audit team appreciates the cooperation and assistance received from the Toll Operations Division during this engagement. August