TxDOT Internal Audit Consultant Engineering Contract Administration Function (1103-1) Department-wide Report

Transcription

1 1103-1F Consultant Engineering Contract Administration Follow-up Audit Original Audit Report Attachment A Consultant Engineering Contract Administration Function (1103-1) Department-wide Report Introduction This report has been prepared for the Transportation Commission, TxDOT Administration and management. The report presents the results of the Consultant Engineering Contract Administration Audit which was conducted as part of the Fiscal Year 2006 Audit Plan. The objective of the audit was to determine if TxDOT s consultant engineering contract administration and management process is in compliance with applicable directives, and that effective controls exist to ensure good work quality, cost containment, and to prevent the procurement of non-engineering services. Scope Audit team members included Karin Faltynek (Lead Auditor), Augustine Nwoko, Craig Easley, Roger Stacy and Dennis Olson (Staff Auditors) with Allen Barr, CIA, CFE (Auditor-in-Charge) providing oversight for the audit. The audit work was conducted during the period of May 2005 to June All work was performed in accordance with the International Standards for the Professional Practice of Internal Auditing of The Institute of Internal Auditors. Audit work included preliminary work (including a risk analysis) with the Administration and the Design Division (DES). Additionally, questions and comments were received from the Transportation Commission and were included in the Audit Program. The Audit Office conducted fieldwork in twelve of the department s twenty-five districts and the TTA Division, and individual reports were issued for noted deficiencies. The fieldwork was also conducted in the remaining twelve districts (except Brownwood) by the District Internal Auditors. The Audit Office provided the Audit Program developed for this audit to those District Internal Auditors and provided on-going assistance. This report reflects the results of the work performed by the Audit Office. The audit focused on the contract selection and negotiation process; the implementation of the new contract procedures outlined in Executive Director Mike Behrens memo of May 5, 2005; the contract management and payment approval process; the pre-letting design review process of deliverables for quality; the post-letting change order process for consultant errors/omissions, and the implementation of the new provider evaluation process as directed in June Active consultant engineering contracts were sampled in each location along with older contracts to determine if scope creep had occurred with those contracts. The audit also included a review of projects which had delayed Lettings due to poor design work by consulting firms, and a review of change orders that had been coded as consultant errors/omissions during the construction phase. The audit did not examine consultant contracts related to Comprehensive Development Agreements (CDAs), those related to the department s aviation program, or professional surveying services, as they were addressed in other audits. Report of 14 August 29, 2006

2 Background Information The department has gradually increased its expenditures for consultant engineers primarily due to an increased workload and Letting volume, and because of loss of expertise in certain areas due to turnover of engineering personnel. Preliminary engineering expenses (PE) (from the TxDOT Preliminary Engineering Summary Reports on the Finance Division s webpage) for consultants have increased in Fiscal Years 2004 through Fiscal Year 2004 was $220,830,133. Fiscal Year 2005 increased 48.0% to $326,770,343. Fiscal Year 2006, through June 2006 (10 months), is $308,184,652 which is an increase of $33,016,376 (12.0%) from the same 10 month period of Fiscal Year The TxDOT Preliminary Engineering Efficiency Reports also show that the percentage of expenditures to consultants has increased from 65 % of total PE expenditures in Fiscal Year 2004 to 72% in Fiscal Years 2005 and In some of the larger districts, the percentage of preliminary engineering expenditures for consultant engineers (versus in-house engineering) has increased dramatically during that same time frame. At the end of Fiscal Year 2004, the Austin District had used consultants for 60% of its preliminary engineering (PE) expenses; San Antonio was 69%; and Dallas was 63%. At the end of Fiscal Year 2005, the Austin District had used consultants for 89%; San Antonio for 87%; and Dallas for 71% of its PE expenditures. The amount of consultant engineering expenditures for the metro districts (Austin, Dallas, Fort Worth, Houston, and San Antonio) in Fiscal Year 2005 was $202,651,191 (or 62% of the total for all districts/divisions). As of June 2, 2006, the department had approximately 1,054 active consultant engineering contracts with a contract value of $2.245 billion. The management of the department s consultant engineering contract program is shared between the Design Division and the Office of General Counsel s Contract Services Section (OGC-CSS). The Design Division s Consultant Contract Office (DES-CCO) provides guidance and direction to the other divisions and districts for their consultant engineering contract operations, maintains databases of all active contracts, precertifications, and provider evaluations, with the Director of the Design Division having authority to execute all consultant engineering contracts up to $5 million. The OGC-CSS also provides guidance/legal advice, maintains the electronic contract monitoring system (FIMS Segment 41 database) and standard contract forms, and is the official repository of the executed copies of the engineering contracts, supplements, and Work Authorizations. In most districts, the responsibility for administering their consultant engineering contracts falls under the direction of the District Transportation & Planning Director (TP&D). Some districts have centralized the primary contracting functions in an office reporting to the District TP&D and in other districts, the function is more de-centralized with more active participation by the Area Offices. Opinion Based upon the audit results, we believe that the department is in general compliance with the applicable directives, but believe improvements are needed in order to achieve better work quality and cost containment in our consultant engineering process. The findings and recommendations outlined below provide detailed information that should improve the process for better work quality and cost containment as well as improved compliance with existing policies/procedures. Further, we found very little evidence of the procurement of non-engineering services during the audit, and believe that risk has been properly addressed by the Administration. Report of 14 August 29, 2006

3 Summary of Findings No. 1: Improvements are needed for cost containment/savings. No. 2: Improvements are needed to achieve good work quality. No. 3: Current consultant engineering operations do not fully comply with existing policies/procedures. Detailed Findings and Recommendations No. 1: The review of sample contracts in twelve districts and the TTA division indicates that improvements are needed for cost containment/savings. Examples include the following: A. Identifying the need to contract: The Audit Team selected 28 recent/active contracts for review, which included both Specific Deliverable and Indefinite Deliverable contracts. All but two of these contracts were executed between 2004 and 2005; one in 2000 and one in We found that the districts are required to annually prepare contract budgets for the following 3-years. This information is used by the Design Division to allocate budgeted funds to each district for their design consultant contracting needs. The Design Division requires that these budget requests be submitted on a standard form which includes project number information, estimated construction cost information, highway locations, estimated design beginning dates, contract numbers (if in progress) and estimated annual expenditures throughout the budget. Information gathered in district interviews indicates that district staff views the process of developing an annual budget as separate from the process of identifying the need to contract, which is the approval mechanism to begin the contract process. We found that the districts do not document the development of a broad scope of work before the contracts are advertised. There was also no documentation to support that the districts then develop a more detailed scope of work and level of effort during the pre-negotiation period as instructed by former Executive Director Wes Heald in The districts reviewed generally did not develop a documented scope of work or level of effort for their consultant engineering contracts until the negotiation phase for Specific Deliverable contracts or the Work Authorization phase for Indefinite Deliverable contracts. Typically, a consultant engineer s first task after being selected for a Specific Deliverable contract is to develop, or assist in developing, a detailed scope of work. Districts often require the consultant engineers to participate in this because the time spent on this task is generally non-billable, and it has become a part of the negotiation phase for the contract. Developing a scope of work prior to contract negotiations seems to be viewed as a burden since the district usually relies on the consultant engineer to participate. TxDOT policy dated May 5, 2005 requires submitting a scope of work for Specific Deliverable contracts for all Notices of Intent posted after May 13, A contract to verify this new process was not available for review during the audit. In addition, we found that identifying the need for Indefinite Deliverable contracts is often not tied to a specific project, because most districts do not determine which design work will be performed by consultant engineers until after these Indefinite Deliverable contracts are executed. For these contracts, Report of 14 August 29, 2006

4 the development of a scope of work occurs in the Work Authorization stage, generally as a coordinated effort between the consultant engineer and the Project Manager as part of negotiation. The Audit Team also selected 18 contracts for review that are 5 years old or older. These contracts were reviewed to determine if the current work performed is still in accordance with the original contract scope and if they are indeed still needed. We found that these contracts are still needed to complete various work and that the work performed under these contracts still conforms to the initial scope of work. We further noted that most of these older contracts selected, primarily Specific Deliverable contracts, have been amended several times for cost increases and extensions of the contract termination dates. The contract amount of 13 of these contracts was increased by an average of %. The reasons for cost increases varied; examples are Metric to English measurement conversions in some of the older plan sets, changes in design standards for the larger project designs, and various other reasons. The contract term of 17 of these contracts was increased by an average of %. The time increases exclude construction support such as shop drawings or updates to plan sheets. Five of these contracts had time added for Construction Support. B. Consultant Negotiation Process: Ten of the twelve districts visited did not prepare documented independent estimates for the level of effort or an independent contract cost estimate prior to negotiations with the consultant. The districts generally discuss in detail the scope/requirements for the work with the consultants in preparation for negotiations and they use the cost estimate prepared by the consultant engineer after these meetings as a starting point to save time. In one district, little evidence was found that the labor rates had been researched prior to negotiations with the selected firms. Attachment A reflects our analysis of contract labor rates reflected in a judgment sample of 43 contracts executed during calendar year 2004 for firms with at least two contracts executed during that time period. We found a number of significant variances (>$20/hour) in the contract labor rates paid by various districts to the same firms for the same employee category. Some of these variances may be due to the firms inconsistent staffing categories and the resulting differences in pay. Other influences on pay may be due to the differences in years of experience of the consultants employees and complexity of the projects. We further noted that most of the negotiated overhead rates were not above the audited rates, and the fixed fees were between 12 % and 15%. We further noted that the Design Division has developed a template for small projects that can be used to develop level of effort estimates. C. Contract Administration Process: The review of three of the selected contracts revealed that the deliverables indicated on the invoices do not match the due dates on the contract or Work Authorization work schedules. We found that the contract or Work Authorization work schedules of some of the contracts are not reviewed for timeliness of the deliverables. Instead, some of these districts rely on verbal confirmation of timeliness, which at times includes changing the due dates of deliverables without adjusting the contract/work Authorization work schedule. Anecdotal information indicates that districts will require consultant engineers to work 24/7 to ensure that letting dates are not delayed if the design deliverables are falling behind schedule. The general feeling in the districts is that close monitoring of timeliness is not necessary as long as the project is not delayed from letting. This diminishes the importance of the contract or Work Authorization work schedules and their monitoring. In nine of the districts reviewed and TTA, we noted that the Project Managers were not reviewing the FIMS Segment 76 (Project Ledger) reports and/or budget monitoring reports for accuracy of charges Report of 14 August 29, 2006

5 to their assigned projects as required in Chapter 2, of the Financial Management Policy Manual and the Project Development Policy Manual. We noted that the Finance Division has submitted an Information Resource Request to enhance FIMS reporting. In three districts, we observed a control weakness in the oversight of Project Managers. These Project Managers assign work, accept work, and authorize payment to the consultant. We realize that the assigning and accepting of work is integral to a Project Manager s duties and that the Project Manager should also review the invoices related to the work. However, the actual authorization of payment should either be made or reviewed by a staff member independent from the Project Manager. D. Consultant Errors/Omissions: In six of the districts reviewed, we found that the districts did not have a process to consistently identify and seek reimbursement from the consultant engineers for additional design/construction costs caused by errors/omissions in the engineer s plans. Since Change Orders are identified and coded in the Area Office, the District Construction Office and District TP&D are not closely involved in the review of these documents. Effect: A. Identifying the need to contract: Without contemplating the full extent of work to be performed, the most appropriate contract type (Specific Deliverable, Indefinite Deliverable, and Discipline Specific) may not be chosen, and/or the contract amount may be considerably understated. Additionally, over-design could occur if the scope of work is not at least partially developed by TxDOT staff prior to development of a scope of work with the consultant engineer. B. Consultant Negotiation Process: By not developing an independent estimate of the expected level of effort, the TxDOT negotiators are at a disadvantage when they negotiate the level of effort with the consultant engineer firm. The TxDOT negotiators are also at a disadvantage when the individual labor rates are not researched prior to negotiations. C. Contract Administration Process: Not monitoring the timeliness of work performed by the consultant engineer could lead to contract time extensions and added contract costs. Anecdotal information gathered during the audit also indicates that lapsed time on consultant engineering contracts is often recouped by last minute marathon work sessions that no doubt lead to errors and omissions in the consultant s work. The project records in FIMS may not be up-to-date, which means that project funding or other related project decisions by TxDOT Executive and District management may be based on inaccurate information. Inaccurate data in FIMS also leads to inaccurate reporting to the Legislature and others. The weakness in oversight of the Project Manager s authorization of invoice payments could potentially lead to fraud or other irregularities. D. Consultant Errors/Omissions: By not seeking reimbursement for additional contract costs caused by a design consultant s error/omission, we allow the consultant to be in breach of the contract. Not seeking reimbursement also reduces our funding resources for roadway construction, limits the consultant engineering firm s accountability, and is a disservice to the taxpayer. Report of 14 August 29, 2006

6 Recommendation: A. Identifying the need to contract: We recommend revising the current policy manuals to reflect the guidance included in former Executive Director Wes Heald s memo of August 3, We also recommend including the current budget planning process in the next update of the Engineering, Architectural and Surveying Services Manual (EAS Manual) as part of Chapter 2, Section 2 so that the planning and budgeting processes are linked. We further recommend requiring a list of proposed Work Authorizations with the request for Indefinite Deliverable contracts similar to the requirement in the latest policy directive for Specific Deliverable contract requests that requires submission of a scope and budget of a sample project. We anticipate that a more detailed contract planning process, coupled with the new $2 million limit for Indefinite Deliverable contracts, should produce more Specific Deliverable or Discipline Specific contracts. We also believe that this may lead to less cost and time extensions while still attracting new and smaller firms to participate in the TxDOT Contract Selection Process. B. Consultant Negotiation Process: We commend the Design Division for beginning to develop a template that will assist the districts in the development of levels of effort for smaller projects. We recommend increasing this effort to include large projects as well. In addition, we recommend expanding this template to allow calculation of estimated contract or Work Authorization costs. We further recommend that the Division develop a database, with similar information as shown in Attachment A that can be used as a monitoring tool to validate the districts use of the recommended labor rate ranges, and to gauge the need for negotiation training. C. Contract Administration Process: We recommend that the Design Division, in conjunction with the Human Resources Division, should develop a training program for Project Managers that would focus on best practices in contract administration and management, so that the Project Manager understands his/her responsibilities from the contract planning phase through the contract execution stages and their role in the construction phase of the project. The training should also stress the need for good work schedules and the consultants adherence to those agreed upon schedules as a condition of payment. We further recommend that the Finance Division accelerate the IRR for the FIMS enhancements so that the reports are more user friendly, and to include training for these reports in the training program for the Project Managers. D. Consultant Errors/Omissions: We recommend that the Design Division should develop additional guidance for the districts to implement the department s errors/omissions policy and include that in the Project Manager s training program as noted above. Additional recommendations regarding errors/omissions are also included in Finding No. 2 below. Management Response and Action Plan: A. Identifying the Need to Contract The Engineering, Architectural and Surveying Services Manual and Contract Management Manual will be updated by December 2007 to reflect previous Executive Director guidance that an independent level of effort estimate should be prepared prior to negotiations. In addition, the current or similar budget development process will be incorporated in the Engineering, Architectural and Surveying Services Manual by December 2007 to emphasize project and contract development planning as the basis for estimating annual consultant expenditures required for statewide cash-flow monitoring. Report of 14 August 29, 2006

7 Beginning November 1, 2006, for indefinite deliverable contracts, the CCO-1 submittal will require a list of projected work authorization projects/assignments including a general description of the type of work and anticipated value. B. Consultant Negotiation Process The purpose of the cost proposal tool being developed is to provide a consistent format for developing a cost estimate by TxDOT or cost proposal by a provider at the contract level for specific deliverable or work authorization level for indefinite deliverable contracts. It is intended to be used initially on small to medium PS&E projects only. With a consistent format of staffing categories and tasks, the level of effort estimate can be incorporated into a database for future reference. Having already gone through a long and unsuccessful process of trying to develop a database from contract data where no consistency existed, it was determined that development and implementation of a standard format would be necessary in order to have a meaningful database, thus the development of the current tool. It is anticipated that the current tool will be implemented and available to districts within the next six months. The current tool is envisioned as a first generation which will provide lessons learned and areas where improvements will be needed to accommodate different and larger project types. Given the lessons learned to-date in developing the current tool, it is anticipated that a second generation tool for larger and different project types could not simply be added as another step of the current process, but will require a different approach involving an actual program. DES-CCO will remain focused on the implementation of the tool being developed and will determine the next best step after initial use can be evaluated. The current tool is intended to support a database for levels of effort, although rate data can be incorporated at the same time. The Design Division does not agree with developing a database containing negotiated rate data that was not developed using standardized staffing categories. This will result in the same difficulties and inefficient process that was encountered as mentioned above. In general, rate data as shown in Attachment A can be misleading and easily misinterpreted without a full explanation of what the numbers are based on, characteristics of the negotiations, and the specific intent of how the numbers should be used. C. Contract Administration Process The Design Division acknowledges the need for training. Based on experience to-date, the best approach appears to be training that covers information from both a contract administration and project management perspective so that it will be useful to attending staff that have responsibilities across a range of roles. The structure of roles and responsibilities currently varies by district and division. It is important that the immediate need for training be balanced with the desire to develop a formal training program that requires significant time and resources. The Design Division will evaluate existing information developed for training/workshop purposes and will determine the feasibility of making this information available within the next 12 months to address immediate needs while working in conjunction with HRD to determine a staged process for developing a more formal program. In response to the specific comment about stressing the need for good work schedules and the consultants adherence to those agreed upon schedules as a condition of payment the current contracts are not structured to make adherence to schedule as a condition of payment. The Design Division does not agree with making the agreement so tight that schedule adherence is a condition for payment. This introduces a significant risk to the provider that will directly affect the cost. In addition, TxDOT would be subject to the same strict terms of meeting every deadline assigned for TxDOT responsibilities which it is anticipated most districts would not be willing to accept. Future training will include information on types of schedules needed and recommended Report of 14 August 29, 2006

8 for contract purposes in comparison to schedules needed for day-to-day project and task management. The Finance Division believes that the data contained in FIMS is materially correct and can be relied upon by management and others. The review of accounting reports by project managers is just one of many controls which are designed to ensure the accuracy of data. Finance will assist Design in preparing training for project managers on the available project accounting reports. Finance will also review the current project accounting reports with the Assistant Executive Director for Engineering Operations to determine whether any immediate modifications or enhancements to the reports are necessary. In developing the planned future enhancements to FIMS reporting, selected project managers will be invited to provide input. D. Consultant Errors/Omissions: The Design Division will develop additional guidance by October 2006 for implementation and inclusion in future training, as applicable. Additional guidance will be maintained on the DES- CCO web site and incorporated into the Engineering, Architectural and Surveying Services Manual. No. 2: The review of sample contracts in twelve districts and the TTA division indicates the quality of work received from the engineering firms could be improved. Examples obtained during the audit include the following: A. Consultant Selection Process: Anecdotal information gathered during audit field work indicates that several districts receive consultant engineering work products that do not undergo the level of Quality Control/Quality Assurance (QC/QA) that is expected. The standard Long List evaluation consists of three criteria (Project Understanding, Similar Experience of Project Manager and Similar Experience of Task Leaders) and does not include a criteria and Relative Importance Factor (RIF) for QC/QA. The standard Interview evaluation consists of five criteria (project understanding, experience of Project Manager and team, team s ability to meet the proposed work schedule, response to interview questions and prior performance) and does not include a criteria and RIF for QC/QA. The review of selected contracts revealed that three districts did not include provider evaluations/references in the Selection Process criteria as required by 43 TAC We also noted that the Interview and Contract Guide and the Request for Proposal template on the DES website do not offer evaluating a firm s QC/QA as an additional criteria. The Design Division recently emphasized reviewing a firm s QC/QA at the deliverable stages. This directive was circulated in the June 15, 2006 memorandum from Assistant Executive Director for Engineering Operations Amadeo Saenz, Jr. B. Contract Administration Process: The review of the selected contracts indicates that four districts and one division are deficient in monitoring the timeliness of the consultant s deliverables: o o updated work schedules were not included in the contracts/work Authorizations; progress reports submitted with the invoices did not include sufficient information to link the billed work to the contract/work Authorization work schedule, so the timeliness of billed tasks could essentially not be evaluated prior to payment approval. Report of 14 August 29, 2006

9 C. Provider Evaluation Process: Not all districts have implemented a monitoring system for provider evaluations and not all have selected an evaluation method for their contracts. Some districts are also not up-to-date on their provider evaluations as required in the guidance provided by the Design Division. We also found that the provider evaluation for project constructability is currently not electronically available as indicated in Chap 5, Sec. 5, of the Construction Contract Administration Manual. This manual requires that the department s construction Project Manager submit evaluation information to the Project Manager for the consultant design contract for further evaluation. We understand that a post construction evaluation is included in a planned update to the current Provider Performance Evaluation Form. D. Errors and Omissions Process: During the audit, we reviewed the district s design review process (pre-letting) and noted that most districts notify the consultant in writing of their errors/omissions. During the construction phase (postletting), we noted that most districts don t notify the consultant (verbally or in writing) when consultant errors/omissions are detected. We further noted that six of the districts reviewed did not have a system in place to review Change Orders for consultant errors/omissions (Reason Codes 1B and 1E), to document the consultant s errors/omissions, and to seek reimbursement for any additional design/construction costs in accordance with the errors and omissions policy included in 43 TAC 9.38(f). The Construction Division provided change order data for FY which indicates that $87.3 million was coded as 1B and 1E. This amount consists of added construction costs, and costs beyond the cost that would have been incurred if the plans were correct. We only identified $1.17 million that had been reimbursed to TxDOT, which indicates that the policy is not being followed. We also reviewed our sample of Change Orders with district staff and noted that a number of the Change Orders in two districts were not accurately coded as consultant error/s omissions, but were caused by Area Engineer preferences that had not been communicated to the design consultant, and various other reasons. We noted in our interviews that there are significant differences of opinion among district staff as to what constitutes a consultant s error/omission, and whether the additional costs should be pursued for reimbursement. In two districts, we noted that district staff made the corrections to the plans rather than requiring the consultant to address those deficiencies at no cost to the department. Finally, we noted in our review that several of the reimbursement deposits did not follow established procedures. The Finance Division subsequently corrected these deposits to apply them to the correct projects after notification by the Audit Office. Effect: A. In a number of instances, department personnel are essentially performing the firm s QC/QA during the deliverable review process instead of requiring the consultant to deliver quality deliverables. In those instances, payments were approved for substandard deliverables. In effect, this increases the consultant s profit on the contract and wastes the time of TxDOT staff in performing the consultants work. B. Timeliness of the deliverables is one of several provider performance measures as indicated in the Provider Performance Evaluation Criteria No. 3 Schedule Management. If timeliness of deliverables is not monitored and documented, then the scoring of this criterion in the Provider Evaluation is not defendable. Absence of monitoring also indicates a lack of schedule enforcement by the Project Manager. C. Without the full implementation of the new Provider Evaluation Process, the department s Project Managers are not fully utilizing an important tool to improve the consultant s performance, and to only select firms with good performance. Report of 14 August 29, 2006

10 D. The department is not in compliance with its own policy, and is not using this mechanism to improve the quality of the consultants work products. Recommendation: A. Consultant Selection Process: We recommend that the Design Division modify the existing standard selection criteria for Long List, Interview and Proposal to include a criteria and RIF for the firm s QC/QA process. Including QC/QA in the selection process, coupled with the recent directive to assess QC/QA during the review of major deliverables and the relatively high (17) base score for Accuracy and Completeness of Deliverables in Criteria No. 1 of the Provider Evaluation Form, should stress the importance of QC/QA. Inclusion of construction phase deficiencies identified as consultant errors/omissions in the Provider Evaluation would place an additional emphasis on QC/QA to the consultant community. B. Contract Administration Process: We recommend that the Design Division and OGC-CSS provide additional guidance to the districts on the level of detail that is actually required in contract/work Authorization work schedules, such as major deliverables or milestones. Additionally, we recommend reviewing current policy manuals to determine which guidelines referring to monitoring the progress of work need to be adjusted so they reflect current expectations. This may also include developing a required format for provider progress reports so the information presented in them is better aligned with the information in the work schedules. Any changes should be included in the rewrite of the Engineering, Architectural and Surveying Services Manual and other manuals as needed. We further recommend that the training for Project Managers should include guidance on the relationship between contract monitoring, documenting the monitoring process and eventually using this documentation during the Provider Performance Evaluation process. C. Provider Evaluations Process: We recommend that the Design Division provide additional guidance to all district contract management staff to stress the need for full implementation of the provider evaluation process. A follow-up discussion could also be included in the next Design/Bridge Conference workshop. We further recommend that the Design Division accelerate the update of the current Provider Evaluation form to include an evaluation of constructability as suggested in the Texas Administrative Code. Including this data will provide cradle to grave information on a provider s performance, which will make the provider evaluation database a powerful tool to be used during the provider selection process. D. Errors and Omissions Process: To strengthen the department s errors/omissions policy, we recommend that the Design Division, in conjunction with the Construction Division, should develop additional guidance for the department s errors/omissions policy including what constitutes a consultant s error/omission, development of a district change order review process, and the amount of additional costs that should be pursued for reimbursement. Additionally, the guidance should include written notification to the consultant firm regarding the errors/omissions, and documentation of the causes of the errors/omissions along with the firm s response for the reimbursement request so that it can be adequately defended. The need to include instances of errors/omissions in the consultant engineer s Performance Evaluation should also be stressed. Report of 14 August 29, 2006

11 Additionally, we recommend that the Design Division coordinate with the Finance Division to provide additional guidance to ensure that all reimbursements for errors/omissions are properly credited to the appropriate fund sources and develop a method to identify these reimbursements in the Financial Information Management System (FIMS) so that comparisons can be made between the Change Orders coded as consultant errors/omissions in Site Manager and CTS, and the collections identified in FIMS for these change orders. We believe the additional emphasis placed upon improved QC/QA, and holding the consultant s accountable through the errors/omissions policy, should improve the overall quality of their work and ultimately reduce the costs of Change Orders. Management Response and Action Plan: The Design Division and OGC-CSS agree with the finding. A. Consultant Selection Process The Design Division will modify the templates by January 2007 for the Notice of Intent, Interview and Contract Guide, and Request for Proposal to recommend evaluation of a firm s QA/QC process as an additional criterion. To avoid redundancy, it will be recommended that the criteria be used either at the NOI or short list stage and not both. As indicated, evaluation during selection will not guarantee the application of an effective QA/QC process during the project. The necessity and importance of quality control must be emphasized throughout the contract management process. This will be addressed during training. B. Contract Administration Process As mentioned under 1.B., future training will include information on types of schedules needed and recommended for contract purposes in comparison to schedules needed for day-to-day project and task management. An example progress report format was distributed as an attachment to the memo summarizing the recommendations from the follow-up to the survey contracts audit. The Design Division will modify as needed and make the example available under Reference Materials and General Guidance on the DES-CCO web site until it can be incorporated into the Engineering, Architectural and Surveying Services Manual and training developed for managing and monitoring provider s work progress. C. Provider Evaluations Process Purpose and importance of completing the provider evaluation process will be incorporated into the contract administration and project management training. Similar to the process used to develop the current provider evaluation form, DES-CCO is planning to use another TxDOT/consultant staff task force to develop a post construction evaluation that can be incorporated into the evaluation database for reference along with the provider evaluation. A task force will be initiated by December 2007 with a proposed evaluation format expected within six months of initiation. D. Errors and Omissions Policy See response to 1.D. DES-CCO will coordinate with FIN to identify the necessary steps to ensure reimbursements are properly credited. No. 3: The review of sample contracts in twelve districts and the TTA division indicate that the selection, negotiation and administration of consultant contracts do not fully comply with existing policies, as described below. A. Selection: Report of 14 August 29, 2006

12 In three of the 28 recent/active contracts reviewed, some of the payment methods stated on the approved requests to contract (Form CCO-1) and some of the evaluation criteria stated in the Notice of Intent were not applied as indicated. Not all districts have a process to track when Letters Of Interest (LOIs) are received in the district. In four of the 28 recent/active contracts reviewed, the verification of information submitted by the firms, such as references and pre-certification, was not always documented. In three of 28 recent/active contracts reviewed, the Interview and Contract Guide did not include all required information or the latest published version was not used. In two of the 28 recent/active contracts reviewed, the scoring of selection criteria was not always documented as required, and some errors in scoring were detected along with deviation from the advertised minimum scores. Guidance on the composition of the Consultant Selection Team is not detailed enough, especially when it comes to specialty contracts such as utility engineering, etc. The required number of the CST members listed in the current Engineering, Architectural and Surveying Services Manual (seems to list 3 members) also disagrees with the current version of 43 TAC 9.34 (which seems to list 4 members). Most districts reviewed had begun implementation of the new selection process directed by the Commission. B. Negotiation: In three of the 28 recent/active contracts reviewed, the negotiation process exceeded the required 30 day time limit. Documentation for a time extension was not available. In one instance, the time extension was not approved at the authorized department level. We found that available guidance on contract negotiation time extensions included in the current Engineering, Architectural and Surveying Services Manual and the current version of 43 TAC 9.37 disagrees on the signature authority for second-time extensions. The manual states that the Deputy Executive Director may grant these extensions and the TAC states that the Assistant Executive Director may grant additional extensions. We found one second time extension that was approved by the DES director instead of the Deputy Executive Director. Our review also found that the number of days permissible per extension in the above quoted guidance was not followed. The manual states that after the automatic extensions, the CRC may grant an additional 10 working days. Our review indicates that at least one time extension for negotiation of a Traffic Signal contract received a first time extension by the DES director for 30 working days instead of 10 working days. C. Contract Administration: Two of the 28 recent/active contracts included Work Authorizations executed by the districts that exceeded $1 million after numerous supplements. Two of the 28 recent/active contracts reviewed included Work Authorizations with an incomplete scope of work. Three of the 28 recent/active contracts reviewed included Work Authorizations that were issued within a few weeks or months prior to contract termination and that reflected a scope of work likely to exceed the contract termination date. We understand that this practice is also known as using place-markers, and were used to justify subsequent contract supplemental agreements that significantly extended the contracts termination dates and increased the contract amounts well beyond what was approved on Form CCO-1 and originally negotiated. Report of 14 August 29, 2006

13 Several instances of non-compliance in the management of Work Authorizations were identified. Some of these instances violate the requirements in the Contract Administration Manual, and in others, the terms of the contract. Examples are changes in payment terms reflected in a Work Authorization without execution of a supplement to the contract; increases in Work Authorization amounts without any changes in scope of work; payment type information not included in the Work Authorization; and Work Authorizations that did not include work schedules or they were not updated. One of the 28 recent/active contracts included non-engineering work in the form of developing a website to communicate preliminary engineering concepts to the public. Effect: A. Insufficient documentation puts TxDOT at a disadvantage in case anyone challenges the selection of a particular consultant firm or that TxDOT is in compliance with the TAC rules. B. Inconsistent application of the TxDOT negotiation time line policy puts the department in a weak position when challenged about its negotiation practices and/or compliance with the TAC rules. C. If TxDOT does not comply with the terms of its own contracts, then the department is at a disadvantage when enforcing these terms. Additionally, the Executive Director and his staff may have to explain these deviations to the Legislature. Recommendation: A. Selection: The Design Division should evaluate the current process and related CCO-Forms to establish which process steps are needed. After this decision is made, the division should update the Engineering, Architectural and Surveying Services Manual and synchronize it with the TAC. In addition, renaming some of the forms may emphasize that the information in the forms supports the purpose of the process. B. Negotiation: The Design Division should evaluate the current negotiation timelines and the current signature authority policy, and then update the EAS Manual and/or the TAC accordingly. C. Contract Administration: The Design Division and the Office of General Counsel should cooperate in the development of contract administration training for district/division staff to provide additional guidance for the correct use of contract supplementals and Work Authorization supplementals. In addition, we recommend that the Design Division and the Office of General Counsel more closely monitor Work Authorizations, Work Authorization supplements, and contract supplements submitted by the districts so that the inappropriate use of supplements is detected and corrective action can be suggested. We further recommend monitoring the overall dollar amount of Work Authorizations (initial amount and supplementals) to avoid circumvention of the district signature authority (currently$1 million) for Work Authorizations. Instances where this authority is exceeded should be researched, and intentional misuse should be reported to the Executive Director for inclusion in the respective District Engineer s performance evaluation. With the new $2 million limit for Indefinite Deliverable contracts, the delegated signature authority for Work Authorizations (currently $1 million) may need to be re-evaluated (possibly lowered to $500 thousand) to improve oversight of these contracts. Management Response and Action Plan: Report of 14 August 29, 2006

14 The Design Division and OGC-CSS agree with the finding. A. Selection DES-CCO will evaluate the forms used throughout the selection and management process to evaluate titles, information collected, and overall necessity. Changes will be incorporated in the Engineering, Architectural and Surveying Services Manual by December 2007 and Texas Administrative Code as updates are made. B. Negotiation DES-CCO will evaluate the negotiation timelines and update the Engineering, Architectural and Surveying Services Manual and Texas Administrative Code as updates are made. C. Contract Administration Similar to the response for 1.C., the Design Division acknowledges the need for training. Based on experience to-date, the best approach appears to be training that covers information from both a contract administration and project management perspective so that it will be useful to attending staff that have responsibilities across a range of roles. The structure of roles and responsibilities currently varies by district and division. It is important that the immediate need for training be balanced with the desire to develop a formal training program that requires significant time and resources. The Design Division will evaluate existing information developed for training/workshop purposes and will determine the feasibility of making this information available within the next 12 months to address the need for a better understanding of the correct use of supplemental agreements to contracts, work authorizations, and supplemental work authorizations. As a way to more closely monitor work authorizations and supplemental work authorizations, it is recommended that OGC-CSS develop a method by January 2007 to review a sample as they are submitted to OGC-CSS after execution. If this process reveals too many problems or does not work effectively, then lowering the work authorization amount for signature authority at the district level can be addressed. Current direction allows signature authority for work authorizations executed with an initial value below $1 million to remain at the district level through the life of the agreement. Beginning November 1, 2006, this direction will change to require Design Division review and execution of supplemental work authorizations that increase the value of a work authorization to $1 million or more. Signature authority for any subsequent supplemental work authorizations will remain with the Design Division Director for the duration of the agreement, regardless of the initial value. Closing Comments The findings and recommendations were discussed in detail in the exit conference with the Design Division Director, his staff, and the OGC-CSS Director on July 19, A briefing was also held on July 20, 2006 with the Finance Division s Accounting Director regarding the issues to enhance FIMS reporting. The Assistant Executive Director for Engineering Operations was briefed on the audit results on July 21, We thank the Construction Division, Design Division, and the Finance Division for their support, the districts staff for their cooperation during field work, as well as the District Internal Auditors who conducted the fieldwork in their districts. We also want to recognize Camille Thomason, DES-CCO, and Janice Mullenix, OGC-CSS, for all of their efforts and contributions to improving the process. Report of 14 August 29, 2006