Internal Audit Report Professional Engineering Procurement Services (PEPS) Consultant Procurement Process TxDOT Internal Audit Division
Objective To evaluate the effectiveness and efficiency of the PEPS process for contract consultant selections. Opinion Based on the audit scope areas reviewed, control mechanisms require improvement and only partially address risk factors and exposures considered significant relative to impacting operational execution and compliance. The organization's system of internal controls requires improvement in order to provide reasonable assurance that key goals and objectives will be achieved. Improvements are required to minimize existing process variation and control gap corrections that may result in potentially significant negative impacts to the organization including the achievement of the organization's business/control objectives. Overall Engagement Assessment Needs Improvement Finding 1 Finding 2 Title Monitoring of Consultant Rotation and Work Authorization Assignment Governance over CST Formation and SOQ Screening Findings Control Design Operating Effectiveness Rating X X Unsatisfactory X X Needs Improvement Management concurs with the above findings and prepared management action plans to address deficiencies. Control Environment The Professional Engineering Procurement Services Division (PEPS) supports the qualifications-based selection process of various professional service disciplines such as consultants for engineering, surveying, and architectural contracts for TxDOT. The selection process includes forming consultant selection teams (CSTs), comprised of subject matter experts assigned from the districts and divisions, to review and score consultant qualifications submitted to TxDOT based on PEPS criteria. Depending on the dollar amount of the contract, CST members must fulfill certain TxDOT requirements to participate, such as a contract procurement training and completion of a nepotism disclosure form, for contracts greater than $1 million. In addition, 43 Texas Administrative Code (TAC) 9.34 provides guidance for CSTs requiring certifications for the specific discipline under review. Once a CST has reviewed and scored consultants based on their qualifications, PEPS signs contracts with these selected consultants and publishes a ranked order listing on the PEPS external website based on final statement of qualifications (SOQ) scoring. Non-federal indefinite deliverable contracts are assigned work authorizations per the listing in accordance with TxDOT policy. The approval of the PEPS division director is required for work authorizations to be assigned out of sequence August 2017 2
Summary Results Audit testing completed resulting in management action plans. Finding Scope Area Evidence 1 2 Consultant Evaluation and Selection Process Consultant Evaluation and Selection Process Consultant Selection Team Formation Based on observation and interviews performed, it was determined that no process exists to monitor the assignment of non-federal indefinite deliverable contract work authorizations. In addition, PEPS could provide no assurance variations to the established rotational process are being detected and/or are being properly approved. Based on review of 29 SOQs and 39 CSTs training requirements, Service Centers are not adhering to established policies and procedures to procure professional services consultants: 17 of 29 (59%) SOQ screening checklists were incomplete 4 of 39 (10%) CST members had not completed required training; however, these CST members were supervised by a PEPS procurement engineer reducing exposure. Audit testing completed not resulting in management action plans. Scope Area Evidence Consultant Evaluation and Selection Process All 29 consultants reviewed were 1) not debarred by any federal or state entities and 2) found to be eligible to work for TxDOT per review of the prequalification check. Each CST for the 9 Request for Qualifications (RFQs) reviewed complied with 43 TAC 9.34, which requires certification for the discipline of the contract. Audit Scope The scope of the audit work focused on the PEPS screening and selection process for contractors procured for state or federally funded, specific or indefinite deliverable contracts, and state funded indefinite deliverable contract work authorization assignments initiated during period of March 2016 through April 2017. Scope 1: Consultant Evaluation and Selection Process - reviewed a sample of RFQs and corresponding sub-selected SOQs and consultants to assess completeness of the screening and procurement of professional services Scope 2: Consultant Selection Team (CST) Formation - reviewed sample of CSTs to validate frequency of participation and completeness of trainings, forms and any necessary approvals. August 2017 3
The audit was performed by Judit Alphonse, Albert Bourque, Alberto Calderon, Jessica Esqueda, Charlita Freeman, and Lindsay Bibeau (Engagement Lead). The audit was conducted during the period from June 5, 2017 to July 26, 2017. Methodology The methodology used to complete the objectives of this audit included: Reviewed TxDOT internal documents such as PEPS Standard Operating Procedures and manuals for the procurement of professional services to identify key procedures that were to be performed. Reviewed sections of Federal Code and Texas Code to identify applicable regulations for the procurement of consultants of professional services. Conducted interviews to discuss the process, risks, or concerns of procuring consultants for professional services. Interviews included key personnel in the PEPS Division such as the Division Director, Deputy Director, PEPS Center of Excellence Section Director, eight service center managers, and individuals in various districts and divisions that may have been on a CST. Judgmentally selected RFQs during the 3 rd and 4 th wave of FY 2016 and 1 st and 2 nd wave of FY 2017 for the respective year s top three disciplines to further test SOQs completeness and selection process requirements for professional services. Reviewed consultant selection teams from the RFQ selections to assess frequency of CST member participation. CSTs were also further reviewed for completeness of required training, forms, and any necessary approvals. Conducted interviews with other state transportation agencies to gain further understanding of their processes for procuring professional services consultants in their respective states. Background This report is prepared for the Texas Transportation Commission and for the Administration and Management of TxDOT. The report presents the results of the PEPS Consultant Procurement Process audit which was conducted as part of the FY 2017 Audit Plan. PEPS, formed in October 2013, has eight service centers across the state. Seven service centers focus on procuring contracts for the districts, while the remaining service center, which was established in April 2017, focuses on procuring contracts for the divisions. PEPS oversees the consultant selection process to ensure each CST selects consultants based solely on their qualifications. PEPS procures consultants over four waves within a fiscal year. Each wave consists of a 1) RFQ, 2) the formation of a CST from the districts and divisions, 3) receipt of consultant SOQs, 4) consultant pre-screening of SOQs by PEPS, 5) scoring of the consultants by the CSTs based on qualifications, 6) creation of a consultant ranked order listing based on scoring, and lastly 7) an executed contract with the consultants. We conducted this performance audit in accordance with Generally Accepted Government Auditing Standards and in conformance with the International Standards for the Professional Practice of Internal Auditing. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. Recommendations to mitigate risks identified August 2017 4
were provided to management during the engagement to assist in the formulation of the management action plans included in this report. The Internal Audit Division uses the Committee of Sponsoring Organizations of the Treadway Commission (COSO) Internal Control Integrated Framework version 2013. A defined set of control objectives was utilized to focus on operational and compliance goals for the identified scope areas. Our audit opinion is an assessment of the health of the overall control environment based on (1) the effectiveness of the enterprise risk management activities throughout the audit period and (2) the degree to which the defined control objectives were being met. Our audit opinion is not a guarantee against operational sub-optimization or non-compliance, particularly in areas not included in the scope of this audit. Benchmarking Discussions with state transportation agencies in three other states revealed that two states are receiving and retaining SOQs through an electronic submission system. The third state had legislative restrictions on using a stand-alone system to receive electronic submissions; however, they do receive SOQ submissions via email or CD. Electronic handling of consultant information may provide for more efficiency in the selection process by further reducing issues such as missing, late, or incorrect submissions. August 2017 5
Detailed Findings and Management Action Plans (MAP) Finding No. 1: Monitoring of Consultant Rotation and Assignment of Work Authorizations Condition The Professional Engineering Procurement Services Division (PEPS) does not have a monitoring process in place to help ensure the ranked order listing of consultant work disposition is being followed or that any change to this order is being properly approved. Effect/Potential Impact By not monitoring the consultant rotation and assignments of the work authorizations for non-federally funded indefinite deliverable contracts, the transparency and effectiveness for the disposition of work to consultants could be negatively impacted. Criteria TxDOT internal policy, Prioritization Process for Assigning Work Authorizations for Non- Federal Indefinite Deliverable Contracts, requires that PEPS assigns work authorizations based on contract rank order first and then on a subsequent rotational basis. After one rotation through the list of contracts, contract capacity (i.e., ability to take on more work authorizations per contract limits) then becomes the rotation factor for assigning work authorizations. PEPS Division Director approval is required to move out of sequential contract rank or remaining contract capacity order. Cause PEPS does not have a monitoring mechanism to track the order of the work authorization assignments. In addition, the Indefinite Deliverable (ID) capacity report, used by service center employees to determine remaining contract capacity, is updated on a daily basis based on usage as work authorizations are assigned; however, the service centers do not keep historical records of the daily reports. Without the historical records, PEPS cannot readily validate the order of the work authorization assignments. In addition, PEPS could not provide assurance that variations to the established rotational process were being detected and/or were being properly approved without recreating the contract. Evidence 8 of 8 (100%) PEPS Service Centers, as well as, the PEPS Center of Excellence noted that no monitoring of work authorization assignments were occurring during the 3 rd and 4 th waves of FY 2016 and 1 st and 2 nd waves of FY 2017 to ensure the assignments made outside of the established rotation were appropriately approved. Management Action Plan (MAP): MAP Owner: Lucio Vasquez, Deputy Division Director, PEPS MAP 1.1: PEPS Administration will provide guidance to division to promote adherence to the division s procedures on assigning work authorizations at the August 2017 Strategy Board meeting and through email communication to staff. August 2017 6
Completion Date: December 15, 2017 MAP 1.2: PEPS will develop and implement training for the management and administration of contracts and work authorizations. Completion Date: December 15, 2017 MAP 1.3: PEPS will create an Excel spreadsheet to track work authorization assignments that are out of rank. For historical purposes, a screenshot will be taken of the order of the work authorization assignments at the time the out of rank request is made. Completion Date: December 15, 2017 MAP 1.4: PEPS will work with Information Management Division (IMD) to modify PS-CAMS to incorporate a method to automatically identify and track work authorizations that are assigned out of rank within the system. In addition, PEPS will revisit the procedure titled Prioritization Process for Assigning Work Authorizations for Non-Federal Indefinite Deliverable Contracts to ensure it corresponds with the modification of PS-CAMS. Completion Date: May 15, 2018 August 2017 7
Finding No. 2: Governance over CST Formation and SOQ Screening Condition Continuous monitoring and adherence to policies and procedures does not exist for Consultant Selection Team (CST) formation and completion of assigned screening responsibilities. Effect/Potential Impact Incomplete application and enforcement of established procedures for procuring consultants and establishing CSTs can result in selecting unqualified consultants and potentially impairing the objective mindset of CSTs member in selecting consultants, respectively. Criteria The PEPS Contracting: Selection Process Manual includes the following: Solicitation Package (SP) Screening Guidance and Instructions states that the PEPS reviewer must mark each item in the checklist indicating that the SP meets the requirement, does not meet the requirement, or that the requirement is not applicable. The guidance also states that the PEPS representative must review, sign, and date each completed checklist. For contracts over $1 Million, nepotism forms must be signed by each Consultant Selection Team (CST) member. CST members are required to complete Consultant Management and Administration training for employees participating in the selection of qualifications based contracts. Lastly, TxDOT Employee Conflict of Interest Policy recommends that supervisors do not serve on a selection team with their subordinates. Exceptions to this recommendation must be approved by the PEPS Division Director and must include factors for mitigation. Cause PEPS does not centrally monitor completion of Statement Of Qualifications (SOQ) screening checklists, approval of CSTs with supervisor/subordinate relationships, and completion of required training or nepotism disclosure forms. Evidence Judgmentally sampled nine Requests for Qualifications (RFQs) during fiscal year (FY) 2016 (3rd and 4th waves) and FY 2017 (1st and 2nd waves) based on three disciplines (engineering, surveying, and architectural). There were 29 consultants/soqs and 39 CSTs identified from the judgmental sample and the following results were noted: Completeness of SOQs and Verification of Consultant Engineering License SOQs were reviewed for completeness of screening checklists and for a valid consultant professional engineering license with the State of Texas. The results were as follows: 17 of 29 (59%) SOQ screening checklists were incomplete due to missing signatures and/or unchecked boxes 1 of 29 (3%) consultants had a State of Texas professional engineering license that expired during the contract term; no work authorizations were executed during the time the license was expired. August 2017 8
Nepotism Disclosure Form Completion Six of the sampled RFQs had contracts valued over $1 Million and were reviewed for completed nepotism disclosure forms. The results were as follows: 1 of 6 (17%) RFQs did not have all required nepotism forms signed, no nepotism was identified. CST Training Requirements CST members were reviewed for completion of required training. The results were as follows: 4 of 39 (10%) CST members had not completed the required training, however these CST members were supervised by a PEPS procurement engineer, reducing exposure CST Composition Requirements Seventeen CST compositions were reviewed for supervisor/subordinate relationships. The results are as follows: Two CSTs did not obtain PEPS Division Director approval for supervisor/subordinate relationship as required Management Action Plan (MAP): MAP Owner: Lucio Vasquez, Deputy Division Director, PEPS MAP 2.1: The following tasks will be performed: PEPS will establish a process to track CST members training each wave. PEPS will revise language in the Statement of Qualifications (SOQ) cover page to ensure the prime provider remains certified over the duration of the contract. PEPS Center of Excellence Section will create a semi-annual process to review a random sample of executed contracts to determine if consultant licenses are active Completion Date: December 15, 2017 MAP 2.2: PEPS will develop a process to incorporate the use of PS-CAMS to electronically monitor the following components of the consultant selection process: Completion of the SOQ screening checklist, Nepotism forms (as required), CST member training, and Approvals (as required). PEPS will work with Information Management Division (IMD) to modify PS-CAMS to assist in tracking the elements in the consultant selection Process. Completion Date: May 15, 2018 August 2017 9
Observations and Recommendations Audit Observation (a): RFQ Receipt and Processing Efficiency Professional Engineering Procurement Services Division (PEPS) receives all consultants Request for Qualifications (RFQ) response documents in paper form via mail carriers. Upon receipt of the response documents, PEPS personnel need to scan the paper documentation into a share drive in order to make them available electronically for review by the consultant selection team (CST) member. Effect/Potential Impact By receiving all consultant documents in paper form via mail carriers, there is an increased risk of delay of receipt and misplacement of consultant documentation. Recommendation PEPS should determine an approach to acquiring an electronic submission system for future receipt of consultant responses to RFQs. Audit Observation (b): CST Selection Participation Frequency Professional Engineering Procurement Services Division does not review the composition of the Consultant Selection Teams (CST) to determine if CST members are repetitively chosen for consultant evaluations. FY 2016 7 of 214 CST members served on three or more CST teams in that fiscal year 3 CST members served on multiple CST teams concurrently in order to maintain a competitive process for consultants who have applied for multiple Requests for Qualifications (RFQs) across disciplines due to potential for preclusions. FY 2017 4 of 188 CST members served on three or more CST teams in that fiscal year 3 CST members served on multiple CST teams concurrently in order to maintain a competitive process for consultants who have applied for multiple Requests for Qualifications (RFQs) across disciplines due to potential preclusions. Effect/Potential Impact A TxDOT employee s repetitive participation in the selection process on consecutive CSTs may create a perception of a conflict of interest and could result in reduced production within their job as the time commitment can be extensive. Audit Recommendation PEPS Service Center personnel should run a report from PS-CAMS prior to each wave and review how often CST members have served on a team and if any are serving on back to back teams. August 2017 10
Summary Results Based on Enterprise Risk Management Framework Closing Comments The results of this audit were discussed with/provided to the Professional Engineering Procurement Services (PEPS) Division Director on August 21, 2017. The Internal Audit Division appreciates the cooperation and assistance received from the PEPS Division during this audit. August 2017 11