RETTEW Associates, Inc. QUALITY ASSURANCE SURVEILLANCE PLAN for (client project/rfp number) (date of proposal submission) Enclosure 1 This proposal includes data that shall not be disclosed outside the government and shall not be duplicated, used, or disclosed in whole or in part for any purpose other than to evaluate this proposal. If, however, a contract is awarded to this offeror as a result of or in connection with the submission of this data, the government shall have the right to duplicate, use, or disclose the data to the extent provided in the resulting contract. This restriction does not limit the government s right to use information contained in this data if it is obtained from another source without restriction.
Table of Contents 1.0 INTRODUCTION... 3 1.1 PURPOSE... 3 1.2 PERFORMANCE MANAGEMENT APPROACH... 3 1.3 PERFORMANCE MANAGEMENT STRATEGY... 4 2.0 GOVERNMENT RESOURCES... 5 2.1 THE CONTRACTING OFFICER... 5 2.2 THE CONTRACTING OFFICER S TECHNICAL REPRESENTATIVE... 5 2.3 TECHNICAL ASSISTANT... 5 2.4 COMPETENCY TECHNICAL AUTHORITY... 5 3.0 ROLES AND RESPONSIBILITIES... 5 3.1 THE CONTRACTING OFFICER... 5 3.2 THE CONTRACTING OFFICER S TECHNICAL REPRESENTATIVE... 5 3.3 TECHNICAL ASSISTANT... 6 3.4 COMPETENCY TECHNICAL AUTHORITY... 6 4.0 METHODOLOGIES TO MONITOR PERFORMANCE... 6 4.1 SURVEILLANCE TECHNIQUES... 6 4.2 CUSTOMER FEEDBACK... 6 4.3 ACCEPTABLE QUALITY LEVELS... 7 5.0 QUALITY ASSURANCE DOCUMENTATION... 7 5.1 THE PERFORMANCE MANAGEMENT FEEDBACK LOOP... 7 5.2 MONITORING FORM... 7 6.0 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT... 8 6.1 DETERMINING PERFORMANCE... 8 6.2 REPORTING... 8 6.3 REVIEWS AND RESOLUTION... 8 7.0 ENCLOSURES... 9 ENCLOSURE 1: SERVICE SUPPORT FOR PERFORMANCE STANDARDS... * ENCLOSURE 2: DATA PERFORMANCE STANDARDS... * ENCLOSURE 3: INCENTIVES... * ENCLOSURE 4: SURVEILLANCE ACTIVITY CHECKLIST... * ENCLOSURE 5: QASP QUALITY ASSURANCE MONITORING FORM... * 2
1.0 Introduction This Quality Assurance Surveillance Plan (QASP) is pursuant to the requirements listed in the Statement of Work (SOW) entitled ( Client Program Support Services here). This plan sets forth the procedures and guidelines that RETTEW Associates, Inc., ( RETTEW ) will use in ensuring the required performance standards or services levels are achieved by the contractor. 1.1 Purpose The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the Contractor is meeting the performance standards/quality levels identified in the SOW and the Contractor s Quality Management Plan (QMP), and to ensure that the government pays only for the level of services received. This QASP defines the roles and responsibilities of all members of the Integrated Project Team (IPT), identifies the performance objectives, defines the methodologies used to monitor and evaluate the contractor s performance, describes quality assurance documentation requirements, and describes the analysis of quality assurance monitoring results. 1.2 Performance Management Approach The SOW structures the acquisition around what service or quality level is required, as opposed to how the contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken by the (client) to monitor and manage the contractor s performance to ensure the expected outcomes or performance objectives communicated in the SOW are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether expected outcomes for the project are being achieved by the contractor. Performance management represents a significant shift from the more traditional Quality Assurance (QA) concepts in several ways. Performance management focuses on assessing whether outcomes are being achieved and to what extent. This approach migrates away from scrutiny of compliance with the processes and practices used to achieve the outcome. A performance-based approach enables the Contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A results focus provides the Contractor flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved and/or the desired performance levels are being met. 3
1.3 Performance Management Strategy The contractor is responsible for the quality of all work performed. The contractor measures that quality through the contractor s own Quality Control Plan (QCP). Quality control is work output, not workers, and therefore includes all work performed under this contract/delivery order regardless of whether the work is performed by contractor employees or by subcontractors. The contractor s QCP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the PWS. The contractor will develop and implement a performance management system with processes to assess and report its performance to the designated government representative. This QASP enables the government to take advantage of the contractor s QCP. 1.3.2 The government representative(s) will assess performance using Contractor Performance Assessment Reporting System (CPARS) to determine how the contractor is performing against communicated performance objectives. The CPAR assesses a contractor s performance, both positive and negative, and provides a record on a given contract during a specified period of time. More information pertaining to CPARS can be found at: http://www.cpars.csd.disa.mil/cparsfiles/pdfs/dod-cpars-guide.pdf Each assessment will be based on objective data (or measurable, subjective data when objective data is not available) supportable by program and contract management data. CPAR performance expectations will be addressed in the Government and contractor s initial post-award meeting. Potential sources of data may include, but are not limited to, the following: Contractor operations reviews Status and progress reviews Production and management reviews Management and engineering process reviews (e.g. risk management, requirements management, etc.) Cost performance reports and other cost and schedule metrics Other program measures and metrics such as: Measures of progress and status of critical resources Measures of product size and stability Measures of product quality and process performance Customer feedback/comments and satisfaction ratings Systems engineering and other technical progress reviews Technical interchange meetings Physical and functional configuration audits Quality reviews and quality assurance evaluations Functional performance evaluations Subcontract Reports 4
2.0 Government Resources 2.1 The Contracting Officer (CO) A person duly appointed with the authority to enter into contracts and make related determination and findings on behalf of the Government. The CO for this contract is (Name, Code). The CO will be designated in the resulting contract. Cos are designated via a written warrant which sets forth limitations of authority. 2.2 The Contracting Officer s Technical Representative (COTR) A person duly designated to be the CO s authorized representative to assist in administering the contract. The COTR will be designated in the resulting contract and individual task orders. The limitations of authority are contained in a written letter of designation. 2.3 Technical An individual designated in writing by the COTR authorized to monitor and review contractor compliance with individual task order requirements. The limitations of authority are contained in a written letter of designation. 2.4 Competency Technical Authority (CTA) An individual designated by the Competency Manager responsible to establish, monitor, and approve technical standards, tools, and processes in conformance with higher authority policy requirements, architectures and standards. 3.0 Roles and Responsibilities 3.1 The Contracting Officer (CO) The Contracting Officer ensures performance of all necessary actions for effective contracting, ensures compliance with the terms of the contract and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer that assures the Contractor receives impartial, fair and equitable treatment under the contract. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the Contractor s performance. 3.2 The Contracting Officer s Technical Representative (COTR) The COTR is responsible for technical administration of the contract and assures proper Government surveillance of the contractor s performance. The COTR is not 5
empowered to make any contractual commitments or to authorize any changes on the Government s behalf. Any changes that the contractor deems may affect contract price, terms or conditions shall be referred to the Contracting Officer for action. 3.3 Technical The TA is responsible for the technical review and contract compliance by the contractor to individual task orders. The TA is not empowered to make any contractual commitments or to authorize any changes on the Government s behalf. Any changes that the contractor deems may affect contract price, terms or conditions shall be referred to the Contracting Officer for action. 3.4 Competency Technical Authority (CTA) The CTA is responsible to ensure that the execution of competency technical standards, tools, and processes adheres to standards and policy that provide a range of technically acceptable alternatives with risk and value assessments. Additionally, the CTA will be responsible for the following: definition of Contract tasks and subtasks, review of requested delivery order work orders, and sourcing concurrence for delivery orders; responsible for key position resume qualification review/approval; and identification by specific contract task and assigned to a Level 3 OBS, as approved by the Level 2 Department Director. 4.0 Methodologies to Monitor Performance 4.1 Surveillance Techniques In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the government to evaluate Contractor performance when appropriate. The primary methods of surveillance are: Random monitoring, which shall be performed by the COTR designated inspector. 100% Inspection Each month, the COTR, shall review the generated documentation and enter summary results into the QS Monitoring Form. Periodic Inspection COTR typically performs the periodic inspection on a monthly basis. 4.2 Customer Feedback The Contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external 6
indicator of the success and effectiveness of all services provided and can be measured through customer complaints. Performance management drives the Contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the COTR, as opposed to the Contractor. Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COTR. The COTR will accept those customer complaints and investigate using the Quality Assurance Monitoring Form Customer Complaint, identified in Attachment 4B. Customer feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints. 4.3 Acceptable Quality Levels The Acceptable Quality Levels (AQLs) included in Attachment 4A, Performance Requirements Summary Table, for Contractor performance are structured to allow the Contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For certain critical activities such as those involving Program, Identity, and PACS Management, the desired performance level is 100 percent, however the AQL is five error or less for each deliverable in the QASP requirements summary based on the required services (tasks/deliverables). 5.0 Quality Assurance Documentation 5.1 The Performance Management Feedback Loop The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the SOW and is assessed using the performance monitoring techniques shown in Attachment 4A. 5.2 Monitoring Form The government s QA surveillance, performed by the COTR, will be reported using the monitoring form in Enclosure (5). The form, when completed, will document the government s assessment of the Contractor s performance under the contract to ensure that the required results are being achieved. The COTR will retain a copy of all completed QA surveillance forms. 7
6.0 Analysis of Quality Assurance Assessment 6.1 Determining Performance The Government shall use the monitoring methods cited to determine whether the performance standards/service levels/aqls have been met. If the contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. 6.2 Reporting At the end of each month, the COTR will prepare a written report summarizing the overall results of the quality assurance surveillance of the Contractor s performance. This written report, which includes the Contractor s submitted monthly report and the completed quality assurance monitoring form (Attachment 4B), will become part of the QA documentation. It will enable the government to demonstrate whether the Contractor is meeting the stated objectives and/or performance standards, including cost/technical/scheduling objectives. 6.3 Reviews and Resolution The COTR may require the Contractor s project manager, or a designated alternate, to meet with the CO and other government IPT personnel as deemed necessary to discuss performance evaluation. The COTR will define a frequency of in-depth reviews with the Contractor, including appropriate self-assessments by the Contractor; however, if the need arises, the Contractor will meet with the COTR as often as required or per the Contractor s request. The agenda of the reviews may include: Monthly performance assessment data and trend analysis Issues and concerns of both parties Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis Recommendations for improved efficiency and/or effectiveness Issues arising from the performance monitoring processes 8
7.0 Enclosures Enclosure (1) - Service Support For Performance Standards Enclosure (2) - Data Performance Standards Enclosure (3) - Incentives Enclosure (4) - Surveillance Activity Checklist Enclosure (5) QASP Quality Assurance Monitoring Form 9
SERVICE SUPPORT FOR PERFORMANCE STANDARDS Description (For Each SOW Element) Performance Standard and Acceptable Quality Level (AQL) Surveillance Method/Measure Incentives Provide Program Planning, Analysis, and Assessments 90% acceptable on first 100% acceptable on subsequent submission to Government Technical Report (Study/Service(s)) Reviewed by Program Technical See enclosure 3. Provide Conference and Program Definition 90% acceptable on first 100% acceptable on subsequent submission to Government Technical Report (Study/Service(s)) Reviewed by Program Technical See enclosure 3 Provide Quantitative Resource Analysis 90% acceptable on first 100% acceptable on subsequent Technical Report (Study/Services(s)) Received by Program Technical See enclosure 3 Provide Program Execution Evaluation 90% acceptable on first 100% acceptable on subsequent Technical Report (Study/Services(s)) Received by Program Technical See enclosure 3 Provide Training Program Development and Management 90% acceptable on first 100% acceptable on subsequent Technical Report (Study/Services(s)) Received by Program Technical See enclosure 3 Provide (additional as SOW directed) 90% acceptable on first 100% acceptable on subsequent Technical Report (Study/Services(s)) Received by Program Technical See enclosure 3 Enclosure 1
DATA PERFORMANCE STANDARDS Description Standard AQL Surveillance Measure Incentives Contractor s Progress, Status Report Submission is in accordance with the Data Item Description (DID) cited in the Contract Data Requirements List (CDRL) 95% acceptable on first submission to Government 100% acceptable on subsequent submission to Government 100% Inspection by Program Technical See enclosure 3 Technical Report (Status/Services(s)) Submission is in accordance with the Data Item Description (DID) cited in the Contract Data Requirements List (CDRL) 95% acceptable on first submission to Government 100% acceptable on subsequent submission to Government 100% Inspection by Program Technical See enclosure 3 Funds and Manpower Expenditure Report Submission is in accordance with the Data Item Description (DID) cited in the Contract Data Requirements List (CDRL) 95% acceptable on first submission to Government 100% acceptable on subsequent submission to Government 100% Inspection by Program Technical See enclosure 3 Enclosure 2
INCENTIVES Assessment Period Acceptable Performance Definition How Measured Incentives Evaluation Period 1 All measurement areas rated Satisfactory. See below Midpoint of the assessment period (6 month) evaluation using the CPARs format covering the previous 6 months (+) award of future orders.* (-) Not awarded future orders unless there is remediation with Contracting Officer s Representative (COTR)* Evaluation Period 2 All measurement areas rated Satisfactory. See below. Midpoint of the assessment period (18 month) evaluation using the CPARs format covering the previous 12 months. (+) award of future orders.* (-) Not awarded future orders unless there is remediation with Contracting Officer s Representative (COTR)* Evaluation Period 3 All measurement areas rated at least Satisfactory. Three or more measurements rated either Very Good or Exceptional. See below. Midpoint of the assessment period (30 month) evaluation using the CPARs format covering the previous 12 months. (+) award of future orders.* (-) Not awarded future orders unless there is remediation with Contracting Officer s Representative (COTR)* Evaluation Period 4 All measurement areas rated at least Satisfactory. Three or more measurements rated either Very Good or Exceptional. See below. Midpoint of the assessment period (42 month) evaluation using the CPARs format covering the previous 12 months. (+) award of future orders.* (-) Not awarded future orders unless there is remediation with Contracting Officer s Representative (COTR)* Evaluation Period 5 All measurement areas rated at least Satisfactory. Three or more measurements rated either Very Good or Exceptional. See below. Midpoint of the assessment period (54 month) evaluation using the CPARs format covering the previous 12 months. (+) award of future orders.* (-) Not awarded future orders unless there is remediation with Contracting Officer s Representative (COTR)* Enclosure 3
* The Government will not award orders unless all regulatory requirements are met and the contractor meets the acceptable performance definition. All SOW/CDRL tasks, including SOW/CDRL sub-tasks, will be assessed focusing on the following. Quality of Product or Service Assess the contractor s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives. Assess how successfully the contractor meets program quality. Schedule Assess the contractor s adherence to the required delivery schedule by assessing the contractor s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions. Cost Control Assess the contractor s effectiveness in forecasting, managing, and controlling contract cost. Is the contractor experiencing cost growth or underrun? If so, discuss the causes and contractorproposed solutions for the cost overruns. For contracts where task or contract sizing is based upon contractor provided person-hour estimates, the relationship of these estimates to ultimate cost should be assessed. In addition, the extent to which the contractor demonstrates a sense of cost responsibility, through the efficient use of resources in each work effort should be assessed. Business Relations Assess the timelines, completeness and quality of problem identification, corrective action plans, proposal submittals, the contractor s reasonable and cooperative behavior, effective business relations, and customer satisfaction. Management Assess the contractor s success with timely award and management of subcontracts, including whether the contractor met small/small disadvantage and women-owned business participation goals. Discuss the extent to which the contractor discharges its responsibility for integration and coordination of all activity needed to execute the contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; communicates appropriate information to affected program elements in a timely manner. Assess the contractor s risk mitigation plans. If applicable, identify any other management areas that are unique to the contract. Other areas Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere. The evaluation ratings are as follows: Exceptional Performance meets contractual requirements and exceeds many to the Government s benefit. The contractual performance of the task and sub-task being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective. Very Good - Performance meets contractual requirements and exceeds some to the Government s benefit. The contractual performance of the task and sub-task being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective. Satisfactory - Performance meets contractual requirements. The contractual performance of the task and sub-task contain some minor problems for which corrective actions taken by the contractor appear or were satisfactory. Enclosure 3
Marginal - Performance does not meet contractual requirements. The contractual performance of the task and sub-task being assessed reflect a serious problem for which the contractor has not yet identified corrective actions. The contractor s proposed actions appear only marginally effective or were not fully implemented. Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the task or sub-task contains a serious problem(s) for which the contractor s actions appear or were ineffective. Enclosure 3
SURVEILLANCE ACTIVITY CHECKLIST Description Surveillance Method/Measure Date Planned Date Completed Quality Schedule Cost Control Business Relation Mgmt. Other Customer and Provide Program Planning, Analysis, and Assessments Government subject matter experts, quarterly feedback TBD Customer and Provide Conference and Program Definition Government subject matter experts, quarterly feedback TBD Customer and Provide Quantitative Resource Analysis Government subject matter experts, quarterly feedback TBD Provide Program Execution Evaluation Customer and Government subject matter experts, quarterly feedback TBD Provide Training Program Development and Management Customer and Government subject matter experts, quarterly feedback TBD Provide (additional as directed by SOW) Customer and Government subject matter experts, quarterly feedback TBD Enclosure 4
QASP QUALITY ASSURANCE MONITORING FORM REQUIRED TASKS/SERVICES: SURVEY PERIOD: SURVEILLANCE METHOD (Check): LEVEL OF SURVEILLANCE (Check): ANALYSIS OF RESULTS: Documented Performance Measurement Rate (# of errors): (AQL = 5 or fewer errors) Reviewer s Performance Assessment (Check): Narrative of Performance During Survey Period: PREPARED BY: DATE: Enclosure 5