BOB CATLIN Director, Contracts, Pricing & Estimating and Risk Management Northrop Grumman Aerospace Systems

Transcription

1 PROPOSAL QUALITY, PRESENTED TO SOUTH BAY NCMA OCTOBER 19, 2017 BOB CATLIN Director, Contracts, Pricing & Estimating and Risk Management Northrop Grumman Aerospace Systems

2 Agenda DA AGEN Definition/Purpose/Keys to Success Built in Quality for Proposal Development Cost Estimating System Requirements Cost Estimating System Quality and DCMA/DCAA Support Improvement Recommendations and Best Practices Government & Industry Collaboration

3 What is Quality? Quality:...a measure of excellence or a state of being free from defects, deficiencies, and significant variations, brought about by the strict and consistent adherence to measurable and verifiable standards to achieve uniformity of output that satisfies specific customer or user requirements. [Business Dictionary.com] Quality is Personal We acknowledge the impact of our actions We accept accountability. We deliver excellence We all make a difference... Meeting requirements the first time, every time. [Video message from Wes Bush, Northrop Grumman Chief Executive Officer and President October 2015]

4 Purpose of Cost Proposal Quality Focus Often, quality concerns involve completeness, structure, and accuracy of pricing; compliance with all RFP requirements; traceability; and documentation of assumptions Government and Industry must have the ability to define the attributes of a high-quality proposal prior to submission. What determines proposal quality? If we can t define it, we can t measure it. And if we can t measure it, we can t manage it

5 SS SUCCE NT DEVEL OPME SAL PROPO TO KEYS Keys to Proposal Development Success Keys for successful proposal practices Early communication of expectations Clear roles and responsibilities Robust Policies and Procedures Comprehensive proposal schedule Rigorous proposal planning Controlled proposal baseline Thorough RFP decomposition and early compliance assessment Complete and compliant outline of all proposal deliverables Establish Proposal CONOPS Efficient processes and tools Early Engagement of Contracts, Pricing and GSC Estimating Continual Focus of Compliant Proposal Products Focused Global Supply Chain (GSC) effort RAA review and approval of all proposal deliverables 5

6 WORK HOME YOUR DO PROAC TIVE, BE CUSTO MER Know Your Customer Be Proactive, Do Your Homework YOUR KNOW ON YOUR RADAR Internal controls and recent/prior audit findings should identify areas of concern Review DCMA website for latest news, trends, focus areas Internet search on Business System training, anecdotes, etc. STANDARD REFERENCE MATERIALS Develop a simple readable system description Who, What, Why, When, Where and How Map Requirements (DFARS, FAR, CAM) to Contractor Polices and Procedures Utilize DCMA/DCAA terminology (DFARS, FAR, CAM) Internal FREQUENT POSITIVE COMMUNICATION Develop a ground swell of understanding of the Business System Requirements External TARGET IDENTIFICATION Know the DFARS requirements Timely flow down requirements to suppliers Monitor Internal Controls for escapes Identify escapes relative to Estimating System Requirements Early interaction with DCAA whenever possible Develop relationships with DCAA and DCMA be on their speed dial 6

7 BUILT IN QUALITY FOR PROPOSAL DEVELOPMENT

8 Quality Starts Day One A review of BOE examples that relate to the proposal and a completed BOE checklist, enables the proposal lead to effectively communicate expected BOE standards to the writers BOE writers are more likely to turn out high quality material the first time, appropriate examples are provided at the kickoff meeting Good, solid examples also enable the writers to work more independently, allowing the proposal lead to focus on quality control

9 P Understanding the Request For Proposal (RFP) A L ( RF P OS T F O R P RO U ES N G T H E R EQ N DI S TA UN D ER Read and understand all sections of the RFP (all functional team members) Clarify requirements Identify constraints which could materially impact cost Review for inconsistencies or conflicting provisions in RFP Specific terms and conditions impact the cost Identify Contract type and incentives Review for Economic Price Adjustment (EPA) clauses Identify Warranties, Liquidated damages, Limitation of liability Coordination between Contracts and Pricing Contract type Fee structure Terms that could impact cost/price Proposal format, content and structure Section L of RFP 9

10 DA AGEN NG MEETI OFF E KICK SAMPL Sample Kick Off Meeting Agenda Welcome Management Overview (importance of bid) Program Background (customer view) Proposal Plan (deliverables, schedule, org chart, etc.) Win Strategy Technical Approach/Offering Management Approach/Offering Cost/Pricing Approach/Offering Proposal Operations (daily meetings, reviews, training, etc.) Admin details (Networks, security, etc.) Summary 10

11 NT DEVEL OPME SAL PROPO TO STEPS ARY SUMM Summary Steps to Proposal Development Built in Quality requires broadly understood common compliance and quality requirements and practices Establish Proposal Team Contracts, Pricing & Estimating (P&E) and GSC Estimating Review/Develop Request for Proposal (RFP), Pricing and Estimating Instructions and Contracting Geometry and Terms & Conditions Ensure proposal development schedule has sufficient time for development and delivery of compliant proposal products. Release RFE and Estimating Kickoff Estimating Instructions Labor Material Intercompany Work Orders (IWO) Begin Proposal Development Labor Basis of Estimate Subcontract Estimates and Price Cost Analyses Bill of Material Other Direct Costs IWO 11

12 Readability Major Areas of Proposal Quality Focus Requirements Estimate Logic Traceability Standard Compliant Development Tools and Templates Sample Proposal Template BOE Training and BOE Checklist Standard Cost Reports Quality Assured Deliverables Well Written Proposal Narratives Quality Well Substantiated BOEs Reliable Compliant Cost Reports

13 Quality Objectives Verify RFP compliance: Every requirement set forth in the RFP must be met. Verify Quality Checklist compliance: Provided as Appendix A of this handbook, compliance with this checklist should be a key goal of every proposal. Each draft of the proposal should be reviewed against this checklist. Assess traceability: All recommendations set forth in the proposal must be directly traceable to the RFP. The requirements matrix is the primary tool used to assess traceability. Assess BOE quality: Direct labor, materials, indirect costs and subcontractor fees must be solidly defended. Assess readability: Basic readability is a valuable asset, particularly during the walk-through. Run-on sentences and excessive use of jargon and buzz words detract from clarity and reduce readability.

14 Validation Methods The major validation methods are: Quality-Focused Peer Review: Well trained, experienced estimators often make the best reviewers (provided they do not review sections they themselves wrote or contributed to). Peer reviews may be performed throughout the cost proposal development process. Proposal Adequacy Checklists: The Checklist is the primary adequacy checklist; other checklists may be created or modified to apply to a specific proposal effort. Adequacy checklists should be reviewed throughout the development process. The final step in a proposal is the proposal adequacy checklist. Cost Proposal Quality Review: Prior to submission a full cost proposal review should be conducted by proposal quality reviewers. This method is the most effective vehicle for attaining a consistently high level of quality on proposals for large and/or complex projects.

15 D AWAR THRU TTAL SUBMI SAL PROPO Summary Steps From Proposal Submittal thru Award FROM STEPS ARY SUMM Management Reviews i.e. Business Unit, Division, Sector, and Corporate Risk and Opportunities Identifying elements of Risk and Opportunities Checklist Execution Review all applicable proposal checklists. Fill out the Proposal Review Checklist with page references and manager approval Cost Volume Generation Ensuring FAR Compliant Cost Volumes Submittal to Customer Method of submittal i.e. Electronic, paper copies Government Evaluation Tech Evaluations, Price Cost Analysis Reports, Audit Assists Pre-Sweep Provide cost and pricing data not previously submitted Negotiations Memorandum of Negotiations Counter Offers Sweep After Handshake Final Price Budget Advice Submittal to Cost Management to develop budget 15

16 COST ESTIMATING SYSTEM REQUIREMENTS

17 Mapping of DFARS Cost Estimating System Requirements to Major Estimating System Components 13. Provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences. 12. Require management review, including verification of the estimating and budgeting policies, procedures, and practices. Estimate Approval 5. Provide for adequate supervision throughout the estimating and budgeting process. 6. Provide for consistent application of estimating and budgeting techniques. 7. Provide for detection and timely correction of errors. 8. Protect against cost duplication and omissions. 10. Require use of appropriate analytical methods. 14. Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process. Internal Controls Estimate Review Roles Estimate Preparation Responsibilities 16. Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as basis to reach a fair and reasonable price. 17. Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement. Estimate Data Sources 1. Establish clear responsibility for preparation, review, and approval of cost estimates and budgets. 2. Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets. 3. Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the established procedures. Training 4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets. 9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate. 11. Integrate information available from other management systems. 15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable.

18 SS PROCE E MAINT ENANC TY QUALI M SYSTE ATING ESTIM Proposal Estimating System Quality Process PLAN Procedures that will produce compliant, high quality cost proposal products and support Develop training materials (including TINA, Proposal Adequacy Checklist (PAC), Estimating System and CAS Compliance) Establish Internal Control Processes DO Communicate Requirements Provide training, maintain attendance logs Develop cost proposals in accordance with procedures In process quality and compliance checks BUILT in QUALITY ACT Monitor and status Estimating System Health - Scorecard (assessments by discipline, cost element, IPT and general areas of concern Respond to Buying Command and Audit Community Concerns (RFIs) Communicate Self Assessment Audit findings Implement Corrective Actions In response internal Corrective Action Requests CHECK Peer reviews and book checks Execute Proposal Adequacy Checklist (PAC) Manager reviews / approves PAC Cost Proposal Reviews Before Submission Perform Self-Assessment Audits After 18

19 M SYSTE ATING ESTIM Estimating System Commitment to Quality Continually improve Estimating System Practices Develop robust procedures which govern proposal development. Proposals are developed by trained staff and reviewed by management prior to release. Estimating System Notices and formal training sessions promote continuous process improvement. Improvements are fueled and characterized by open and frequent communications within Industry, the Buying Commands, DCMA and DCAA. Focus on Improving the Following Areas: The best preparation for any audit begins with a quality proposal 19

20 TS IMPRO VEMEN SAL PROPO ON Emphasis on Deliverables & Process Improvements ASIS EMPH Deliverable Improvements: Cost Volume Basis of Estimates Price Cost Analysis Reports Supplier & IWO Cost or Pricing Data Adequacy Documentation of Commerciality and Competition Integrity of Data Sources Data Analytics-Descriptive and Predictive Analysis & Matrices Process Improvements: Standard Template Cost Proposal Quality Review BOE Review Checklist/Tool PCAR Template, Checklist, Trng GSC and IWO Review Checklist Modifications Identifying potential Improvements in the Sweep/RFI Tool Reducing time between Negotiation Handshake and Submitting Certificate of Cost and Pricing Data 20

21 COST ESTIMATING SYSTEM QUALITY AND DCMA/DCAA SUPPORT

22 ELL NUTSH IN A POLICY MS SYSTE ESS BUSIN THE The Business Systems Policy in a Nutshell 22 The Cognizant Contracting Officer (CO) in consultation with DCAA and/or Functional Specialists shall: Determine the acceptability of the contractor s business systems Approve or disapprove the systems Pursue correction of significant deficiencies Withhold Payments in accordance with the requirements of , Contractor Business Systems, until significant deficiencies are corrected

23 ED DEFIN ENCY DEFICI ICANT Significant Deficiency Defined SIGNIF 23 Significant Deficiency means a shortcoming in the system that materially affects the ability of officials of the Department of Defense to rely upon information produced by the system that is needed for management purposes Standard Definition in all business system related clauses

24 ESS PROC ATION DETE RMIN EM SYST NG ESTI MATI COST Cost Estimating System Determination Process DCAA Office Reviews Recent Proposal Audit Reports for trends or recurring issues DCAA Conducts Proposal Audit DCAA Opens an Estimating System Deficiency Action Factfinding DCAA Management Review of SOCAR Contractor provides a Rejoinder DACO Provides Initial Determination to Contractor DCAA Conducts Proposal Audit DCAA review of Factfinding Results DCAA Approval to Release SOCAR to Contractor DCAA Reviews Rejoinder and closes Audit or Passes SOCAR onto DACO Contractor Provides a Rejoinder DCAA Conducts Proposal Audit DCAA Office development of Statement of Conditions and Recommendations (SOCAR) DCAA holds Exit Conference with Contractor to Deliver SOCAR DACO reviews DCAA SOCAR Determines if to Proceed Final Determination By DACO Possible Holdbacks if not Closed OUT

25 DCAA Evaluation of Contractor Business Systems Contractor business systems and related internal controls, including the estimating system, are the first line of defense against waste, fraud, and abuse. Examples of deficiencies reported by DCAA included the contractor s failure to: Adhere to its disclosed estimating practices Consider historical experiences Support cost estimates Eliminate duplicated costs in proposals

26 CONTR ACTOR ) AND COMM AND, DCAA NG (BUYI TION COMM UNICA Encourage Early Communication (Buying Command, DCMA, DCAA and Contractor) EARLY URAGE ENCO FAR Exchanges With Industry Before Receipt of Proposals. c) Agencies are encouraged to promote early exchanges of information about future acquisitions. An early exchange of information among industry and the program manager, contracting officer, and other participants in the acquisition process can identify and resolve concerns regarding the acquisition strategy, including proposed contract type, terms and conditions, and acquisition planning schedules; the feasibility of the requirement, including performance requirements, statements of work, and data requirements; the suitability of the proposal instructions and evaluation criteria, including the approach for assessing past performance information; the availability of reference documents; and any other industry concerns or questions. 26

27 TS IMPRO VEMEN SS PROCE NT OP/IM PLEME DCMA Proposal Support DEVEL Commerciality Determination Directorate established Jan 2015 Develop listing of Commercial Suppliers for Off Cycle Assistance NS TO CONVE RSATIO ATE GENER Integrated Cost Analysis Team (ICAT) Technical and Pricing Teams very interested in further engagement Leverage DCMA in process, discuss areas of concern and brainstorm off cycle 27

28 IMPROVEMENT RECOMMENDATIONS AND BEST PRACTICES

29 STAKEHOLDER IMPROVEMENT RECOMMENDATIONS Buying Command Reasonable proposal preparation timelines considering FAR requirements Mature SOW and other pertinent data for inclusion with RFP (No TBDs) Continued coordination with DCAA/DCMA and Contractor for Proposal RFI Responses Communication to DCAA of Outstanding PCARs Close Out Actions Copies of DCAA reports to contractor for identification of DCAA issues Contractors Ongoing improvements in communication with external customers, notify Buying Command of Vendor Information Schedule Universal RFI Log to be shared with Contractor/Buying Command/Audit Agencies Continued push for Higher Level Estimating Increased Offline Estimate requirements development commerciality, PCARs 29

30 NS RECO MMEN DATIO T IMPRO VEMEN SS PROCE AND RS PLAYE RY PRIMA STAKEHOLDER IMPROVEMENT RECOMMENDATIONS DCAA Better detailed explanation of audit concerns in Proposal Exit Briefings examples of specific areas of concern Acceptance of Higher Level price analysis Bill of Material Communication to Buying Command of Outstanding PCARs Utilization of alternative reasonableness determination methods DCMA Written Request for Information during proposal audits with established bilateral response time Continued active partnership with Contractor, DCMA ICAT and Commerciality Directorate Rates and Factors Direct Labor, Cost Estimating Factors, and Indirect Rates Quick release of Forward Pricing Rate Recommendations (FPRRs) Drive to Forward Pricing Rate Agreements (FPRAs) 30

31 CES PRACTI BEST Sole Source Process Best Practices Shoulder to Shoulder (S2S) meetings throughout the proposal process: Preliminary data to help define the DRFP (SOW language, Specs., etc.) Understand the DRFP and propose changed for the final RFP Understand the final RFP and coordinate Ground Rules and Assumptions Avoids unexpected costs before proposal submittal Discussion of draft TD/BOEs (if able to send early) Fact Finding Questions after submittal, before negotiations Opportunity to discuss definitization schedule early in process Receipt of Draft RFP s for proposal efforts >$50M, 30 days prior to final RFP enables: Opportunity for questions/comments/redlines prior to release of the final RFP Pre-coordination and agreement on various contractual documents (SOW, CDRLS, etc.) and proposal artifacts (TD/BOE s format/level) Pre coordination of required resources in support of the proposal process Global Supply Chain to proceed with draft RFP s to suppliers Identification of bill of material (BOM) requirements 31

32 (CONTINU ED) CES PRACTI BEST Sole Source Process Best Practices RFPs immediately disseminated to key program stakeholders upon receipt from customer Proposal baseline schedule Assign dedicated Proposal Lead and proposal team shortly after RFP (preferably before if DRFP available) Develop and disseminate proposal schedule at the proposal kickoff Maintain close supplier coordination for timely and responsive supplier proposals or quotes Maintain daily stand-ups with clear requirements of what is due when Plan in required reviews (cost, peer, functional, etc.) Schedule sign-offs early Weekly Tag-Ups With Customer: Detailed discussion of the status of all proposals (in work and pending negotiation/definitization) NGAS Program Team: Detailed run through of all outstanding proposal activity with Program VP, Proposal Leadership, other stakeholders 32

33 (CONTINU ED) CES PRACTI BEST Sole Source Process Best Practices Bi-Monthly Contracts Collaboration Meetings with program office and contractor stakeholders Face-to-Face interaction (Contractor/USG) Open communication related to pressing contractual issues Opportunity to receive Voice of the Customer (VOC) Proposal Quality, Open Contract Actions, etc. Established new processes and tools to address DCAA data requirements and response time for proposal audits For complex proposals, offer to the DCAA a proposal walk-through to become familiar with the proposal components Established regular communication meetings with DCAA leadership to precoordinate audit, clarify expectations, address open actions, identify proposal activity and maintain constructive dialogue Established early delivery of labor data to DCAA, either with proposal delivery or shortly thereafter, for use in labor data sources sampling by DCAA Employed closed system portal for transmission of proposal and cost/pricing data for DCAA (virtual drop box) Embedded a sector P&E compliance official with the program to improve our communication and enhance our response (2 nd set of eyes) 33

34 GOVERNMENT & INDUSTRY COLLABORATION

35 GOVERNMENT & INDUSTRY COLLABORATION Government & Industry Collaboration COST IMPROVEMENT PRODUCTS TEAMS (CIPTS) Cost Improvement Products Teams (CIPTs) NRO ODNI Joint Space Cost Council Air Force - Space NASA Aviation Industry Partners 35

36 KEY CIPT INITIATIVES/TOPICS Key CIPT Initiatives/Topics Higher Level Estimating Data Driven Basis of Estimates Common Cost Estimating Approaches: Cost Improvement, Obsolescence, other Standardized Basis of Estimate Format for Customer Agency Use of Pre-FPRP Discussion on Cost Estimating (Shoulder to Shoulder) High Level Bill of Materials Cost Risk Assessment Approaches & Techniques Data Reporting Government/Industry working together to update CSDR, 1921 forms, and EVM reporting Supporting Customer Cost Agencies in developing cost positions 36

37 Proposal Cycle Time (AKA Velocity) Process Changes Shoulder to Shoulder (S2S) meetings throughout the proposal process Receipt of Draft RFP s for proposal efforts >$50M, 30 days prior to final RFP RFPs immediately disseminated to key program stakeholders upon receipt from customer Agreed to Proposal baseline schedule Weekly Tag-Ups with Customer Bi-Monthly Contracts Collaboration Meetings with program office and NGAS stakeholders Established new processes and tools to address DCAA data requirements and response time for proposal audits Government & Industry Collaboration Cost Improvement Products Teams (CIPTs)

38 N PREPA RATIO Actual Proposal Cycle Time (AKA Velocity) Improvement SAL PROPO Draft RFP X/XX/14 Submittal X/XX/15 Negotiations Start XX/XX/15 Contract Award X/XX/16 LRIP X ~ 28 Months from Draft RFP to Submittal LRIP X+1 Revised RFP XX/XX/16 Submittal X/XX/17 Negotiations Start X/XX/17 Contract Award X/XX/17 ~ 8 Months from Revised RFP LRIP X+2 Draft RFP X/XX/17 Final RFP X/XX/17 Submittal X/XX/17 Negotiations Start XX/XX/17 Contract Award XX/XX/17 51 Days: FRFP to Proposal ~ 7 Months from Draft RFP Continue to Reduce Time and Investment from RFP to Award 38

39 Find the right mix of skills when selecting team. Considerations Along the Way Include narratives that can act as bridges to readability. Costs must include detailed analysis and relevance to the estimate. Make Sure It All Plugs Together COMMUNICATE!

40 THAT S ALL FOLKS!!!