The Baseline. From the CEVM: Assurance and Data. Department of the Navy (DON) Earned Value Management (EVM) Quarterly Newsletter.

Transcription

1 The Baseline Department of the Navy (DON) Earned Value Management (EVM) Quarterly Newsletter April 2018 From the CEVM: Assurance and Data Written By: David Tervonen At its core, Earned Value Management (EVM) within the Department of Defense (DoD) involves two main principles: assurance that the contractor has a disciplined approach to program management and data needed for effective Government program management oversight. Inside this issue: From the CEVM: Assurance and Data IPM Learning Culture 2-3 IMS Forecasting Metrics: Current Execution Index IBR: The Hidden Stories 6-7 When the cost, schedule, and/or technical risks are of a sufficient concern, it is in the best interest of the Government that the contractor has an integrated approach to program management. An Earned Value Management System (EVMS) integrates inputs from the planning, scheduling, budgeting, work authorization, and cost accumulation processes with the program work breakdown structure and organizational structure. The Federal Government has adopted the Electronic Industries Alliance Standard 748 (EIA-748) as the industry standard to ensure the program management system, or EVMS, used by the contractor has the necessary level of discipline. Project Controls 8 EVM Training Materials Available Online 9 Crossword Puzzle 10 Calendar 11 Within the DoD, the Integrated Program Management Report (IPMR) is used to define the program management data needed for effective Government oversight. The IPMR Data Item Description (DID) describes the data necessary to communicate program cost and schedule information between the contractor and the Government. The IPMR DID will ensure the Government receives a reliable output of the contractor s EVMS. The reliability of the data report is vital to provide for Government program management decisions based on an accurate account of the contracted effort.

2 Page 2 Empowering Speed and Innovation through an IPM Learning Culture Written By: Matthew Wilkinson NAVAIR NAVAIR is a high-performing organization that leadership challenges to be fast, innovative, and increasing speed to the fleet (Grosklags, 2017). One aspect of how NAVAIR-4.2.3, Integrated Project Management (IPM) Division, maintains this expectation is by encouraging a learning culture. A learning culture is a collection of organizational conventions, values, practices and processes. These conventions encourage employees and organizations to develop knowledge and competence. An organization with a learning culture promotes continuous learning and believes that systems influence each other. Since constant learning elevates an individual as a worker and as a person, it opens opportunities for the establishment to transform continuously for the better. (Nabong, 2015) High-performing organizations are three times as likely to have a strong learning culture (Mallon, 2009). In industries where innovation is prized, such as military RDT&E, there is a high correlation between the drivers of innovative organizations and learning cultures (Mallon, 2009), see Table 1. This continuous learning goal has persisted from NAVAIR leadership with the development of NAVAIR University (NAVAIRU) and its nine colleges, RADM Eastburg s emphasis on a learning organization since 2007 has led to VADM Dunaway s 2013 NAVAIR Long-Range Strategy (Dunaway, 2013) and the current Navy Civilian Framework (Richardson, 2016). Value Drivers of Innovation and Learning Cultures (Mallon, 2009) Values Time Spent Learning New Knowledge and Skills Values Mistakes and Failures as Opportunities for Reflection Empowers Employees to have Influence over Tasking Rewards Learning Clearly Defines Decision-making Processes Encourage Questioning Promote Employee Responsibility for Personal Development Challenge Employees with Tasks beyond Current Abilities Sample NAVAIR Implementation High-Quality Instructional Design for Formal Classroom Offerings Training Roadmap and Expectations Brown Bags and Sharing Lessons Learned Continuous Improvement through Training Feedback (Questionnaires, Online Forms, and Examinations for Retention) Innovation Captures and Competency Initiatives Included in the Professional Development Rating for Analysts Supervisory and Process Group Alignment Documented Standard Processes (Toolkits) Program Office integration for IPM Analysis and IGS Development Processes Practical Workshops in lieu of Lecture-based Training Self-paced Workshops Competency Initiatives as Targeted Growth Opportunities Advanced Analytics to Support Customer Requests and Needs Advanced Scheduling Techniques to Support Customer Requests and Needs Table 1: NAVAIR Integrated Project Management Implementation of a Learning Culture The IPM Division has more than 180 assigned analysts. Within the IPM Division, creating a learning culture for this many analysts started with department and division leadership establishing: a training roadmap of well-designed training offerings, expectations regarding professional development, and alignment between supervisory chains (branch heads) and Subject Matter Experts (process group SMEs). First, AIR training offerings (Table 2, next page) were developed in coordination with professional instructional designers to optimize courses for learner retention with continuous improvement from course evaluations and learner feedback. This approach has led to a variety of course implementations based on the complexity of the subject matter and the optimal learning environment, including: formal classroom offerings, practical supervised workshops, and self-paced workshops. This provides our IPM teammates with the skills necessary for Earned Value Management (EVM) analysis, Integrated Government Schedule (IGS) development/maintenance and other IPM services for our customers. Next, AIR-4.2 has included professional development as one of three categories for employee ratings, recognizing the inherent value of a learning culture. Finally, this implementation creates synergies between AIR supervisors and the process groups resulting in a dialogue between analysts, developers, SMEs, and leadership that proliferates lessons learned via white papers, toolkit amendments, tool improvements, one-hour brown bag presentations, innovation captures and competency initiatives. In summary, IPM leadership was necessary to create the environment where a learning culture could flourish, resulting in the wide-ranging expertise, experience, and processes that the command values. (continued on Page 3)

3 Page 3 IPM Learning Culture cont. Of course, the knowledge, skills, and attributes of an EVM analyst or IGS developer do not provide NAVAIR value until they are implemented for project success. Therefore, establishing a learning culture was not only intra-divisional but required enculturating lessons learned and best practices across the command. AIR developed a variety of courses in coordination with AIR-4.2 s Program Performance Team that target Integrated Project Team (IPT) members, Program Office leadership, and the larger NAVAIR community for project planning and execution success. This series of courses includes Framing EVM for IPTs, Foundation for Project Success, Managing with EVM, Managing Uncertainty for Project Control, and Integrated Schedule Management Project Controls. All have been included in NAVAIRU, a command-wide initiative that provides a one-stop shop for all NAVAIR-specific training that is integrated with official human resource training records. Additionally, AIR analysts, developers, and SMEs espouse these IPM methods, benefits, and products via targeted training, customer discussions, and preparatory training before major reviews or risk assessments, such as Schedule Risk Assessments, Estimates to Complete, and Integrated Baseline Reviews. Establishing a learning culture where technical and programmatic experts can cross-train with IPM techniques produces a highly-functioning project management group across the command. Following this approach, a learning culture has been established which has resulted in an 80% increase in demand for our services, with the workforce growing from 105 in 2007 to 188 in Since 2010, AIR training is consistently rated very high, with a 94% average for internal training and a 91% average for enculturation training (Program Performance Team, 2018). The quality of the training for learner retention and an enculturation focus have been key contributors to successfully establishing a learning culture, which routinely reinforces itself and produces the benefits of high-performance and innovation resulting in increased speed to the fleet. Bibliography Dunaway, D. (2013, September). NAVAIR s Long-Range Strategy. Retrieved February 2, 2018, from Naval Air Systems Command: fuseaction=home.download&id=551 Grosklags, P. (2017) Commander s Intent. Patuxent River: Naval Air Systems Command. Mallon, D. (2009, May). Learning Culture and Innovation: How Your Organization s Learning Culture Affects Your Company s Ability to Get Ahead and Stay Ahead. Bersin & Associates. Nabong, T. A. (2015, April 7). Creating a Learning Culture for the Improvement of your Organization. Retrieved February 2, 2018, from Training Industry: Program Performance Team. (2018, February 2). Survey Data. NAVAIR-4.2. Table 2: NAVAIR Training Roadmap Richardson, J. (2016, October). Navy Civilian Workforce Framework, v1.0. Retrieved February 2, 2018, from Navy.mil:

4 Page 4 Integrated Master Schedule (IMS) Forecasting Metrics: Current Execution Index (CEI) Written By: John Collins NELO Careful Integrated Master Schedule (IMS) performance measurement is important to understanding the status of your program. The IMS provides an ever-changing window into the contractor s progress (or lack thereof). After all, not only is your IMS the backbone of earned value metrics, the IMS is also your early warning system for potential problems. How the contractor executes the baseline gives us the basis of all earned value metrics. However, as you get further away from your Integrated Baseline Review (IBR) or your contract falls further behind schedule, the ability to forecast the upcoming work accurately becomes paramount. This is where the Current Execution or CEI metric becomes valuable. CEI is simply an IMS metric that measures how accurately the program is forecasting and executing to its forecast from one month to the next. In other words, what did the contractor tell me he was going to finish this month (in last month s IMS) and how well did he do it! It measures the ability to forecast from month to month vice the ability to execute the baseline plan. Current Execution Index = # of tasks completed in the month as forecast in prior month # of tasks forecast to complete in the month per the prior forecast The National Defense Industrial Association (NDIA) has set the standard for industry at 75% efficiency. A program that is doing well is always executing to their forecast. Those programs that are struggling are not. One of the real benefits of using CEI is an increased program emphasis on ensuring the accuracy of the forecast schedule. After all, if the contractor can t forecast what they are going to do next month how well do you think they are forecasting system delivery? Typical monthly CEI tracking metrics look something like this: In this example, we can see that over the past six months, the contractor has failed in all months to meet the 75% target. (continued on Page 5)

5 Page 5 Current Execution Index (CEI) cont. The previous chart can be broken down even further by identifying problem Control Account Managers (CAMS) or the Work Breakdown Structure elements. Both the contractor s Organizational Breakdown Structure (OBS) and Contract WBS are required to be traceable to relevant effort in the IMS by the Integrated Program Management Report (IPMR) Data Item Description (DID). A breakdown of the current month s performance by CAM could be presented in a simple table like the one below. CAM Number Forecasted to Finish Number Actually Finished Number Missed Percentage Hit Rogers % Wilson % Walters % Richards % Whitman % Williams % Parker % Barton % Walker % Danvers % Totals: % In this example, CAMs Rogers, Wilson, Walters and Richards are failing to meet the 75% goals, and Rogers is the worst performer by far. Use of the CEI metric at this level, identifies both positive and negative performance issues and drives ownership and accountability behaviors at the contractor that are necessary for program success. If you find that one or two CAMs are usually responsible for the contractor failing the forecasting metric, it s a good bet that the CAM or CAMs may be experiencing difficulty and not have a good handle on how his/her work will proceed.

6 Page 6 Integrated Baseline Review (IBR): The Hidden Stories Written By: Victor Thomas NAVAIR As one of the Department of the Navy s (DON) most experienced gate keepers of the IBR process, the NAVAIR Cost Estimating and Analysis Department s Integrated Project Management Division (AIR-4.2.3) discovered a hidden story in capturing, maintaining and trending IBR data. Within the Analysis Process Group (APG) and Management Systems and Training Process Group, an IBR database has been established to capture IBR results including: the Control Account Managers (CAMs) summary slides (~980 slides to date) along with data from 83 different programs, 19 defense contractors at 28 different locations from IBRs covering the past ten years. This information is used to monitor contractor IBR history, prepare for future IBRs, and use in EVM analysis. The hope is that this tool will enable analysts to be proactive with the contractor by identifying potential risk factors early (before the next IBR event) and examining potential areas of concern while developing monthly analysis. The goal was to populate this database and use it to understand the findings from previous IBRs with the ability to sort, trend and analyze by contractor/site etc. Data is gathered from the IBR documents including the Appendix F (IBR Summary Form) and CAM Interview Summary to populate the database: IBR Database: Populated using data and findings from CAM summary slides Common Findings were established to create some similarities amongst the data, reduce the total number, and create a meaningful way to categorize the risks. The risks are bucketed into one of these common findings when input in the database. Based on extensive IBR history and lessons learned, a list of Typical IBR Findings had already been established and was used to create the common findings list. List of Common Findings (continued on Page 7)

7 Page 7 IBR: The Hidden Stories cont. The development of the common findings is credited in part to the IBR training provided to analysts and IPT s in preparation for an upcoming event. Based on extensive IBR history and lessons learned, a list of Typical IBR Findings had already been established and was used to assemble the common findings list. This enables the analysts to track and trend the effectiveness of the contractor s corrective actions after the IBR event is deemed closed and compare with current issues found during monthly IPM analysis. Below is a sample of how the data is used to support the IPM analyst while preparing for an SRA: As a time saving measure, a proposal was put forward for the SRA to be performed internally, providing results for review and approval. The IBR database was used to understand the Contractor and site IMS risk during past IBR s for evaluation of this proposal. After finding significant historical issues with the health of schedules, it was recommended not to allow for the acceptance of an internal SRA from that site/contractor. Below is another example of the data that can be gathered using our Excel raw data export tool: This chart illustrates the results of many IBR s at a particular site. The inadequate budgets finding was described the most at this site during IBR s. However, after examining the overall risk (as categorized by our established risk criteria), cost and schedule carry the most risk at this particular site. This is still a new product in the early stages, but it has helped us better understand the obscure hidden story issues seen in analysis before the IBR (based on past history) as well after the IBR, addressing the propensity of effective corrective actions. This data can be used: In preparing for follow-on reviews and IBR training In understanding site/contractor behavior To assist with understanding monthly IPM analysis To understand the effectiveness of the contractor s corrective action plan suggested action(s)

8 Page 8 Project Controls Written By: Andrew Wargo SSP The development of a finely tuned process for project planning and delivery and rigorous program management practices is paramount for successful delivery on all capital projects. The key to achieving this, although straightforward, is simultaneously both unassailable and inaccessible to many organizations: project controls. Project controls may seem too simple; however, successful planning and completion of a project require an understanding of project scope, divided into manageable pieces, combined with a realistic schedule of discrete effort, and integrated with insight into the resources available. While many companies are adept at obtaining contracts for projects of staggering proportions, one specific thing enables the delivery of each of those projects and will enable the delivery of future projects: the company is willing to assume risk due to its confidence in project execution. As such, execution risk falls on the company and its partners, not the customer, in the event of overruns or delays. Similarly, understanding the scope desired within the required time frame is paramount to successfully executing resources. Unbounded execution of resources without a plan can have significant consequences. Accordingly, the fundamental concept in a disciplined project controls environment is to begin with the end in mind and to plan from the beginning. Project controls help to identify deviations from the plan along the way, add structural stability to the management of the project via change control, and contribute to effective risk and opportunity management by integrating the analysis of each into management decisions. Each major portion of large projects technologies, capabilities, or pieces of construction requires management at all levels. Ultimately the configuration controls, change controls, coordinated limits, and management of the overall project must be informed by a rigorous system of integrated planning, measurement, change control, management decisions, and customer engagement. Customer engagement and trust on massive projects hinge upon the reputation of the performing organizations and disciplined discussions of performance that rely on project controls to inform and drive sound decisions at all levels. Begin with the end in mind. -Stephen Covey

9 Page 9 EVM Training Materials Available Online Defense Acquisition University (DAU): EVM 101 Fundamentals of Earned Value Management In a virtual classroom environment, professionals learn additional information about earned value management (EVM), which is introduced in the prerequisite course. The course summarizes the language, data reports, metrics, graphs, and management processes associated with EVM as they apply to DoD acquisition management. Professionals also learn the processes related to the Performance Measurement Baseline, the Integrated Baseline Review, and the 32 Guidelines prescribed in Section 2 of the Electronic Industries Alliance Standard (EIA-748) for EVM systems. Finally, professionals evaluate and compute basic EVM metrics and EVM metric -based estimates at completion. EVM 202 Intermediate Earned Value Management Professionals taking this course work as members of an integrated product team for an ACAT I program. In the context of integrated program management, participants review, develop, and experience the Earned Value Management (EVM)-related processes associated with requirements generation, acquisition strategy development, Request for Proposal development, source selection, risk management, Integrated Baseline Review, and analysis during program execution. EVM 262 EVMS Guidelines and Compliance The Earned Value Management System (EVMS) guidelines and compliance course provides knowledge needed to review integrated management systems and to determine their compliance with the Electronic Industries Alliance EVMS (EIA 748) standard. Course material, individual exercises, and group exercises review in-depth the 32 EIA 748 EVMS guidelines and the processes associated with approval and surveillance of contractor and government integrated management systems. EVM 263 Principles of Schedule Management This course provides knowledge needed to interpret network schedules required by DoD policy and the Electronic Industries Alliance (EIA) 748 Earned Value Management System (EVMS) Standard. Course material, individual and group exercises demonstrate the schedule development, maintenance, and analysis processes. The exercises reinforce: the Precedence Diagram Method of scheduling; schedule analysis using a schedule assessment model to analyze a complex; 700-line Microsoft Project network schedule; and schedule risk analysis utilizing Monte Carlo simulation. Students will be required to create a Microsoft Project network schedule. Other training and certifications are available from sources to include, but not limited to: the College of Performance Management, AACE International, and Project Management Institute.

10 Page 10 Crossword Puzzle

11 Page 11 Calendar April May June Su M Tu W Th F Sa Su M Tu W Th F Sa Su M Tu W Th F Sa EVM Events 30 Apr - 3 May NDIA Integrated Program Management Division Meeting Federal Holidays 28 May Memorial Day 30 May - 1 Jun EVM World 2018, Fort Lauderdale, FL ASN(RD&A) Events 9 Apr The Athena Project (Deckplate Innovation Event) at Sea Air Space 11 Apr Small Business Facebook Live Event at Sea Air Space 12 Apr HASC Seapower Subcommittee Hearing 14 Apr LCS 17 Indianapolis Christening 17 Apr SASC Seapower Subcommittee Hearing 21 Apr LPD 27 Portland Commissioning 25 Apr Investing in the National Security Innovation Base 2 May Modular Open Systems Summit Dec 18 Edition s Crossword Answers: This publication was created by the Naval Center for Earned Value Management (CEVM) with contributions from the Systems Commands. Please contact us at for any questions on the content of this publication. For more information on EVM within the Department of the Navy: