PART THREE: Work Plan and IV&V Methodology (RFP 5.3.3)

PART THREE: Work Plan and IV&V Methodology (RFP 5.3.3) 3.1 IV&V Methodology and Work Plan 3.1.1 NTT DATA IV&V Framework We believe that successful IV&V is more than just verification that the processes are performed correctly and validation that the product is correct. This approach leaves several critical gaps specifically, in proactive risk management, performance management and stakeholder management. Even if the system is correct and is built correctly, it can fail during implementation if it does not meet performance goals due to a lack of business process improvements, does not comply with new legislation, fails to support the interface requirements of outside stakeholders, or is rejected by users because of ineffective organizational change management. We protect our customer s interests with our holistic approach that expands the IV&V viewpoint past the processes, products and artifacts to encompass the bigger program assurance perspective. As your IV&V vendor, NTT DATA will use our IV&V tool NAVV which addresses: Project Management. Project management assessments focus on verifying that the formal software life cycle methodologies, including Waterfall, Unified Process and Agile, are followed. We have extensive experience with all types of development approaches and are knowledgeable on the benefits and risks associated with it. We integrate standards, best practices and metrics with our expertise in effective project management techniques into a NAVV. We will use this model for our assessments of the management controls, practices, processes, procedures, tools and methods used by development vendors in areas such as project integration, scope, time, cost, quality, staffing, communications, and risk/issue management. Our goal is to identify potential problems as early as possible to verify the project is on track to meet your implementation scheduled and to make recommendations for efficient use of funds. Our team also provides guidance and training on PM processes. Product Management. We perform IV&V assessments throughout the SDLC to verify that documents, artifacts, and products meet business and technical requirements. We will evaluate products for correctness, completeness, reliability, and maintainability at each step in the development process. Correctness means the product satisfies requirements; completeness signifies all required functions are implemented the program life cycle; reliability indicates the final product can be expected to perform its intended function without error or failure; and maintainability requires that the developed program products be designed to facilitate and simplify life cycle maintenance and modifications. Performance Management. While the product management perspective focuses on what the system is required to do in terms of functional support, the performance management viewpoint focuses on the envisioned outcome of the system in terms of metrics-based strategic goals and objectives. These objectives represent the broad policy priorities and the services provided to constituents, and they serve as the basis for measuring the results in terms of the defined business outcomes. This focus on business process improvement produces measurable, quantifiable benefits derived by the new system implementations. We also assess overall project performance, to determine the probability of future project progress and success, and identifying impediments to successful project completion. Risk Management. At project initiation, we perform a risk/gap analysis based on our experience with multi-vendor project integration efforts and their risks. From this assessment, we will identify and prioritize critical risk factors that may keep the project from achieving stated objectives. Based on the impact and probability assessment, we suggest 2015 NTT DATA, Inc. October 07, 2015 1

proactive and executable mitigation strategies or actions to either prevent a risk from occurring (reduce probability) or to reduce the impact if it were to occur. We will monitor all risks and issues throughout the project life cycle. Stakeholder Management. As part of our holistic approach to IV&V, we extend our assessments beyond developer plans to observe stakeholder and end user involvement and commitment, executive level support of the program, definition of roles and responsibilities between supporting organizations, MOA s and SLA s with outside stakeholders. The approach to organizational change management for field implementation, and the "marketing" of the new system to stakeholders, are also evaluated as all of these factors can turn into potential risks impacting project success. Our independence from the development effort allows us to provide a fresh viewpoint identifying subtle issues that may be overlooked or ignored by those who are working closely on the project on a day-to-day basis. It also allows us to pay close attention to the communication, relationships and involvement among stakeholders, project teams and functional business representatives. We measure stakeholder buy-in and the effectiveness of knowledge transfer to assure the success of the project. The NAVV framework includes the structure, workflow, processes, templates, and checklists required to provide an objective assessment throughout the project life cycle and insure that our team has the organization and knowledge to be productive on day one. NAVV is based on industry best practices, metrics, and a proprietary database of lessons learned and findings from previous engagements. This combination provides visibility, accountability and fact-based decision making while promoting systems that deliver business benefits within budget and on schedule. NAVV is tactical; for example, it helps validate that the product built matches the requirements. It is also strategic, assuring the proper alignment of the project with organizational goals and strategy. This is because NAVV is not just reactive (measuring against baselines and progress) but also proactive by assuring that future resources, funding and requirements are appropriately aligned for upcoming development activities and modules. Our proactive and independent approach, as compared to an auditing approach, results in fewer delays, reduced cost, higher product quality, and improvement of the development process itself. Most importantly, it increases the likelihood that the delivered system will provide the expected benefits and outcomes. 3.1.2 NTT DATA IV&V Assessment Methodology Included in our NAVV framework is our repeatable five-step IV&V assessment process that assures the artifact readiness before an assessment is started. We perform a quick-check to identify any initial critical risks, and then we perform a methodical, detailed analysis. From this analysis, we develop recommendations and prioritize our findings and improvements in our peer-reviewed reports. The NTT DATA team uses standard task processes and workflow, so we are organized and prepared from the start, saving our clients valuable time and money. The five steps of our assessment process (which is used for both IV&V reports and vendor deliverable reviews) are: Planning and Research. During this phase, we develop the scope of work for the activity describing the specific activities, resources, artifacts and participants required for the review. Through meticulous research, we identify a baseline understanding of the current developer's artifacts in respect to the particular task (what has been done to date). This initial short, focused analysis promotes the developer's readiness for assessment and identifies potential project risks. Supporting information such as standards and guidelines 2015 NTT DATA, Inc. October 07, 2015 2

are assembled. The NTT DATA team modifies the applicable NAVV templates and checklists as required to meet the assessment requirements. Assessment. We conduct the assessment by comparing the artifacts to the relevant requirements, applicable standards, and industry best practices to determine whether artifacts conform to defined requirements and whether the challenges faced during solution implementation processes are either resolved or mitigated. Activities in this phase typically include evaluation, review, inspection, analysis, assessment, and testing of products and processes. Using interviews, observations, document reviews, product reviews, product walk-throughs, and product demonstrations we evaluate a variety of factors including project management and technical factors such as cost, schedule, quality, standards, compliance, risk, stakeholder involvement, knowledge transfer, viability of technical components and performance improvement metrics. This phase is crucial, because, in it, we dissect the overall system from multiple perspectives in a variety of technical and management areas in an objective consideration of business and performance goals. The key outputs of this phase include a list of documented observations and improvement recommendations based on objective evidence. Analysis. We identify findings and risks based on an in-depth analysis of the assessment activities. These findings and risks form the foundation for corrective or preventive actions necessary to see to it that project deliverables meet requirements. If we identify a critical issue, we will provide a "heads up" with the caveat that recommendation for solutions is still forthcoming. During this phase, our team meets to assess the findings and risk and their potential impact from a system-wide viewpoint. If necessary, the NTT DATA team has a deep pool of technical expertise to provide assistance in resolving complex system integration issues and deriving optimal solutions. Our team will develop objective recommendations that focus on assuring project success. Report. We understand that it is vitally important to provide our findings and recommendations in a format that allows the systems integrator to rapidly integrate them into the development process or product. We prioritize our findings in terms of their potential impact, their relevance, and the audience so that the appropriate development team members can focus on the important issues first. We use a variety of methods for reporting findings, risks, and recommendations, including formal reports such Deliverable Reviews, Status Reports, management briefings, e-mail alerts, status updates, issues log, risk register, and an IV&V deficiency log. These reports and logs provide a continuous, up to date status at the executive or summary level to monitor progress toward the resolution of findings and recommendations. All reports are peer reviewed before we release them to you. In addition, all IV&V deliverables, processes, plans, and applicable reference material will be available upon your request. Close. Once an IV&V assessment is accepted by the Commonwealth, all actions are transferred to tracking reports and logs as applicable. We do this so that all findings are tracked until resolution. 3.1.3 Compliance with Governance and Standards NAVV is based on a number of industry standards and best practices. The bedrock of these standards includes the IEEE and ISO standards for software engineering projects (IEEE 1012: Standard for Software Verification and Validation (V&V) and ISO/IEC 12207: Systems and Software Engineering Software Life Cycle Processes). We further apply full project management practices as described in the PMBOK Guide, as well as CMMI for Development. The NAVV model is customized for each project to ensure applicable Federal and State statutes and administrative rules and guidelines are followed. 2015 NTT DATA, Inc. October 07, 2015 3

3.2 Work Plan No one wants a failed IT system, especially when public funds are involved and failure could become a headline. Effective IV&V is a critical component to program assurance. Our independent assessments provide a fresh look and a second opinion verifying that systems are being built correctly and validating that the right product is built. The work plan for a specific IV&V project will be developed using the general work plan activities described in the following sections. 3.2.1 Project Initiation and Planning Phase The combination of an experienced team and the head-start provided by NAVV will allow us to quickly establish our IV&V team so that we can become productive immediately in the Initiation and Planning Phase of a project. At contract start, we will establish the IV&V project environment needed to efficiently perform our work. The most important part of the Project Initiation and Planning stage of this project will be the finalization of the IV&V project management plan, including the IV&V work breakdown structure and the schedule based on the projects implementation schedule. The IV&V PM plan also provides a detailed description of our IV&V services including methodologies, strategies, standards, and approach for executing each of the IV&V activities within the SOW. A critical component of our project initiation is an initial assessment. This is an assessment of the current environment to: Review the Project Management Plan and the project's schedule, resources, tasks, structures, processes, and procedures, to assess the overall maturity of the project s planning Evaluate sufficiency of project scope, staffing, budget and objectives, including alignment to legislative intent and its impact on agency processes and services Assess the project s organizational, governance, and communication processes Review the impact of any procurement requirements and deadlines and the ability to verify compliance during project execution Review solicitation, procurement, or contract documents to verify that, at a minimum, the evaluation criteria and requirements are clearly articulated and are consistent with project objectives, and that the obligations of the Vendor, subcontractors, and external staff are clearly defined and aligned to facilitate success. The assessment results are documented in an Initial Review Report (IRR). The report will describe findings in terms of the project context and will include data on each area analyzed, detailed recommendations on how to improve project processes, and recommended next steps. 3.2.2 Execution Phase Our approach to providing IV&V services focuses on verifying that quality is built in throughout the project life cycle to answer the question Is the product built right? Our ongoing IV&V assessments result in exception-based reports that objectively outline the strengths and weaknesses of projects and provide recommendations for improvement or correction as appropriate. We verify that the vendor s deliverables and services supply the required business and solution artifacts and possess the following attributes: completeness, accuracy, traceability, consistency, readability, and testability. The NTT DATA team will recommend acceptance or rejection of an artifact as a condition for moving forward with subsequent project activities and we will support formal gate reviews as part of the decision-making process. 2015 NTT DATA, Inc. October 07, 2015 4

The NTT DATA team will also provide validation activities to answer the question Is the right product being built? We will validate that the projects achieve your vision and objectives and that the production system satisfies business requirements. As we perform evaluation of the processes, artifacts, work products, milestones and deliverables we will: Verify that the vendor's products and services supply the required artifacts and are accurate, complete, and in compliance with industry standards, project standards, and best practices. Verify that all life cycle deliverables are feasible and consistent with the overall project direction. This includes requirements documents, requirements traceability matrixes, data conversion and migration plans, design specifications, technical architecture, coding standards, acceptance criteria, and training materials Monitor and evaluate the effectiveness of organizational change management, knowledge transfer, customer satisfaction and acceptance Evaluate project management performance with schedule effectiveness metrics and earned value measures identifying strengths, variances from best practices, and deficiencies and providing recommendations for course corrections Summarize the results of ongoing project monitoring and provide findings and recommendations for improvement of project management and processes. Verify compliance of system support activities such as configuration and change management, quality assurance, testing, security, implementation, disaster recovery and sustained engineering with approved plans Support formal gate reviews and make recommendations for the project s ability to meet future milestones and deliverables and the readiness of the project and agency to proceed to the next phase. Validate that project supports the stated objectives and meet contractual and functional requirements. We will provide information on potential risks, mitigation, issues, and assessments of project status in the form of monthly project assessment reports, weekly project assessment updates, phase gate assessment reports and deliverable review reports. These reports will provide timely feedback to the decision makers about the status of critical components of the projects. We also provide, on an exception basis, special briefings should a critical issue require immediate attention. All findings, risks and issues will be tracked in consolidated logs, and prioritize based on agreed on guidelines and reported in the monthly project assessment report. In the report we will also discuss project changes and associated impacts, summarize findings, risks and issues and identify corrective actions and opportunities for improvement. The report will highlight all completed and upcoming deliverables. We also provide dashboards summarizing key characteristics for the project, as well as a summary of IV&V activities for the month. 3.2.3 Project Closure Phase The preparation and delivery of our final report and a project archive will complete the formal deliverables for the IV&V project. This information will include lessons learned, best practices, and challenges and opportunities for use on future similar initiatives. We will also confirm that all data related to the IV&V project is organized in a central repository, providing a complete achieve of all project deliverables, status reports and special studies prepared by the IV&V team 2015 NTT DATA, Inc. October 07, 2015 5