AUDIT OF THE CANADIAN FORCES HEALTH INFORMATION SYSTEM (CFHIS) PROJECT

Transcription

1 AUDIT OF THE CANADIAN FORCES HEALTH INFORMATION SYSTEM (CFHIS) PROJECT October (CRS) FV1.0_CFHIS Rpt_Oct04_CH

2 NOTICE OF CAVEAT TO THE READER This review is not intended to assess the performance of contractors; rather it is an internal review of processes and practices within the DND/ CF. Chief Review Services

3 SYNOPSIS In January 2000, the Department of National Defence and the Canadian Forces (DND/CF) initiated a major healthcare restructuring project known as Rx2000. The goal of Rx2000 is to develop and implement solutions for reported healthcare deficiencies thereby improving the standard of care provided to CF members at home and abroad. One of the areas of concern included the manner in which health information was being collected, stored, shared or utilized 1. The Canadian Forces Health Information System (CFHIS) is intended to address this issue. The CFHIS is a $108 million project (excluding GST) to integrate a series of Commercial Off-the Shelf (COTS) applications that are to provide an enterprise-wide, health information system. The CFHIS project will be implemented in three phases over a five-year period. At the time of the audit, project completion was targeted for Due to the cost, schedule and significance of the CFHIS, Chief Review Services (CRS), in partnership with KPMG, conducted an audit of the first phase of the project to determine if appropriate processes and controls were in place to manage and successfully deliver the CFHIS. The CRS audit was conducted from August to October This report presents the results of the audit, which focused on the following areas: The effectiveness of the project s management control framework; The soundness of the project s risk management strategies; and The project s use of information for decision-making and reporting. Although the first phase of the CFHIS project was forecast to be on schedule and on budget at the time of the audit, over the long-term, fundamental weaknesses in key areas will make it difficult for the CFHIS project management office (PMO) to meet final deliverables within the prescribed budget and timeframes. Specifically, this audit report addresses weaknesses in the implementation of core project management processes, the application of risk management practices and the availability of project documentation and information. Without structured and rigorous project processes and outputs to manage the overall CFHIS project, critical interdependencies and risks will be overlooked and may significantly affect projected costs and schedule. The key recommendations of this audit are targeted at achieving a higher level of diligence and rigour in the management of the CFHIS project in order to increase the probability of success in downstream phases. Management action plans provided by the ADM(IM)/DGIMPD and the CFHIS project management team demonstrate constructive attention to the recommendations contained in this report and we are generally satisfied that corrective actions have been, or will be, implemented. It is important to emphasize that developing and implementing project plans and processes are only the start of achieving the required level of rigour. Equally important is ensuring that project plans and processes are maintained, communicated, used and monitored. Recommendations and corresponding management action plans are presented in matrix format at Annex F of this report. 1 Additional information on the Rx2000 initiative can be found at the following website: Chief Review Services i/i

4 TABLE OF CONTENTS RESULTS IN BRIEF... I INTRODUCTION... I BACKGROUND... I OVERALL ASSESSMENT... III KEY RESULTS...V KEY RECOMMENDATIONS...VI MANAGEMENT ACTION PLANS...VII ANNEX A AUDIT OBJECTIVES, SCOPE AND METHODOLOGY... A-1 ANNEX B CONTROL OBJECTIVES FOR INFORMATION AND RELATED TECHNOLOGY (COBIT )... B-1 ANNEX C COBIT CONTROL OBJECTIVES BY TB INTERNAL AUDIT POLICY GROUPING... C-1 ANNEX D CFHIS ORGANIZATIONAL CHART (SIMPLIFIED VERSION)... D-1 ANNEX E GLOSSARY OF DND ACRONYMS AND PROJECT MANAGEMENT TERMS...E-1 ANNEX F MANAGEMENT ACTION PLANS...F-1 Chief Review Services

5 INTRODUCTION RESULTS IN BRIEF The successful implementation of information technology (IT) projects is a recognized challenge within the IT industry and the federal government, including the Department of National Defence and the Canadian Forces (DND/CF). In response to this challenge, the Treasury Board Secretariat (TBS) developed the Enhanced Framework for the Management of IT Projects. To strengthen IM project management support and oversight within the DND/CF, the ADM(IM)/Director General Information Management Project Delivery (DGIMPD) Division was created in April 2000, devoted exclusively to the delivery of IM projects. One of the ADM(IM)/DGIMPD s primary responsibilities is the implementation of a Project Delivery Management (PDM) discipline designed to assure the successful delivery of major IM projects through the rigorous application of performance management processes and project management techniques 2. It is within this context that Chief Review Services (CRS) conducted an audit of the first phase of the Canadian Forces Health Information System (CFHIS) project. BACKGROUND The DND/CF has determined that a requirement exists for an integrated, enterprise-wide, health information system for the collection, provision and sharing of health information as required by health care providers, CF members and decision-makers throughout the DND/CF. The CFHIS involves the integration of a series of Commercial Off-The-Shelf (COTS) applications and ancillary products that, upon completion, are to provide full health information capabilities and support including: Master Patient Index; Laboratory; Diagnostic Imaging; Pharmacy; Order Entry and Results Reporting; Dental and Medical charting; and Clinical and Executive decision-making support. The CFHIS project is linked to many other on-going health care reform initiatives and projects within the DND/CF, particularly Rx2000, the umbrella project to develop and implement solutions for reported CF health care deficiencies. Additional information on the Rx2000 initiative can be found at the following website: 2 Director Information Management Project Planning and Control (DIMPPC) Project Delivery Management Concept of Operations Version 1.1, 29 June Chief Review Services I/VII

6 In November 2002, a contract was awarded to a prime contractor to develop and implement the first phase of the CFHIS project with options for subsequent phases based on performance. At the time, this three-phased, $108 million project (excluding GST) was targeted for full implementation by February Phase I work began immediately following contract award. Overall responsibility for the project resides with the CFHIS project management office (PMO). This includes overseeing and contributing to prime contractor deliverables in addition to coordinating and managing all internal DND/CF project management and departmental activities required to implement and deliver the CFHIS. A breakdown of the approved 3 CFHIS budgeted project funding requirements by phase is presented in Figure 1 below. $M 50.0 CFHIS Project Funding Requirements - $108M (excluding GST) $47M $33M $28M 0.0 Phase I - Dec 02 to Feb 04 Phase II - Mar 04 to Feb 06 Phase III - Mar 06 to Feb 08 Escalation Contingency Internal DND/CF Contract-VP Contract-FP Figure 1: Budgeted CFHIS Project Funding Requirements Source: CFHIS Budget and Escalated Costing (v3.15) The CFHIS contract consists primarily of a fixed-price (FP) portion with a variable-price (VP) portion for completing approved work packages. Work packages are required for delivering system functionality and for project management support that are not fixed-price deliverables (i.e., interfacing with other departmental systems or integrated project team (IPT) effort). At the time of the CRS audit, actual project expenditures totalled $9.8 million (excluding GST) and total commitments were $8.2 million (excluding GST). In total, 74 per cent (or $18 million) of Phase I budgeted project costs (excluding contingency and escalation) were spent or committed as at 08 September The CFHIS PMO was also forecasting that Phase I would be delivered on time and on budget without a requirement for contingency funds. 3 On 25 July 2002, the CFHIS Program Management Board (PMB) granted departmental approval for the CFHIS project at a revised indicative cost of $108 million (excluding GST). The project was formally approved on 21 November Chief Review Services II/VII

7 Due to the overall cost, schedule and importance of the CFHIS, CRS, in partnership with KPMG, conducted an audit of the first phase of the project during the first year of its five-year implementation plan. The audit was conducted from August to October 2003 to determine whether appropriate processes and controls were in place to manage and successfully deliver the CFHIS. Industry accepted project management and IT governance frameworks were utilized to develop the audit methodology such as the Project Management Body of Knowledge (PMBOK ) and the Control Objectives for Information and related Technology (COBIT ). COBIT is a generally applicable and accepted standard for good IT security and control practices. It provides a reference framework for management, users and IT audit and security practitioners to manage the risks associated with IT-related processes. COBIT also provides a measurement tool to evaluate and rate the maturity of processes from non-existent (0) to optimized (5). Additional information on the PMBOK and COBIT can be found at the following websites: and OVERALL ASSESSMENT We encountered a team of people who are clearly dedicated to the success of the CFHIS project. However, the audit team has rated the project overall, at a Level 2, using the COBIT Maturity Model. A Level 2 maturity rating, on a scale of 0 to 5, indicates that processes in place for the project are repeatable but intuitive. Essentially, the CFHIS PMO is placing a relatively high degree of reliance on the knowledge of individuals, and less on clearly defined, documented and communicated processes. Processes follow a regular pattern but many are only partially designed or implemented. This is evident in the COBIT maturity ratings summarized by individual project process in Figure 2, presented further on in this section. At the same time, team member experience and expertise is not being fully leveraged. The CFHIS PMO is largely making use of the prime contractor s processes rather than developing and implementing overall project processes that integrate both prime contractor deliverables and internal DND/CF activities. Since the majority of Phase I activities are fixed-price deliverables, the CFHIS PMO is currently able to manage with this approach. However, project complexity will increase in successive phases of the project, as many more concurrent activities are required to be completed. The fixed-price portion of the contract will decrease from 65 per cent of total Phase I budgeted project costs (excluding contingency and escalation) to 53 per cent and 27 per cent in Phases II and III respectively. Unless structured and rigorous processes and outputs are implemented to manage the overall CFHIS project, critical interdependencies and risks will be overlooked and may significantly affect project cost and schedule. It is recognized that at this stage in the project, the observations can only speak to potential risks and impacts. However, there are certain attributes, or red flags, which have some similarity to those associated with a previous DND/CF IM project that was eventually cancelled after four years and a total expenditure of $65 million. In 1993, the RIIP/RAMS (Reserve Automated Management System, a component of the Reserve Integrated Information Project) project budget was set at $79.3 million and was scheduled for completion in five years. In 1997, it was concluded that the project could not deliver the key functional capabilities within the prescribed budget or timeframes. It was Chief Review Services III/VII

8 subsequently cancelled and CRS was requested to perform an audit of the circumstances 4. Although the RIIP/RAMS project faltered in large part due to an unproven procurement strategy, weaknesses in the application of project management systems and a lack of follow through on risk mitigation strategies exacerbated the situation. Another significant issue was the absence of clear project documentation that resulted in poor corporate memory and difficulty for new project members and stakeholders. While it is certainly not the intention to be alarmist, these same weaknesses are, at least in some degree, evident with respect to the CFHIS project and should not go unattended. Due to the known risks related to large, multi-year IM projects and the increasing complexity of the CFHIS project, weaknesses in project processes should be addressed immediately. The project processes shown in Figure 2 below, should have at a minimum, a COBIT Maturity Model rating of Level 3. This indicates that processes are clearly defined, documented and communicated. To demonstrate added due diligence and to maximize value for money, project teams should be challenged to set even higher target maturity levels. CRS Audit Objectives 0 Non- Existent 1 Initial / Ad Hoc Project Management Control Framework - Project Management Framework 1 COBIT Maturity Model Ratings 2 Repeatable But Intuitive - Scope Management 2 - Master Project Plan 1 - Project Phase Approval 2 - HR Management 1 - Communications Mgmt (Internal) 1 - Communications Mgmt (External) 2 3 Defined Process - Cost Management 3 - Quality Management 1 - End-User Participation 2 - Training Strategy 3 Risk Management Strategies - Risk Management 1 - Service Level Management 1 - System Security Plan / Compliance with External Requirements 2 3 Information for Decision-Making and Reporting - Project Documentation 1 - Business Processes 1 Figure 2: COBIT Control Objectives assessed in the audit have been categorized by TB audit policy grouping and PMBOK knowledge areas. 4 Managed / Measurable 5 Optimized 4 CRS File Number , October The CRS report can be found at the following website: Chief Review Services IV/VII

9 KEY RESULTS Over the course of the audit, the ADM(IM)/DGIMPD and the CFHIS PMO team were very quick to respond to CRS concerns and actions were underway in October 2003 to begin resolving the identified issues summarized below. The CFHIS project s Management Control Framework lacks clear definition and structure: There is no evidence of a CFHIS master project plan; a complete CFHIS work breakdown structure (WBS) for the overall project; an integrated CFHIS project schedule; an overall change control process for both project and product scope; or a CFHIS human resources management plan. Although many project sub-plans have been partially developed or exist in draft format, there are no overall plans that include both the prime contractor and internal DND/CF efforts. The ADM(IM)/DGIMPD s quality management of CFHIS project management processes is largely reactive rather than proactive in nature. Key Project Oversight PDM Practices applied to the CFHIS project such as Delivery Management Meetings (DMMs) are not functioning as well as intended. Senior-level oversight mechanisms such as a Senior Review Board (SRB) and a Program Review Board (PMB) are in place for the project. The SRB met six times from March 1999 to June 2003 and the PMB met four times in approximately the same timeframe. Approval of the CFHIS WBS and the project implementation plan (PIP) is included as a responsibility in the SRB Terms of Reference. Discussion specifically related to these project plans were not found in the SRB or PMB meeting minutes provided. Risk Management Practices are not well developed or actively followed: Key project team members are not highly involved with risk management activities. The audit team noted that in one of the project s risk registries, risks had been assigned to various project team members. When interviewed, project team members were unaware that they had been assigned this responsibility. A CFHIS Risk Management Plan was developed at the beginning of the project by a consulting firm. However, it was not implemented and identified risks have not been tracked or monitored over time. The CFHIS PMO is currently trying to revise and implement a more robust risk management process, but assigned project team members do not demonstrate strong project management experience in this area. A total of $15.6 million in contingency funds has been included in the CFHIS project budget based on the CFHIS Project Profile and Risk Assessment (PPRA), yet there is no evidence that identified risks are being actively managed or monitored. Risk Identification, Risk Mitigation and Risk Monitoring are key to keeping the CFHIS project on schedule and on budget. Chief Review Services V/VII

10 There is a lack of project documentation and information from structured project management processes: The CFHIS project documentation repository is not structured effectively and project documentation is not consistently updated. Management decisions are being made in the absence of project outputs from structured project management processes. For example, additional project resources are being hired without a human resources management plan identifying required resources. A lack of documentation also results in poor corporate memory and weak project communication. New project team members and other stakeholders will encounter difficulty in making informed decisions without clear documentation capturing prior events, decisions and reasoning. This is more significant for projects with lengthy and complex implementation plans such as the CFHIS. KEY RECOMMENDATIONS It is recommended that the following actions be taken to achieve a greater level of rigour and increase the probability of success in the downstream phases of the project: ADM(IM)/DGIMPD clearly identify, define and prioritize the core project management processes and outputs considered integral to the successful delivery of the CFHIS project. At a minimum, CFHIS project outputs should include: a clear and succinct phased project scope document; a phased work breakdown structure (WBS) capturing total project scope; an integrated project schedule; and an integrated project master plan. The CFHIS PMO establish a process to identify, monitor and manage key CFHIS project interdependencies using project outputs such as the recommended WBS and project schedule and update project outputs as required (i.e., CFHIS budgeted funding requirements). ADM(IM)/DGIMPD and the CFHIS PMO develop a CFHIS human resources management plan including skills mapping, training plans and clarification of roles and responsibilities for all project team positions and personnel. The CFHIS PMO develop a comprehensive CFHIS risk management plan by consolidating risks from existing risk registries as well as categorizing, prioritizing, quantifying, assigning, monitoring and following-up on identified risks. ADM(IM)/DGIMPD strengthen the monitoring and oversight of the CFHIS project to ensure that project management processes remain in place and are followed and that accurate and consistent outputs can be used for management decision making. ADM(IM)/DGIMPD and the CFHIS PMO develop a strategy and schedule to ensure that identified core project management processes are implemented in a timely manner. Consideration should be given to acquiring external project management expertise on a short-term basis to ensure that recommendations are implemented promptly and without undue delay to project progress. Chief Review Services VI/VII

11 MANAGEMENT ACTION PLANS The ADM(IM)/DGIMPD and the CFHIS PMO acknowledge the audit findings and have accepted the audit recommendations. As indicated in the management action plans, provided in matrix format at Annex F, work has already commenced on addressing the key recommendations included in this report. Of particular note is the assignment of a risk manager, the completion of a comprehensive risk management plan and the implementation of risk management processes. We are generally satisfied that the management action plans appropriately address the concerns raised in this report and that corrective actions have been, or will be, implemented. It is important to emphasize that developing and implementing project plans and processes are only the starting point of achieving the required level of rigour. Equally important is ensuring that these plans and processes are maintained, communicated, used and monitored. It is this type of diligence and discipline that will help to increase the probability of successfully delivering the CFHIS in downstream phases of the project. Chief Review Services VII/VII

12 AUDIT OBJECTIVE ANNEX A AUDIT OBJECTIVES, SCOPE AND METHODOLOGY The objective of the audit was to assess the project management framework of the DND/CF Canadian Forces Health Information System (CFHIS) project and to determine whether appropriate processes and controls were in place to manage the project. The audit objective is aligned with the Treasury Board (TB) Internal Audit policy and includes: SCOPE the effectiveness of the project s management control framework; the soundness of the project s risk management strategies; and the project s use of information for decision-making and reporting. The scope of the audit included the project management framework and controls established for the implementation, management and delivery of the CFHIS project. As the CFHIS project was in the first year of its five-year implementation plan at the time of the audit, the scope was limited to the processes and controls that were planned or in place for the first phase of the CFHIS project. METHODOLOGY Industry accepted project management and IT governance frameworks were utilized to develop the audit methodology such as the PMBOK, published by the Project Management Institute and COBIT, published by the IT Governance Institute. COBIT is a generally applicable and accepted standard for good practices in IT security and control. It provides a reference framework for management, users and IT audit and security practitioners for managing the risks associated with IT-related processes. COBIT also includes a measurement tool to evaluate and rate the maturity of processes. The COBIT Maturity Model ratings range from 0 to 5 and are defined below: COBIT Maturity Levels 0 - Non-Existent Management processes are not applied at all. 1 - Initial / Ad Hoc Ad hoc approaches in place of standardized processes. 2 - Repeatable But Intuitive Processes follow a regular pattern. 3 - Defined Process Processes are documented and communicated. 4 - Managed and Measurable Processes are monitored and measured. 5 - Optimized Processes have been refined to a level of best practice. Chief Review Services A-1/2

13 ANNEX A The CRS audit team, in collaboration with KPMG, used the COBIT Maturity Model to measure and rate each of the COBIT Control Objectives assessed in the audit. Further detail on COBIT is provided at Annex B. A summary of the individual COBIT Control Objectives assessed in the audit are provided at Annex C and have been crossreferenced to the TB audit policy groupings and PMBOK knowledge areas presented earlier in Figure 2. Key project team members interviewed by the audit team are highlighted in the CFHIS organizational chart provided at Annex D. Chief Review Services A-2/2

14 ANNEX B CONTROL OBJECTIVES FOR INFORMATION AND RELATED TECHNOLOGY (COBIT ) 5 COBIT is published by the IT Governance Institute and is recognized internationally as a generally applicable and accepted standard for good practices in IT governance, security and control. COBIT is used or recommended by auditors and management worldwide, including the US National Association of State Chief Information Officers (NASCIO) and the Office of Inspector General (OIG) of the US House of Representatives 6. For the CFHIS audit, the audit team tailored and assessed 19 of the 34 COBIT High-level Control Objectives from the Planning and Organization (PO), Acquisition and Implementation (AI) and Delivery and Support (DS) domain areas. COBIT High-level Control Objectives assessed in the audit are provided at Annex C. 5 COBIT 3 rd Edition Management Guidelines, July Released by the COBIT Steering Committee and the IT Governance Institute. 6 DM Review Online, October Editorial Staff. Chief Review Services B-1/3

15 ANNEX B COBIT also provides the ability for organizations to measure and rate the maturity of their management processes. The COBIT maturity models are derived from the Software Engineering Institute (SEI) Capability Maturity Model 7 and allow organizations to grade existing processes from non-existent (0) to optimized (5). COBIT maturity levels are defined as follows: COBIT Maturity Levels 0 - Non-Existent Management processes are not applied at all. 1 - Initial / Ad Hoc Ad hoc approaches in place of standardized processes. 2 - Repeatable But Intuitive Processes follow a regular pattern. 3 - Defined Process Processes are documented and communicated. 4 - Managed and Measurable Processes are monitored and measured. 5 - Optimized Processes have been refined to a level of best practice. A fundamental feature of the maturity model is that it enables organizations to define to be or target maturity levels and also assists with developing strategies to achieve them. Additionally, it provides a reference tool for organizations to benchmark themselves against other IT organizations. Benchmarking Information In response to the many COBIT user requests, the Information Systems Audit and Control Association (ISACA) conducted a self-assessment survey from March to June 2002, to assess the average maturity of the 15 most important COBIT processes 8. In total, 168 valid responses were received, distributed over different geographies, sizes and sectors as presented below in Figure 3. Figure 3: Survey Respondents by Location, Size and Sector 7 Capability Maturity Model for Software, Version 1.1 Technical Report CMU/SEI-93-TR-024, Software Engineering Institute, Carnegie Mellon University, PA, February Control and Governance Maturity Survey: Establishing a Reference Benchmark and a Self-assessment Tool. Information Systems Control Journal, Volume 6, E. Guldentops, CISA, W. Van Grembergen, Ph.D., and S. De Haes. Chief Review Services B-2/3

16 ANNEX B In line with the overall focus of the CFHIS audit, the CFHIS overall project rating has been presented below in Figure 4 along with the ISACA unweighted average survey results for PO 10, the COBIT Control Objective designed for managing projects. Due to the nature of self-assessment surveys, it is important that these results are interpreted with necessary care and that they are positioned in context. For the purposes of this report, the survey results are to be used as a point of reference. PO 10 - Manage Projects COBIT Maturity Levels CFHIS Project Overall Results Large Org'n Mid Org'n Small Org'n Public Sector Finance Sector Retail / Mfg Sector Global Org'n Asia / Oceania EMEA 0 PO 10 - Manage Projects Figure 4: ISACA Control and Governance Maturity Self-Assessment survey results for PO 10 Manage Projects. Due to the cost, schedule and significance of the CFHIS project, project processes should have, at a minimum, a COBIT Maturity Model rating of Level 3 indicating processes are clearly defined, documented and communicated. This is similar to the ISACA survey results for global and large organization respondents. To demonstrate added due diligence and to maximize value for money, project teams should be challenged to set even higher target maturity levels. Chief Review Services B-3/3

17 ANNEX C COBIT CONTROL OBJECTIVES BY TB INTERNAL AUDIT POLICY GROUPING The CRS audit team, in collaboration with KPMG, tailored and assessed 19 of the 34 COBIT High-level Control Objectives from the Planning and Organization (PO), Acquisition and Implementation (AI) and Delivery and Support (DS) domain areas. CRS Audit Objectives / TB Audit Policy COBIT Control Objectives Assessed in the Audit Project Management Control Framework - Project Management Framework PO 10 Manage Projects PM Framework - Scope Management PO 10 Manage Projects Project Definition AI 1 3 Automated Solutions AI 6 Manage Changes - Master Project Plan PO 10 Manage Projects Master Project Plan - Project Phase Approval PO 10 Manage Projects Phase Approval - HR Management PO 4 Define the IT Organization PO 7 Manage Human Resources PO 10 Manage Projects Project Team - Communications Mgmt (Internal) PO 6 Communicate Aims and Directions - Communications Mgmt (External) PO 6 Communicate Aims and Directions - Cost Management PO 5 Manage the IT Investment - Quality Management PO 10 Manage Projects - Quality Plan PO 10 Manage Projects - Test Plan PO 11 Manage Quality AI 5 Install and Accredit Systems - End-User Participation PO 10 Manage Projects - User Involvement - Training Strategy PO 10 Manage Projects - Training Plan Risk Management Strategies - Risk Management PO 9 Assess Risks PO 10 Manage Projects Project Risk - Service Level Management DS 1 Define and Manage Service Levels DS 3 Manage Performance and Capacity DS 13 Manage Operations PO 10 Planning of Assurance Methods - System Security Plan / DS 5 Ensure Systems Security Compliance with External Req ts PO 8 Compliance with External Requirements Information for Decision-Making and Reporting - Project Documentation PO 2 Information Architecture Definition - Business Processes AI 4 Develop and Maintain Procedures Chief Review Services C-1/1

18 ANNEX D CFHIS ORGANIZATIONAL CHART (SIMPLIFIED VERSION) Project Sponsor ADM (HR-Mil) Project Leader ADM (HR-Mil)/DGHS Project Director PD Project Implementation ADM (IM) Project Leader ADM (IM)/DGIMPD Deputy Project Leader DPL CFHIS Committee Governance Project Management Board Senior Review Board Procurement ADM (Mat)/DCPS CFHIS PMO Project Manager PM DCPS Procurement Officer Project Control Office PCO Admin Coordinator Deputy Project Manager DPM Test Manager Functional Lead Technical Lead Denotes CFHIS personnel interviewed * Denotes those not interviewed Resources Training Lead TrL Resources * Interviews were also conducted with the PWGSC contracting officer and key members of the prime contractor s team (LM Program Manager, Chief Engineer and Project Scheduler). Chief Review Services D-1/1

19 ANNEX E GLOSSARY OF DND ACRONYMS AND PROJECT MANAGEMENT TERMS ADM(IM) Assistant Deputy Minister Information Management OIG Office of Inspector General ADM(Mat) Assistant Deputy Minister Materiel LC Limited Capability AI Acquire and Implementation (COBIT domain area) MA&S Materiel Acquisition and Support CF Canadian Forces OPI Office of Primary Interest CFHIS Canadian Forces Health Information System PCO Project Control Officer CMM Capability Maturity Model PD Project Director COBIT Control Objectives for Information and related Technology PDM Project Delivery Management COTS Commercial Off the Shelf PIR Project Independent Review (PDM process) CRS Chief Review Services PM Project Manager DCPS Director Common Procurement and Supply PMB Program Management Board DESC Defence Enterprise Service Centre PMBOK Project Management Body of Knowledge DGHS Director General Health Services PMO Project Management Office DGIMPD Director General Information Management Project Delivery PO Planning and Organizing (COBIT domain area) Dir IM Secur Directorate of IM Security PPMP Project Performance Management Plan (PDM process) DMM Delivery Management Meetings (PDM process) PPRA Project Profile and Risk Assessment DND Department of National Defence PSR Project Status Reports (PDM process) DPL Deputy Project Leader PWGSC Public Works and Government Services Canada DPM Deputy Project Manager RAMS Reserve Automated Management System DS Delivery and Support (COBIT domain area) RIIP Reserve Integrated Information Project FAA Financial Administration Act RFP Request for Proposal FC Full Capability SEI Software Engineering Institute FMAS Financial Managerial Accounting System SLA Service Level Agreements HR Human Resources SME Subject Matter Expert HRMS Human Resources Management System SRB Senior Review Board IM Information Management TB Treasury Board ISACA Information Systems Audit and Control Association TBS Treasury Board Secretariat IT Information Technology TrL Training lead NASCIO National Association of State Chief Information Officers WBS Work Breakdown Structure Chief Review Services E-1/1

20 ANNEX F MANAGEMENT ACTION PLANS CRS Recommendations OPI(s) Management Action Plans 1. Clearly identify, define and prioritize the core project management processes and outputs considered integral to the successful delivery of the CFHIS project. At a minimum, CFHIS project outputs should include: ADM(IM)/DGIMPD DGIMPD s Project Delivery Management (PDM) has established cost, time, scope and risk as key project management processes that must be managed. Project Management processes for these processes will include the development and maintenance of performance indicators. These performance indicators will be incorporated into the monthly Project Status Reports (PSRs) to assist in the management of these processes. An updated Project Performance Management Plan (PPMP) will be produced for CFHIS Phase II, and the CRS audit findings and detailed analysis and recommendations will be incorporated into the creation of the PPMP for CFHIS Phase II. It is expected that the Phase II PPMP will require additional changes to existing project management processes in use within CFHIS. Progress has already been made since the audit was performed, towards the implementation of a more robust risk management process. a. a clear and succinct phased project scope document; b. a phased work breakdown structure (WBS) capturing total project scope; a. While the project scope is outlined in various documents such as the Synopsis Sheets, the PPRA, and Project Charter, and more completely defined the project SOR document which is supplemented by the Enterprise Architecture Plan and detailed requirements in a CFHIS PMO maintained DOORS database, we acknowledge that there does not exist a single document that consolidates and links CFHIS scope and aligns this with the three phases of the project. The project will prepare a Scope consolidation document that will outline CFHIS scope by phases, provide linkages to other scope documentation such as the SOR and DOORS requirements database. As well, this document will address the detailed findings of the CRS audit related to phased project scope ambiguities. b. DND and the prime contractor, LMC, are working together for the Phase II integrated WBS. A preliminary WBS for Phase III will also be developed during Phase II. The integrated WBS will ensure all project scope is addressed. Chief Review Services F-1/4

21 CRS Recommendations OPI(s) Management Action Plans ANNEX F c. an integrated project schedule; and d. an integrated project master plan. 2. Establish a process to identify, monitor and manage key CFHIS project interdependencies using project outputs such as the recommended WBS and project schedule. Update project outputs as required (i.e., CFHIS budgeted funding requirements). 3. Develop a CFHIS human resources management plan including: a. skills mapping; CFHIS PMO ADM(IM)/DGIMPD and CFHIS PMO c. DND and the prime contractor, LMC, are working together for the project schedule. d. CFHIS will develop a Project Master Plan consistent with DND project management guidance contained in the Capability Initiatives Database (please refer to paragraph 1a). A number of improvements in formalizing the processes, documented capture, execution and control have been made since the CRS audit was performed. An informal process for the identification of internal project interdependencies had been followed since project inception. A formal process has now been implemented within which Technical and Functional Team Leads. On a weekly basis, any issues that could have an effect on the other teams are identified. The issues are documented and presented to a Senior Management Committee composed of the PM, PD and Contractor personnel for resolution in the event the issue cannot be resolved at the team level. If necessary, schedule, WBS and budgets are updated weekly. As part of developing an integrated project schedule, project internal dependencies will be identified and incorporated into the project schedule. The project acknowledges that no documented HR Plan existed or was explicitly communicated at the time of the audit. a. A process has now been established whereby requirements for specific expertise are identified on an on-going basis and the skilled resources are obtained to fulfill these specific roles. This may be through government staffing or through contract. The prime contractor may also be considered as a source or personnel resources, depending on the expertise required and available. Chief Review Services F-2/4

22 CRS Recommendations OPI(s) Management Action Plans ANNEX F b. training plans; and c. clarification of roles and responsibilities for all project team positions and personnel. 4. Develop a comprehensive CFHIS risk management plan by consolidating risks from existing risk registries as well as categorizing, prioritizing, quantifying, assigning, monitoring and following-up on identified risks. 5. Strengthen the monitoring and oversight of the CFHIS project to ensure that project management processes remain in place and are followed and that accurate and consistent outputs can be used for management decision-making. CFHIS PMO ADM(IM)/DGIMPD b. Each situation brings its own specific training issues. Training requirements are identified, prioritized and then actioned on an individual basis, based on any experience or formal training gaps. c. The situation is being reviewed and an HR Plan will be finalized. In addition, specific responsibilities will be reviewed and information will be communicated to all project personnel to ensure that all understand their responsibilities. A risk manager has been assigned to consolidate and maintain an integrated and comprehensive CFHIS Risk Management Plan. The plan includes processes for categorizing risks, prioritizing, quantifying, assigning, monitoring, developing mitigation plans and reviewing identified risks through interaction with all team members. DGIMPD has been monitoring the CFHIS project on an ongoing basis. PSRs are reviewed during the monthly Delivery Management Meetings (DMMs) between DPDMIS (Director Project Delivery Management Information Systems), the PM and key project staff. There have been ongoing improvements to the project management processes used within the CFHIS project. In part, this is as a result of the DGIMPD oversight provided as part of the PDM practices of a monthly PSR and DMM. Since the audit was conducted from August to October 2003, there has been progress made in finalizing documented project management plans. Of note is the completion of a Risk Management Plan, the appointment of a Risk Manager and implementation of risk management processes. Chief Review Services F-3/4

23 ANNEX F CRS Recommendations OPI(s) Management Action Plans The Project Performance Management Plan (PPMP), which establishes the reporting standard to be followed in the PSR, will be updated for Phase II of CFHIS. The update will use the findings of the CFHIS audit as input. This will form the baseline for CFHIS monitoring throughout Phase II of the project. A DGIMPD self-validation, a Project Readiness Review, will also be undertaken to confirm the presence and appropriateness of documented project management processes as well as other project documentation observed on by the CRS audit. 6. Develop a strategy and schedule to ensure that identified core project management processes are implemented in a timely manner. ADM(IM)/DGIMPD and CFHIS PMO PDM includes a Project Independent Review (PIR) process, performed on a recurring basis throughout the life of a project, to assess the project management processes in use within a project. The CRS audit performed the function of the DGIMPD PIR of CFHIS for Phase I. For the next PIR of CFHIS, the CRS audit report will be used as input to confirm appropriate action has been taken within the project. The core project management processes required for Phase II will be reviewed during the update of the PPMP. Where new processes or updates to existing processes are required they will be noted and added to the DMM action items list and status reviewed at the monthly DMMs. Identified required resources will be acquired when needed. This was the case in the Risk Management processes where a risk manager was acquired and formal risk management processes implemented during Phase I. These processes will continue to be monitored by the DPL through monthly DMM meetings. Chief Review Services F-4/4