2012 Medicaid Enterprise System Conference

Transcription

1 2012 Medicaid Enterprise System Conference O3: Leveraging Independent Verification and Validation (IV&V) to improve project success

2 Introduction of Panel

3 Agenda PCG North Highland, CSG, Washington

4 Role of IV&V Presenter: Rick Zelznak, North Highland

5 Role of IV&V What IV&V is and is not Risk Management Identify and report risk Risk management plan Corrective action plans Reporting to stakeholder groups Project Governance Risk Management Identify and report risk Risk management plan Corrective action plans Reporting to stakeholder groups

6 What You Should Expect from IV&V IV&V Is Not: Program or project management: IV&V strengthens by assuring the effectiveness of management processes Software testing: IV&V does, however, identify deficiencies in test plans and recommends modifications and enhancements. Supplemental staffing: IV&V is not additional staffing to deliver the client s tasks or to help the project maintain schedule. Quality assurance: Typically integrated with PMO/Project team rather than independent Auditors: IV&V service providers have the responsibility to develop actionable and practical recommendations to address project gaps and deficiencies not just highlight the problems and issues. Just for MMIS: IV&V is a widely accepted and expected component of high risk technology project in both the private and public sector

7 What You Should Expect from IV&V IV&V Is: Verification that project management disciplines are planned and performed according to best practices and work products are of high quality Validation that deliverables and project results meet the goals, objectives and business requirements as intended Assistance in protecting a client s interest through provision of subject matter expertise and independent oversight Unbiased perspective should look at effectiveness of both state and vendor. As a result, courts typically hold IV&V opinion higher than vendor or state. Verification: Doing The Right Things Validation: Doing Things Right

8 IV&V in Planning and Execution

9 Vendor Qualities Been there, done that meaning, has worked client side, vendor side and is experienced with like size projects allows vendor to see many perspectives and understand motivations Pragmatic and helpful Assertive without being a distraction or impediment Understands cultural norms of the client Preference given to a track record of successful system implementations over deep subject matter expertise A measure of successful IV&V is the satisfaction of both the state and the technology vendor

10 Lessons Learned ü ü ü IV&V should flex and adapt to the project, integrating smoothly into the project team, not get in the way Creativity should be employed in each project based on available data, client objectives and make-up of project team IV&V should be involved early in the project life cycle ½ of the value is lost after decisions have been made ü Projects grow in size and complexity, vendor maturity increases states must keep pace ü Covert, compliance centric and periodic IV&V has its place, but don t just check the box -- apply experience, practical knowledge and always keep the end objective in mind: Total Project Success

11 IV&V Tied to Certification Presenter: Deneen Omer, CSG Government Solutions

12 IV&V Tied to Certification IV&V Tied to Certification How do IV&V and the certification process relate? IV&V is forward looking Forward looking to a successful project What defines a successful project? One measure of project success: achieving certification! There is no Success button How can you maximize the IV&V for successful certification?

13 IV&V Tied to Certification CMS view of the certification process CMS has transformed the certification process from a single event occurring after the implementation of a new MMIS to an interactive model that engages CMS and the State (including its contractors), in a collaborative effort from the initial planning for major system improvements through the final achievement. We all know it is much easier to build it in throughout the process rather than assessing and correcting after the fact! Leverage your IV&V to make sure your project is building it (certification criteria) in to your project.

14 IV&V Tied to Certification Medicaid Enterprise Certification Toolkit (Toolkit) Contains a set of checklists Keyed to Medicaid business areas Business objectives for each area supported by review criteria IV&V is engaged in checking your project along the way Leverage IV&V to look at the certification criteria as part of the IV&V review criteria Leverage IV&V to ensure a no surprises philosophy!

15 Meeting CFR Requirements Presenter: Rick Zelznak, North Highland

16 What s New Regarding IV&V Defining IV&V IV&V Triggers (a) Application of IV&V Provider Acquisition (b&c) Expectations of Provider (b&c) Independence

17 Defining IV&V IV&V can be defined differently depending on its application For the purpose of Subpart F, CMS defines IV&V as: A well-defined standard process for examining the organizational, management, and technical aspects of a project to determine the effort s adherence to industry standards and best practices, to identify risks, and make recommendations for remediation, where appropriate

18 IV&V Triggers 8 CMS will require IV&V if project is 1. at risk of missing statutory or regulatory deadlines for automation that is intended to meet program requirements; 2. at risk of failing to meet a critical milestone; or 3. indicate the need for a new project or total system redesign; or 4. developing systems under waivers pursuant to sections 452(d)(3) or 627 of the Social Security Act; or 5. at risk of failure, major delay, or cost overrun; or 6. fail to timely and completely submit APD updates or other required systems documentation; or 7. state s procurement policies put the project at risk, including a pattern of failing to pursue competition to the maximum extent feasible; or 8. state s failure to adequately involve the State program offices in the development and implementation of the project.

19 IV&V Application Typically, expect that IV&V will be required for historically high risk projects such as MMIS and eligibility and enrollment systems In practice, introducing IV&V early in the process and integrating closely with PMO functions tends to mitigate project failure States are able to request a waiver of the IV&V requirement

20 Provider Acquisition The acquisition document and contract for selecting the IV&V provider (or similar documents if IV&V services are provided by other State agencies) must ü ü ü include requirements regarding the experience and skills of the key personnel proposed for the IV&V analysis; specify by name the key personnel who actually will work on the project; and be submitted to the Department for prior written approval.

21 Expectation of Providers Selected IV&V provider must: ü ü ü ü ü ü Develop a project work plan Review and make recommendations on project management and technical aspects of the project (state and vendor) Consult with all stakeholders and assess user involvement and buy-in regarding system functionality and system s ability to support program business needs Conduct analysis of past project performance sufficient to identify and make recommendations for improvement Provide risk management assessment and capacity planning services Develop performance metrics which allow tracking project completion against milestones set by the State The plan and results of analysis must be provided directly to the Department at the same time it is given to the State.

22 Independent IV&V must be conducted by an entity that is independent from the State (unless the State receives an exception from the Department) Avoid Conflict of Interest Can an IV&V vendor also provide resources to work on other aspects of the project? Can a vendor that worked on the project prior to the state hiring an IV&V vendor serve as the IV&V provider? Can a state department hire or assign a staff member to provide IV&V services?

23 Relationship to Seven Standards and Conditions Presenter: Deneen Omer, CSG Government Solutions

24 Relationship to Seven Standards & Conditions IV&V Relationship to Seven Standards and Conditions (SS&C) SS&C are another critical input to overall success Build these characteristics into the project - From conception through procurement to design and implementation How can you maximize IV&V for adherence to the SS&C? Quick review of the SS&C 42 CFR : 1. Modularity Standard 2. MITA Condition 3. Industry Standards Condition 4. Leverage Condition 5. Business Results Condition 6. Reporting Conditions 7. Interoperability Condition

25 Relationship to Seven Standards & Conditions Modularity Standard: Use of a modular, flexible approach to systems development. Compliance is addressed by use of SDLC methodology, open interfaces and exposed APIs, use of business rules engines, and submission of business rules to HHS-designated repository. IV&V reviews methodology and approach to design Engage IV&V early in the process to provide independent review of the design and implementation approach MITA Condition: Align to and advance increasingly in MITA maturity. Compliance is addressed by performing an annual State Self-Assessment (SS-A), submitting a MITA 5-year roadmap for maturity progression, and implementing a Concept of Operations and Business Process Models. IV&V assess to ensure the solution to aligns to vision and objectives

26 Relationship to Seven Standards & Conditions Industry Standards Condition: Ensure alignment with, and incorporation of, industry standards. Compliance is met by implementing standards. IV&V review for standards compliance including Health Insurance Portability and Accountability Act of 1996 (HIPAA) Standard Transaction Sets Section 508 of the Rehabilitation Act ACA Section 1104 Operating Rules ACA Section 1561 Electronic Eligibility and Enrollment Leverage Condition: Promote sharing, leverage, and reuse of Medicaid technologies and systems. Solution examples include project collaboration with other states, implementation of Commercial Off-the-Shelf (COTS) software, and identification of components for reuse by other states or within the state s Medicaid Enterprise components. IV&V input to identify opportunities for reuse to maximize success

27 Relationship to Seven Standards & Conditions Business Results Condition: Support accurate and timely processing of claims. Evaluation is based on the degree of automation, web-based customer service for providers and beneficiaries, and the enactment of performance standards. IV&V looks for service level agreements and level of automation Reporting Conditions: Produce transaction data, reports and performance information. Solution examples include electronic transmission of program evaluation reports through open interfaces to management and federal agencies. IV&V reviews for accuracy and transparency to support management Interoperability Condition: Ensure seamless coordination and integration with HIE and HIX and other federal agencies. Systems must be built using standardized messaging, protocols, and architecture. IV&V looks at the enterprise view beyond silos

28 Relationship to Seven Standards & Conditions Why SS&C? 90% match! Simply stated - to ensure the implementation achieves the business goals that the funding was supposed to accomplish CMS performance measures will tie directly to the standards and conditions Leverage IV&V as an important partner to assist in achieving Seven Standards and Conditions

29 Modular Development Use of existing or new backbone Presenter: Jeff Hellzen, PCG

30 Modular Development Use of Existing or New Backbone 7 Standards and Conditions Modularity Standard Use of a modular, flexible approach to systems development, including the use of open interfaces and exposed application programming interfaces; the separation of business rules from core programming; and the availability of business rules in both human and machine readable formats.

31 Modular Development Use of Existing or New Backbone Use of a modular, flexible approach to systems development, including the use of open interfaces and exposed application programming interfaces;

32 Modular Development Use of Existing or New Backbone Existing Does your existing backbone support open standards? q q q Does it support Open Interfaces? (recognized by industry groups or standards organizations) Is there an Application Programming Interface (API) available? For multiple languages and environments? Is there a path forward can both sides of the interface change on differing schedules? New Does the new backbone support open standards? q q q Does it support Open Interfaces? (recognized by industry groups or standards organizations) Is there an Application Programming Interface (API) available? For multiple languages and environments? Is there a path forward can both sides of the interface change on differing schedules? If the answer to these ques?ons is no, you should look elsewhere

33 Modular Development Use of Existing or New Backbone the separation of business rules from core programming; and the availability of business rules in both human and machine readable formats.

34 Modular Development Use of Existing or New Backbone Existing Does your existing backbone use a separate business rules component? q q q q q Can the rules be modified without touching the main application? Can the rules be used by other applications? Can rules execution be orchestrated? Can the rules be date-driven? Can your business analysts read and understand the rules? New Can your new backbone use a separate business rules component? q q q q Can the rules be re-used by a number of applications / other rules? Can rules execution be orchestrated? Can the rules be date-driven? Can your business analysts read and understand the rules? If the answer to these ques?ons is no, you should look elsewhere

35 Modular Development Use of Existing or New Backbone The previous answers are just parts of the puzzle: Use your IV&V vendor to answer the ili#es ques?ons: q Scalability q Capability q Maintainability q Reliability q Availability q Extensibility q Feasibility

36 Modular Development Use of Existing or New Backbone So what about the rest of my system assets? Use your IV&V vendor to: Help create an applications, tools, interfaces, and documentation inventory (portfolio) Determine the feasibility of re-use and architectural fit Determine where the portfolio is in its life cycle Facilitate a discussion of reusability with your users and technical staff

37 Experience with IV&V and the RFP Process Presenter: John Anderson- State of Washington

38 Experience with IV&V and the RFP Process What do you want the IV&V Vendor to do? IV&V is a very broad topic, need to understand what you need out of an IV&V engagement Factors to consider: project s scope of work, your team s strengths, your team s weaknesses, budget, project schedule, state/federal requirements (MITA, Security testing, ADA testing, independent testing, etc.) Where to Look for Guidance IEEE 1012; PMBOK 3rd Ed.; ITIL; CMMI, MITA Other State s IV&V RFP s

39 IV&V Scope of Work Areas to Consider Independent Testing Technical Deliverable Reviews Code Reviews Technical Assessments

40 IV&V Scope of Work Areas to Consider Technical Assessments (periodic or one time) Software Development Processes and Standards Configuration/version control Processes and tools Data Conversion/migration processes and tools Testing plans, processes, standards and tools Hardware/software architecture and sufficiency Capacity/performance Database design Portability/re-usability Operational Readiness Implementation/turnover plans System Back-Up/Recovery procedures Disaster Recovery/ Business Continuance Plan

41 What type of engagement do you want? Full time on-site presence Regular presence in Project Meetings Regularly scheduled reports (e.g. monthly) Regular status updates Periodic On site assessments Each assessment has a defined SOW Assessment interviews/reviews Assessment Report Status Updates with each assessment report

42 How do want the contract and payment terms structured? Deliverables Based Initial Strategy & Plan Assessment Reports Regular Monthly Reports Corrective Action Plan(s) Performance Based Usually deliverables based with SLA s Time & Materials Hybrid

43 Experience Managing IV&V Contract Presenter: Lynda Douglas, PCG

44 Experience Managing IV&V Contracts IV&V Reporting Map IV&V tasks to Project schedule Periodic or Continuous Customize Reporting structure to client s needs Regular debrief meetings Managing Results of IV&V recommendations Separate from Vendors Risk Management Project Accepts or Rejects recommendation Corrective Action Plans

45 Experience Managing IV&V Contracts Stakeholder Involvement Vendor Agency Management Team CMS Gate Reviews APD updates Flexibility Cycle scheduling Add-ons Technical deep dives

46 Questions & Answers Please complete your conference evaluation Thank You.