Dr. Nader Mehravari Research Scientist, CERT Division
|
|
- Jonah Palmer
- 6 years ago
- Views:
Transcription
1 Everything You Always Wanted to Know About Maturity Models Dr. Nader Mehravari Research Scientist, CERT Division Dr. Nader Mehravari is with the CERT Program at the Software Engineering Institute (SEI), a unit of Carnegie Mellon University in Pittsburgh, PA. His current areas of interest and research include operational resilience, protection and sustainment of critical infrastructure, preparedness planning, and associated risk management principles and practices. Nader was with Lockheed Martin from 1992 through In his most recent assignment, he was the Director for Business Resiliency. In this capacity, he led and oversaw all preparedness planning and associated governance and compliance activities. He was responsible for building and leading Lockheed Martin's resiliency program where he successfully implemented a modern, integrated, risk management based approach to disaster recovery, business continuity, pandemic planning, crisis management, emergency management, and workforce continuity for all of Lockheed Martin.
2 Notices Copyright 2014 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. Carnegie Mellon and CERT are registered marks of Carnegie Mellon University. DM
3 Outline Setting the Stage The need for measuring operational activities & their effectiveness Are we doing the right things? Are we using the right tools to measure? Are we measuring the right things? ABCs of Maturity Models What are Maturity Models? Types of Maturity Models Examples of Maturity Models Closing Thoughts A few cautions Determining when and which type to use
4 Setting the Stage The need for measuring operational activities & their effectiveness Are we doing the right things? Are we using the right tools to measure? Are we measuring the right things?
5 Today s Operating Environment Rapid changes in technology and its application in a wide range of industries. Introduction of many new systems, business processes, markets, risks, and enterprise approaches. Many immature products and services being consumed by enterprises that themselves are in a state of change.
6 Challenges at Hand How can you tell if you are doing a good job of managing these changes? How best to monitor your progress on an ongoing basis? How do you manage the interactions of systems and processes that are continually changing? How do poor processes impact interoperability, safety, reliability, efficiency, and effectiveness?
7 Which tool should I use? Your organization wants to know SOMETHING about your mission operation: How EFFECTIVE are we? Do we have the right SKILLS and CAPABILITIES? Do we have the right TECHNOLOGIES? OR
8 Observation The development and use of maturity models in security, continuity, IT operations, & resilience space is increasing dramatically.
9 Do maturity models measure the right thing? v May not measure what you think it measures Ø Practice maturity vs. organizational maturity? v May give you inaccurate data on which to base decisions Ø Process performance vs. product performance? v Can increase cost but reduce benefit Ø An improved process may not result in compliance v May provide a false sense of confidence Ø A robust process may not stop all malware
10 ABCs of Maturity Models What are Maturity Models? Types of Maturity Models Examples of Maturity Models
11 Maturity Model Defined An organized way to convey a path of experience, wisdom, perfection, or acculturation. Depicts an evolutionary progression of an attribute, characteristic, pattern, or practice. The subject of a maturity model can be objects or things, ways of doing something, characteristics of something, practices, controls, or processes.
12 Maturity Models Provide Means for assessing and benchmarking performance Ability to assess how a set of characteristics have evolved Expression of body knowledge of best practices Identification of gaps and improvement plans Roadmap for model-based improvement Demonstrated results of improvement efforts Common language or taxonomy
13 Key Components of a Maturity Model Levels The measurement scale The transitional states Domains Logical groupings of like attributes into areas of importance to the subject matter and intent of the model Logical groupings of like practices, processes, or good things to do Attributes Core content of the model arranged by domains and levels Typically based on observed practices, standards, or expert knowledge Diagnostic Methods For assessment, measurement, gap identification, benchmarking Improvement Roadmaps To guide improvement efforts (e.g., Plan-Do-Check-Act)
14 Types of Maturity Models There are three types of maturity models Progression Maturity Models Capability Maturity Models (CMM) Information Technology/Risk Management Part Two focuses on using CERT-RMM to establish a foundation for sustaining operational resilience management processes in complex environments where risks rapidly emerge and change. Part Three details all 26 CERT-RMM process areas, from asset definition through vulnerability resolution. For each, complete descriptions of goals and practices are presented, with realistic examples. Part Four contains appendices, including Targeted Improvement Roadmaps, a glossary, and other reference materials. This book will be valuable to anyone seeking to improve the mission assurance of high-value services, including leaders of large enterprise or organizational units, security or business continuity specialists, managers of large IT operations, and those using methodologies such as ISO 27000, COBIT, ITIL, or CMMI. informit.com/seiseries cert.org/resilience Cover illustration by nikamata/istock.com Text printed on recycled paper ISBN-13: ISBN-10: Caralli Allen White The authors are senior technical staff members within the CERT Program of the Software Engineering Institute (SEI). Richard A. Caralli, Resilient Enterprise Management technical manager, develops and delivers methods, tools, and techniques for enterprise security and resilience management. He led the development of CERT-RMM. Julia H. Allen develops and transitions executive outreach programs in enterprise security and governance and researches software security and assurance. She served as the SEI s Acting Director and Deputy Director/COO and authored The CERT Guide to System and Network Security Practices (Addison-Wesley, 2001). David W. White, a core member of the CERT-RMM development team, develops CERT-RMM and related products and helps organizations apply them. A Maturity Model for Managing Operational Resilience SEI SERIES t A CERT BOOK Part One summarizes the value of a process improvement approach to managing resilience, explains CERT-RMM s conventions and core principles, describes the model architecturally, and shows how it supports relationships tightly linked to your objectives. CERT Resilience Management Model This book both introduces CERT-RMM and presents the model in its entirety. It begins with essential background for all professionals, whether they have previously used process improvement models or not. Next, it explains CERT-RMM s Generic Goals and Practices and discusses various approaches for using the model. Short essays by a number of contributors illustrate how CERT-RMM can be applied for different purposes or can be used to improve an existing program. Finally, the book provides a complete baseline understanding of all 26 process areas included in CERT-RMM One or more may be appropriate for your particular needs CERT -RMM, Version 1.1 CERT Resilience Management Model (CERT-RMM) is an innovative and transformative way to manage operational resilience in complex, risk-evolving environments. CERT-RMM distills years of research into best practices for managing the security and survivability of people, information, technology, and facilities. It integrates these best practices into a unified, capability-focused maturity model that encompasses security, business continuity, and IT operations. By using CERT-RMM, organizations can escape silo-driven approaches to managing operational risk and align to achieve strategic resilience management goals. CERT Resilience Management Model Hybrid Maturity Models Richard A. Caralli Julia H. Allen David W. White $79.99 US CANADA Not all maturity models are CMMs
15 Progression Maturity Models Simple progression or scaling of an attribute, characteristic, pattern, or practice Levels describe higher states of achievement, advancement, completeness, or evolution Levels can be arbitrary as agreed upon by users, industry, etc.
16 Progression Maturity Models - Example A Maturity Progression for Toy Building Bricks Lego Mindstorms Lego Architecture Lego Technic Lego City Lego Duplo
17 Progression Maturity Models - Example A Maturity Progression for Human Mobility Fly Sprint Run Jog Walk Crawl A Maturity Progression for Authentication Three-factor authentication Two-factor authentication Addition of changing every 60 days Use of strong passwords Use of simple passwords Progress does not necessarily equate to maturity
18 Progression Maturity Models - Example Higher levels may be characterized as tool- enabled These characteriza,ons are typically arbitrary Lower levels may be characterized as primi6ve A Maturity Progression for Counting Computer Calculator Adding machine Slide rule Abacus Pencil and paper Sticks/Stones Fingers
19 Progression Model Example: SGMM Smart Grid Maturity Model Characteris6cs: Features you 3 would expect to see at each stage of the smart grid journey SMR Strategy, Management, & Regulatory OS Organization & Structure GO Grid Operations WAM Work & Asset Management TECH Technology CUST Customer VCI Value Chain Integration SE Societal & Environmental
20 Benefits and Limitations of Progression Models Benefits v Provides a transformative roadmap v Simple to understand and adopt; low adoption cost v Easy to recalibrate as technologies and practices advance Limitations v Levels are arbitrarily defined and may be meaningless v Achieving higher levels does not necessarily translate into maturity v Often confused with CMMs thus users inaccurately project traits of CMMs on progression models
21 Capability Maturity Models (CMM) A more complex instrument Characterizes the maturity of processes the degree to which processes are institutionalized the degree to which the organization demonstrates process maturity the maturity of the culture of the organization Levels reflect the degree to which a particular set of practices have been institutionalized Institutionalized processes are more likely to be retained during times of stress. CERT Resilience SEI SERIES A CERT BOOK CERT -RMM, Version 1.1 CERT Resilience Management Model Richard A. Caralli Julia H. Allen David W. White A Maturity Model for Managing Operational Resilience
22 What do these organizations have in common? Customer Happiness Chain of Command Unit Cohesion Regulations Strong Culture Customer Service Tradition Protection
23 CMM Levels An Example Processes are acculturated, defined, measured, and governed Prac,ces are performed Prac,ces are incomplete Level 3 Defined Level 2 Managed Level 1 Performed Level 0 Incomplete Higher degrees of institutionalization translate to more stable processes that are repeatable produce consistent results over time are retained during times of stress
24 CMM Levels Another Example Example 3 Optimized Measured Defined Repeatable Ad hoc Nonexistent Continuous and effective, integrated, proactive, usually automated Well-managed, formal, often automated, evaluated frequently Intuitive, not documented, occurs only when necessary Intuitive, not documented, occurs only when necessary Occasional, not consistent, not planned, disorganized Not understood, not formalized, need is not recognized
25 Examples of CMM Levels Example 1 Optimized Quantitatively Managed Defined Managed Ad hoc Example 2 Externally integrated Internally integrated Managed Performed Initiated Example 3 Shared Defined Measured Managed Planned Performed but ad hoc Incomplete
26 Capability Maturity Model Example: CERT-RMM Framework for managing and improving opera6onal resilience an extensive super-set of the things an organization could do to be more resilient. - CERT-RMM adopter
27 CMM Example: CERT-RMM CERT-RMM Process Areas (Domains) Access Management Asset Definition and Mgmt. Communications Compliance Controls Management Enterprise Focus Environmental Control External Dependencies Financial Resource Mgmt. Human Resource Management Identity Management Incident Management & Control Knowledge & Information Mgmt. Measurement and Analysis Monitoring Organizational Process Focus Organizational Process Definition Organizational Training & Awareness People Management Resilience Requirements Development Resilience Requirements Mgmt. Resilient Technical Solution Engr. Risk Management Service Continuity Technology Management Vulnerability Analysis & Resolution
28 CMM Example: CERT-RMM Consider the Incident Management and Control (IMC) domain from CERT-RMM: Goal 1: Establish the IMC process Goal 2: Detect events Goal 3: Declare incidents Goal 4: Respond to and recover from incidents Goal 5: Establish incident learning
29 CMM Example: CERT-RMM Level 0 Level 1 Level 2 Level 3 Incomplete Performed Managed Defined..... We don t do (all of) the prac6ces. We do the prac6ces. Ins,tu,onaliza,on is cumula,ve We do the prac6ces AND we plan and govern the process, resource it, train people to do it, monitor it, etc We do everything in level 2 AND we have a defined process and collect improvement informa6on.
30 Benefits and Limitations of CMMs Benefits v Provides for measurement of core competencies v Provides for rigorous measurement of capability the ability to retain core competencies under times of stress v Can provide a path to quantitative measurement Limitations v Sometimes difficult to understand and apply; high adoption cost v Maturity may not translate into actual results v Potential false sense of achievement: achieving high maturity in security practices may not mean the organization is secure
31 Compare: Progression vs CMM Level 3 Run Level 3 Defined Level 2 Jog Level 1 Walk Level 0 Crawl Core prac6ces Distribu,on of core prac,ces across levels Level 2 Managed Level 1 Performed Level 0 Incomplete Core prac6ces Distribu,on of ins,tu,onalizing features Progression Model Capability Model
32 Hybrid Maturity Model Combines the best features of progression and capability maturity models Allows for measurement of evolution or achievement as in progression models Adds the ability to measure capability or institutionalization with the rigor of a CMM Levels reflect both achievement and capability Transitions between levels: Similar to a capability model (i.e., describe capability maturity) Architecturally use the characteristics, indicators, attributes, or patterns of a progression model
33 Hybrid Maturity Models Domains: Specific categories of azributes, characteris6cs, pazerns, or prac6ces that form the content of the model Domain 1 Domain 2 Domain 3 Domain 4 Domain n Capability or maturity levels Level 4 Defined Level 3 Measured Level 2 Managed Level 1 Planned Level 0 Incomplete Maturity levels: Defined sets of characteris6cs and outcomes, plus capability considera5ons Model content: Specific azributes, characteris6cs, pazerns, or prac6ces that represent progression and capability
34 Hybrid Model Example: ES-C2M2 Electricity Subsector Cybersecurity Capability Maturity Model (ES- C2M2)
35 Benefits and Limitations of Hybrid Models Benefits v Provides for easy measurement of core competencies as well as approximation of capability v Can adapt easily to evolution of technologies and practices without sacrificing capability measurement Limitations v Maturity concept is approximated; not as rigorous as CMM v Combination of attributes with institutionalizing features at each level can be arbitrary v Low adoption cost
36 Closing Thoughts A few cautions Determining when and which type to use
37 First and Foremost Have a clear understanding of your business objectives for using any type of improvement model How the model will meet these objectives Understand how this initiative fits with others that are mainstream for the organization (not a new add-on) Have visible sponsorship of executives and senior leaders who are essential for success Have well-defined outcome measures that are regularly reported and reviewed Have a plan and committed resources
38 A Few Cautions Progression models may be easier to adopt but may not be sustainable (aka sticky) Definitions of levels can be arbitrary Measuring process performance and maturity is useful but may not be sufficient Exercise care when using maturity models for specific purposes
39 Progression Models May Not Be Sustainable A progression model provides a roadmap or scale of a particular characteristic, indicator, attribute, pattern, or practice Focuses on practices or controls and their progression from least mature to most mature Cannot be used to measure the extent to which an organization is capable of sustaining the practice in times of disruption and stress (the practice has not become part of the DNA) A hybrid or capability maturity model adds the dimension of organizational capability to practice progression Thus able to measure an organization s resilience in the presence of disruption and stress
40 Definitions of Levels Can Be Arbitrary Often defined by consensus of subject matter experts Can simply reflect a plateau or a place in a progression or scale Often have not been validated or are difficult to validate based on experience and measurement May neglect to represent the capability and capacity of an organization to sustain operations in the presence of disruption and stress 40
41 Measuring Process Performance May Not Be Sufficient Experience demonstrates that the quality of the process directly affects the quality of the product However, process performance and maturity are only one aspect Also need to consider the performance and maturity of The product and its outcomes The supporting technologies The environment within which the product operates Knowledge, skills, and abilities of people with respect to all of these Which of these dimensions to emphasize given product objectives 41
42 When Does It Make Sense to Use Maturity Models? Requirement for a structured approach Demonstrated, measurable results based on an established body of knowledge A defined roadmap from a current state to a desired state An ability to monitor and measure progress, particularly in the presence of change Response to a strategic improvement or new product/new market objective 42
43 When Does It Make Sense to Use Maturity Models? (cont.) Desire to answer these questions in a repeatable, predictable manner: How do I compare with my peers? (ability to benchmark) How can I determine how secure I am and if I am secure enough? How do I measure my current state? Characterize my desired state? What concrete actions do I need to take to improve? And in what order? How do I measure progress toward my desired state? How do I adapt to change? 43
44 Exercise Care When Using Maturity Models If the immediate need is to respond to an in-progress disruptive event Robust processes are not yet in place Current protection and defensive mechanisms are failing Need to stop the bleeding, stabilize operations, rely on experts In response to current and new compliance requirements In a highly regulated industry Must demonstrate compliance with specific laws, regulations and standard(s) Standard, defined processes and mapping new compliance requirements to these can be quite effective 44
45 References Crosby, P. B. (1979). Quality is Free. New York: New American Library. ISBN Nolan, R. L. (July 1973). "Managing the computer resource: A stage hypothesis". Comm. ACM 16 (7): doi: / Humphrey, W. S. (March 1988). "Characterizing the software process: A maturity framework". IEEE Software 5 (2): doi:10.119/ Humphrey, W. S. (1989). Managing the Software Process. SEI series in software engineering. Reading, Mass.: Addison-Wesley. ISBN Paulk, Mark C.; Weber, Charles V.; & Curtis, Bill. The Capability Maturity Model: Guidelines for Improving the Software Process. Addison- Wesley Professional, CMMI Overview. (2012). Chris McClean and Khalid Kark, Introducing the Forrester Information Security Maturity Model, Forrester Research, July 27, Caralli, Richard A.; Allen, Julia H.; & White, David W. CERT Resilience Management Model: A Maturity Model for Managing Operational Resilience. Addison-Wesley, Caralli, Richard A. Discerning the Intent of Maturity Models from Characterizations of Security Posture. Software Engineering Institute, Carnegie Mellon University, Electricity Subsector Cybersecurity Maturity Model. Carnegie Mellon University, %20Subsector%20Cybersecurity%20Capabilities%20Maturity%20Model%20%28ES-C2M2%29%20-%20May% pdf Smart Grid: Tools and Methods. (2012). Resilience Management. (2012).
46 As projects continue to grow in scale and complexity, effective collaboration across geographical, cultural, and technical boundaries is increasingly prevalent and essential to system success. SATURN 2012 will explore the theme of Architecture: Catalyst for Collaboration. Introduction to the CERT Resilience Management Model February 18-20, 2014 (SEI, Arlington, VA) June 17-19, 2014 (SEI, Pittsburgh, PA) See Materials Widget for course document
Risk and Resilience: Considerations for Information Security Risk Assessment and Management
Risk and Resilience: Considerations for Information Security Risk Assessment and Management Julia Allen and Jim Cebula CERT Program Software Engineering Institute Session ID: GRC-202 Session Classification:
More informationDefining a Maturity Scale for Governing Operational Resilience
Defining a Maturity Scale for Governing Operational Resilience Katie Stewart Julia Allen Audrey Dorofee Michelle Valdez Lisa Young March 2015 TECHNICAL NOTE CMU/SEI-2015-TN-004 CERT Division http://www.sei.cmu.edu
More informationCERT Resilience Management Model, Version 1.2
CERT Resilience Management Model, Organizational Process Focus (OPF) Richard A. Caralli Julia H. Allen David W. White Lisa R. Young Nader Mehravari Pamela D. Curtis February 2016 CERT Program Unlimited
More informationCARNEGIE MELLON UNIVERSITY
CARNEGIE MELLON UNIVERSITY 1 Integrated Risk Management for the Enterprise Brett Tucker December 2018 Carnegie Mellon University Software Engineering Institute Carnegie Mellon University Pittsburgh, PA
More informationThe Smart Grid Maturity Model & The Smart Grid Interoperability Maturity Model. #GridInterop
The Smart Grid Maturity Model & The Smart Grid Interoperability Maturity Model #GridInterop Maturity Models Dueling or Complementary? SGMM? SGIMM? SGIMM? SGMM? #GridInterop Phoenix, AZ, Dec 5-8, 2011 2
More informationCERT Resilience Management Model, Version 1.2
CERT Resilience Management Model, Asset Definition and Management (ADM) Richard A. Caralli Julia H. Allen David W. White Lisa R. Young Nader Mehravari Pamela D. Curtis February 2016 CERT Program Unlimited
More informationWhat Metrics Should a CSIRT Collect to Measure. Success?
What Metrics Should a CSIRT Collect to Measure (Or What Questions Should We Be Asking and How Do We Get the Answers?) Robin Ruefle, Audrey Dorofee June 15, 2017 Software Engineering Institute Carnegie
More informationIntroduction to Software Product Lines Patrick Donohoe Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213
Introduction to Software Product Lines Patrick Donohoe Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 2014 by Carnegie Mellon University Copyright 2014 Carnegie Mellon University
More informationComplexity and Software: How to Meet the Challenge. NDIA CMMI Technology Conference
Complexity and Software: How to Meet the Challenge NDIA CMMI Technology Conference Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Paul Nielsen November 15, 2011 2011 Carnegie
More informationApplication of the CERT Resilience Management Model at Lockheed Martin
Application of the CERT Resilience Management Model at Lockheed Martin March 2011 David White Carnegie Mellon University Software Engineering Institute dwhite@cert.org Dr. Nader Mehravari Executive VP,
More informationCERT Resilience Management Model Capability Appraisal Method (CAM) Version 1.1
CERT Resilience Management Model Capability Appraisal Method (CAM) Version 1.1 Resilient Enterprise Management Team October 2011 TECHNICAL REPORT CMU/SEI-2011-TR-020 ESC-TR-2011-020 CERT Program http://www.sei.cmu.edu
More informationUnderstanding Model Representations and Levels: What Do They Mean?
Pittsburgh, PA 15213-3890 Understanding Model Representations and Levels: What Do They Mean? Mary Beth Chrissis Mike Konrad Sandy Shrum Sponsored by the U.S. Department of Defense 2004 by Carnegie Mellon
More informationCMMI Version 1.3: Are you Ready for Release?
CMMI Version 1.3: Are you Ready for Release? Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Eileen Forrester October 2010 2 3 How to Participate Today Open and close your
More informationA Case Study: Experiences with Agile and Lean Principles
A Case Study: Experiences with Agile and Lean Principles Jeff Davenport Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University This material is based upon work
More informationMeasuring What Matters Lisa Young
SESSION ID: GRC-R05 Measuring What Matters www.cert.org/rsa/ Lisa Young Senior Engineer CERT-Software Engineering Institute-Carnegie Mellon University Notices Copyright 2016 Carnegie Mellon University
More informationSGMM Model Definition A framework for smart grid transformation
SGMM Model Definition A framework for smart grid transformation Authors: The SGMM Team Version 1.2 September 2011 TECHNICAL REPORT CMU/SEI-2011-TR-025 ESC-TR-2011-025 CERT Program Research, Technology,
More informationDesigning the Infrastructure for an Enterprise IT System
Designing the Infrastructure for an Enterprise IT System William E. Novak Patrick R.H. Place Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University This material
More informationImproving Operational Resilience Processes
IEEE International Conference on Social Computing / IEEE International Conference on Privacy, Security, Risk and Trust Improving Operational Resilience Processes The CERT Resilience Management Model Richard
More informationAcquisition Overview: The Challenges
Acquisition Overview: The Challenges Rita Creel Robert J. Ellison June 2007 ABSTRACT: The challenges of acquiring software-intensive systems continue to grow along with the increasingly critical role software
More informationIncremental Lifecycle Assurance of Critical Systems
Incremental Lifecycle Assurance of Critical Systems Peter Feiler Incremental Lifecycle Assurance of Critical of Critical Systems Systems Oct 2016 2016Carnegie Mellon University 1 Copyright 2016 Carnegie
More informationAchieving Agility and Stability in Large-Scale Software Development. Ipek Ozkaya Senior Researcher, Research, Technology, and System Solutions Program
Achieving Agility and Stability in Large-Scale Software Development Ipek Ozkaya Senior Researcher, Research, Technology, and System Solutions Program Ipek Ozkaya is a senior member of the technical staff
More informationCreating a Computer Security Incident Response Team Action Plan
Creating a Computer Security Incident Response Team CERT Training and Education Networked Systems Survivability Software Engineering Institute Carnegie Mellon University This material is approved for public
More informationAgile In Government: A Research Agenda for Agile Software Development
Agile In Government: A Research Agenda for Agile Software Development Will Hayes Suzanne Miller Eileen Wrubel Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 March 201720
More informationBusiness Process Improvement Guided by the BPMM i
Business Process Improvement Guided by the BPMM i In this first column, we introduce the idea of organizational maturity by describing the overall content of the Business Process Maturity Model (BPMM),
More informationReducing Architecture Complexity with AADL
Reducing Architecture Complexity with AADL Julien Delange Jerome Hugues Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Copyright
More informationMEASURING PROCESS CAPABILITY VERSUS ORGANIZATIONAL PROCESS MATURITY
MEASURING PROCESS CAPABILITY VERSUS ORGANIZATIONAL PROCESS MATURITY Mark C. Paulk and Michael D. Konrad Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213-3890 Abstract The
More informationOh No, DevOps is Tough to Implement!
[DISTRIBUTION STATEMENT Please copy and paste the appropriate distribution statement into this space.] Oh No, DevOps is Tough to Implement! Hasan Yasar Copyright 2018 Carnegie Mellon University. All Rights
More informationCERT Resilience Management Model
CERT Resilience Management Model A Maturity Model Approach to Managing Operational Resilience SEI Webinar Series 28 July 2010 Rich Caralli Technical Manager CERT Resilient Enterprise Management Team Report
More informationPrioritizing IT Controls for Effective, Measurable Security
Prioritizing IT Controls for Effective, Measurable Security Daniel Phelps Gene Kim Kurt Milne October 2006 ABSTRACT: This article summarizes results from the IT Controls Performance Study conducted by
More informationEmpower your Smart Grid Transformation
Empower your Smart Grid Transformation David White SGMM Project Manager 10 March 2011 TWITTER: #seiwebinar 2 3 How to Participate Today Open and close your Panel View, Select, and Test your audio Submit
More informationMethodology for the Cost Benefit Analysis of a Large Scale Multi-phasic Software Enterprise Migration
Methodology for the Cost Benefit Analysis of a Large Scale Multi-phasic Software Enterprise Migration Bryce Meyer Jerry Jackson Jim Wessel Software Engineering Institute Carnegie Mellon University Pittsburgh,
More informationEffective Reduction of Avoidable Complexity in Embedded Systems
Effective Reduction of Avoidable Complexity in Embedded Systems Dr. Julien Delange Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Copyright 2015 Carnegie Mellon University
More informationAdapting Agile to the. Framework. Mary Ann Lapham, PMP, CSM Principal Engineer Software Engineering Institute
Adapting Agile to the Defense Acquisition Framework Mary Ann, PMP, CSM Principal Engineer Software Engineering Institute Carnegie Mellon University Agile?? 2013 Carnegie Mellon University Agenda The Problem
More informationArchitecture Support for Testing
Architecture Support for Testing Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Paul Clements 29 March 2011 2010 Carnegie Mellon University Goal: Explore how architecture
More informationI ve Evaluated My Architecture. Now What?
Experience with the Architecture Improvement Workshop Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Larry Jones, SEI Rick Kazman, SEI SATURN Conference, May 7, 2009 I ve
More informationFall 2014 SEI Research Review. Team Attributes &Team Performance FY14-7 Expert Performance and Measurement
Fall 2014 SEI Research Review Team Attributes &Team Performance FY14-7 Expert Performance and Measurement Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Jennifer Cowley
More informationSupporting Safety Evaluation Process using AADL
Supporting Safety Evaluation Process using AADL Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Julien Delange and Peter Feiler 12/09/2013 Safety Analysis issues (aka the
More informationFocus on Resiliency: A Process Improvement Approach to Security
Focus on Resiliency: A Process Improvement Approach to Security Introducing the Resiliency Engineering Framework Rich Caralli & Lisa Young Software Engineering Institute CSI 33 rd Annual Security Conference
More informationPRM - IT IBM Process Reference Model for IT
PRM-IT V3 Reference Library - A1 Governance and Management Sysem PRM-IT Version 3.0 April, 2008 PRM - IT IBM Process Reference Model for IT Sequencing the DNA of IT Management Copyright Notice Copyright
More informationTSP Performance and Capability Evaluation (PACE): Customer Guide
Carnegie Mellon University Research Showcase @ CMU Software Engineering Institute 9-2013 TSP Performance and Capability Evaluation (PACE): Customer Guide William R. Nichols Carnegie Mellon University,
More informationApplying CERT-RMM: Users Group Workshop Experiences. 12 th Annual CMMI Technology Conference and User Group
Applying CERT-RMM: Users Group Workshop Experiences 12 th Annual CMMI Technology Conference and User Group Julia Allen; Software Engineering Institute/CERT Program Lynn Penn; Lockheed Martin IS&GS 7 November
More informationImproving Acquisition in Government Requirements Management Leading Practices: CMMI-ACQ Visualization
the way we see it Improving Acquisition in Government Requirements Management Leading Practices: CMMI-ACQ Visualization July 2008 Capgemini Government Solutions Table of Contents 1 The Challenge: Increase
More informationArchitecture-Centric Procurement
Architecture-Centric Procurement SATURN Conference April 29 May 3, 2013 Minneapolis, MN John Bergey Larry Jones Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213-2612 Presentation
More informationAgile Development and Software Architecture: Understanding Scale and Risk
Agile Development and Software Architecture: Understanding Scale and Risk Ipek Ozkaya Research, Technology and Systems Solutions (RTSS) Program Ozkaya is a senior member of the SEI technical staff within
More informationInside of a ring or out, ain t nothing wrong with going down. It s staying down that s wrong. Muhammad Ali
MANAGING OPERATIONAL RISK IN THE 21 ST CENTURY White Paper Series Inside of a ring or out, ain t nothing wrong with going down. It s staying down that s wrong. Muhammad Ali 2 In today s competitive and
More informationCreating a Computer Security Incident Response Team Attendee Workbook
Creating a Computer Security Incident Response Team Attendee Workbook CERT Training and Education Networked Systems Survivability Software Engineering Institute Carnegie Mellon University This material
More informationThe Business Case for Systems Engineering: Comparison of Defense-Domain and Non- Defense Projects
The Business Case for Systems Engineering: Comparison of Defense-Domain and Non- Defense Projects Presenter: Joseph P. Elm The Software Engineering Institute (SEI) a DoD Research FFRDC Copyright 2014 Carnegie
More informationBusiness Resiliency Health Index Implementation
Business Resiliency Health Index Implementation April 26, 2017 15th Annual Lockheed Martin Enterprise Business Continuity & Recovery Neeta Adkar Jeremy Adkins Agenda Lockheed Martin (LM) Business Resiliency
More informationPractical Risk Management: Framework and Methods
New SEI Course! Practical Risk Management: Framework and Methods September 23-24, 2009 Arlington, VA Register at: www.sei.cmu.edu/products/courses/p78.html 1 13 th International Software Product Line Conference
More informationIt Takes an Ecosystem Gary Chastek John D. McGregor
Gary Chastek John D. McGregor Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Gary Chastek April 25, 2012 Introduction During the second half of 2011, the Software Engineering
More informationCMMI for Services (CMMI-SVC): Current State
: Current State Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Eileen Forrester April 2011 What I will cover Explain what the CMMI-SVC is and why we built it Discuss service
More informationOSATE overview & community updates
OSATE overview & community updates Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Julien Delange 04/22/2013 Overview of OSATE2 Eclipse-based AADL editor Support for AADLv2.1,
More informationAnalyzing and Evaluating Enterprise Architectures John Klein Senior Technical Staff
Analyzing and Evaluating Enterprise Architectures John Klein Senior Technical Staff John has over 20 years experience developing systems and software. He joined SEI in 2008. Before joining SEI, John was
More informationThe Business Case for Systems Engineering: Comparison of Defense-Domain and Non- Defense Projects
The Business Case for Systems Engineering: Comparison of Defense-Domain and Non- Defense Projects Presenter: Joseph P. Elm The Software Engineering Institute (SEI) a DoD Research FFRDC Report Documentation
More informationSoftware Project Management Sixth Edition. Chapter Software process quality
Software Project Management Sixth Edition Chapter 13.2 Software process quality 1 Product and Process Quality A good process is usually required to produce a good product. For manufactured goods, process
More informationSOFTWARE MEASUREMENT GUIDEBOOK. Revision 1
SOFTWARE ENGINEERING LABORATORY SERIES SEL-94-102 SOFTWARE MEASUREMENT GUIDEBOOK Revision 1 JUNE 1995 National Aeronautics and Space Administration Goddard Space Flight Center Greenbelt, Maryland 20771
More informationFinding a Vendor You Can Trust in the Global Marketplace
Finding a Vendor You Can Trust in the Global Marketplace Art Conklin Dan Shoemaker August 2008 ABSTRACT: This article introduces the concept of standardized third-party certification of supplier process
More informationRational Software White Paper TP 174
Reaching CMM Levels 2 and 3 with the Rational Unified Process Rational Software White Paper TP 174 Table of Contents Abstract... 1 Introduction... 1 Level 2, Repeatable... 2 Requirements Management...
More informationSafety Evaluation with AADLv2
Safety Evaluation with AADLv2 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Julien Delange 09/24/2013 Agenda Overview of AADL Error-Model Annex Approach for Safety Evaluation
More informationImplementation of the CO BIT -3 Maturity Model in Royal Philips Electronics
Implementation of the CO BIT -3 Maturity Model in Royal Philips Electronics Alfred C.E. van Gils Philips International BV Corporate Information Technology Eindhoven, The Netherlands Abstract: Philips has
More informationLockheed Martin Benefits Continue Under CMMI
Lockheed Martin Benefits Continue Under CMMI CMMI Technology Conference 2004 November 17, 2004 Joan Weszka Lockheed Martin Corporate Engineering & Technology Systems & Software Resource Center. CMMI is
More informationDeveloping a successful governance strategy. By Muhammad Iqbal Hanafri, S.Pi., M.Kom. IT GOVERNANCE STMIK BINA SARANA GLOBAL
Developing a successful governance strategy By Muhammad Iqbal Hanafri, S.Pi., M.Kom. IT GOVERNANCE STMIK BINA SARANA GLOBAL it governance By NATIONAL COMPUTING CENTRE The effective use of information technology
More informationOCTAVE -S Implementation Guide, Version 1.0. Volume 9: Strategy and Plan Worksheets. Christopher Alberts Audrey Dorofee James Stevens Carol Woody
OCTAVE -S Implementation Guide, Version 1.0 Volume 9: Strategy and Plan Worksheets Christopher Alberts Audrey Dorofee James Stevens Carol Woody January 2005 HANDBOOK CMU/SEI-2003-HB-003 Pittsburgh, PA
More informationUse and Organizational Impact of Process Performance Modeling in CMMI High Maturity Organizations
Use and Organizational Impact of Process Performance Modeling in CMMI High Maturity Organizations Dennis R. Goldenson James McCurley Robert W. Stoddard, II 13th Annual PSM Users Group Conference Orlando,
More informationAn Overview of the Smart Grid Maturity Model (SGMM)
An Overview of the Smart Grid Maturity Model (SGMM) Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Steve Masters 5 th annual SMART GRIDS conference March 10, 2009 Background
More informationSecurity Measurement and Analysis
Security Measurement and Analysis Christopher Alberts Julia Allen Robert Stoddard Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 This presentation is entitled. It describes
More informationWhat is ITIL 4. Contents
What is ITIL 4 Contents What is ITIL and why did ITIL need to evolve?... 1 Key Concepts of Service Management... 1 The Nature of Value... 2 How Value Creation Is Enabled Through Services... 2 Key Concepts
More informationQuality Management of Software and Systems: Software Process Assessments
Quality Management of Software and Systems: Software Process Assessments Contents Temporal development of the CMM and the assessment procedures Mature and Immature Processes Structure of the Capability
More informationThis chapter illustrates the evolutionary differences between
CHAPTER 6 Contents An integrated approach Two representations CMMI process area contents Process area upgrades and additions Project management concepts process areas Project Monitoring and Control Engineering
More informationEvaluating CSIRT Operations
Evaluating CSIRT Operations FIRST 2006 CERT Training and Education Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213-3890 CERT, CERT Coordination Center, and Carnegie Mellon
More informationImplementing Product Development Flow: The Key to Managing Large Scale Agile Development
Implementing Product Development Flow: The Key to Managing Large Scale Agile Development Will Hayes SEI Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University
More informationComplexity and Safety (FAA) SEI Board of Visitors. Sarah Sheard, Team Lead Team: Mike Konrad, Chuck Weinstock, Bill Nichols, Greg Such
Complexity and Safety (FAA) SEI Board of Visitors October 27, 2016 Sarah Sheard, Team Lead Team: Mike Konrad, Chuck Weinstock, Bill Nichols, Greg Such Software Engineering Institute Carnegie Mellon University
More informationBusiness Continuity Maturity Model (BCMM) Overview & Standards Compliance Assessment v2.5
Business Continuity Maturity Model (BCMM) Overview & Standards Compliance Assessment v2.5 Virtual Corporation, Inc. 100 Enterprise Drive Suite 301 Rockaway, NJ 07866 973-426-1444 virtual-corp.com/business-continuity
More informationGiven the competitive importance of
Making Work A successful measurement process becomes a way of doing business. is embedded in the organization, and performance improves because people are making fact-based decisions. This article describes
More informationMake Innovation Real with Unique, Leading-edge Software Solutions
SAP Brief SAP Innovative Business s Make Innovation Real with Unique, Leading-edge Software s SAP Brief Achieve your greatest goals with SAP Innovative Business s Smart digital enterprises know that success
More informationArchitecture-led Incremental System Assurance (ALISA) Demonstration
Architecture-led Incremental System Assurance (ALISA) Demonstration Peter Feiler Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 [DISTRIBUTION STATEMENT A] This material
More informationAn IT Governance Journey April Disclaimer: opinion being those of presenter(s) and not necessarily State Farm
An IT Governance Journey April 2018 Disclaimer: opinion being those of presenter(s) and not necessarily State Farm Agenda Opportunities Getting Ready COBIT 5 Application Benefits IT Governance Pattern
More informationShaping Systems Engineering and INCOSE for the Future
Shaping Systems Engineering and INCOSE for the Future Garry Roedler, ESEP INCOSE President-Elect, INCOSE Fellow and Founder Recipient, IEEE-CS Golden Core, Lockheed Martin Fellow, Engineering Outreach
More informationEileen Forrester CMMI for Services Product Manager
CMMI for Services (SVC): The Strategic Landscape for Service Eileen Forrester CMMI for Services Product Manager Forrester is the manager of the CMMI for Services Project at the Software Engineering Institute,
More informationBenchmarking Functional Verification by Mike Bartley and Mike Benjamin, Test and Verification Solutions
Benchmarking Functional Verification by Mike Bartley and Mike Benjamin, Test and Verification Solutions 36 Introduction This article describes asuremark - the Functional verification Capability Maturity
More informationImproving Your Play Book Lessons Learned from Proposal Benchmarking
Improving Your Play Book Lessons Learned from Proposal Benchmarking Presented by Howard Nutt, Executive Director to APMP SoCal Chapter Training Camp, 22 October 2010 Capability Maturity Model and CMM are
More informationBenchmarking Software Assurance Implementation. Michele Moss SSTC Conference May 18, 2011
Benchmarking Software Assurance Implementation Michele Moss SSTC Conference May 18, 2011 Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information
More informationProcess Quality Levels of ISO/IEC 15504, CMMI and K-model
Process Quality Levels of ISO/IEC 15504, CMMI and K-model Sun Myung Hwang Dept. of Computer Engineering Daejeon University, Korea sunhwang@dju.ac.kr 1. Introduction 1.1 Background The quality of a product
More informationExtended Enterprise Architecture ViewPoints Support Guide
Extended Enterprise Architecture ViewPoints Support Guide Editorial Writer: J. Schekkerman Version 1.8 2006 Preface An enterprise architecture (EA) establishes the organization-wide roadmap to achieve
More informationImproving Customer Satisfaction A People CMM Perspective
Improving Customer Satisfaction A People October 11, 2010 Palma Buttles Senior Member of the Technical Staff Software Engineering Institute Shane McGraw Program Development & Transition Software Engineering
More informationCMM,,mproving and,ntegrating
Pittsburgh, PA 15213-3890 CMM,,mproving and,ntegrating Mike Phillips Mary Beth Chrissis Mike Konrad Sandy Shrum SM SCAMPI, SCAMPI Lead Appraiser, SEPG, and SEI are service marks of Carnegie Mellon University.,
More informationMediterranean Journal of Social Sciences MCSER Publishing, Rome-Italy
Doi:10.5901/mjss.2016.v7n3s2p71 Abstract Prioritization of Components of the System of Communications and Coordination Using the People Capability Maturity Model, Case Study: Ghom General Office of Foundation
More informationPresented By: Mark Paulk
Presented By: Mark Paulk Brought To You By: Sponsored By: ASQ Software Division Invites You to Attend Held concurrently with the ASQ World Conference on Quality and Improvement May 6 8, 2013 in Indianapolis,
More informationProcess Maturity Models
EMAIL ADVISOR Volume 5, Number 5 March 20, 2007 March Sponsor Process Maturity Models Process Maturity Models are currently a hot topic in the BPM community. The Gartner Group described its BP Maturity
More informationExpert Reference Series of White Papers. ITIL Implementation: Where to Begin
Expert Reference Series of White Papers ITIL Implementation: Where to Begin 1-800-COURSES www.globalknowledge.com ITIL Implementation: Where to Begin Michael Caruso, PMP, DPSM Introduction The Information
More informationCMMI Today The Current State
CMMI Today The Current State CMMI Technology Conference 2003 November 18, 2003 Ron Paulson Vice President, Engineering Lockheed Martin Corporation CMMI is registered in the U.S. Patent and Trademark Office
More informationOrganizing For Systems Management
Organizing For Systems Management By Rich Schiesser in conjunction with Harris Kern s Enterprise Computing Institute Factors to Consider in Designing IT Organizations Few employees enjoy departmental re-structuring,
More informationChapter 6. Software Quality Management & Estimation
Chapter 6 Software Quality Management & Estimation What is Quality Management Also called software quality assurance (SQA) s/w quality:- It is defined as the degree to which a system, components, or process
More informationThe escm-sp v2: T H E e S O U R C I N G C A P A B I L I T Y M O D E L F O R S E R V I C E P R O V I D E R S ( e S C M - S P ) v 2
The esourcing Capability Model for Service Providers (escm-sp) v2 2 P A R T The escm-sp v2: T H E e S O U R C I N G C A P A B I L I T Y M O D E L F O R S E R V I C E P R O V I D E R S ( e S C M - S P )
More informationThe information contained herein is subject to change without notice.
The information contained herein is subject to change without notice. This is a QAI Copyrighted work which may not be reproduced without the written permission of QAI. You may not use these materials to
More informationBPM & PROCESS-LED TRANSFORMATION
BPM & PROCESS-LED TRANSFORMATION FEATURES Fast track BPM with comprehensive lifecycle offerings from Oracle Financial Services Domain expertise to help bridge the business IT gap and align BPM with Enterprise
More informationTechnical Report CMU/SEI-96-TR-002 ESC-TR Software Capability Evaluation Version 3.0 Method Description. Paul Byrnes Mike Phillips
Technical Report CMU/SEI-96-TR-002 ESC-TR-96-002 Software Capability Evaluation Version 3.0 Method Description Paul Byrnes Mike Phillips April 1996 Technical Report CMU/SEI-96-TR-002 ESC-TR-96-002 April
More informationCORROSION MANAGEMENT MATURITY MODEL
CORROSION MANAGEMENT MATURITY MODEL CMMM Model Definition AUTHOR Jeff Varney Executive Director APQC Page 1 of 35 TABLE OF CONTENTS OVERVIEW... 5 I. INTRODUCTION... 6 1.1 The Need... 6 1.2 The Corrosion
More informationEnhancing business continuity management to address changing business realities
IBM Global Technology Services November 2017 Thought Leadership White Paper Enhancing business continuity management to address changing business realities A business-centric approach to help reduce business
More informationTaking the Next Step: Water Sector Steering Group Review of Effective Utility Management
Taking the Next Step: Water Sector Steering Group Review of Effective Utility Management Fact Sheet - February 2016 BACKGROUND In 2007 a historic agreement was signed to jointly promote Effective Utility
More information