QUEST Boston As Requirements Go, So Goes the Project. Thursday, April 7 th, :00 PM 2:00 PM. PRESENTER: Charlene Gross

Transcription

1 Thursday, April 7 th, :00 PM 2:00 PM QUEST Boston 2011 As Go, So Goes the Project PRESENTER: Charlene Gross COMPANY: Software Engineering Institute

2 This was page intentionally left blank

3 As Go, So Goes the Project Charlene C. Gross QUEST Conference Agenda Facts and Figures to Sell Better Classical Categories Mapping Classical Categories to Capability Maturity Model Integration (CMMI ) Terms Elements of Sound Engineering Rights of Customers and Analysts Not Mutually Exclusive 2

4 If Architects Had to Work like Software Developers Please design and build me a house. I am not quite sure of what I need, but my house should have between two and forty-five bedrooms. Please bring me the cost breakdown for each configuration so that I can arbitrarily pick one. The house I ultimately choose must cost less than my current one. Please take care to use modern design practices and the latest materials in construction, as I want an up-to-date showplace. However, the kitchen should be designed to accommodate, among other things, my 1952 Gibson refrigerator. Do not worry at this time about acquiring the resources to build the house itself. Your first priority is to develop detailed plans and specifications. Once I approve these plans, however, I would expect the house to be under roof within 48 hours. We like our neighbor s house a great deal, particularly the 75-foot swimming pool. With careful engineering, I believe that you can design this into our new house without impacting the final cost. 3 Dear Architect: (cont.) Please prepare a complete set of blueprints, but you don t need a real design at this time. But be advised that you will be held accountable for any increase of construction costs as a result of later design changes. You must be thrilled to be working on this interesting project! Using the latest techniques and materials, as well as having such freedom in your designs, is something that can't happen very often. Sincerely, The Client P.S. My wife disagrees with many of the instructions I've given you in this letter. As architect, it is your responsibility to resolve these differences. If you can't handle this responsibility, I will have to find another architect. P.P.S. Perhaps what I need is not a house at all, but a travel trailer. Please advise me as soon as possible if this is the case. 4

5 Dear Architect: Discussion What adjectives describe this set of house requirements? Is there anything positive about the requirements? What would you do as the architect? 5 Facts and Figures Things to Say in the Elevator 6

6 Facts and Figures - A Short Group Quiz 1. The majority of defects have their root cause in. a) poorly trained developers b) overworked testers c) poorly defined requirements 2. Finding and fixing a requirements error after delivery can cost you as much as fixing it in the requirements phase. a) 400 times b) 100 times c) 110 times 2. A requirements error can take as much time to correct in testing as in the requirements phase. a) 50 times b) 5 times c) 25 times Source: Reifer 2007; Boehm and Basili, 2001;Grady, 1999; Dabney and Barber, 2003; Nelson et al., Facts and Figures The Answers 1. The majority of defects do have their root cause in poorly defined requirements. (That s the gimme.) 2. Finding and fixing a requirements error after delivery can cost you 400 times or 100 times or 110 times as much as fixing it in phase, depending on what report you read. 3. A requirements error can take 50 times or 5 times or 25 times as much time to correct in testing as in the requirements phase, depending on what report you read. There is no single answer, BUT these results from various studies provide you with a starting point. It is the needs and measures of your organization that determine the right justification for you. 8

7 Facts and Figures as Cost Driver The earlier you discover these errors missing, wrong, conflicting, and ambiguous requirements the cheaper it is to fix them. Finding and fixing a requirements error during the requirements phase might cost you $25 to $100. If you don t find it until the construction or testing phase, fixing that same error is going to cost you $500 to $1000 (20 to 40 times more). Left undetected, that requirements error will cost you as much as 300 to 1,000 times more. That $25 cost becomes $10,000! [Reifer, 2007] Barry Boehm remarked on the impact of non-functional requirements: A tiny change in NFRs (non-functional requirements) can cause a huge change in the cost. Boehm cited the tripling of a $10 million project to $30 million when the response time (of a NFR) went from four seconds to one. [Robertson 2005]. 9 Facts and Figures as Effort Driver Source: From James Martin, as cited by Richard Bender, Based Testing Process Overview. RBTInc

8 Facts and Figures as Project Management Driver Quality Driver o The greatest control on software quality can be exercised during requirements phases. [Stevens, 1999] Planning Driver o Clear and concise communication to all the team members o Alive and active throughout the lifecycle o Solution must reflect requirements ROI Driver o BASIS FOR RESOURCE AND EFFORT ESTIMATES and thus cost and profit 11 Classical Categories 12

9 Classical Categories Business User Functional Non-functional 13 Category - Business Meaning: What the organization hopes to achieve The business benefits that the product will offer Examples of Questions for Business : o o o How will this project (product) improve your business or organization? What will you be able to do that you cannot do now? How will you know that the product is delivering the improvement you want? 14

10 Requirement Category - User Meaning: What the user requires to complete tasks Business rules, data requirements, and acceptance criteria that user will employ o o o o Examples of Questions for User Tasks that you need to accomplished? Rules you use to complete tasks? Procedures you follow? Products you produce? 15 Category - Functional Meaning: Descriptions of o Data to be entered o Operations performed o Work-flows o System reports and outputs o System access levels o Applicable regulatory requirements Examples of Questions for Functional : ofederal, state, and based payroll tax tables oemployee tax accruals and reports osupports for year-end processing osupports User defined financial reports 16

11 Requirement Type Non-Functional Meaning: Criteria for performance of the system, rather than specific behaviors Quality of service May lead to functional requirements Examples of Nonfunctional Requirement Types: o Usability o Performance o Scalability o Security o Flexibility o Portability 17 Mapping Classical Categories to Capability Maturity Model Integration (CMMI ) Terms 18

12 Brief CMMI Model Overview Process Management Premise The Process Management Premise - The quality of a system is highly influenced by the quality of the process used to acquire, develop, and maintain it. A CMMI model is not a process. A CMMI model describes the characteristics of effective processes. All models are wrong, but some are useful. George Box (1987) (Quality and Statistics Engineer) 19 CMMI Model Overview Three Complementary Constellations DEV = Installation ACQ = Acquisition SVC = Services Shared PA Each of the model constellations provides ways of implementing process improvement to achieve business objectives 20

13 Brief CMMI Model Overview Process Maturity Levels Process measured and controlled Process characterized for the organization and is proactive Process characterized for projects and is often reactive Process unpredictable, poorly controlled, and reactive Focus on continuous process improvement ML 1 ML 2 ML 3 ML 4 ML 5 Optimizing Quantitatively Managed Defined Managed Initial 21 CMMI and Categories Customer Discover & Allocate Refine Product Allocate Product- Component Derived 22

14 CMMI and Customer Customer o An understanding of what will satisfy stakeholders o Transformed stakeholder needs, expectations, constraints, and interfaces o May be stated in technical or non-technical terms o May also provide specific design requirements 23 CMMI and Product Example Product Work Product Delivered to Customer o More detailed and precise sets of requirements o Expressed in technical terms or parameters Functionality, including actions, sequence, inputs, and outputs Qualities it must possess Constraints that the system and its development must satisfy 24

15 CMMI and Product Component Example Product-Component Lower Level Components of the Product o Allocated from product requirements o Complete specification, including fit, form, function, performance, and any other requirement o Sufficiently technical for use in the design of the product component o Example - a car engine and a piston are product components of a car (the product) 25 CMMI and Derived Example Derived Discovered and/or Implied from: o Customer requirements o Applicable standards, laws, policies, common practices, and management decisions o Contractual commitments such as data rights for delivered software products, interfaces, terms and condition, delivery dates, and milestones with exit criteria o Factors arise as part of selected architecture, design decisions, or developer s unique business considerations o May also address the cost and performance of other lifecycle phases and other non-technical requirements: 26

16 Relating Two Schemas Customer Business Traditional Terms Product User Terms used in CMMI Product- Component Derived Functional Non-Functional 27 Relating Two Schemas Customer Business Traditional Terms Product User Terms used in CMMI Product- Component Derived Functional Non-Functional 28

17 Relating Two Schemas Customer Business Traditional Terms Product User Terms used in CMMI Product- Component Derived Functional Non-Functional 29 Relating Two Schemas Customer Business Traditional Terms Terms used in CMMI Product Product- Component Derived User Functional Non-Functional 30

18 Relating Two Schemas Customer Business Traditional Terms Product Terms used in CMMI Product- Component Derived User Functional Non-Functional 31 Elements of Sound Engineering 32

19 Use a Variety of Techniques to Elicit Selected Examples o Questionnaires, interviews, use cases, and scenarios (operational, sustainment, and development) obtained from end users o Operational, sustainment, and development walkthroughs and end-user task analysis o Quality attribute elicitation workshops with stakeholders o Prototypes and models o Market surveys o Business case analysis o Extraction from sources such as documents, standards, or specifications o Observation of existing products, environments, and workflow patterns o Delivering small incremental vertical slices of product functionality 33 Ensure Training of Analyst Examples training topics for Analysts soft skills o Basic facilitation skills and rules o Active listening skills and interpreting body language o Techniques for leading effective group dynamics o How to handle problem people o Presentation skills Examples training topics for Analysts requirements writing skills o Different levels of requirements o Construction - active voice and complete sentences with proper grammar, spelling and punctuation o Language usage clear, concise, consistent, unambiguous o Use case development and tools o Authoring operational concepts and scenarios 34

20 Ensure Training of User Examples of training topics for Users o How requirements fit into the Project Life Cycle o Understanding the different levels of requirements o Characteristics and guidelines for writing effective business requirements o Practical exercises in writing problem statements, business objectives, requirements lists, user stories and non-functional requirements o How to trace requirements o Package software evaluation and selection process o What to expect during scoping sessions and requirements gathering sessions 35 Implement Management Best Practices - Change Control Change Control Processes controlling and authorizing changes o Documentation and baseline of requirements o Submission and documentation of changes o Impact analysis and negotiation with stakeholders o Change Control Board Infrastructure o Update and recording of disposition of change request 36

21 Implement Management Best Practices Configuration Management Configuration Management Processes ensuring correct version availability o Configuration management of requirements repository and deliverables o Version maintenance and history throughout iterations o Designated read, write, delete and update permissions o Check In-Check out capability o Labeling and annotation schemas 37 Implement Management Best Practices Traceability Traceability Processes forward, backward, and sideways o Vertical end-to-end traceability o Horizontal traceability impact of changes on networking, hardware, interfaces, etc o Capture of allocation rationale, accountability, and test/validation o Capabilities to view/trace links o History of requirements changes 38

22 Implement Management Best Practices Status Tracking Status Processes status of activity on requirements o Categories for status, e.g., proposed, approved, implemented, verified, deleted, and/or rejected o Methods of tracking o Escalation standards 39 Implement Management Best Practices Measures Measures Measuring requirements process activities o change requests status, number, age o Number of requirements in a particular status category o Time spent on traceability and other requirements activities o Volatility requirements withdrawn and inserted throughout the process 40

23 Managing Customer Expectations A Bill of Rights and a Bill of Responsibilities 41 Origin and Importance Developed by Karl E. Wiegers for his book, Software Delineates what customer should expect from project team Clarifies what customer needs to commit to providing to project team 42

24 Customer Bill of Rights 1. Expect analysts to speak your language. 2. Expect analysts to learn about your business and your objectives. 3. Expect analysts to structure the information you present during requirements capture into a written software requirement specification. 4. Have developers explain all work products created from the requirements process. 5. Expect developers to treat you with respect and to maintain a collaborative and professional attitude throughout your interactions. 6. Have developers provide you with ideas and alternatives both for your requirements and for implementation of the product. 43 Customer Bill of Rights 2 7. Describe characteristics of the product that will make it easy and enjoyable to use. 8. Be presented with opportunities to adjust your requirements to permit reuse of existing software components. 9. Be given good-faith estimates of cost, impacts, and tradeoffs when you request a change in the requirements. 10. Receive a system that meets your functional and quality needs, to the extent that those needs have been communicated to the developers and agreed upon. 44

25 Customer Bill of Responsibilities 1. Educate analysts about your business and define business jargon. 2. Spend the time it takes to provide requirements, clarify them, and iteratively flesh them out. 3. Be specific and precise when providing input about the system s requirements. 4. Make timely decisions about requirements when requested to do so. 5. Respect a developer s assessment of the cost and feasibility of requirements. 6. Set priorities for individual requirements, system features, or use cases 7. Review requirements documents and prototypes. 45 Customer Bill of Responsibilities 2 8. Communicate changes to the project requirements as soon as you know about them. 9. Follow the development organization s defined process for requesting requirements changes. 10. Respect the processes the developers use for requirements engineering. 46

26 Bibliography 1 1. Bender, R. " Based Testing Process Overview". Bender RBT Inc Boehm, B. and Basili, V.R. Software Defect Reduction Top 10 List. Software Management, January Box, G. E. P. and Draper, N. R. Empirical Model-Building and Response Surfaces. Wiley Dabney, J. B. and Barber, G. Direct Return on Investment of Software Independent Verification and Validation: Methodology and Initial Case Studies. Assurance Technology Symposium, June Grady, R. B. An Economic Release Decision Model: Insights into Software Project Management. Proceedings of the Applications of Software Measurement Conference Nelson, M., Clark, J., and Spurlock, M.A.. Curing the Software and Cost Estimating Blues. The Defense Acquisition University Program Manager Magazine, November December Bibliography 2 7. Reifer, D.J., Profiles of Level 5 CMMI Organizations. Crosstalk, January Robertson, S. and Robertson, J. Preface. -Led Project Management Discovering David s Slingshot. By Barry Boehm. Pearson Education, Stevens, R. and Martin, J. What is Management? QSS, Inc. (1999). 10. Wiegers, K. E. Software. Redmond, WA: Microsoft Press. (1999). 48

27 Questions 49 Contact Information Slide Format Presenter / Point of Contact Charlene C Gross Senior Member of Technical Staff Acquisition Support Program Telephone: jgross@sei.cmu.edu cgross@sei.cmu.edu Web U.S. Mail Software Engineering Institute Customer Relations 4500 Fifth Avenue Pittsburgh, PA USA Customer Relations info@sei.cmu.edu Telephone: SEI Phone: SEI Fax:

28 THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the trademark holder. This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at This work was created in the performance of Federal Government Contract Number FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to the copyright license under the clause at