Software and Systems Measurement and Analysis

Transcription

1 Software and Systems Measurement and Analysis Cheryl Jones Paul Janusz US Army TACOM-ARDEC Software Intensive Systems Evaluation July 25, 2000 PSM User s Conference - 1

2 Goal Implement measurement on TACOM- ARDEC programs - Use practical experience base - Share lessons learned - Integrate SQA and process improvement activities PSM User s Conference - 2

3 Objectives Show Project Leaders How to Plan and Implement the M&A Process for QED Projects Implement Project Level M&A - Practical Software and Systems Measurement - Technical Guidance and Practice Provide an Understanding of the QED M&A Policy and Procedures Help Develop and Tailor M&A Project Plans for Specific QED Projects PSM User s Conference - 3

4 Approach M&A Overview Measurement & Analysis Overview and Perspective PSM PSM 4.0 Curriculum M&A Policy and Procedures M&A Project Planning Measurement and Analysis Policy and Procedures Measurement and Analysis Project Plan Development PSM User s Conference - 4

5 Types of Measurement Measuring the Initiative - Performance measurement - PEG Infrastructure Progress, Adoption, Impact Measuring the Processes - Individual process capability, improvements - QED organizational maturity Measuring the Projects - Software and Systems - Project activities and products PSM User s Conference - 5

6 Process Improvement Initiative WBS 1.0 Infrastructure 2.0 Project Application 1.1 Initiative Management 2.1 Software Quality Technical Tasks Management & Organizational Plans Acquisition Planning Support Financial Management Source Selection Support Resource Management Risk Management Management Reviews Measurement & Analysis 1.2 Process Definition & Implementation Documentation Analysis Policy Development Process Assessments Procedure Development Design Analysis 1.3 Support Resource Definition & Implementation Code Analysis Measurement Repository Software Safety Analysis Process Asset Library Software Evaluation Planning DM/CM Repository Software Evaluation Execution 1.4 Training Functional/Physical Conf. Audits Manage Training Program Materiel Release Preparation Plan Training Generic SW Quality PM Support Conduct Training Other 1.5 Internal Assessment & Evaluation 2.2 Software Quality Task Management Internal Quality Assurance WBS Development & Estimating Organizational Assessments Project Plan Development 1.6 Config./Data Management Peer Reviews Configuration Management Internal Quality Assurance Data Management Subcontract Management 1.7 Performance Management Internal Performance Measurement Risk Management Management Reviews Measurement & Analysis Submission of Lessons Learned PSM User s Conference - 6

7 The ARDEC Measurement Conspiracy Practical Software Measurement CMMI M&A ISSUE DRIVEN MEASUREMENT PROCESS ISO/IEC PSM User s Conference - 7

8 Measurement and Analysis History Practical Software Measurement Contains Detailed How-To Guidance on Implementing Measurement - PSM Was the Base Document for ISO/IEC ISO/IEC 15939, Software Measurement Process, Contains the Required Activities and Tasks for a Measurement Process Was the Source for the CMMI M&A Process Area Definition CMMI M&A Process Area Describes How to Evaluate a Measurement Process PSM User s Conference - 8

9 Performance Management Policy Addresses Three Process Areas: - Measurement and Analysis - Risk Management - Decision Analysis and Resolution PSM User s Conference - 9

10 Measurement and Analysis Policy Applies to QED Projects Issue-driven Process Based on Project Information Needs Information Needs Will Be Identified An Appropriate Set of Measures, Based on the Information Needs, Will Be Identified PSM User s Conference - 10

11 Measurement and Analysis Policy Measurement Activities Will Be Planned and Documented in a Project Measurement Plan Required Data Will Be Collected, Stored, Analyzed, and Results Interpreted Report Analysis Results to Management at Defined Points to Support Decisions Project Team Will Implement the Measurement Process IAW Project Measurement Plan and Procedures PSM User s Conference - 11

12 Measurement and Analysis Policy Measurement Process and Measures Will Be Evaluated and Improvements Noted Integrate Measurement Policy With Other Policies Resources Shall Be Allocated, Responsibility and Authority Assigned Measurement Procedures May Be Tailored - Don t Tailor Policy PSM User s Conference - 12

13 Measurement Activities Information Needs Technical and Management Processes User Feedback Analysis Results Core Measurement Process Implement Process Tailor Measures Apply Measures Measurement Plan Scope of PSM Improvement Actions Evaluate Process Analysis Results and Performance Measures PSM User s Conference - 13

14 Tailoring Measures Project Information and Characteristics Information Needs Identify and Prioritize Project Issues New Issues Select and Specify Project Measures Process Characteristics Proposed Changes Proposed Changes Integrate Into the Technical & Management Processes Measurement Plan PSM User s Conference - 14

15 Applying Measures Measurement Plan Collect and Process Data Data Periodic Assessment New Issues Project Context Information Analyze Issues Information Questions Risk Management and Financial Performance Results Make Recommendations Analysis Results, Performance Measures PSM User s Conference - 15

16 Implementing a Measurement Process Obtain Organizational Support Define Responsibilities Provide Resources PSM User s Conference - 16

17 Evaluating a Measurement Program Evaluate Measures and Indicators Analysis Results Performance Measures Evaluate the Measurement Process Proposed Changes Update the Experience Base Proposed Changes Identify and Implement Improvements Improvement Actions PSM User s Conference - 17

18 Measurement and Analysis Procedure Activity Task Step Responsible Parties Product Produced/Output Documented In Link to CMMI PSM User s Conference - 18

19 Measurement and Analysis Procedure (cont) CMMI Activity Task Step Responsible Parties Goal 1, Activity 2, SP 1.2 Select and Specify Project Measures Identify Measurement Category PL, IPT 7 Goal 1, Activity 2, SP 1.2 Select Applicable Measures PL, IPT 8 Goal 1, Activity 2, SP 1.2 Specify Data Requirements IPT 9 PSM User s Conference - 19

20 Measurement and Analysis Procedure (cont) 7 For each common issue area, identify candidate measurement categories to identify the type of information required. This can be done by comparing the Project Manager's questions to the questions identified in the measurement categories in Practical Softwa 8 Select appropriate measures that report back on each information need and common issue area. Practical Software Measurement, Part 3, Measurement Selection and Specification Tables, and Part 5, Measurement Analysis and Indicator Examples, provide examples. 9 Develop the Measurement specification for each measure. Guidance is provided in Practical Software Measurement, Part 3, Measurement Selection and Specification Tables. Follow the template in Appendix B. Product Produced/Output Measurement Categories, Questions Addressed Selected Measures General Measurement Specification and Specific Data Specification Documented in Project Measurement Plan, Project Measurement Database (PSM Insight) Project Measurement Plan, Project Measurement Database (PSM Insight) Project Measurement Plan, Project Measurement Database (PSM Insight) PSM User s Conference - 20

21 Measurement Plan Template Introduction Project Description Measurement Approach Project Software Information Needs Software Measurement Specifications Project Aggregation Structures Project Reporting Approach PSM User s Conference - 21

22 Project Measurement Specification Information Need - Information Need Name - Information Need Description - Reporting Mechanism Indicators - Indicator Name - Indicator Description - Analysis Methods - Decision Criteria Derived Measures - Derived Measure Name - Derived Measure Description - Aggregation Structure PSM User s Conference - 22

23 Project Measurement Specification (cont) Base Measures - Base Measure Name - Base Measure Definition - Aggregation Structure - Attributes - Criteria for Counting Actual - Collection Mechanisms - Collection Periodicity PSM User s Conference - 23

24 Available SWISE Measurement Support Recommendations on Project Plans Review of Completed Project Plans Facilitation of Measurement Planning Workshops Support on Project Measurement Planning and Implementation Measurement Analysis PSM User s Conference - 24

25 For More Information SWISE - PSM Support Center John McGarry Cheryl Jones Paul Janusz Mia Hernandez x7007 jmcgarry@pica.army.mil x2644 cljones@pica.army.mil x4849 janusz@pica.army.mil x7227 miahern@pica.army.mil (973) (FAX) psm@pica.army.mil PSM Home Page: PSM User s Conference - 25