Quality Assurance in SW Engineering for Telecommunication Systems

Transcription

1 Quality Assurance in SW Engineering for Telecommunication Systems Johan Lahtivuori, Quality Manager at Mobile Switching Systems Unit, L M Ericsson in Finland Quality Assurance in SW Engineering for Telecommunication Systems 1 Johan Lahtivuori Outline z Introduction What is Quality The role of an Operation System Quality Organisation Quality Assurance z Quality Management Planning & Reporting Management by Facts z Quality Control Why reviews? Type of reviews Preparation rate experience z Process Compliance Select your process with care Check for compliance through audits and assessments Type of findings z Preventive Quality Work Risk analyses and follow-up Prediction of possible problem products Root Cause Analyses z Quality is an Endless Journey The experience battery A safe improvement journey and some noted experiences Quality Assurance in SW Engineering for Telecommunication Systems 2 Johan Lahtivuori

2 Quality is... Quality Assurance in SW Engineering for Telecommunication Systems 3 Johan Lahtivuori Challenge: How to Get Happy Customers High Operation System Customer Satisfaction Customer Focus Commitment Competence Right product on-time with right quality Low Resources Quality Assurance in SW Engineering for Telecommunication Systems 4 Johan Lahtivuori

3 Operation System - Base for Customer Satisfaction z Right Sized Operation System Efficient work processes Planning and tracking Quality control during design Verification after design A learning organization Customer Focus Commitment Competence Resources Operation System Right product on-time with right quality High Customer Satisfaction Low Quality Assurance in SW Engineering for Telecommunication Systems 5 Johan Lahtivuori Operation Systems Can Be Misused (= Not Used) z Too much: bureaucracy police function statistics, statistics long and boring documents z Too little: no planning, only fire fighting unnecessary work re-inventing the wheels unclear tasks lack of information flow Quality Assurance in SW Engineering for Telecommunication Systems 6 Johan Lahtivuori

4 Our Operation System (= Quality System) z Available on Intranet (WWW), includes for example Responsibilities & Authorities Operational Plans Improvement Programs Performance Follow-up Process Management and Process Environment Project Management Competence Management Quality Assurance z ISO 9001 Certified Preliminary ISO 9001:2000 used internally Processes & Project Goals, Policies, Organisation Improvement, Metrics, Follow-up People Quality Assurance in SW Engineering for Telecommunication Systems 7 Johan Lahtivuori Quality Organisation = Quality is Everyone s Job Line Organisation Project Management Team Line Managers Process Owners Quality Manager Product Committees,... Development Group Project Members Reviews Quality Coordinator Main Project Quality Coordinator Happy Customer = Quality Makers = Quality Assurance Elements Quality Assurance in SW Engineering for Telecommunication Systems 8 Johan Lahtivuori

5 No quality in, no quality out z Miracles with respect to quality usually not happen in SW design if the input is of low quality, so is the output z Miracles can happen from time aspect overtime and deviations from agreed work flow might result in employee dissatisfaction zit is a relief to get ready, not an accomplishment z What do we do? Focus on requirement capturing and impact analyses Develop the work flow towards an incremental approach: start when you can, control total project and changes Cross functional teams: requirement, system, code and test aspects from beginning to the end Quality Assurance in SW Engineering for Telecommunication Systems 9 Johan Lahtivuori Quality Assurance Means to know and show where we are going (Quality Management) to control what will be delivered (Quality Control) to maintain law and order (Process Compliance) to avoid problems and learn from mistakes (Preventive Quality Work) Quality Assurance in SW Engineering for Telecommunication Systems 10 Johan Lahtivuori

6 Quality Management to know and show where we are going Quality Assurance in SW Engineering for Telecommunication Systems 11 Johan Lahtivuori Project Management by Facts Results and Control Mechanisms Plans & Goals Project Specification Budget Time Plan Quality Assurance Plan Test Plan Costs Test Data Review Data Work, Work -> Actual Data Progress Data Delivery Data Faults Quality Assurance in SW Engineering for Telecommunication Systems 12 Johan Lahtivuori

7 Quality Planning z Annual goals for the line organisation z Strategies and actions how to achieve the goals z Improvement programs established z Project specific goals and strategies are based on line organisation visions and goals and project sponsor s goals z Priorities are defined per project Quality <-> Cost <-> Functionality <-> Time z A Quality Assurance Plan deploys the Quality Assurance practices to the project Quality Assurance in SW Engineering for Telecommunication Systems 13 Johan Lahtivuori Project Quality Reporting z Monthly Quality Reports by Quality Coordinator Summary Goals and Results Events in Quality Assurance: zprocess Compliance zpreventive Quality Work zquality Control Proposals for Quality Improvements z Info shots and seminars for project members Audit findings, review statistics, instructions,... Quality Assurance in SW Engineering for Telecommunication Systems 14 Johan Lahtivuori

8 Success Factors Clear goals and strategies $ZDUHQHVV $QDO\VHV $FWLRQV Quality Assurance in SW Engineering for Telecommunication Systems 15 Johan Lahtivuori Good news, bad news and statistics z It is easy to gather a lot of data, the tough thing is to use it z What do we do? Provide the users of statistics with analyses of the data, not only the data Connect the measurements to higher level strategies and visualise it, for example through a Balanced Scorecard approach Quality Assurance in SW Engineering for Telecommunication Systems 16 Johan Lahtivuori

9 Quality Control to control what will be delivered Quality Assurance in SW Engineering for Telecommunication Systems 17 Johan Lahtivuori Why reviews? z Would you like to drive in a city where no one care about the brakes? z An inspected and approved document makes life easier for everyone Brakes not inspected! z Hints on the road Plan the reviews Assign viewpoints, everyone does not need to read everything Prepare, focus on major remarks Require and use the feedback Quality Assurance in SW Engineering for Telecommunication Systems 18 Johan Lahtivuori

10 Type of Reviews Walkthrough Early on-track indication Get to know the contents Support Alternatives Early Reviews Approve Product Committees Technical Editors Area Experts Expert Reviews Reviews (Inspections) Check / Approve Formal review process Preparations Inspection Record Remark Follow-up Quality Assurance in SW Engineering for Telecommunication Systems 19 Johan Lahtivuori Defects per Inspected Page vrs Preparation Rate z There is a strong correlation between the used preparation rate (pages read per hour during preparation) and the number of major defects found per page. z More defects per page will be found if the preparation rate is kept low. z Optimum preparation rate seem to be at 3-5 pages per hour. Quality Assurance in SW Engineering for Telecommunication Systems 20 Johan Lahtivuori

11 w Hey, there is no timee for revievs! z Time is constant, the problems could be delays to settle requirements, delays in previous phase it is hard to say NO! to changes in requirements effort estimations are faulty or incomplete pressure and progress are very closely related zdelay of code delivery is more visible than delay of specifications z What do we do? Team work and frequent (weekly) reviews of documents and code zrequires investments like team building, Controlled freedom to teams -> commitment Increased understanding of benefits with reviews Quality Assurance in SW Engineering for Telecommunication Systems 21 Johan Lahtivuori But no problem, we have the testing... z Ratio of faults (x %) slipping through tests to a live network is rather constant the more faults we introduce, the more faults are delivered to customer (and vice versa!) System Test In-service Function Test z Faults should be eliminated as close to introduction as possible - invest early or suffer later +LJK TXDOLW\ LV DFKLHYHG LQ GHVLJQ WHVWLQJ PDNHV LW H[FHOOHQW Quality Assurance in SW Engineering for Telecommunication Systems 22 Johan Lahtivuori

12 Process Compliance to maintain law and order Quality Assurance in SW Engineering for Telecommunication Systems 23 Johan Lahtivuori We shall choose proper way of working Quality Assurance in SW Engineering for Telecommunication Systems 24 Johan Lahtivuori

13 The chosen way will be checked for compliance z The chosen way is base for Estimations Schedules and resource needs Goal setting Expectations and interfaces to associated processes z Known deviations shall be agreed on from start Budget Product Quality Customer Satisfaction ON TIME Quality Assurance in SW Engineering for Telecommunication Systems 25 Johan Lahtivuori Deviations Lead to Problems 50 Quality Assurance in SW Engineering for Telecommunication Systems 26 Johan Lahtivuori

14 Process Compliance Checkpoints Project Audits Tollgate Assessments TG TG TG TG TG TG TG TG Project Project A Operation System Process Audits ISO9001 Audits CMM Assessments Product Audits Quality Assurance in SW Engineering for Telecommunication Systems 27 Johan Lahtivuori Types of Audit Findings and Examples z Deviations Reviews not done as agreed z Shortcuts Parallel working without endangering project goals z Opportunities for Improvements Introduce a known good practice to decrease lead time z Need for Information Not aware of all needed input documents for a design phase z Faults in Processes or Work Instructions Inconsistency with associated process Quality Assurance in SW Engineering for Telecommunication Systems 28 Johan Lahtivuori

15 Preventive Quality Work to avoid problems and learn from mistakes Quality Assurance in SW Engineering for Telecommunication Systems 29 Johan Lahtivuori Risk Analyses and Follow-up &KDQJHV z Risk analysis is held at start of project z Followed-up during the project z New risks added as needed Analyse risks, decide actions Follow-up & close risks Add new risks and actions Quality Assurance in SW Engineering for Telecommunication Systems 30 Johan Lahtivuori

16 Prediction of Potential Problem Products z Based on facts / feelings regarding impacts on single SW units design base quality of modified SW units stability of requirements SW unit complexity interface complexity z Actions initiated as needed expert support, actions to settle requirements, review plan focus,... z Status is followed-up during project, for example by amount of faults in tests. Quality Assurance in SW Engineering for Telecommunication Systems 31 Johan Lahtivuori Root Cause Analyses of Faults z Fault Description How it was found and the visual consequences The actual fault (in SW, in interactions, ) z Root Causes When introduced and why Where should it have been found before testing Why not found in testing z Counter Measures Concrete and clear suggestions how similar faults could be avoided in the future. Quality Assurance in SW Engineering for Telecommunication Systems 32 Johan Lahtivuori

17 Quality is... Quality Assurance in SW Engineering for Telecommunication Systems 33 Johan Lahtivuori The Experience Battery &KDQJH,PSURYH &RQWLQXH 6KDUH $QDO\VHV )DFWV DQG )LJXUHV $ OHDUQLQJ RUJDQLVDWLRQ *RRG SUDFWLFHV FRPPRQ SUREOHPV FRPPRQ VROXWLRQV 3ODQQLQJ 3UHGLFWLRQ &RQVWDQWV Quality Assurance in SW Engineering for Telecommunication Systems 34 Johan Lahtivuori

18 A Safe Improvement Journey be prepared for the unexpected... Reserve time and resources Prioritize in the big picture Involve everybody Make frequent changes a way of life and support it Know your problems and ways to deal with them Think return of investment in long term perspective Quality Assurance in SW Engineering for Telecommunication Systems 35 Johan Lahtivuori