Build It fast & Build It Right John Watkins International Conference On Software Testing, Analysis & Review November 19-23 Stockholm, Sweden P r e s e n t a t i o n T6 Thursday 22nd November, 2001
Thursday 22 November 2001 T6 Build it Fast & Build it Right John Watkins John holds Masters Degrees in both Computer Science and Object-Orientation, has over 20 years experience in the field of software development, with some 15 years in the field of software testing, and is a Fellow of the British Computer Society. During his career as a testing professional, John has been involved at all levels and phases of testing, and has providing high level testing consultancy, training and mentoring to numerous Blue Chip Companies. John currently works for Rational Software Limited where he is the UK Testing Product Manager. John is a regular presenter at international testing conferences and events, having recently addressed the EuroSTAR 2000 conference in Copenhagen, as well providing the keynote address at the Ohjelmistotestaus 2001 conference in Helsinki. Most recently, John has published a book on Testing Process with Cambridge University Press ("Testing IT : an Off-the-shelf Software Testing Process" see www.cup.org/titles/052179546x.html).
Build it Fast and Build it Right John Watkins Product Manager - Testing, Rational UK jwatkins@rational.com
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
The New Economy: The World Runs on Software
Software is Business Speed Time-tomarket is critical Technology and standards change rapidly e-business e-infrastructure e-devices Quality Software drives customer experience High cost of failure
The software Paradox Faster Time to Market Higher Quality
Feeling the Squeeze? Today s Risks in Software Development Dissatisfied customers Weakened competitive advantage Long integration cycles Squandered creativity of developers Diminished team motivation No money ( ) No sleep (zzzz) No fun (!!!!)
Projects are Failing Gartner Group stated that: 76% of projects fail to come in on time, to budget or with acceptable quality This IS NOT ACCEPTABLE Testing is a key discipline to address these issues
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
It s a Cliché! Everyone agrees testing should occur early Everyone agrees more testing should be conducted It is well recognised that catching defects early saves time, effort and money It is well recognised that catching defects early improves quality and manages risk
It s a Cliché! Everyone agrees testing should occur early Everyone agrees more testing should be conducted It is well recognised that catching defects early saves time, effort and money It is well recognised that catching defects early improves quality and manages risk But how many developers/organisations do it?
It s a Cliché! Everyone agrees testing should occur early Everyone agrees more testing should be conducted It is well recognised that catching defects early saves time, effort and money It is well recognised that catching defects early improves quality and manages risk But how many developers/organisations do it? Not Many!
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
Quality by Design: Earlier Detection & Repair Bus. Modeling Anal. & Requirements Design Goal: find problems early in lifecycle Implementation Most problems currently found here Testing Deployment C O S T Catch quality issues as early as possible Increase project and quality predictability Improve speed, quality and customer satisfaction Decrease total project costs
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
Difficulties of Early Testing A system built of components <<Form>> dlg_order <<Class Module>> Order <<Class Module>> Customer <<Module>> Db <<Module>> Persistence <<Form>> dlg_orderrow <<Class Module>> Customers <<Class Module>> OrderRow <<Class Module>> Articles <<Module>> Article
Difficulties of Early Testing whose individual reliability is acceptable.80 <<Class Module>> Order <<Module>> Db <<Form>> dlg_order.90 <<Class Module>> Customer.95 <<Module>> Persistence.90 <<Form>> dlg_orderrow.90.95 1.0 <<Class Module>> Customers <<Class Module>> OrderRow <<Class Module>> Articles.75 <<Module>> Article.80
Difficulties of Early Testing can combine into likely failure! <<Form>> dlg_order.90 <<Form>> dlg_orderrow.90.80.90.95 1.0 <<Class Module>> Order <<Class Module>> Customer <<Class Module>> Customers <<Class Module>> OrderRow.80 x.90 x.95 x 1.0 x.90 x.90 x.95 x <<Module>>.75. Db x.80 = 0.32.95 <<Class Module>> Articles.75 <<Module>> Persistence <<Module>> Article.80
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
How do you test a Component? Two stage strategy Test individual operations of a single component Component Server Invoke Operation Return Result Component
How do you test a Component Two stage strategy Test individual operations of a single component Test sequences of operations across multiple components Component Server Invoke Operation Return Result Component
The Need for Test Harness Development No GUI suitable for testing Dependent components not yet ready Manual creation of drivers and stubs is painful Takes time away from development Impossible to test all use cases of all operations of all components Increases the cost of quality Increases the cost of development Test Harness should be tested Component C Component A Component B Component D
The Need for Realistic Test Data Test Data must be representative and realistic Typically Test Data must be hand crafted Rigorous design must be employed: Boundary analysis Partition analysis State Transition analysis Test Data must itself should be tested
The Need for Correct Test Scenarios Requirements for the Component should be tested Correct behaviour of the Component must be tested Correct interaction with other components must be tested Must include: Correct behaviour Error conditions Boundary behaviour Test Procedures should be reviewed for correctness
Component Testing the Bottom Line Test Harness development & test, Test Data development & test, Test Scenario/Procedure development & test - All take time, effort & money, and Developers are already working under significant time constraints and pressure to deliver Its no wonder Component Test is so often performed poorly, or even skipped altogether
Component Testing the Bottom Line Test Harness development & test, Test Data development & test, Test Scenario/Procedure development & test - All take time, effort & money, and Developers are already working under significant time constraints and pressure to deliver Its no wonder Component Test is so often performed poorly, or even skipped altogether After all, finding defects that s what the Testers are paid to do at System Test, isn t it?
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
Component Testing Philosophy of Test Here s a thought A Good Tester Tries to Demonstrate software doesn t work But, more often than not A Developer is much more interested in demonstrating their code does work This basic difference in philosophy of test is KEY Education will be a major issue here
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
Design for Test the Basic Principles Developers must collaborate with testers to improve quality Software Development is a Team Sport Make as much use of existing information as possible Analysis and Design information can be exploited to: Automate Test Harness Creation Automate Test Scenario Generation Automate Test Data Creation Component Testing must be made as simple as possible for Developers to perform, and perform well
Design validation Quality has to be built from the beginning Quality can be measured at design Testability is a key facet of quality
Generate Tests Automatically From Visual Model Model Developer Generate Skeleton Code code code code code code code code code code code code code code code code code code code code code code code code Provides Business Logic Consider how Developers use Modeling Case Tools today code code code code code code code code code code code code code code code code code code code code code Model Provides Code Framework Developer Provides Code Logic
Generate Tests Automatically From Visual Model Model Developer Generate Test Skeleton Code code code code code code code code code code code code code code code code code code code code code code code code Provide Test Data Consider how Developers use Modeling Case Tools today Now go one step beyond code code code code code code code code code code code code code code code code code code code code code Model Provides Test Framework Developer Provides Test Data
Exploiting Design Information <<Class>> <<Module>> <<Form>> <<Form>> <<Class>> <<Class>> <<Class>> <<Module>> Converts designs into Component tests <<Class>> <<Module>> Automated Component testing providing comprehensive coverage
Architect for Performance Model Multi-User Stress Test on Model Approved Architecture to Production
Exploiting Design in Performance Testing Design for Test approach can also be used generate Performance Tests from UML (Unified Modelling Language) Take Class diagram & Sequence diagram information Generate Performance Testing Scenarios Execute them against the Server Application Identify Architecture Issues and Scalability Problems Without Writing a single line of code! Very effective means of managing Project Risk
Sample Application Architecture
Test your design before you implement Design for Test converts system design into system test Test your design before you code
Agenda Introduction Contemporary Software Challenges Challenges of Early Testing Late Detection of Defects Component v System Testing How do you test a Component? People Issues Introducing Design for Test Wrap Up
Wrap Up Testing MUST occur earlier and more often Catching defects early saves time, effort and cost, improves quality, and helps to manage project risk Challenging project timescales and human nature mean that Component Testing is often poorly done or skipped Valuable design information needs to be utilised to simplify Component Testing The Design for Test approach allows design information to be used to simplify testing, making it more effective and more efficient
Wrap Up Testing MUST occur earlier and more often Catching defects early saves time, effort and cost, improves quality, and helps to manage project risk Challenging project timescales and human nature mean that Component Testing is often poorly done or skipped Valuable design information needs to be leveraged to simplify Component Testing The Design for Test approach allows design information to be used to simplify testing, making it more effective and more efficient Bottom Line Lets get Developers & Testers working together to deliver quality software
White Papers A White Paper discussing the Design for Test Initiative and A Technical Paper describing Rational QualityArchitect Are available from: www.rational.com/qacsuk
Build it Fast and Build it Right John Watkins Product Manager - Testing, Rational UK jwatkins@rational.com