Front end GUI automation ROI: a changing definition

Transcription

1 Frnt end GUI autmatin ROI: a changing definitin Capgemini India Private Limited July 2013 Prepared by: Angshuman Dutta

2 1 Table f Cntents 1. Abstract Autmatin ROI backgrund Traditinal expectatins Changed trend Autmatin ROI- the frmula Cnclusin Authr s Bigraphy...5

3 2 1. Abstract It has been mre than decades nw that frnt end GUI autmatin is cntinued t be a buzz wrd in sftware testing wrld. Hwever, the actual return n investing in autmatin is still nt clear with it s end users. Traditinally, there are many ppular benefits f using test autmatin can be discussed abut but the biggest barrier t adptin remains the level f maintenance required t sustain it. At times, significant time & cst needs t investment in achieving a mderate level f autmatin. With recent trends f mre business driven sftware develpment & increased cmplexity f frnt end applicatins, the benefits ut f investing sftware test autmatin cmes under questin. 2. Autmatin ROI backgrund 2.1 Traditinal expectatins Originally test autmatin was cnsidered used as a tl that can reduce the cst f executin by remving the manual resurces/testers. Custmers were ready t invest n autmatin as a netime investment & expecting fllwing benefits: Executin time can be drastically reduced (almst zer) cmpared t manual executin Tests can be easily repeated with different set f test data seamlessly. All tests can be autmated Zer ramp up time Autmated cmprehensive test planning One tl that fits perfectly With the benefits lk extremely prmising mre & mre custmers started investing in autmatin. The practical results were nt prved t be as beautiful as it was dcumented n thery. In fact sme f the autmatin prjects had failed s badly that the custmers started dubting the benefits f autmatin. The frmula f ROI eventually appears t be nt s simple fr test autmatin. 2.2 Changed trend As sftware test mdels becme mre & mre mature, the custmers are analysing the facts befre investing in autmatin. It becmes clear t everyne that investment fr any autmatin is nt directly prprtinal t it s benefits. Similar t any ther prject, planning, identifying risk & mitigatin plan becmes key fr the success f any autmatin prject. It s becme imperative t analyse the belw aspects f any testing prject while deciding t include autmatin Pririty - pririty is driven by business & hence the need fr autmatin. Test suitability cmplicated system with mre applicatins are difficult t autmate. Test effrt if the manual effrt itself is less, there is little scpe t imprve further. Reuse effrt mre the executin, mre autmatin benefits can be prjected. Test cverage - functinal testing that cntains large areas f repletin are the right ne t autmate. Dcumentatin autmatin is difficult t start if detailed dcumentatin is nt available.

4 3 Test envirnment stability f test envirnment is key fr autmatin. Technlgy special cntrls, nn-standard develpment practices can be a bttle neck fr autmatin. With this the success rate f autmatin prjects started imprving cnsistently & custmers started lking int the lng term benefits f autmatin. 3. Autmatin ROI- the frmula Autmatin in mdern day is nt nly abut implementing advance tls r efficient cding techniques but mre f adpting strategic appraches /mdels. The varius autmatin mdels are further custmized as per business needs t enhance mre benefits ver cst. A) Pririty based autmatin: Priritizing autmatin f simpler & business critical manual checks (like smke test /Ready fr business) shws immediate return n investments. These checks are almst used n daily basis with minimal maintenance.mre the usage, mre the return n autmatin investment. The ROI f autmatin in this case can be demnstrated t the custmer withut taking help f any cmplicated frmula. ROI = (Benefits f autmatin)/(cst f autmatin) B) Framewrk driven autmatin: With the advent f framewrks, the heavy cst f test autmatin maintenance reduced t significant level. The autmatin scripts becme mre maintainable & they becme available during mst f the executin cycles. Custmers able t see lng term benefit f investing additinal cst & time in develping autmatin framewrk. Develp framewrk autmatin Autmatin scripts develped quickly fr the applicatin leveraging the develped framewrk Less-maintenance turnarund time Is maintenance required? Execute autmatin scripts ROI = (Benefits f autmatin + reduced maintenance cst + reduced script develpment cst)/ (Cst f autmatin + cst f framewrk develpment)

5 4 C) Autmatin service mdel: Develpment ging int agile mde, it s becme imperative t have a hlistic apprach twards autmatin. Autmatin service mdels are primarily designed t define autmatin strategies driven by future ROI prjectins. Cllect & analyse data fr varius applicatins Calculate prjected ROIs fr each applicatin Based n ROI inputs priritize applicatins t autmate Develp framewrk autmatin Develp scripts fr Applicatin -1 Develp scripts fr Applicatin -2 Develp scripts fr Applicatin -3 Develp scripts fr Applicatin -4 Executin service mdel t strategise the executin apprach & cnduct autmatin Cntinuusly cllect pst executin data t calculate ROI This mdel prves t prvide maximum return n investment nt nly saves direct csts but als prvide benefits in leveraging autmatin artifacts at framewrk level acrss different applicatins f separate business units. Belw are primary data pints that have been based lined t derive ROI frmula - Manual executin effrt Autmatin executin per release Releases tested per year Other autmatin executin per year Develpment duratin Replacement savings Avidance saving Framewrk leverage Test set leverage Test envirnment dwntime.

6 5 The data cllected is further analysed t derive the ttal ROI f test autmatin effrt. Eventually, the ttal ROI in this case is fund t be a cmbinatin cst savings identified at different levels Maintenance saving Leverage saving Executin time saving 4. Cnclusin Return n investing in autmatin prjects is n lnger a myth nw. The expectatin frm test autmatin in tday s planet is much mre beynd a direct cst saving mdel. Advancement f technlgies in bth sftware develpment & new autmatin tls has taken the custmer expectatins t new level. Custmers are lking fr many intangible benefits apart frm direct cst save. This enhances the necessity f mre accurate ROI calculatin s as t prject mre realistic benefits. 5. Authr s Bigraphy Angshuman Dutta is a PMP certified & a Subject Matter Expert in Autmatin Testing. He wrks as a Test lead in Capgemini Sftware Private Limited. He has arund 11 years f IT experience and arund 8 years f experience in Test autmatin. He is a active member f Autmatin Centre Of Excellence fr a large accunt in Capgemini and has been instrumental in defining methdlgies, designing and driving varius ff-the-shell cst effective and highly prductive autmatin mdels.