During the past few years, the

Transcription

1 The Rosetta Stone Makng Estmates Work wth II Donald J. Refer, Refer Consultants, Inc. Barry W. Boehm and Sunta Chulan, Unversty of Southern Calforna As part of our efforts to help Constructve Cost Model () users, we, the research team at the Center for Software Engneerng at the Unversty of Southern Calforna (USC), have developed the Rosetta Stone to convert fles to run usng the new II software cost estmatng model. The Rosetta Stone s extremely mportant because t allows users to update estmates made wth the earler verson of the model so that they can take full advantage of the many new features ncorporated nto the II package. Ths artcle descrbes both the Rosetta Stone and gudelnes to make the job of converson easy. Durng the past few years, the team at USC has been workng to update the 19 verson of the estmatng model ( ) [1]. The new verson of the model, called II, bulds on the experences that ndustral afflates of the USC Center for Software Engneerng have had and addresses lfecycle processes and paradgms that have become popular snce the orgnal model was frst ntroduced n 19. These new paradgms nclude reuse-drven approaches, commercal-off-the-shelf lfecycle developments, component-based software engneerng approaches, use of object-orented methods, and other mprovements to the way we do busness stmulated by process mprovement ntatves. Ths artcle focuses attenton on a tool we have developed to permt our users to update ther orgnal fles so that they can be used wth the II model. We call the tool the Rosetta Stone because t s not unlke the black stone slab found n Egypt by French troops n On t were engraved three scrpts (Greek, Demotc, and heroglyphcs), whch enabled archaeologsts to construct translatons among the three languages. Our Rosetta Stone permts ts users to translate fles prepared wth the orgnal model to be compatble wth II. Many of our afflates thought creaton of the Rosetta Stone was mportant because they had wanted to use the new verson of the model to take advantage of ts many advanced capabltes, ncludng the II package s autocalbraton features. However, they could not make the move because they had fles that requred older versons of the model to run, e.g., and Ada-. Others wanted to calbrate the new verson of the model usng ther hstorcal databases, but the new verson of the model had a new structure, altered mathematcs, and dfferent parameters and parametrc ratngs. Under such crcumstances, convertng fles was no easy task. The II Estmatng Model The major dfferences between and II, why they are mportant, and cost drver defntons are summarzed n Table 1. These changes are mportant because they reflect how the state of software engneerng technology has matured durng the past two decades. For example, programmers were submttng batch jobs when the model was frst publshed. Turnaround tme mpacted ther productvty. Therefore, a parameter TURN was used n the model to reflect the average wat programmers experenced before recevng ther job back. Such a parameter s no longer mportant because most programmers have nstant access to computatonal facltes through ther workstatons. Therefore, the parameter has been removed n the II model. The followng summary hghlghts the major changes made to the orgnal verson of as II was developed. II addresses the followng three phases of the spral lfecycle: applcatons development, early desgn, and post-archtecture. The three modes n the exponent are replaced by fve scale factors. The followng cost drvers were added to II: DOCU, RUSE, PVOL, PEXP, LTEX, PCON, and SITE. The followng cost drvers were deleted from the orgnal : VIRT, TURN, VEXP, LEXP, and MODP. The ratngs for those cost drvers retaned n II were altered consderably to reflect more upto-date calbratons. The Rosetta Stone As llustrated n Table 2, users need to convert factors n the equatons (such as the exponent, the sze estmate, and the ratngs for the cost drvers) from the orgnal to the new verson of the model. We suggest that users employ the followng four steps to make the converson so orgnal fles can be used wth the II model. Update Sze The orgnal cost estmatng model used delverable source lnes of CROSSTALK The Journal of Defense Software Engneerng 11

2 Software Engneerng Technology Model Structure Mathematcal Form of Effort Equaton Exponent code (DSI) as ts measure of the sze of the software job. DSI were orgnally represented by card mages, e.g., ncludes all non-comment, nonblank carrage returns. II uses the followng three measures to bound the volume of work assocated wth a software job: source lnes of code (SLOC), functon Sngle model that assumes you start requrements allocated to software. E Effort = A(c (Sze) xponent ) wth Exponent = Fxed constant selected as a functon of mode. Organc = 1.05 Semdetached = 1.12 Embedded = 1.20 S ze LOC (wth extensons for FPs). II Three models that assume you progress through a spral-type development to soldfy your requrements, soldfy the archtecture, and reduce rsk. Effort = A( c (Sze) Exponent ) Exponent = Varable establshed based on ratng of fve scale factors. PREC: Precedentedness FLEX: Development Flexblty RESL: Archtecture or Rsk Resoluton TEAM: Team Coheson PMAT: Process Maturty S Object ponts, FPs, or SLOCs. Cost Drvers (c ) 15 drvers, each of whch must be rated Other Model Dfferences Table 1. Model comparsons. RELY: Relablty DATA: Database Sze CPLX: Complexty TIME: Executon Tme Constrant STOR: Man Storage Constrant VIRT: Vrtual Machne Volatlty TURN: Turnaround Tme ACAP: Analyst Capablty PCAP: Programmer Capablty AEXP: Applcatons Experence VEXP: Vrtual Machne Experence LEXP: Language Experence TOOL: Use of Software Tools MODP: Use of Modern Programmng Technques SCED: Requred Schedule Model based on Lnear reuse formula. Assumpton of reasonably stable requrements. 17 drvers, each of whch must be rated RELY: Relablty DATA: Database Sze CPLX: Complexty RUSE: Requred Reusablty DOCU: Documentaton TIME: Executon Tme Constrant STOR: Man Storage Constrant PVOL: Platform Volatlty ACAP: Analyst Capablty PCAP: Programmer Capablty AEXP: Applcatons Experence PEXP: Platform Experence LTEX: Language and Tool Experence PCON: Personnel Contnuty TOOL: Use of Software Tools SITE: Multste Development SCED: Requred Schedule Has many other enhancements, ncludng Nonlnear reuse formula. Reuse model that looks at effort needed to understand and assmlate. Breakage ratngs used to address requrements volatlty. Autocalbraton features. ponts (FPs), and object ponts. SLOCs are counted usng logcal language statements per Software Engneerng Insttute (SEI) gudelnes [2], e.g., IF- THEN-ELSE, ELSE IF s consdered one, not two, statements. Table 2 provdes gudelnes to convert sze n DSI to SLOCs so that they can be used wth the II model. Whenever possble, we recommend usng counts for the actual sze of the fle nstead of the orgnal estmate. Such practces allow you to correlate your actuals, e.g., the actual applcaton sze wth the effort requred to do the work assocated wth developng the software. The reducton n sze for II estmates s attrbutable to s need to convert card mages to SLOC counts. As already noted, the par IF-THEN-ELSE and END IF would be counted as two card mages n and as a sngle source nstructon n II. The gudelnes offered n Table 2 are based on statstcal averages to smplfy conversons; however, we encourage you to use your actuals f you have them. The followng are two common msconceptons about s use of SLOC and FPs: Msconcepton 1: does not support the use of FPs FP versons of have been avalable snce the Before You Leap commercal software package mplementaton n As noted n Table 1, II supports use of ether SLOC or FP metrcs. In both cases, ths s done va backfrng tables, whch permt you to convert FPs to SLOCs at dfferent levels. Msconcepton 2: Although t s rresponsble to use SLOC as a general productvty metrc, t s not rresponsble to use FP as a general szng parameter for estmaton Ths msconcepton breaks down nto the two followng cases. Your organzaton uses dfferent language levels to develop software. In ths case, t s rresponsble to use SLOC as your productvty metrc, snce you get hgher productvty per SLOC at hgher language levels; however, t also s rresponsble to use FP as a general szng metrc because pure FP wll generate the same cost (or schedule or qualty) estmate for a program wth the same functonalty developed 12 CROSSTALK The Journal of Defense Software Engneerng

3 The Rosetta Stone: Makng Estmates Work wth II Table 2. Convertng sze estmates. usng dfferent language levels. Ths s clearly wrong. To get responsble results n ths case, FP-based estmaton models need to use some form of backfrng to account for the dfference n language level. Your organzaton always uses the same programmng language (level). Here, t s responsble to use pure FP as your szng metrc for estmaton. But t also s responsble to use SLOC as your productvty metrc. Both metrcs work n practce. Convert Exponent Convert the orgnal modes to scale factor settngs usng the Rosetta Stone values n Table 3. Then, adjust the ratngs to reflect the actual stuaton. For example, the Rosetta Stone rates process maturty (PMAT) low because most projects usng are assumed to have been at Level 1 on the SEI process maturty scale [5]. However, the project s actual ratng may have been hgher and an adjustment may be n order. An excepton s the PMAT scale factor, whch replaces the Modern Programmng Practces (MODP) multplcatve cost drver. As seen n Table 4, MODP ratngs of very low (VL) or low (L) translate nto a PMAT ratng of VL or a low level on the Software Capablty Maturty Model scale. An MODP ratng of normal (N) translates nto a PMAT ratng of L or a hgh Level 1. An MODP ratng of hgh (H) or Table 3. Model scale factor converson ratngs. Mode and Scale Factors Precedentedness Development (FLEX) (PREC) Flexblty Archtecture and Rsk Resoluton (RESL) Team Coheson (TEAM) DSI Second-Generaton Languages Thrd-Generaton Languages Fourth-Generaton Languages Object-orented Languages Functon Feature Ponts Ponts Organc Semdetached Embedded II SLOC [3] Reduce DSI by 35 percent. Reduce DSI by 25 percent. Reduce DSI by 40 percent. Reduce DSI by 30 percent. Use the expanson factors developed by Capers Jones [4] to determne equvalent SLOCs. Use the expanson factors developed by Jones to determne equvalent SLOCs. Process Maturty (PMAT) L L L VH Capers N very hgh (VH) translates nto a PMAT ratng of N or CMM Level 2. As wth the other factors, f you know that the project s actual ratng was dfferent from the one provded by the Rosetta Stone, use the actual value. The movement from modes to scale factors represents a major change n the model. To determne the economes and dseconomes of scale, fve factors have been ntroduced. Because each of these factors can nfluence the power to whch sze s rased n the equaton, they can have a profound mpact on cost and productvty. For example, to ncrease the ratng from H to VH n these parameters can ntroduce as much as a 6 percent swng n the resultng resource estmate. Most of these factors are modern n ther dervaton. For example, the concept of process maturty was not n ts formatve stages when the orgnal model was publshed. In addton, the fnal three factors, RESL, TEAM, and PMAT, show how an organzaton can exercse management control over ts dseconomes of scale. Fnally, the frst two, PREC and FLEX, are the less controllable factors contrbutng to modes or nteractons. Rate Cost Drvers The trckest part of the converson s the cost drvers. Cost drvers are parameters to whch cost s senstve. For example, as wth the scale factors, you would expect that use of experenced staff would make a software development less expensve; otherwse, why use them? Because the new verson of the model uses altered drvers, the Rosetta Stone converson gudelnes outlned n Table 4 are mportant. For those nterested n more detals about the cost drvers, we suggest you refer to the II Model Defnton Manual [6]. Agan, the ratngs need to be adjusted to reflect what actually happened on the project. For example, the orgnal estmate may have assumed that analyst capablty was very hgh; however, the calber of analysts actually assgned mght have been nomnal because key employees were not avalable to the project when they were needed. Users should take advantage of ther knowledge of what occurred on the project to make ther estmates more reflectve of what really went on as the applcaton was developed. Use of such knowledge can mprove the credblty and accuracy of ther estmates. The TURN and TOOL ratng scales have been affected by technology changes snce 19. Today, vrtually everyone uses nteractve workstatons to develop software. TURN has therefore been dropped from II and ts calbraton assumes the TURN ratng s L. Table 5 provdes alternatve multplers for other TURN ratngs. The tool sutes avalable n the 1990s far exceed the VH TOOL ratng, and vrtually no projects operate at the VL or L TOOL levels. II has shfted the TOOL ratng scale two levels hgher so that a N TOOL ratng corresponds to a VL II TOOL ratng. Fgure 5 also provdes a CROSSTALK The Journal of Defense Software Engneerng 13

4 Software Engneerng Technology Drvers RELY DATA CPLX TIME STOR VIRT II Drvers Converson Factors R ELY D ATA C PLX T IME S TOR P VOL T URN Use values n Table 5. ACAP PCAP VEXP AEXP LEXP TOOL MODP SCED A CAP P CAP P EXP A EXP L TEX T OOL Use values n Table 5. Adjust PMAT settngs. Table 4. Cost drvers conversons. If MODP s rated VL or L, set PMAT to VL. N, set PMAT to L. H or VH, set PMAT to N. S CED R USE Set to N or actual, f avalable. D OCU If Mode: = Organc, set to L. = Semdetached, set to N. = Embedded, set to H. P CON Set to N or actual, f avalable. S ITE Set to H or actual, f avalable. set of II multplers correspondng to project ratngs. Some mplementatons of II, such as the USC II package, provde slots for extra user-defned cost drvers. The values n Fgure 5 can be put nto those slots (f you do ths, use an N ratng n the normal II TOOL slot). To learn more about the cost drvers and ther ratngs, refer to the USC Web ste ( II) or several of the Center for Software Engneerng s other publcatons [7, 8]. Because the goal of ths artcle s to present the Rosetta Stone, we dd not thnk t was necessary to go nto the detals of the model and an explanaton of ts many parameters. Expermental Accuracy To assess the accuracy of the translatons, the team used the Rosetta Stone to convert 89 projects. These projects were clustered subsets of the databases we used for model calbraton. Clusters were doman-specfc. We updated our estmates usng actuals whenever we could. We then used the autocalbraton feature of the USC II package to develop a constant for the effort equaton, e.g., the A n the equaton Effort = A(sze) P. Fnally, we compared our estmates to actuals and computed the relatve error as a functon of the followng cases. Usng the Rosetta Stone wth no adjustments. Usng the Rosetta Stone wth knowledge-base adjustments, e.g., updatng the estmate fles wth actuals when avalable. Usng the Rosetta Stone wth knowledge-base adjustments and doman clusterng, e.g., segmentng the data based on organzaton or applcaton area. The results of these analyses, whch are summarzed n Table 6, were extremely postve. They show that we can acheve an acceptable degree of estmatng accuracy when usng the Rosetta Stone to convert fles to run wth the II software cost model. Summary and Conclusons The Rosetta Stone was developed to provde ts users wth a process and a tool to convert ther orgnal fles so that they can be used wth the new II estmatng model. The Stone represents a startng pont for such efforts. It does not replace the need to understand ether the scope of the estmate or the changes that occurred as the Table 5. TURN and TOOL adjustments. Table 6. Estmate accuracy analyss results. C ases Accuracy (Relatve Error) Usng Ratng the II model as calbrated. Usng the II model as calbrated usng developer or doman clusterng. Usng the Rosetta Stone wth no adjustments. Usng the Rosetta Stone wth knowledge- base adjustments. Usng the Rosetta Stone wth knowledge- base adjustments and doman clusterng. VL L N H VH II Multpler: TURN II Multpler: TOOL Estmates are wthn percent of the tme. percent of actuals 76 percent of the tme. 60 percent of the tme. 68 percent of the tme. 74 percent of the tme. 14 CROSSTALK The Journal of Defense Software Engneerng

5 The Rosetta Stone: Makng Estmates Work wth II project unfolded. Rather, the Stone takes these consderatons nto account as you update ts knowledge base wth actuals. The value of the Rosetta Stone was demonstrated convncngly based on an accuracy analyss of an 89-project database. As expected, the accuracy ncreased as we adjusted the estmates usng actuals and looked at results based on doman segmentatons. We are encouraged by the results. We plan to contnue our efforts to provde structure and support for such converson efforts. References 1. Boehm, B., Software Engneerng Economcs, Prentce-Hall, Englewood Clffs, N.J., Park, R., Software Sze Measurement: A Framework for Countng Source Statements, CMU/SEI-92-TR-20, Software Engneerng Insttute, Pttsburgh, Pa., Refer, D., personal correspondence, Jones, C., Appled Software Measurement: Assurng Productvty and Qualty, McGraw-Hll, New York, Paulk, M., C. Weber, B. Curts, and M. Chrsss, The Capablty Maturty Model: Gudelnes for Improvng the Software Process, Addson-Wesley, Readng, Mass., Boehm, B., et al., II Model Defnton Manual, Verson 1.4, Unversty of Southern Calforna, Los Angeles, Calf., Boehm, B., et al., The 2.0 Software Cost Estmaton Model, Amercan Programmer, July 1996, pp Clark, B. and D. Refer, The Rosetta Stone: Makng Your Estmates Work wth II, Software Technology Conference, Salt Lake Cty, Utah, About the Authors Donald J. Refer s a leadng fgure n software engneerng and management, wth over 30 years progressve experence n government and ndustry. He has been chef of the Ada Jont Program Offce, techncal advser to the Center for Software, and drector of the Department of Defense (DoD) Software Reuse Intatve under an Intergovernmental Personnel Act assgnment wth the Defense Informaton Systems Agency. He s currently presdent of RCI, a small consultng frm servcng Fortune 500 companes, and he s a vstng assocate at USC, where he serves on the team. He has a bachelor s degree n electrcal engneerng, a master s degree n operatons research, and a certfcate n busness management (master s equvalent). Hs many honors nclude the Secretary of Defense s medal for Outstandng Publc Servce, the NASA Dstngushed Servce Medal, the Freman Award, and the Hughes Arcraft Fellowshp. He has over 100 publcatons, ncludng hs popular IEEE Software Management Tutoral and a new Wley book enttled Practcal Software Reuse. Refer Consultants, Inc. P.O. Box 4046 Torrance, CA Voce: E-mal: d.refer@eee.org Barry W. Boehm s consdered one of the fathers of the feld of software engneerng. He s currently drector of the Center for Software Engneerng at USC, and for many years drected key technology offces n the DoD, TRW, and Rand Corporaton. Hs contrbutons to software engneerng nclude, the spral model of the software process, the Theory W approach to software management and requrements determnaton, and the TRW Software Productvty System and Quantum Leap advanced software engneerng envronments. Hs current software research nterests nclude process modelng, requrements engneerng, archtectures, metrcs and cost models, engneerng envronments, and knowledge-based engneerng. Boehm has a bachelor s degree from Harvard Unversty and a master s degree and a doctorate from the Unversty of Calforna at Los Angeles, all n mathematcs. He has served on the board of several scentfc journals and has served as charman of numerous promnent engneerng socety commttees. He s the recpent of many of software engneerng s hghest awards. He s an Amercan Insttute of Aeronautcs and Astronautcs Fellow, an Assocaton for Computng Machnery Fellow, an Insttute of Electrcal and Electroncs Engneers Fellow, and a member of the Natonal Academy of Engneerng. Center for Software Engneerng Unversty of Southern Calforna 941 West 37th Place Los Angeles, CA Voce: E-mal: boehm@sunset.usc.edu Sunta Chulan s a research assstant at the Center for Software Engneerng at the Unversty of Southern Calforna. She s an actve partcpant on the II research team and s workng on a Bayesan approach to data analyss and model calbraton. She s also workng on a cost and qualty model that wll be an extenson to the exstng II model. Her man nterests nclude software process mprovement wth statstcal process control, software relablty modelng, rsk assessment, software cost estmaton, and software metrcs. She has a bachelor s degree n computer engneerng from Bombay Unversty and a master s degree n computer scence from USC. She s currently a doctoral canddate at the Center for Software Engneerng at USC. Center for Software Engneerng Unversty of Southern Calforna 941 West 37th Place Los Angeles, CA Voce: E-mal: sdevnan@sunset.usc.edu CROSSTALK The Journal of Defense Software Engneerng 15