CSCI 510 Final Exam, Fall 2017 v10 of solution & rubric Monday, December 11, questions, 300 points

Transcription

1 CSCI 510 Final Exam, Fall 2017 v10 of solution & rubric Monday, December 11, questions, 300 points If registered DEN student, please circle: Yes Last Name: First Name: USC ID: Question 1 (48) Question 2 (64) Question 3 (68) Question 4 (120) Total (300) When working exam questions, please assume that non-simple estimating situations do not occur in the problem, unless they are mentioned explicitly. Specifically, unless mentioned otherwise in the problem statement: The Post-Architecture model is used (not the Early Design model); There is no Adapted (reused) software (i.e., all software is new); Requirements Volatility (REVL) is zero; There is no automatic translation; The Inception and Transition phases are not part of the estimate (i.e., the model equations give the effort and schedule directly); No risk reserves are needed; The nominal schedule is to be used (i.e., SCED% = 100%); and Only one module is to be produced. As a related shortcut, a problem statement will frequently say something like "all other Effort Multipliers should be taken to be Nominal". As you have learned, the numeric value for any Effort Multiplier at the Nominal level is 1.0. As another shortcut, there is a standard approximation for Scale Factors: When all Scale Factors are Nominal, you should use 1.1 as the value for the exponent E. Use at least 3 significant digits for your final answer (and your intermediate results). More significant digits is acceptable. (To avoid some rounding difficulties, 4+ significant digits is recommended.)

2 1. ICSM Principles Violation, 48 points, based on the Unaffordable Requirement case study in Chapter 4 of the ICSM book, reproduced with some modifications below. (Usually graded by Anandi) 4.1 Failure Story: The Unaffordable Requirement In the early 1980s, a large government organization contracted to scale up a useful 50-person department level information query and analysis system. The scaled-up system would provide more than 1,000 users, spread across a large building complex, with similar query and analysis capabilities. In an attempt to simplify the project, the customer prepared a contract to develop a system that would provide the existing department-level capabilities to the overall organization, which would then be extended to add or modify capabilities needed by the other users. A particularly valuable feature of the department-level system was its ability to respond to almost all queries in less than one second. To preserve this, the Customer fixed into the contract a requirement for a system response time of less than one second. It satisfied the System Requirements Review criteria at the beginning of the contract of being unambiguous, testable, and free of design commitments. The winning bidder for the job of Developer assumed that the job could be done by using the COTS product supporting the department system, enabling its access from the full set of Customer facilities, scaling up the workload, and tuning the resulting Initial Operational Capability (IOC) system to meet the one-second response time requirement. Subsequently, though, the Developer architects found that the COTS product could only be tuned to reach a 2.5 second response time for the full workload, and that sub-second performance could only be provided via a highly customized design that attempted to anticipate query patterns and cache copies of data so that each user s likely data would be within one second s reach (a 1980 s precursor of Google). The resulting hardware architecture had more than 25 super-midicomputers (an earlier term for a computer with performance and capacity between a minicomputer and a mainframe) busy caching data according to algorithms whose actual performance defied easy analysis. Creating a hardwaresoftware architecture with this scope and complexity took 15 months of effort, and brought the estimated cost of the system to nearly $100 million, driven primarily by the requirement for a one-second response time, and required the project s Preliminary Design Review to be delayed by 15 months. 2

3 Faced with this unattractive prospect (far more than the Customer s budget and schedule for the system), the Customer and Developer decided to develop a prototype of the system s user interface and representative capabilities to test the need for a 1-second response time for all queries. The results showed that a foursecond response time would satisfy users 90 percent of the time. A four-second response time, with special handling for high-priority transactions, enabled the existing COTS product to be used, and dropped development costs to considerably less than Customer s budget of $30 million. Fortunately, this showed that an affordable Initial Operational Capability (IOC) was achievable, but only after 15 months of wasted effort on the expensive-system architecture and a 15-month delay in delivery. However, it did not show that the IOC would satisfy the needs of the other user organizations, or of its Maintainer organization. Cost 3

4 Exam Question Since the Unaffordable Requirement project was the failure example for ICSM Principle 4, Evidence and Risk-Based Decisions (violations of this principle were well covered in the ICSM book), this question asks you to identify two Unaffordable Requirement violations of each of the other three Principles 1, 2, and 3, by the system s Customer and Developer organizations. 8 points each will be awarded for up to 2 well-defined violations of each of Principles 1, 2, and 3. Number your violations. [Example violations are given below for Principle 4. Other violations are determined by the grader s judgment, based on how well the proposed violation agrees with the definition of the Principle. However, a violation that essentially duplicates another student violation on the same Principle gets no credit.] As examples of the type of violations of the principles to be provided, below are some for Principle 4. (Showing how any of these violates one (but no more) of the other Principles is OK.) The Customer did not attempt to provide evidence at the System Requirements Review that the system could scale to a 1-second response. The Customer did not attempt to provide evidence at the System Requirements Review that the IOC would satisfy the needs of the other user organizations. The Customer did not attempt to provide evidence at the System Requirements Review that the IOC would satisfy the needs of its Maintainer organization. The Developer did not attempt to provide evidence at the System Requirements Review that the developed IOC could scale to a 1-second response. The Developer did not attempt to provide evidence at the System Requirements Review that the developed IOC could be easily modified to satisfy the needs of the Customer s other user organizations. 4

5 Principle 1. Provide three violations of Principle 1, Stakeholder Value-Based Guidance Examples: The Customer did not attempt to involve other users in determining whether the IOC would be easily modifiable to meet their needs. The Customer did not attempt to involve its Maintainer organization in determining whether the IOC would be easily maintainable. The Developer did not attempt to involve the Customer in determining whether the IOC would satisfy the 1-second response time requirement, or whether a custom solution would be affordable. The Developer did not attempt to involve the Customer s Maintainer organization in determining whether the IOC would be easily maintainable. 5

6 Principle 2. Provide three violations of Principle 2, Incremental Commitment and Accountability Examples: The Customer did not attempt to provide plans at the System Requirements Review for the other user organizations to determine how the IOC would be modified to satisfy their needs. The Customer did not attempt to provide plans at the System Requirements Review for its Maintainer organization to work with the Developer to ensure that the IOC would be easily maintainable. The Developer did not attempt to provide plans at the System Requirements Review for working with the Customer on an initial increment to determine its users response time needs. The Developer did not attempt to provide plans at the System Requirements Review for working with the Customer on a followon increment to develop the needed infrastructure support for its users response time needs. 6

7 Principle 3. Provide three violations of Principle 3, Concurrent Multi- Discipline Engineering The Customer did not attempt to provide plans at the System Requirements Review for the other user organizations to concurrently determine with the Developer how the IOC would be modified to satisfy their needs. The Customer did not attempt to provide plans at the System Requirements Review for its Maintainer organization to concurrently work with the Developer to ensure that the IOC would be easily maintainable. The Developer did not attempt to provide plans at the System Requirements Review for concurrently working with the Customer on an initial increment to determine its users response time needs. The Developer did not attempt to provide plans at the System Requirements Review for concurrently working with the Customer on a followon increment to develop the needed infrastructure support for its users response time needs. 7

8 2. COCOMO II Estimate, 64 points, based on the case study in Question 1. (Usually graded by Elaine) Given that the COTS-based system could easily provide a 4-second response time to queries, and should be able with some modifications to provide 2- second responses to about 10% of the queries, the Developer proposed a 2- phase approach to reach a full-service solution. Phase 1 would involve about 25 KSLOC of access extensions to enable the Customer s other users to access the COTS capabilities provided to the current user-department. Phase 2 would identify and develop the highest-priority 100-KSLOC of additional capabilities to enable the COTS product to address the other users highpriority needs. The Developer agreed to provide some of its best performers to ensure timely and high-quality results, and to involve the maintainers in the development to ensure a highly maintainable system. Perform COCOMO II estimates for the effort, cost, and schedule of the two phases (30 points each). Show your work in estimating the effort, cost, and schedule for each option. It is best to show your work in case earlier calculations are wrong. Calculations are shown with three significant digits; many-digit calculations are also acceptable. 2.a Phase 1 estimates (32 points). Given that the Developer is providing High-capability performers for the project, its COCOMO II ACAP and PCAP ratings will be High, and the Developer costs will be $10K per person-month. The complexity of the COTS product and its interfaces, and the size of the database lead to High ratings for the COCOMO II CPLX and DATA cost drivers, and a Low rating for the Platform Experience cost driver. The other cost drivers and the scale factor ratings will be Nominal for Phase 1; use 1.1 for the scale factor exponent and for the TDEV exponent. As stated above, the size will be 25 KSLOC. 8

9 Effort [17 points total.] High DATA multiplier = High CPLX multiplier = High ACAP multiplier = High PCAP multiplier = Low PLEX multiplier = EAF = 1.14 * 1.17 * 0.85 * 0.88 * 1.09 = [EAF: 7 points total: 1 points for each correct multiplier. 2 points if EAF is numerically correct.] PM = 2.94 * (Size ^ E) * EAF = 2.94 * (25 ^ 1.1) * = 2.94 * 34.5 * = person-months [Rest of PM calculation: 10 points total: 7 points if form of equation is correct (partial credit allowed on incorrect forms). 3 points for numerically correct answer; 2 points if correct with student EAF; 1 point if a computational error at this point.] Cost [5 points total] = PM * $10K/PM = PM * $10K/PM = $1,103K = $1.103M. [3 points if form of equation is correct (partial credit allowed on incorrect form). 2 points if numerically correct answer; 1 point if numerically correct based on student PM.] Schedule (months) with Nominal SCED [10 points total] = 3.67 * (PM ^ F) = 3.67 * (110.3 ^ 0.318) = 3.67 * 4.46 = 16.4 months. [7 points if form of equation is correct (partial credit allowed on incorrect form). 3 points if numerically correct answer; 2 points if numerically correct based on student PM.] 9

10 2.b Phase 2 estimates (32 points). Phase 1 would create an Initial Operational Capability (IOC) for the entire Customer organization to use the Department-level functionality. The Developer s proposed Phase 2 would begin with the Customer working with the Developer having already identified, sized, and prioritized these, and selecting the highest-priority capabilities that would fit within a 100-KSLOC Full Operational Capability (FOC) system. Lower-priority capabilities would then become a prioritized backlog for the maintenance phase. Perform effort, cost, and schedule estimates for an approach that adds further high-capability Developer personnel to develop the 100-KSLOC Full Operational Capability (FOC) system. The Phase 1 Developers experience in developing the IOC enables the Developer to increase their Phase 2 Platform Experience (PLEX) rating from Low to Nominal. The Phase 1 High ratings for DATA, CPLX, ACAP, and PCAP remain the same for Phase 2, as do the Nominal ratings for the remaining cost drivers and scale factors. Effort [17 points total.] High DATA multiplier = High CPLX multiplier = High ACAP multiplier = High PCAP multiplier = EAF = 1.14 * 1.17 * 0.85 * 0.88 = [EAF: 7 points total: 5 points for correct multipliers. 2 points if EAF is numerically correct.] OK to get the result by dividing by the 1.10 multiplier for Low PLEX in Phase 1, although this rounds to vs

11 PM = 2.94 * (Size ^ E) * EAF = 2.94 * (100 ^ 1.1) * = 2.94 * * = (or ) person-months [Rest of PM calculation: 10 points total: 7 points if form of equation is correct (partial credit allowed on incorrect forms). 3 points for numerically correct answer; 2 points if correct with student EAF; 1 point if a computational error at this point.] Cost [5 points total] = PM * $10K/PM = PM * $10K/PM = $4,651K = $4.651M (or $ M). [3 points if form of equation is correct (partial credit allowed on incorrect form). 2 points if numerically correct answer; 1 point if numerically correct based on student PM.] Schedule (months) with Nominal SCED [10 points total] = 3.67 * (PM ^ F) = 3.67 * (465.1 ^ 0.318) = 3.67 * = 25.8 months (or months). [7 points if form of equation is correct (partial credit allowed on incorrect form). 3 points if numerically correct answer; 2 points if numerically correct based on student PM.] 11

12 3. Business Case Analysis, 68 points, based on the case study from Question 1. (Usually graded by Jim) The Customer would like to get an earlier understanding of which additional capabilities will be most needed by the other users. Rather than wait until the end of Phase 1 to start doing this, while the Developer is creating in Phase 1 an Initial Operational Capability (IOC) for the entire Customer organization to use the Department-level functionality, the Customer proposes that the Developer provide additional high-capability personnel to work with the Customer s other end users to identify, prototype, size, and select the highestpriority capabilities that will fit within the 100-KSLOC Full Operational Capability (FOC) system. As before, lower-priority capabilities would then become a prioritized backlog for the maintenance phase. The Customer indicates that this would not only provide earlier and better selection of the Phase 2 capabilities, it would provide the Developer with the prototypers experience with the nature of these capabilities. Given that this would increase the Developer s cost-driver ratings for Applications, Platform, and Language and Tool Experience, it could be that the reduced cost of Phase 2 would more than pay for the costs of engaging the prototyping team. 3.a Phase 1 estimates (19 points). Perform a COCOMO II estimate of the effort, cost, and schedule for a Developer prototyping team creating 20-KSLOC worth of prototypes of the higher-priority other users needed capabilities. Relevant cost driver ratings are High ratings for CPLX, DATA, ACAP, and PCAP, and a Low rating for PLEX (the same as for the Phase 1 developers; OK to take EAF calculation from question 2.a) Effort [12 points total.] High DATA multiplier = High CPLX multiplier = High ACAP multiplier =

13 High PCAP multiplier = Low PLEX multiplier = EAF = 1.14 * 1.17 * 0.85 * 0.88 * 1.09 = [EAF: OK to take from question 2.a] PM = 2.94 * (Size ^ E) * EAF = 2.94 * (20 ^ 1.1) * = 2.94 * 27.0 * = 86.3 person-months [PM calculation: 12 points total: 2 points for correct EAF; 7 points if form of equation is correct (partial credit allowed on incorrect forms). 3 points for numerically correct answer; 2 points if correct with student EAF; 1 point if a computational error at this point.] Cost [2 points total] = PM * $10K/PM = 86.3 PM * $10K/PM = $863K = $0. 863M. [1 points if form of equation is correct (partial credit allowed on incorrect form). 1 points if numerically correct answer; 0.5 point if numerically correct based on student PM.] Schedule (months) with Nominal SCED [5 points total] = 3.67 * (PM ^ F) = 3.67 * (86.3 ^ 0.318) = 3.67 * 4.13 = 15.1 months. [3 points if form of equation is correct (partial credit allowed on incorrect form). 2 points if numerically correct answer; 1 point if numerically correct based on student PM.] 13

14 3.b Phase 2 estimates (23 points). Again, Phase 1 would create an Initial Operational Capability (IOC) for the entire Customer organization to use the Department-level functionality. The Customer s proposed Phase 2 would begin with the Customer working with the unified Developer team, including the developers of the Phase 1 IOC, and the parallel Phase 1 prototyping team working with the Customer s additional end users in identifying, sizing, prototyping and prioritizing their desired capabilities, and selecting the highest-priority capabilities that will fit within a 100-KSLOC Full Operational Capability (FOC) system. Lower-priority capabilities would then become a prioritized backlog for the maintenance phase. Perform effort, cost, and schedule estimates for an approach that adds the high-capability Developer prototyping personnel to the Phase 1 IOC team, to develop the 100-KSLOC Full Operational Capability (FOC) system. The combination of the Phase 1 Developers experience in developing the IOC combines with the prototypers experience on the added Phase 2 capabilities to increase their Phase 2 Platform Experience (PLEX) and their Applications Experience (APEX) from Low to High. The Phase 1 High ratings for DATA, CPLX, ACAP, and PCAP remain the same for Phase 2, as do the Nominal ratings for the remaining cost drivers and scale factors. Effort [13 points total.] High DATA multiplier = High CPLX multiplier = High ACAP multiplier = High PCAP multiplier = High PLEX multiplier = High APEX multiplier = EAF = 1.14 * 1.17 * 0.85 * 0.88 * 0.91 * 0.88 = Alternatively, from Phase 1 EAF: 14

15 EAF = / 1.09 * 0.91 * 0.88 = [EAF: 3 points total: 2 points for correct multipliers. 1 point if EAF is numerically correct.] PM = 2.94 * (Size ^ E) * EAF = 2.94 * (100 ^ 1.1) * = 2.94 * * = person-months [Rest of PM calculation: 10 points total: 7 points if form of equation is correct (partial credit allowed on incorrect forms). 3 points for numerically correct answer; 2 points if correct with student EAF; 1 point if a computational error at this point.] Cost [5 points total] = PM * $10K/PM = PM * $10K/PM = $3,723K = $3.723M. [3 points if form of equation is correct (partial credit allowed on incorrect form). 2 points if numerically correct answer; 1 point if numerically correct based on student PM.] Schedule (months) with Nominal SCED [5 points total] = 3.67 * (PM ^ F) = 3.67 * (372.3 ^ 0.318) = 3.67 * 6.57 = 24.1 months. [3 points if form of equation is correct (partial credit allowed on incorrect form). 2 points if numerically correct answer; 1 point if numerically correct based on student PM.] 15

16 3.c Perform a Return on Investment (ROI) analysis for investing in the prototyping team (18 points). Note: Items in parentheses are numbers of previous questions. Recall that the cost of the Developer Strategy was the cost of creating an IOC (2.a) plus the cost of simply building a 100-KSLOC FOC (2.b). Also recall that the cost of the Customer Strategy was the cost of creating an IOC (2.a) plus the cost of developing a 20-KSLOC prototype (3.a) plus the cost of the 100-KSLOC FOC built after prototyping (3.b); the added cost of the Customer Strategy is the cost of developing the prototype. The Net Benefit then is the cost of the Developer Strategy minus the cost of the Customer Strategy. Compute the cost of the Developer Strategy, the cost of the Customer Strategy, and the Net Benefit. Using the Net Benefit and the added cost of the Customer Strategy, compute the ROI of using the Customer Strategy. [Student answers from Questions 2.a, 2.b, 3.a, and 3.b are taken to be the correct starting values.] Total cost of the Developer Strategy is the cost of the IOC (2.a) ($1,103K) plus the cost of the FOC (2.b) ($4,651K) = $5,754K. [3 points total: 2 for correct formula, 1 for correct values. Partial credit allowed.] Total cost of the Customer Strategy is the cost of the IOC (2.a) ($1,103K), plus the cost of the prototyping team (3.a) ($863K), plus the cost of the full-team Phase B effort (3.b) ($3,723K) = $5,689K. [4 points total: 3 for correct formula, 1 for correct values. Partial credit allowed.] Net Benefit = $5,754K - $5,689K = $65K. [3 points total: 2 for correct formula, 1 for correct values. Partial credit allowed.] Added cost of the Customer Strategy (3.a) = $863K. [2 points total, for explicitly naming and/or using the 3.a value, in the ROI calculation.] Return on Investment (ROI) = (Net Benefit)/Cost 16

17 = $65K/$863K = (7.53%) [6 points total: 4 for correct formula, 2 for correct values. Partial credit allowed.] 17

18 3.d Perform a schedule comparison (8 points). Compute the total schedule for the Developer Strategy and the total schedule for the Customer Strategy. How much time does the Customer Strategy save? [Student answers from Questions 2.a, 2.b, 3.a, and 3.b are taken to be the correct starting values.] Developer Strategy schedule: Phase1 + Phase2 = = 42.2 months. [2 points total. Partial credit allowed.] Customer Strategy: Phase1 + Phase2 = = 40.5 months (prototyping is 15.1 months that runs in parallel with Phase1). [3 points total: 2 for correct formula. 1 for correct values. Partial credit allowed.] The gain in schedule is: = 1.7 months. [3 points total: 2 for correct formula; 0 for subtracting the Developer Strategy from the Customer Strategy. 1 for correct values. Partial credit allowed.] 18

19 4. True-False Questions; 120 points, 6 points each. (Usually graded by Elaine) Put either a T or F on the line preceding the statement. Grading will be based on this alone, not any additional statements or explanations provided. 1. T A system development will succeed if and only if it makes winners of its successcritical stakeholders. (ICSM pg 10) 2. OK COCOMO II's TEAM scale factor accounts for the sources of project turbulence and entropy because of difficulties within the development team. (EP-2 pg 34) [Question is badly worded (should be only within the development team. Full credit for either T or F.] 3. T Len Cayetano analyzes projects he's worked on using COCOMO II scale factors as a framework. (EC-21 #48, 52, 56, 60, 64, 65, 66) 4. T By using a Figure of Merit we can produce a single decision criterion from a combination of several decision criteria. (EP-6 pg 9) 5. F A conclusion from the lecture on information buying is that one should always build a prototype or simulation. (EC-12 #19) 6. T If R is a risk with loss L, where the probability of L = 1.0, then R is actually not a risk. (EC-13 #4, 5) 7. F If scalability risks are high and personnel turnover is frequent, it is recommended to tailor the life-cycle process around plan-driven risks. (EC-17 #29, 30) 8. F The elements of WinWin negotiation are win conditions, issues, risks, and agreements. (EC-23 #8) 9. T Under the ICSM, the only situation in which the Operations and Support phase ends is some kind of end of life situation. (EC-14 #17) 19

20 10. T In general, concurrent engineering will have to bring its threads together and perhaps suspend development for evidence-based reviews. (EC-19 #2) 11. T One part of the ACM/IEEE Software Engineering Code of Ethics says that a software engineer should always put the public's interest first. (EC-25 #12) 12. T COCOMO II includes a non-linear reuse model. (EC-3 #13, 14) 13. F COCOMO II estimating is not of use to an agile project, especially one that's underway. (EC-7 #28) 14. F The ICSM can't be committed to incrementally. (ICSM section 0.5) 15. F Managing Operations & Maintenance costs is like herding cattle. (EC-22 #2, 3) 16. T Planning to implement only sunny-day requirements can cause a project to take the lower branch of the Cone of Uncertainty. (EC-22 #6, 7) 17. F Warren Reid recommends that one not make an actual commitment on a project until the cone of uncertainty suggests that the cost estimate is within plus or minus 5% of actual cost. (EC-24 #11) 18. T COCOMO II can be regarded as a production function which relates development man-months, as inputs, to delivered SLOCs, as outputs. (EP-5 #21) 19. T A good practice is to concurrently engineer a project's requirements and design. (ICSM pg 94) 20. F A good practice is to concurrently engineer a project's design and development. (ICSM pg 85) 20