CA Clarity PPM Release 14.2

Transcription

1 CA Clarity PPM Release 14.2 Oracle Performance Whitepaper CA Clarity Performance Team 3/19/2015 Version 1.0 Copyright 2015 CA. This document contains data and conclusions from the execution of performance tests against CA Clarity PPM

2 Contents Disclaimer... 4 Executive Summary... 5 Methodology... 5 Results and Conclusion... 5 Comparison by Functional Area (13.3 and 14.2)... 6 Test Results... 7 Overview of Test Methodology... 7 Test Result Summary User Load Test... 7 Test Environment... 8 Application & Background server software... 8 Application server JVM memory parameters... 8 Oracle database server software... 8 MS SQL Server software... 8 Application & Background server hardware... 8 Database server hardware... 8 Data Warehouse... 9 Expected Growth Rates... 9 Expected Load Times... 9 Functional Areas That Influence Growth User Load Test Performance Graphs App Server CPU Utilization Heap usage Appendix A: Use Cases Use case 1 - Create Projects Use case 2 - PM Projects Use case 3 - PM Financial Planning view and edit Cost Plan Use case 3a - PM Financial Planning - Cost Plan Use case 3b - PM Financial Planning - Cost Plan Use case 3c - PM Financial Planning - Cost Plan Use case 3d - PM Financial Planning - Cost Plan Use case 4 - Create Project from Template Use case 5 - Gantt Toolbar Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 2

3 Use case 6 - Gantt WBS List Use case 7 - Interactive Gantt Chart Use case 8 - Auto-Schedule Use case 10 - General UI Behavior Use case 11 - PM Resource Management: Working with resources Use case 12 - Timesheets Use case 13 - PM Financial Transactions Use case 14 - Departments Use case 15 - Navigator All Page Script Use case 16 - PM Rate Matrix Use case 18 - Administration All Page Script Use case 19 - Approve Timesheet Use case 21a New Portfolio Use case 21b New Portfolio Appendix B: Load Test Details User Load Test Details (all response times in seconds) Appendix C: Oracle Parameters Oracle INIT.ORA parameters Appendix D: Database Size Details Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 3

4 Disclaimer This white paper is not meant as a sizing guide or as a recommendation guide, but instead it portrays possible user loads given a particular CA Clarity PPM configuration. We understand that customer requirements and environments differ. The CA Clarity PPM functionality exercised during these performance load tests may or may not match the functionality that individual companies deploy. CA does not recommend that customers acquire hardware based on this white paper, but instead seek the sizing expertise of our technical pre sales and professional services personnel. The test environments outlined in this document do not consider reporting, process engine, and background service or failover requirements. MS SQL is a trademark or registered trademark of Microsoft Corporation in the United States and/or other countries. This document is for your informational purposes only. CA assumes no responsibility for the accuracy or completeness of the information. To the extent permitted by applicable law, CA provides this document as is without warranty of any kind, including, without limitation, any implied warranties of merchantability, fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document, including, without limitation, lost profits, business interruption, goodwill or lost data, even if CA is expressly advised in advance of the possibility of such damages. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 4

5 Executive Summary To establish and publish performance benchmarks, CA conducted a series of tests for CA Clarity PPM Release 14.2 on a LINUX clustered environment. Below are the high level enhancements made to the test infrastructure: In order to more closely simulate real world activity, the user concurrency has been increased from 200 users earlier to 500 users in Release Existing customer application logs were analyzed to ensure test throughput (number of actions performed) is on par with peak load for 500 concurrent users. Two new use cases related to Reports & Cost Plan enhancements were incorporated. Reports were executed in volume to simulate well above peak usage of ~150 reports in an hour with an average report run time of 10 seconds. In addition to the UI actions performed, a background process was run in volume during the test. A project process ran ~600 times in an hour to export an updated project via XOG. The test objectives were to: Demonstrate the ability of the product to serve a large user base (5,000 users) on Intel-based hardware using an Oracle 11gR2 ( ) database and Apache Tomcat Version application servers on a LINUX clustered environment. Establish performance benchmarks under load using a core set of application functionality on Intel-based hardware and a Linux environment. Methodology The load tests use a mixture of inquiry and update actions to simulate end-user interaction with the product. The tests used 500 virtual users to simulate a peak period of usage for a total user population of 5,000 registered users 1. Wait times ranged between 3 and 45 seconds, depending on the type of action. The performance test is measured for 150 minutes, including a ramp up time of 15 minutes. The tests used a single database server with two application servers and one BG server, all running on individual physical servers. The actual experience of customers will depend on how hardware is set up, how many modules are deployed, how the user actions are distributed among the population and the rights granted to the users. Results and Conclusion When comparing the average response times of the same functionality, Clarity Release 14.2 ( ) performs 28% better than Clarity Release 13.3 ( ) on similar hardware comparing the same counters 2. The 28% increase also reflects a 250% increase in user concurrency with added background reports and jobs running under the same hardware configurations from past Clarity performance tests. See the Test Environment section of this document for details on the deployment architecture. See Appendix B: Load Test Details for full test results. 1 Testing at a 10% concurrency rate, 500 VUs represent 5,000 registered users. 500/.10=5000, therefore 500 VUs represent 5000 registered users. 10% concurrency is typical for CA Clarity PPM customers on peak usage times. 2 ((v13.3 avgresp V14.2 avgresp)/ V13.3 avgresp)) = % improve or (( )/0.94) = 0.28 or 28% improvement Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 5

6 Comparison by Functional Area (13.3 and 14.2) Average Response Time For Functional Area Name Project Management Resource Management Financial Management General Navigation Timesheets Portfolios All timers Clarity functional area wise comparison V Vs V Avg Response Time (sec) Project Resource Financial Management Management Management General Navigation Functional Area Timesheets Portfolios Reports All timers Each area consists of the following use cases: Project Management (Use-Case1, 2, 4, 5, 6, 7, 8) Resource Management (Use-Cases 11, 13, 14) Financial Management (Use-Cases 3, 3a, 3b, 3c, 3d, 16) General Navigation (Use-Cases 10, 15, 18) Timesheets (Use-Cases 12, 19) Portfolios (Use-Cases 21a & 21b) 3 These figures are the response time average of all timers and not the average of the individual functional area averages. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 6

7 Test Results Overview of Test Methodology The following types of test were run: Load (500 concurrent users) All tests were written and executed using Load Runner Each virtual user simulates one real user actively and repeatedly clicking through a given scenario. Wait times reflect only interaction delays (for example, completing a form before clicking Submit), not idleness and task interruption. Appendix A: Use Cases contains the detailed test scenario descriptions. The background server ran all jobs, reports, and processes to completion prior to the start of the performance test. Test Result Summary Two Quad core X GHz Xeon servers, each running one v14.2 application instance, can support a simulated load for 5,000 registered users (500 simultaneous virtual users) with Oracle. 500 User Load Test Results are drawn from the 500 virtual user 2.0 hour load tests. Appendix B: Load Test Details shows the detailed results. Average Transaction Response Time 0.67 seconds Average Database Server CPU Utilization: 6.7 % Average Application Server 1 CPU Utilization: 38.4 % Average Application Server 2 CPU Utilization: 31.2 % Average Background Server CPU Utilization: 10.2 % Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 7

8 Test Environment For the Load tests, we used two application servers and one background server, each on their own hardware server. Application & Background server software: OS RHEL 5.5 Java 64 Bit jdk1.7.0_67 App Tomcat Application server JVM memory parameters: -Xms1024m -Xmx4096m -XX:MaxPermSize=196m Oracle database server software: OS RHEL 6.4 DB Oracle MS SQL Server software: OS Windows Server 2012 Standard DB SQL Server 2012 EE 64 Bit Application & Background server hardware: CPU Quad core X GHz Xeon RAM 8 GB Database server hardware: CPU 32 core E v2@2.00ghz Xeons RAM 126 GB /16G for Oracle Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 8

9 Data Warehouse The new Data Warehouse (DWH) feature is optimized specifically for reporting and analytics. The Data Warehouse is installed on a separate schema from the CA Clarity PPM database schema. The Data Warehouse is populated at scheduled intervals by a CA Clarity PPM job. For customers upgrading to CA Clarity PPM 14.2, the following sections provide examples of expected growth rates and loading times based on existing database sizes. Expected Growth Rates We sampled 10 customer data sets from various industries to get an idea of expected growth rates. Industry OLTP Size DWH Size % of OLTP IT_1 56 GB 13 GB 23 % IT_2 65 GB 7 GB 11 % Financial 223 GB 33 GB 15 % Services 168 GB 37 GB 22 % Retail_1 300 GB 10 GB 4 % Retail_2 15 GB 2 GB 13 % Insurance 37 GB 2 GB 5 % Financial 5 GB 1 GB 20 % Transportation 2 GB 500 MB 25 % Medical 14 GB 300 MB 2 % Expected Load Times We sampled 10 customer data sets from various industries to get an idea of expected initial load times. Industry OLTP Size DWH Size DWH Full Load IT_1 56 GB 13 GB 41 m IT_2 65 GB 7 GB 35 m Financial 223 GB 33 GB 5 h Services 168 GB 37 GB 7 h Retail_1 300 GB 10 GB 34 m Retail_2 15 GB 2 GB 21 m Insurance 37 GB 2 GB 25 m Financial 5 GB 1 GB 10 m Transportation 2 GB 500 MB 11 m Medical 14 GB 300 MB 10 m Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 9

10 Functional Areas That Influence Growth The following data points outline areas that influenced the size/load of the DWH in the above tables. Industry Resources Teams Assignments Tasks Projects Timesheets Time Entry WIP Financial Plans Retail_1 50 k 175k 650 k 500 k 12 k 600 k 1.5 m 0 12 k Financial 364 k 1 m 2 m 8 m 180 k 1.6 m 4.1 m 21 m 560 k Services 146 k 1.6 m 7.2 m 6.6 m 102 k 10 m 26 m 40 k 499 IT_1 347 k 5 k k m 31 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 10

11 500- User Load Test Performance Graphs App Server CPU Utilization Heap usage The following graph shows the JVM heap usage during and after the load test for the application servers and the BG Server. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 11

12 Appendix A: Use Cases Use case 1 - Create Projects 1. Go to Homepage. 2. Log in to the application.. 3. Click Projects link. 4. Click New button on project list. Wait 20 seconds. 5. Click Save. 6. Click Team tab. 7. Click Add button. Wait 15 seconds. 8. Filter for desired resources. Wait 15 seconds. 9. Select and add resources to project. 10. Filter for desired resources. Wait 15 seconds. 11. Select and add resources to project. 12. Filter for desired resources. Wait 15 seconds. 13. Select and add resources to project. 14. Click Task tab Open the Gantt Editing Session Insert a subproject. Wait 50 seconds over 4 requests 17. Drill into the subproject (Note: In v13, this opens a new Gantt editing window). 18. Insert a subproject. Wait 35 seconds over 3 requests. 19. Drill into the subproject (In v13, this opens a new Gantt editing window). Wait 15 seconds. 20. Make a change to one of the tasks by editing the start and finish dates; save. Wait 55 seconds over 4 requests. 21. Close the Gantt window. 22. Refresh the parent Gantt window. 23. Close this Gantt window. 24. Refresh the master project Gantt window. 25. Expand All - verify that the changed task dates are displayed correctly. Wait 9 seconds. 26. Close the Gantt editing window. 27. Log out. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 12

13 Use case 2 - PM Projects 1. Go to Homepage. 2. Log in to the application.. 3. View Project List.. 4. Open an existing Project.. 5. Access Project Dashboard tab with OOTB portlets.. 6. Access Project Properties and Project Team tabs Wait 15 seconds over 2 requests 7. Use "Set Allocation" object action to update 15 selected team members to have 50% allocation. Wait 20 seconds over 2 requests. 8. Use object action for "Commit Planned Allocation" to hard book selected team members. Wait 10 seconds over 2 requests. 9. Log out. Use case 3 - PM Financial Planning view and edit Cost Plan 1. View Cost Plan list page should have 500 rows of cost plans. Wait 60 seconds divided over 6 requests. 2. Open the detail of an existing Cost Plan. 3. Update cost plan by repopulating and adding 4 additional detail rows. Wait 60 seconds divided over 6 requests. 4. (Auto-addition of 3 rows by re-populate and manual addition of 1 row repeat for both re-populate types). Wait 120 seconds divided over 12 requests. 5. View Benefit Plan list page (15 Benefit plans). Wait 50 seconds over 5 requests. Use case 3a - PM Financial Planning - Cost Plan 2 1. Create Cost Plan (Two grouping attributes) and add a cost plan detail row. Wait 235 seconds over 24 requests. Use case 3b - PM Financial Planning - Cost Plan 3 1. Create Cost Plan from Investment Team (Project duration 1 year, # of team members 15, FTP type - Monthly). Wait 27 seconds over 20 requests Use case 3c - PM Financial Planning - Cost Plan 4 1. Create Cost Plan from Task Assignments (Project duration 1 year, # of task assignments 15, FTP type - Monthly) and open Benefit Plans page. Wait 90 seconds over 14 requests. Use case 3d - PM Financial Planning - Cost Plan 5 1. Log in.. 2. Click on Projects link. 3. Expand the Projects filter. 4. Filter for Project. 5. Open Project. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 13

14 6. Click on Financial Plans tab. 7. Click on New Cost Plan from Investment Team. 8. Select Grouping Attributes. 9. Click Save for new Cost Plan from Investment Team. 10. Go to Cost Plan Detail Click Return for Cost Plan Detail Click New Cost Plan from Task Assignments. 13. Select Grouping Attributes Click Save for New Cost Plan from Task Assignments. 15. Go to Cost Plan Detail Click Return for Cost Plan Detail Filter for New Cost Plan Open New Cost Plan Select Populate from Task Assignments. 20. Click Return For Cost Plan Filter for New Cost Plan Open New Cost Plan Click Populate From Investment Team. 24. Click Return For Cost Plan Click Project Properties. 26. Click Return For Projects. 27. Logout of the application. Use case 4 - Create Project from Template 1. Click Projects link. Wait 35 seconds over 4 requests. 2. Click Create New from Template. 3. Filter for desired template. 4. Select template. 5. Click Save.. 6. Logout Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 14

15 Use case 5 - Gantt Toolbar 1. Open the 2000 Task project s Gantt editing window. After each action in the list below, save the changes. Wait 55 seconds over 8 requests. 2. Using toolbar, create 10 new tasks. Wait 100 seconds over 10 requests. 3. Using toolbar, copy a set of tasks from another project. Wait 35 seconds over 7 requests. 4. Using toolbar, insert a subproject. Wait 25 seconds over 4 requests. 5. Using toolbar, create a new project and insert it. Wait 15 seconds over 2 requests. 6. Using toolbar, assign a resources to a selected task. Wait 30 seconds over 4 requests. 7. Using toolbar, select 3 tasks and indent them. Then select 2 of those tasks and indent them one more level, then select the bottom task and indent it one more level. Wait 40 seconds over 6 requests. 8. Using toolbar, select 3 top-level tasks and indent them. Then select them again and outdent. Wait 25 seconds over 3 requests. 9. Using toolbar, select a top-level task and use the Move action to move it to another place in the WBS hierarchy. Wait 20 seconds over 2 requests. 10. Using the toolbar, link 3 tasks. 11. Using the toolbar, unlink the 3 tasks. 12. Change the dates on 5 tasks. Save. Using the toolbar, create a new project baseline. 13. Select the 5 tasks and using the toolbar, update the baseline on the selected tasks. Wait 90 seconds over 11 requests. 14. Select 10 tasks and using the toolbar, delete the tasks. Wait 10 seconds over 2 requests. 15. Using the toolbar, view the Gantt Legend. 16. Using the 2000 task project, expand all tasks Using the 2000 task project, collapse all tasks.. Use case 6 - Gantt WBS List 1. Use inline edit to edit 1 row and save. Wait 55 seconds over 11 requests. 2. Use inline edit to edit 20 rows and save. Wait 105 seconds over 21 requests. 3. Expand a single branch that has at least 4 levels of nested tasks. Wait 25 seconds over 5 requests. 4. Inline edit: Assign three resources to a task using auto-suggest on the MVL Assigned Resources field in the grid. Wait 35 seconds over 6 requests. 5. Insert one new blank row; edit; save. Wait 15 seconds over 3 requests. 6. Drag and drop a task into a new position (from top to bottom of the page).. 7. Drag and drop a task into a new position (causing the list to scroll).. 8. Scroll down through the 2000 task project and record how long it takes when each new set of 100 records is fetched. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 15

16 Wait 100 seconds over 20 requests. 9. Expand Filter section. 10. Filter using the OOTB fields: a) Task name. b) List view. c) Assigned Resources. Wait 15 seconds over 3 requests. 11. Drill into a subproject from master project - this opens a separate Gantt editing session. 12. Change start and finish dates for a task; save. Wait 25 seconds over 5 requests. 13. Close the subproject Gantt. 14. Refresh the master project Expand the subproject and confirm updated task dates are seen in the WBS list.. Use case 7 - Interactive Gantt Chart 1. Use Gantt task bar (drag) to change finish date; save. Wait 50 seconds over 8 requests. 2. Use Gantt task bar (drag) to change start date, save. Wait 20 seconds over 2 requests. 3. Use Gantt task bar (drag) to move entire task to later dates; save. Wait 20 seconds over 2 requests. 4. Change timescale on the Gantt chart using the toolbar widget. 5. Move Gantt time periods forward one column. 6. Move Gantt timescale forward one set of time periods. 7. Link two tasks using drag and drop for F-S dependency. 8. Link two tasks using drag and drop for S-S dependency. Wait 20 seconds over 2 requests. Use case 8 - Auto-Schedule 1. Create a tentative schedule without running auto-schedule. Wait 40 seconds over 8 requests. 2. Change task start and finish dates. Wait 15 seconds. 3. Publish the tentative schedule.. 4. Use toolbar to run auto-schedule to create a tentative schedule. Wait 15 seconds. 5. Delete the tentative schedule.. Use case 10 - General UI Behavior 1. Page refresh on Project List page that configured to list 50 projects per page. Wait 70 seconds over 6 requests. 2. Drill-down on a project, and click on the Team tab and Detail subtab. Wait 28 seconds over 3 requests. 3. Add a new TSV column in Detail page Wait 83 seconds over 7 requests. 4. Replace Assignment Roles. Wait 20 seconds over 2 requests. 5. Select a Capacity Planning scenario from the Team tab Wait 40 seconds over 2 requests. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 16

17 6. Edit team member start date, team member finish date and allocation for at least 5 team members in this scenario; Save. Wait 42 seconds over 6 requests. 7. Switch back to Plan of Record. Use case 11 - PM Resource Management: Working with resources 1. View the Resource List. Wait 25 seconds over 2 requests. 2. Open a Resource that is allocated to at least 25 investments (Projects and NPIOs). 3. Go to Resource Allocations tab and view data on the Summary page; go to Page 2 and then back to Page 1 (assumes rows per page is set to 20).. 4. Go to the Allocations - Detail page (this one has a TSV on it); advance time periods forward one set of periods. Wait 30 seconds over 4 requests. 5. Click on Resource Planning menu. 6. Open each Resource Planning tab and view data in each portlet. Wait 15 seconds over 3 requests. Use case 12 - Timesheets 1. Log in. Wait 3 seconds. 2. Navigate to Timesheets page. 3. Filter Timesheet for desired user & time period. Wait 3 seconds. 4. Click on Timesheet icon to display/edit timesheet. Wait 3 seconds. 5. Click on Populate button to populate timesheet. Wait 3 seconds. 6. Confirm repopulate, click on Yes button.. 7. Select and delete time entries from timesheet. Wait 3 seconds. 8. Confirm Delete. Wait 3 seconds. 9. Click on Add Task button Select task & Add task to timesheet. Wait 15 seconds. 11. Submit timesheet. Wait 3 seconds. 12. Open/Edit timesheet. Wait 3 seconds. 13. Return timesheet. Wait 3 seconds. 14. Log out. Use case 13 - PM Financial Transactions 1. View Transaction Vouchers List Page (20 records on first page). Wait 25 seconds over 5 requests. 2. View Transactions (Detail) List Page - Should Include 20 Transactions.. 3. Create a new transaction, retain values then click Submit and Create new. Wait 40 seconds over 7 requests. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 17

18 Use case 14 - Departments 1. Navigator. Wait 15 seconds over 2 requests. 2. Departments (show 50 departments with 9 columns).. 3. Select department.. 4. Select resources.. 5. Select investments.. Use case 15 - Navigator All Page Script 1. Overview Wait 20 seconds over 3 requests 2. General 3. Organizer a. Tasks b. Calendar c. Process d. Notifications 4. Dashboards a. New i. Cancel 5. Portlets 6. Chart a. Select one i. Select Graph and Next ii. Back 7. Grid a. Select one i. Cancel 8. Filter a. Cancel 9. Interactive a. Cancel 10. Timesheets 11. Reports and jobs a. Jobs Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 18

19 b. Reports 12. Account settings a. Proxy b. Notifications c. Software downloads d. Personal information 13. Organization 14. Departments a. New i. Cancel 15. Knowledge Store 16. IT Service Management 17. Services a. New i. Cancel 18. Requirements Planning 19. Release Planning a. New i. Cancel 20. Releases a. New i. Cancel 21. Requirements a. New i. Cancel 22. Portfolios a. New i. Cancel 23. Projects a. New b. Cancel 24. New from template a. Cancel Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 19

20 25. Select project a. Dashboard b. Team i. Detail ii. Role Capacity iii. Participants iv. Participant Groups v. Requisitions vi. Staff c. Task d. i. Task List ii. Assignments e. Chargebacks i. Options ii. Transactions iii. Debit Rules 1. New f. Processes g. Properties i. Cancel 26. Programs a. New i. Cancel 27. Products a. New i. Cancel 28. Other Work a. New i. Cancel 29. Demand Management a. Ideas a. Cancel Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 20

21 b. New i. Cancel c. Incidents 30. Resource Management 31. Resources a. New i. Cancel b. Select one. c. Skills i. Add 1. Back d. Allocations i. Add 1. Back e. Document Manager 32. Resource planning a. Allocations b. Capacity c. Investments d. Workloads 33. Resource Finder a. Filter 34. Enter Resource ID a. Search i. Select one. 35. Skills a. Add i. Back 36. Allocations a. Add i. Back 37. Document Manager Allocations Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 21

22 38. Calendar 39. Properties a. Cancel 40. Resource Requisitions 41. Transaction Entry a. New b. Transaction Entry 42. Post to WIP. 43. Create WIP Adjustment. 44. Approve WIP Adjustment. 45. Transactions 46. Invoices Use case 16 - PM Rate Matrix 1. Open Rate/Cost Matrix. 2. Create a new Matrix row properties page (5 assigned columns). Wait 45 Seconds over 5 requests Use case 18 - Administration All Page Script 1. Organization and Access i) Resources (1) Select resource (2) OBS and Partitions (3) Groups (a) Add (b) Back (4) Resource s Access Rights global (a) Add (b) Back (5) Access to this resource Full view, (a) Continue (b) Add (c) Exit (6) Access to this resource group Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 22

23 (a) Add (b) Exit (7) Access to this resource obs unit (a) Add (b) Exit (c) (8) Groups (9) select group (a) Resources (i) Add (ii) Back (b) Group s Access Rights Instance (i) Add (ii) Back (c) Group s Access Rights OBS Unit (i) Add (ii) Exit (d) Group s Access Rights Global (i) Add (ii) Back (10) OBS (a) New (b) Back 2. Studio i) Objects (1) Select Project: (a) Attributes (i) Select one (ii) Back (iii) New attribute (iv) Back (b) Linking (i) Select one (ii) Back Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 23

24 (iii) New (iv) Back (c) Actions (i) Select one (ii) Close (iii) New (iv) Close (d) Views (e) Audit Trail (i) Access To This Object (ii) Full View (iii) Resource (iv) Group (v) OBS Unit ii) Queries (1) New (2) Back iii) Portlets (1) Select One (a) Access To This Portlet 1. Full View 2. Resource 3. Group 4. OBS Unit (b) Properties (c) Back (2) New (a) Chart Portlet Wait 4 seconds over 2 requests (b) Grid Portlet Wait 4 seconds over 2 requests (c) Filter Portlet Wait 4 seconds over 2 requests (d) HTML Portlet Wait 4 seconds over 2 requests (e) Interactive Portlet Wait 4 seconds over 2 requests iv) Portlet Pages Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 24

25 (1) Select one (a) Link Parameters a. New b. Back (b) Tabs a. New b. Back (c) Access To This Object a. Back (2) New (a) Back v) Add-Ins vi) UI Themes vii) Views (2) Select one (a) Exit 4. Data Administration a) Time Slices i) Select one (1) Back ii) New (1) Back b) Lookups i) Select one (1) Parent Window (2) Values (a) Select one (i) Back (3) Auto Suggest (a) Back (4) General (a) Back ii) New Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 25

26 (1) Back c) Reports and Jobs i) Select one: (1) Parameters (a) Access one (i) Back (2) Parameter order (3) Incompatible Jobs (a) Add (i) Back (4) Associated Categories (a) Add (i) Back (5) Access to this Job (a) Full View (b) Resource (c) Group (d) OBS Unit d) Skills Hierarchy i) New (1) Back e) Notifications i) Select one. (1) Cancel 5. Project Management a) Timesheet Options b) Time Reporting Periods i) New (1) Back c) Charge Codes i) New (1) Back d) Input Type Codes Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 26

27 i) New (1) Back e) Settings f) Base Calendars i) New (1) Close g) Migrate Methods h) Risk Settings 6. General Settings a) System Options b) Site Links i) New (1) Back c) Client Downloads Use case 19 - Approve Timesheet 1. Open a Timesheet. Wait 40 seconds over 5 requests. 2. Approve Timesheet.. Use case 21a New Portfolio 1. Log in. Wait 10 seconds over 2 requests. 2. Click on Portfolios Link.. 3. Click on New portfolio button. Wait 20 seconds. 4. Enter details and click on Save button.. 5. Click on Manual link. Wait 20 seconds. 6. Click on Contents Editor tab. 7. Select Projects, Assets and Applications and click on Include button.. 8. Click on Build Power Filter link for Projects.. 9. Select Description as power filter. 10. Add Description as to power filter Click on Evaluate button to validate the filter.. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 27

28 12. Click Save and Return for Projects power filter Click on Build Power Filter link for Assets Select Description as power filter. 15. Add Description as to power filter Click on Evaluate button to validate the filter Click Save and Return for Assets power filter Click on Build Power Filter link for Applications Select Description as power filter. 20. Add Description as to power filter Click on Evaluate button to validate the filter Click Save and Return for Applications power filter Click on Properties tab Click on Synchronization Schedule link. 25. Click on Sync Now button. Wait 10 seconds over two requests. 26. Log out of the application. Use case 21b New Portfolio 1. Log in. Wait 10 seconds over two requests. 2. Click on Portfolios link.. 3. Expand the filter. Wait 3 seconds. 4. Filter for portfolio. Wait 3 seconds. 5. Open portfolio. Wait 3 seconds. 6. Click on Plans tab.. 7. Click on New button for new plan. 8. Enter details and save plan Select the created plan and click on Copy button. 10. Enter details and save plan Click on Investments tab Click on Plans tab.. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 28

29 13. Select plan 1 to edit. 14. Edit and save plan Click on Targets tab. 16. Select plan 2 to edit. Wait 20 seconds. 17. Edit and save plan Return to plans list Select plans 1 and 2 to delete Delete selected plans Log out of the application. Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 29

30 Appendix B: Load Test Details 500 User Load Test Details (all response times in seconds) Transaction Name Average HomePage Login Project link New Project button Create Project Team Tab Add button Filter Resource Icon Filter Resource - I Filter Resource - II Filter Resource - III Add Resource - I Add Resource - II Add Resource - III Tasks Tab Open Gantt Existing Sub Project Link Filter Sub Project Icon Filter Sub Project Add Sub Project Open Sub Project Add Existing Sub Project link Filter Sub Sub Project Add Sub Sub Project Open Sub Sub Project Click to Edit Task dates Edit Task Start date Edit Task Finish date Save Task Start & Finish dates Refresh Sub Gantt Refresh Gantt Delete Sub Sub Project Expand all Logout 0.04 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 30

31 Transaction Name Average HomePage Login Projects SelectProject Dashboard a-V13- Properties Team Select20MembersAllocate p Select20MembersAllocate Allocate50PercentSubmit Select20MembersConfirmAllocation p Select20MembersConfirmAllocation ConfirmAllocation Logout Home Page Login Click Projects Filter For Project Open Project Click Financial Plans Filter For Cost Plan Open Cost Plan Click Populate From Investment Team a-V13-Select InlineEdit Role b-V13-Select InlineEdit ChargeCode Click Add Detail a-V13-InlineEDit New Detail Click Save For Team Detail Click Save For Team Detail Select Populate From Task Assignments a-V13-Select InlineEdit Role b-V13-Select InlineEdit ChargeCode Click Add New Task Detail b-V13-InlineEdit New Task Detail Click Save For Task Detail Click Save For Task Detail Select Details And Delete Confirm Delete Details 1.09 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 31

32 Transaction Name Average Click Cost Plan Properties Click Save And Return Cost Plan Select Benefit Plans Open Benefit Plan Return From Benefit Plan Click Project Properties Click Save And Return For Project Click Projects Logout a-001-HomePage a-002-Login a-003a-V13-Expand Filter a-003-Click Projects a-005-Filter For Project a-006-Open Project a-007-Click Financial Plans a-008a-V13-InlineEdit Attrib a-008b-V13-InlineEdit Attrib a-008-Click New Manual Plan a-011-Click Save For Cost Plan a-012-Click Cost Plan Detail a-013-Click Add Detail a-014-V13-Select InlineEdit Role a-016-V13-Select InlineEdit Charge Code a-017a-V13-Edit Inline Detail Values a-017-Click Save Detail a-019-Save New Detail Details a-020-Click Return For Cost Plan a-021-Filter For Cost Plan a-022-Click Project Properties a-023-Click Return For Project a-024-Clear Project Filter a-025-Logout b-001-Homepage b-002-Login b-003_1-Exand Filter b-003a-Filter Project b-003b-select Project 0.31 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 32

33 Transaction Name Average 03b-003-Projects b-005-FinancialPlans tab b-006.a-CP from Inv Team b-006.a-V13-Enter Mandates b-006.b-V13-Save and Return b-006.c-Filter costplan b-007b-close Costplan b-007-Open Costplan b-008-logout c-000 HomePage c-000 Login c-001 Projects c-002 Filter Project c-003 Open Project c-004 Financial Plans c-005 New from TA_RP c-006 Create Cost Plan c-007-V13- Detail Tab c-008 Add Cost Plan Detail c-009 Filter Charge Code c-010 Add Filtered Charge Code c-011 Filter Role c-012 Add Filtered Role c-013 Populate from TA_RP c-014 Return c-015 Benefit Plans Tab c-016 Logout d-001 Home Page d-002 Login d-003 Click Projects d-004 ExpandProjectFilter d-005 Filter For Project d-006 Open Project d-007 Goto Financial Plans d-008 Click New Cost Plan from InvestmentTeam d-009 Select Grouping Attributes d-010 Click Save for new CP from INV Team d-011 Goto CP Detail Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 33

34 Transaction Name Average 03d-012 Click Return for CP detail d-013 Clcik New CP from Task Assignments d-014 Select Grouping Attributes d-015 Click Save for New CP from Task Assignments d-016 Gogo Cost Plan Detail d-017 Click Return for Cost Plan d-018 Filter for New Cost Plan d-019 Open New Cost Plan d-020 Select Populate from Task Assignments d-021 Click Return For Cost Plan d-022 Filter for New Cost Plan d-023 Open New Cost Plan d-024 Clcik Populate From Investment Team d-025 Click Return For Cost Plan d-026 Click Project Properties d-027 Click Return For Projects d-028 Logout HomePage Login Project link New From Template Filter Project Icon Filter Porject Template Select Template Create Project from Template Logout HomePage Login Projects Link Filter Project icon Filter Project Open Project Tasks Tab Gantt Create Task Icon Create Task - I Create Task - II Create Task - III 1.00 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 34

35 Transaction Name Average Create Task - IV Create Task - IX Create Task - V Create Task - VI Create Task - VII Create Task - VIII Create Task - X Filter Tasks Copy Task from Template Icon Filter Project Icon Filter Project Select Project Expand Tasks Copy Tasks from Template Add Existing Sub Project Icon Filter Project Icon Filter Project Add Sub Project Create New Project Icon Create New Project Assign Resource Icon Filter Resource Filter Resource Add Resource Indent 3 Resources Expand Task containing 3 Indent tasks Indent 2 Tasks Expand Task containing 2 Indent tasks Indent 1 Task Expand Task containg 1 Indent Task Indent 3 Top-Level tasks Expand Tasks containing 3 Indent Tasks Outdent 3 tasks Select Task & Click Move Task Move Task Link 3 Tasks Unlink 3 Tasks Edit Task Dates 0.21 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 35

36 Transaction Name Average Edit Task Date - I Edit Task Date - II Edit Task Date - III Edit Task Date - IV Edit Task Date - V Save Five Task Dates Create Baseline Icon Create Baseline Project Update Baseline for 5 Tasks Update Baseline for 5 Tasks - Yes Delete 10 Tasks Delete 3 Tasks Show All Expand All Collapse All Logout Home Login Porject link Filter Project Icon Filter Project Open Project Tasks Tab Gantt Edit 1st Row Edit Date Save 1st Row Edit Row I Edit Row II Edit Row III Edit Row IV Edit Row IX Edit Row V Edit Row VI Edit Row VII Edit Row VIII Edit Row X Edit Row XI 0.05 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 36

37 Transaction Name Average Edit Row XII Edit Row XIII Edit Row XIV Edit Row XIX Edit Row XV Edit Row XVI Edit Row XVII Edit Row XVIII Edit Row XX Save Changes - 20 Rows Insert button Inline edit Save Inline edit Drag and drop a task from top to bottom of the page Drag and drop a task into a new position-causing the list to scroll Scroll Up scroll down 100 records I scroll down 100 records II scroll down 100 records III scroll down 100 records IV scroll down 100 records IX scroll down 100 records V scroll down 100 records VI scroll down 100 records VII scroll down 100 records VIII scroll down 100 records X scroll down 100 records XI scroll down 100 records XII scroll down 100 records XIII scroll down 100 records XIV scroll down 100 records XIX scroll down 100 records XV scroll down 100 records XVI scroll down 100 records XVII scroll down 100 records XVIII Filter task Expand Level II Expand Level III 0.16 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 37

38 Transaction Name Average Expand Level IV Expand Task I Filter task Edit Resource Fileld Assign Resource I Assign Resource II Assign Resource III Save & Assign Resource Delete task Filter List view Filter with Assigned Resources Filter SubProject Open SubProject Edit task dates Edit Start date Edit finish date Save Task dates Refresh Master Project Expand SubProject Logout Homepage Login a-Filter Projects Projects Openproject Tasktab a- Find gantt date gantt Change Finish date using gantt task bar save the Changes for Finish date Change start date using gantt task bar save the Changes for start date move entire task to later date using gantt task bar drag save after entire date movement change time scale Forward one Column Forward one set of time period Save and Return 0.00 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 38

39 Transaction Name Average F-S dependecy S-S dependecy delete F-S and S-S dependecies Logout Homepage login projects filterforproject openproject Unlock taskstab clicknewschduletocreatetentativeschedule V13-change start and finish dates submit publishtentativeschedule yestopublish clickautoschedule autoschdulebutton DeleteTentativeSchedule yestodelete logout Homepage Login Click Projects ChangeProjectDisplayOptions ClickOptions ChangeRowsPerPageAndSaveReturn OpenProject ClickTeamTab ClickTeamDetailTab ClickConfigureTSVOptions ClickFields ClickNewField ClickNextForField ClickSaveReturnForTSV ClickLayout ClickSaveReturnForTSVApply V13-SelectRoleforResource 0.26 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 39

40 Transaction Name Average SelectResourceAndClickSave ClickTeamStaff SelectScenario a-V13-GetFirstRowDetails b-V13-GetFirstRowDetails c-V13-GetFirstRowDetails d-V13-GetFirstRowDetails e-V13-GetFirstRowDetails ClickSaveForScenario ChangetoPlanOfRecrod Logout HomePage Login Resources Filter Resource a SelectResource ResourcesAllocations ResourcesAllocationsPage ResourcesAllocationsPage ResourcesAllocationsDetail ResourcesAllocationsDetailNextPeriod ResourcePlanning ResourcePlanningAllocations ResourcePlanningInvestments ResourcePlanningCapacity Logout HomePage Login Timesheets Filter Timesheet Edit Timesheet Populate Timesheet Repopulate Confirmation Delete Task Confirm Delete Add Task Select Task & Add Submit Timesheet 0.29 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 40

41 Transaction Name Average Return Timesheet Logout Homepage Login Transaction Entry Filter Voucher Open Voucher a-Select Transaction Date b-Select Investment ID Create New Transaction c-Select Task d-Select Charge Code e-Select Resource ID f-Select Input Type Code Save and Create New Logout Homepage Login Departments ResourcesTab InvestmentsTab Logout Homepage Login Personal-Overview-General Personal-Organizer Personal-Organizer-tasks Personal-Organizer-Calendar Personal-Organizer-Process Personal-Organizer-Notifications Personal-Organizer-ActionItems Personal-Organizer-ActionItems-New Personal-Organizer-ActionItems-New-Cancel Personal-DashBoards Personal-DashBoards-New Personal-DashBoards-New-Cancel Personal-Portlets Personal-Portlets-NewPortlet-GraphPortlet 0.58 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 41

42 Transaction Name Average Personal-Portlets-NewPortlet-GraphPortlet-SelectOne-Next Personal-Portlets-NewPortlet-GraphPortlet-SelectOne-Next-Cancel Personal-Portlets-NewPortlet-Grid Portlet Personal-Portlets-NewPortlet-Grid Portlet-SelectOne-Next Personal-Portlets-NewPortlet-Grid Portlet-SelectOne-Next-Cancel Personal-Portlets-NewPortlet-Interactive Portlet Personal-Portlets-NewPortlet-Interactive Portlet-Cancel Personal-Timesheets Personal-ReportsandJobs Personal-ReportsandJobs-Jobs Personal-ReportsandJobs-Reports Personal-AccountSettings Personal-AccountSettings-Proxy Personal-AccountSettings-Notifications Personal-AccountSettings-SoftwareDownloads Personal-AccountSettings-Personal Information Organization-Departments Organization-Departments-New Organization-Departments-New-Cancel Organization-Knowledge Store IT Service Management-Services IT Service Management-Services-New IT Service Management-Services-Cancel Requirement Planning-Release Planning Requirement Planning-Release Planning-New Requirement Planning-Release Planning-New-Cancel Requirement Planning-Releases Requirement Planning-Releases-New Requirement Planning-Releases-New-Cancel Requirement Planning-Requirements Requirement Planning-Requirements-New Requirement Planning-Requirements-New-Cancel Portfolio Management-Portfolios Portfolio Management-Portfolios-New Portfolio Management-Portfolios-New-Cancel Portfolio Management-Projects Portfolio Management-Projects-New Portfolio Management-Projects-New-Cancel 1.08 Copyright 2015 CA Technologies, Inc. All rights reserved. No unauthorized copying or distribution permitted. (Version 1.0) 42