Rational ClearQuest 8.0 Release Report

Transcription

1 Rational ClearQuest 8.0 Release Report Dated: 01 November, 2011 Updated: 18 July, 2012 IBM Corporation 2012

2 Trademarks IBM, the IBM logo, and ibm.com are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at The listed trademarks of the following companies require attribution: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of The Minister for the Cabinet Office, and is registered in the U.S. Patent and Trademark Office. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Other product and service names might be trademarks of IBM or other companies. Dated 18 July 2011 Printed 19 July 2012 Page 2 of 24

3 Contents 1 Introduction Rational ClearQuest Version 8.0 Feature Business Value Ease of Administration Full Text Search ClearQuest ALM ClearQuest Reporting ClearQuest Java Independently install & update WAS 7 and IHS Flexible ClearQuest Enabled ClearCase Unified Change Management Integration Collaborative Lifecycle Management (CLM) ClearQuest Charts on RTC dashboards WebSphere Application Server User Improvements Web Help ClearQuest new platforms support Testing and Methodology Metrics APAR and Defect Fixes Testing Strategy Functional Verification Testing Coverage Systems Verification Testing Coverage In-House Deployment Testing Beta Testing Key Process Improvement Areas ClearQuest Web Performance, Scalability and Availability ClearQuest Web Performance ClearQuest Web Scalability ClearQuest Web Availability Testing Topologies and Transaction Diagrams Customer Feedback The Design Partner Program Beta Program Future Feedback...24 Dated 18 July 2011 Printed 19 July 2012 Page 3 of 24

4 1 Introduction This document provides a high-level overview of the content of the IBM Rational ClearQuest 8.0 release, and highlights the value of this release to IBM Rational customers. Rational ClearQuest Version 8.0 focuses on the following areas: Business Value: Improving features and addressing Requests For Enhancement (RFEs) to benefit our customers. Testing Methodology and Metrics: Testing the code so that it is designed to meet feature requirements and does not regress the performance of previous releases. System Performance: Providing that the Rational ClearQuest release is scalable, reliable and meets performance requirements. Customer Feedback: Working with customers to understand and incorporate their feedback into Rational ClearQuest to better meet business needs. Disclaimer Any data contained herein, including performance-related data, were ascertained in a controlled environment. The results obtained in other operating environments may vary significantly. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Any test results are specific to the product software, test configuration, workload, and environment that were used. Product performance in other environments or conditions may be different compared to the data reported in this document. Statements regarding IBM s future direction and intent are subject to change or withdrawal without notice and represent goals and objectives only. Dated 18 July 2011 Printed 19 July 2012 Page 4 of 24

5 2 Rational ClearQuest Version 8.0 Feature Business Value To succeed, a product must provide business value to customers. The following aspects of enhancing business value were the focus of this release: The following sections detail the features included in the release, 2.1 Ease of Administration ClearQuest 8.0 comes with many enhancements that will help with Ease of administration. Full Text Search Admin Tool: Use the IBM Rational ClearQuest Full-Text Search Administrator tool in ClearQuest version 8.0 to set up, configure, and administer the full-text search feature. This tool is included in the product as of ClearQuest version 8.0. ClearQuest ALM: The Application Lifecycle Management (ALM) packages provide a solution for ensuring ALM best practices in existing Rational ClearQuest change management systems. Application Lifecycle Management enables streamlining a team s ability to produce a software release. Users can manage and configure their WebSphere Application Server and IBM HTTP Server environments independent of the ClearQuest environment. Flexible ClearQuest Enabled ClearCase Unified Change Management Integration Dated 18 July 2011 Printed 19 July 2012 Page 5 of 24

6 2.1.1 Full Text Search Use the IBM Rational ClearQuest Full-Text Search Administrator tool in ClearQuest version 8.0 to set up, configure, and administer the full-text search feature. This tool is included in the product as of ClearQuest version 8.0. This tool replaces existing full-text search tools such as cqtssetup.pl and cqtsdbcrawler.pl, which are included in ClearQuest 7.1.x. The tool hides the complexity and low-level details of setting up, configuring, and administering full-text search. Use the Rational ClearQuest Full-Text Search Administrator tool to run a set of command-line options from the server where the full-text search feature is installed. The Rational ClearQuest Full-Text Search Administrator tool is available in the following languages: English, Brazilian, Portuguese, French, German, Italian, Spanish, Simplified-Chinese, Traditional-Chinese, Japanese, and Korean ClearQuest ALM 1.3 The Application Lifecycle Management (ALM) packages provide a solution for ensuring ALM best practices in existing Rational ClearQuest change management systems. Application Lifecycle Management enables streamlining a team s ability to produce a software release. It involves the coordination of software development activities and assets to produce and manage software applications throughout their life cycles. The ClearQuest ALMWork package builds on the ALMProject package. These packages comprise a collection of tightly integrated record types that create a foundation for managing software or other kinds of development projects. The ClearQuest Attachments 2.1 package provides external storage of ClearQuest attachments in off-site locations, such as FTP servers. This release is certified and supported for ClearQuest ALM schema, any other schema, and supports stateless record-type. ClearQuest Attachments 2.1 augments the existing attachment solutions provided by the ClearQuest Attachments 1.0 solution. The ClearQuest Plus 2.1 package provides enhanced notification functionality to ClearQuest users. This release is certified and supported for ClearQuest ALM schema, any other schema, and supports stateless record-type. Plus 2.1 augments the existing notification solutions provided by the ClearQuest 1.0 package. Highlights of the ALM 1.3 package include: Improved usability by leveraging CQ pop-up form support to reduce user mouse clicks Pre-population of some fields for workflow automation Highlights of the Plus 2.1 package include: Support for rich text, form layout and rich rules on notifications from ClearQuest Addresses over 60 customer RFE's Dated 18 July 2011 Printed 19 July 2012 Page 6 of 24

7 Enable package to be applied on any schema Highlights of the Attachments 2.1 package include: Store ClearQuest attachments on a remote FTP site Control the file size of attachments Enable package to be applied on any schema Additional information can be found at: ClearQuest Reporting Rational ClearQuest provides a flexible reporting solution to design and test reports with Business Intelligence and Reporting Tools (BIRT) or Crystal Reports reporting tools configured with the ClearQuest Reporting Driver. Users can share ClearQuest reports with the Report Launcher for ClearQuest or access reports from Rational Team Concert to facilitate agile planning. topic=/com.ibm.rational.clearquest.webadmin.doc/topics/c_reporting_ovw.htm Crystal Reports Migration Tool A utility to help migrate your Crystal Reports data-push reports to data-pull reports, which are recommended in ClearQuest 7.1 and later. Report Exporter - The IBM Rational ClearQuest Report Exporter is a command line tool used to migrate report files from the Report Server for ClearQuest repository and save the files to a local file system ClearQuest Java 6 Java applications that use CMAPI must run using Java 6 Java Runtime Environment (JRE). ClearQuest performance is improved due to this upgrade in Java Independently install & update WAS 7 and IHS 7 Users can manage and configure their WebSphere Application Server and IBM HTTP Server environments independent of the ClearQuest environment. Dated 18 July 2011 Printed 19 July 2012 Page 7 of 24

8 Recommended fixpacks for WebSphere Application Server and IBM HTTP Server can be applied without upgrading ClearQuest. Customers can deploy ClearQuest Web in WAS profiles which run as a user other than root. This addresses security concerns related to ClearQuest Web running as root on UNIX/Linux servers. Customers can deploy ClearQuest Web along with other applications such as ClearCase CCRC WAN Server in the same WAS profile on servers with low scalability requirements. Footprint of a ClearQuest installation and size of a ClearQuest repository download is reduced significantly. Upgrade of ClearQuest is light weight in size and provides faster upgrade time Flexible ClearQuest Enabled ClearCase Unified Change Management Integration The Unified Change Management ClearQuest integration has been re-implemented using the Open Services for Lifecycle Collaboration (OSLC) interface, leading to two deployment enhancements. A ClearCase client on a Local Area Network no longer needs to include a ClearQuest client to support the Unified Change Management ClearQuest integration. In a Wide Area Network installation, the ClearQuest Web Server and ClearCase CCRC WAN Servers may be installed on different physical machines, allowing a truly centralized installation of ClearQuest while continuing to support the Unified Change Management ClearQuest integration. 2.2 Collaborative Lifecycle Management (CLM) You can use Collaborative Lifecycle Management (CLM) integrations with products based on Jazz technology, including IBM Rational Team Concert, IBM Rational Quality Manager, and IBM Rational Requirements Composer, to connect the work of all team members. Collaborative Lifecycle Management integrations with Rational ClearQuest Web are also referred to as the Rational ClearQuest Bridge. Collaborative Lifecycle Management integrations provide a common approach to artifact linking, dashboards, security, and user interface frameworks. Cross-product links support traceability, web-like navigation, commenting, and status tracking across project repositories. Collaborative Lifecycle Management integrations provide the following capabilities: Link to existing artifacts across repositories in integrated products; for example, ClearQuest records are linked to work items, test cases, and requirements. Hover over links to quickly check that status of associated work; for example testers can monitor the status of a defect that they reported to the development team. Add a widget to a dashboard to report query results or display ClearQuest charts. ClearQuest 8.0 is a Open Services for Lifecycle Collaboration Change Management (OSLC-CM) 2.0 provider and an OSLC-CM 1.0 consumer. Dated 18 July 2011 Printed 19 July 2012 Page 8 of 24

9 2.2.1 ClearQuest Charts on RTC dashboards ClearQuest 8.0 provides out of the box charting widgets that can be added to any Jazz dashboard starting with Jazz Team Server These charts also provide similar functionality to other Jazz charting widgets such as drill down WebSphere Application Server 7 ClearCase, ClearQuest and Jazz servers are WebSphere Application Server (WAS) applications. All three products support WAS 7 supporting common WAS 7 based flexible deployment on the same server or independent servers. Each web server application may be deployed in its own WAS profile or combined with other applications in a common profile. Configuration is based on customer requirement, usage model and scalability requirements. 2.3 User Improvements Web delivery of help ClearQuest new platforms support Web Help Beginning with version , ClearQuest now provides the option to retrieve help content from the web dynamically. This capability enables you to always have the latest content that is available from within your product. For customers with Internet restrictions help content can be served by an intranet server. You can also install the help content locally on your computer. For additional information on the help system: topic=/com.ibm.help.common.rational.getstart.doc/topics/getting_started_with_help.html Dated 18 July 2011 Printed 19 July 2012 Page 9 of 24

10 2.3.2 ClearQuest new platforms support ClearQuest continues to update platform support in fixpacks and major releases like 8.0. Please refer to link below for the latest platforms and integrations supported. IBM Rational ClearQuest 8.0 Release Notes Dated 18 July 2011 Printed 19 July 2012 Page 10 of 24

11 3 Testing and Methodology Metrics To ensure the reliability of Rational ClearQuest, tests were run to validate as much of the product s functionality as possible, while providing comprehensive client and server platform coverage. 3.1 APAR and Defect Fixes The release also included many more defect fixes that were the result of internally found defects or externally reported APARs from customers. Please refer to charts below for metrics around quality goals for this release. The IBM Corporate Instruction definition of reliability is that the software must execute with no program errors and produce correct results in all client usage scenarios. One element of reliability commitments is to focus on the improvement of the quality of the product in terms of APARs, which are defects that have been reported by a customer. For Rational ClearQuest version 8.0, 71% of APARS reported since V7.1.2 were closed. The chart below shows the percentage of APARs by severity that were fixed in Rational ClearQuest version 8.0. APAR resolution percentages: 76.0% 74.0% 72.0% 73% 74% 70.0% 71% 68.0% 66.0% 67% 64.0% 62.0% DS1 DS2 DS3/4 Total Figure 1: APAR resolution percentages The quality goals for the 8.0 release are to fix 100% DS1 & DS2 defects (current release & internal backlog), and 95% all DS1 DS4s found in current release (Non-APAR). Both goals were met or exceeded. The severity distribution for the internally found defect fixes in this release is shown below, with DS1 being most severe and DS4 being least severe. Dated 18 July 2011 Printed 19 July 2012 Page 11 of 24

12 Defect Resolution Percentages 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% 100% 97% DS1 DS2 DS3/DS4 Figure 2: Defect resolution percentages 3.2 Testing Strategy As part of our adoption of agile development practices, testing was conducted within each iteration throughout the release targeting the changes delivered in that iteration. In order to facilitate agile best practices, the entire development team uses ClearQuest integrated with Rational Team Concert (RTC) and Rational Quality Manager (RQM) to support agile planning, defect tracking, test planning, test tracking and overall status reporting via dashboards. The testing strategy for Rational ClearQuest 8.0 includes Functional testing, System testing, In-House Deployment, Performance testing and Beta testing. Functional tests examine each feature area or component and validate that the user can perform the expected actions. System tests examine typical customer usage, for example, deploying Rational ClearQuest and Rational ClearQuest MultiSite into a large enterprise environments as well as integrations with other products. Rational ClearQuest and Rational ClearQuest MultiSite are used in the development process. Before Rational ClearQuest is installed on internal servers or clients, testing occurs to ensure that there have been no regressions since the last release and that all new features are qualified. This is followed by In-House Deployment testing, where the product is used in production by the entire ClearCase and ClearQuest development and support organizations. Performance testing is covered throughout the release targeting specific performance goals. Beta testing was conducted throughout the release including two open beta programs available to all existing or potential customers worldwide Functional Verification Testing Coverage As part of adoption of agile development practices, Functional Verification Testing (FVT) planning of the release begins with Epic and User Stories. These stories define requirements and new capabilities for the release that include identifying acceptance criteria for new functionality. Acceptance criteria includes the set of tests that must pass in order to declare that the feature is ready to be released. Additionally FVT includes overall regression testing that must be conducted during the release to validate that overall functionality continues to function as expected, and that no regressions were introduced. Dated 18 July 2011 Printed 19 July 2012 Page 12 of 24

13 FVT attempts to cover as much as of the product functionality as possible via both automation and manual testing. New tests are developed to cover new or changed functionality during each iteration. The chart below shows the ClearQuest test coverage by Functional Area. ClearQuest FVT Coverage by Area 14% 3% 4% 19% ClearQuest Web Clients/CQ ALM Upgrade/Install Full Text Search 11% Reporting Core 7% 29% Web Help/Documentation Integrations 13% Figure 3: FVT coverage by area Systems Verification Testing Coverage System Verification Testing (SVT) encompasses a variety of testing arenas. Complex customer-like schemas are tested before and after upgrading ClearQuest from a prior release to this version. SVT also covers integration testing between Rational ClearQuest and other applications. Particular focus was on testing integrations between ClearQuest and the Collaborative Lifecycle Management tools: Rational Team Concert, Rational Quality Manager, Rational Requirements Manager as well as DOORS. Reporting and report migration was also a focus area for the System Test Verification In-House Deployment Testing The testing strategy also includes In-House Deployment in which tested development versions of the ClearQuest product are installed on ClearQuest client systems as well as production servers that are widely used by the development, testing and support teams. The ClearQuest application is a complex multi-sited schema with integrations to many Rational products including Rational Team Concert (RTC) and Rational Quality Manager (RQM) as well integrations to other business systems within IBM. New features such as ClearQuest data visualisation using RTC dashboards were tested in our production environment. Dated 18 July 2011 Printed 19 July 2012 Page 13 of 24

14 ClearQuest Web was deployed on a WAS 7.0 server along with Full Text Search using the Full Text Search Administration tool Beta Testing At IBM Rational, we are committed to providing high-quality products through a regular program of thorough product testing both inside and outside of the company. Internal controls in product development, quality assurance, and technical support ensure the technical quality of the product. But it is the Beta testing at client sites that ensures products can withstand the challenges of real-world processing. The Rational Beta Program works with product teams to choose the beta program that best suites their goals. Beta programs allow you to evaluate and provide feedback on IBM products before the products' general availability. All Beta programs require that you accept the terms of use of the beta product. These programs are typically confidential. There are two types of beta programs that were run in the 8.0 release described below. Managed beta: This evaluation requires a high level of commitment from the customers to provide feedback through periodic status calls and surveys. Customers might be asked to provide a quote or become a reference for the product. Several customers participated in the managed beta programs for ClearQuest for the 8.0 Release. Open beta: These programs are not confidential and are open to anyone. Many customers participated in the open beta program for this release. This can range from hundreds on some products to thousands on other products. All defects reported and survey results are evaluated for inclusion in the current product and if not included may be deferred to a future release. ClearQuest had 2 beta programs during the 8.0 Release. Some key statistics are shown below. # of PRs Entered 71 # of Defects Entered 43 # of Defects Resolved 36 % Defects Resolved 83.70% # of RFEs Entered 18 # of RFEs Fixed 12 % RFEs Fixed 67.00% Figure 5: Beta statistics *PRs are Problem Reports entered into the Problem Reporting Tool. *Defects and RFEs (Request For Enhancement) are actual Work Items that were created. Dated 18 July 2011 Printed 19 July 2012 Page 14 of 24

15 3.2.5 Key Process Improvement Areas Test Process Improvement Testing was conducted within each iteration that included a new or changed feature or capability. In this way, defects were discovered and fixed while the code was being developed. Automated build-triggered tests were run on a wide cross section of supported operating system platforms and database versions with extensive coverage of ClearQuest core components. Junit Testing Junit tests were developed and enhanced to cover ClearQuest Web Server capabilities including OSLC-based integrations. Automation In addition to unit and functional test coverage for the ClearQuest core and ClearQuest Web Server components, test automation was expanded to cover the Graphical User Interfaces (GUI) for ClearQuest Web, the ClearQuest Designer (Eclipse based) and ClearQuest client (Eclipse based). A subset of these tests written with Rational Functional Tester (RFT) were executed daily via the build-triggered system. More comprehensive suites of RFT-based tests were run on a wider variety of configurations throughout the release. Dated 18 July 2011 Printed 19 July 2012 Page 15 of 24

16 4 ClearQuest Web Performance, Scalability and Availability In terms of system performance, the following areas were the focus for IBM Rational ClearQuest Version 8.0: Performance: The ability of a system to respond acceptably to user requests Scalability: The ability of a system to respond to increased load and still respond with acceptable performance Availability: The ability of a system to respond for long durations with consistently acceptable performance Disclaimer for Performance Data: Any Performance data contained herein was determined in a controlled environment. The results obtained in other operating environments may vary significantly. Actual results may vary. Users of this document should verify the applicable data for their specific environment. These test results are specific to the product software, test configuration, workload, and environment that were used. Product performance in other environments or conditions may be different compared to the data reported in this document. 4.1 ClearQuest Web Performance IBM Rational ClearQuest Web version 8.0 offers significant single-user performance improvements. The following graph (Figure 6) compares response times between Rational ClearQuest Web version and Rational ClearQuest Web version using WebSphere version 7. The following graph (Figure 7) compares response times between Rational ClearQuest Web version and Rational ClearQuest Web version using WebSphere version The test schema was the ALM schema populated with approximately 350,000 records. Most all measured transactions are faster in the ClearQuest Web version 8.0 release compared to the ClearQuest Web version release. The average transaction response time improvement for these measured transactions averages 20%. Note: IBM encourages customers to use WAS 8 wherever possible. Dated 18 July 2011 Printed 19 July 2012 Page 16 of 24

17 ClearQuest Web Single-User Benchmark, vs Single CQ Web + WAS Instance, ALM Schema SSL enabled, server-side data, untuned WAS Response Time (sec Login Create new request Save new request Find request Generate task workload (2) Find task Open task Modify task (initiate) Modify task (commit) Generate activity workload (4) Save new activity Transaction Open activity Activate activity Complete activity Modify activity (initiate) Modify activity (commit) Run saved query Load from query result Logout Figure 6: Rational ClearQuest Web Single-user Benchmark, vs ClearQuest Web Single-User Benchmark, vs Single CQ Web + WAS Instance, ALM Schema SSL enabled, server-side data, untuned WAS Response Time (sec Login Create new request Save new request Find request Generate task workload (2) Find task Open task Modify task (initiate) Modify task (commit) Generate activity workload (4) Save new activity Transaction Open activity Activate activity Complete activity Modify activity (initiate) Modify activity (commit) Run saved query Load from query result Logout Figure 7: Rational ClearQuest Web Single-user Benchmark, vs Dated 18 July 2011 Printed 19 July 2012 Page 17 of 24

18 4.2 ClearQuest Web Scalability IBM Rational ClearQuest Web version 8.0 scales very well. Scalability testing for Rational ClearQuest Web version 8.0 measured single and multi-user response time. The graph below (Figure 8) compares single and multi-user performance (75 and 150 users) on ClearQuest Web using WebSphere version 7.0. The graph below (Figure 9) compares single and multi-user performance (75,150 and 200 users) on ClearQuest Web using WebSphere version Note: IBM encourages customers to use WAS 8 wherever possible. ClearQuest Web Scalability Benchmark, , 75, 150 users, averaging 25 transactions-per-hour-per-user Single CQ Web + WAS Instance, ALM Schem a SSL enabled, server-side data Response Time (sec.) Login Create new request Save new request Find request Generate task workload (2) Find task Open task Modify task (initiate) Modify task (commit) Generate activity workloa... Save new activity Open activity Activate activity Complete activity Modify activity (initiate) Modify activity (commit) Run saved query Load from query result Logout Transaction 1 user 75 users 150 users Figure 8: IBM Rational ClearQuest Web scalability comparison Dated 18 July 2011 Printed 19 July 2012 Page 18 of 24

19 Response Time (sec.) Login Create new request Save new request Find request Generate task workload (2) Find task ClearQuest Web Scalability Benchmark, , 75, 150, 200 users, averaging 25 transactions-per-hour-per-user Single CQ Web + WAS Instance, ALM Schema SSL enabled, server-side data Open task Modify task (initiate) Modify task (commit) Generate activity workload (4) Save new activity Open activity Transaction Activate activity Complete activity Modify activity (initiate) Modify activity (commit) Run saved query 1 user 75 user 150 user 200 user Load from query result Logout Figure 9: IBM Rational ClearQuest Web scalability comparison Scalability testing also compared IBM Rational ClearQuest Web version with IBM Rational ClearQuest Web version 8.0. For the ALM schema tested in the test environment, the response time of 200 ClearQuest Web version 8.0 users on the same server compared favourably with the average response time of 150 ClearQuest Web version users. In this test environment with this test schema we show that IBM Rational ClearQuest Web version 8.0 can sustain 33% more users than a comparable Rational ClearQuest Web version load and also demonstrate better performance. As with any new release of software, IBM recommends testing and validating Rational ClearQuest Web in your own environment and with your own schema. Dated 18 July 2011 Printed 19 July 2012 Page 19 of 24

20 ClearQuest Web Scalability Comparison vs : 150 users averaging 23transactions-per-hour-per-user : 200 users averaging 25 transactions-per-hour-per-user Single CQ Web + WAS Instance, ALM Schema; SSL enabled, server-side data Response Time (se Login Create new request Save new request Find request Generate task workload (2 Find task Open task Modify task (initiate) Save new activity Modify task (commit) Generate activity workload (4 Open activity Transaction Activate activity Complete activity Modify activity (initiate) Run saved query Modify activity (commit) u u Load from query resul Logout Figure 10: IBM Rational ClearQuest Web Scalability comparison between versions and ClearQuest Web Availability Throughout the 8.0 development cycle, the availability of IBM Rational ClearQuest was continuously tested via the use of 3-day workloads. These workloads were designed to keep the system running at very high loads during which metrics were taken at regular intervals to validate the server s availability. The Rational ClearQuest availability scenarios tested the ClearQuest Web Server architecture on all supported server platforms. A subset of configurations also included integrations points such that leverage OSLC, and additionally where IBM Rational ClearCase is hosted on the same server. The test environment was built using WebSphere version 7.0 to ensure that all of the functionality found in a larger WebSphere deployment would be incorporated. The following goals of the ClearQuest availability scenarios were as follows: Assess the availability of the single ClearQuest Web server under conditions that simulate the maximum number of supported users, a reasonable user activity rate (15 transactions-per-hour-per-user), and long run duration (3 days) Measure and record performance characteristics under load and stress to ensure no major degradation during a 3-day time period System under load remained stable and responsive for extended period (3 days) with no unplanned restarts System under load remained stable and responsive under normal and stress level workloads without availability failures or data corruption The ClearQuest Web Server availability tests completed with no performance degradation on Windows, AIX and Linux. This scenario had 200 active users with each user performing 15 transactions per hour. The ClearQuest repository was populated with 100,000 records. The ClearQuest OSLC availability tests completed with no performance degradation on Windows, AIX and Linux. This scenario had 200 active users with each user performing 16 transactions per hour. The Dated 18 July 2011 Printed 19 July 2012 Page 20 of 24

21 ClearQuest repository was populated with 100,000 records while the Jazz repository had 10,000 work items. Additionally there were 10,000 OSLC links between ClearQuest records and Jazz work items. The ClearQuest Web Server with OSLC availability tests completed with no performance degradation on Windows, AIX and Linux. This scenario had 300 active users with each user performing 13 transactions per hour. The ClearQuest repository was populated with 100,000 records while the Jazz repository had 10,000 work items. Additionally there were 10,000 OSLC links between ClearQuest records and Jazz work items Testing Topologies and Transaction Diagrams Figure 11: ClearQuest Web Server Availability Testing Topology and Testing Transaction Distribution Figure 12: ClearQuest OSLC Availability Testing Transaction Distribution Dated 18 July 2011 Printed 19 July 2012 Page 21 of 24

22 Figure 13: ClearQuest Web Server with OSLC Availability Testing Topology and Transaction Distribution Dated 18 July 2011 Printed 19 July 2012 Page 22 of 24

23 5 Customer Feedback The Rational ClearQuest version 8.0 release development cycle involved a high level of customer interaction to create a usable and consumable product. Customers influenced the product through the IBM Rational Design Partner Program (DPP), Beta Program, and additional customer feedback though the Rational Software Innovate Conference, Voice of the Customer Event (VoiCE), and reports from customers to the IBM Rational Support system. 5.1 The Design Partner Program The Design Partner Program (DPP) helps IBM acquire continuous customer feedback that can be leveraged early in the product development cycle, and used to influence the product strategy, direction, and features. Customers from various industries such as Healthcare, Electronics, and Financial Services participate in the program and share their ideas. The goal of continued collaboration with Design Partners is to deliver the right features and improve consumability, while increasing customer satisfaction and loyalty. The DPP is also interested in vetting product strategy, new cutting-edge development, and ensuring our future roadmaps align with the overall business needs of our customers. As a result of the Design Partner Program, IBM is able to create products that customers are excited about and committed to, because these customers exerted considerable influence on the development plans. Select design partners participated in the Beta Program for Rational ClearQuest version 8.0, some deploying on large distributed test environments, and others in small and medium test environments. Design Partners participated in activities such as feature presentations and feedback discussions, individual customer interviews, surveys, and usability testing on Rational ClearQuest version Beta Program Rational conducted a two phase Beta program for this release. Over 20 customers from around the world downloaded and evaluated early versions of the code. Interactions with beta customers included: Weekly meetings with customers and engineers to educate customers on the features of the product and to gather feedback on the customers experience with the beta code. Quality Surveys to gather feedback on specific strengths and weaknesses of the beta. Engineering was able to make adjustments to the final release based on this customer feedback. Customers of the Rational ClearQuest beta took particular interest in the new Reporting and infrastructure changes, WAS and Installation Manager features. The Rational Beta Program provides customers with the following benefits: Early access to beta code Free education Free support Early development of skills Direct input to Rational Development and Product management to influence the development of the software Early ability to assess how the new release can benefit their organization Dated 18 July 2011 Printed 19 July 2012 Page 23 of 24

24 The Beta Program provides Rational with the following benefits: Validation of product acceptability (functionality, usability) with key customers to provide input for this and future releases Validation of General Availability (GA) readiness and support capability Direct customer input to development, marketing, and sales References and testimonials from early users of the product Ability to provide selected IBM customers and Business Partners with the opportunity to gain early experience with the product 5.3 Future Feedback Customers interested in learning more about the Rational Design Partner Program, future Beta Evaluation programs, usability testing or Rational Software Innovate Conferences should contact their local sales representative. Dated 18 July 2011 Printed 19 July 2012 Page 24 of 24