3. NEED FOR ACDT 5. PROCESS FLOW 4. ACDT SOLUTION

Transcription

1 TCS ACDT Automated Code Deployment Tool Code deployment automation solution for faster and accurate deliveries Mahesh Gupta Tata Consultancy Services Solution Architect Mumbai Guru KP Rao Tata Consultancy Services Rational Lead Bangalore Nita Khare Tata Consultancy Services IBM SW Solutions Lead Mumbai ABSTRACT The White Paper describes the challenges & issues faced by manual application deployment process in multitier application environments and solution framework with desired functions and features in any deployment tool for complete automation of deployment. TARGET AUDIENCE Audience of this document is expected to have knowledge of deployment process. List of Abbreviations Used RSA RAD RTC RFT RPT XML 1. INTRODUCTION Rational Software Architect Rational Application Developer Rational Team Concert Rational Functional Tester Rational Performance Tester EXtensible Markup Language Table 1 List of Abbreviations This solution framework mainly helps in faster and accurate deployments. In short, ACDT helps in the deployment process automation and focuses mainly on below: Deployment Components Deployment Components List Source and Target Details Deployment Dependencies Deployment Activities Pre Deployment Post Deployment Standardization & Validation Bulk Import / Export Deployment Schedule Deployment Checklist Workflow Management Managing Deployment Workflows Notifications and Alerts Successful / Failure Deployments. Planned Deployments Reports Deployment Summary Dashboard Product Specific Adaptors Tool based Build/Packaging 2. PAIN AREAS OF DEPLOYMENT PROCESS Almost everyone who has worked in a project experiences the state when the deployment process has failed or was inaccurate. The business users are breathing down the neck for information for their important decision making. This results in On Time delivery failure and also missing of the SLAs result in loss of business and additional cost of rework. The common pain areas of the deployment processes are: Manual processes Incomplete Deployments Version Mismatch No Timely Notifications & Collaborations Missing Approvals Verification and Validation mechanism Absence of Pre & Post Deployment Verifications Diverse Tool / Technology dependent processes Lack of Deployment traceability and Logs Deployment Knowledge Retention Lack of scheduling and defining interdependencies Complex Failed Deployment Recovery Management Copyright 2010 Page 1 of 7

2 3. NEED FOR ACDT Code Deployment is crucial and integral part of release management and SDLC. Many times code deployment across environment is repeated manual process. Many projects handle hundreds of components to be deployed across platform through manual processes which results in error due to issues of missing, mismatch, wrong version etc of code component. And these manual processes are different for different platform/technology & COTS tools. This repeated manual process is mastered over time with some level of automation by the deployment team. But these components are not reusable for other applications & environments as these are not integrated, non-standardized & not transparent to other users. Due to manual intervention the monitoring & reporting of code deployment is another pain area of the manual deployment process. For code deployment governance, there is sheer need of defining work flow & ownership with right stakeholders to approve the complete process from initiation to post deployment housekeeping. One of the pain/challenge of manual deployment is to recover in case of failure during deployment. The failure can be in any part/module/process of deployment and such failure can cause major issues in business continuity. 2. Collaborative: Should support high collaboration among the stake holders like notification & alerts 3. Workflow: Should support organized tasks execution with defined ownership among stakeholders 4. Reporting: Should provide a single and consolidate view of deployment processes 5. Integration with Configuration tools: Integration with the enterprise configuration management tool 6. Reusable: Should support to reuse the mastered deployment processes across applications & environments. 7. Security Management: Environment wise role based security management. And reusability of environment details (parameters, access credential etc) in secured fashion. 5. PROCESS FLOW To summarize, following are key points to express why ACDT is required. Centralized Single tool for deployment Defined process Standardization Deployment governance & reporting Reduced risk of wrong deployment Traceability & Transparency Efficient Failed Deployment Recovery Objective is to have single centralized automated tool for code deployment across the platform and environments within organization. 4. ACDT SOLUTION ACDT is based on modularizing the deployment process and activities. At high level all deployment processes can be categorized in below groups. Code Movement Code Build (Tool /platform specific processing) Database script execution OS Script execution Deployment governance Setup & Configuration (Environment parameters, Security setup etc.) Except last one all of the above processes can be automated. Apart from automation, the desired tool should also support below functionalities to facilitate smooth accurate deployment Every Time & On Time. 1. Modularized: Should allow users to break the deployment work in multiple steps/tasks Diagram 1 ACDT Process Flow There are several ways of automating the entire deployment process. Projects over the period of time try to handle the deployment issue in half hazard manner i.e. by building custom scripts or using some deployment tool available in the market. But, this is not a standard, systematic and complete way. Teams start building custom tool for deployment. The code developed is limited to project / customer or specific product use. Explore tools available in the market and find the best fit for your enterprise. One such tool is RTC Rational Team Concert. In this paper we focus on how RTC tool supports the desired features and functionaries required for ACDT and how we can use for Cognos and Datastage application deployments. Copyright 2010 Page 2 of 7

3 RTC - IBM Rational Team Concert is built on the Jazz Platform, an extensible technology platform that helps teams integrate tasks across the software life cycle. Rational Team Concert is built on client-server architecture and it enables software development teams to track all aspects of their work, such as work items, source control, reporting, and build management, in a single product. Rational Team Concert has an Eclipse-based client interface, a Microsoft Visual Studio client interface, and a Web interface. Additionally it provides since version 4.0 a windows shell integration within the Windows Explorer for source control of files within the Rational Team Concert repository. The client interfaces provide an integrated development environment for developers to build and deliver artifacts. Users can use the Web interface to administer servers and projects, access project areas, browse repository information, update tasks, or read about recent events. Now let us analyze how RTC maps to the ACDT features. 5.1 Modularized Build Management capabilities of RTC support build templates like Ant, Maven, command line etc. Most of the cases Build Engineers using RTC will consider using Ant Script template which uses build.xml for build & deployment. Below we have detailed how we can break the deployment work into multiple targets using build.xml structure. And define interdependency among them. The depends attribute can be included in the target tag into XML to specify that this target requires another target to be executed prior to being executed itself. Multiple targets can be specified and separated with commas. <target name="one" depends="two, three"> Here, target "one" will not be executed until the targets named "two" and "three" are, first. Example of using the depends attribute Here is an example of a build file that executes three targets in order, first, middle and last. Note that the order the targets appear in the build file is unimportant: <project name="test Project" default="dist" basedir="."> <description> simple example build file </description> <!-- set global properties for this build --> <property name="src" location="d:\releng\sample Project"/> <property name="build" location="d:\rtc Build\builddir"/> <property name="dist" location="d:\rtc Build\final"/> <target name="init"> <!-- Create the time stamp --> <tstamp/> <!-- Create the build directory structure used by compile --> <mkdir dir="${build}"/> <target name="compile" depends="init" description="compile the source " > <!-- Compile the java code from ${src} into ${build} --> <javac srcdir="${src}" destdir="${build}"/> <target name="dist" depends="compile" description="generate the distribution" > <!-- Create the distribution directory --> <mkdir dir="${dist}/lib"/> <!-- Put everything in ${build} into the MyProject- ${DSTAMP}.jar file --> <jar jarfile="${dist}/lib/myproject-${dstamp}.jar" basedir="${build}"/> </project> 5.2 Collaborative IBM RTC is a Collaborative project planning tool on Jazz platform, which contains added features of Source Code Management & Build/Deployment Management. Its web based environment allows team members to receive & share updates / notifications on all the team activities getting executed. Each member can receive notifications or alerts on the tasks / actions, provided he is subscribed for the same. Also RTC Web provides rich Dashboards for individual team members. One can go through the available build specific widgets (like Builds, Build Health etc) in the RTC Dashboard & get more information on the number of builds executed, how many are passed / failed etc., and generate reports on build statistics. Dashboards are easy to create and update, and are highly customizable. You can add/remove any viewlet, drag & drop to move them around, organize them under different tabs, and configure each one to show exactly what you want. The appearance of each viewlet is also customizable. You can change titles, trim colors, and even make them trim-less so they stand out. Copyright 2010 Page 3 of 7

4 practices. It's built from the ground up with advanced features to support collaborative development, lifecycle integration, traceability, and control. Teams using RSA / RAD can source control their models, code directly by deploying RTC plugin on top of these tools. Also, Test artifacts like Test cases, Test Scripts too can be controlled by deploying RTC client plugin onto specific testing tools like RFT, RPT etc. In a collaborative environment, a team member can associate his changes with the work items he has been assigned to. Later on, the approver can refer the changes applied during approving/rejecting the respective work item / task. Diagram 2 Customizable Dashboard 5.3 Workflow Using RTC s Task management capabilities, team lead can assign specific tasks to his team members. If the notifications are enabled, the member will be notified about the task assignment & he will start working on the activity. Once complete, he can update the task status & ask the team lead to review, approve his activity. Team lead can then review the results & can then approve / reject the assigned task. Each member of the team can use the available widgets / add custom widgets to get an understanding on the current status of the project (tasks assigned to him/ assigned by him / pending for his approval etc.). Using the rich mouse-hover features, he can directly open the task page with more details of that particular task. A wide array of work item trend reports allows you to view new work items per day, closed work items per day, a comparison of open work items between teams, open work items by type, open vs. closed work items, and others. Team Lead can create a Project plan (Release Backlog/ Spring Backlog/ Product Backlog etc.), create necessary tasks under the plan & assign them to the respective team members. He can monitor the progress of the tasks by viewing the available widgets in his dashboard. We can organize the task execution based on the priorities, by developing custom plugins using RTC Rest APIs. 5.4 Advanced Source Control Management One of the key feature build-in with RTC is the powerful SCM capabilities. Its focus is on supporting geographically distributed teams and has strong support for parallel development, component based development, and agile Although, RTC does not have built-in integration capabilities with tools like Cognos & Data stage, still these associated artifacts can be controlled under RTC s Source control. One can use the RTC Synchronizer Solution developed by Rational COE & synchronize Cognos / Data Stage updated artifacts into RTC workspace & then delivers it to RTC SCM Stream. Similarly new members can download / copy the latest version of the files from RTC Stream into their local workspace & then start work on the latest version of the artifact (by importing them into local Cognos environment). RTC Synchronizer is a GUI developed using eclipse plugin extensions. Diagram 3 Custom built GUI for Code Synchronization Member can also deploy the latest version checked into the RTC stream into n number of server locations using the Deployment option provided in the solution. 5.5 Reporting Build Reports When the build fails, people need to know why it failed and how to fix it. Over time, interested stakeholders would also like to know about the build's record - how often does it fail? How long does it take to fix a broken build? What has caused it to break the most frequently? Is the build taking longer to complete than before? It is possible to answer these questions, and others, using a number of build-related reports, including Build Health, Code Coverage, Code Health, Frequent Test Failures, and Frequent Test Regressions. Copyright 2010 Page 4 of 7

5 2. Subversion integrates directly with Rational Team Concert in these ways: Bridge and an importer. Subversion Bridge lets you connect Jazz Work Items to Subversion Repositories to work items. Development teams who wish to use Subversion as their version control tool but associate change sets in Subversion with work items in Team Concert can use this bridge to enable Team Concert collaboration, dashboards and agile planning with subversion. Importer can import the data from Subversion repository into Jazz SCM. Both Subversion Bridge and an importer are included with all Team Concert Editions. 3. Rational Team Concert integrates with CVS through CVS Importer. The importer imports data from CVS into Jazz SCM. The importer maintains version history. The CVS importer is included with any Team Concert product Edition. Diagram 4 Deployment Status When there is more detailed information available for a particular item, you can hover on the item to get a tooltip showing a little more, zoom in on the viewlet to see more details for all the items, or you can click on the link to see the complete information. This includes work items, plans, builds, reports, feeds, and other key data. E 4. Rational Team Concert integrates with Git through Git Bridge. Development teams who wish to use Git as their version control tool but associate commits in Git with work items in Team Concert can use this integration to enable Team Concert collaboration, dashboards and agile planning with Git. The Git integration uses the Team Concert RESTful API for modifying work items and their attributes and it provides direct linking between Team Concert work items and Git changes. The Git integration is supported with any Team Concert product Edition. 5. Rational Synergy integrates with Rational Team Concert (RTC) using a bridge. Using the Synergy bridge to RTC allows integrating Synergy with RTC so that you utilize the benefits of Work Items, Agile Planning, Collaboration and Dashboard capabilities without using Jazz source control Diagram 5 Deployment Process status 5.6 Integration with Configuration tools Integration with RTC itself has built in SCM features. Still it can integrate with other SCM tools like IBM Clear Case, Sub version, Synergy, Git etc. 5.7 Reusable The ANT scripts, Reports, Dashboard etc. built and developed for one application environment can be easily reused for multiple applications across environments. 1. The Rational ClearCase Bridge provides a traceability link from Rational ClearCase artifacts to work items in Rational Team Concert. This is primarily for a customer who wants to continue to use Rational ClearCase as a source control system, but who also wants to take advantage of other features of Rational Team Concert, such as agile planning, dashboards, and etc Copyright 2010 Page 5 of 7

6 5.8 Security Management In this part we specifically focus on the Authentication mechanism supported by IBM Jazz Server. We can use the ant-contrib <trycatch> task, as detailed in the example below. We can have the scripts to be executed (& which may fail during execution) within the <try> <catch> block & can check for the property value in the <finally> block. Based on the Property value we can recover back to the previous deployment state. Diagram 6 Security Integration There are basically 2 methods of User Authentication into Jazz 1. Form : default method Jazz team server web UI presents HTML form to collect user credentials first time user attempts to access a secured resource Login session is established between the browser and the server; once authenticated, the server only needs to validate the session but does no need to re-authenticate credentials. 2. Basic : method used by LDAP The web browser itself presents a dialog to collect user credentials the first time the user attempts to access a secured resource Credentials are passed on every request from client to the server; on each request the server (Web sphere) validates the credentials or (Tomcat) re-authenticates credentials. 5.9 Failed Recovery Management Recovery from failed deployments is very critical to restore the original state of the solution to ensure business as usual. In this automated deployment framework, entire deployment process has to be coded (Scripted) as module with defined interdependencies. All such modules should have separate recovery module in case of failure. Deployment failure recoveries are of two types: Module/Process Failed Recovery: Developer has to code the recovery script in such a way so as to completely recover the module irrespective of the point of failure within module. For example: <target name="taskfailcheck"> <trycatch property="taskfailed" reference="errorref"> <try> <antcall target="normaltarget" /> </try> <catch> <echo>error caught: ${errorref}</echo> </catch> <finally> <echo>check if we failed and perform task otherwise perform other task.</echo> <if> <isset property="taskfailed" /> <then> <antcall target="failtarget" /> </then> <!-- otherwise perform succesful task --> <else> <antcall target="successtarget" /> </else> </if> </finally> </trycatch> Replace the normaltarget, failtarget, and successtarget with actual targets; We could also just check the taskfailed property in the other targets themselves, for example: <target name="failedtarget" if="taskfailed"> then we would have a target setup like this to setup the order: <target name="mastertarget" depends="taskfailcheck, successtarget, failedtarget"/> This task would run the trycatch then it would run either the successtarget or the failedtarget depending on the success and failure of the normaltarget target. Within Module/Process failed or Partial module Recovery: Partial or in between process failure recovery is complicated but can be achieved by capturing the control data during process or module execution at each logical Copyright 2010 Page 6 of 7

7 commit of the changes. The implementation team should weight the cost and affinity of such incidences to decide if they need to implement partial process fail recovery. Considering our scenario using RTC as a build management tool with Ant scripts, we could use an additional 'Master' ant script. The master script (a new one) includes all public targets from the original one and delegates the work to the corresponding task in your build script (ant calls) 6.1 Cost Effort Analysis Mentioned below are the Effort related factors that can be considered while performing the Cost Benefit of the solution. The weights can be assigned appropriately as per the requirements. If the delegate fails, the master should be able to recognize the failure and could call the 'clean-up' task (either on the 'master' or on the original build file) We can make use of the <try><catch> block as explained in the Process Failed Recovery section in this case as well. But we will be having sub blocks for each of the tasks that need to be executed, with a recovery plan for individual tasks (rather than for all together, as in the earlier case). 6. BENEFITS Mentioned below are the benefits of using ACDT solution. Ease of Deployment - Reduced deployment effort (30% reduction in the deployment effort). Gain will increase as the reusability increases Less Time to deploy Reduced deployment time (30% reduction in the deployment time). Gain will increase over time. Governance Transparency in the entire deployment process, automates building deployment process knowledge base. Automated approval management Risk Minimization pertaining to loss of knowledge and human errors. Automated Dashboard based Monitoring Robust, Flexible and Scalable automation process which is in line with industry best practices. Reduced risk of non-compliance. Technical effectiveness and efficiency of the deployment solution. Control and standardization. Effective code management. Diagram 7 Cost Effort Analysis Here are few of the other Factors that can be considered while deciding to go for Automated Deployment. Applications Count Application environment complexity Support team size Nos. of users Frequency of deployment Stability of applications Nos. of modules / components Type of Architecture 7. SUMMARY ACDT solution framework is applicable and qualified for all products where deployment is a main concern. 8. Useful Links Copyright 2010 Page 7 of 7