Performance Engineering to Enhance Efficacy of Business Operations

Transcription

1 Performance Engineering to Enhance Efficacy of Business Operations Vipul Kamat, TCS 1

2 Contents Introduction Overview: Business Process Operations IT System Landscape Business Operations Context Unique BPO Ecosystem and Impact of System Performance Observations on BPO Floor Approach to Enhance System Performance Performance Assessment Network, Server, Browser side assessments Other Areas to Improve System Performance Break Processes Intelligently Leverage UX-Analysis Case Study 2

3 Overview: Business Process Operations Overview: BPO Details - BPO Services Data Services Voice Services BPO Knowledge Services Voice Services Customer Contact Centre Collection Services Data & Knowledge Services Account Opening Credit Card Account Loan Account Trade Finance LC Issuance Doc. Scrutiny Cash Management Cash Collections & Recon Payables Management 3

4 IT System Landscape: Business Operations Context Typical BPO Landscape: Logical View Performance Areas BPO User View IT System Performance Considers time spent processing within Application server as well as Database Network Performance Considers Network performance between BPO Delivery center and Customer Data center where IT systems are hosted User Desktop/Browser Performance Considers performance from User point of view i.e. up to the point where application screen is fully rendered on BPO users screen to enable them to start processing 4

5 Unique BPO Ecosystem and Impact on Performance Factors Impacting IT System Performance Data Packet Size Higher the packet size, longer it takes to transmit the data Number of Concurrent Users Higher the number of users, lower are the spare resources like memory, CPU, network bandwidth available for individual user requests Distance/Hops Higher the distance/hops, higher is the latency Typical Observations in BPO Data Entry Heavy Processes Data heavy processing involves large data packet size Repetitive work by large number of associates High nos. of concurrent users High nos. of transactions executed during periods preceding specific periods like month-end, quarter/year end Process Delivery from Remote Geographical Locations Large Distance/Higher Hops Factors Impacting Performance Uniquely Combine & Occur in BPO Ecosystem 5

6 Observations on BPO Floor System Latency Across Trade Finance Ops floor, hundreds of associates access web based application through VDI Desktops Thousands of Processing Minutes Lost Non Productive time spent by associates waiting for application screen to refresh and resume processing Sub-optimal User Experience Adverse Impact on Accuracy linked KPIs Increased Fatigue System Peak Performance Across Cash Management Ops floor, hundreds of associates access web based application through VDI Desktops Critical Transactions can Remain Unprocessed Month-end or Financial Year-end volumes are high Heavy Volumes result in incomplete transactions or transactions missing their TAT causing financial liability Heavy Volumes coupled with deadline to meet can result in anxiety and erroneous processing Ensuring Optimal System Performance is Critical for Efficient Operations 6

7 Approach to Enhance System Performance..I Conduct Performance Assessments Goal of Performance Assessment is to Validate how IT Systems are aligned to handle high user base from remote locations Validate how shared IT infrastructure, say message gateways, can handle seasonal loads like month end or year end volume spikes Outcome Get Recommendations to be implemented to manage system performance OR Set up workarounds or fall back mechanisms in case changes suggested cannot be implemented in requisite timeframe Network v/s Application Performance Focus Analysis First on Network Performance before analyzing Application Performance IT Application performance baseline data would already be available and Production Deployment environment in which Application is deployed is the only change impacting performance (assuming other factors are same) 7

8 Approach to Enhance System Performance..II Analyze Performance Identify Bottlenecks linked to Network Server Network and Server Browser ScrutiNet is a proprietary network sniffer based application profiling tool to help identifying if the bottleneck is in the client, network or application layer. Jensor is a light-weight, low-overhead Java Profiler. PerfExt is a proprietary performance extrapolation tool that uses modelling techniques to extrapolate application performance for a large number of users. Dynatrace is a commercial tool that provides performance monitoring and diagnostics options into JavaScript execution, DOM access, rendering activities for analysing and troubleshooting web applications. Performance Bottleneck Identification Process 8

9 Approach to Enhance System Performance..III Performance Assessments Network Identifying Network Issue instead of Application Issue is the starting point to initiate in-depth network side analysis and monitoring of network infrastructure Horizontal & Vertical Analysis of Network Horizontal Analysis: Tool should be able to view end-to-end network and enable path analysis up to the end users device Vertical Analysis: Tool should be able to give insight within the layers of network Extend Analysis beyond own network and study traffic to and fro from shared resources/servers Model and Simulate Network layout Performance Assessments Server Application Transaction Profiling Drill down at application tier level, that is, Web layer, Business layer or Database layer Transaction response time break up to be analyzed Output: Identification of candidates for performance tuning 9

10 Approach to Enhance System Performance..IV Performance Assessment - Server Tool like ScrutiNet can be used for these in combination with tools like Jensor that can be used for profiling java code to analyse: Method Response Time Object Count Heap Utilization Performance Assessment - Database Database side bottlenecks are relatively easy to address and typically involves SQL analysis Involves solution around: Optimizing query plans or query cost Introducing and usage of table indexes Partitioning of tables and data archival strategies 10

11 Approach to Enhance System Performance..V Steps Involved resolving DNS names, setting up TCP connections, transmitting HTTP requests, downloading resources from server, fetching resources from cache, parsing and executing scripts, and rendering objects on the page Performance Assessment - Browser Last Leg of Bottleneck Measured From User s View Difference in time between page request and page fully rendered on browser screen for next action 11

12 Approach to Enhance System Performance..VI Performance Assessment - Browser Analysis of following parameters is essential First Impression Time Time when user gets the first visual of the page/ screen On Load Time Time when OnLoad event is triggered Java script code begins execution to build the page or enhance the page Total Load Time Time when all content, downloads and execution triggered by OnLoad are completed Page Weight The total page weight would measure number of requests and bytes transferred to generate a given page Cache-Request/ Response Headers HTTP Compression Probable Tools Shortlist and Analyze Heavily used screens with tools like Fiddler Dynatrace 12

13 Approach to Enhance System Performance..VII System Performance Other Areas Break Processes Intelligently Typically a business decision Incorrect Decision would Impact Business Operations Performance Insights can help drive the outcome Typical Recommendations Co-locate related/linked processes Conduct BPM-based process modelling and simulations to identify bottleneck areas and identify weakest link from system performance standpoint Conduct UX-assessment to enhance User Experience Use Performance Extrapolation and Enable Proactive Provisioning Month-end, Year-end workload Increase BPO Associates Increase System Resources Private, Public or Hybrid cloud adoption as feasible Adopt Optimized VDI Technologies Adopt VDI Technologies with architecture that reduce latency between virtualized desktop and shared storage 13

14 Case Study Background and Context Background and Context The Trade Finance system used by large multinational bank across the globe Web based system leveraging JEE and n-tier architecture Hundreds of concurrent users across the globe Challenges High wait time for associates during transaction processing Month-end, Quarter-end volume spikes lead to SLA breach Financial Impact Loss of Reputation Sub-Optimal User Experience Increased Chances of Error 14

15 Objective Case Study Objective and Scope To enhance the End-User experience in terms of Trade Finance system response time, conduct exercise to collect data for performance analysis to recommend next steps towards remedial action. Sample size & Coverage Area: Business Regions, Delivery locations, Process areas The Trade Finance system used by large multinational bank across the globe and the performance engineering assessment intended to cover the Business Regions, Delivery Locations and Process Areas as depicted below Major Business Regions North America EMEA APAC Delivery Locations Mumbai Chennai Process Areas Domestic Trade International Trade Coverage of Trade Finance System 15

16 Case Study Performance Assessment Plan Sr. No. Tasks TCS IT TCS Ops Customer Team *Week 1 & 2 Week 3 Week 4 Week Completion of Pre-requisites for arriving at Scope, Data points to be captured and feasibility of tool for analysis Fine tune context setting (post Pre-Work) and sharing of Objective to be achieved Confirmed Acceptance from Customer on Scope, Objectives, data points planned for collection for Analysis and Reporting Deployment of tool for collecting data points as per agreed sample size Perform Assist Perform Perform Assist Assist Perform Assist Perform 5 Configuration of tool for collecting required data points Assist Perform Dummy run of tool and initial assessment of data collected to check suitability of tool configuration Perform Assist Perform 6 TCS to ensure optimal utilization of the machines in-scope for analysis, for the period during assessment Perform 7 Enable collection of data points during assessment period Assist Perform 8 9 Facilitate provisioning of performance related logs collected to designated consultants for detailed analysis and mapping of screens to URLs being analyzed Perform analysis and provide reports and further action - Draft followed by Final Recommendations Assist Perform Perform Assist 16

17 Case Study Execution & Sample Recommendations Execution Scope Frequently Used Steps Monitor Performance & Collect Data Shortlist Problem Areas Deep Dive to Locate Problems Recommend Solutions Representative Analysis Summary: Typical Data Points Analysis Recommendations Page 1 - Response Time Page 1 has heavy Java script Split the Java script load, Achieve functionality using AJAX Page 2 - Render Time Page 2 has multiple images Create alternate images, explore usage of Image Sprites Page 3 Render Time Page 2 has multiple tabs Load Primary window data as per UX Recommendations 17

18 Case Study Outcome Detailed Time and Motion Analysis Performed Process Level Data across process teams and regions captured for different products like Export Advising (EXA), Export Confirmation (EXC), Reimbursements (RMB), Reinsurance (RIN), Standby Issuance (SBD), Export Payments (EXP) and Customer Service (CSV) Savings Due to Higher No of Txns Processed per month: USD ~ 750,000 Region Western Europe North America Process Before (minutes) After (minutes) Volumes Al l-m Al l-c CSV EXA,EXC,RIN RMB SBD EXP EXC LATAM Al l-m

19 Thank You 19