Nicolai M. Josuttis IT-communication.com 09/08 1 2 Independent consultant continuously learning since 1962 Nicolai Josuttis Systems Architect, Technical Manager finance, manufacturing, automobile, telecommunication SOA experience for multiple years incl. bringing SOA in production with >50 international systems >500 services >10 million service calls per day Author of SOA in Practice http://www.soa-in-practice.com
3 Table of Contents Why SOA? ESB in Practice Reuse in Practice BPM in Practice Conway s Law Summary Why SOA? 4
5 The Context of the 21 st Century Globalization We scale more and more Business is highly distributed Harmonization is not an option We are on the road towards ULS (ultra-large-scale) systems Internet Health care systems Traffic and transportation... 6 Characteristics of Large Systems Characteristics of large systems: Decentralization Heterogeneous, inconsistent, and changing elements Continuous evolution and deployment Redundancies Inherently conflicting, evolving, and diverse requirements Normal failures Erosion of the people/system boundary These characteristics may appear in today s systems can become critical in system landscapes dominate in ultra-large-scale systems SOA
7 Distributed Systems to Deal with Millions of Customers Thin Client Smart Client (Call Center Desktop) Mobile Phone / Fax Frontends Portal Processes Order Fulfillment Backends CRM Billing Logistics 8 Sign Contract with Mobile Phone Distributed Systems to Deal with Millions of Customers Thin Client Smart Client (Call Center Desktop) Mobile Phone / Fax Frontends Portal Process Order Processes Order Fulfillment AddContract( ) SendBill( ) ShipMobilePhone( ) Backends CRM Billing Logistics
9 Distributed Systems to Deal with Millions of Customers Thin Client Frontends Portal Wireless Business Unit Smart Client (Call Center Desktop) Mobile Phone / Fax Processes Order Fulfillment Backends CRM Billing Logistics 10 Distributed Processes Over Multiple Companies Wired Business Unit Wireless Business Unit Billing CRM Insurance Company Property CRM Life
11 Distributed Processes Over Multiple Companies We moved houses Our new Address is: Wired Business Unit Wireless Business Unit Billing CRM Insurance Company UpdateAddress( ) UpdateAddress( ) Property CRM Life 12 Reality: Systems in Multiple Countries, BU s, Companies Austria International USA UPS FedEx ESB B2B Gateway ESB Vendor Wired Germany UK
13 Dealing with Business Requirements findparcelstatus() Austria International USA UPS FedEx ESB B2B Gateway ESB Where is my phone? Vendor Wired checkphonestatus() findshippingdata() Germany findprocess() findcustomer() UK 14 New World of IT System Development Maintenance of System Landscapes
15 SOA SOA is a concept for the maintenance of system landscapes It implies Change Management of multiple business requirements for heterogeneous systems with different product owners ESB in Practice 16
17 ESB Approaches individual integration of interfaces: just a common protocol: contracts with value-added services: Consumer Consumer Consumer Consumer Consumer Consumer Contract Contract Provider Provider Provider possible EAI approach native Web Services??? 18 Web Services ESB s Might Internally Use Other Middleware Consumer 1 Consumer 2 send requests to: 192.168.1.01 SOAP send requests to: 192.168.1.02 SOAP deployed at: 192.168.1.01 deployed at: 192.168.1.02 Interceptor Interceptor App-Server MQ, JMS, JBI, App-Server Interceptor Interceptor SOAP SOAP deployed at: 192.168.1.51 deployed at: 192.168.1.52 Provider A Provider B
19 Service Development Lifecycle Rose Repository HTML Consumer1 Consumer2 Magic Draw Repository HTML Intermediate format: Signature SLA s URL for documentation Central Registry abstract WSDL full WSDLs service call service call Deployment Information abstract WSDL internal Provider design time runtime 20 Service Development Lifecycle HTML Rose Consumer1 Consumer2 Repository Magic Draw Repository external WSDL HTML design time Intermediate format: Signature SLA s URL for documentation Deployment Information external Provider Central Registry abstract WSDL full WSDLs abstract WSDL mapping service call service call internal Provider runtime
Reuse in Practice 21 22 Consumer Specific Services Loading all customer data at once took too long: Consumer Provider getall()
23 Consumer Specific Services Loading all customer data at once took too long Consumer Provider getall() So, split the service into two specific calls: Consumer getpart1() Provider Cache 24 Consumer Specific Services Loading all customer data at once took too long: Consumer Provider getall() So, split the service into two specific calls: Consumer getpart1() Provider getpart2() Cache
25 You will find: Reused services Redundant services Custom-made services Unused services The Performance Reusability Tradeoff 5 4 Average number of consumers per service 3 2 1 1 2 3 4 5 Years 26 Service Monitoring by a Suisse Bank ~900 services used by > 150 applications > 42 % services reused On average: 1.77 clients/service (3.7 clients/interface) > 12,000 Person-Days saved by reuse No new application that don't use at least 1 service 200% 180% 160% 140% 120% 100% 80% 60% 40% 20% 0% 1 2 3 4 5 6 7 8 9 4 3.5 3 Percent of Svcs reused 2.5 2 1.5 1 0.5 0 Avg No of clients per Svc (only new) Avg No of clients per interface slide by: Hermann Schlamann @JAX2007
Business Process Modeling in Practice 27 28 customer order customer calls in identify customer record order create product ship product send bill Business Process View sub process sub process sub process BPM Tools: Process Service Composed Composed Composed Composed Service Service Service Service Service View CRM Order & Logistics Billing
29 customer order customer calls in identify customer record order create product ship product send bill Business Process View sub process sub process sub process BPM Tools: Process Service Composed Composed Composed Composed Service Service Service Service Service View CRM Order & Logistics Billing 30 customer order Standard: BPEL For orchestration of services customer BPEL-Engines provide identify runtime record support create ship send - Business Process Assembler calls in customer order product product bill But there are limits: it - plug-and-play needs services - different levels of abstraction sub sub sub - error handling process process process - no orchestration of SLAs BPM- BPEL is not for frontend workflows Process Tools: - BPEL is not a generic state machine Service Business Process View Service View Composed Composed Composed Composed Service Service Service Service CRM Order & Logistics Billing
31 Business Process Modeling Notations UML Unified Modeling Language Use Case Diagram Activity Diagram... BPMN Business Process Modeling Notation EPC Event-driven Process Chain 32 Business Notations versus BPEL UML: BPEL: BPMN:??? EPC:
33 collect data and fulfill Standard Approach for Process Services 1. Collect data to start the process workflow with human interaction steps can be composed services 2. Fulfill process (usually) no human interaction can be a process service start process Frontend Backend possible response 34 Process Services and Human Interaction backend waits for interaction White paper: BPEL4people Proprietary BPEL extensions Upcoming standards: WS-HumanTask (WS-HT) WS-BPEL4People (WS-B4P) start process Frontend 1. poll request 2. push response queue 1. push request 2. poll response UI Backend
35 Thin Client Smart Client (Call Center Desktop) Back Office Frontend Portal & validate execute Process Compose Backend domain domain domain 36 Thin Client Smart Client (Call Center Desktop) Back Office Frontend Portal Process Compose Backend domain domain domain
Conway s Law 37 38 SOA and the Organization Systems: A System Landscape: CRM Client Inventory Client Production Client Billing Client CRM Inventory Production Billing based on: Eberhard Wolff, Interface21 @JAX2006
39 Conway s Law Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure. Melvin Conway, 1968 in How Do Committees Invent? http://www.melconway.com/law/index.html 40 SOA and the Departments New organizational structures channel specific frontends process oriented backends domain specific backends deal with new requirements solution managers handle projects
41 Development Process Sales &Marketing, Controlling, Solution Management Requirements High-level Design System: Product Owner A&D Development Business Case Release Plans Test Management Development (weekly iterations) 42 Distribution Costs Many people involved Solution Manager System Analysts System Designers System Implementers System Testers Solution Integrators Solution Testers Delivery Managers Incident Managers
Summary 43 44 Lessons Learned Distribution costs Distributed design (multiple parties involved) Needs organizational culture (collaboration, trust) and structures Distributed testing Might turn out to be the biggest day-to-day problem Distributed debugging Your infrastructure/esb is your debugger Distributed quality Overall quality depends on individual qualities Loose Coupling costs Any form of indirection and heterogeneity has a price You can t buy SOA Tools can only help Main tasks are Architecture, Processes, Culture
45 SOA Essence If you can avoid distribution, avoid it SOA should never be an end in itself Use SOA only, if distribution is required If monoliths are impossible or too expensive If independent companies or business units are involved SOA is a strategy Establish step-by-step Needs top management support 46 Q&A Nicolai Josuttis www.it-communication.com josuttis@it-communication.com Gaussstr. 29 D - 38106 Braunschweig Tel.: +49 531 / 129 88 86 +49 700 / 5678 8888 +49 700 / JOSUTTIS