Web Services and Application Integration Conference Los-Angeles May 5-7, 2003

Transcription

1 From Chaos To Order - Delivering e-business Integration Solutions Web Services and Application Integration Conference Los-Angeles May 5-7, 2003 Business Integration in Action - Lessons learned from the growing convergence of application development and application integration Max Dolgicer, CTO mdolgicer@isg-inc.com International Systems Group (ISG), Inc International Systems Group (ISG), Inc. 1

2 Agenda Development Projects vs. Integration Projects The Traditional Divide Development for Integration Built to integrate Web Services an Implementation Example for SODA Integration Through Development Composite Applications Where Current Solutions Do Well and Where They Fall Short Case Studies Convergence Outlook 2004 Development or Integration? 2003 International Systems Group (ISG), Inc. 2

3 Development Projects vs. Integration Projects Traditionally, IT projects used to be segregated between development and integration The traditional divide Different project focus Different business objectives and drivers Different technology base for existing systems vs. new applications Different solutions provided by vendors Different skill sets required Frequently, even different budgets 2003 International Systems Group (ISG), Inc. 3

4 Project Category: EAI (A2A and B2B) Project focus: Enterprise Application Integration (EAI) Within the enterprise (A2A) Across trading partners (B2B) Large-scale integration of legacy systems and packaged applications Integration-centric approach Integration of existing systems is the overriding concern Development of new business logic can be important, but is a secondary concern 2003 International Systems Group (ISG), Inc. 4

5 Project Category: EAI (A2A and B2B) There is no such thing as typical integration project Many projects have requirements for AI (Application Integration), without enterprise scope A wide variety of approaches are employed Point to point vs. hub and spoke Batch vs. near real-time vs. real-time Message-based vs. event/notification driven systems Message Brokers were initially positioned for large-scale projects ( true EAI) Requires careful ROI analysis Most projects have not gone beyond integrating 3-6 applications The IT organization may need to change in order to exploit the technology Technology capabilities may create business opportunities but who is to drive it? 2003 International Systems Group (ISG), Inc. 5

6 Project Category: Application Development Project focus: development of new componentbased e-business applications Development-centric approach Development of new business logic is the overriding concern Integration with existing systems is usually important, but is a secondary concern J2EE Application Servers and.net have become technology of choice 2003 International Systems Group (ISG), Inc. 6

7 Development For Integration Old thinking: Integration is an afterthought (is addressed after applications have been developed) New thinking: Integration requirements become part of the application development process Applications are instrumented for (later) integration The actual integration can be implemented through Variety approaches including Business Process Management (BPM) Development of Composite Applications Service Oriented Development Of Applications (SODA) 2003 International Systems Group (ISG), Inc. 7

8 Integration Through Development The Next Wave: Development of Composite (i.e. Virtual) applications Composite Applications may combine building blocks such as: Existing assets (custom and packaged applications) Reusability through encapsulation New component-based applications and Web Services Combining building blocks can be viewed as a typical application development task New business logic needs to be implemented Interaction between the building blocks needs to be implemented (i.e. new business logic) 2003 International Systems Group (ISG), Inc. 8

9 Positioning of Composite Applications Integration focus projects (coarse-grain granularity) EAI tools for large scale application integration Adapters, data transformation, BPM EAI Composite Applications High level tools for component development and assembly Composition from new and existing business functions and services Components Low level tools for component construction Wrapping of existing business logic Development of new components Development focus projects (amount of new business logic) 2003 International Systems Group (ISG), Inc. 9

10 Example: Travel Reservation Application Travel Portal Airline - Mainframe Hotel J2EE Components Message Broker Functionality State management Synchronous Asynchronous Event correlation Composite Application Rental Car Web Service Exception handling 2003 International Systems Group (ISG), Inc. 10

11 Message Broker Challenges Focus on integration, not the development and deployment of new business components No notion of a standard container that manages components Focus on integration glue, do not apply proven concepts and rigor of application development Conceptually similar to customizable, pre-built applications Are mostly not built on a standard platform (i.e. J2EE) Some vendors have already embraced standards, but technology remain to a large degree proprietary Results in shortage of skilled personnel Require relatively high up-front cost Difficult to achieve ROI Message Broker vendors have not been able to scale their sales model yet 2003 International Systems Group (ISG), Inc. 11

12 Java Tools And Application Server Challenges Provide good infrastructure foundation for deployment of Composite Applications Scalability, reliability, availability Services like transactions, security, etc Focus on the development and deployment of new business components, not on integration Low level tools for component construction JBuilder, TogetherJ, etc Integration capabilities are evolving BUT: Point solutions (e.g. adapters to packaged software) Loose integration with Message Brokers Built-in Message Broker, e.g WebLogic Integration (WLI) Building Composite Applications requires coding to many lowlevel interfaces Similar perspective on.net 2003 International Systems Group (ISG), Inc. 12

13 The Technology and Skills Gap Efficient technology for building Composite Applications is not widely available Extensive engineering effort is required Requires teams skilled in different disciplines Development vs. integration, multitude of interfaces Cost effective and timely solution delivery is limited by the number of skilled developers EAI: 25,000-35,000 J2EE: 350, ,000 General: 9,000, International Systems Group (ISG), Inc. 13

14 Case Study 1 Integration + BPM 2003 International Systems Group (ISG), Inc. 14

15 Processing Of Patent Applications Server 1 (UNIX) Server 2 (UNIX) Log file EFW application RAM application Log file EAI product LAN EAI product Log file EFS application Oracle DB-A DB-B DB-C XML files Development Windows NT/ International Systems Group (ISG), Inc. 15

16 Some of the Processing Highlights Collate complete collection of patent application files Parse files against DTDs and Schemas Process elements in XML files Check for existence Validate Truncate, remove, replace data Extract XML data from files and convert to relational structure Extract XML data from files and convert to Java structure 2003 International Systems Group (ISG), Inc. 16

17 Collating & Processing XML Files 2003 International Systems Group (ISG), Inc. 17

18 Sample XML Structure <inventor-block label="[inventor Block] "> <first-named-inventor label="[first Named Inventor] "> <name label="[name:] "> <name-prefix label="[name Prefix:] "> Mr </name-prefix> <given-name label="[given Name:] "> Spencer </given-name> <middle-name label="[middle Name:] "> T </middle-name> <family-name label="[family Name:] "> Bessette </family-name> </name> 2003 International Systems Group (ISG), Inc. 18

19 Manipulating XML Data 2003 International Systems Group (ISG), Inc. 19

20 Case Study 2 Development + Point Integration 2003 International Systems Group (ISG), Inc. 20

21 Mutual Funds Company External Data Feed Employee Web Server Entrust PKI Authentication Business Communication Layer Presentation Layer Business Components Layer User Session Management Dealer Fax Shareholder Business Request Router Account Funds Middleware Components Layer Name Service Monitoring Security Label Printing Faxing Out Other Functions Data Validation Fax Functionality Encapsulation Data Abstraction Layer Logging Presentation Label Printing Encapsulation Auditing CICS/ADABAS RDBMS File Net Imaging 2003 International Systems Group (ISG), Inc. 21

22 Development + Point Integration Project focus on development of new business applications Strong emphasis on components A limited number of integration points are required: CICS, Filenet, Fax and External data feeds Integration points implemented as custom built adapters (or wrappers ) 2003 International Systems Group (ISG), Inc. 22

23 Case Study 3 Development + Integration 2003 International Systems Group (ISG), Inc. 23

24 Development and Integration are equal citizens Driven by the desire to have one common architecture for the development and deployment of new e-business applications as well as for application integration Application development and application integration are equally important Technology choices are not obvious 2003 International Systems Group (ISG), Inc. 24

25 Major Investment Bank Portfolio of Fixed Income Applications Development of reusable middleware framework Must be capable to support development of new applications Must serve as Enterprise Integration Platform (EAI) for 23+ existing applications diverse platforms, APIs, connectivity options Central information repository The BIG consolidation and streamlining of business processes diverse mix of applications populate repository and receive notifications of data changes publish/subscribe model 2003 International Systems Group (ISG), Inc. 25

26 Durable XML Enabled EAI Architecture Bloomberg Date Server Bloomberg Application Bolt Application Pricing Application Date Adapter Sec. Master Application Service Base Service Base Service Base Service Base Service Base Data Transform Engine SMF Monitor/Activator SMF Central Logger/Auditor Naming/Configuration LDAP System Management Framework TIBCO Rendezvous MQSeries OrbixTalk RMI Abstract Transport Layer Devon Adapter Tandem Adapter Data Abstraction Data Abstraction Data Abstraction Service Base Service Base Service Base Service Base Service Base Devon Tandem RAW BIG NY BIG NJ 2003 International Systems Group (ISG), Inc. 26

27 Reasons for Using XML Defines canonical data format across the enterprise E.g. standardization on common trade message Defined as superset of proprietary trade messages Many applications (30+) have their own specific flavors of a trade message Application specific flavors are derived from common message 2003 International Systems Group (ISG), Inc. 27

28 Reasons for Using XML Configuration information used for the infrastructure (i.e. middleware) is stored in LDAP repository in XML form Validation schemes and data definitions (DTD or XML schema) can be shared across the enterprise 2003 International Systems Group (ISG), Inc. 28

29 Service Oriented Architecture Canonical/XML data is published across the enterprise via the middleware infrastructure Existing (legacy) as well as new applications are encapsulated as services Integration Adapters live at the boundary between legacy applications and the middleware infrastructure Convert between proprietary and canonical form New development During development of new applications a service base is utilized Provides automatic instrumentation of application for integration with the SOA 2003 International Systems Group (ISG), Inc. 29

30 Databases Become Part Of The SOA DTD schema/canonical form is designed to mirror the database schema Allows implementation to be generic, i.e. the infrastructure components that maintain the database need not know the format of the data ahead of time No direct database calls embedded in applications Databases become services 2003 International Systems Group (ISG), Inc. 30

31 Convergence Scenario 2004 Development Integration Application Servers Message Brokers Web Services Pure-play platforms Development+ Integration 2003 International Systems Group (ISG), Inc. 31

32 Two Sides Of The Same Coin Integration through Development Componentbased development Servicebased interface New Components SO A Servicebased interface Composite Application Development Legacy Applicati on Packaged Applicati on Development for Integration 2003 International Systems Group (ISG), Inc. 32

33 Development meets Integration? Composite Applications Good fit if reasonable amount of new business logic needed to complete the integration solution Contrast to BPM for business event propagation But: high-level development/assembly tools still missing Web Services Most current implementation choice for development of services Where possible/feasible encapsulate back-end applications/business processes as Web Services But : Today Web Services represent only the tip of the integration iceberg Possible Product Approaches: Application Platform Suite (APS) Best-of-Breed Specialists Focused, maybe even tactical but useful (and less expensive) 2003 International Systems Group (ISG), Inc. 33

34 From Chaos To Order - Delivering e-business Integration Solutions 32 Broadway, Suite 414 New York, NY Tel: (212) Fax: (212) isg@isg-inc.com International Systems Group (ISG), Inc. 34