(a) Introduction Chapter 3. The Integration as a Service Paradigm - Computing eras * Mainframe: one centralized large system provided services to thousands of users one-tomany * Personal computer (PC): each user has his/her own PC one-to-one * Smart devices: each user has his/her own smart devices (nomadic, wearable, portable, implantable) many-to-one * Cloud computing: everything is in the cloud * Ambient intelligence (AmI): ambient sensing, networking, perception, decision-making technologies - New models brought by cloud computing * Deployment: over the cloud * Delivery and consumption: on demand * Pricing: pay per use * Quality of service: availability, high performance, on-demand scalability/elasticity, affordability, global-scale accessibility and usability, energy efficiency etc. - Status of the cloud * Cloud service providers (CSP) are experimenting new cool things 3-1
# Infrastructure as a service (IaaS), platform as a service (PaaS), software as a service (SaaS), security as a service (SaaS), storage as a service (STaaS), network as a service (Naas), database as a service (DBaaS), communication as a service (Caas), monitoring as a service (MaaS), cloud migration as a service (C-Maas), IT as a service (ITaaS), * Enterprises are moving their business services and applications to the cloud # Currently, enterprise data and applications are linked via standard-compliant integration platforms within the corporate intranet # Enterprises are going global # Moving to and integration with the cloud: ERP, B2B, B2C, - An enterprise s cloud integration * Enterprise-to-enterprise (E2E) * Enterprise-to-cloud (E2C) * Cloud-to-cloud (C2C) (b) The challenges of SaaS - Challenges of SaaS * Integration problem # SaaS applications are point solutions and service one line of business # Problem: data synchronization between multiple lines of business * APIs are insufficient # Accessing and managing data via API requires a significant amount of coding and 3-2
maintenance due to frequent API updates # There is little or no standardization on SaaS APIs: IT department expends time dealing with the APIs of each SaaS applications * Data transmission security # Data are secure within the hosted environment # Data transfer between on-premise systems and SaaS applications may not be secure # Require integration efforts for the diverse, distributed, and decentralized data # Data integrity, confidentiality, quality, and value have to be preserved (c) Integration as a service - Integration as a service (IaaS) * Migration of the functionality of enterprise applications into the cloud and provide smooth data transport between enterprise and SaaS applications * Data may be duplicated between on-premise and off-premise applications: need to be completely and compactly synchronized * Dynamic SaaS APIs worsen the integration problem * Limited access to the cloud presents another problem # Accessing local applications is much easier than clouds # Set local integration points are easy in local applications, but difficult in SaaS applications # E.g., SalseForce.com s API does not support transactions against multiple records Integration code has to handle that logic * Service integration 3-3
# Enterprise application integration (EAI): use of software and computer systems architectural principles to integrate a set of enterprise computer applications # Enterprise service bus (ESB): a software architecture model for interaction and communication between applications in the service-oriented architecture (SOA) * Data integration # Enterprise data bus (EDB): ensure timely and accurate availability of data for all applications - Three scenarios of cloud integration * Within a public cloud # Two different applications in a cloud owned by the same company or different companies # Role of cloud integration middleware (cloud-based ESB or Internet service bus, ISB): enable these applications to talk to each other * Homogeneous clouds # Applications are in two geographically separated cloud infrastructures # The cloud integration middleware can be in cloud 1 or 2 or in a separated cloud 3-4
# Need data and protocol transformations which are done by the ISB * Heterogeneous clouds # One application is in a public cloud and the other in a private cloud # Most common scene for cloud integration - Integration methodologies * Three types of integration # Using traditional enterprise integration tools - Examples: EAI, ESB, 3-5
- Develop special connectors on these tools to access cloud-located applications - Special drivers, connectors, and adapters need to be built and incorporated on the existing integration platforms to enable bidirectional connectivity with the cloud services # Host traditional enterprise integration tools in the cloud - Same as the first approach except that the integration software suite is hosted in the cloud, i.e., the enterprise outsources the integration projects to the cloud service provider - Good for cloud-to-cloud (C2C) integration, but may require a secure VPN tunnel to access on-premise corporate data - Example: PowerCenter provided by Informatica # Integration as a service or on-demand integration offerings - SaaS applications designed to deliver the integration securely over the Internet - Even on-premise system can be integrated with other on-premise applications via this integration service - Good for companies who insist about ease of use and maintenance, time to deployment, and a tight budget - Example: Informatica s On-Demiand Integration Services * Characteristics of integration solutions and products # Connectivity: connect the source and target systems using available native interfaces - Problem connecting standard interfaces (e.g., web services) and proprietary interfaces # Semantic mediation: account for the differences of application semantics between two or more systems - Semantics: how information gets understood, interpreted, and represented within 3-6
information systems # Data mediation: convert data from a source data format into destination data format # Data migration: transfer data between storage types, formats, or systems # Data security: insure that information from the source systems is securely placed into target systems # Data integrity: data is complete and consistent - Important for data synchronization between on-premise and SaaS-based systems # Governance: managing changes to core information resources, including data semantics, structure, and interfaces (d) SaaS integration: products and platforms - Current market for SaaS integration * SaaS integration is difficult due to heterogeneity and multiplicity-induced complexity * Standards-compliant integration suites are being transitioned into services * Currently, need for data integration is greater than service integration or message-based integration * Next, there will be a huge market for application and service integration * Composition and collaboration will become critical for the mass adoption of clouds * Interoperability is the most fundamental thing 3-7
- Jitterbit * A fully graphical integration solution that provides users a versatile platform and a suite of productivity tools to reduce the integration efforts * Can be used standalone or with existing EAI infrastructure * Enables the integration among confidential and corporate data, enterprise applications, web services, XML data sources, legacy systems, simple and complex flat files * Consists of two major components: # Integration environment: quickly configure, test, deploy, and manage integration project 3-8
# Integration server: a scalable run-time engine that processes all the integration operations - Dell Boomi * An integration service that is completely on-demand * Connects any combination of SaaS, PaaS, cloud, and on-premise applications without the burden of installing and maintaining software packages or appliances * Build, deploy, and manage simple to complex integration processes using only a web browser - OpSource Connect * Allow customers to consume and publish applications across a common web services infrastructure 3-9
* Integration by unifying different SaaS applications in the cloud as well as legacy applications running behind a corporate firewall * OpSource Services Bus (OSB), based on SOA, allows applications running on the OpSource On-Demand platform to quickly and easily access web services - SnapLogic * A capable, clean, and uncluttered solution for data integration * Can be deployed in enterprise as well as in cloud landscapes * SnapLogic professional: provides production management, increased capacity, and multi-user features * Can run SnapLogic on-premise or hosted in the cloud - Pervasive DataCloud * Provides dynamic compute capacity for deploying on-demand integration and other datacentric application * The first multi-tenant platform for delivering integration as a service for both hosted and onpremise applications * Pervasive Data Integrator: support the following integration scenarios: data migration/consolidation/conversion, ETL/Data warehouse, B2B/EDI integration, EAI, SaaS integration, SOA/ESB/Web services, data quality/governance, hubs # ETL: Extract data from outside sources, Transform it to fit operational needs, and Load it into the end target 3-10
- Others: Bludwolf, Linxter, OnlineMQ, CloudMQ, - Example: Gmail Salesforce.com Integration - Link enterprise and cloud systems via messaging * Messaging: asynchronous information delivery as oppose to a request-response architecture * Message-oriented middleware (MoM): software or hardware infrastructure that supports 3-11
sending and receiving messages between distributed systems * Allows applications to be distributed over heterogeneous platforms * Reduces the complexity of developing applications that span multiple operating systems and network protocols * Cloud services and applications could be loosely or lightly coupled, or decoupled * Examples: # Informatica On-Demand Services - On-demand data integration in SaaS applications # Microsoft Internet Service Bus (ISB) - Microsoft.NET acts as the cloud middleware -.NET service bus: provides the messaging channel for connecting cloud applications to on-premises applications, services and systems (see right figure) -.NET access control service: user name and password authentication and authorization services via HTTP requests -.NET workflow service: rule-based control flow based on Windows Workflow Foundation 3-12
(WWF) (e) Business-to-business integration services - Business-to-business integration (B2Bi) Extend business processes to business partners including customers, vendors, suppliers, distributors, and other alliances * Pure EAI: only for internal data share * B2Bi has the following capabilities: # Encrypt files for safe passage across the public network # Manage large data volumes, transfer batch files, and convert disparate file formats # Guarantee data accuracy, integrity, confidentiality, and delivery * Provides connectors for major ERP, CRM, SCM, - Cloud-based enterprise mashup integration services for B2B scenarios * Mashup: combination of different and distributed resources including content, data or application functionality # Resources represent the core building blocks for mashups, and can be accessed through APIs # Widgets or gadgets put a face on the underlying resources by providing a graphical representation and piping the data received from the resources # Piping can include data aggregation, merging, or filtering * Enterprise mashups: users integrate heterogeneous digital data and applications from multiple 3-13
sources * In an enterprise mashup, the product is typically a combination of internal corporate data and applications with externally sourced data, SaaS, and Web content * Why enterprise mashups are needed? # Mashups gives faster answers # Mashups improve resource use (of both personnel and soft/hard computing resources) # Mashups help address new business opportunities by letting users assemble internal and external data in an opportunistic way * SOA: one of the promising solutions to enterprise mashup - Enterprise mashups platforms and tools: FAST project * Mashup integration services are being implemented as a prototype (see the following figure) * The prototype architecture shows the services and their relations to each other * Users access the services through a mashup platform of their choice * Mashup platforms are connected via APIs to the mashup integration services * All data coming from the users go through a translation engine to unify the data objects and protocols, so that different mashup platforms can be integrated - B2Bi examples * Clarizen Salesforce.com Integration * Cast Iron Systems: rapidly complete application-specific integrations using a configuration, not coding approach 3-14
3-15