SCRIBE WHITE PAPER HOW SCRIBE ONLINE WORKS

Transcription

1 SCRIBE WHITE PAPER HOW SCRIBE ONLINE WORKS

2 Introduction Scribe Online is a leading integration platform as a service (ipaas) from Scribe Software. It has a multi-tenant architecture that scales by distributing workloads across agents that belong to each tenant. The cloud handles administrative tasks such as user management, connection management, integration design, scheduling and monitoring. Agents carry out instructions they receive from the cloud and report results back to the cloud. Scribe Online User Interface Third Party SaaS Application Connectors API Connector Development Kit Scribe Online Cloud Shared Components Connections Maps Solutions Tenant Specific Components Execution History Applications Customer Site On-premise Agents Scribe Ecosystem Cloud Agents Scribe Software Corporation All rights reserved scribesoft.com 1

3 The platform uses connectors to communicate with specific applications, databases and file systems. Connectors address the specifics of each type of endpoint, including authentication, discovery of metadata, query formation and target operations. They interact with the platform through an open interface that allows anyone with a Scribe Online account to create a connector. A comprehensive security model encompasses all aspects of the platform. The user interface, agents, and third-party applications communicate with the cloud through secure channels. Connectors use secure protocols to communicate with the APIs for the respective applications. Credentials are stored in encrypted containers, and IP safe lists restrict access to specific tenants. The functionality of the platform is available through a modern REST API, which allows partners to embed the platform s capabilities into their applications. Continue reading to learn the details of how the components work. Scribe Software Corporation All rights reserved scribesoft.com 2

4 Organizations and Users Each tenant in the Scribe Online cloud service is called an organization. It represents the company using the service, and there are three types: 1. A normal organization stands alone and represents a company that buys the Scribe Online service directly from Scribe or through a reseller. Normal organizations often begin with a trial subscription initiated through the Scribe website. 2. A parent organization belongs to a company that purchases the Scribe Online ipaas on a wholesale basis with the ability to create and manage child organizations for its customers. 3. A child organization falls under a specific parent organization and represents a company that is a customer of the owner of the parent organization. Most cloud applications restrict each user to a single organization. Scribe Online allows users to belong to more than one organization. The user interface provides users who belong to more than one organization with a picklist to switch from one organization to another. An administrator in an organization can invite other users to that organization and remove users from the organization. Child organizations automatically inherit the users of the parent organization. Each child organization can also have its own users in addition to the users from the parent organization. Each organization has one or more subscriptions to Scribe Online services. Scribe Software Corporation All rights reserved scribesoft.com 3

5 Services Scribe Online offers three different services: 1. The Integration Service performs bidirectional integration between two or more endpoints with scheduling, support for complex logic and transformations, and the ability to recognize records that are new or changed since the last integration job ran. 2. The Replication Service creates and maintains a copy of the data in a cloud application such as Salesforce.com or Microsoft Dynamics CRM in a database. The database can be on premise or in the cloud. 3. The Migration Service enables Scribe partners to migrate data from one application to another. It offers most of the features of the Integration Service, with the exception of scheduling and detection of changed records. The Migration Service is only available as a wholesale service that the partner manages through a parent organization. Each service is available in multiple tiers with different features and options. Scribe Software Corporation All rights reserved scribesoft.com 4

6 Connectors A connector links an application, database or file system to the Scribe Online platform. Most connectors support two-way communication, and they have several basic functions, including: Authentication with a set of user credentials Discovery of metadata, often including custom objects and custom fields Sending queries, usually with filters, and receiving the results for processing Accepting inbound messages for processing Translation of Scribe Online platform operations such as Create, Lookup, Update/ Upsert and Delete into specific API calls that the application understands Connectors have the ability to process large datasets using paging, batches or bulk loads. They can process hundreds or even thousands of records at a time in one API call, improving performance and reducing the number of API hits. The Scribe Online user interface includes a Marketplace of connectors. Through the Marketplace, users enable the connectors they need in their organizations. Scribe provides many of the connectors, and third parties have created many others. Scribe Software Corporation All rights reserved scribesoft.com 5

7 Connections Once a connector is enabled in a Scribe Online organization, it can be used to make a connection to a specific instance of an application, database or file system. In most cases, a connection includes user credentials for the application or database and a URL or other location of the application. Scribe Online tests the credentials before creating the connection and stores the connection details in an encrypted container for later use when running maps. Connections are reusable -- an existing connection can be used again and again in different maps. Scribe Software Corporation All rights reserved scribesoft.com 6

8 Agents The Scribe Online architecture includes separate design and run-time components. The cloud platform handles administration, design and monitoring, while delegating execution of replication and integration jobs to its agents. The agent retrieves its commands from the cloud through Microsoft s Azure Service Bus, carries out the instructions and reports the results to the cloud. Scribe Online offers two types of agents: Cloud agents handle integrations between cloud-based endpoints such as SaaS applications, cloud databases and cloud filesystems like Dropbox. For example, a cloud agent can be used to integrate leads between Marketo and Microsoft Dynamics CRM Online. Scribe hosts the agents in its private cloud. On-premise agents handle integrations where one or more endpoints exist behind a firewall, inaccessible to a cloud agent. Users install an on-premise agent behind the firewall where the agent can communicate with the local endpoint as well as any cloud endpoints. For example, one would use an onpremise agent to replicate Salesforce.com data to a local database. All agents run on the same codebase and operate semi-autonomously, increasing the resilience and availability of the overall system. Even if an agent loses communication with the cloud, it continues to run the latest compiled versions of its maps according to the schedule. Once connectivity is reestablished, it reports the results to the cloud. This means agents continue to run even during scheduled maintenance of the cloud platform. Scribe Software Corporation All rights reserved scribesoft.com 7

9 When Scribe releases a new version of Scribe Online or a connector, the agent finishes any work it is doing, then downloads and runs the new code. This ensures that new software does not interrupt integration jobs. Maps The Scribe Online cloud provides a graphical design environment where users create and maintain integration maps. Each map begins with one or more connections from the set defined in an organization. Scribe Online offers three types of maps: 1. Query maps use one of the connections associated with the map to get data for processing. The query can include filters to restrict the results. In most cases, this will include a net change filter, which Scribe Online uses to retrieve only records that are new or changed since the last time the map ran. 2. Message maps allow a cloud application like Salesforce.com to push messages to the map. Each message map has a unique URL. Users configure the cloud application to deliver records to the URL. When records arrive, the agent executes the map and uses one of the connections in the map to interpret the message payload. Message maps process asynchronously. 3. Request/Reply maps allow an external application to send records to a map in JSON format through a REST API. Like message maps, each request/ reply map has its own unique URL. Unlike the other two map types, the application sending the request does not need a connection in Scribe Online. As the agent executes the map, it builds a response, which it sends back to the calling application. Request/reply maps process synchronously. Scribe Software Corporation All rights reserved scribesoft.com 8

10 The Scribe Online user builds the map with the visual designer by dragging blocks onto a canvas to define a logic sequence. Control blocks direct the logic flow. The main control block, For Each Result, creates a loop that executes once for each record the query produces. Others provide if-then-else and other conditional tests. Operation blocks take action on one of the connections in the map, such as creating, reading, updating, upserting or deleting a record. They also handle field mapping and data transformation with a library of dozens of Excel-like functions. For update, upsert and delete operations, users configure match fields to identify the correct record in the target connection. Scribe Online also provides a map debugger that allows a user to execute a map one step at a time or to set break points when certain conditions occur. This aids in troubleshooting map design. Scribe Software Corporation All rights reserved scribesoft.com 9

11 Solutions In many cases, an integration process involves more than one map. A solution is a set of ordered maps that are assigned to run on a specific agent. Each time a solution runs, it executes the maps in order until the final map completes. Solutions can run according to a pre-set schedule, on-demand through the user interface or via an API call. Solutions can be exported and imported. The exported maps contain the definition of the connections used, data flow, and mappings/formulas used. Export and import allow you to move solutions between test and production environments or from one customer s organization to another. REST API Scribe Online offers a REST API that provides application programmers with access to much of the functionality of the platform. This allows you to embed the functionality of Scribe Online in another SaaS application. Developers can automate processes for: Creating users and organizations Installing agents Adding connectors and creating connections Creating a solution with a set of maps Scheduling solutions Monitoring the results of integration tasks Scribe Online enforces a safe list for the API based on the IP address making the API call. Users configure one or more IP address ranges that can make API calls for an organization. Scribe Software Corporation All rights reserved scribesoft.com 10

12 New Connector Development SaaS companies and integration consulting firms can create their own connectors for Scribe Online. The scope of a connector development project includes authentication, metadata discovery, queries and target operations like create, update and delete. There are three options available for building Scribe Online connectors. The Messaging Framework works with applications that send outgoing webhooks in either JSON or XML format. It allows you to quickly build a connector from a sample message in Microsoft Visual Studio. You can use the connector as the source in a message map but target operations are not supported. The Fast Connector Framework is a wizard-driven tool that helps you build a connector to an HTTP web service by completing a Microsoft Visual Studio Solution with the specific details of the application you re connecting to. This approach allows you to build connectors that support both source and target operations. The Connector Development Kit (CDK) offers more flexibility and access to additional functionality, such as complex filtering in queries, picklist values in metadata, additional target operations beyond create and update, and support for related entities. Scribe offers a certification process for those who wish to make a connector available to others. Once certified, your connector can be listed in the Scribe Online Marketplace. Scribe Software Corporation All rights reserved scribesoft.com 11

13 Monitoring and Notifications Each time an agent runs an integration job, it reports the results back to the Scribe Online cloud. Results include how long the job ran, how many records were processed, how many failed, and any errors that occurred. Users can view the Execution History related to a solution in the Scribe Online user interface. Scribe Online can reprocess failed records for up to 45 days. Users can also configure Scribe Online to send notifications via when certain events occur. Notifications are available for solution and record failures, system updates, connector installation and lack of agent communication with the cloud. These settings are specific to each user in each organization because users can belong to multiple organizations and might want different settings for each one. Scribe Software Corporation All rights reserved scribesoft.com 12

14 About Scribe Software Scribe is an established global provider of application and data integration software. The company s easy to use platforms, comprehensive integration lifecycle management capabilities and open APIs offer the fastest way to create, run, and adapt integrations. More than 12,000 customers and 1,200 partners including enterprises, software as a service (SaaS) providers, systems integrators (SIs) rely on Scribe s integration platform as a service (ipaas) and on-premises platform to accelerate integration projects and time-to-value and to meet the complex connectivity demands of today s cloud and hybrid environments. Scribe and its solution development partners serve customers in every industry, including financial services, life sciences, manufacturing, public sector, retail, utilities, higher education and media and entertainment. For a free trial of Scribe s software platforms go to: CORPORATE HEADQUARTERS 150 Dow Street Manchester, NH USA EMEA OFFICE Bezuidenhoutseweg 161, 2594 AG Den Haag, The Netherlands +31 (0) Scribe Software Corporation All rights reserved scribesoft.com 13