OWB EE 11.2: From ETL to DI 16 November 2009 Antonio Romero, Senior Product Manager, Data Integration

Transcription

1 OWB EE 11.2: From ETL to DI 16 November 2009 Antonio Romero, Senior Product Manager, Data Integration

2 Roadmap: DI Beyond ETL

3 Overlapping Data Integration Spaces Dimensional modeling and ETL Bulk Data Movement Changed Data Capture Data Quality for Data Loads DW Data Integration BI Data Integration for BI applications Event driven BI Operational Analytics SOA Data Services Process Integration Event driven data integration Requirements converging over time

4 Oracle DI Products Oracle Warehouse Builder Question: Does this help me build an Oracle DW? DW is the overriding use case Other features (e.g. DQ, modeling) serve DW and BI needs Highly dependent on Oracle DB for critical elements: connectivity, CDC, SOA, repository, runtime agent Very successful, many thousands of customers Oracle Data Integrator Question: Does this help me integrate more data, better? Mixed environments No single stack vendor DW is one of many important use cases Strategic platform for future development

5 Oracle Data Integration: The Road to 12g Milestones Toward Convergence Unified Team Oct 2008 ODI-EE License Jan 2009 OWB-EE 11gR2 Sep 2009 ODI 11g 2010 ODI 12g (Unified Platform) ODI Strategic Data Integration Platform Knowledge Modules OWB-EE 11gR2 ODI-EE License In-Database ETL OWB Investments are Fully Protected No Forced Migrations or Switches Regular Upgrade to Unified Platform ODI 12g (Unified Platform) aims to be a Superset of Existing Products no regression 5

6 How OWB-EE 11gR2 Helps You Protect existing investments in OWB designs and skills ODI infrastructure presented in familiar mapping model Oracle DB, DW and BI tool support are also enhanced Meets near-future requirements around new DI options Minimize or eliminate switching costs No value to switching twice Smooth long-term transition to ODI 12g in the future When 12g is ready, OWB capabilities will be there; migrate your designs then Result: Best Short-Term ROI Fastest Time to Value through easy mapping migration Fully Supported in the Roadmap

7 OWB-EE 11.2: Code Template Mappings

8 OWB 11gR2 EE: ETL Mapping Options Code Template Mappings: ODI Technology in OWB Logical view: Look just like classic mappings Abstraction: Can combine any operators in a mapping Execution view: Pick your own code generation Code templates = ODI 10.1 KMs (more or less) Runtime Workload can be pushed to multiple sources/targets Connectivity: JDBC, native load/unload, other Easy to migrate existing mappings to CT mappings Classic Mappings: Still Available Generate SQL, PL/SQL, SQL Loader, ABAP code Limited operators depending on mapping language Internal code templates for different mapping languages Runtime: target DB/Control Center Service Connectivity: Database Links, Gateways Still work as in previous releases, no forced migration

9 OWB 11.2 Code Templates: Extract, Load, Integrate Same basic pattern as in ODI 10.1 Pluggable Code Template Framework Custom Metadata Import CDC Load Control Integrate Load CDC Control Integrate/ Oracle Tgt Sample Seeded Code Templates Oracle LogMiner SQLServer Triggers DB2 UDB Triggers SQL to SQL Oracle Control Oracle SQL*Loader Oracle Target Oracle DBLink

10 OWB 11.2: Code Templates CTs define sequence of tasks Tasks define code snippets to run at source or target Code snippets contain Code snippets contain placeholders expanded during generation

11 OWB 11.2: Code Template Mappings: Example

12 OWB 11.2: Code Template Mappings: Example

13 OWB 11.2: Code Template Mappings: Example

14 OWB 11.2: CT Mappings Oracle Target CT

15 OWB 11.2 CT Mappings: Flat Files for Fast DW Loads SQL Server BCP Unload Oracle Target (10.2 / 11.1) FTP External Tables Oracle Source FTP Data Pump Unload Oracle Warehouse Builder 11g R2 Load Code Templates Integration Code Templates

16 OWB 11.2 CT Mappings: Fastest Exadata Loading SQL Server BCP Unload Oracle Target (11.2) FTP External Tables Oracle Source SCP Data Pump Unload DBFS in its own instance holds files in SecureFiles

17 Easy, Flexible CDC Mappings Supported using Journaling Code Templates from ODI Multiple capture methods Triggers or log-based Consistent or simple For new CDC technology, write new Journaling CT Setting up CDC Set up source module with change tables and CDC template Define subscribers Create CT mappings to process changes Start capture 17

18 Getting and Applying OWB Code Templates Some CTs pre-packaged with OWB Currently certified KMs ship with OWB Can implement new CTs directly in OWB Check for Updates Migrate OWB Classic Mappings to use CTs instead Cut and Paste to move existing mapping to CT framework Use Oracle Target CT for complex transformation once data is in Oracle Change default execution units and CTs for optimal performance

19 More New DI Features: SOA, Event-Based DI

20 Web Service Integration Easy to publish a mapping or process flow as a web service Runs at Control Center Agent Can be invoked by Oracle BPEL or other web service consumer Process flow activity to invoke Web services

21 Advanced Queues for Event-Based Integration Advanced Queues as Sources or Targets in Mappings Real-Time (Trickle-Feed) and Batch supported Built on Oracle Streams Great post on OWB Blog with lots of details

22 Resources: Information and Community OWB on OTN OWB Blog OWB Forum OWB LinkedIn Group

23

24