Beruflich Dokumente
Kultur Dokumente
DataConventional
IntegrationData Integration BI & DataBI & DW Projects
Warehouse
• Bulk-load historical data to new application • Real-time Data Warehouse for BI
• Synchronize new and old applications • Populate Warehouse with High Performance ODI
• Complex transformations, CDC • Aggregates and aligns data for operational
• Ensure database-level consistency across analytics, performance management, etc
applications bidirectional
Modernization Initiatives
Modernization Migrations / Consolidation
Data Migrations & Consolidation
• Mainframe / iSeries sources to Oracle RAC • Upgrade Applications or Migrate to New Schema
• Re-architect legacy batch processing to SOA and • Single-time Bulk load and/or keep in Sync with
ODI-EE ongoing translation, delivery of data.
• Unification of structured and unstructured data • Applications merge support
SOA Projects
SOA Initiatives Master
Master Data Data Management
Management
• Establish Messaging Architecture for Integration • Create Single View of the Truth in real time
• Incorporate Efficient Bulk Data Processing with ODI • Aggregating multiple operational sources
• Providing data access, transformation etc. as • Synchronize Data with ODI-EE
services within SOA • Supports master data stores, publishes master data
changes to all consumers
ODI-Enterprise Edition at Oracle
Pervasive Data Integration in the Oracle Ecosystem
New
11
Differentiator: E-LT Architecture
High Performance
Conventional: Separate ETL Server Conventional ETL Architecture
• Proprietary ETL Engine
• Poor Performance
Extract Transform Load
• High Costs for Separate Standalone Server
• IBM & Informatica’s approach
Oracle: No New Servers
• Lower Cost: Leverage Compute Resources &
Partition Workload efficiently
• Efficient: Exploits Database Optimizer
• Fast: Exploits Native Bulk Load & Other
Database Interfaces Next Generation Architecture
• Scalable: Scales as you add Processors to
Source or Target
Benefits
• Optimal Performance & Scalability
“E-LT”
• Better Hardware Leverage Transform Transform
Extract Load
• Easier to Manage & Lower Cost
1212
E-LT Architecture with Data Integrator
Comprehensive, Lightweight Data Integration
• Key Architecture Benefits: 100% Java, Open APIs, very fast E-LT
• Embeddable Java Agent consumes very little CPU or RAM
• E-LT Architecture uses DBMS CPUs, not the Application CPUs
• Open APIs, Open Metadata, Open XML Knowledge Modules
LKM IKM
JKM A
D
B C$_0
I$ E$ (Errors)
LKM C$_1
File
C IKM CKM
RKM
Extract-Load Transform Check-Load
Differentiator: Knowledge Modules
“Hot-Pluggable,” Modular, Extensible
Staging Tables
Load Integrate
Services
CDC
Check Target Tables
Journalize
Sources
Error Tables
1414
Differentiator: Changed Data Capture
Event-Driven Framework for Realtime Data Integration
Change Data Capture (CDC) allows for real time data to be available for any
type of data integration solution. Change Data Capture is accomplished
by capturing just the changed records and not the full data set,
dramatically reducing time and resources over the life of the data
integration solution.
15
Easy Steps to CDC
Simple Framework for Business Intelligence & Warehousing
1616
Differentiator: Flexible Error Workflow
Error Recycling, Native Integration with BPEL & Human
Workflow
Conventional: Rigid/Non-
Existent Workflow Target
• Proprietary workflow baked-in Static
• No workflow / Integration Sales Check
3rd party solution Process
Co
Er
• Flexible Data Correction Workflow
rre
ror
• Static Constraint Checking and
cti
s
Error Recycling
on
Validation
s
• Flow-based Constraint
Checking and Validation
• Error Recycling
• Standards-Based Process Flow
• BPEL Process Manager / Human
BPEL4People Workflow
• Reuse Standardized Error Handling BPEL Process Manager
Procedures
• Integrated with Human Workflow
• Manage Long Running Transactions
• Manage Complex Approval and
Overriding Patterns
17
Differentiator: Data Quality
Inline E-LT Data Integrity Framework
1818
Differentiator: Dimensional Modeling
Warehouse Builder Enterprise ETL Modeling Framework
• Creation of
objects
• Configuration
for multiple
databases
• Validation
• DDL Code
generation
• Deployment to
multiple
databases
• Data viewing
Differentiator: Native SOA Architecture
SOA and ODI for Evolution to Modern IT Architecture
Conventional: Monolithic/Legacy C++
• Decade(s) old heavy-weight architectures SOA Clients
Business Intelligence | Web 2.0 Mashups etc.
• SOA is a “bolt-on” – not integral
Data Access Transformation Batch Data
• Java is a “wrapper” – not native Service Service Service
Data
Custom
(Unstructured Sources) Legacy DB
ER
• Easy SOA Integration P Apps
• Invoke Data Flow as part of ESB ESB Files /
ESB Files /
XML
• Invoke Workflow via SOA for XML
2020
ODI for SOA Use Cases
Oracle SOA Suite and Data Integrator
Add Comprehensive Data Service Capabilities to SOA
22
ODI for Transforming Large Payloads
Reference
Product
Order DB
3 Reference Suppliers
7 8
Reference
Shared
Metadata
Repository
ODI Invoke
Data Integrator 5
4 Transform
6
Shared
Container/JVM
Shared SOA Server
23
ODI for Loading Bulk Data
Reference
Product
Suppliers Reference DB Server
3 8
7
Reference
Shared
Metadata
Repository Any DB
ODI Invoke Transform
Records
6
B
Data Integrator Records
4 5 <zyx>
A
Shared
Container/JVM
Shared SOA Server
24
ODI Loads a BI Warehouse
(with Error Hospital Human Workflow)
c Human
1 Workflow
Event Oracle BPEL PM or ESB
2
7
d
3 Instruction
1. An event occurs which 6
prompts a data load Confirmation
Shared
2. Execution starts within Metadata
BPEL/ ESB process Repository
b
3. Instruction to start ODI ODI Invoke
4. ODI extracts & loads data e
a) Some errors w/in batch
b) Notification to BPEL
Data Integrator
Transform
Human Workflow
engine Shared
Container/JVM
c) Repair & approvals
Shared SOA Server
process
a
d) Repairs are committed
e) ODI looks for repaired DB Server 4 5 DW Server Business
data when “Recycle” Intelligence
option is on
5. ODI transforms the data Transform
6. ODI confirms job complete Any DB Records
7. BPEL/ESB process B
continues any additional Records
process execution steps A
25
ODI for SOA-based ERP Integration
Messaging and bulk data
1. A business process ERP may leverage the same unique
object IDs – thereby ensuring ERP
for Migration (initial App 1
bulk data load) or uniform ERP data objects App 2
Replication (ongoing (eg: PK123 = Pkxyz)
synch) is invoked UI UI
2. BPEL/ESB sends
instruction to ODI Oracle BPEL PM or ESB
3. ODI performs E-LT XML 1event XML
a) ODI creates App 5
App
Unique ID for new
ERP data objects
b) ODI updates 2 Instruction
Unique ID for c
existing objects Confirmation
Shared
4. ODI confirms job 4
Canonical
ID XREF
5. BPEL/ESB begins Lookups
processing ERP ODI Invoke
business
transactions Data Data Integrator Data
c) BPEL/ESB Transform
3
leverage same Shared
Unique IDs for Container/JVM b
canonical XML DDL
6. All SOA and ETL jobs DDL a
keep business data
aligned with Unique
IDs
26
Data Integration Suite
What is ODI Suite?
Extend SOA with Data Services
28
What is ODI Suite?
Best of Breed SOA Middleware
• Packaged Components
• Oracle Data Integrator Enterprise Edition (ODI-EE)
• Data Relationship Management (DRM - Hyperion MDM)
• BPEL Process Manager (w/Human Workflow)
• Oracle Service Bus
• WebLogic Suite (Coherence, B2B, Rules…)
• Optional Components
• Oracle Data Quality (ODQ) & Oracle Data Profiling (ODP)
• Oracle Data Service Integrator (ODSI)
• Adapters (ERP, B2B, Unstructured, Mainframe, etc)
29
Data Integration
The Challenge of Integration
Client Applications
Need:
Information
how and
when you
want it
Have: Data
In Disparate
Sources
SOURCES
DATA
Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid | Content Management | Search
30
Leverage a Comprehensive Foundation
Client Applications
Data Connectivity
Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid | Content Management | Search
Deploy with the Flexibility of Services
SOA Data Services Strategy
Data Connectivity
SOURCES
DATA
Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid | Content Management | Search
32
Oracle Data Integration Suite
Oracle Fusion MDM Business Activity SOA
Applications Applications Applications Intelligence Monitoring Platforms
Bulk Data Core Realtime Data Core Master Data Core Data Quality Core
Bulk Acquisition Change Data Capture Realtime Query Data Classification Cleanse/Parse
Complex Transforms Integrity Constraints Query Federation Data Reconciliation Cluster Analysis
Data Connectivity
Relational Databases Non-Relational DBs Various File Formats XML Data
Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid | Content Management | Search
Best-of-Breed Data Integration Platform
Oracle Fusion MDM Business Activity SOA
Applications Applications Applications Intelligence Monitoring Platforms
Bulk Data Core Realtime Data Core Master Data Core Data Quality Core
ETL
Bulk &
ETL & CDC
CDC Core
Acquisition Core Data Services
Change Data Capture Realtime Query Data Quality
Data Classification
Master
Master Data
Data
Cleanse/Parse
(From
(From Oracle
Oracle Data
Data (From Trillium Joint
Trickle Feeds
Integrator) Event-Based (From
(From BEA
BEA ALDSP)
ALDSP)
Data Caching Lifecycle Mgmt
Development)
(From
(From Hyperion
Hyperion MDM)
MDM)
Matching/Dedupe’s
Integrator) Development)
Complex Transforms Integrity Constraints Query Federation Data Reconciliation Cluster Analysis
Data Connectivity
Knowledge Relational
Knowledge Modules
Modules for Databases
for XML,
XML, Flat
Flat Non-Relational
Licensed AdaptersDBs for Legacy
Various
& File Formats
Stellent ContentXML Data
Adapters
File, Relational,
File, Relational, Basic
Basic Apps
Apps Specialized Apps B2B Adapters
Packaged Applications
Packaged Applications Standard Msg Formats
Standard Msg Formats Message Queues
Message Queues Semi-Structured Data
Semi-Structured Data
Oracle Database
Oracle or 3rd Party: Applications | Analytics | Identity | SOA Platforms | Data Grid
Warehouse
Warehouse
Builder
Builder
High Level Reference Architecture
For a Comprehensive Data Services Strategy
Client Applications
BPEL
Service DSP Object.wsdl ODIInvoke.wsdl ODI SKM.wsdl
Bus Cloud
Canonic
Data Bulk Data
Data
al Events and Grid
Data Event
Data Service Service
Services BAM/
Coherence
CEP
Service Bus
Data
Service
Oracle Data
ODSI Quality
Servers (BEA ODI
SOA DB Cleansing
DSP)
Key: Adapter
API Calls Business Intelligence
(ESB or
Data Flows
BPEL
Weblogic Server BIEE HYSL
hosted)
DW
BULK DATA
35
Data Services Personnel
Specialist Roles Require Enterprise-class Platforms and Tools
ERP Applications
36
Data Services Personnel
Key Roles for a Complete Solution are at Every Tier
Process Data
Architect Steward Data
Taxonomist
ODI
Business Business Master
Suite
User Process Data
Workflow SOA
Architect
BI … Apps
Biz SOA
Rules Bus
Data
Data Grid
Architect
Quality
Data
Integration
DBA
Data Data
Business Steward Taxonomist Business
User User
Hyperion Data Relationship Management
Operational Analytic
Hierarchy Governance
Legacy Analytics
& &
Enterprise
People Soft Oracle Oracle Reporting
Data Data
Quality Quality
E-Business Suite Financial Apps
Planning &
HR Forecasting
38
Oracle Data Integration Suite
Competitive Summary for Data Services
IBM Info Business
Feature Oracle Informatica
Server Objects EIM
Performance ODI Beats in ODI Beats in ODI Beats in
transforms (ELT) transforms (ELT) transforms (ELT)
39
ODI-EE for Oracle
Focus on Oracle BIEE Suite Plus
Unify Data Integration with Business Intelligence
• Report-to-source lineage
Design Data • Drill-in from a Report
& Drill Enterprise Data
Flow • Get query, DW schema, source
Warehouse
• Oracle RAC
tables, transforms
• Teradata
• Netezza…etc
• Gain confidence in analytics
41
Report to Source Data Lineage
Explore Lineage from Target Columns to Source Columns
Focus on OBI Analytic Applications
Overview
ODI Knowledge
Modules for BI
Pre-Built ODI Interfaces and • Extensible by Customers & System
Analytic
Transformation Packages for Integrators
Analytic Applications
Applications
Changed
Build from Scratch Pre-built BI Apps
Metadata Bulk Data Data
Data
Repository Transform. Integrity Quality Training
Capture
Metrics &
ODI Connectivity Framework Dashbrds
DW Design
Hyperion Data
Relationship
• Hierarchy management
Management • Derivation/inheritance
• Classification, categorization
DRM
Import/Export
• Attribute management
Profiles
• Change management
• Streamlined, automated
• Valid-from, Valid-to ranges
Oracle Data Integrator
• Compare any 2 versions
ODI Knowledge Modules for DRM • Business-user driven
Metadata Bulk
Changed
Data Data
• Easy user declarative tool
Data
Repository Transform.
Capture
Integrity Quality
• Specify business rules
ODI Connectivity Framework
• Multi-user collaboration
• What-if & historical analysis
• Meets IT requirements
Bulk and Real-Time Data Processing • Synchronize with operational systems
Information
Oracle
EBS SAP/R Data
• Complete auditing, fine-grained
CDC 3
Assets Other
Sources
People
Soft
MessageWarehouse
Queues
security
• Configurable, no coding
46
Focus on Business Activity Monitor
Use Data Integration with High Speed Event Processing
47
ODI for Oracle Fusion
Data Integration as a Key Fusion Differentiator
• Fusion Applications
• Built-in, Automated Data Replication among ERP Applications
• Fusion w/Essbase
• Fusion Financials (Automatic Financial Consolidation)
• Pre-built Essbase Content with Apps and Business Intelligence
Focus on Pillar Replication
Embedded ODI Architecture
Design time Runtime
JDeveloper
OC4J
ADF BC
Pillar Editor FSCM DB with Pillar
ODI Extensions
CDC
SQL
CRM
HCM
MDS Data
Replication
Enterprise FSCM Oracle Data Integrator
EPM
Manager ELM
Pillar Definitions
OC4J
ODI Designer Replication ODI Agent
Interfaces ADF BC
SQL
HCM DB with Pillar
Pillar Metadata Generator ODI
ODI Agent ODI Extensions
Pug in SDK
ODI CDC
Repository Knowledge
Pillar Deployment Modules ODI Agent
Configuration UI Pug in
Data
Replication
SQL
Service OC4J
EM DB Registry ADF BC
CRM CRM DB with Pillar
Pillar Deployment HCM
Configurations ODI Extensions
FSCM
CDC
EPM
ELM
Focus on Pillar Replication
Patterns Covered for Conflict Resolution
Pattern Solution
Physical Conflict Pillar Replication Framework
(Insert/Update/Delete) default KMs to detect & resolve
database level conflicts
Logical Conflicts (Primary Flag Application Specific logic built
& Date Effectivity) using ODI Knowledge Modules
to detect & resolve conflicts.
Column sub-setting All or some columns can be
selected from EOs using the
RDS Editor.
Row sub-setting A Filter can be added to a
replication data set using the
RDS Editor
Q&
A
The preceding is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.