Oracle BI Applications Technical Bootcamp

Extending BI Applications

Peak Indicators Limited

Agenda

Peak Indicators Limited

Agenda

Understand the different types of customisations Estimate the impact of a customisation Implementation
Approach Roles

Peak Indicators Limited

 Customisations

Peak Indicators Limited

Customisations

Is the process by which the Oracle Business Analytics Warehouse tables and columns, custom extract, transform and load (ETL) mapping templates, are modified to accommodate new data for analysis.

Peak Indicators Limited

Customisations in General

Supported source systems are pre-built applications too BI Apps setup is always required (installation during Day1) Transactional Application Modifications

User Interface Data Model Extensions New entities (business components) Different relationships between business components Modified visibility rules Different authorisation/authentication Different reporting requirements (dashboards, navigation, requests) New KPIs or reporting entities based on existing ones Different hierarchies Different visibility rules New facts and/or dimensions (unsupported functionality) Additional source systems or data warehouses Change data capture logic

BI Applications Modifications

Peak Indicators Limited

Customisation Scenarios

Customisation scenarios are categorised across data sources and modification types.

Peak Indicators Limited

Customisation Scenarios

Category 1.

In a Category 1 customisation, you add additional columns from source systems that have pre-packaged adapters and load the data into existing data warehouse tables. In a Category 2 customisation, you use prepackaged adapters to add new fact or dimension tables to the data warehouse. Category 2 customizations normally require that you build new SDE and SIL mappings. In a Category 3 customisation, you use the Universal adapter to load data from sources that do not have pre-packaged adapters.

Category 2.

Category 3.

Peak Indicators Limited

Upgrade Considerations

Follow the recommended customisation methodology to minimise the effort required to reapply customisations after an upgrade.

Peak Indicators Limited

Type I Customisation: Adding Columns to Existing Tables

Type I customisations involve extracting additional columns from source systems that are already mapped (for example, Siebel or Oracle) and loading the data into existing data warehouse tables.

Peak Indicators Limited

10

Type I Customisation: Extending Mappings

Existing mappings and tables are extensible. Sample placeholders demonstrate how to pass and store additional data. Oracle BI Applications provides a methodology to extend preconfigured mappings to include additional columns and load the data into existing tables.

Always copy OOTB mappings first into a custom folder and modify existing logic or columns in the custom mappings.

Peak Indicators Limited

11

Type I Customisation: Safe Path

Most mappings have a single placeholder column, named X_CUSTOM, that marks a safe path through the mapping.

All extension logic should follow the same route through the mapping as X_CUSTOM. You can add additional transformations to the mapping, but they should follow the same route through the mapping as X_CUSTOM. The graphic shows the pre-configured logic in grey. You should not modify anything contained within these objects. You should add customisations to the existing mapping, which allows them to run parallel to the existing logic.

Peak Indicators Limited

12

Type I Customisation: Extension Categories

Exposed objects: Sources, Targets, nonreusable transformations

Can be changed in the form of extensions (additive) Must be copied to custom folders before being modified

Encapsulated objects: Mapplets, reusable transformations

Should not be changed. Can be added to a mapping

Custom objects: Objects not shipped by Oracle

Peak Indicators Limited

13

Type I Customisation: Impact of Customisation on Upgrade

If you extend a mapping and the mapping:

Does not change during the upgrade, all extensions are retained Experiences changes to the encapsulated logic, all extensions are retained Experiences changes to the exposed objects, extensions to these objects are lost but the underlying extension logic is retained Extensions to exposed objects must be manually reapplied.

Peak Indicators Limited

14

Type I Customisation: Points to Remember

Encapsulated objects must never be customised unless directed by Oracle. Exposed objects can be extended but must never be otherwise be modified. Custom objects are never changed during an upgrade. Minimise the number of changes to exposed objects by using custom objects. Evaluate options and determine the best approach for your environment

Custom object approach is the preferred approach if ETL time is acceptable.

Peak Indicators Limited

15

Type I Customisation: Typical Steps to Extend Mappings

1. 2. 3. 4.

5.
6.

Copy the appropriate mappings to a custom Informatica folder. Extend the source and target tables by making changes to the tables in the database. Extend the Source Dependent Extraction (SDE) and the Source Independent Load (SIL) mappings by bringing in the additional columns. Copy the appropriate workflows to the custom Informatica folder. Modify workflows and sessions as needed. Update the Data Warehouse Administration Console (DAC) with the necessary changes.

Peak Indicators Limited

16

Type II Customisation: Adding Additional Tables

Use pre-packaged adaptors to add new fact or dimension tables to the datawarehouse, regardless of whether they are already mapped. Build new SDE and SIL mappings.

Peak Indicators Limited

17

Type II Customisation: Considerations

Use required system columns. Register tables and indices in the DAC. Register new tasks for Informatica workflows, assemble subject areas and build execution plans in the DAC. Use the naming convention: WC_TABLENAME_<table type>

Peak Indicators Limited

18

Type II Customisation: Required Columns

Staging tables:

INTEGRATION_ID, DATASOURCE_NUM_ID INTEGRATION_ID, DATASOURCE_NUM_ID, ROW_WID, ETL_PROC_WID

Fact, dimension and extension tables:

Peak Indicators Limited

19

Type II Customisation: DATASOURCE_NUM_ID

Is part of the unique user key for all tables in the warehouse schema. Permits rows to be loaded in the same warehouse tables from different sources, provided that the column is given a different value for each source. Reserves the value 1 for Siebel, this and other reserved values cannot be used for other datasources

Peak Indicators Limited

20

Type II Customisation: Custom Folders

Create custom SDE and SIL folders in Informatica and make changes in them.

Do not store customised SDE and SIL mappings in the same folder.

Do not change objects in shipped folders.

Copy existing objects from the shipped folders into the corresponding custom folders. Reuse existing business components, source and target definitions,transformations, mapplets and mappings.

Peak Indicators Limited

21

Type II Customisation: Custom Workflows

Create custom workflows for all customised mappings.

Each workflow should load only one table. The workflow name should match a session name that is used inside the workflow. Set the appropriate source and target connection values in Informatica Designer. Register workflows in the DAC.

Peak Indicators Limited

22

Additional Customisation Considerations

See next slides:.

Table definitions in Informatica. Update Strategy ETL process Truncating target tables ETL_PROC_WID DATASOURCE_NUM_ID Creating Indices Naming Conventions Configuring the DAC

Peak Indicators Limited

23

Table Definitions in Oracle Format

By default, table definitions are set to Oracle in the Informatica repository. Set SQL style to Oracle when importing new table definitions from external datasources regardless of database type.
Used by Informatica to interpret at run time what the datatypes are going to be, depending on the source and target specified in the session properties Does not affect behaviour of mappings or logic of how the data is loaded; database types get converted appropriately

If SQL style is not set to Oracle, mappings that use table definition will be invalid.

Peak Indicators Limited

24

Update Strategy

Design a custom process to detect new and modified records. Process should be designed to load only incremental data. If data is loaded without incremental process, previously loaded data will be updated again. Example logic in standard SIL mappings:
Mapping looks up destination tables based on INTEGRATION_ID and DATASOURCE_NUM_ID 2. If combination exists, ROW_WID is returned and record updated. 3. In some cases e.g. eBS, the last update date stored in the the target tables is also compared to the columns mentioned to determine insert or update. 4. If combination does not exist, lookup returns NULL and a record is inserted.
1.

Peak Indicators Limited

25

ETL Process

When using multiple sources for the data warehouse, you can decide to load from all of them at the same time, or at different time frequencies using different execution plans. Integration of data from different sources and update frequency in SCD have impact on sequential or parallel loads

Peak Indicators Limited

26

Truncating Target Tables

Truncating should be done through the DAC A single DAC task has one placeholder for a full load, and one for an incremental load. Truncate tables based on the following guidelines:

For SDE workflows, the commands for full and incremental load are the same. They should have the Truncate Always flag selected in the DAC. For SIL workflows, the command can be different for full and incremental loads. They should have the Truncate For Full Load option selected in the DAC. When a table is truncated, the indices are automatically dropped and created after the data is loaded.

Peak Indicators Limited

27

ETL_PROC_WID

Is a foreign key to the W_ETL_RUN_S table, which maintains the history of the ETL process

To use the same ETL_PROC_WID, copy the reusable lookup LKP_ETL_PROC_WID to mapping; input to lookup is a constant (hard-coded to 1)

DATASOURCE_NUM_ID
Use a parameter to define this value in the mapping.
The DAC will automatically create a parameter file with the correct DATASOURCE_NUM_ID, which will be picked up by the parameter in the mapping.

Peak Indicators Limited

28

Creating Indicies

Staging tables typically do not require indices. Create indices on all columns that the ETL uses for dimensions and facts. For example:
ROW_WIDs of dimensions and facts INTEGRATION_ID DATASOURCE_NUM_ID Flags

Carefully consider on which columns to put filter conditions. Define indices to improve query performance

Inspect standard repository objects for guidance The DAC server drops and re-creates indices during a full load.

Register new indices in the DAC.

Peak Indicators Limited

29

Naming conventions

Name all the newly created tables with the prefix WC_.

This helps isolate the new tables from the shipped tables Helps when upgrading the warehouse

Keep good documentation of customisations.

DWH Fact Table Dimension Table Dimension Hierarchy Table Aggregated Fact Table

Base WC_XXX_F WC_XXX_D WC_XXX_DH WC_XXX_A

Staging WC_XXX_FS WC_XXX_DS WC_XXX_DHS

Peak Indicators Limited

30

Configuring DAC

Customisation changes must be registered in the DAC:

Table, column and index definitions Workflows Custom folders

Peak Indicators Limited

31

 Implementation

Peak Indicators Limited

32

General Considerations

Which BI Applications modules are being implemented? Is the customer coming with specific reporting/BI requirements? What are the source systems and does this include multiple instances of one source system type? Are adaptors available for the relevant module and source system? Is the project to be phased in delivery of modules? What are the levels of UI and security integration between BI Apps and source systems? Level of customisation of source systems? Is BI Apps being implemented at the same time as source systems? Implementation vs Upgrade?

Peak Indicators Limited

33

Iterative Delivery approach

Key requirement selected Definition

Install and populate vanilla BI Apps warehouse Requirements Modelling Gap Analysis, design and configuration

Tailor and test BI Apps for first key requirement

Construction

Deploy BI Apps to production Production

Start work on next set of requirement

60-90 day release cycles Peak Indicators Limited

34

Typical Effort & Customization balance

Additional dashboards and reports, guided and conditional navigations, iBots, etc. Additional derived metrics, custom drill paths, exposing extensions in physical, logical and presentation layer, etc. Extension of DW Schema for extension columns, additional tables, external sources, aggregates, indices, etc. Extension of ETL for extension columns, descriptive flexfields, additional tables, external sources, etc.

Dashboards & Reports

Easy

OBIEE Metadata

Moderate

DW Schema

Intermediate

ETL

Involved

Degree of Customization
Peak Indicators Limited

Level of Effort
35

Typical Customization

Peak Indicators Limited

36

BI Application Extensions
1

Type 1 Simple Extensions

Extension of existing facts or dimensions with new information

New Facts or Bridge with existing dimensions

Type 2 New Facts or Bridge table

Type 3 Fact granularity change

Add new Dimensions to an existing Fact

Type 4 New star schema

New facts and some new dimensions integrated with existing dimensions Adding a second home-grown or non-supported source Modified data access control and security mechanism

Type 5 Additional Source Systems

Type 6 Security

Type 7 Hierarchy Changes

Modified hierarchies because of dimension extensions More or less attributes

Type 8 Slowly Changing Dimension Changes

Type 9 Other than standard source tables part of CDC

Peak Indicators Limited

37

BI Apps Implementation Steps

Project Management

Knowledge Transfer

1.

Vanilla implementation
A. B.

C.

D. E. F. G.

Tech and Func. Architecture / Install and basis configuration Global setup Security / Mapping ETL / DAC Repository / OBI EE Repository / Dashboards Per module : Standard configuration and load Per module : Data quality check Per module : Integration testing Assistance to functional UAT Assistance for transition to production

2.

Tailoring

Peak Indicators Limited

38

BI Apps Implementation Steps

1.
2.

Vanilla Implementation Tailoring

Gap analysis : global and per module B. Per module : data modeling C. Per module : ETL customization and load D. Per module : OBIEE customization and new dashboards E. Per module : Unit Testing and Data quality check F. Per module : Integration testing G. Assistance to functional and technical UAT H. Assistance for transition to production
A.

Project Management

Knowledge Transfer

Peak Indicators Limited

39

BI Apps Example Rollout

Roll-out priorities
Procurement 2. Accounts Payable 3. GL + Accounts Receivable 4. Custom development : Grants Management
1.

Procurement AP

Standard

Custom

Standard

Custom

GL + AR GM

Standard

Custom

Custom

Peak Indicators Limited

40

Profile of an OBI Apps Implementation Team

Role
Project Manager Technical / Solutions Architect

Responsibility
Responsible for the management of the project team and all client relations Applications Deployment Gap Analysis - Technical Solution and Technical design mitigating gap to OOB product Requirements Definition Gap Analysis Functional Report & Dashboard Development Testing - Functional End user Training Back End Installation, configuration and development of custom ETL code Metadata Configuration Dashboard Configuration

Skill Sets
Project Management with experience in OBI Apps deployment Client and Vendor escalations Depth and Breadth on technical knowledge of the products used. Knowledge on Functional Area and maybe industry SME on Functional area and industry Knowledge of ootb content for OBI Apps modules Documentation and Training

SME Functional Analyst

ETL Developer

Informatica Development Working Knowledge of OBI Apps

OBIEE Configurator

OBI Metadata and Dashboard Configurator Knowledge of OOB content for OBI Apps modules
41

Peak Indicators Limited

Summary

This module provided an introduction to: Customisations Implementation approach Implementation roles

Peak Indicators Limited

42

Helping Your Business Intelligence Journey

Peak Indicators Limited