Informatica PowerMart Training

Course Structure
What is ETL? What is Informatica?

Informatica Products
Informatica PowerMart training
Session I
- Overview of PowerMart

Session II
- Working with PowerMart Repository Manager

Session III
- Working with PowerMart Designer

Session IV
Working with PowerMart Server Manager Performance tuning Case Study - Test your Informatica skills Quiz

What is ETL?
ETL(Extraction, Transformation and Loading) is a process by which data is integrated and transformed from the operational systems into the datawarehouse environment

Extraction

80 tables

Oracle

50 tables

Sybase Target

Text files

Transformation
Source
Emp id
10001 10002

Last Name
Jones Holmes

First Name
Indiana Sherloc k

Staging Area

Name = Concat(First Name, Last Name) Indiana Jones Sherlock Homes

Loading
Data Warehouse
Direct Load

Source

Staging Area

Cleaning, Transformation & Integration of Raw data

What is Informatica?
A market leading provider of e-business infrastructure and analytic software which enables customers to automate the integration, analysis and real time delivery of critical corporate information via web,wireless and voice Informatica applications include
eCRM application eBusiness Operations application eProcurement

More than 1,370 customers, including 60 percent of the Fortune 100 companies are using Informaticas analytic solutions More than 900 companies are using Informatica products

What is Informatica
Founded in 1993 HQ : Redwood City, CA

Informatica Products
Informatica provides the following suite of products for data integration
PowerCenter- enterprise data integration hub PowerMart- application deployment platform PowerCenter.e- PowerCenter extension for e-business data PowerConnect- high performance data extraction PowerPlug- data model import utilities PowerBridge- Metadata bridge to Hyperion Essbase Analytic Business Components- developer productivity tools Mobile Access- delivery of corporate data and analytics via wireless devices and voice recognition

PowerCenter 5 and PowerMart 5

eBusiness Integration XML Sourcing/Targeting XML data can be source analyzed, read in, and written out. Source analysis can be performed via direct parsing of the XML data file, via DTD's, or via XML schemas. Object-Level Import/Export Objects from XML files can be imported/exported using the client tools. Appropriate validation is performed on imported objects. Web metadata reports More than 20 new, HTMLformatted metadata reports have been added to provide documentation to mappings in areas such as dependencies and attribute listings. These reports are extensible through modification of XSL templates.

PowerCenter 5 and PowerMart 5

Extensibility IBM MQ Series Compatibility IBM MQ Series data can be source analyzed, read in, and written out via the MQ Series PowerConnect. Requires Informatica PowerCenter. Siebel Systems Compatibility Siebel data can be source analyzed, read in, and written out via the Siebel PowerConnect. Requires Informatica PowerCenter. SAP R/3 Compatibility SAP R/3 data can be source analyzed, read in, and written out via the SAP/R/3 PowerConnect. Requires Informatica PowerCenter. PeopleSoft Compatibility PeopleSoft data can be source analyzed, read in, and written out via the PeopleSoft PowerConnect. Requires Informatica PowerCenter. Router (splitter) When designing mappings, a new transformation object can be used to provide a CASE statement for routing rows to different pipelines

PowerCenter 5 and PowerMart 5

Performance Parallel Transformation Pipelining (Informatica PowerCenter only) The DTM is multithreaded. Users can parallelize their transformation pipeline by specifying the number of data partitions. Special considerations were made for aggregator and join transformations. Dynamic Lookup Cache Users can insert rows into the lookup cache during an active session. Shared Lookup Cache The persistent lookup cache file can be shared across mappings. Copy Session Users can copy a session from one folder to another, from one repository to another. Mapping Variables Parameters can be specified within various portions of a mapping. Debugger With the Debugger, users can perform debugging operations such as run mappings from Designer tool, set break points, examine port values, step through execution, and modify port values. Full Screen View Users have the option of editing mappings in full screen mode.

PowerCenter 5 and PowerMart 5

Repository Manager Privileges Administrators can see a display of all privileges granted to a specific user in the repository manager. Server Manager Read Only Sessions A user with Session Operator privileges will be able to view session properties via the Session property dialog. However, this dialog will only be made available in read-only mode when a user has session operator privileges. Workspace Search A user can search for a port name within a selected object, a port name across all objects in the workspace, or a name of an object on the workspace. Show Invalid Mappings Invalid mappings will be distinguished with a different icon in the Navigator outline. Copy As (Mappings and Maplets) A user can save a mapping or maplet under a different name. Compare Folder Repository Manager provides a "compare" option in the Folder menu. The user can compare the contents of two folders and get a listing of the discrepancies.

PowerCenter 5 and PowerMart 5

Resizable SQL Editor and Expression Editor A user can resize the dialog box for the SQL Editor (for SQL Override) and also the Expression Editor. Global Session Invalidation A user can choose a folder or an individual session and select "Validate Session" from the menus. In the case of validation at the folder level, the user can choose any number of the invalid sessions to validate all at once. Shortcut Keys A user can use keyboard commands to perform certain common functions within the Designer client tool. Folder open on double-click A user can double-click on a folder icon in the navigator outline tree and expand the outline tree to show the contents of the folder. Enhanced overall look Users will notice an improved user interface appearance in the form of gradation shading on the title bars, and 3D beveled borders on transformation objects and links.

PowerMart Training - Course Contents

Session I
Introduction to PowerMart and its components PowerMart Architecture System requirements Overview of PowerMart Repository Manager Creating Repository Creating folders Creating Users, Groups and assigning privileges

Session II

Viewing/removing locks Generating metadata reports Import/export registry Overview of PowerMart Designer Create/import source in Source analyzer Create/import target in Warehouse designer Understanding Transformation Objects Suggested naming conventions

PowerMart Training - Course Contents

Session III
Create transformation objects, mappings in Mapping designer Mapping wizards Create mapplets in Mapplet designer Overview of Server Manager What is a session/batch? Configure Server Manager Adding database connections

Session IV

Create sessions Create batches Run a session/batch Performance tuning techniques Case study Quiz

Session I
Objective
Familiarize with Informatica Powermart and its components Hands on with Repository Manager

What will you learn?

Creating Repository Creating Folders Creating users Creating user groups Assigning privileges

Overview of Informatica PowerMart

A complete suite of components for building a datamart/datawarehouse Provides ability to source large volumes of fast changing data from multiple platforms Handles complex transformations Supports high speed loads Metadata repository co-ordinates and drives the extraction, transformation, loading and management

Components of PowerMart
PowerMart Designer
Multi-faceted tool for visually defining mappings and transformations

PowerMart Repository
An open metadata store for definitions about mappings, transformations and other data mart details

PowerMart Repository Manager

A facility for managing user activities and metadata storage in the repository

PowerMart Server
A pipelined, multi-threaded server engine that is able to overlap data extraction, transformation and loading

PowerMart Server Manager

An administrative interface for configuring and scheduling extract/transform/load sessions

Informatica PowerMart Architecture

System Requirements

PowerMart Repository Manager

Overview of Repository Manager

Used for administering the repository Allows to do the following tasks
Perform Repository functions
- Create,backup,copy,restore,upgrade, and delete repositories

Perform Folder functions

- Create, edit and delete folders - Copy a folder within a repository or to other repository

Implement Repository security

- Create, edit and delete repository users and user groups - Assign and revoke repository privileges and folder permissions

- View locks and un-lock objects, versions and folders

Overview of Repository Manager

Add and remove repository reports Import and export repository connection information in the registry Analyze source/target, mapping dependencies View the properties of repository objects

PowerMart Repository Manager Workspace

Menu Bar

Std. Tool Bar

Navigator

Dependency

Status Bar

Main

Creating the Repository

Create an ODBC data source to connect to the database where you want the repository tables to be created Open Repository Manager module Click on Create Repository

Enter the Repository name,database username and password and Select the ODBC data source created previously and finally the native connect string and click OK. The list of tables created for the repository will be visible in the Output window

Creating the Folder

Click on Folders>Create Enter folder name, description, owner, group and permissions

Creating Groups
Connect to a repository To create Groups choose Security>Manage Groups Click Add

Enter the name of the new group and a description

Click OK

Creating Users
To create Users choose Security>Manage Users

Click Add
Enter a username Enter the password

twice to confirm it

Creating Users
Click Group Memberships

To Add the user to a group, select the group in the Not Member list, and click Add To Remove the user from a group, select the group in the Member List, and click Remove

Assign Privileges
Choose Security> Manage Privileges

Select a privilege from the Privilege list

Use Designer Browse Repository Create Sessions and batches Session Operator Administer Repository Administer Server Super User

Assign Privileges
Click Add
Repository groups without the selected privilege appear

Click Show Users to view the repository users

Select the user or the group

Click Add Click OK to save changes

Session II
Objective
Familiarize with Repository Manager Familiarize with Designer

What will you learn?

Viewing/Removing locks Generating metadata reports Import/export registry Creating/Importing source using Source Analyzer Creating/Importing target using Warehouse Designer Types of transformation objects

Types of ports
Naming conventions for different objects

View/Remove Locks
Locks prevent the users from duplicating or overriding work Choose Edit > Show Locks to view all the locked objects

The dialog box shows

Username locking the object Folder containing the object Version containing the object Object type(folder, session, reusable transformation etc.) Object name Lock type(Read,Write,Execute,Save,Fetch) Lock time, hostname,Application(Server Manager, Designer, Server etc.)

Generate Reports
Types of Default Reports
Mapping report source/target dependencies report

Target table report

executed session report

To add a report choose Report>Add Report

Generate Reports
Click on Add to add a report to the installed report list

Select from the default reports available or custom reports created using Crystal reports and click OK To view or print a report select it from the Reports menu Select Print Preview to view the report

Generate Reports
Provide username, password and ODBC data source name

Click Print Preview and select the wildcard character if reports for all the folders, tables and versions have to be viewed

Import/Export Registry
Purpose- To simplify the process of setting up client systems The registry contains the following connection information
Repository name Database user name and password Repository user name and password ODBC data source name(DSN)

Dont forget to create the DSN before importing the registry as it does not include the ODBC data source itself

Import/Export Registry
To export the registry, choose Tools>Export Registry

In the dialog box enter the name for the file and Save To import the registry, choose Tools>Import Registry Select the file and Open it A dialog box confirms the merging of data source information

PowerMart Designer

Overview of PowerMart Designer

Designer provides five tools with which to create mappings
Source Analyzer
- to import or create source definitions for flat file, ERP, and relational sources

Warehouse Designer
- to import or create target definitions

Transformation developer
- to create reusable transformations

Mapplet designer
- to create mapplets

Mapping designer
- to create mappings

Designer allows to work in multiple folders and repositories at a time

Designer Work Space

Menu Bar Std. Tool Bar

Navigator

Source Analyzer

Transformation developer Mapplet Designer

Mapping Designer

Warehouse Designer

Output

Workspace

Status Bar

Source Analyzer
Reads, analyzes and "reverse engineers" schema information of operational databases and flat files Stores metadata information in the repository

How to import a source?

choose Sources > Import from database/flat file/cobol file To import a table enter ODBC data source, username, password

Source Analyzer
Click on Connect button and the list of tables appears Select the tables which you will be using as source tables Click OK to add the selected tables into the Source analyzer work space

How to create a new Source?

Choose Sources > Create

Give a name to the new source, select the database type Click Create and then Done buttons to make the blank source appear in workspace

Source Analyzer
To enter column names, data types and field lengths double click on the newly created source structure Click on the Columns tab

Click on the Add a new column button to add new columns in the source and specify the details Click OK to accept changes

Warehouse Designer
Provides the following features
Create a new target Import the target structure

Create cubes and dimensions for multi-dimensional metadata analysis

How to create a new target?

Choose Targets > Create Enter the name for the target table and the database type Double click on the empty target structure and enter the column names, data type and field length same as done for creating sources

Select the newly created table and choose from menu bar Target >Generate/Execute SQL

Warehouse Designer

Connect to the warehouse by giving ODBC data source,user id and password Select the appropriate Generation options and click on Generate SQL file button To view/ edit the SQL click on Edit SQL file

To generate the tables in the database click on Execute SQL file

Look at the Output window to check for errors in generation of the target table in database

Warehouse Designer
How to import target table definition?
Choose Target > Import from database Connect to the database by selecting the ODBC data source and the enter the user name and password

Select tables from the list of tables available in the database which will be used as targets Click OK to get the tables in the workspace

Mapping Designer
Visual aid to creating and editing source-to-target mappings. Dataflow diagramming.
Method of creating dataflow links through combinations of PowerMart 4.6's transformation objects.
Sources, targets and transformation objects can be dragged and dropped into a workspace to construct the transformation pipeline.

Transformation Objects
To create a transformation
Click on Transformation > Create

Transformation Objects
Select the transformation object and give a name to it Click on Create and then Done

Aggregator
Performs an aggregate calculation(Count, Average etc.)

Transformation Objects
Expression
Perform custom calculations of a simple or complex nature, using data from one or more input ports

Transformation Objects
Filter
Performs a test on all records before allowing them to be sent to the next object

Transformation Objects
Joiner
Joins data from disparate sources, such as mainframes, flat files and relational databases

Transformation Objects
Lookup
Looks up values

Transformation Objects
Sequence Generator
Generates unique ID values in the same fashion as a sequence in a relational database

Transformation Objects
Source Qualifier
Represents data temporarily stored on the PowerMart server

Transformation Objects
Stored Procedure
Calls a stored procedure and captures return values

Transformation Objects
Update Strategy
Defines how the PowerMart server should handle updates to existing records in targets

Transformation Objects
Rank
Performs comparisons and groupings

Suggested Naming Conventions

Transformations
Aggregator Expression AGG_TransformationName EXP_TransformationName

Filter
Joiner Lookup Normalizer Rank Sequence Generator Stored Procedure Source Qualifier Update Strategy

FIL_TransformationName
JNR_TransformationName LKP_TransformationName NRM_TransformationName RNK_TransformationName SEQ_TransformationName SP_TransformationName SQ_TransformationName UPD_TransformationName

Suggested Naming Conventions

Targets
T_TARGETNAME

Mappings
m_MappingName

Mapplets
mplt_MappletName

Sessions
s_MappingName

Sequential Batches
bs_SequentialBatchName

Concurrent Batches
bc_SequentialBatchName

Mapping Designer
Ports
To design the basic flow of data between source and targets Types: input, output, variable

Variable Ports
For aggregator, expression and rank transformations Use variable to
simplify complex expression store temporary data store values from prior rows

Provides Wizards to create simple pass through mappings/Slowly changing dimensions

Session III
Objective
Hands on with Designer Familiarize with Server Manager

What will you learn?

Creating Mapping Types of Mapping wizards Mapplet designer Creating Mapplets Configuring Server Manager Adding database connection

Mapping Creation
How to create a Mapping?
Open the Mapping designer workspace Choose Mappings > Create

Give a proper name to the mapping

Open the Sources from the navigator which you would have created/imported using source analyzer Drag and drop the source table from navigator into the workspace

Mapping Creation

Mapping Creation
Choose Transformation > Create to create a transformation object Select the type of transformation object you want to create and give a proper name to it

Mapping Creation
Drag and drop the required fields(ports) from Source Qualifier to the transformation object

Mapping Creation
Add new ports in the transformation object and define them as variable ports to do complex transformation

Mapping Creation
Open the Targets from the navigator which you would have created/imported using Warehouse Designer Select the target table, drag and drop it into the designer workspace

Mapping Creation
Drag and drop the output ports from the last transformation object to the corresponding ports in the target

Choose Repository > Save to store the mapping Check the output window for any errors

Mapping Wizards
Wizards help to create mappings quickly and easily Wizards designed to create mappings for loading and maintaining star schemas

Mappings created using wizards can be edited and customized

Getting Started Wizard
Simple Pass Through
mappings to load static fact and dimensions Use this when you want to drop all existing data from the table before loading new data

Slowly Growing Target

mappings to load slowly growing dimension or fact tables Use this to load new data when existing data does not require updates

Mapping Wizards
Slowly Changing Dimensions Wizard
Type I Dimension Mapping
- Keep most recent values in target

Source
Emp id Name Email Emp id

Target
Name Email

1001

Shane

Shane @xyz.c om

1001

Shane

Shane @xyz.c om

Source
Emp id Name Email Emp id

Target
Name Email

1001

Shane

Shane@ abc.co.in

1001

Shane

Shane@ abc.co.in

Shane@xyz. com

Mapping Wizards
Slowly Changing Dimensions Wizard
Type II Dimension Mapping
Version Data Mapping
- insert new and changed dimensions with version number and incremented primary key - full history and progress of changes

Target Source
Emp id 10 Name Shane Email Shane @xyz. com
PM_ PRIM ARY KEY 1000 Emp id Name Email PM_V ERSI ON_ NUM BER 0

10

Shane

Shan e@xy z.co m

Type II Dimension/Versioning
Source
Emp id 10 Name Email

Shane Shane@ abc.co.in

PM_PRI MARYKE Y 1000 1001

Emp id 10 10

Name

Email

PM_VERSION_N UMBER 0 1

Shane Shane

Shane@ xyz.com Shane@ abc.co.in

Target

Type II Dimension/Versioning
Source
Emp id 10 Name Email

Shane Shane@ abc.com

PM_PRI MARYKE Y

Emp id 10 10 10

Name

Email

PM_VERSION_N UMBER 0 1 2

Target

1000 1001 1003

Shane Shane Shane

Shane@ xyz.com Shane@ abc.co.in Shane@ abc.com

Mapping Wizards
Slowly Changing Dimensions Wizard
Type II Dimension Mapping
Flag Current Mapping
- insert new and changed dimensions with flags and incremented primary key - full history and flagging only current dimensions

Target Source
Emp id 10 Name Shane Email Shane @xyz. com
PM_ PRIM ARY KEY 1000 Emp id Name Email PM_CU RRENT _FLAG

10

Shane

Shan e@xy z.co m

Type II Dimension/Flag Current

Source
Emp id 10 Name Email

Shane Shane@ abc.co.in

PM_PRI MARYKE Y 1000 1001

Emp id 10 10

Name

Email

PM_CURRENT_F LAG N Y

Shane Shane

Shane@ xyz.com Shane@ abc.co.in

Target

Type II Dimension/Flag Current

Source
Emp id 10 Name Email

Shane Shane@ abc.com

PM_PRI MARYKE Y

Emp id 10 10 10

Name

Email

PM_CURRENT_F LAG N N Y

Target

1000 1001 1003

Shane Shane Shane

Shane@ xyz.com Shane@ abc.co.in Shane@ abc.com

Mapping Wizards
Slowly Changing Dimensions Wizard
Type II Dimension Mapping
Effective Date Range Mapping
- insert new and changed dimensions with date range to define current dimension data - full history and tracking changes with an exact effective date range

Source
Emp id 10 Name Email

Target
PM_ PRIM ARY KEY
1000

Emp id

Name

Email

PM_B EGIN _DAT E

01/0 1/00

PM_E ND_ DATE

Shane Shane @xyz. com

10

Shane

Shan e@xy z.co m

Type II Dimension/Effective Date

Source
Emp id 10 Name Email

Shane Shane@ abc.co.in

PM_PRIMA RYKEY 1000

Emp id

Name

Email

PM_BEGIN _DATE 01/01/00

PM_END _DATE 03/01/00

10

Shane

Shane @xyz. com Shane @ abc.co .in

1001

10

Shane

03/01/00

Target

Type II Dimension/Effective Date

Source
Emp id 10 Name Email

Shane Shane@ abc.com

PM_PRI MARYKE Y 1000 1001 1003

Emp id 10 10 10

Name

Email

PM_BEGIN _DATE 01/01/00 03/01/00 05/02/00

PM_END_ DATE 03/01/00 05/02/00

Shane Shane Shane

Shane@ xyz.com Shane@ abc.co.in Shane@ abc.com

Target

Mapping Wizards
Slowly Changing Dimensions Wizard
Type III Dimension Mapping
- insert new and update values in existing dimensions

- keep current and previous values (partial history)

Target Source
Emp id 10 Name Email
PM_P RIMA RYKE Y 1 Emp id Name Email PM_P rev_ Colu mnN ame PM_E FFECT _DAT E 01/01 /00

Shane Shane @xyz. com

10

Shane

Shane @xyz. com

Type III Dimension

Source
Emp id 10 Name Email

Shane Shane@ abc.co.in

PM_PRIMA RYKEY 1

Emp id

Name

Email

PM_Prev_ ColumnNa me Shane@xy z.com

PM_EFFE CT_DATE 01/02/00

10

Shane

Shane @ abc.co .in

Target

Type III Dimension

Source
Emp id 10 Name Email

Shane Shane@ abc.com

PM_PRI MARYKE Y 1

Emp id 10

Name

Email

PM_Prev_ ColumnNa me Shane@ abc.co.in

PM_EFFEC T_DATE 01/03/00

Shane

Shane@ abc.com

Target

Mapplet Designer
Mapplet
Reusable object that reflect set of transformation logic to use in multiple mappings

Like a single transformation object

Expands at run time

Each Mapplet must include the following

Exactly one input transformation or Source Qualifier transformation
At least one output transformation

Rules for Objects in Mapplets

Use only reusable Sequence Generators Do not use pre- or post-session stored procedures

Mapplet Designer
Rules for Objects in Mapplets
Do not use the following in a mapplet
Joiner Cobol Source definition Normalizer Target definitions

Mapplet Designer
How to create a Mapplet?
Open the Mapplet designer workspace Choose Mapplets > Create

Give a name to the Mapplet

Create an input transformation to define mapplet input ports if the mapplet contains no sources

Creating a Mapplet

Double click on the object to define the input ports

Creating a Mapplet
Create the transformation objects to be used in the mapplet One input transformation can be connected to only one transformation, so to pass same values to two separate data flows,connect the input transformation to another transformation and then split the data flow Use Output transformation to create output ports, creating one Output transformation for each mapplet output group

Connect all the input ports, ports in transformation objects and the output ports to complete the data flow

Creating a Mapplet

Choose Repository > Save to store the mapplet Check the output window for validation status If mapplet is not valid,correct the problem and re-save the mapplet

PowerMart Server Manager

Overview of Server Manager

Tasks performed in Server Manager
Configure the Informatica Server Configure Database, External loader and FTP connections

Manage sessions and batches

Create,edit,validate,and delete sessions and batches Copy and move sessions and batches within a folder Start and Stop sessions and batches, and abort sessions View session logs,session details and session performance details

Monitor, add, edit, and delete Informatica server information in the repository

Stop the Informatica Server

Customize Server Manager

Server Manager
Sessions
set of instructions that tell Informatica Server how and when to move data from sources to targets

Batch
Group of sessions Types of batch
Sequential
Runs the sessions one after the other

Concurrent
Runs all the sessions at the same time

Server Manager Workspace

Menu Bar Std. Tool Bar

Navigator

Monitor

Output

Configure

Status Bar

Configuring Server Manager

How to Register the Server?
Make sure the Informatica Server is running In Server Manager connect to the repository

Choose Server configuration > Register Server

In the dialog box that appears select the network protocol (TCP/IP or IPX/SPX) and enter the following settings
Server Name

Host Name or IP address

Port Number Timeout

Configuring Server Manager

To enter server variables,click the Advanced button

Enter a valid root directory for $PMRootDir

Enter the other server variables, as desired Click OK to save your changes

Server Manager
How to add database connections?
Connect to a repository Choose Server Configuration > Database Connections

Click Add
Enter the following information

Click OK to add this connection to the Data Sources list Click Close to save all the changes

Session IV
Objective
Hands on with Server Manager Understand how to tune performance in Informatica

Test your knowledge gathered through this training

What will you learn?

Creating sessions Creating batch Running a session/batch Performance tuning features Implementing case study

Creating Session
How to create a Session?
Click on the folder in the navigator which contains the mapping

Creating Session
Select Operations > Add Session from menu bar

Select the mapping for which session has to be created

Click OK

Creating Session
Session wizard will appear

Select the Source type, how to treat rows and the database connection name for relational sources
For target, select the type and the database connection name(if relational)

Click Next

Creating Session
On the time page, enter a schedule for the session

Click Next

Creating Session
On the log files page, enter the settings for Session Log File name, Stop on(after no. of errors), Perform recovery, override tracing etc.

Click Next

Creating Session
On the Transformations page, enter override mapping or mapplet transformation attributes as needed

Click Finish to create the session

Creating Batches
How to create a batch?
Select the folder in the navigator for which sessions have been created Choose Operations > Add batch

Enter the Batch name

Creating Batches

Specify whether a concurrent one Enter the schedule for the batch

Click OK to add the batch to the folder

To add sessions to the batch, drag and drop the required sessions onto the batch icon

Run a Session/Batch
Select the session/batch to be run from the folder in the navigator Make sure the Monitor option of the server is checked

Click on the Start a session/batch icon

Check the output window to make sure the server acknowledged the request to run the session/batch Click on the Poll sessions icon to check the status of the run in the monitor window Double click on the session in the monitor window Click on the Refresh continuously button to monitor whether the session is running, got completed successfully or got failed

Run a Session/Batch

Read the number of rows loaded, failed or the first error message from the dialog box Click on the Open Log File button to dig into why a session failed

Performance Tuning
Check Collect Performance data option in the session properties of a session in Server Manager

Performance Tuning
Source Qualifier
Buffer Input efficiency BufferOutput efficiency

Target

Evaluation

Buffer Input efficiency

Buffer Output efficiency

High

Low

High

Low

Source database slow, eliminate read bottleneck Target database slow, eliminate write bottleneck DTM slow, optimize session or mapping

Low

Low/High

Low

High

Low

High

High

Low

Efficiency Percentage High= 80-100% Low= 0-20%

Performance Tuning
The counters help identify
Read/Write/DTM bottlenecks Caching problems

Transformation errors
Shared memory allocation problems

To avoid read bottleneck

Reduce the number of records processed by filtering the source Index the source Replace default query with an optimized query in SQL override of Source Qualifier

Performance Tuning
To avoid write bottleneck
Utilize SQL loader facility of the database Drop indexes before load and rebuild after the load Increase the database block size

To avoid DTM bottleneck

Reduce the amount of detail in the session logs

Performance Tuning
To avoid DTM bottleneck
Run parallel sessions in Concurrent batches Use incremental aggregation for mappings that use aggregation

Performance Tuning
To avoid DTM bottleneck
Optimize mapping Optimize session

For cache problems

Increase index and data cache

Click

Performance Tuning
For shared memory allocation problems
Increase the shared memory size

Performance Tuning
To optimize sessions
Increase shared memory size Increase buffer block size for very large row sizes

Consider the commit interval. More commits=slower performance

Disable decimal arithmetic if possible

Performance Tuning
To optimize mapping
Utilize single pass reads, use SQL override Place filters, aggregators as close to source as possible

Trim spaces before comparing strings

Operators are faster than functions,i.e.,|| vs Concat Cache lookup table columns

Advanced server Features

Data Transformation Transformation process process

Session shared memory 64k..

Reader

DTM

Writer

Advanced server Features

Session memory is divided into buffers Buffer are used by the reader to store data from the source By the DTM, store the transformed data By the writer, transformed data will be written to the target

Advanced Server Features

E-Mail configuration Send email message after a session run variables %s session name %l total rows loaded %r total rows rejected %e session error status %b session start time %c session elapsed time %g attaches the session log to the message

Case Study
Case I Product data is captured in two platforms. One in relational table and other in flat file. Combine data from these two sources and put them into the Product dimensional table Case II In a data warehouse, we create surrogate keys for defining the primary keys in dimension tables. Create surrogate keys for the product table created in previous case. Populate the fact table with the surrogate keys created in product dimension table as the foreign keys

Case Study
Case III The Employee table has data for employees situated in all the countries.

Whenever new employee data gets added or the existing data gets modified into the source table, the new/modified data needs to be loaded into the Employees dimension.
Create a mapping which checks for new/changed data in the source and loads only those records into the target

Case Study
Case IV
Create the following tables in the source database

1. Customers
Customer_id, Country Customer_name, Address, City, State

2. Employees
Employee_id, Address 3. Products Product_id, Product_name, item, unitprice First_name, last_name, Designation

Case Study
4. Orders order_id, customer_id, employee_id, order_date required_date 5. Order details order_id, product_id, unitprice, quantity

discount
In the target side, 1. create the dimension table for Customer,products and employees by adding an extra surrogate key columns respectively.

Case Study
2. Design a mapping for loading the data on the above mentioned tables by using slowly changing dimension wizard. Try to make use of Type1,Type2 and Type3 3. Create order_fact in the target db for loading orders. Design a mapping for loading the data to handle insert and update strategies. Populate customer_key and employee_key instead of populating customer_id,employee_id by using lookup transformations. 4. Create a order_details_fact table with the following measures Order_id, no_of_products, Tot_qty, Tot_price Try to make use of aggregator transformation.

Quiz
1. Where do you generate reports on Metadata?
a. Designer b. Repository manager

c. Server manager
d. Server

2. What is the use of cube and dimensions in the designer?

a. Metadata analysis b. OLAP tool analysis

c. Report generation
d. Creating sessions

Quiz
3. When only one object of an entire mapping is reusable it is called?
a. Reusable transformation

b. Mapplet
c. Repeat transformation d. Duplicate transformation

4. When a group of objects in a mapping are reusable it is called?

a. Reusable transformation b. Mapplet c. Repeat transformation d. Duplicate transformation

Quiz
5. Where do you copy a mapping from one folder to another folder?
a. Designer

b. Repository manager
c. Server manager d. Server

6. Where do you copy a session from one folder to another folder?

a. Designer b. Repository manager c. Server manager d. Server

Quiz
7. How many repositories can you create in one database?
a. 1

b.
c. d.

2
3 Any number

8.
a. b. c. d.

Where do you clear the locks on the objects in Informatica?

Designer Repository Manager Server Manager Not possible

Quiz
9. Where do you create folders?
a. b. Designer Repository Manager

c.
d.

Server Manager
None of the above

10. A mapplet must include the following

a. b. At least one input transformation, at least one output transformation Exactly one input transformation, at least one output transformation

c.
d.

No input transformation, at least one output transformation

At least one input transformation, no output transformation

Quiz
11. A mapplet should not have the following transformation
a. Source Qualifier

b.
c. d.

Joiner
Expression Aggregator

12. Where do you specify 'Constraint based load ordering'?

a. b. c. d. Designer Repository Manager Server Manager None of the above

Quiz
13. How much is the default size for the index cache?
a. b. 100 MB 10 MB

c.
d.

1 MB
100 KB

14. How much is the default size for the data cache?
a. b. c. 100 MB 200 MB 2 MB

d.

100 KB