Beruflich Dokumente
Kultur Dokumente
Table of Contents
Overview ..........................................................................
...................................................................................
.............. 3 Implementation
Scenario ..........................................................................
........................................................................ 3 Before
You
Begin..............................................................................
................................................................................ 4
PowerExchange
Installation ......................................................................
................................................................... 4 Information
Required for CDC
Configuration......................................................................
.......................................... 5 Oracle CDC Restrictions and
Requirements ......................................................................
.......................................... 7 Create Target
Tables.............................................................................
....................................................................... 7 Oracle
Configuration
Tasks .............................................................................
................................................................. 8 Step 1. Specify
an Archive Log
Destination .......................................................................
.......................................... 8 Step 2. Set the Oracle
transaction_auditing
Parameter .........................................................................
...................... 8 Step 3. Enable ARCHIVELOG
Mode ..............................................................................
............................................. 8 Step 4. Stop and Restart the Oracle
Database ..........................................................................
.................................. 8 Step 5. Grant the User Privileges Required for
CDC ...............................................................................
.................... 8 Step 6. Configure Oracle Minimal Global Supplemental Logging
................................................................................ 9
Step 7. Create a Table Space Exclusively for Oracle LogMiner Use
(Optional)........................................................... 9 Step 8.
Copy the Oracle Catalog to the Archived
Logs ..............................................................................
................ 10 PowerExchange Configuration
Tasks..............................................................................
............................................... 10 Step 1. Customize the
dbmover.cfg File on
UNIX ..............................................................................
........................ 11 Step 2. Customize the PowerExchange Logger pwxccl.cfg
File ..............................................................................
.. 16 Step 3. Start the PowerExchange
Listener...........................................................................
...................................... 19 Step 4. Customize the dbmover.cfg Files
on the Windows and PowerCenter Integration Service Machines............ 20 Step 5.
Create Capture
Registrations......................................................................
................................................... 20 Step 6. Add Change Indicator
or Before Image Columns to Extraction Maps
(Optional)........................................... 22 Step 7. Perform a Database
Row Test of the Extraction
Maps ..............................................................................
.... 23 Step 8. Materialize the Target
Tables.............................................................................
............................................ 23 Step 9. Start the PowerExchange
Logger ............................................................................
...................................... 23 Step 10. Allow Changes to Be Written to
the Source
Tables ............................................................................
......... 24 PowerCenter Configuration
Tasks .............................................................................
..................................................... 24 Step 1. Import Extraction
Maps for Source
Tables.............................................................................
........................ 24
2
Step 2. Import Target Table
Definitions........................................................................
.............................................. 25 Step 3. Create a
Mapping............................................................................
............................................................... 25 Step 4.
Configure an Application Connection for the Oracle
Sources........................................................................ 25
Step 5. Create a Workflow and
Session............................................................................
......................................... 26 Step 6. Configure the Restart Token
File ..............................................................................
..................................... 29 Step 7. Cold Start the CDC
Workflow...........................................................................
.............................................. 29 Additional
Information .......................................................................
.............................................................................. 29
Overview
This article summarizes the steps that are necessary to successfully implement
PowerExchange 8.6.1 Oracle LogMiner CDC for multiple Oracle source tables in a non-
RAC environment on UNIX. It includes example configuration files and best
practices. PowerExchange Oracle LogMiner CDC uses Oracle LogMiner to read change
data from Oracle redo logs. To move the change data to target tables, PowerExchange
uses the PowerExchange Client for PowerCenter (PWXPC) in conjunction with
PowerCenter. To implement Oracle CDC, you need to perform configuration tasks in
Oracle, PowerExchange, and PowerCenter. Often, these tasks are performed by more
than one person, for example, the Oracle DBA, target RDBMS DBA, system
administrator, PowerExchange administrator, and PowerCenter developer. This
implementation uses the PowerExchange Logger for Linux, UNIX, and Windows, which is
optional for Oracle LogMiner CDC. The PowerExchange Logger captures change data
from Oracle redo logs and writes only the successful units of work (UOWs), in
chronological order based on commit time, to PowerExchange Logger log files. The
change data is extracted from the PowerExchange Logger log files in continuous
extraction mode to provide nearreal-time access to the data on the target machine.
The benefits of using the PowerExchange Logger include fewer database accesses,
faster CDC restart, and no need to prolong retention of Oracle redo files for
change capture. Note: Informatica strongly recommends that you use the
PowerExchange Logger for Oracle LogMiner CDC. If you use real-time extraction mode
without the PowerExchange Logger, PowerExchange starts a separate Oracle LogMiner
session for each extraction session. Running multiple, concurrent LogMiner sessions
can significantly degrade performance of the system where LogMiner runs. Although
this article focuses on UNIX, the information herein also applies to Oracle sources
on Linux and Windows.
Implementation Scenario
Scenario: You need to keep a data warehouse synchronized with an Oracle production
database in near real time. This scenario makes the following assumptions about
your CDC environment and requirements: y y y y y You are using PowerExchange CDC
8.6.1 or later with Oracle 10g or 11g. You need to capture changes from about 50
Oracle source tables that are on a UNIX machine. The UNIX machine is remote from
the PowerCenter Integration Service machine. The PowerExchange Navigator and
PowerCenter Client reside on the same Windows machine, which is remote from the
PowerCenter Integration Service machine. The Oracle source tables are not in an
Oracle Real Application Cluster (RAC) environment. You must move change data in
near real time to multiple target tables on a remote machine.
3
y y
You run the PowerExchange Logger for Linux, UNIX, and Windows on the Oracle source
machine to capture change data continuously to PowerExchange Logger log files. You
extract change data from the PowerExchange Logger log files in continuous
extraction mode. Continuous extraction mode provides near real time access to
change data but is much less resource intensive than real-time extraction mode. The
Oracle source machine has sufficient memory and resources to handle CDC efficiently
without offloading PowerExchange Logger processing. You use PWXPC connections and
PowerCenter workflows to extract the change data from the PowerExchange Logger log
files and to write the data to multiple target tables. Note: PWXPC is a native
plug-in that is installed with PowerCenter on the Windows and PowerCenter
Integration Service machines.
y y
Gather the information about your Oracle source and environment that is required
for CDC configuration. Review CDC limitations and restrictions. Create the target
tables.
PowerExchange Installation
PowerExchange installation on UNIX and Windows is all-inclusive. You do not install
PowerExchange components separately.
4
On the UNIX machine with the Oracle source tables, extract files from the 32-bit or
64-bit PowerExchange tar file for your UNIX operating system. Then run install.sh
to perform the installation. After installation completes, set the following
environment variables: y y y Set the PWX_HOME environment variable to point to the
PowerExchange installation directory. Edit the PATH and LIBPATH environment
variables to append the PowerExchange installation directory. Verify that the LANG
and LC_ALL environment variables are set to “C.” If these environment variables are
not set to “C,” ask your UNIX administrator to change them.
On the Windows machine, run the PowerExchange 32-bit executable. You must install
the Windows 32-bit executable to use the PowerExchange Navigator, even on a Windows
64-bit machine. On the PowerCenter Integration Service machine, follow the
PowerExchange installation instructions for the operating system type and bit-
level. If the Integration Service runs on a 64-bit machine, use the PowerExchange
64-bit installation executable or tar file. If the Integration Service runs on a
32-bit machine, use the PowerExchange 32-bit executable or tar file. For more
information about UNIX and Windows installation procedures, see the PowerExchange
Installation Guide.
Table space for Oracle LogMiner Datafile for the Oracle LogMiner table space User-
defined name for the LISTENER statement, which is used to start or stop the
PowerExchange Listener on the UNIX machine Port number that the PowerExchange
Listener uses on the UNIX machine User-defined name for the NODE statement that is
used to connect to the UNIX machine
5
Information Host name for the UNIX machine
Your Value
• Path to PowerExchange message log files on the UNIX machine Subdirectory of the
LOGPATH directory for alternative message log files for the PowerExchange Logger
Path to the directory that stores the CCT and CDCT files on the UNIX machine. These
files store information about capture registrations and PowerExchange Logger log
files. Path to the directory that stores extraction maps on the UNIX machine User-
defined collection ID for an ORACLEID statement for the Oracle source instance.
Maximum length is eight characters.
LOGPATH statement in dbmover.cfg on the UNIX machine PFX option of the TRACING
statement in dbmover.cfg on the UNIX machine CAPT_PATH statement in dbmover.cfg on
the UNIX machine
/pwx/logs altlogs
/pwx/capture
/pwx/capture/xtramaps
ORACOL01
Also specified in: • Collection ID field when you create a registration group in
the PowerExchange Navigator, or Instance field when you view registration group
properties. DBID parameter in pwxccl.cfg
6
Information Oracle database name
Your Value
A database service name that PowerExchange uses to connect to the Oracle database
orcl01.world
Oracle schema.table names for the source tables, and the columns from which to
capture change data Path and unique prefix to use for generating the PowerExchange
Logger log files. To create the file names, the Logger appends the following:
.CND.CPyymmdd.Thhmmssnnn
pwx/pwxccl
Path and base name to use for generating the PowerExchange Logger checkpoint files
Retention period, in days, for CDCT records and PowerExchange Logger log files
Database user ID Database password or encrypted password
pwx/pwxccl.chkpt
30
oracapt oracapt
PowerExchange does not capture changes from columns that have certain datatypes
such as LOB or LONG. For more information about datatype restrictions, see the
PowerExchange CDC Guide for Linux, UNIX, and Windows.
7
Oracle Configuration Tasks
To configure Oracle for CDC, use the sample Oracle configuration script file,
oracapt.sql, which is located in the PowerExchange installation directory. This
script file contains sample SQL statements and comments to help you complete the
required Oracle configuration steps.
8
GRANT "CONNECT" TO "ORACAPT";
3.
Issue the following GRANT statements to provide the minimum privileges for running
Oracle CDC to the user “ORACAPT” :
GRANT SELECT ON "PUBLIC"."V$INSTANCE" TO "ORACAPT"; GRANT SELECT ON
"PUBLIC"."V$DATABASE" TO "ORACAPT"; GRANT SELECT ON "SYS"."DBA_LOG_GROUPS" TO
"ORACAPT"; GRANT SELECT ON "SYS"."DBA_LOG_GROUP_COLUMNS" TO "ORACAPT"; GRANT SELECT
ON "PUBLIC"."V$TRANSACTION" TO "ORACAPT"; GRANT SELECT ON "PUBLIC"."V$ARCHIVED_LOG"
TO "ORACAPT"; GRANT SELECT ON "PUBLIC"."V$LOGMNR_CONTENTS" TO "ORACAPT"; GRANT
SELECT ON "PUBLIC"."V$PARAMETER" TO "ORACAPT"; GRANT SELECT ON
"PUBLIC"."V$NLS_PARAMETERS" TO "ORACAPT"; --- If you are using Oracle 10g or later,
include the following statement: -- GRANT SELECT ANY TRANSACTION TO "ORACAPT";
-GRANT EXECUTE ON "SYS"."DBMS_FLASHBACK" TO "ORACAPT"; GRANT EXECUTE ON
"SYS"."DBMS_LOGMNR_D" TO "ORACAPT"; GRANT EXECUTE ON "SYS"."DBMS_LOGMNR" TO
"ORACAPT";
Note: The GRANT LOCK ANY TABLE statement is not required because this scenario uses
GENRLOCK=N for the ORCL CAPI_CONNECTION. With this setting, PowerExchange does not
lock Oracle tables while creating restart points. 4. Issue the following GRANT
statement if you plan to use the Oracle user ID that you specified in the CREATE
USER statement to create capture registrations and to automatically run the DDL
that PowerExchange generates for creating supplemental log groups at registration
completion:
GRANT ALTER ANY TABLE TO "ORACAPT";
For a description of the minimum required Oracle system privileges and object
privileges, see the CDC Guide for Linux, UNIX, and Windows.
Note: You must also define a supplemental log group for each Oracle source table.
When you register an Oracle table in the PowerExchange Navigator, PowerExchange
generates DDL for adding a supplemental log group for the table.
Step 7. Create a Table Space Exclusively for Oracle LogMiner Use (Optional)
Create a table space exclusively for Oracle LogMiner use to prevent the SYSTEM
table space (for Oracle version 9i) or SYSAUX table space (for Oracle 10g or later)
from becoming full during CDC. This step is necessary only if you have not
previously configured LogMiner for use with other Oracle features such as logical
standby databases, Oracle Streams, or native Oracle change capture process. 1. To
create the table space, issue the following DDL:
CREATE TABLESPACE "LOGMNRTS" NOLOGGING DATAFILE '/oracle_path/datafilename.ora'
SIZE 50M REUSE AUTOEXTEND ON NEXT 10M MAXSIZE 100M EXTENT MANAGEMENT LOCAL;
9
Notes: y Specify NOLOGGING if you use Oracle LogMiner only for PowerExchange CDC
and an occasional query. Change NOLOGGING to LOGGING if you use any of the
following Oracle features: logical standby databases, Oracle Streams, or native
Oracle change capture process. For the DATAFILE value, specify a file name based on
your local Oracle database file naming standards for the data files that comprise
this table space.
y 2.
If this statement fails with the ORA_01353 message, see the comments in oracapt.sql
for more information. 3. To recompile the SYS.DBMS_LOGMNR_D package, enter the
following command:
ALTER PACKAGE SYS.DBMS_LOGMNR_D COMPILE BODY;
Tip: Periodically, PowerExchange requests Oracle to recopy the catalog. You can use
the CATBEGIN, CATEND, and CATINT parameters in the ORCL CAPI_CONNECTION statement
of dbmover.cfg to control how often Oracle copies the catalog and the time period
within which the copy occurs.
• • •
•
•
10
Step 1. Customize the dbmover.cfg File on UNIX
On the UNIX machine that contains the Oracle source tables, customize the
dbmover.cfg configuration file. Tip: A sample file for Linux and UNIX machines is
located in your PowerExchange installation directory. Copy this file to another
directory and customize the copy. This practice preserves your customized file if
you upgrade PowerExchange. The following sample dbmover.cfg file includes required
and recommended statements for Oracle CDC.
/********************************************************************/ /*
PowerExchange Configuration File
/********************************************************************/
LISTENER=(pwxlst,TCPIP,2480) NODE=(local,TCPIP,127.0.0.1,2480)
NODE=(pwxlst,TCPIP,127.0.0.1,2480) NODE=(default,TCPIP,x,2480) /* APPBUFSIZE=128000
COLON=: CONSOLE_TRACE=Y DECPOINT=. DEFAULTCHAR=* DEFAULTDATE=19800101 MAXTASKS=5
MSGPREFIX=PWX NEGSIGN=PIPE=| POLLTIME=1000 TIMEOUTS=(60,3600,3600) EXT_CP_SUPPT=Y
LOGPATH=/pwx/logs TRACING=/PFX=PWXLOG,RECLEN=255,FILENUM=3,APPEND=Y,FLUSH=99)
CAPT_PATH=/pwx/capture CAPT_XTRA=/pwx/capture/xtramaps /* /* An ORACLEID statement
is required for each Oracle instance involved in CDC.
ORACLEID=(ORACOL01,orcl01,orcl01.world,orcl01.world) /* /* CAPI connection
statements: /* /* A CAPX CAPI_CONNECTION is required for continuous extraction
mode. CAPI_CONNECTION=(NAME=CAPX01,TYPE=(CAPX,DFLTINST=ORACOL01)) /* /* Both UOWC
and ORCL CAPI_CONNECTION statements are required for Oracle CDC.
CAPI_CONNECTION=(NAME=CAPIORA1,TYPE=(ORCL,ARRAYSIZE=1000,BYPASSUF=Y,
CATBEGIN=00:01,CATEND=23:59,CATINT=1440,ORACOLL=ORACOL01,SELRETRY=0))
CAPI_CONNECTION=(NAME=CAPIUOW1,TYPE=(UOWC,CAPINAME=CAPIORA1,MEMCACHE=50000)) /*
CAPI_CONN_NAME=CAPIUOW1 /* /* To use pwxcmd to issue commands to PowerExchange
Listener and Logger /* processes, include the following statements:
SVCNODE=(pwxlst,8500) SVCNODE=(pwxccl_svc,8501)
CMDNODE=(pwxlst,LISTENER,unix_host01,8500)
CMDNODE=(pwxccl_svc,CONDENSE,unix_host01,8501) /*
Key Parameter Descriptions The following table describes key parameters for Oracle
CDC using the PowerExchange Logger:
Parameter CAPI_CONN_NAME Description Optional. Default CAPI_CONNECTION statement,
across all source types.
11
Parameter CAPT_PATH
Description Required. Specifies a path to the local directory where the CCT and
CDCT files reside. This directory must exist on the UNIX machine. The CCT file
contains capture registrations. The CDCT file contains information about
PowerExchange Logger log files. Required. Specifies a path to the local directory
where extraction maps reside. This directory must exist on the UNIX machine.
Optional. Specifies a service name and connection information for the PowerExchange
Listener or PowerExchange Logger process to which you want to send pwxcmd commands
from this machine. Note: In this scenario, you run the PowerExchange Listener and
PowerExchange Logger in background mode. To issue commands to background processes,
you must use the pwxcmd program. However, you can also use pwxcmd to issue commands
in foreground mode or from a script file. For more information, see the
PowerExchange Command Reference. For this scenario, define CMDNODE statements in
the dbmover.cfg files on the UNIX and Windows machines so that you can issue pwxcmd
commands from either machine. Syntax is:
CMDNODE=(service_name,service_type,host_name,port_number)
CAPT_XTRA CMDNODE
Where: • service_name is a service name. This name can be any name you want to use.
For consistency, you can match the service name in the corresponding SVCNODE
statement in the dbmover.cfg file on this UNIX machine. service_type is either
“LISTENER” for the PowerExchange Listener or “CONDENSE” for the PowerExchange
Logger. host_name is the host name or IP address of the machine where the
PowerExchange Listener or PowerExchange Logger runs. In this case, enter the host
name or IP address of this UNIX machine. port_number is the port number that is
specified in the corresponding SVCNODE statement of the dbmover.cfg file on this
UNIX machine. This port is the one on which the PowerExchange Listener or
PowerExchange Logger listens for pwxcmd commands.
• •
ORACLEID
• •
•
12
Parameter
Description database. Tip: Informatica recommends that you set the following
environment variables, whenever possible, to enable connection to the Oracle
database without using the capture connect string and to bypass the use of Oracle
SQL*Net: o o o o ORACLE_HOME ORACLE_SID PATH One of the following variables,
depending on your UNIX platform type: LD_LIBRARY_PATH, LIBPATH, or SHLIB_PATH
If you can bypass the use of SQL*Net when connecting to your Oracle database, you
can improve PowerExchange Logger performance, even when running on the same machine
as the Oracle database. The capture string and capture connect string are required
if you have multiple Oracle instances and you are capturing changes from an
instance other than the default instance. You can specify up to 20 ORACLEID
statements in a dbmover.cfg. CAPX CAPI_CONNECTION Required for continuous
extraction mode. Named set of parameters for continuous extraction of change data
from PowerExchange Logger log files. Syntax:
CAPI_CONNECTION=( NAME=name, [DLLTRACE=dll_trace_name], [TRACE=trace_name],
TYPE=(CAPX, DFLTINST=collection_id, [FILEWAIT=seconds] ))
• ORCL CAPI_CONNECTION
Required. Named set of parameters that the CAPI uses to connect to the Oracle
change stream and control extraction processing. Syntax:
CAPI_CONNECTION=( NAME=name, [DLLTRACE=dll_trace_name], [TRACE=trace_name],
TYPE=(ORCL, [ARRAYSIZE=nnnnnnnnnn], [BYPASSUF=Y|N], [CATBEGIN=hh:mm],
[CATEND=hh:mm], [CATINT=minutes], [COMMITINT=minutes], [GENRLOCK=Y|N],
[LOGDEST=dest],
13
Parameter
Description
[LGTHREAD=thread], [ONLINECAT=Y|N], ORACOLL=collection_id, [SELRETRY=nnnnnnnnnn],
[SNGLINST=ora_instance] ) )
Tip: Include the CATBEGIN, CATEND, and CATINT parameters to control how often
Oracle copies the Oracle catalog to the redo logs and the time period within which
the copy occurs. If you do not have restrictions for this time period, do not
specify CATBEGIN and CATEND. For more information about optional parameters, see
the PowerExchange Reference Manual. SVCNODE Optional. Specifies the port on which
the command-handling service for a PowerExchange Listener or PowerExchange Logger
process listens for pwxcmd commands. Define a SVCNODE statement for each
PowerExchange Listener or PowerExchange Logger service to which you want to send
pwxcmd commands. Syntax is:
SVCNODE=(service_name,port_number)
Where: • service_name is:
14
Parameter
Description o o • For the PowerExchange Listener: The node name that is specified
in the LISTENER statement. For the PowerExchange Logger: The service name that is
specified in the CONDENSENAME statement of the pwxccl.cfg file.
For more information about other optional parameters, see the PowerExchange
Reference Manual. UOWC CAPI_CONNECTION Required. Named set of parameters for the
UOW Cleanser. The UOW Cleanser reconstructs intermingled changes from multiple UOWs
into complete and consecutive UOWs that are in chronological order by end time.
This statement also points to the ORCL CAPI_CONNECTION. Syntax:
CAPI_CONNECTION=( NAME=name, [DLLTRACE=trace_id], [TRACE=trace], TYPE=(UOWC,
CAPINAME=capiname, [BLKSIZE=block_size], [MEMCACHE=kilobytes] ) )
Where:
15
Parameter
For more information about all dbmover.cfg parameters, see the PowerExchange
Reference Manual.
Key Parameter Descriptions The following table describes key parameters for the
PowerExchange Logger:
Parameter CAPT_IMAGE Description Data image type that the PowerExchange Logger
captures to its log files. Enter AI to capture after images only, or enter BA to
capture both before and after images of the data. Informatica recommends that you
specify BA so that you have the flexibility to use either AI or BA for extraction
and apply processing.
16
Parameter CAPTURE_NODE_EPWD or CAPTURE_NODE_PWD
CAPTURE_NODE_UID CHKPT_BASENAME
CHKPT_NUM COLL_END_LOG
COND_CDCT_RET_P
CONDENSENAME
This service name must also be specified in a SVCNODE statement of the dbmover.cfg
file on the machine where the PowerExchange Logger runs. CONN_OVR Recommended. Name
of the override UOWC CAPI_CONNECTION statement to use for the PowerExchange Logger.
If you do not specify CONN_OVR, the PowerExchange Logger uses the default
CAPI_CONNECTION if one is specified in dbmover.cfg. Note: In this scenario, the
CONN_OVR statement is included for completeness but is not required because the
default CAPI_CONNECTION is used. DBID Required. A source identifier, or instance
name, that is defined in capture registrations. With DB_TYPE, it defines selection
criteria for capture registrations in the CCT file. For Oracle, this value must
match the first positional parameter in the ORACLEID statement in dbmover.cfg. This
value should also match the ORACOLL value in the ORCL CAPI_CONNECTION statement and
the collection ID in the registration group. Required. Source RDBMS type. For
Oracle, this value must be ORA.
DB_TYPE
17
Parameter EXT_CAPT_MASK
Description Required. An existing directory path and a unique prefix to be used for
generating the PowerExchange Logger log files. PowerExchange considers any file
that matches this path and prefix to be a PowerExchange Logger log file. Maximum
length is 256 characters. Example:
/capture/pwxlog
Warning: Do not use the same EXT_CAPT_MASK value for multiple PowerExchange Logger
processes. Otherwise, a PowerExchange Logger process might corrupt log files that
are used by another PowerExchange Logger process. FILE_FLUSH_VAL Recommended. File
flush interval, in seconds, that the PowerExchange Logger waits before writing data
to the current log file on disk. This parameter affects the latency of change data
extractions that use continuous extraction mode. A file flush must occur before
data can be read by extraction sessions. Enter a value from 0 through 86400.
Default is -1, which causes the PowerExchange Logger process to not flush data to
the current log file. Usually, the -1 value is used only when running extractions
in batch extraction mode. Warning: A value of 0 can degrade PowerExchange Logger
and file system performance. Enter a value this is appropriate for your CDC
environment. Values that are too high can increase change extraction latency, and
values that are too low can degrade PowerExchange Logger and system performance.
For this scenario, enter a value of 10. FILE_SWITCH_CRIT FILE_SWITCH_MIN Optional.
Units to use for the FILE_SWITCH_MIN and FILE_SWITCH_VAL parameter values. Enter M
for minutes or R for records. Default is M. Use R for better control of file size.
Optional. File-switch criteria that the PowerExchange Logger uses when it
encounters change data for a new source. You can use this parameter to reduce
change data latency when running extractions in continuous extraction mode. Syntax
is:
FILE_SWITCH_MIN=(min_val,min_val_ign)
Where: • min_val is the minimum number of FILE_SWITCH_CRIT units that must elapse
after the PowerExchange Logger encounters a change record for a source that has no
entry in the CDCT file, before a file switch can be performed. Valid values are -1
(which causes PowerExchange to ignore this parameter) through 2147483647.
min_val_ign is the minimum number of FILE_SWITCH_CRIT units that must pass during a
PowerExchange Logger cold start before the PowerExchange Logger uses the min_val
value. Before the min_val_ign threshold is met, only FILE_SWITCH_VAL controls file
switch activity. Valid values are 0 (which causes PowerExchange to use the min_val
value) through 2147483647. This value is ignored if the PowerExchange Logger is
warm started.
Default is (-1,0) Warning: A value of (0,0) can result in a large number of file
switches when the PowerExchange Logger is cold started and you have registered many
source tables for PowerExchange Logger processing.
18
Parameter FILE_SWITCH_VAL
NO_DATA_WAIT
Recommended. If you run the PowerExchange Logger in continuous mode, specify the
number of minutes that the PowerExchange Logger must wait before starting the next
logging cycle. Default is 60. For continuous extraction mode, enter the minimum
value of 1 so that the next logging cycle starts shortly after the current one
completes. If the value of FILE_SWITCH_CRIT is M and the value of FILE_SWITCH_VAL
is less than the value of NO_DATA_WAIT, the PowerExchange Logger uses the
FILE_SWITCH_VAL value instead. In this scenario, enter 1 to log changes on a near
continuous basis.
NO_DATA_WAIT2
Recommended. Number of seconds that PowerExchange waits at the end-of-log for more
change data before returning control to the PowerExchange Logger. If this wait
period elapses and new change data has not been received, the PowerExchange Logger
stops the current logging cycle. Default is 600. If you set FILE_SWITCH_CRIT to M,
for minutes, Informatica recommends that you enter a NO_DATA_WAIT2 value that is 60
seconds less than the FILE_SWITCH_VAL value. In this scenario, this value is 840
seconds.
Optional. Parameters that define a restart point for starting change data
processing when a PowerExchange Logger is cold started. • • • If you enter 0s for
both parameters, processing starts from the point in the redo logs where the last
Oracle catalog write occurred. If you enter token values other than 0, processing
starts at the specific point defined by these token values. If you do not specify
these parameters, processing starts from the current end-of-log position.
Note: These parameters are ignored for a warm start. For this scenario, leave these
parameters commented out. SIGNALLING Indicates whether the PowerExchange Logger
attempts to take automatic action in the event of certain errors. Default is N.
Enter Y to have the PowerExchange Logger automatically handle certain errors such
as memory corruption. After the PowerExchange Logger handles an error, it attempts
to shut down in a controlled manner.
For more information about dbmover.cfg parameters, see the PowerExchange Reference
Manual.
For the scenario covered in this article, enter: For more information, see the
PowerExchange Command Reference.
Step 4. Customize the dbmover.cfg Files on the Windows and PowerCenter Integration
Service Machines
You must also customize the dbmover.cfg files on the Windows machine that hosts the
PowerExchange Navigator and PowerCenter Client and on the PowerCenter Integration
Service machine to add NODE statements that point to the UNIX machine that contains
the Oracle source tables. PowerExchange uses these NODE statements to connect to
the UNIX machine to read Oracle change data or metadata. Note: You do not need to
define a LISTENER statement in these dbmover.cfg files. The PowerExchange Listener
runs only on the Oracle source machine to listen for SQL requests for data or
metadata.
The CMDNODE statements are optional. Include them to be able to issue pwxcmd
commands from the Windows machine to the PowerExchange Listener and PowerExchange
Logger processes that run on the UNIX machine. For more information about issuing
pwxcmd commands and the command syntax, see the PowerExchange Command Reference.
20
Field Location
Description Enter the user-defined node name for the UNIX machine that contains the
Oracle source tables, as specified in a NODE statement of the local dbmover.cfg on
the Windows machine. The PowerExchange Navigator uses this information to connect
to the UNIX machine. Select ORACLE. Enter a database user ID that has sufficient
privileges to access the Oracle source tables for registration. This value should
match the CAPTURE_NODE_UID value in pwxccl.cfg. Enter the database password that is
associated with the specified user ID. This value should match the CAPTURE_NODE_PWD
or CAPTURE_NODE_EPWD value in pwxccl.cfg. Enter the name that you specified for the
first parameter in the ORACLEID statement of dbmover.cfg. This value should also
match the ORACOLL value in the ORCL CAPI_CONNECTION statement and the DBID value in
pwxccl.cfg. Select this option to have PowerExchange display the Add Capture
Registration – Name and Table Filter page so that you can add capture registrations
for Oracle source tables. Tip: Alternatively, to add the registrations later, open
the registration group and click Add > Capture Registration.
Type User ID
ORACLE oracapt
Password
oracapt
Collection identifier
ORACOL01
Add Registration
N/A
3.
On the Add Capture Registration – Name and Table Filter page, enter the following
information: y y Required. In the Name field, enter a descriptive name for the
registration in lowercase. This name can be up to eight alphanumeric characters in
length and must begin with a letter. Optional. In the Schema Name and Table Name
fields, enter schema name and table name masks to filter the list of tables to
register. Use the asterisk (*) anywhere in one of these masks to represent one or
more characters.
When you click Next, the Add Capture Registration – Tables and Columns page appears
and lists all tables that match your filter criteria. 4. 5. 6. In the Tables list,
double-click a table that you want to register. The table columns appear in the
Columns list. Select the columns for which to capture changes. Then click Next. On
the Add Capture Registration – Type page, enter the following information:
Field Type Status Condense Supplemental Log Group Name Description Select
Synchronous. Select Active to set the registration status to active when it is
created. Select Part to use the PowerExchange Logger. Enter a supplemental log
group name, or accept the default name, which has the format schema_tablename. A
supplemental log group must be defined for each table for which you want
PowerExchange to capture Oracle change data. PowerExchange generates DDL for
creating the supplemental log group for the table when you click Finish. You must
save this DDL to a file. You can give the file to your Oracle DBA, or have
PowerExchange run the DDL at registration completion.
21
Field Execute DDL now
Description Select this option to have PowerExchange run the DDL for creating the
supplemental log group when you click Finish. Do not select this option if you do
not have the privileges that are required to create supplement log groups on the
Oracle table. Instead, give the file with generated DDL to your Oracle DBA.
7. 8.
Click Finish. The Save ALTER TABLE SQL file dialog box appears. Enter the name of
the file to which to save the DDL for creating the supplemental log group, and
click Save. PowerExchange saves the DDL file to the specified directory and writes
the capture registration to the CCT file on the UNIX source machine. If you
selected Execute DDL now, PowerExchange also runs the DDL. If a supplemental log
group already exists for the table, an error occurs. Tip: For change capture to
occur, you must create a supplemental log group for every table that has an active
capture registration with Condense set to Part in the registration group. Each
supplemental log group must include all of the columns from which PowerExchange is
to capture changes. If a supplemental log group with the same name already exists
for a table, an error occurs.
Step 6. Add Change Indicator or Before Image Columns to Extraction Maps (Optional)
In the PowerExchange Navigator, you can edit the extraction maps to add
PowerExchange-generated change indicator (DTL_CI) and before image (DTL_BI)
columns. A DTL_CI column indicates if UPDATEs caused data in the column to change.
Based on this information, you can perform some processing, such as capture data
only for the columns for which the indicator is Y. A DTL_BI column stores the
before image of data in a column that was updated. If you do not add DTL_BI
columns, only the after image is captured. 1. 2. 3. In the PowerExchange Navigator,
open an extraction map for an Oracle source table. Right-click in the Extraction
Definition window, and select Amend Change Indicator/Before Image Extensions. The
Extract Definition - Amend Columns dialog box appears. To add DTL_CI or DTL_BI
columns, perform the following actions: y y 4. On the Change Indicators tab, select
columns in the Available Columns list to which to add a DTL_CI column, and click
Add. On the Before Images tab, select the columns In the Available Columns list to
which to add a DTL_BI column, and click Add.
Tip: You can also remove columns from an extraction map. Open the extraction map
and clear the column check boxes. Changes to the column data are still captured but
not extracted. For more information, see the PowerExchange Navigator User Guide.
22
Step 7. Perform a Database Row Test of the Extraction Maps
In the PowerExchange Navigator, you can perform a database row text on an
extraction map to verify that PowerExchange can use it to access PowerExchange
Logger log files and capture change data in near real time. This step is
recommended. 1. 2. 3. 4. 5. 6. 7. 8. In the PowerExchange Navigator, select an
extraction map for an Oracle source table. Click File > Database Row Test. The
Database Row Test dialog box appears. In the DB_Type list, select CAPXRT. In the
Application field, enter any value. This value is required but serves no function
for a row test. In the Location, User ID, and Password fields, verify that the
values match the values that you specified for the related registration group. In
the Fetch list, verity that Data is selected. Verify that the SQL Statement box
displays a SELECT statement for the correct extraction map. Click Go. When the row
test completes, the Database Row Test window displays the change data retrieved
from the PowerExchange Logger log files. If the window is blank, change data might
not yet have been written to the log files. A blank window does not necessarily
indicate an error. Tip: After the PowerExchange Logger is running and has logged
changes, you can perform a database row test to verify that change data is being
logged to the PowerExchange Logger log files. Before running the row test, you must
click Advanced and specify the CAPI_CONNECTION statement name, CAPX01, in the CAPI
Connection Name field of the CAPXRT Advanced Parameters dialog box to use the
continuous extraction connection. For more information, see the PowerExchange
Navigator User Guide.
If you store the pwxccl.cfg file in a directory other than the PowerExchange
installation directory, as recommended, you must include the “cs” option to point
to the file:
pwxccl cs=directory\pwxccl.cfg
After the PowerExchange Logger has been cold started once, you can run it
continuously in the background by entering the following command:
nohup pwxccl cs=directory\pwxccl.cfg 2>&1 < /dev/null &
23
For more information, see the PowerExchange Command Reference.
4. 5.
Click Connect. The Select Datamaps box lists the extraction maps that match your
criteria. Select one or more extraction maps to import.
24
6.
Click OK. In the Repository Navigator, the Oracle source table definitions appear
under ORACLE_location.
25
Attribute Image Type
Value Select AI, even though the CAPT_IMAGE parameter in the PowerExchange Logger
pwxccl.cfg file specifies BA. In this scenario, both before images and after images
are captured but only the after images are extracted and written to targets
Restart Token File Folder Number of Runs to Keep Restart Token File
For more information about all attributes, see PowerExchange Interfaces for
PowerCenter. 5. 6. Click OK. Click Close.
Recovery Strategy
Properties
26
Attribute or Option DTM buffer size
Tab Properties
Value Amount of memory allocated to the session from the Data Transformation
Manager (DTM) process for reading, writing, and transforming data. By default, the
Integration Service determines the DTM buffer size at run time. The minimum is 12
MB. Often, the default amount is too low for CDC processing. Increase this value if
necessary. The Integration Service interprets your value as bytes unless you append
KB, MB, or GB. Note: This value is related to the Default buffer block size on the
Config Object tab. The minimum number of buffer blocks must be greater than the
total number of sources and targets. The number of buffer blocks in a session = DTM
buffer size / buffer block size. Default settings create enough buffer blocks for
83 sources and targets. If the session contains more than 83, you might need to
increase DTM buffer size or decrease default buffer block size. For more
information, see the PowerCenter Workflow Basics Guide.
Enter 1 to ensure target data and restart token integrity is maintained. Select the
first Oracle source table in the left pane. Then, under Connections, select
Application in the Type field. In the corresponding Value field, select the generic
CDC Real Time connection object that you created. For Oracle source tables other
than the first table, select None in the Type field. Note: In the next step, you
customize connection attributes for the first source table and then apply the
connection type and attributes to all of the other tables, which have a connection
type of None.
5. 6.
On the Mapping tab, under Connections, click the pencil icon next to the Value
field while the first Oracle source table is still selected in the left pane. In
the Connection Object Definition dialog box, customize the Oracle source connection
attributes to be used for the session. The following table describes the connection
attributes that you should customize for a PowerExchange CDC session:
Attribute Application Name Value Enter a unique application name. Maximum length is
20 characters. Warning: If you use the same application name for different CDC
sessions, data loss or damage to data integrity might occur. Restart Token File
Name UOW Count Enter a unique name for the restart token file. Enter the number of
units of work (UOWs) that PWXPC must process before it flushes the data buffer to
commit the change data to the targets. Default is 1. A value of -1 disables this
attribute. Note: If you enter a large number, the RDBMS might hold many locks on
tables until the UOW count is reached and commit processing is done. For the commit
to occur, the Minimum Rows Per commit threshold, if specified, must also be met.
You can also control commits that occur at UOW boundaries based on time by
specifying the Real-time Flush Latency attribute. You can specify UOW Count or
Real-time Flush Latency or both.
27
Attribute Idle Time
Value Enter the number of seconds the PowerExchange Listener remains idle after
reaching the end of the change log, as indicated by the message PWX-09967, before
returning an end-offile (EOF). Default is -1, which causes an EOF to not be
returned and the session to run continuously. A value of 0 causes an EOF to be
returned at the end of log and the session to end successfully. Enter the number of
milliseconds that must pass before PWXPC flushes the data buffer to commit the
change data to the targets. When the real-time latency period expires, PWXPC
continues to read the changes in the current UOW until the end of that UOW is
reached. Then, PWXPC flushes the data buffer to commit the change data to the
targets. Valid values are 0 to 86400. If you specify a value from 1 through 1999 or
accept the default of 0, PWXPC uses 2000 milliseconds (or 2 seconds). A value of -1
disables this attribute. For the commit to occur, the Minimum Rows Per commit
threshold, if specified, must also be met. You can also control commits that occur
at UOW boundaries based on number of UOWs by specifying the UOW Count attribute.
You can specify UOW Count or Real-time Flush Latency or both.
Enter the maximum number of change records that PWXPC must process before it can
flush the data buffer to commit the change data to the targets. If necessary, PWXPC
continues to process change records across UOW boundaries until the maximum rows
value is met. PWXPC does not wait for a UOW boundary to commit the change data.
Default is 0, which disables this attribute. For the commit to occur, the Minimum
Rows Per commit threshold, if specified, must also be met. Tip: Usually, you set
this attribute only if you have very large UOWs.
Enter the minimum number of change records that PowerExchange must read from the
change stream before it can pass any commit record to PWXPC. Before reaching this
minimum, PowerExchange passes only the change records, without any commit records,
to PWXPC. Default is 0, which disables this attribute. Tip: This attribute is
useful if you have a large number of small UOWs and want a consistent flow of same-
sized UOWs.
Enter the name of the CAPX CAPI_CONNECTION statement that you defined in the
dbmover.cfg on the UNIX machine with the Oracle source tables. In this scenario,
the value is “CAPX01.”
For more information about Oracle connection attributes, see the PowerExchange
Interfaces for PowerCenter guide. Tip: When you are done and return to the Edit
Tasks dialog box, right-click the Application field and select Apply Connection
Attributes > To all Instances to apply the connection object attributes to the
other Oracle source tables. 7. On the Mapping tab, select the first target table in
the left pane and customize attributes. The attributes that you set depend on your
target type. You should at least set the following attributes: y y 8. 9. For the
Target load type attribute, select Normal. For the Table Name Prefix attribute,
enter the schema or owner name of the target table. Apply this attribute to all
instances.
For more information about creating sessions and workflows, see the PowerCenter
Workflow Basics Guide.
28
Step 6. Configure the Restart Token File
The restart token file provides restart tokens for specifying the point in the
change stream from which to start new CDC sessions. You can create a restart token
file that sets specific restart tokens for each Oracle source table or that
generates restart tokens that represent the current end of the change stream.
Alternatively, you can let PWXPC create a restart token file that contains null
restart tokens. In this scenario, let PWXPC create a restart token file that
contains null restart tokens. With the null restart tokens, change extraction
processing will begin with the first PowerExchange Logger log file. Important:
Review the restart and recovery information in the PowerExchange Interfaces for
PowerCenter guide. You must understand basic concepts about PowerExchange CDC
restart and recovery to prevent change data loss and to preserve target data
integrity. During session initialization, PWXPC uses the name specified in the
Restart Token File Name attribute to create an empty restart token file, if one
does not exist. The restart token file name is one of the following values, in
order of precedence: the Application Name value in the connection attributes, the
workflow name, or the Restart Token File Name value in the connection attributes.
By default, the workflow name is used. However, the workflow name might not result
in a unique restart token file name. The restart token file name must be unique for
each CDC session. Otherwise, session failures and potential data loss might occur.
To ensure a unique name, verify that you entered an application name value in the
source connection attributes. PWXPC creates the restart token file in the directory
that is specified in the Restart Token File Folder attribute of the source CDC
connection attributes. If this attribute contains the default value of
$PMRootDir/Restart, PWXPC also automatically creates this directory, if it does not
exist. During change data extraction processing, PWXPC updates restart tokens and
stores the updated token values in the PM_REC_STATE table in the target database.
You can use the restart token file to override the restart tokens in the
PM_REC_STATE table, if necessary, for a warm start.
Additional Information
For more information about PowerExchange CDC for Oracle, see the following
resources: y y y y y PowerExchange CDC Guide for Linux, UNIX, and Windows: For
configuration of the PowerExchange Logger and of CDC for Oracle and other data
sources on Linux, UNIX, or Windows. PowerExchange Interfaces for PowerCenter: For
PWXPC information and PowerExchange CDC attributes and options in PowerCenter.
PowerExchange Reference Manual: For descriptions of all dbmover.cfg parameters,
pwxcmd configuration, and PowerExchange security configuration. How-To article:
“Configuring PowerExchange 8.6.1 Oracle LogMiner CDC Parameters Based on
Environment Type” Key Knowledge Base (KB) articles: KB 108325: “Knowledge Link:
Working with PowerExchange Oracle CDC sessions”
29
-
KB 115883: “FAQ: Where should PowerExchange CDC for Oracle components run?”
You can access these publications and KB articles from my.informatica.com. New
articles and KBs on Oracle CDC are added on a continuing basis. Search the
Knowledge Base for additional articles and KBs periodically.
Author
Virginia Pfeifle Principal Technical Writer
30