Goldengate Mapping and Transformation for ETL

There is situation when you want to take advantage of GG. it can map and transform data in shape which you want on
Target.
we had same one situation, when Source table have 90+ rows and 5 million rows gets Replicated/day and Target table is
not partitioned.

so we did partition Target table not just partitioned but we sub-partition it.

Table will be first Partitioned based on date column by New 11g Interval partition and
then sub partitioned by site id which is again subset of account number, Account number column is Varchar2 with some 20
digits Number.so site id will be parsed from first 3 digits from Account Number.
In nutshell we are decomposing account Number on Target.

In addition to this ,we added three extra column.

R_TGT_CREATED_DT = Whenever record inserted first time on target it will have target current timestamp.
R_TGT_UPDATED_DT = Whenever record inserted & updated on target it will have target timestamp.
R_ACTION_CD = type of DML will be captured in this column, (insert/update/delete)

Again addition to this, we want to preserve rows which will be deleted in separate partitioned archived table.
so we had again archive table called ${TABLE_NAME}_DEL , Sorry using variable of Unix.

Note: As source and Target DDL is not identical, You must use defgen utility to generate source DDL. First time You
must load data using Initial Load, so rows will fall into right partitions and Extra mapping columns will also
populated as same time.

Note: When you want use INSERTDELETES on Targetside, You must use NOCOMPRESSDELETES on Source.
DDL of Source table.
CREATE TABLE DUMMYEVENT
(
ACC_NUM VARCHAR2(20 BYTE) NOT NULL,
ORIGINAL_ACC_NUM VARCHAR2(20 BYTE),
RULE_NUMBER NUMBER(1),
-----few columns Trim.
PREVIOUS_EVENT_REF VARCHAR2(16 BYTE),
PREVIOUS_EVENT_SEQ NUMBER(9),
PRODUCT_SEQ NUMBER(9),
TWIN_EVENT_BOO VARCHAR2(1 BYTE)
)

--DDL of Target table.

CREATE TABLE DUMMYEVENT
( SITE_ID NUMBER(3),
ACC_NUM VARCHAR2(20 BYTE) NOT NULL,
ORIGINAL_ACC_NUM VARCHAR2(20 BYTE),
RULE_NUMBER NUMBER(1),
-----few columns Trim.
PREVIOUS_EVENT_REF VARCHAR2(16 BYTE),
PREVIOUS_EVENT_SEQ NUMBER(9),
PRODUCT_SEQ NUMBER(9),
TWIN_EVENT_BOO VARCHAR2(1 BYTE)
R_TGT_CREATED_DT DATE,
R_TGT_UPDATED_DT DATE,
R_ACTION_CD VARCHAR2(100 BYTE)
)
TABLESPACE DUMMY_ADMIN
PARTITION BY RANGE (ACC_NUM)
INTERVAL(NUMTODSINTERVAL(1, 'DAY'))
subpartition by list (SITE_ID)
(PARTITION DUMMYEVENT1 VALUES LESS THAN (TO_DATE('20-02-2013', 'DD-MM-YYYY'))
(
SUBPARTITION CO_1 VALUES (1),
SUBPARTITION CO_126 VALUES (126),
SUBPARTITION CO_131 VALUES (131),
SUBPARTITION CO_132 VALUES (132),
SUBPARTITION CO_135 VALUES (135),
SUBPARTITION CO_182 VALUES (182),
SUBPARTITION CO_186 VALUES (186),
SUBPARTITION CO_214 VALUES (214),
SUBPARTITION CO_215 VALUES (215),
SUBPARTITION CO_216 VALUES (216),
SUBPARTITION CO_238 VALUES (238),
SUBPARTITION CO_239 VALUES (239),
SUBPARTITION CO_333 VALUES (333),
SUBPARTITION CO_334 VALUES (334),
SUBPARTITION CO_342 VALUES (342),
SUBPARTITION CO_436 VALUES (436),
SUBPARTITION CO_476 VALUES (476),
SUBPARTITION CO_477 VALUES (477),
SUBPARTITION CO_541 VALUES (541),
SUBPARTITION CO_580 VALUES (580),
SUBPARTITION CO_609 VALUES (609),
SUBPARTITION CO_UK VALUES (DEFAULT)
)
);

---ARCHIVED DELETE TABLE TO PRESERVE DELETED ROWS.

CREATE TABLE IDENTITY.DUMMYEVENT_DEL
( SITE_ID NUMBER(3),
ACC_NUM VARCHAR2(20 BYTE) NOT NULL,
ORIGINAL_ACC_NUM VARCHAR2(20 BYTE),
RULE_NUMBER NUMBER(1),
-----few columns Trim.
PREVIOUS_EVENT_REF VARCHAR2(16 BYTE),
PREVIOUS_EVENT_SEQ NUMBER(9),
PRODUCT_SEQ NUMBER(9),
TWIN_EVENT_BOO VARCHAR2(1 BYTE)
R_TGT_CREATED_DT DATE,
R_TGT_UPDATED_DT DATE,
R_ACTION_CD VARCHAR2(100 BYTE)
)
TABLESPACE IDM_DATA
PARTITION BY RANGE (R_TGT_UPDATED_DT)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION costed1 VALUES LESS THAN (TO_DATE('20-02-2013', 'DD-MM-YYYY'))
TABLESPACE IDM_DATA
);

---Goldengate Mapping On source

Please keep in mind that you put NOCOMPRESSDELETES in Source parameter file.
TABLE ICBS_ADMIN.DUMMYEVENT, KEYCOLS(ACC_NUM);

---Goldengate Mapping On Target, please keep in mind that you need to put ALLOWDUPTARGETMAP in Target
Mapping file.
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
--THIS IS FOR INSERT AND DELETE RECORD ONLY.
ALLOWDUPTARGETMAP
GETINSERTS
GETDELETES
IGNOREUPDATES
MAP ICBS_ADMIN.DUMMYEVENT, TARGET ICBS_ADMIN.DUMMYEVENT,
KEYCOLS(ACC_NUM),
INSERTMISSINGUPDATES,
COLMAP (USEDEFAULTS,
R_TGT_CREATED_DT= @DATENOW (),
R_TGT_UPDATED_DT= @DATENOW (),
R_ACTION_CD= @GETENV ("GGHEADER","OPTYPE")
);
--THIS IS FOR UPDATE RECORD ONLY.
GETUPDATES
IGNOREINSERTS
IGNOREDELETES
MAP ICBS_ADMIN.DUMMYEVENT, TARGET ICBS_ADMIN.DUMMYEVENT,
KEYCOLS(ACC_NUM),
INSERTMISSINGUPDATES,
COLMAP (USEDEFAULTS,
R_TGT_UPDATED_DT= @DATENOW (),
R_ACTION_CD= @GETENV ("GGHEADER","OPTYPE")
);
---------------------------------------------------------------------------------
--To preserve deletes.
---------------------------------------------------------------------------------
GETDELETES
IGNOREINSERTS
IGNOREUPDATES
INSERTDELETES
MAP ICBS_ADMIN.DUMMYEVENT, TARGET ICBS_ADMIN.DUMMYEVENT_DEL,
KEYCOLS(ACC_NUM),
INSERTMISSINGUPDATES,
COLMAP (USEDEFAULTS,
SITE_ID= @STREXT (ACC_NUM, 1, 3),
R_TGT_CREATED_DT= @DATENOW (),
R_TGT_UPDATED_DT= @DATENOW (),
R_ACTION_CD= @GETENV ("GGHEADER","OPTYPE")
);