How to find the number of success , rejected and bad records in the same mapping.

How to find the number of success , rejected and bad records in the same mapping. In this Mapping we will see how to find the number of success , rejected and bad records in one mapping.

Source file is a flat file which is in .csv format . Click here to download the source file.The table appears like as shown below.. EMPNO 100 101 102 103 105 104 106 SANTA SMITHA NAME RAJ JOHN MAON HIREDATE SEX 21-APR 21-APR-08 08-APR 22-APR-08 22-APR-08 22-APR-08 M M M M F F M

In the above table it shows few values are missing in the table .ANd also the date format of few records are improper.This must be considered as invlaid records and should be loaded into Bad_records table ( target table which is relational). Other than 2 , 3 , 5, 6 records ,remaining all are invalid records because of NULL values or improper DATE format or both . INVALID & VALID RECORDS :: First we seperate this data using Expression transformation.Which is used to flag the row for 1 or 0 .The condition as follows .. IIF(NOT IS_DATE(HIREDATE,'DD-MON-YY') OR ISNULL(EMPNO) OR ISNULL(NAME) OR ISNULL(HIREDATE) OR ISNULL(SEX) ,1,0) FLAG =1 is considered as invalid data and FLAG =0 is considered as valid data .This data will be routed into next transformation using router transformation .Here we added two user groups one as FLAG=1 for invalid data and the other as FLAG=0 for valid data. FLAG=1 data is forwarded to the expression transformation .Here we take one variable port and trwo ouput ports .One for increament purpose and the other for flag the row ... INVALID RECORDS INCREAMENT :: PORT COUNT_INVALID V_PORT EDIT EXPRESSION V_PORT ( output port ) V_PORT+1 ( variable)

INVALID DATE :: PORT INVALID_DATE EDIT EXPRESSION IIF( IS_DATE(O_HIREDATE,'DD-MON-YY'), O_HIREDATE, 'INVALID DATE')

This data will be moved to the BAD_RECORDS table.Look at the below table:: EMPNO NAME 100 102 103 106 RAJ NULL NULL HIREDATE SEX COUNT 1 2 3 4

INVALID DATE M 22-APR-08 NULL M M

MAON INVALID DATE M

VALID RECORDS :: In this we will have the valid records.But here we dont want the Employee ,who is 'F' (Female).So our goal is to load MALE employee info., into the SUCCESS_RECORDS target table. For this we need to use a Router transformation and declare the user group as follows \ IIF( sex='M',TRUE,FALSE) And the defined group will capture teh rejected records which are nothing but employee who is FEMALE . This data passed to the REUSABLE Expressiona transformation.Where the Increamental logic is applied to get the count value for the the no., of success and rejected records which are passing it.And loaded into the target table. Look at the below tables ::: SUCCESS_RECORDS:: EMPNO NAME 101 JOHN HIREDATE 22-APR-08 SEX COUNT M 1

REJECTED_RECORDS:: EMPNO NAME 105 106 HIREDATE SEX COUNT F F 1 2

SANTA 22-APR-2008 SMITHA 22-APR-2008