cat --- for creating and displaying short files

chmod --- change permissions

cd --- change directory

cp --- for copying files

date --- display date

echo --- echo argument

ftp --- connect to a remote machine to download or upload files

grep --- search file

head --- display first part of file

ls --- see what files you have

lpr --- standard print command (see also print )

more --- use to read files

mkdir --- create directory

mv --- for moving and renaming files

ncftp --- especially good for downloading files via anonymous ftp.

print --- custom print command (see also lpr )

pwd --- find out what directory you are in

rm --- remove a file

rmdir --- remove directory

rsh --- remote shell

setenv --- set an environment variable

sort --- sort file

tail --- display last part of file

tar --- create an archive, add or extract files

telnet --- log in to another machine

wc --- count characters, words, lines

SQL SELECT

SQL DISTINCT

SQL WHERE

SQL AND OR

SQL IN

SQL BETWEEN

SQL Wildcard

SQL LIKE

SQL ORDER BY

SQL GROUP BY

SQL HAVING

SQL ALIAS

SQL AS

SQL SELECT UNIQUE

SQL INSERT INTO

SQL INSERT INTO SELECT

SQL UPDATE

SQL DELETE FROM

Steps to be followed for implementing SCD II

Read the incoming records through any input stage like sequential file/dataset/table.

Do the required processing for the incoming data.

After the above processing step, pass the data into the change capture stage.

The change capture should be having two input links- one is the before dataset and the other is the
after dataset. For our job, the before dataset should be the active records present in the table. The
active records are all those records which are having EXPR_DT=2999-12-31. The after dataset will be
the incoming data passed into change capture after all the necessary processing.

The change capture stage compared the before dataset and after dataset and produces 4
change_codes for each of the records. The 4 change codes are as follows:

0 Copy code (The code indicates the after record is a copy of the before record)

1-Insert code (The code indicates a new record has been inserted in the after set that did not exist in
the before set.)

2-Delete code(The code indicates that a record in the before set has been deleted from the after set)

3-Edit code(the code indicates the after record is an edited version of the before record)

The copy records are not passed in the change captured stage as since we need only edited, insert
records fro SCD II implementation.

Use a filter stage to separate the records that needs to be expired and inserted.

Filter the records with change_code = 1 or 3 into the insert records link. Filter the records with
change_code= 3 into update/expiry link.

The records with change_code=3 are edited records. So the original records corresponding to these
edited records are to be made in-active (expired). We can make the records inactive by changing the
EXPR_DT<> 2999-12-31.So to make the record inactive change the EXPR_DT with a valid date. For
e.g. you can use make the EXPR_DT as the date one less than the date on which you are loading the
data into the table. We will assume that we are loading the data on 2008-08-15.So the EXPR_DT for
inactive records would become 2008-08-14. The date 2008-08-15 can be made as the EFCT_DT for
records to be inserted.

To get the original records which needs to be expired, look-up the target table for all the records with
change_code=3 which are filtered out separately. Get the original record along with the EFCT_DT of the
original record. Then update the records EXPR_DT to 2008-08-14 in the table. Now the original records
are made inactive (expired).

The new updated record (change_code=3) needs to be in table along with the new insert
records(change_code=1).This data is filtered out from the filter stage and inserted into the table with
EFCT_DT=Data of loading i.e. 2008-08-15 and EXPR_DT=2999-12-31

String functionsalnum,alpha,compact white space etc

Logical Functions

Date and Time Functionscurrent date,current time,current timestamp,day since from date,months from date,nextweekday from date
etc

Mathematical Functions

Null Handling Functions

Number Functions

Raw Functions

Vector Function

Type Conversion Functions

Utility Functions

if you want to remove duplicates in transformer stage

use one of the partition technic hash partition you can
easily remove duplicatess