Various Approaches to Conditional Truncate Reload

Truncate-Reload:
In Truncate-Reload Logic, the target table is always truncated before data is loaded to the target
table.
Normally, the truncate command or Stored-procedure (SQL Command/ Stored Procedure) is
called in the Pre-SQL Session property or the Truncate Table option is checked in the Session
Properties.

Our Requirement:
In our Project there was a requirement, that:
1. The Target table should only be truncated in case where the number of Source Records
are greater than 0.
2. If the No. of Source Records are equal to 0 then there should not be any truncate/load
on the Target table.

How to Achieve this:

I found 3 ways to incorporate this logic:
1. Target Load Plan at the Mapping Level in the same Mapping
2. Creating multiple Mappings/Sessions in the same Workflow
3. Using a Unix Script in the same Session through Pre-Session Command

1. Target Load Plan at the Mapping Level (in the same mapping)
Overview:
In this method, the requirement can be achieved by using 2 different pipelines in the
same mapping and then loading the respective targets using Target Load Plan.
Steps to achieve the requirement:
1. Create another pipeline with the same source structure in the same Mapping.
2. Copy and connect all the data filter business transformations (if any) to the newly
created pipeline.
3. Count the Total No. of Source Records using an Aggregator transformation.
4. Add a Filter to the pipeline to check and proceed only if the record count > 0.
5. Add a Stored procedure transformation to the Pipeline to truncate the target table.
[Before hand, create a Stored Procedure in the Target Database to truncate
the table]

6. Add a Flat File as the Target to the Pipeline.

[You can also store the record count and System date in the Target Flat File
that will help in keeping a track of the date the table was last truncated
and the record count]

7. Go the Target Load Plan option in the Mapping tab and :

a. Select the newly created pipeline (Flat File Target) to load first.
b. Select the existing pipeline (RDB Target) to load second.
8. Save the Mapping and run the Session.

Fig 1.1 (Two Pipelines in the same mapping with the same Source structure)

Fig 1.2 (Target Load Plan Option)

Load)

Fig 1.3 (Setting the Order of Target

2. Creating multiple Mappings/Sessions in the same Workflow

Overview:
Create another Mapping/ Session to count the no. of Source records and truncate the
target table if the count is greater than 0. Link the newly created Session to execute first
and then the existing Session (that loads data to the target) in the same workflow.
Steps to achieve the requirement:
1. Create another Mapping with the same source structure in the same Mapping.
2. Copy and connect all the data filter business transformations (if any) to the newly
created Mapping.
3. Count the Total No. of Source Records using an Aggregator transformation.
4. Add a Filter to the pipeline to check and proceed only if the record count > 0.
5. Add a Stored procedure transformation to the Pipeline to truncate the target table.
[Before hand, create a Stored Procedure in the Target Database to truncate
the table]

6. Add a Flat File as the Target to the Pipeline.

[You can also store the record count and System date in the Target Flat File
that will help in keeping a track of the date the table was last truncated
and the record count]

7. Save the Mapping and create a new Session.

8. Link the newly created Session in the Workflow to run First and then link the existing
(Data Loading) Session to run after that.

3. Using a Unix Script in the same Session through Pre-Session

Command
Overview:
Create a Unix Script and connect to the database from the Script. Perform record count
and truncate the table accordingly in the Script. Link the Unix Script (Command Task/ PreSession Command) to the Workflow before the existing Session (That loads the Target)
Steps to achieve the requirement:
1.
2.
3.
4.

Create a Unix Script and connect to the database through the Script.
Run the Source Qualifier Query through the Script to count the records.
Check the no. of Records and put the truncate table condition accordingly.
Connect the Script/Task to execute before the Session.