SQL* LOADER

2 April 2009

CONTENTS
1. OVERVIEW OF SQL* LOADER 2. ARCHITECTURE 3. TYPES OF FILE USED 4. STRUCTURE OF THE INPUT FILE 5. EXECUTING SQL* LOADER 6. CASE STUDY 7. TUNING DATA LOADS

1. OVERVIEW OF SQL* LOADER

SQL* LOADER (sqlldr) IS USED TO LOAD DATA FROM EXTERNAL FILES TO THE ORACLE DATABASE TABLE. THE PHYSICAL RECORDS IN EXTERNAL FILE ARE CONTROLLED BY THE CONTROLLER FILE.
records Loaded

SQL*LOADER

External File

Oracle Database

2. ARCHITECTURE
EXTERNAL FILES

SQLLDR Field Validation

CONTROL FILE

DISCARD discarded

Rejected

Record Validation

BAD Rejected

LOG

ORACLE SERVER
loaded

Oracle DB

3. TYPES OF FILE USED

THE FOLLOWING TYPES OF FILES ARE USED

1. INPUT FILE
1.1 Data File or External File :- Collection of Physical Records 1.2. Control File :- Instruct SQL*Loader where to find the data, how to parse and interpret the data, and where to insert the data

2. OUTPUT FILE
2.1 Bad File :Contain all records, rejected either due to formatting errors or because of Oracle errors.

2.2 Discard File :- Contain all records that do not meet any of the loading condition specified in the control file. 2.3 Log File :Stores the state of the load, including a description of any errors that occurred during the load.

4. STRUCTURE OF INPUT FILE

4.1 EXTERNAL FILE FIXED-LENGTH RECORD FORMAT- All records in a External File are of same length. Example: EMP# 10002 10 EMP_NAME Satya Pratap Srijib Roy SALARY 48000 4800 DOB 01/03/1979 01/03/1982

In this case we give a specific position where the Control file can expect a data field: VARIABLE-LENGTH RECORD FORMAT- Record fields are separated by a delimiters like , | \ . In this case we specify a delimiter in the Control File Example: 10002|Satya Pratap|48000|01/03/1979 10|Srijib Roy|4800|01/03/1982 contd..
6

4.2 CONTROL FILE OVERALL STRUCTURE

[OPTIONS (Opt_type)] --Options Clause LOAD DATA INFILE <file_ name> [RECSIZE <integer>] [BUFFERS <integer>] ---modes [APPEND/INSERT/REPLACE/TRUNCATE] INTO TABLE " schema_ name ". " table _ name" FIELDS TERMINATED BY <delimiter> --for variable length records in External file [OPTIONALLY ENCLOSED BY <<enclosing character> >] [TRAILING NULLCOLS] -- columns that are not present in the record treated as null

[WHEN <condition>]

--filter condition

( <column_name> [<data_type>] [ <sql_function>] , --for variable length records <column_name> [<data_type>] [ <sql_function>] , <column_name> [ <data_type>] [<sql _function>] )
contd..
7

4.2.1 CONTROL FILE STRUCTURE FOR FIXED - LENGTH RECORD OPTIONS (ERRORS = 55) LOAD DATA INFILE <file name> INTO TABLE <table name> ( <column_name> POSITION(<integer>:<integer>) <data_type> , <column_name> POSITION(<integer>:<integer>) <data_type> ) 4.2.2 CONTROL FILE STRUCTURE FOR VARIABLE - LENGTH RECORD OPTIONS (ROWS = 5) LOAD DATA INFILE <file name> INTO TABLE <table name> FIELD TERMINATED BY | ( <column_name> <data_type> , <column_name> <data_type> ) contd..
8

OPTIONS CLAUSE:
1.1 1.2 1.3 1.4

Some SQL*Loader options are-

BAD : Name of the bad file BINDSIZE: The size of the bind array in bytes. CONTROL : The name of the control file DIRECT : [FALSE] Specifies whether or not to use a direct path load (dpl) or conventional 1.5 DISCARD : Name of the discard file 1.6 DISCARDMAX : [ALL] Max no. of records to discard. 1.7 ERRORS : [50] No. of allowed errors. 1.8 LOAD : [ALL] No. of logical rows to load. 1.9 LOG : Name of the log file. 1.10 MULTITHREADING : Flag to indicate if it should be used used during dpl 1.11 PARFILE : Parameter file contains additional load parameter specification. 1.13 PARALLEL : [FALSE] Perform parallel load. (increases performance) 1.14 READSIZE: [1 MB] Size of the read buffer. 1.15 RESUMABLE: [FALSE] Resumable flag for the current session. 1.16 RESUMABLE_NAME : Text identifierfor resumable statement. 1.17 RESUMABLE_TIME : [7200 sec.] Wait time for resumable option. 1.18 ROWS : [64] No. of rows to be committed at a time. [ALL] for dpl 1.19 SILENT: Suppress load message. 1.20 SKIP : [0] Allow us to skip n number of logical records. 1.21 USERID : The Oracle username and password
9

5. EXECUTING SQL* LOADER

Once we are ready with the control file and/or data file(s), the sqlldr command is written to load the date into our target table in oracle database. Running the SQL* LOADER from UNIX:
$ sqlldr userid = user_name/pwd control = <control filename> log = <Log filename> Example :- $ sqlldr userid = system/admin@123 control = xyz.ctl log = mylog.log

SQL* LOADER xyz.ctl mylog.log

ORACLE TABLE

10

6. EXAMPLE
TEMP TABLES : EMP ( emp_no, emp_name, sal, dept_no)
10 | Satya | 24000 | D0101

Demo 1 : VARIABLE - LENGTH RECORD

LOAD DATE INFILE c:\Extfile\emp1.dat APPEND INTO TABLE EMP FIELD TERMINATED BY | ( emp_no integer, emp_name char UPPER(:emp_name), sal number, dept_no char )

17 | Sumanto | 24000 | D0112

emp1.ctl

- - function

Executed as: $ sqlldr userid = scott/tiger control=c:\control\emp1.ctl log=c:\mylog\emp1.log

11

EXAMPLE
10 Satya 240000 D010 Demo 2 : FIXED - LENGTH RECORD 17 Sumanto 24000 D0112 LOAD DATE INFILE c:\Extfile\emp2.dat DISCARDFILE 'c:\discard\emp2.dsc' APPEND INTO TABLE EMP WHEN sal ! = ' ' ( emp_no POSITION(1:5) integer, emp_name POSITION(7:15) char UPPER(:emp_name), sal POSITION(17:22) number, dept_no POSITION(24:30) char ) Executed as: $ sqlldr userid = scott/tiger control=c:\control\emp2.ctl log=c:\mylog\emp2.log

12

EXAMPLE
Demo 3 : Loading multiple files into multiple tables in a singe control file
LOAD DATE INFILE c:\Extfile\emp1.dat ---Data file 1 INFILE c:\Extfile\emp2.dat ---Data file 2 APPEND INTO TABLE EMP1 WHEN sal <= 30000 ( emp_no POSITION(1:5) integer, emp_name POSITION(7:15) char UPPER(:emp_name), sal POSITION(17:22) number, dept_no POSITION(24:30) char) INTO TABLE EMP2 WHEN sal > 30000 ( emp_no POSITION(1:5) integer, emp_name POSITION(7:15) char UPPER(:emp_name), sal POSITION(17:22) number, dept_no POSITION(24:30) char ) Executed as: $ sqlldr userid = scott/tiger control=c:\control\emp3.ctl log=c:\mylog\emp3.log
13

EXAMPLE
Demo 4 : When data file is in the control file
LOAD DATE INFILE * DISCARDFILE 'c:\discard\emp2.dsc' INSERT INTO TABLE EMP ( emp_no POSITION(1:5) integer, emp_name POSITION(7:15) char UPPER(:emp_name), sal POSITION(17:22) number , dept_no POSITION(24:30) char ) BEGINDATA 10 Satya 240000 D010 17 Sumanto 24000 D0112

Executed as: $ sqlldr userid = scott/tiger control=c:\control\emp2.ctl log=c:\mylog\emp2.log

14

7. TUNING DATA LOADS

Till now Ive discussed conventional method in which, array of rows are inserted with standard SQL INSERT statement. Hence performance decreases when the volume of data increases. To overcome this performance issue, use following loads :

Direct Pate Load : In this SQL* Loader writes directly to our databases
datafiles. Because it bypasses standard SQL statement processing, it results in a significant performance improvement. Enforces only primary key, unique, and NOT NULL constraints. INSERT trigger not fired. DIRECT = TRUE - - options specified in command line/control file.

Parallel Direct Pate Load : Attempts to load datafiles by creating temporary

segments for each parallel sessions and merge them into one segments . Finally, this segment is then added to the table.

PARALLEL = TRUE - - options specified in command line/control file. contd..

15

Parallel load Architecture :

file1.dat Cont1.ctl

SQL* Loader

SEGMENT segment 1

file2.dat Cont2.ctl

SQL* Loader

segment 2

2 parallel session
TABLE append

Few Restriction on Parallel Direct Path Load 1. Referential integrity and check constraints must be disabled.
2. Triggers must be disabled. 3. Rows can only be appended. This is due to the individual loads not been coordinated.

16

17