Oracle 10g Data Pump 101

Babette Turner-Underwood
The Pythian Group
Ottawa ON, Canada
babette@pythian.com
Wednesday, April 26, 2006 2

Agenda
• Pros and Cons of Data Pump
• Data Pump Components and Features
• Data Pump Dictionary Views
• Using Data Pump
– Command line, interactive, dbms_datapump
• Monitoring Data Pump

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 3

Exp / Imp disadvantages

• Non-restartable
• Client-side, single threaded execution
• Limited tuning mechanisms
• Difficiult to monitor progress
• Limited object filtering

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 4

Data Pump Advantages

• Restartable. Improved control.
• Server-side, parallel execution
• Automated performance tuning
• Simplified monitoring
• Improved object filtering

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 5

Data Pump Disdvantages

• ONLY metadata compression
• Won’t overwrite file if exists
• Can’t use UNIX pipes, etc
• NEVER run as SYS ( / as sysdba)

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 6

Data Pump Components

• Jobs
• Master Processes
• Master Table

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 7

Data Pump Features

• Writes either
– Direct Path unloads
– External tables (part of cluster, has LOB,
etc)
• Command line interface
• Writing to external tables
• DBMS_METADATA
• DBMS_DATAPUMP

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 8

Data Pump Data Dict. Views

• DATABASE_EXPORT_OBJECTS
• SCHEMA_EXPORT_OBJECTS
• TABLE_EXPORT_OBJECTS
desc database_export_objects;
Name Null? Type
----------------------------- -------- --------------------
OBJECT_PATH NOT NULL VARCHAR2(200)
COMMENTS VARCHAR2(2000)
NAMED CHAR(1)

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 9

Data Pump Data Dict. Views

col object_path format a30 word_wrapped
col comments format a30 word_wrapped

select named, object_path, comments from

database_export_objects;
'
N OBJECT_PATH COMMENTS
- ------------------------------
------------------------------
Y ALTER_FUNCTION Recompile functions
Y ALTER_PACKAGE_SPEC Recompile package
specifications
. . .

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 10

Data Pump Data Dict. Views

TTITLE 'Active DataPump Operations'
COL owner_name FORMAT A06 HEADING 'Owner'
COL job_name FORMAT A20 HEADING 'JobName'
COL operation FORMAT A12 HEADING 'Operation'
COL job_mode FORMAT A12 HEADING 'JobMode'
COL state FORMAT A12 HEADING 'State'
COL degree FORMAT 9999 ' HEADING 'Degr'
COL attached_sessions FORMAT 9999 HEADING 'Sess'

SELECT owner_name
,job_name
,operation
,job_mode
,state
,degree
,attached_sessions
FROM dba_datapump_jobs;

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 11

Data Pump Command Line

impdp help=y expdp help=y
LOGFILE / NOLOGFILE FILESIZE
DUMPFILE FLASHBACK_SCN
'
DIRECTORY QUERY
JOBNAME INCLUDE
PARALLEL EXCLUDE

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 12

Using Data Pump – example 1

create directory btu as '/opt/db/oracle/working/babette';
Directory created.

expdp DUMPFILE=btu:avail.dmp LOGFILE=btu:avail.log

Export: Release 10.2.0.1.0 - 64bit Production on Thursday, 19 January…

Copyright (c) 2003, 2005, Oracle. All rights reserved.
'
Username: avail
Password:

Connected to: Oracle Database 10g Release 10.2.0.1.0 - 64bit Production

Starting "AVAIL"."SYS_EXPORT_SCHEMA_01": avail/******** DUMPFILE=btu:avail.dmp
LOGFILE=btu:avail.log
Estimate in progress using BLOCKS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 27.68 MB
Processing object type SCHEMA_EXPORT/USER
. . .

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 13

Using Data Pump – example 1

. . .
. . exported "AVAIL"."MYTABLE" 13.95 KB 380 rows
. . exported "AVAIL"."MYTABLE2" 26.82 KB 586 rows
. . exported "AVAIL"."OTHERONE" 0 KB 0 rows
. . exported "AVAIL"."PROCESSONE" 0 KB 0 rows
Master table "AVAIL"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded
'
*************************************************************************
Dump file set for AVAIL.SYS_EXPORT_SCHEMA_01 is:
/opt/db/oracle/working/babette/avail.dmp
Job "AVAIL"."SYS_EXPORT_SCHEMA_01" successfully completed at 04:39:16

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 14

Using Data Pump – example 2

/opt/db/oracle/working/babette> cat avail.parfile
DIRECTORY=btu
SCHEMAS=avail
JOB_NAME=avail_test2
DUMPFILE=btu:avail.dmp
LOGFILE=btu:avail.log
'
/opt/db/oracle/working/babette> expdp avail PARFILE=avail.parfile
Export: Release 10.2.0.1.0 - 64bit Production on Thursday, 19 January...
. . .
Master table "AVAIL"."AVAIL_TEST2" successfully loaded/unloaded
*************************************************************************
Dump file set for AVAIL.AVAIL_TEST2 is:
/opt/db/oracle/working/babette/avail.dmp
Job "AVAIL"."AVAIL_TEST2" successfully completed at 04:45:08

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 15

Using Data Pump – example 2

But FAILS if file already exists :
/opt/db/oracle/working/babette> expdp avail PARFILE=avail.parfile

Export: Release 10.2.0.1.0 - 64bit Production on Thursday, 19 January.. .

'
Copyright (c) 2003, 2005, Oracle. All rights reserved.
Password:

Connected to: Oracle Database 10g Release 10.2.0.1.0 - 64bit Production

ORA-39001: invalid argument value
ORA-39000: bad dump file specification
ORA-31641: unable to create dump file
"/opt/db/oracle/working/babette/avail.dmp"
ORA-27038: created file already exists
Additional information: 1

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 16

Using Data Pump – example 3

Import only specific objects:
impdp avail/avail DUMPFILE=btu:avail.dmp include=PROCEDURE

Import: Release 10.2.0.1.0 - 64bit Production on Thursday, 19 January . .

'
Copyright (c) 2003, 2005, Oracle. All rights reserved.
Connected to: Oracle Database 10g Release 10.2.0.1.0 - 64bit Production
Master table "AVAIL"."SYS_IMPORT_FULL_01" successfully loaded/unloaded
Starting "AVAIL"."SYS_IMPORT_FULL_01": avail/********
DUMPFILE=btu:avail.dmp include=PROCEDURE
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Processing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE
ORA-31684: Object type PROCEDURE:"AVAIL". "MYPROC1" already exists
ORA-31684: Object type PROCEDURE:"AVAIL"."NEXTPROC" already exists
. . .
Processing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE
Job "AVAIL"."SYS_IMPORT_FULL_01" completed with 2 error(s) at 12:38:29

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 17

Data Pump Interactive Mode

ADD_FILE Add dumpfile to dumpfile set.
CONTINUE_CLIENT Return to logging mode. Job will be
re-started if idle.
EXIT_CLIENT Quit client session, leave job running.
FILESIZE Default filesize (bytes) for subsequent
ADD_FILE 'commands.
HELP Summarize interactive commands.
KILL_JOB Detach and delete job.
PARALLEL Change # active workers for current job.
PARALLEL=<number of workers>.
START_JOB Start/resume current job.
STATUS Freq (secs) job status to be monitored
default(0) show new status when avail
STOP_JOB Clean shutdown of job, exit client.
STOP_JOB=IMMEDIATE does immediate stop

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 18

Data Pump Interactive Mode

Disconnect without killing export
create directory dpump as '/database/btu/exports';

expdp avail directory=dpump dumpfile=avail.dmp logfile=avail.log

'
Export: Release 10.2.0.1.0 - 64bit Production on Thursday, 19 January...
Copyright (c) 2003, 2005, Oracle. All rights reserved.
Password:

Connected to: Oracle Database 10g Release 10.2.0.1.0 - 64bit Production

Starting "AVAIL"."SYS_EXPORT_SCHEMA_01": avail/******** directory=dpump
dumpfile=avail/********.dmp logfile=avail/********.log
Estimate in progress using BLOCKS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
. . .

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 19

Data Pump Interactive Mode

Press Control-C for interactive mode
Export> status

Job: SYS_EXPORT_SCHEMA_01
Operation: EXPORT
Mode: SCHEMA '
State: EXECUTING
Bytes Processed: 27,340,024
Percent Done: 99
Current Parallelism: 1
Job Error Count: 0
Dump File: /database/btu/exports/avail.dmp
bytes written: 27,660,288

Worker 1 Status:
State: EXECUTING

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 20

DBMS_DATAPUMP
declare
2 handle number;
3 begin
4 handle := dbms_datapump.open ('EXPORT', 'SCHEMA');
5 dbms_datapump.add_file(handle, 'avail.dmp', 'BTU');
6 dbms_datapump.metadata_filter(handle, 'SCHEMA_EXPR','=''AVAIL''');
7
8
'
dbms_datapump.set_parallel(handle, 2);
dbms_datapump.start_job(handle);
9 dbms_datapump.detach(handle);
10 exception
11 when others then
12 dbms_output.put_line(substr(sqlerrm, 1,254));
13 end;
14 /
ORA-39001: invalid argument value

PL/SQL procedure successfully completed.

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 21

DBMS_DATAPUMP
btu> !rm avail.dmp

btu> @dump.sql

ORA-39002: invalid operation

'
*** PARALLEL NOT ALLOWED IN STANDARD EDITION ***

PL/SQL procedure successfully completed.

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 22

DBMS_DATAPUMP
(dump.sql)
declare
2 handle number;
3 begin
4 handle := dbms_datapump.open ('EXPORT', 'SCHEMA');
5 dbms_datapump.add_file(handle, 'avail.dmp', 'BTU');
6 dbms_datapump.metadata_filter(handle,
'SCHEMA_EXPR','=''AVAIL'''); '
7 -- dbms_datapump.set_parallel(handle, 2);
8 dbms_datapump.start_job(handle);
9 dbms_datapump.detach(handle);
10 exception
11 when others then
12 dbms_output.put_line(substr(sqlerrm, 1,254));
13 end;
14 /

PL/SQL procedure successfully completed.

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 23

Monitoring Data Pump

Use Client “STATUS” information
Use Data Dictionary Views
SQL> select sid, serial# '
2 from v$session s, dba_datapump_sessions d
3* where s.saddr = d.saddr

SID SERIAL#
---------- ----------
1916 3269
1892 4602
1954 1332

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 24

Monitoring Data Pump

Alert log information
Thu Jan 19 11:54:10 2006
kupprdp: master process DM00 started with pid=131, OS id=5738
to execute - SYS.KUPM$MCP.MAIN('SYS_EXPORT_SCHEMA_01', 'AVAIL',
'KUPC$C_1_20060119115410', 'KUPC$S_1_20060119115410', 0);
kupprdp: worker process DW01 started with worker id=1, pid=128,
OS id=5741 to execute – '
SYS.KUPW$WORKER.MAIN('SYS_EXPORT_SCHEMA_01', 'AVAIL');
kupprdp: master process DM00 started with pid=132, OS id=6294
to execute - SYS.KUPM$MCP.MAIN('SYS_EXPORT_FULL_01', 'AVAIL',
'KUPC$C_1_20060119115947', 'KUPC$S_1_20060119115947', 0);
kupprdp: worker process DW01 started with worker id=1, pid=133,
OS id=6296 to execute –
SYS.KUPW$WORKER.MAIN('SYS_EXPORT_FULL_01', 'AVAIL');
Thu Jan 19 11:59:54 2006
Memory Notification: Library Cache Object loaded into SGA
Heap size 2479K exceeds notification threshold (2048K)
KGL object name :SELECT /*+rule*/ SYS_XMLGEN(VALUE(KU$), X. . .

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 25

Monitoring Data Pump

Using Alert log information in queries
select sid, program from v$session where paddr in
(select addr from v$process where pid in (128,133));
'
select sid from v$px_session where qcsid = 23;

select sid, serial#, sofar, totalwork

from v$session_longops
where opname = 'SYS_EXPORT_SCHEMA_01' and sofar != totalwork;

Babette Turner-Underwood Oracle 10g Data Pump 101

Wednesday, April 26, 2006 26

Questions ?
• Babette Turner-Underwood
babette@pythian.com

Babette Turner-Underwood Oracle 10g Data Pump 101