External Tables Concepts

External Tables
Submitted by Natalka Roshak on Sun, 2006-03-05 18:00

RDBMS Server

External Tables let you query data in a flat file as though the file were an Oracle table. In 9i, only read operations were permitted; in 10g, you can also write out data to an external table, although you can't write to an existing table. While external tables can be queried, they're not usable in many ways regular Oracle tables are. You cannot perform any DML operations on external tables other than table creation; one consequence is that you can't create an index on an external table. External tables are largely used as a convenient way of moving data into and out of the database. Oracle uses SQL*Loader functionality, through the ORACLE_LOADER access driver to move data from the flat file into the database; it uses a Data Pump access driver to move data out of the db into a file in an Oracle-proprietary format, and back into the database from files of that format. While there are some behaviour differences and restrictions, you can think of external tables as a convenient, SQL-based way to use SQL*Loader and Data Pump functionality. For example, suppose that you receive a daily .csv report from another department. Instead of writing a SQL*Loader script to import each day's .csv file into your database, you can simply create an external table and write an "insert ... select" SQL query to insert the data directly into your tables. Place the day's CSV file in the location specified in the external table definition, run the query, and you're done

The external tables feature is a complement to existing SQL*Loader functionality. It enables you to access data in external sources as if it were in a table in the database. Prior to Oracle Database 10g, external tables were read-only. However, as of Oracle Database 10g, external tables can also be written to. Note that SQL*Loader may be the better choice in data loading situations that require additional indexing of the staging table. See Behavior Differences Between SQL*Loader and External Tables for more information about how load behavior differs between SQL*Loader and external tables. To use the external tables feature, you must have some knowledge of the file format and record format of the datafiles on your platform if the ORACLE_LOADERaccess driver is used and the datafiles are in text format. You must also know enough about SQL to be able to create an external table and perform queries against it.

Oracle External Tables

Last Updated on Monday, 21 November 2011 14:12 Written by Saurav Mitra

The Oracle external tables feature allows us to access data in external sources as if it is a table in the database. This is a very convenient and fast method to retrieve data from flat files outside Oracle database.

What is an Oracle External Tables?

The Oracle external tables feature allows us to access data in external sources as if it is a table in the database. External tables are read-only. No data manipulation language (DML) operations is allowed on an external table. An external table does not describe any data that is stored in the database.

So, how do I create an external table?

To create an external table in Oracle we use the same CREATE TABLE DDL, but we specify the type of the table as external by an additional clause - ORGANIZATION EXTERNAL. Also we need to define a set of other parameters called ACCESS PARAMETERS in order to tell Oracle the location and structure of the source data. To understand the syntax of all these, let's start by creating an external table right away. First we will connect to the database and create a directory for the extrnal table. CONN SYS/SYS_PWORD AS SYSDBA CREATE OR REPLACE DIRECTORY ext_tab_dir AS 'C:\External_Tables';

GRANT READ,WRITE ON DIRECTORY ext_tab_dir TO scott;

Flat File Structure

We will start by trying to load a flat file as an external table. Suppose the flat file is namedemployee1.dat with the content as: empno,first_name,last_name,dob 1234,John,Lee,"31/12/1978" 7777,Sam,vichi,"19/03/1975" So our CREATE TABLE syntax will be something like below

CREATE TABLE Example for External Table

CREATE TABLE emp_ext( empno NUMBER(4), first_name CHAR(20), last_name CHAR(20), dob CHAR(10) ORGANIZATION EXTERNAL( TYPE ORACLE_LOADER DEFAULT DIRECTORY ext_tab_dir ACCESS PARAMETERS ( RECORDS DELIMITED BY NEWLINE SKIP 1 BADFILE 'bad_%a_%p.bad' LOGFILE 'log_%a_%p.log' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' MISSING FIELD VALUES ARE NULL REJECT ROWS WITH ALL NULL FIELDS (empno INTEGER EXTERNAL (4), first_name CHAR(20), last_name CHAR(20), dob CHAR(10) DATE_FORMAT DATE MASK "dd/mm/yyyy") ) LOCATION ('employee1.dat','employee2.dat') ) PARALLEL REJECT LIMIT 0; LRTRIM

SELECT * FROM emp_ext; Now we can insert this temporary read only data to our oracle table say employee. INSERT INTO employee (empno, first_name, last_name, dob) (SELECT empno, first_name, last_name, dob FROM emp_ext);

Global temporary table example

Oracle Tips by Burleson Consulting March 3, 2012

Question: I need an example of how a global temporary table works. How does a global temporary table differs from a traditional temporary table or a WITH clause? I need a working example of a global temporary table. Do you need to drop the Global temporary table after each use? Answer: When using a global temporary table you need only define the table once for the entire schema and then any user can insert directly into the table without the need to re-create the global temporary table. drop table t1; create table t1 as select sum(quantity) all_sales from stores; Instead, a global temporary table does not required that the table be dropped and recreated and the global temporary tables are shared, such that each user gets a local copy of their rows, using the common definition. create global temporary table sum_quantity (sum_qty number); Once created, everyone shares the table definition and rows are private to each session. Here is an example of using a global temporary table:

SQL> insert into sum_quantity 2 (select sum(qty) from sales); Note that the rows disappear after the SQL statement has completed, so that a GTT can be used over-and-over without redoing the table. SQL> select * from sum_quantity; SUM_QTY ---------499 SQL> commit; Commit complete. SQL> select * from sum_quantity; no rows selected

Global Temporary Tables

Applications often use some form of temporary data store for processes that are to complicated to complete in a single pass. Often, these temporary stores are defined as database tables or PL/SQL tables. In Oracle 8i, the maintenance and management of temporary tables can be delegated to the server by using Global Temporary Tables. Creation of Global Temporary Tables Miscellaneous Features

Creation of Global Temporary Tables

The data in a global temporary table is private, such that data inserted by a session can only be accessed by that session. The session-specific rows in a global temporary table can be preserved for the whole session, or just for the current transaction. The ON COMMIT DELETE ROWS clause indicates that the data should be deleted at the end of the transaction. CREATE GLOBAL TEMPORARY TABLE my_temp_table ( column1 NUMBER, column2 NUMBER ) ON COMMIT DELETE ROWS;

In contrast, the ON COMMIT PRESERVE ROWS clause indicates that rows should be preserved until the end of the session. CREATE GLOBAL TEMPORARY TABLE my_temp_table ( column1 NUMBER, column2 NUMBER ) ON COMMIT PRESERVE ROWS;

Miscellaneous Features
If the TRUNCATE statement is issued against a temporary table, only the session specific data is trucated. There is no affect on the data of other sessions. Data in temporary tables is stored in temp segments in the temp tablespace. Data in temporary tables is automatically deleted at the end of the database session, even if it ends abnormally. Indexes can be created on temporary tables. The content of the index and the scope of the index is the same as the database session. Views can be created against temporary tables and combinations of temporary and permanent tables. Temporary tables can have triggers associated with them. Export and Import utilities can be used to transfer the table definitions, but no data rows are processed. Statistics on temporary tables are common to all sessions. Oracle 12c allows session specific statistics. There are a number of restrictions related to temporary tables but these are version specific.

This chapter discusses the following topics:

How Are External Tables Created? Using External Tables to Load and Unload Data Datatype Conversion During External Table Use Parallel Access to External Tables Performance Hints When Using External Tables External Table Restrictions Behavior Differences Between SQL*Loader and External Tables

How Are External Tables Created?

External tables are created using the SQL CREATE TABLE...ORGANIZATION EXTERNAL statement. When you create an external table, you specify the following attributes:

- specifies the type of external table. The two available types are the ORACLE_LOADER type and the ORACLE_DATAPUMP type. Each type of external table is supported by its own access driver. The ORACLE_LOADER access driver is the default. It can perform only data loads, and the data must come from text datafiles. Loads from external tables to internal tables are done by reading from the external tables' text-only datafiles. The ORACLE_DATAPUMP access driver can perform both loads and unloads. The data must come from binary dump files. Loads to internal tables from external tables are done by fetching from the binary dump files. Unloads from internal tables to external tables are done by populating the external tables' binary dump files. DEFAULT DIRECTORY - specifies the default location of files that are read or written by external tables. The location is specified with a directory object, not a directory path. See Location of Datafiles and Output Files for more information. ACCESS PARAMETERS - describe the external data source and implements the type of external table that was specified. Each type of external table has its own access driver that provides access parameters unique to that type of external table. See Access Parameters. LOCATION - specifies the location of the external data. The location is specified as a list of directory objects and filenames. If the directory object is not specified, then the default directory object is used as the file location.
TYPE

The following example shows the use of each of these attributes:

CREATE TABLE emp_load (employee_number CHAR(5), employee_last_name CHAR(20), employee_first_name CHAR(15), employee_middle_name CHAR(15)) ORGANIZATION EXTERNAL (TYPE ORACLE_LOADER DEFAULT DIRECTORY ext_tab_dir ACCESS PARAMETERS (RECORDS FIXED 62 FIELDS (employee_number CHAR(2), employee_dob CHAR(20), employee_last_name CHAR(18), employee_first_name CHAR(11), employee_middle_name CHAR(11))) LOCATION ('info.dat'));

The information you provide through the access driver ensures that data from the data source is processed so that it matches the definition of the external table. The fields

listed after CREATE TABLE emp_load are actually defining the metadata for the data in the info.dat source file. The access parameters are optional. Access Parameters When you create an external table of a particular type, you can specify access parameters to modify the default behavior of the access driver. Each access driver has its own syntax for access parameters.

See Also:

Chapter 14, " The ORACLE_LOADER Access Driver" Chapter 15, " The ORACLE_DATAPUMP Access Driver"

Location of Datafiles and Output Files The access driver runs inside the database server. This is different from SQL*Loader, which is a client program that sends the data to be loaded over to the server. This difference has the following implications:

The server must have access to any files to be loaded by the access driver. The server must create and write the output files created by the access driver: the log file, bad file, and discard file, as well as any dump files created by theORACLE_DATAPUMP access driver.

The access driver does not allow you to specify a complete specification for files. This is because the server may have access to files that you do not, and allowing you to read this data would affect security. Similarly, you might overwrite a file that you normally would not have privileges to delete. Instead, you are required to specify directory objects as the locations from which to read files and write files. A directory object maps a name to a directory name on the file system. For example, the following statement creates a directory object named ext_tab_dir that is mapped to a directory located at/usr/apps/datafiles.
CREATE DIRECTORY ext_tab_dir AS '/usr/apps/datafiles';

Directory objects can be created by DBAs or by any user with the CREATE ANY DIRECTORY privilege.

After a directory is created, the user creating the directory object needs to grant READ and WRITE privileges on the directory to other users. These privileges must be explicitly granted, rather than assigned through the use of roles. For example, to allow the server to read files on behalf of user scott in the directory named byext_tab_dir, the user who created the directory object must execute the following command:
GRANT READ ON DIRECTORY ext_tab_dir TO scott;

The name of the directory object can appear in the following places in a CREATE TABLE...ORGANIZATION EXTERNAL statement:

The DEFAULT DIRECTORY clause, which specifies the default directory to use for all input and output files that do not explicitly name a directory object. The LOCATION clause, which lists all of the datafiles for the external table. The files are named in the form directory:file. The directory portion is optional. If it is missing, the default directory is used as the directory for the file. The ACCESS PARAMETERS clause where output files are named. The files are named in the form directory:file. The directory portion is optional. If it is missing, the default directory is used as the directory for the file. Syntax in the access parameters enables you to indicate that a particular output file should not be created. This is useful if you do not care about the output files or if you do not have write access to any directory objects.

The SYS user is the only user that can own directory objects, but the SYS user can grant other users the privilege to create directory objects. Note that READ orWRITE permission to a directory object means only that the Oracle database will read or write that file on your behalf. You are not given direct access to those files outside of the Oracle database unless you have the appropriate operating system privileges. Similarly, the Oracle database requires permission from the operating system to read and write files in the directories. Example: Creating and Loading an External Table Using ORACLE_LOADER The steps in this section show an example of using the ORACLE_LOADER access driver to create and load an external table. A traditional table named emp is defined along with an external table named emp_load. The external data is then loaded into an internal table. 1. Assume your .dat file looks as follows:
2. 56november, 15, 1980 baker mary alice

3. 87december, 20, 1970 4.

roper

lisa

marie

5. Execute the following SQL statements to set up a default directory (which contains the data source) and to grant access to it:
6. CREATE DIRECTORY ext_tab_dir AS '/usr/apps/datafiles'; 7. GRANT READ ON DIRECTORY ext_tab_dir TO SCOTT; 8.

9. Create a traditional table named emp:

10. CREATE TABLE emp (emp_no CHAR(6), last_name CHAR(25), first_name CHAR(20), middle_initial CHAR(1)); 11.

12. Create an external table named emp_load:

13. CREATE TABLE emp_load (employee_number CHAR(5), employee_last_name CHAR(20), 14. employee_first_name CHAR(15), 15. employee_middle_name CHAR(15)) 16. ORGANIZATION EXTERNAL (TYPE ORACLE_LOADER DEFAULT DIRECTORY ext_tab_dir 17. ACCESS PARAMETERS (RECORDS DELIMITED BY NEWLINE FIELDS 18. (employee_number CHAR(2), 19. employee_dob CHAR(20), 20. employee_last_name CHAR(18), 21. employee_first_name CHAR(11), 22. employee_middle_name CHAR(11))) 23. LOCATION ('info.dat')); 24.

25. Load the data from the external table emp_load into the table emp:
26. INSERT INTO emp (emp_no, first_name, middle_initial, last_name) 27. (SELECT employee_number, employee_first_name, 28. substr(employee_middle_name, 1, 1), 29. employee_last_name 30. FROM emp_load); 31.

32. Perform the following select operation to verify that the information in the .dat file was loaded into the emp table:
33. 34. 35. 36. 37. 38. 39. SQL> SELECT * FROM emp; EMP_NO -----56 87 LAST_NAME ------------------------baker roper FIRST_NAME -------------------mary lisa M a m

Notes about this example:

The employee_number field in the datafile is converted to a character string for the employee_number field in the external table. The datafile contains an employee_dob field that is not loaded into any field in the table.

The substr function is used on the employee_middle_name column in the external table to generate the value for middle_initial in table emp.

External Tables
Submitted by Natalka Roshak on Sun, 2006-03-05 18:00

RDBMS Server

External Tables let you query data in a flat file as though the file were an Oracle table. In 9i, only read operations were permitted; in 10g, you can also write out data to an external table, although you can't write to an existing table. While external tables can be queried, they're not usable in many ways regular Oracle tables are. You cannot perform any DML operations on external tables other than table creation; one consequence is that you can't create an index on an external table. External tables are largely used as a convenient way of moving data into and out of the database. Oracle uses SQL*Loader functionality, through the ORACLE_LOADER access driver to move data from the flat file into the database; it uses a Data Pump access driver to move data out of the db into a file in an Oracleproprietary format, and back into the database from files of that format. While there are some behaviour differences and restrictions, you can think of external tables as a convenient, SQL-based way to use SQL*Loader and Data Pump functionality. For example, suppose that you receive a daily .csv report from another department. Instead of writing a SQL*Loader script to import each day's .csv file into your database, you can simply create an external table and write an "insert ... select" SQL query to insert the data directly into your tables. Place the day's CSV file in the location specified in the external table definition, run the query, and you're done