How to Improve Database Performance With Full-Table Scans

Jeff Maresh 10/11/2002

So whats a full-table scan?

A full-table scan is the process of reading all blocks of a database table sequentially When a full-table scan occurs, indexes on the scanned table are not used Because of the volume of data processed, they can consume large amounts of I/O bandwidth

Why the bad reputation?

In Oracle 6, full-table scans flooded the buffer cache because there was no difference in the way that blocks were handled between full-table scans and reads by index Prior to Oracle 8, Full-table scans often occurred when the optimizer developed the less than optimal execution plan More often than not, full-table scans were the cause of bad query performance Often times, bad overall database performance resulted from many full-table scans DBAs spent many a fortnight exorcising the database of full-table scans to avoid floggings by users hence the prevailing belief that all full-table scans are bad

That was then, this is now

Beginning with Oracle 7.0, blocks read by fulltable scans dont flood the buffer cache because the space they are allowed to occupy is restricted to a small amount Hash joins and parallel query, added in Oracle 7.3, rely heavily on full-table scans for efficiency There have been significant improvements in the cost-based optimizer to minimize the inappropriate use of full-table scans

That was then, this is now

Disk subsystems have improved dramatically to increase I/O bandwidth, relative to overall database size and number of users Significant performance gains can now be achieved by properly using full-table scans The usual reasons why full-tables scans now cause bad performance are
Missing indexes on large tables A poor indexing strategy on large tables Missing table or index statistics

What is performance?
Subjective measures
Slow response time on specific queries Overall sluggish database response

Objective measures
CPU consumption I/O consumption Physical I/O block reads from disk Logical I/O block reads from buffer cache Optimizer query cost

Which one is best?

Subjective performance measures are important because the end-uses must be satisfied There is no perfect objective measure Logical I/O is the least flawed objective measure in most cases because:
I/O requires the majority of clock time to execute queries Between 30 and 50% of query CPU consumed is spent on logical I/O Minimizing logical I/O usually minimizes physical I/O, CPU consumption, and response time

Table access methods

Full-table scan read all data blocks in the table in sequence B*Tree index unique scan read one table row by obtaining the ROWID from an index that has one index entry for each table row B*Tree index range scan read many table rows by obtaining multiple rowids from an index that has multiple ROWIDs for each table row

FTS effects on buffer cache

Large tables use few buffer cache blocks as specified by the value of db_file_multi_block_read_count, typically 8 to 64 blocks per table scanned Small tables may use up to 2% of buffer cache specified by _small_table_threshold Default behavior inserts rows at the LRU side of the buffer cache LRU lists Can load on MRU side of the LRU lists using CACHE clause on ALTER TABLE Can load on MRU side of the LRU lists using CACHE query hint

FTS effects on buffer cache

Buffer Cache LRU List
Most Recently Used (MRU) Block aging Least Recently Used (LRU)

Blocks accessed by index start here Blocks accessed by FTS start here if CACHEd

Blocks accessed by default FTS start here

A practical example
Table has 100,000 rows Average row length is 100 bytes All blocks are 90% full (PCTFREE=10) Table is stored in 1,790 blocks Full table scan requires 1,790 logical reads

Full-table scan access

A full-table scan reads all data blocks in the table in sequence

1789

1790

Unique B*Tree index

B*Tree index on an ascending sequencegenerated numeric surrogate key 1 index key value corresponding to each of the 100,000 rows in the table Index height is 2 1 branch block 222 leaf blocks 3 logical reads per table row retrieved

Unique index access

SELECT

WHERE id=1345

Index branch block

..

..
Index contains ROWID of row in data block

Index leaf blocks

1789

1790

Data Blocks

Nonunique index
The table contains a code value that could represent a customer type code Nonunique index is created on the code value The table has 1,000 rows corresponding to each of the 100 code values Index height is 2 1 branch block 307 leaf blocks Number of logical reads per access is not obvious

Some additional info

Rows were inserted into the table such that the value of the code was incremented after each row was inserted The code value ranges between 0 and 99 and is reset to zero after it reaches a value of 99 This process is repeated 1,000 times The result is that few rows for each key value will be contained in a single data block A few index leaf blocks in the nonunique index contain ROWIDs that correspond to many data blocks

Data Load Sequence

1 2 3 4 5 6 7 8 9 10 11 12 13 14 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 100

Nonunique index access

SELECT

WHERE type=5

Index branch block

Performance: 503 Logical Reads per key value 1,000 rows/key

..

..

Index leaf blocks Few rows having the same key value reside in the same data block

1789

1790

Data Blocks

Another data load possibility

Rows are inserted into the table such that all 1,000 rows with the same code value are loaded sequentially Code value is incremented by 1 This process is repeated 100 times The result is that all rows with the same key value will be located within a few adjacent data blocks Few index leaf blocks in the nonunique index contain ROWIDs that correspond to few data blocks

Data Load Sequence

11111111111111111111 22222222222222222222 33333333333333333333 44444444444444444444 55555555555555555555 66666666666666666666 77777777777777777777

Nonunique index access

SELECT WHERE type=5

Index branch block

Performance: 53 Logical Reads per key value 1,000 rows/key

..

..

Index leaf blocks

All rows having the same key value reside in a few adjacent data blocks

1789

1790

Data Blocks

How do we determine that the second index will perform better than the first?

Clustering factor
Helps to understand the efficiency of nonunique indexes Clustering factor is available in dba_indexes and dba_ind_partitions in clustering_factor column after statistics are gathered The range for clustering factor is between the number of data blocks and number of table rows The first nonunique index case (worst case)
Clustering factor = 100,000 = # of rows The high clustering factor case

The second nonunique index case (best case)

Clustering factor = 1,786 = # of data blocks The low clustering factor case

Low clustering factor is more efficient than high clustering factor

Index summary
Unique Index Height Branch blocks Leaf blocks Distinct keys LR/access Rows/access Clustering Factor 2 1 222 100,000 3 1 1,786 Nonunique Index high CF 2 1 307 100 503 1,000 100,000 Nonunique index low CF 2 1 307 100 53 1,000 1,786

Access summary
Logical Reads required to get a specific number of rows from the table
Percent Rows of Table Retrieved Full Table Scan Unique Index Nonunique Nonunique Index Index High CF Low CF

0.001
1

1
1,000

1,790
1,790

3
3,000

N/A
503

N/A
53

5
10 25

5,000
10,000 25,000

1,790
1,790 1,790

15,000
30,000 75,000

2,515
5,030 12,575

265
530 1,325

50

50,000

1,790 150,000

25,150

2,650

Access method conclusions

Unique index access methods achieve the lowest number of logical reads of all access methods when retrieving a very small number of table rows Nonunique index access methods are most efficient when retrieving between 1% and 35% of table rows depending upon
Clustering factor, determined by data load order relative to key value Distribution of data on key values Number of rows per key value Length of indexed columns

Access method conclusions

Full-table scans are usually the most efficient access method when more than 35% of the rows are retrieved Full-table scans may be the most efficient access method when retrieving as few as 5% of total table rows When retrieving between 5 and 35% of the table rows, use clustering factor to determine which one is most efficient Test the results

Database parameters for FTS

db_file_multi_block_read_count
Maximum number of blocks read for each physical I/O operation during a full-table scan OS dependent Typically 8 to 64 (Default=8 on Solaris)

optimizer_mode
ALL_ROWS and CHOOSE (Default) modes are becoming better at determining when to perform full-table scans RULE and FIRST_ROWS favor index access methods over full-table scans Recommendation: Use the default value and correct the few errant queries that result using query hints It is imperative to periodically collect statistics on tables and indexes for the optimizer to determine the optimal execution plan

Full-table scan performance

Full-table scans read all data blocks up to the high water mark of the table Periodically rebuild tables that have considerable delete activity to minimize the number of empty table blocks that will be read during full-table scans If appropriate, consider increasing PCTUSED on frequently scanned tables to reduce total number of data blocks Consider caching frequently scanned small tables Must have accurate statistics on tables and indexes for optimizer to choose correct access path Must have correct indexes on large tables to avoid inappropriate full-tables scans Drop indexes on small tables unless they support primary or unique key constraints

Practical use: hash joins

Application
Small row source joined to a large row source Or, two smaller row sources of equal size joined to each other FTS on smaller tables are usually most efficient access method

How they work

Small row source is read and bitmapped, eliminating rows with limiting conditions Large row source is read using appropriate access method, eliminating rows with limiting conditions Join is effected using a hashing algorithm as the large row source is read

Hash join schematic

Small row source
1. Read entire small row source and create hash table 2. Read row from large row source and hash the key value

3. Check for matching hash value in hash table

Large row source

Hash table contains bitmap of key values

4. A match yields a joined row

Result set
5. Repeat steps 2-4 for all rows in large row source

Example join
tax_info table has 1,200 rows, 21 blocks orders table has 600,000 rows, 9,000 data blocks Query returns 4,000 rows

Nested loops join method

SELECT o.order_num, o.order_date, s.state_name, s.tax_rate FROM orders o, tax_info s WHERE o.juris_cd = s.juris_cd AND o.order_date = 01-AUG-02; NESTED LOOPS JOIN TABLE ACCESS BY INDEX ROW ID ORDERS INDEX RANGE SCAN ORDERS_IDX1 TABLE ACCESS BY INDEX ROW ID TAX_INFO INDEX UNIQUE SCAN TAX_INFO_IDX1

For every row retrieved from orders, 3 logical reads are required to retrieve the row in tax_info by the unique index. 4,000 x 3 = 12,000 logical reads to access tax_info

Hash join method

SELECT o.order_num, o.order_date, s.state_name,s.tax_rate FROM orders o, tax_info s WHERE o.juris_cd = s.juris_cd AND o.order_date = 01-AUG-02; HASH JOIN TABLE ACCESS FULL TAX_INFO TABLE ACCESS BY INDEX ROW ID ORDERS INDEX RANGE SCAN ORDERS_IDX1

tax_info appears first in the execution plan so it is the table that is bitmapped One full-table scan (21 Logical Reads) is required to retrieve all rows from tax_info Compare to 12,000 logical reads in the nested loops join case

Hash join performance gains

Factor of 5 to 15 reduction in logical reads is possible Factor of 5 to 15 reduction in execution time is possible Cost-based optimizer usually does a good job of determining when hash joins are most appropriate if statistics are available on the associated tables and indexes Hash joins often use full-table scans on small tables Caching frequently scanned small tables may be appropriate to minimize physical reads

Practical Use: Parallel FTS

Goal is to obtain full system resource utilization (100% I/O, 100% CPU) If utilization is significantly lower, why not use it? Use Parallel Execution (PX) facility to perform full-table scans to reduce run-time on longrunning nonselective queries if resources are available This method may use more system resources than index access method but response time is improved significantly if employed correctly

Nonparallel full-table scans

Server Process

A single process scans all data blocks

Table data blocks

Parallel full-table scans

Server Process
Each slave process scans a range of data blocks, or table partition

PX Slave Process

PX Slave Process

PX Slave Process

Table data blocks

Summary
Full-table scans may be the most efficient access method when between 5% and 35% of rows are retrieved depending upon available indexes and physical structure Full-table scans will usually be the most efficient access method when more than 35% of rows are retrieved Significant performance gains can be realized by employing features that can effectively utilize full-table scans Tune database parameters for optimal full-table scan and feature performance Test parameters and queries before migrating!

Questions and Answers