Presentation Outline

SQL Writing Process

SQL Standards
Using Indexes The Optimizer FROM, WHERE Clauses EXPLAIN SQL Trace Sub-Selects and Joins Tips and Tricks
1

SQL Writing Process

Step 1: What information do I need? Columns Step 2: Where is it? Tables Step 3: Write SQL:
SELECT columns FROM tables WHERE ... (joins, filters, subqueries)

I'M FINISHED!

SQL Writing Process

YOU'RE NOT FINISHED YET! You've got the results you want, but at what cost? There are many, many ways to get the right results, but only one is the fastest way1000-to-1 improvements are attainable! Inefficient SQL can dramatically degrade the performance of the entire system

Developers and DBAs must work together to tune the database and the application

SQL Standards
Why are SQL standards important?

Maintainability, readability
Performance: If SQL is the same as a (recently) executed statement, it can be re-used instead of needing to be reparsed

SQL Standards
Question: which of these statements are the same?

A. SELECT LNAME FROM EMP WHERE EMPNO = 12;

B. SELECT lname FROM emp WHERE empno = 12; C. SELECT lname FROM emp WHERE empno = :id; D. SELECT lname FROM
WHERE empno = 12; emp

SQL Standards
Answer: None

Whitespace, case, bind variables vs. constants all matter

Using standards helps to ensure that equivalent SQL can be reused.

Tables Used in the Examples

DEPT deptno dname loc
EMP empno mgr job deptno fname lname comm hiredate grade sal
7

SALGRADE grade losal hisal

SQL Standards: Example

SELECT E.empno, D.dname FROM emp E, dept D WHERE AND OR E.deptno = D.deptno (D.deptno = :vardept E.empno = :varemp); Keywords upper case and left-aligned Columns on new lines Use std. table aliases

Separate w/ one space

Use bind variables AND/OR on new lines

No space before/after parentheses

Indexes: What are they?

An index is a database object used to speed retrieval of rows in a table. The index contains only the indexed value--usually the key(s)--and a pointer to the row in the table. Multiple indexes may be created for a table Not all indexes contain unique values Indexes may have multiple columns

Indexes and SQL

If a column appears in a WHERE clause it is a candidate for being indexed. If a column is indexed the database can used the index to find the rows instead of scanning the table. If the column is not referenced properly, however, the database may not be able to used the index and will have to scan the table anyway. Knowing what columns are and are not indexed can help you write more efficient SQL
10

Example: Query without Index

No index exists for column EMPNO on table EMP, so a table scan must be performed:
Table: EMP SELECT * FROM emp WHERE empno = 8
empno 4 9 1 3 5 2 7 8 6 fname lisa jackie john larry jim mary harold mark gene lname... baker miller larson jones clark smith simmons burns harris

11

Example: Query with Index

Column EMPNO is indexed, so it can be used to find the requested row:
SELECT * FROM emp WHERE empno = 8 Index: PK_EMP EMP (EMPNO)
1, 4

Table: EMP
5 5, 9 empno 4 9 1 3 5 2 7 8 6 fname lisa jackie john larry jim mary harold mark gene lname ... baker miller larson jones clark smith simmons burns harris

1 2

7 8 9

12

Indexes: Caveats
Sometimes a table scan cannot be avoided Not every column should be indexed--there is performance overhead on Inserts, Updates, Deletes Small tables may be faster with a table scan Queries returning a large number (> 5-20%) of the rows in the table may be faster with a table scan

13

Indexes: Column Order

Example: Index on (EMPNO, DEPTNO)
SELECT * FROM emp WHERE deptno = 10;

SELECT FROM WHERE AND

* emp empno > 0 deptno = 10;

Question: which of these statements will use the index?

14

Indexes: Column Order

First statement will not use index. Second statement will use index.

Must use the leading column of the index.

15

Indexes: Functions
Question: which of these statements will use the index?
SELECT FROM WHERE ... WHERE * emp TRUNC(hiredate) = TRUNC(SYSDATE); fname || lname = 'MARYSMITH';

SELECT * FROM emp WHERE hiredate BETWEEN TRUNC(SYSDATE) AND TRUNC(SYSDATE)+1 ... WHERE fname = 'MARY' AND lname = 'SMITH';
16

Indexes: Functions
First statement will not use index. Second statement will use index. Using a function, calculation, or other operation on an indexed column disables the use of the Index.

17

Indexes: NOT
Question: which of these statements will use the index?
SELECT FROM WHERE ... ... * dept deptno != 0; deptno NOT = 0; deptno IS NOT NULL;

SELECT * FROM dept WHERE deptno > 0;

18

Indexes: NOT
First statement will not use index. Second statement will use index. Using NOT excludes indexed columns.

19

The Optimizer
The WHERE/FROM rules on the following pages apply to the Rule-based optimizer (Oracle). If the Cost-based Optimizer is used, Oracle will attempt to reorder the statements as efficiently as possible (assuming statistics are available). DB2 and Sybase use only a Cost-based optimizer The Optimizer's access paths can be overridden in Oracle and Sybase (not DB2)

20

The Optimizer: Hints

Return the first rows in the result set as fast as possible:
SELECT /*+ FIRST_ROWS */ empno FROM emp E dept D, WHERE E.deptno = D.deptno;

Force Optimizer to use index IDX_HIREDATE:

SELECT /*+ INDEX (E idx_hiredate) */ empno FROM emp E WHERE E.hiredate > TO_DATE('01-JAN-2000');

21

FROM Clause: Driving Table

Specify the driving table last in the FROM Clause:
SELECT * FROM dept D, -- 10 rows emp E -- 1,000 rows WHERE E.deptno = D.deptno;
SELECT * FROM emp E, -- 1,000 rows dept D -- 10 rows WHERE E.deptno = D.deptno;

Driving table is EMP

Driving table is DEPT

22

FROM Clause: Intersection Table

When joining 3 or more tables, use the Intersection table (with the most shared columns) as the driving table:
SELECT * FROM dept D, salgrade S, emp E WHERE E.deptno = D.deptno AND E.grade = S.grade;

EMP shares columns with DEPT and SALGRADE, so use as the driving table

23

WHERE: Discard Early

Use WHERE clauses first which discard the maximum number of rows:
SELECT FROM WHERE AND * emp E E.empno IN (101, 102, 103) E.deptno > 10;

3 rows 90,000 rows

24

WHERE: USING AND

Question: which of these statements will more effective?
SELECT FROM WHERE AND * emp E E.sal > 50000 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno)

SELECT * FROM emp E WHERE 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno) AND E.sal > 50000
25

WHERE: USING AND

When using an "AND" sub query, place it first. It will more cost effective.

26

WHERE: USING OR
Question: which of these statements will more effective?
SELECT * FROM emp E WHERE 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno) OR E.sal > 50000 SELECT FROM WHERE OR * emp E E.sal > 50000 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno)
27

WHERE: USING OR
Using an "OR" sub query, place it last. It will more cost effective.

28

WHERE: Filter First, Join Last

When Joining and Filtering, specify the Filter condition first, Joins last.
SELECT * FROM emp E, dept D WHERE (E.empno = 123 OR D.deptno > 10) AND E.deptno = D.deptno;

Filter criteria Join criteria

29

Subqueries: IN vs. EXISTS

Question: which of these statements will more effective?
SELECT E.* FROM emp E WHERE E.deptno IN ( SELECT D.deptno FROM dept D WHERE D.dname = 'SALES'); SELECT * FROM emp E WHERE EXISTS SELECT FROM WHERE AND

( 'X' dept D D.deptno = E.deptno D.dname = 'SALES');

30

Subqueries: IN vs. EXISTS

IN : Both tables are scanned. EXITS : Only outer table is scanned;Sub query used the index.

Use EXISTS instead of IN in sub queries.

31

Subquery vs. Join

Question: which of these statements will more effective?
SELECT * FROM emp E WHERE E.deptno IN ( SELECT D.deptno FROM dept D WHERE D.dname = 'SALES'); SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno; 32

Subquery vs. Join

IN : Both tables are scanned. JOIN : Only one table is scanned,other use index.

Use Join instead of Sub query.

33

Join vs. EXISTS

Best performance depends on subquery/driving table:
SELECT * FROM emp E WHERE EXISTS SELECT FROM WHERE AND EXISTS: better than Join if the number of matching rows in DEPT is small

( 'X' dept D D.deptno = E.deptno D.dname = 'SALES');

SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno;
34

JOIN: better than Exists if the number of matching rows in DEPT is large

Explain
Display the access path the database will use (e.g., use of indexes, sorts, joins, table scans) Oracle: Sybase: DB2: EXPLAIN SHOWPLAN EXPLAIN

Oracle Syntax:
EXPLAIN PLAN SET STATEMENT_ID = 'statement id' INTO PLAN_TABLE FOR

statement

Requires Select/Insert privileges on PLAN_TABLE

35

Explain
Example 1: IN subquery
SELECT * FROM emp E WHERE E.deptno IN ( SELECT D.deptno FROM dept D WHERE D.dname = 'SALES');

Result:

3 joins MERGE JOIN 1 dynamic view SORT (JOIN) 2 table scans TABLE ACCESS (FULL) OF EMP 3 sorts SORT (JOIN) VIEW SORT (UNIQUE) TABLE ACCESS (FULL) OF DEPT
36

Explain
Example 2: "EXISTS" subquery
SELECT * FROM emp e WHERE EXISTS SELECT FROM WHERE AND

( 'x' dept d d.deptno = e.deptno d.dname = 'SALES');

1 table scan 1 index scan 1 index access

Result:

FILTER TABLE ACCESS (FULL) OF EMP TABLE ACCESS (BY INDEX ROWID) OF DEPT INDEX (UNIQUE SCAN) OF PK_DEPT (UNIQUE)
37

Explain
Example 3: Join (no subquery)
SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno;

Result:

1 table scan 1 index scan 1 index access

NESTED LOOPS TABLE ACCESS (FULL) OF EMP TABLE ACCESS (BY INDEX ROWID) OF DEPT INDEX (UNIQUE SCAN) OF PK_DEPT (UNIQUE)

38

SQL Trace
Use SQL Trace to determine the actual time and resource costs for for a statement to execute. Step 1: ALTER SESSION SET SQL_TRACE TRUE; Step 2: Execute SQL to be traced:
SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno;

Step 3: ALTER SESSION SET SQL_TRACE FALSE;

39

SQL Trace
Step 4: Trace file is created in <USER_DUMP_DEST> directory on the server (specified by the DBA).

Step 5: Run TKPROF (UNIX) to create a formatted output file:

tkprof echd_ora_15319.trc $HOME/prof.out table=plan_table explain=dbuser/passwd

Trace file Formatted output file destination for Explain user/passwd for Explain

40

SQL Trace
Step 6: view the output file:
... SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno; call count ------- -----Parse 1 Execute 1 Fetch 2 ------- -----total 4

TIMED_STATISTICS must be turned on to get these values

rows ---------0 0 6 ---------6

cpu elapsed disk query current -------- ---------- ---------- ---------- ---------0.00 0.00 0 0 0 0.00 0.00 0 0 0 0.00 0.00 4 19 3 -------- ---------- ---------- ---------- ---------0.00 0.00 4 19 3

Misses in library cache during parse: 0 Optimizer goal: CHOOSE Parsing user id: 62 (PMARKS) Rows ------6 14 14 14 Row Source Operation --------------------------------------------------NESTED LOOPS TABLE ACCESS FULL EMP TABLE ACCESS BY INDEX ROWID DEPT INDEX UNIQUE SCAN (object id 4628)

EXPLAIN output

41

Tips and Tricks: UNION ALL

Use UNION ALL instead of UNION if there are no duplicate rows (or if you don't mind duplicates):
SELECT * FROM emp UNION SELECT * FROM emp_arch; SELECT * FROM emp UNION ALL SELECT * FROM emp_arch;

UNION: requires sort

UNION ALL: no sort

42

Tips and Tricks: HAVING vs. WHERE

With GROUP BY, use WHERE instead of HAVING (if the filter criteria does not apply to a group function):
SELECT deptno, AVG(sal) FROM emp GROUP BY deptno HAVING deptno IN (10, 20); SELECT deptno, AVG(sal) FROM emp WHERE deptno IN (10, 20) GROUP BY deptno;
43

HAVING: rows are filtered after result set is returned

WHERE: rows are filtered first--possibly far fewer to process

Tips and Tricks: EXISTS vs DISTINCT

Use EXISTS instead of DISTINCT to avoid implicit sort (if the column is indexed):
SELECT DISTINCT e.deptno, e.lname FROM dept d, emp e WHERE d.deptno = e.deptno;

DISTINCT: implicit sort is performed to filter duplicate rows

SELECT e.deptno, e.lname EXISTS: no sort FROM emp e WHERE EXISTS ( SELECT 'X' FROM dept d WHERE d.deptno = e.deptno);
44

Tips and Tricks: Consolidate SQL

Select from Sequences and use SYSDATE in the statement in which they are used:
SELECT SYSDATE INTO :vardate FROM dual; FROM dual;

SELECT arch_seq.NEXTVAL INTO :varid

INSERT INTO archive BEFORE: 3 statements VALUES (:vardate, :varid, ...) are used to perform 1 Insert
INSERT INTO emp_archive VALUES (SYSDATE, emp_seq.NEXTVAL, ...) AFTER: only 1 statement is needed
45

Tips and Tricks: Consolidate SQL

Consolidate unrelated statements using outer-joins to the the DUAL (dummy) table:
SELECT dname FROM dept WHERE deptno = 10; SELECT lname FROM emp WHERE empno = 7369;

SELECT d.dname, e.lname FROM dept d, emp e, dual x WHERE d.deptno AND e.empno AND NVL('X', AND NVL('X',

BEFORE: 2 round-trips

AFTER: only 1 round-trip

(+) = 10 (+) = 7369 x.dummy) = NVL('X', e.ROWID (+)) x.dummy) = NVL('X', d.ROWID (+));
46

Tips and Tricks: COUNT

Use COUNT(*) instead of COUNT(column):
SELECT COUNT(empno) FROM emp;

SELECT COUNT(*) FROM emp;

~ 50% faster

47

Tips and Tricks: Self-Join

Use a self-join (joining a table to itself) instead of two queries on the same table:
SELECT mgr INTO :varmgr FROM emp WHERE deptno = 10; LOOP... SELECT mgr, lname FROM emp WHERE mgr = :varmgr; SELECT E.mgr, E.lname FROM emp E, emp M WHERE M.deptno = 10 AND E.empno = M.mgr;

BEFORE: 2 round-trips

AFTER: only 1

48

Tips and Tricks: ROWNUM

Use the ROWNUM pseudo-column to return only the first N rows of a result set. (For example, if you just want a sampling of data):
SELECT * FROM emp WHERE ROWNUM <= 10; Returns only the first 10 employees in the table, in no particular order

49

Tips and Tricks: ROWID

The ROWID pseudo-column uniquely identifies a row, and is the fastest way to access a row:
CURSOR retired_emp_cur IS Instead of selecting the SELECT ROWID key column(s), ROWID is FROM emp used to identify the row WHERE retired = 'Y'; for later use ... FOR retired_emp_rec IN retired_emp_cur LOOP SELECT fname || ' ' || lname INTO :printable_name FROM emp WHERE ROWID = retired_emp_rec.ROWID; ...
50

Tips and Tricks: Sequences

Use a Sequence to generate unique values for a table:
SELECT INTO FROM ... INSERT VALUES MAX(empno) :new_empno emp; INTO emp (:new_empno + 1, ...); MAX(empno) requires a sort and an index scan INSERT could fail with a Duplicate error if someone else gets there first

Using a Sequence INSERT INTO emp VALUES (emp_seq.NEXTVAL, ...); ensures that you always have a unique number, or and does not require any SELECT emp_seq.NEXVAL table reads INTO :new_empno FROM dual;
51

THANK YOU

52

Tips and Tricks: Cartesian Products

Avoid Cartesian products by ensuring that the tables are joined on all shared keys:
SELECT FROM * dept, -- 10 rows salgrade, -- 20 rows emp; -- 1,000 rows

10 * 1000 * 20 = 200,000 rows

SELECT FROM * dept, -- 10 rows salgrade, -- 20 rows emp -- 1,000 rows E.deptno = D.deptno E.grade = S.grade;
53

WHERE AND

1,000 rows