1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

2 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Direct NFS and CloneDB with
the Latest Generation of
Database Technology
Akshay Shah
Principal Member of Technical Staff
3 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
The following is intended to outline our general product direction. It is intended
for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracles products
remains at the sole discretion of Oracle.
4 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB architecture
Conclusion
5 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB architecture
Conclusion
6 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Introduction
NAS Attractive for database storage
Easy to deploy and simple to administer
Runs over standard Ethernet
Cheaper than Storage Area Networks (SAN)
Shared file system for Grid Computing (RAC)
Why is NAS not more popular for databases?
NAS performance is inferior to SAN
NFS client stack not optimized for database access patterns
Client management complexities
Configuration varies across different platforms.
Tuning parameters are quite complex.
High system CPU utilization
7 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS Client
New feature introduced in Oracle Database 11
g
Release 1
NFS version 3 client embedded in Oracle database kernel
Improves NAS storage performance for Oracle database
Improves high availability of NAS storage
Optimizes scalability of NAS storage
Vastly reduces system CPU utilization
Significant cost savings for database storage
Simplifies client management and eliminates config errors
Consistent configuration and performance across different platforms
Even Microsoft Windows
8 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Standard NFS Architecture
Database
NAS Storage
Standard NFS client

LGWR I/O queue

DBWR I/O queue

PQ slave I/O queue

RMAN I/O queue

RPC I/O daemon
limits Oracle to 128
concurrent I/Os
Just 1 TCP
connection per mount
Linux NFS client RPC daemon
Each Oracle process
can issue 100s of
concurrent I/Os
9 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS Architecture
Database
NAS Storage
Direct NFS client

LGWR I/O queue

DBWR I/O queue

PQ slave I/O queue

RMAN I/O queue

LGWR TCP connection
DBWR TCP connection
PQ slave TCP connection
RMAN TCP connection
Direct NFS can issue 1000s
of concurrent operations due
to the parallel architecture
Every Oracle
process has its own
TCP connection
10 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS Performance
Direct NFS improves NAS performance for Oracle
Direct and Asynchronous I/O
Bypass operating system caches
Eliminate operating system write-ordering locks
Reduce system CPU by bypassing file system stack
Massively parallel I/O architecture
Each Oracle process creates its own TCP connection to storage
Not bound by the number of TCP connections to storage
Not constrained by rpciod (Limited to 128 concurrent operations, default is 16)
11 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Scalability and High Availability
Direct NFS improves HA and optimizes server scalability
Supports up to 4 parallel network paths to storage
Displays linear scaling over both 1GbE and 10GbE paths
Automatically load balances across all network paths
No extra configuration necessary!!
Failover messages in the event of a path failure
1
2
5
6
7
8
3
4
Automatic load
balancing across
all paths
Path 3
goes down
3
7
12 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Cost Savings and Administration
Direct NFS provides significant cost savings
Simple Ethernet for Storage Connectivity
Eliminates expensive components in storage stack
Redundant host bus adaptors (HBA)
Fibre Channel switches
Bonded network interfaces
Direct NFS simplifies NAS administration
Standard NFS client implementation across all Oracle supported platforms
NFS is a viable option on platforms that dont natively support NFS, e.g. Windows
Supports Real Application Cluster (RAC) database files
Direct NFS automatically optimizes mount points for RAC
No NIC bonding required as Direct NFS manages HA for database files
13 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Partner Validation Quotes
Direct NFS is an optimized NFS client for Oracle databases that
provides a high-bandwidth solution for data transfer with the Sun
ZFS Storage Appliance
Direct NFS is an excellent choice for customers who want to obtain
FCP-like performance for OLTP workloads at NFS costs
Direct NFS is easy to setup and the ease-of-use nature of the
option means it needs no maintenance once configured
Fibre Channel speeds and throughput and can be achieved
using pure NAS/ IP storage architectures with Oracle 11
g
Direct NFS
14 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB architecture
Conclusion
15 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
41698
94578
94231 11.6
5.8
6
0
2
4
6
8
10
12
14
0
20000
40000
60000
80000
100000
120000
w/o dNFS dNFS 10GbE FCoE
L
a
t
e
n
c
y

i
n

m
s
O
E

T
r
a
n
s
a
c
t
i
o
n
s

p
e
r

m
i
n
OLTP transactions and I/O latency
Order Entry Transactions DB file sequential read
OLTP Performance
Order Entry Transaction processing workload
1 TB database, 8000 warehouses, 250 concurrent users
Mix of transactions 65% reads, 35% writes
dNFS and FCoE
have same
throughput
dNFS latency is
slightly lower
than FCoE
16 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Table Scan Performance
Scan entire large table with 150 million rows
High degree of parallelism on the scan query
Linear throughput scaling with additional NICs
Throughput is limited by server-side bottleneck with 4 NICs
Low CPU utilization on the database host
112
224
328
380
7
16
26
34
0
20
40
60
80
100
0
50
100
150
200
250
300
350
400
1 2 3 4
C
P
U

u
t
i
l
i
z
a
t
i
o
n
(
A
v
g
.

%
)
N
e
t
w
o
r
k

T
h
r
o
u
g
h
p
u
t
(
M
B
/
s
e
c
)
Number of GbE connections
Network Throughut Host CPU utilization
Linear network
scaling
Low CPU
utilization
17 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Disk Write Performance
RMAN write from /dev/zero
Reading from /dev/zero to avoid ingest bottlenecks
Direct NFS almost saturates 2x10GigE pipes
Limited by disk bottleneck in the end
Great backup solution for high-end systems like Exadata
Supports sustained backup rates of up to 9 TB/hour with Exadata
0
500
1000
1500
2000
RMAN write throughput
W
r
i
t
e

t
h
r
o
u
g
h
p
u
t
(
M
B
/
s
e
c
)
Direct
NFS
Kernel
NFS
dNFS is
250% faster!!!
18 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB architecture
Conclusion
19 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS Client Configuration (1)
1 simple step to get started with Direct NFS!!!
Disabling Direct NFS is just as simple
prompt> cd $ORACLE_HOME/rdbms/lib
prompt> make f ins_rdbms.mk dnfs_on
Enable the Direct NFS Client
prompt> cd $ORACLE_HOME/rdbms/lib
prompt> make f ins_rdbms.mk dnfs_off
Disable the Direct NFS Client
20 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS Client Configuration (2)
Configure oranfstab config file for multi-pathing
Located in /etc or $ORACLE_HOME/dbs
server: myNFSServer1
local: 192.168.1.1 path: 192.168.1.2
local: 192.168.2.1 path: 192.168.2.2
local: 192.168.3.1 path: 192.168.3.2
local: 192.168.4.1 path: 192.168.4.2
export: /vol/oradata1 mount: /mnt/oradata1
export: /vol/oradata2 mount: /mnt/oradata2
mnt_timeout: 600
Parameter Usage
Server Unique identifier for the NFS server
Local Network paths (up to 4) on the database host
Path Network paths (up to 4) on the NFS server
Export The exported volume on the NFS server
Mount The local mount point for the exported volume
mnt_timeout Time in seconds to wait for the first mount
dontroute Prevents OS routing of outgoing messages
management Network path for NFS server management interface
oranfstab parameters
Sample oranfstab
21 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS Client Monitoring
Direct NFS v$ views
v$dnfs_servers
Lists the NFS servers that are mounted by Direct NFS client
v$dnfs_files
Lists the files that have been opened using Direct NFS client
v$dnfs_channels
Lists the TCP connections established by Direct NFS to the NFS server
v$dnfs_stats
Statistics on the different NFS operations issued by Oracle processes
select * from v$dnfs_servers;
svrname dirname mntport nfsport rtmax wtmax
mynfsserver /vol/oradata1 4046 2049 65536 65536
22 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS MOS notes
1496040.1
Direct NFS Frequently Asked Questions
1495104.1
Recommended patches for Direct NFS 11.2.0.3
1495739.1
Direct NFS monitoring and v$views
23 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB architecture
Conclusion
24 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB
Do you have NAS storage?
Do you take a backup of your production database?
Do you create Test/Dev instances?
Test clones
Production
database
25 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
CloneDB overview
Introduced in Oracle 11.2.0.2
Clone production databases with minimal impact
Production data is safe and secure
Uses a simple RMAN backup
Refresh test instances with incremental backup
Based on copy-on-write technology
Huge storage space saving with thin provisioning
Works with single instance and RAC databases
26 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Why CloneDB?
Easy to setup
No special storage software needed
Works on all platforms
Instantaneous cloning
No copies of the data
Create multiple clones based on one backup
Integrated with ZFS/NetApp/EMC snapshots
Refer to MOS note 1210656.1 for setup details
27 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
CloneDB Use Cases
Hardware/Software upgrades
Application/OS patching
Backup verification
Application development/testing
Recover Oracle objects
Run read-only report queries
28 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Test/dev setup
Application development
Storage size 5G
Clone DB #2
Production
Database
Size 1TB
Weekly
Production
Database
Backup
RMAN
Test hardware / software upgrades
Storage size 500 MB
Clone DB #1
dNFS
29 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Database upgrade
Production
Database
11.2.0.2
Weekly
Production
Database
Backup
RMAN
Clone
Database
11.2.0.2
dNFS
Clone
Database
11.2.0.3
Apply DB
patch
30 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB architecture
Conclusion
31 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Architecture
All datafiles in clone environment are sparse files
Copy blocks on demand
Changed blocks bitmap using bloom filter
Information about the backup file recorded in cloned database
Size of clone database is determined by rate of change not size
of the production database
32 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Architecture
Cloned Database
Production
Database
Backup
Read dba (1,400)
1,200
Oracle process p1
Block 400, on datafile 1 retrieved from production backup as it
was never updated by clonedb
No copies maintained in clonedb, no storage occupied
Read dba (1,400)
1,800
1,1
1,500
1,900 1,800
1,2
1,400
1,600
33 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Architecture
Cloned Database
Production
Database
Backup
Read dba (1,400)
Oracle process p1
Clonedb updates data in block 400 of datafile 1 and writes it to disk
Data gets written to the clonedb storage
Production database backup is intact
1,1
1,500
1,900 1,800
1,2
1,400
1,600
Write dba (1,400)
1,200
1,800
1,400
34 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Architecture
Cloned Database
Production
Database
Backup
Oracle process p1
Updated data retrieved locally from the clonedb
Application sees the latest changed data
1,1
1,500
1,900 1,800
1,2
1,400
1,600
1,200
1,800
1,400
Read dba (1,400)
Write dba (1,400)
Read dba (1,400)
35 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Performance
Around 10% performance degradation between
production and clonedb
Production database performance is not impacted
Performance limited by the storage and the number
of test instances sharing backup
Data blocks non-contiguous in clone database
Full table scans can take longer than master
Redo log files, control files are always local
36 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Direct NFS CloneDB Performance
Both production and clone run on a similar hardware and storage
OLTP workload running on both databases
Clonedb TPS within 10% of the production TPS
0
500
1000
1500
2000
2500
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65
T
P
S
OLTP Throughput
MASTER
CLONE
37 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Program Agenda
Direct NFS introduction
Direct NFS performance
Direct NFS configuration and best practices
Direct NFS CloneDB overview
Direct NFS CloneDB deep dive
Conclusion
38 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Conclusion
Direct NFS
Extremely simple and easy-to-use
Performance comparable to Fibre Channel SAN
Built-in high availability and network scalability
Consistent performance across different platforms
Significant cost savings for database storage
CloneDB
Instantaneous cloning
Platform/storage independent
Thin Provisioning
Works on NAS
39 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Collateral
Direct NFS
Direct NFS documentation
Oracle Direct NFS white paper
Sun ZFS storage appliance backup best practices
NetApp protocol comparison white paper
EMC Direct NFS white paper
Clone DB
Oracle Clone DB MOS note
EMC Clone DB white paper
Direct NFS MOS notes
1496040.1, 1495104.1, 1495739.1
40 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Q&A
41 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
42 Copyright 2012, Oracle and/or its affiliates. All rights reserved.